math: fix nextafter and nexttoward on maxdbl and maxflt input