math: fix some invalid checks (remquo, lrint, lround, y0, y1, yn)