math: fix signed int left shift ub in sqrt