fix undefined behavior in scanf core

as reported/analyzed by Pascal Cuoq, the shlim and shcnt
macros/functions are called by the scanf core (vfscanf) with f->rpos
potentially null (if the FILE is not yet activated for reading at the
time of the call). in this case, they compute differences between a
null pointer (f->rpos) and a non-null one (f->buf), resulting in
undefined behavior.

it's unlikely that any observably wrong behavior occurred in practice,
at least without LTO, due to limits on what's visible to the compiler
from translation unit boundaries, but this has not been checked.

fix is simply ensuring that the FILE is activated for read mode before
entering the main scanf loop, and erroring out early if it can't be.

diff --git a/src/stdio/vfscanf.c b/src/stdio/vfscanf.c
index 9e030fc..b5ebc16 100644 (file)
--- a/src/stdio/vfscanf.c
+++ b/src/stdio/vfscanf.c
@@ -76,6 +76,9 @@ int vfscanf(FILE *restrict f, const char *restrict fmt, va_list ap)
 
        FLOCK(f);
 
+       if (!f->rpos) __toread(f);
+       if (!f->rpos) goto input_fail;
+
        for (p=(const unsigned char *)fmt; *p; p++) {
 
                alloc = 0;