Somewhat surprising behaviour of \scantokens after \endinput

[jfbu]

Le 8 juin 2021 à 11:39, jfbu <jfbu at free.fr> a écrit :

> Hi,
> 
> I have encountered a phenomenon with \scantokens executed after \endinput:
> 
> ---
> % execute this file testscan.tex with etex on command line
> \def\test{%
>    \begingroup
>    \endlinechar-1\everyeof{\noexpand}%
>    \edef\foo{\scantokens{}}%
>    \endgroup
> }%
> \test % ok
> % \test\endinput % would be ok
> \endinput\test % not ok
> ---
> 
> executing "etex testscan" (or pdftex, xetex, luatex, eptex) gives this:
> 
> Runaway definition?
> ->
> ! File ended while scanning definition of \foo.
> <inserted text> 
>                }
> \test ...of {\noexpand }\edef \foo {\scantokens {}
>                                                  }\endgroup 
> l.12 \endinput\test
>                    % not ok^^M
> ? X
> 
> I would be curious if some expert could elucidate.
> 
> It seems the \noexpand which is usual trick to
> be able to use \scantokens inside an \edef
> fails here, 
> 
> and this is caused by the \scantokens being executed
> after the \endinput token was already encountered
> 
> I have not looked at the sources and I can't
> say a priori if this is to be expected or is
> a problem with \scantokens
> 
> All I remember from whatever TeX documentation
> I have read in the past
> is that TeX executes the macros on the same
> line as \endinput
> 
> And that a priori, it is unexpected behaviour
> if one does not know the innards of \scantokens
> 
> I looked at https://tug.org/mailman/listinfo
> but did not find an eTeX specific list,
> so posting here as it is not pdftex specific either
> 
> Best,
> 
> Please CC my address if replying
> 
> Jean-François
> 
> 


In contrast this has no issue

----
\def\test{%
    \input .tex\relax
}%
\endinput\test \message{This is ok}
----


produces

$ etex testbug4b
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021) (preloaded format=etex)
 restricted \write18 enabled.
entering extended mode
(./testbug4b.tex (/usr/local/texlive/2021/texmf-dist/tex/latex/tools/.tex)
This is ok)
*\bye
No pages of output.
Transcript written on testbug4b.log.


so it is not that executing \input after \endinput is forbidden.

Same with putting \message{This is ok} inside the file to \input

$ etex testbug4c
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021) (preloaded format=etex)
 restricted \write18 enabled.
entering extended mode
(./testbug4c.tex (./thisisok.tex This is ok))
*\bye
No pages of output.
Transcript written on testbug4c.log.