[tex-k] \pdfprimitive and back_input

[Hironori KITAGAWA]

Hello all,

H. Yamashita reported another issue that "\pdfprimitive\ \q" does not
issue an error "Undefined control sequence".

A MWE is the following source:
---- pdfprimitive-vmode.tex ----
\par % vmode

\ifdefined\pdfprimitive % pdfTeX
  \pdfprimitive\vrule\q % <-- (1)
\else % XeTeX
  \primitive\vrule\q
\fi
\end
----

I looked into this issue, and found that the cause is cur_tok is not set
in the following WEB code:

@<Cases of |main_control| that are not part of the inner loop@>=
any_mode(relax),vmode+spacer,mmode+spacer,mmode+no_boundary:do_nothing;
any_mode(ignore_spaces): begin
   (...)
    if cur_cs<>undefined_primitive then begin
      cur_cmd := prim_eq_type(cur_cs);
      cur_chr := prim_equiv(cur_cs);
      goto reswitch;
      end;
    end;
  end;

\vrule (which is a result of \pdfprimitive\vrule) in vmode causes 
back_input to unscan the token (and begins a paragraph). 
However, back_input only looks cur_tok, which represents 
the internalized \pdfprimitive at that time. 
So "\pdfprimitive\vrule\q" becomes "(internalized \pdfprimitive)\q",
hence no error (and \vrule disappears).

Note that e-(u)pTeX "resolved" this issue by commit r45482.
In pdfTeX the solution might be much simpler:
 - adding "cur_tok:=(cur_cmd*@'400)+cur_chr;" in above WEB code
 - raise cs_token_flag to something like "FFFF to avoid clash
But e-(u)pTeX couldn't took this principle, since a Japanese character
token uses a value between "1000 and cs_token_flag (raised to "FFFF or more).
So, I merged "prim_eqtb" array into main "eqtb" array
(between frozen_primitive and frozen_null_font).

I put a preliminary patch for pdfTeX and XeTeX at
https://gist.github.com/h-kitagawa/8c0bb24f7436a9fb5c851358b577f054 .


Regards
Hironori KITAGAWA <h_kitagawa2001 at yahoo.co.jp>