File descriptor error for pdflatex.fmt creation on android 11

[Henrik Grimler]

Hi,

I noticed two errors in my small test program (return value of
open_output was not checked and gzFile was uninitialized). I fixed
those but am then back at getting the fdsan error with the macro, but
not with it defined as a function. Compiling and running the attached
test.c gives the error, but removing the wopenout-macro and un-
commenting the wopenout-function makes it work.

Since using a function instead of a macro works for the small test
program I tried building texlive with wopenout converted to a function
(see attached wopenout.patch), but pdftex then seem to be unable to
open the fmtfile, it stops with:

```
(/data/data/com.termux/files/usr/share/texlive/texmf-
dist/tex/latex/firstaid/la
tex2e-first-aid-for-external-files.ltx)
 ) )
Beginning to dump on file pdflatex.fmt
! Could not write 1 4-byte item(s) to pdflatex.fmt.
 (preloaded format=pdflatex 2021.4.4)
```

Which indicates that I have made some error in the patch, or that there
is some other error in wopenout. Can anyone see any problems in
wopenout.patch?

One workaround that works is to not compress the format file, i.e.
undef FMT_COMPRESS in texmfmp.h.

Best regards,
Henrik Grimler

On Sat, 2021-04-03 at 20:50 +0200, Henrik Grimler wrote:
> Hello again, 
> 
> I have created a small test program (attached as test.c) that
> reproduces the problem by condensing the relevant code in
> texk/web2c/texmfmp.h and texk/web2c/lib/openclose.c.
> 
> It compiles without errors with clang-11 on both android (arm) and
> linux (x86_64) with `clang -lz -o test test_reproduced.c`, and works
> fine on linux, but when run on android it gives one of those file
> descriptor errors:
> 
> fdsan: attempted to close file descriptor 3, expected to be unowned,
> actually owned by FILE* 0xacdb000c
> 
> The problem seem to have to do with how the format file is opened. In
> texmfmp.h we have:
> 
> ```c
> #define wopenout(f) (open_output ((FILE**)&(f), FOPEN_WBIN_MODE) \
>  && (f = gzdopen(fileno((FILE*)f), FOPEN_WBIN_MODE)) \
>  && (gzsetparams(f, 1, Z_DEFAULT_STRATEGY) == Z_OK))
> ```
> 
> If I convert the macro into a function instead:
> 
> ```c
> int wopenout (gzFile f)
> {
>   open_output ((FILE**)&(f), "wb");
>   f = gzdopen(fileno((FILE*)f), "wb");
>   return gzsetparams(f, 1, Z_DEFAULT_STRATEGY) == Z_OK;
> }
> ```
> 
> the error no longer occurs. 
> 
> My first thought was that the issue is related to how gzFile is casted
> to FILE, but since the problem disappears when defined as a function I
> suppose this might be a bug in the android libc and/or fd sanitizer. 
> 
> Could macros, as the one defined for wopenout, be problematic? I.e.
> would it make sense to convert wopenout into a normal function or is
> this most likely an android-specific bug?
> 
> Best regards,
> Henrik Grimler
> 
> 
> 
> On Fri, 2021-04-02 at 15:45 +0200, Henrik Grimler wrote:
> > Dear all,
> > 
> > Android 11 introduced a new file descriptor sanitizer ("fdsan", see
> > [1]
> > for docs) which detects possible issues when opening and closing
> > files
> > across multiple threads.
> > 
> > Installing texlive on android 11 gives a fdsan error when creating
> > pdflatex.fmt. If I run:
> > 
> >   pdftex -ini -jobname=pdflatex -progname=pdflatex -translate-
> > file=cp227.tcx *pdflatex.ini
> > 
> > it aborts with an error
> > 
> >   fdsan: attempted to close file descriptor 3, expected to be
> > unowned,
> > actually owned by FILE* 0xb6c6100c
> > 
> > Does anyone have experience with this type of issue? For debugging it
> > the docs suggests replacing all `int fd` with a special `unique_fd
> > fd`
> > type, but I am not yet sure how to use that for software like texlive
> > (rather than android apps), so any hints are greatly appreciated.
> > 
> > If I run the command above in gdb I get this backtrace (see [2] for
> > full log):
> > 
> > [...]
> > Hyphenation trie of length 6081 has 183 ops out of 35111
> >   2 for language 1
> >   181 for language 0
> > 0 words of pdfTeX memory
> > fdsan: attempted to close file descriptor 3, expected to be unowned,
> > actually owned by FILE* 0xb6c6100c
> > 
> > Program received signal SIGABRT, Aborted.
> > 0xb64ec9e8 in fdsan_error(char const*, ...) () from
> > /apex/com.android.runtime/lib/bionic/libc.so
> > (gdb) bt
> > #0  0xb64ec9e8 in fdsan_error(char const*, ...) () from
> > /apex/com.android.runtime/lib/bionic/libc.so
> > #1  0xb64ec6fe in android_fdsan_close_with_tag () from
> > /apex/com.android.runtime/lib/bionic/libc.so
> > #2  0xb64ecd6e in close () from
> > /apex/com.android.runtime/lib/bionic/libc.so
> > #3  0xb68b18f4 in gzclose_w () from
> > /data/data/com.termux/files/usr/lib/libz.so.1.2.11
> > #4  0x7f56abd8 in storefmtfile () at pdftexini.c:3584
> > #5  0x7f570806 in mainbody () at pdftexini.c:5651
> > #6  0x7f5609c0 in main (ac=<optimized out>, av=<optimized out>)
> >     at /home/builder/.termux-build/texlive-
> > bin/src/texk/web2c/lib/texmfmp.c:1098
> > 
> > The above output is for pdftex built from the svn54456 tag. 
> > Please let me know if I can provide any additional information!
> > 
> > Best regards
> > Henrik Grimler
> > 
> > [1]
> > https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md
> > [2] https://grimler.se/logs/pdftex.log
> > 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: text/x-csrc
Size: 437 bytes
Desc: not available
URL: <https://tug.org/pipermail/tex-live/attachments/20210404/16a9cc37/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wopenout.patch
Type: text/x-patch
Size: 1092 bytes
Desc: not available
URL: <https://tug.org/pipermail/tex-live/attachments/20210404/16a9cc37/attachment-0001.bin>