Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runLaTeX: fixed empty table of contents + refactor #104185

Merged
merged 2 commits into from Nov 27, 2020
Merged

runLaTeX: fixed empty table of contents + refactor #104185

merged 2 commits into from Nov 27, 2020

Conversation

ilyakooo0
Copy link
Contributor

@ilyakooo0 ilyakooo0 commented Nov 18, 2020 •
edited

Motivation for this change

The current behavior of nixpkgs.texFunctions.runLaTeX does not always yield a valid document. More precisely, the builder determines whether LaTeX should be run additional times based on the files written during the previous run and whether the logs mention the need for a rerun. However, there are situations when the document needs to be rebuilt but there is no indication of it. An example of such a situation is the presence of a table of contents in the document. If the document is built once, the table of contents will be empty. LaTeX needs to be run twice to get a proper table of contents.

The current versions of nixpkgs yields a document with an empty table of contents:

cat <<EOF > default.nix
(import (builtins.fetchGit {
  url = "https://github.com/NixOS/nixpkgs";
  rev = "f2e4de720d80775eb198b252f0f31af6cbcce99e";
}) {}).texFunctions.runLaTeX {rootFile = ./doc.tex;}
EOF

cat <<EOF > doc.tex
\documentclass[12pt]{article}
\begin{document}
\tableofcontents
\section{Hello}
\end{document}
EOF

nix-build

doc.pdf

Screenshot 2020-11-18 at 19 28 27

My branch yields a non-empty table of contents which seems to clearly be more desireable:

cat <<EOF > default.nix
(import (builtins.fetchGit {
  url = "https://github.com/ilyakooo0/nixpkgs";
  ref = "fix-latex-empty-contents";
}) {}).texFunctions.runLaTeX {rootFile = ./doc.tex;}
EOF

cat <<EOF > doc.tex
\documentclass[12pt]{article}
\begin{document}
\tableofcontents
\section{Hello}
\end{document}
EOF

nix-build

doc.pdf
Screenshot 2020-11-18 at 19 29 47

Perofmance considerations

The only situation where the additional runLaTeX would not have been needed is when there is no table of contents and runLaTeX would have been run once (when bibTeX was not used). If building the document called runLaTeX multiple times before this PR, then the number runLaTeX calls will not be changed.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ilyakooo0
runLaTeX: fixed empty table of contents + refactor.
5fa5eeb 
Building a simple LaTeX document without bibTeX with a table of contents yielded a document with an empty table of contents since LaTeX was only run once
@ilyakooo0 ilyakooo0 changed the title Always run LaTeX twice runLaTeX: fixed empty table of contents + refactor Nov 18, 2020
@ilyakooo0 ilyakooo0 marked this pull request as ready for review November 18, 2020 16:45
@veprbl
Copy link
Member

veprbl commented Nov 18, 2020

I wonder if latexmk fails to handle your document as well

@veprbl veprbl added the 6.topic: TeX Issues regarding texlive and TeX in general label Nov 18, 2020
@ilyakooo0
Copy link
Contributor Author

ilyakooo0 commented Nov 18, 2020 •
edited

@veprbl I just tested it and latexmk builds the document properly:

doc.pdf
Screenshot 2020-11-18 at 20 04 32

I didn't investigate the way it detects the necessity to rebuild though...

latexmk log 
Latexmk: This is Latexmk, John Collins, 26 Dec. 2019, version: 4.67.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "doc.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./doc.tex
LaTeX2e <2020-02-02> patch level 5
L3 programming layer <2020-03-06>
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/article.cls
Document Class: article 2019/12/20 v1.4l Standard LaTeX document class
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/size12.clo))
(/usr/local/texlive/2020/texmf-dist/tex/latex/l3backend/l3backend-pdfmode.def)
(./doc.aux)
No file doc.toc.
[1{/usr/local/texlive/2020/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
(./doc.aux) )</usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsfonts/cm
/cmbx12.pfb></usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsfonts/cm/
cmr12.pfb>
Output written on doc.pdf (1 page, 16870 bytes).
Transcript written on doc.log.
=== TeX engine is 'pdfTeX'
Latexmk: Missing input file: 'doc.toc' from line
  'No file doc.toc.'
Latexmk: Log file says output to 'doc.pdf'
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'doc.aux'
      'doc.toc'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "doc.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./doc.tex
LaTeX2e <2020-02-02> patch level 5
L3 programming layer <2020-03-06>
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/article.cls
Document Class: article 2019/12/20 v1.4l Standard LaTeX document class
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/size12.clo))
(/usr/local/texlive/2020/texmf-dist/tex/latex/l3backend/l3backend-pdfmode.def)
(./doc.aux) (./doc.toc) [1{/usr/local/texlive/2020/texmf-var/fonts/map/pdftex/u
pdmap/pdftex.map}] (./doc.aux) )</usr/local/texlive/2020/texmf-dist/fonts/type1
/public/amsfonts/cm/cmbx12.pfb></usr/local/texlive/2020/texmf-dist/fonts/type1/
public/amsfonts/cm/cmr12.pfb>
Output written on doc.pdf (1 page, 16892 bytes).
Transcript written on doc.log.
=== TeX engine is 'pdfTeX'
Latexmk: Log file says output to 'doc.pdf'
Latexmk: All targets () are up-to-date

@veprbl
Copy link
Member

veprbl commented Nov 18, 2020

@ilyakooo0 If it manages to also build a more trivial document in a single pass, we might want to just switch to using it.

@ilyakooo0
Copy link
Contributor Author

ilyakooo0 commented Nov 18, 2020 •
edited

@veprbl I just tried running latexmk on a trivial document without a table of contents, and it seems like latexmk solves the problem precisely the same way – by always calling pdflatex twice – look for Running 'pdflatex -recorder "doc.tex"' in the logs below.

The file I tested it with:

\documentclass[12pt]{article}
\begin{document}
\section{Hello}
\end{document}
latexmk log 
❯ latexmk -pdf -verbose doc.tex
Latexmk: This is Latexmk, John Collins, 26 Dec. 2019, version: 4.67.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "doc.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./doc.tex
LaTeX2e <2020-02-02> patch level 5
L3 programming layer <2020-03-06>
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/article.cls
Document Class: article 2019/12/20 v1.4l Standard LaTeX document class
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/size12.clo))
(/usr/local/texlive/2020/texmf-dist/tex/latex/l3backend/l3backend-pdfmode.def)
(./doc.aux) [1{/usr/local/texlive/2020/texmf-var/fonts/map/pdftex/updmap/pdftex
.map}] (./doc.aux) )</usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsf
onts/cm/cmbx12.pfb></usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsfo
nts/cm/cmr12.pfb>
Output written on doc.pdf (1 page, 16137 bytes).
Transcript written on doc.log.
=== TeX engine is 'pdfTeX'
Latexmk: Log file says output to 'doc.pdf'
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'doc.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "doc.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./doc.tex
LaTeX2e <2020-02-02> patch level 5
L3 programming layer <2020-03-06>
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/article.cls
Document Class: article 2019/12/20 v1.4l Standard LaTeX document class
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/size12.clo))
(/usr/local/texlive/2020/texmf-dist/tex/latex/l3backend/l3backend-pdfmode.def)
(./doc.aux) [1{/usr/local/texlive/2020/texmf-var/fonts/map/pdftex/updmap/pdftex
.map}] (./doc.aux) )</usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsf
onts/cm/cmbx12.pfb></usr/local/texlive/2020/texmf-dist/fonts/type1/public/amsfo
nts/cm/cmr12.pfb>
Output written on doc.pdf (1 page, 16137 bytes).
Transcript written on doc.log.
=== TeX engine is 'pdfTeX'
Latexmk: Log file says output to 'doc.pdf'
Latexmk: All targets () are up-to-date

@veprbl
Copy link
Member

veprbl commented Nov 18, 2020

Seems like it reacts to the presence of the aux file and LaTeX seems to generate it even if TOC is not possible.

with (import (builtins.fetchGit {
  url = "https://github.com/NixOS/nixpkgs";
  rev = "f2e4de720d80775eb198b252f0f31af6cbcce99e";
}) {});

runCommand "latexmk-test" {
  nativeBuildInputs = with texlive; [ (combine { inherit scheme-small latexmk; }) ];
  file = builtins.toFile "doc.tex" ''
    \documentclass{minimal}
    \begin{document}
    test
    \end{document}
  '';
} ''
  cp "$file" doc.tex
  latexmk -pdf -verbose doc.tex
  mkdir "$out"
  mv * "$out"/
''

veprbl
veprbl approved these changes Nov 18, 2020
View reviewed changes
Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ilyakooo0
Copy link
Contributor Author

@veprbl I actually just noticed that when running pdflatex on a document with a \tableofcontents then an additional *.toc is generated. If I remove \tableofcontents from the document, then the *.toc file is not generated.

I am not intimately familiar with corner-cases of LaTeX, but if we use the presence of *.toc to detect the need for an additional pass, then the same problem might arise with some other latex tables the document writes to (lof, lot).

@veprbl
Copy link
Member

veprbl commented Nov 19, 2020

I suppose it's fine to run at least twice, but it would be nice to leave a comment in the code explaining the reasoning for doing the second iteration.

@ilyakooo0
Copy link
Contributor Author

@veprbl can we merge this?

@veprbl veprbl merged commit eb7d367 into NixOS:master Nov 27, 2020
KyleOndy added a commit to KyleOndy/dotfiles that referenced this pull request Nov 28, 2020
@KyleOndy
niv nixpkgs: update c6a8fcd9 -> d26aa329
783c3c8 
## Changelog for nixpkgs:
Commits: [NixOS/nixpkgs@c6a8fcd9...d26aa329](NixOS/nixpkgs@c6a8fcd...d26aa32)

* [`2b0ee787`](NixOS/nixpkgs@2b0ee78) mosquitto: systemd service sandboxing
* [`43b32e6f`](NixOS/nixpkgs@43b32e6) fractal: mark broken on Darwin
* [`57784466`](NixOS/nixpkgs@5778446) bcompare: 4.3.5.24893 -> 4.3.7.25118
* [`921b15ef`](NixOS/nixpkgs@921b15e) python3Packages.nipype: remove Python 2 support (dependency nibabel already unsupported)
* [`f31be2ec`](NixOS/nixpkgs@f31be2e) python3Packages.nipype: avoid use of neurdflib by default since it causes a package collision when rdflib is in the closure
* [`8d8cd890`](NixOS/nixpkgs@8d8cd89) pasystray: ayatana appindicators instead of libappindicator
* [`65c9bb25`](NixOS/nixpkgs@65c9bb2) clipit: enable appindicator support
* [`ec34ce96`](NixOS/nixpkgs@ec34ce9) pyditz: 0.10.3 -> 0.11
* [`39e33a47`](NixOS/nixpkgs@39e33a4) mpvScripts.mpris: Use mpv-unwrapped
* [`c69dd499`](NixOS/nixpkgs@c69dd49) waifu2x-converter-cpp: 5.2.4 -> 5.3.3
* [`ca3dd4b5`](NixOS/nixpkgs@ca3dd4b) waifu2x-converter-cpp: add OpenCL support
* [`41e9a530`](NixOS/nixpkgs@41e9a53) powertop: strip hcitool FSH path
* [`ed93d1de`](NixOS/nixpkgs@ed93d1d) elementary-planner: 2.4.6 -> 2.5.4
* [`adbe903c`](NixOS/nixpkgs@adbe903) jdupes: 1.18.2 -> 1.19.0
* [`58ab512a`](NixOS/nixpkgs@58ab512) libdigidocpp: 3.14.3 -> 3.14.4
* [`6a1a3207`](NixOS/nixpkgs@6a1a320) keepassrpc: 1.8.0 -> 1.14.0
* [`173bf39c`](NixOS/nixpkgs@173bf39) todoist-electron: 1.23.0 -> 1.24.0
* [`59455622`](NixOS/nixpkgs@5945562) todoist-electron: Switch to MIT license
* [`8cfeba2b`](NixOS/nixpkgs@8cfeba2) todoist-electron: Use desktop icon
* [`3d9ee22d`](NixOS/nixpkgs@3d9ee22) wine{Unstable,Staging}: 5.20 -> 5.21
* [`dfcd45de`](NixOS/nixpkgs@dfcd45d) wineStable: 5.0.2 -> 5.0.3
* [`152147a8`](NixOS/nixpkgs@152147a) wine-mono: 4.9.4 -> 5.1.1
* [`22da5d92`](NixOS/nixpkgs@22da5d9) wine{Unstable,Staging}: 5.21 -> 5.22
* [`4d5ab8d9`](NixOS/nixpkgs@4d5ab8d) autofs: fix compilation fail due to libtirpc changes
* [`9144ebaf`](NixOS/nixpkgs@9144eba) wineStable: revert cert-path patch for stable
* [`626f9310`](NixOS/nixpkgs@626f931) wineUnstable: use own set of patches
* [`db17db53`](NixOS/nixpkgs@db17db5) nginx: 1.19.4 -> 1.19.5
* [`587ef348`](NixOS/nixpkgs@587ef34) postgresqlPackages.pg_hll: 2.14 -> 2.15
* [`5183864d`](NixOS/nixpkgs@5183864) linux: explicitly enable RAS
* [`d9d8c8a7`](NixOS/nixpkgs@d9d8c8a) blender: use ffmpeg instead of ffmpeg_3
* [`492f3b80`](NixOS/nixpkgs@492f3b8) blender: fix on darwin
* [`05440fda`](NixOS/nixpkgs@05440fd) shotcut: use qt-5.15, rework
* [`8f46b51e`](NixOS/nixpkgs@8f46b51) ammonite: 2.2.0 -> 2.3.8
* [`1b14ab86`](NixOS/nixpkgs@1b14ab8) _1password-gui: 0.9.4-1 -> 0.9.5-2
* [`b3efb719`](NixOS/nixpkgs@b3efb71) avocode: 4.7.0 -> 4.10.3
* [`92672cff`](NixOS/nixpkgs@92672cf) qmapshack: 1.15.0 → 1.15.1
* [`fcbedf65`](NixOS/nixpkgs@fcbedf6) Revert "nmap: 7.80 -> 7.90"
* [`00afc855`](NixOS/nixpkgs@00afc85) gitter: 4.1.0 -> 5.0.1
* [`63dd41f7`](NixOS/nixpkgs@63dd41f) bombadillo: init at 2.3.3
* [`89f75b02`](NixOS/nixpkgs@89f75b0) geeqie: 1.4.0 -> 1.5.1
* [`706ed34e`](NixOS/nixpkgs@706ed34) containerd: 1.4.1 -> 1.4.2
* [`3ed321df`](NixOS/nixpkgs@3ed321d) z3: enable build on non-x86_64 unix; checked the build on aarch64-linux
* [`06f2fd5f`](NixOS/nixpkgs@06f2fd5) jetbrains: updates
* [`6aea53c3`](NixOS/nixpkgs@6aea53c) chromedriver: Switch to Chromium's upstream-info.json (NixOS/nixpkgs#105054)
* [`abe1271f`](NixOS/nixpkgs@abe1271) perlPackages.Appcpm: init at 0.994
* [`0229bebd`](NixOS/nixpkgs@0229beb) cimg: 2.9.3 -> 2.9.4
* [`3546e475`](NixOS/nixpkgs@3546e47) fondo: 1.3.9 -> 1.3.10
* [`ac3f6009`](NixOS/nixpkgs@ac3f600) pythonPackages.browsermob-proxy: drop insecure python-cryptography transitive dependency; only used for Marionette anyway
* [`0d6bc2f7`](NixOS/nixpkgs@0d6bc2f) broot: 1.0.5 -> 1.0.6
* [`94486021`](NixOS/nixpkgs@9448602) enlightenment.evisum: 0.5.7 -> 0.5.8
* [`64939cff`](NixOS/nixpkgs@64939cf) neomutt: 20201120 -> 20201127
* [`763e8380`](NixOS/nixpkgs@763e838) ephemeral: 6.4.1 -> 7.0.4
* [`34d28198`](NixOS/nixpkgs@34d2819) ammonite: Fix arguments
* [`f98df98d`](NixOS/nixpkgs@f98df98) google-chrome: Remove msteen from the list of maintainers (NixOS/nixpkgs#105152)
* [`eb7d3672`](NixOS/nixpkgs@eb7d367) runLaTeX: rerun at least twice + refactor (NixOS/nixpkgs#104185)
* [`77e367e9`](NixOS/nixpkgs@77e367e) datovka: 4.15.5 -> 4.15.6
* [`09a0f4ca`](NixOS/nixpkgs@09a0f4c) discord-ptb: 0.0.22 -> 0.0.23
* [`df1c8e3f`](NixOS/nixpkgs@df1c8e3) python3Packages.snowflake-connector-python: Fix boto3 replace
* [`08c74a2d`](NixOS/nixpkgs@08c74a2) python3Packages.snowflake-sqlalchemy: Disable running tests
* [`d0f04c7c`](NixOS/nixpkgs@d0f04c7) frostwire-bin: 6.8.7 -> 6.8.8
* [`67d7a671`](NixOS/nixpkgs@67d7a67) tilt: 0.17.11 -> 0.17.12
@KyleOndy KyleOndy mentioned this pull request Nov 28, 2020
Closed
@ilyakooo0 ilyakooo0 deleted the fix-latex-empty-contents branch November 28, 2020 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@veprbl veprbl veprbl approved these changes

Assignees
No one assigned
Labels
6.topic: TeX Issues regarding texlive and TeX in general 10.rebuild-darwin: 0 10.rebuild-linux: 0 12. first-time contribution
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@ilyakooo0 @veprbl @Janik-Haag