-
-
Notifications
You must be signed in to change notification settings - Fork 15.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
texlive: 2019 (wip) #61469
texlive: 2019 (wip) #61469
Conversation
Status? |
Seems to work well for my needs? And I'm not confident about the hosting situation, but mostly I have no idea what to expect from NixOS'ers tex usage ^_^. And apparently this latest conflict which I'll address tomorrow / next push, time for bed for me O:). |
Minor ping, this would be great to have in 19.09 😄 |
@dtzWill what's missing from https://www.tug.org/historic/ , that needs to be hosted at wdtz.org? It seems like the 2019 texlive sources are there, too… |
The ctan sources-- aka everything in texlive that's not the core binaries :(. Nothing new on this front, just we need to play another round of "host the tarballs" for 2019 :). |
Closing in favor of #71419! \o/ If there's anything useful here not over there, LMK O:). |
Actually, the "-texmf" tarball contains exactly those packages from CTAN. And, in theory, it should be possible to build our texlive using that 2.6 GB tarball. The problem is that we would have to duplicate their build system. This is not impossible to do, and, in fact, guix did this [1] (with mere 7k lines of handwritten Guile code). They overcome the problem of the big tarball by fetching from svn [2], but this probably would not work for us since we are looking to provide 3.5k individual packages unless we are going to break compatibility. Right now, I don't see any shortcuts on this path. [1] https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tex.scm?h=bf7b08c4fe7d14c25a83bde99f19eca1119d88ff |
For some context, this was previously and recently discussed: NixOS#61469 (comment) NixOS#71419 (comment) Fortunately it appears, due to developments since those discussion, that there is now a versioned mirror for ctan contents!! Naturally this commit is about moving to that mirror: updating the package lists and regenerating the hashes. This has two huge advantages: * Updating the pinned ctan packages can be done more frequently than the current "once a year". * We don't have to rely on community members to create a snapshot and mirror it indefinitely and at their own cost. Minor "advantage": * I should be writing but instead I was working on this :P It appears new, I stumbled upon a mention of it here[1]: > Native TL updates ultimately come from a directory named tlnet (on CTAN). > An archive for tlnet is available, with daily updates since > September 2019, and the final yearly release going back about ten years. [1] https://www.tug.org/texlive/pkginstall.html ---- Concerns and theirs status: * Package integrity: There may be concerns about ensuring the package hashes are obtained from a trusted source-- I haven't looked into this at all "yet". * Mirror reliability and not-abusing-nice-things-from-good-people If the mirror is a small operation it may be hostile to point the network request fury of the Nixpkgs texlive community ;). This doesn't appear to be the case and there's no mention of it, and presumably wouldn't be linked from tug.org if wasn't to be used? Would be good to chase down an announcement or otherwise ensure this is expected usage. Time will tell re:mirror reliability (or speed?), so far it seems like a solid service meant to be used and unlikely to be problematic. * Is the archive only updated once a day? The archive is organized by date, which is wonderful (!) but might mean the contents may see changes during the day. I suspect this is not the case but just a thought. Regardless, any problems seem easily avoided by simply not updating to the current day and use yesterday instead. ---- If this is the way we move forward, there are changes/fixes/improvements to be made in terms of documentation as well as (possibly) putting together some update automation tooling. --- I have only tested this with a single project so far, and it doesn't have any especially esoteric or demanding needs. Help testing by checking your projects to see that they build and produce expected results. Building against the before/after set and using `diffpdf` is how I usually vet these updates, if that's useful. The `diffpdf` especially helps identify the minor appearance changes I usually miss visually-- tweaks to microtype behavior or slightly different layout/placement of figures or handling of beamer slides, etc.
For some context, this was previously and recently discussed: NixOS#61469 (comment) NixOS#71419 (comment) Fortunately it appears, due to developments since those discussion, that there is now a versioned mirror for ctan contents!! Naturally this commit is about moving to that mirror: updating the package lists and regenerating the hashes. This has two huge advantages: * Updating the pinned ctan packages can be done more frequently than the current "once a year". * We don't have to rely on community members to create a snapshot and mirror it indefinitely and at their own cost. Minor "advantage": * I should be writing but instead I was working on this :P It appears new, I stumbled upon a mention of it here[1]: > Native TL updates ultimately come from a directory named tlnet (on CTAN). > An archive for tlnet is available, with daily updates since > September 2019, and the final yearly release going back about ten years. [1] https://www.tug.org/texlive/pkginstall.html ---- Concerns and theirs status: * Package integrity: There may be concerns about ensuring the package hashes are obtained from a trusted source-- I haven't looked into this at all "yet". * Mirror reliability and not-abusing-nice-things-from-good-people If the mirror is a small operation it may be hostile to point the network request fury of the Nixpkgs texlive community ;). This doesn't appear to be the case and there's no mention of it, and presumably wouldn't be linked from tug.org if wasn't to be used? Would be good to chase down an announcement or otherwise ensure this is expected usage. Time will tell re:mirror reliability (or speed?), so far it seems like a solid service meant to be used and unlikely to be problematic. * Is the archive only updated once a day? The archive is organized by date, which is wonderful (!) but might mean the contents may see changes during the day. I suspect this is not the case but just a thought. Regardless, any problems seem easily avoided by simply not updating to the current day and use yesterday instead. ---- If this is the way we move forward, there are changes/fixes/improvements to be made in terms of documentation as well as (possibly) putting together some update automation tooling. --- I have only tested this with a single project so far, and it doesn't have any especially esoteric or demanding needs. Help testing by checking your projects to see that they build and produce expected results. Building against the before/after set and using `diffpdf` is how I usually vet these updates, if that's useful. The `diffpdf` especially helps identify the minor appearance changes I usually miss visually-- tweaks to microtype behavior or slightly different layout/placement of figures or handling of beamer slides, etc. (cherry picked from commit 9c75eec)
For some context, this was previously and recently discussed: NixOS#61469 (comment) NixOS#71419 (comment) Fortunately it appears, due to developments since those discussion, that there is now a versioned mirror for ctan contents!! Naturally this commit is about moving to that mirror: updating the package lists and regenerating the hashes. This has two huge advantages: * Updating the pinned ctan packages can be done more frequently than the current "once a year". * We don't have to rely on community members to create a snapshot and mirror it indefinitely and at their own cost. Minor "advantage": * I should be writing but instead I was working on this :P It appears new, I stumbled upon a mention of it here[1]: > Native TL updates ultimately come from a directory named tlnet (on CTAN). > An archive for tlnet is available, with daily updates since > September 2019, and the final yearly release going back about ten years. [1] https://www.tug.org/texlive/pkginstall.html ---- Concerns and theirs status: * Package integrity: There may be concerns about ensuring the package hashes are obtained from a trusted source-- I haven't looked into this at all "yet". * Mirror reliability and not-abusing-nice-things-from-good-people If the mirror is a small operation it may be hostile to point the network request fury of the Nixpkgs texlive community ;). This doesn't appear to be the case and there's no mention of it, and presumably wouldn't be linked from tug.org if wasn't to be used? Would be good to chase down an announcement or otherwise ensure this is expected usage. Time will tell re:mirror reliability (or speed?), so far it seems like a solid service meant to be used and unlikely to be problematic. * Is the archive only updated once a day? The archive is organized by date, which is wonderful (!) but might mean the contents may see changes during the day. I suspect this is not the case but just a thought. Regardless, any problems seem easily avoided by simply not updating to the current day and use yesterday instead. ---- If this is the way we move forward, there are changes/fixes/improvements to be made in terms of documentation as well as (possibly) putting together some update automation tooling. --- I have only tested this with a single project so far, and it doesn't have any especially esoteric or demanding needs. Help testing by checking your projects to see that they build and produce expected results. Building against the before/after set and using `diffpdf` is how I usually vet these updates, if that's useful. The `diffpdf` especially helps identify the minor appearance changes I usually miss visually-- tweaks to microtype behavior or slightly different layout/placement of figures or handling of beamer slides, etc.
Motivation for this change
Another year, another update!
This uses snapshot hosted on wdtz.org,
which may not be up to the task long-term.
Not sure how hard folks hit our TL mirrors...
Commit history is messy, showing path taken so far.
(Will clean up if/when reach something ready to merge)
Help testing and suggestions for cleanup appreciated!
For example, dvisvgm was having troubles I ultimately resolved
by setting
LDFLAGS
but likely has a better solution.Fixes #60890.
Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nix-review --run "nix-review wip"
./result/bin/
)nix path-info -S
before and after)