python3Packages.pytorch: create separate lib output #92617
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
do you mind posting the closure savings? as that's the primary goal of the pr? |
At some point pytorch.dev was added to expose the libtorch headers and libraries to non-Python users of libtorch. However, this output currently has two disadvantages: 1. An application that compiles against the dev output will also have the libtorch header files in its closure. This is not so nice when e.g. building Docker images of applications that use libtorch. 2. The dev output has a large transitive closure with many dependencies that are not necessary when compiling against libtorch. This change adds the `lib` output so that applications that only link against libtorch libraries have a small closure. Before this change, the libtorch dependency adds 746MiB: % nix path-info -S `realpath result-dev` /nix/store/10rmy81bjk628sfpbj2szxlws6brq1xn-python3.8-pytorch-1.5.1-dev 782203848 With this change it is reduced to 196MiB: % nix path-info -S `realpath result-lib` /nix/store/bck65lf0z7gdhcf89w1zs5nz333lhgwa-python3.8-pytorch-1.5.1-lib 205865056
danieldk
force-pushed
the
pytorch-lib-output
branch
from
68ef45c
to
0bdc835
Compare
|
Good point. I have amended the commit message with the following: |
|
Any objections to merging this? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
At some point pytorch.dev was added to expose the libtorch headers and
libraries to non-Python users of libtorch. However, this output
currently has two disadvantages:
the libtorch header files in its closure. This is not so nice when
e.g. building Docker images of applications that use libtorch.
that are not necessary when compiling against libtorch.
This change adds the
liboutput so that applications that only linkagainst libtorch libraries have a small closure.
Tested CUDA build ✔️
Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)nix path-info -Sbefore and after)