Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pythonPackages.tensorflow: repair cuda-dependent variant #30058

Merged
merged 1 commit into from Oct 10, 2017

Conversation

jyp
Copy link
Contributor

@jyp jyp commented Oct 3, 2017

merge the outputs of cudatoolkit locally in the tensorflow
derivation, using symlinkJoin

Fixes #29798

Motivation for this change

See #29798

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • Linux
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

merge the outputs of cudatoolkit locally in the tensorflow
derivation, using symlinkJoin

Fixes NixOS#29798
Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not tested, but looks good.

@jyp
Copy link
Contributor Author

jyp commented Oct 5, 2017

@abbradar what do you think?

@abbradar
Copy link
Member

abbradar commented Oct 5, 2017

I don't have access to my trusty nvidia yet, but the problem is that it cannot find the library at runtime, correct? If so, instead of using symlinkJoin you could just add both paths to rpath -- should be more clean.

@jyp
Copy link
Contributor Author

jyp commented Oct 5, 2017

@abbradar adding both to the rpath actually did not work. Apparently this is due to a quirk in the pre-built tensorflow wheel. The symlinkJoin trick was found by @TravisWhitaker #29575

@abbradar
Copy link
Member

abbradar commented Oct 5, 2017

Ah, no problem then. Looks good to me!

@jyp
Copy link
Contributor Author

jyp commented Oct 5, 2017 via email

@jyp
Copy link
Contributor Author

jyp commented Oct 9, 2017

@FRidh Any chance to merge this? Or is someone else in charge?

@abbradar abbradar merged commit 244f347 into NixOS:master Oct 10, 2017
@abbradar
Copy link
Member

Tested that it imports but not runs (trust you have tested this with CUDA). Thank you!

andersk added a commit to andersk/nixpkgs that referenced this pull request Apr 7, 2018
Similar to NixOS#30058 for TensorFlow.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
FRidh pushed a commit that referenced this pull request May 4, 2018
* pytorch-0.3 with optional cuda and cudnn

* pytorch tests reenabled if compiling without cuda

* pytorch: Conditionalize cudnn dependency on cudaSupport

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: Compile with the same GCC version used by CUDA if cudaSupport

Fixes this error:

In file included from /nix/store/gv7w3c71jg627cpcff04yi6kwzpzjyap-cudatoolkit-9.1.85.1/include/host_config.h:50:0,
                 from /nix/store/gv7w3c71jg627cpcff04yi6kwzpzjyap-cudatoolkit-9.1.85.1/include/cuda_runtime.h:78,
                 from <command-line>:0:
/nix/store/gv7w3c71jg627cpcff04yi6kwzpzjyap-cudatoolkit-9.1.85.1/include/crt/host_config.h:121:2: error: #error -- unsupported GNU version! gcc versions later than 6 are not supported!
 #error -- unsupported GNU version! gcc versions later than 6 are not supported!
  ^~~~~

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: Build with joined cudatoolkit

Similar to #30058 for TensorFlow.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: 0.3.0 -> 0.3.1

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: Patch for “refcounted file mapping not supported” failure

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: Skip distributed tests

Signed-off-by: Anders Kaseorg <andersk@mit.edu>

* pytorch: Use the stub libcuda.so from cudatoolkit for running tests

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants