Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tensorflow: 1.9.0 -> 1.11.0 #45709

Merged
merged 2 commits into from Oct 15, 2018
Merged

tensorflow: 1.9.0 -> 1.11.0 #45709

merged 2 commits into from Oct 15, 2018

Conversation

jyp
Copy link
Contributor

@jyp jyp commented Aug 28, 2018

Motivation for this change

Upstream update

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@jyp jyp requested a review from FRidh as a code owner August 28, 2018 14:17
@jyp jyp changed the title Tf 1.10.1 tensorflow 1.10.1 Aug 28, 2018
@Mic92 Mic92 changed the title tensorflow 1.10.1 tensorflow: 1.9.0 -> 1.10.1 Aug 28, 2018
@globin
Copy link
Member

globin commented Aug 28, 2018

@GrahamcOfBorg build python3.pkgs.Keras python2.pkgs.Keras

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python3.pkgs.Keras, python2.pkgs.Keras

Partial log (click to expand)

copying path '/nix/store/4kcc23p7axcxzq8swsgir7lzprxlnjbs-python2.7-Keras_Applications-1.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/131j7m6fgccgd32xvynnq08clpa8w3x3-python3.6-h5py-2.8.0' from 'https://cache.nixos.org'...
copying path '/nix/store/8yg7j9ydc1l2v31ysx1liycr2gcxzfj4-python3.6-scipy-1.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/425h8ngjjjrkv2pqz8ss61sqiwb1z0za-python2.7-Keras_Preprocessing-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/c3qxblr816af05cf9rk7wmqbffc71s2w-python3.6-Keras_Applications-1.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/sqmxyrn9nx464wlwsdzzvkralakjz8pk-python2.7-Keras-2.2.2' from 'https://cache.nixos.org'...
copying path '/nix/store/y2kzbyjdz512sa30iwbsbqcks2fjy9if-python3.6-Keras_Preprocessing-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/s9ncmqf7569bblwcdwdv1g5hkn500xim-python3.6-Keras-2.2.2' from 'https://cache.nixos.org'...
/nix/store/s9ncmqf7569bblwcdwdv1g5hkn500xim-python3.6-Keras-2.2.2
/nix/store/sqmxyrn9nx464wlwsdzzvkralakjz8pk-python2.7-Keras-2.2.2

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python3.pkgs.Keras, python2.pkgs.Keras

Partial log (click to expand)

copying path '/nix/store/y57biarfbl988j25rkms17q7kkvlhyz1-python2.7-Keras_Applications-1.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/d03g3fr5qsg3v97rjvflm98bgyk4s5kf-python3.6-numpy-1.14.5' from 'https://cache.nixos.org'...
copying path '/nix/store/9rzy38fqnig6q8v2yl8i35xdirlwi7wa-python2.7-Keras-2.2.2' from 'https://cache.nixos.org'...
copying path '/nix/store/9gbm7c4bl2dmi4rx4183rnh838dqhhl5-python3.6-h5py-2.8.0' from 'https://cache.nixos.org'...
copying path '/nix/store/s3wjqkdi6sz8xp9cgn9mnqdjsscfknmr-python3.6-scipy-1.1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/dailh3c67xbq4jc6y864wngg64q8qzvh-python3.6-Keras_Applications-1.0.4' from 'https://cache.nixos.org'...
copying path '/nix/store/xfs0ilxqwj0gky8w6zknxk227ncvhs01-python3.6-Keras_Preprocessing-1.0.2' from 'https://cache.nixos.org'...
copying path '/nix/store/4dry92hn300f1qlwykf9c4yjrlx1wvwl-python3.6-Keras-2.2.2' from 'https://cache.nixos.org'...
/nix/store/4dry92hn300f1qlwykf9c4yjrlx1wvwl-python3.6-Keras-2.2.2
/nix/store/9rzy38fqnig6q8v2yl8i35xdirlwi7wa-python2.7-Keras-2.2.2

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python3.pkgs.Keras, python2.pkgs.Keras

Partial log (click to expand)

Requirement already satisfied: six>=1.9.0 in /nix/store/xibkimbc2faad16k9rsvvssxspxajdly-python3.6-six-1.11.0/lib/python3.6/site-packages (from Keras==2.2.2) (1.11.0)
Requirement already satisfied: h5py in /nix/store/ni5bffgvfv09m248fzgadqvfrvaydyy8-python3.6-h5py-2.8.0/lib/python3.6/site-packages (from Keras==2.2.2) (2.8.0)
Requirement already satisfied: pyyaml in /nix/store/sym4r11pq4bvf4pwaj2gk1n8njisnciv-python3.6-PyYAML-3.13/lib/python3.6/site-packages (from Keras==2.2.2) (3.13)
Installing collected packages: Keras
Successfully installed Keras-2.2.2
/private/tmp/nix-build-python3.6-Keras-2.2.2.drv-0/Keras-2.2.2
post-installation fixup
strip is /nix/store/df6k4mgdjxciy0f637lryp7c9ln7n1m3-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/00pizxvsbgjz5wp79nyyyld7wqsv8vyf-python3.6-Keras-2.2.2/lib
patching script interpreter paths in /nix/store/00pizxvsbgjz5wp79nyyyld7wqsv8vyf-python3.6-Keras-2.2.2

@globin
Copy link
Member

globin commented Aug 28, 2018

@GrahamcOfBorg build python3.pkgs.tensorflow python2.pkgs.tensorflow

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python3.pkgs.tensorflow

The following builds were skipped because they don't evaluate on x86_64-darwin: python2.pkgs.tensorflow

Partial log (click to expand)

strip is /nix/store/df6k4mgdjxciy0f637lryp7c9ln7n1m3-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/lib  /nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin
patching script interpreter paths in /nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1
/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/lib/python3.6/site-packages/tensorflow/aux-bin/tflite_convert: interpreter directive changed from "/usr/bin/env python" to "/nix/store/dlswl7xjvh3bsfydygqzi2n0f82p5d9p-python3-3.6.6/bin/python"
wrapping `/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin/toco_from_protos'...
wrapping `/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin/freeze_graph'...
wrapping `/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin/tflite_convert'...
wrapping `/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin/saved_model_cli'...
wrapping `/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1/bin/toco'...
/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: python3.pkgs.tensorflow, python2.pkgs.tensorflow

Partial log (click to expand)

unpacking sources
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
building
installing
/build/dist /build
tensorflow-1.10.1-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.
builder for '/nix/store/z5gvmddm4zgq3h8snad3dhmfdagq7ycy-python3.6-tensorflow-1.10.1.drv' failed with exit code 1
error: build of '/nix/store/ycsnndig14bqx42cn9rm8jlqhicm7w6y-python2.7-tensorflow-1.10.1.drv', '/nix/store/z5gvmddm4zgq3h8snad3dhmfdagq7ycy-python3.6-tensorflow-1.10.1.drv' failed

@@ -0,0 +1,6 @@
{ callPackage, lib, ... }:

lib.overrideDerivation (callPackage ./generic-v3.nix {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should use overrideAttrs instead according to https://nixos.org/nixpkgs/manual/#sec-pkg-overrideDerivation

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python3.pkgs.tensorflow, python2.pkgs.tensorflow

Partial log (click to expand)

patching script interpreter paths in /nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1
/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/lib/python3.6/site-packages/tensorflow/aux-bin/tflite_convert: interpreter directive changed from "/usr/bin/env python" to "/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/bin/python"
checking for references to /build in /nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/toco'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/saved_model_cli'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/tflite_convert'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/toco_from_protos'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/freeze_graph'...
/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1
/nix/store/85m63m4f503xfdb73ana1napizsz83zd-python2.7-tensorflow-1.10.1

@jyp
Copy link
Contributor Author

jyp commented Aug 29, 2018

@symphorien Thanks for your feedback, I have applied your suggestion.

@xeji
Copy link
Contributor

xeji commented Aug 30, 2018

@GrahamcOfBorg build protobuf3_6 python3.pkgs.tensorflow python2.pkgs.tensorflow python3.pkgs.tensorflow-tensorboard python2.pkgs.tensorflow-tensorboard

@@ -9863,7 +9863,7 @@ in {
protobuf = callPackage ../development/python-modules/protobuf {
disabled = isPyPy;
doCheck = !isPy3k;
protobuf = pkgs.protobuf3_5;
protobuf = pkgs.protobuf3_6;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change affects more packages that will going to need to be checked.

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: protobuf3_6, python3.pkgs.tensorflow, python3.pkgs.tensorflow-tensorboard, python2.pkgs.tensorflow-tensorboard

The following builds were skipped because they don't evaluate on x86_64-darwin: python2.pkgs.tensorflow

Partial log (click to expand)

/private/tmp/nix-build-python2.7-tensorflow-tensorboard-1.10.0.drv-0
post-installation fixup
strip is /nix/store/df6k4mgdjxciy0f637lryp7c9ln7n1m3-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/k6ydn3vk0i07f2im7jips21fwfr40xak-python2.7-tensorflow-tensorboard-1.10.0/lib  /nix/store/k6ydn3vk0i07f2im7jips21fwfr40xak-python2.7-tensorflow-tensorboard-1.10.0/bin
patching script interpreter paths in /nix/store/k6ydn3vk0i07f2im7jips21fwfr40xak-python2.7-tensorflow-tensorboard-1.10.0
wrapping `/nix/store/k6ydn3vk0i07f2im7jips21fwfr40xak-python2.7-tensorflow-tensorboard-1.10.0/bin/tensorboard'...
/nix/store/kpawns8pc8n7l9bm1bampkgbdzxkln49-protobuf-3.6.1
/nix/store/305iabbn9da0bqkhpk4yi67xc71vh5c0-python3.6-tensorflow-1.10.1
/nix/store/avrvprkxba6s3pwwygqqp8gf0xr20190-python3.6-tensorflow-tensorboard-1.10.0
/nix/store/k6ydn3vk0i07f2im7jips21fwfr40xak-python2.7-tensorflow-tensorboard-1.10.0

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: protobuf3_6, python3.pkgs.tensorflow, python2.pkgs.tensorflow, python3.pkgs.tensorflow-tensorboard, python2.pkgs.tensorflow-tensorboard

Partial log (click to expand)

wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/freeze_graph'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/toco'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/saved_model_cli'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/toco_from_protos'...
wrapping `/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1/bin/tflite_convert'...
/nix/store/1krnj68gh7b5cg3qpcf291k9g5gy1ky7-protobuf-3.6.1
/nix/store/li1j6jpgin0h8x9w34nxcc3p3d3vncyq-python3.6-tensorflow-1.10.1
/nix/store/85m63m4f503xfdb73ana1napizsz83zd-python2.7-tensorflow-1.10.1
/nix/store/033z8afzig8h59r3bn2c0rn878jpg92z-python3.6-tensorflow-tensorboard-1.10.0
/nix/store/hczcsrmqhvaqd5ccc882sp78fw2z501d-python2.7-tensorflow-tensorboard-1.10.0

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: protobuf3_6, python3.pkgs.tensorflow, python2.pkgs.tensorflow, python3.pkgs.tensorflow-tensorboard, python2.pkgs.tensorflow-tensorboard

Partial log (click to expand)

unpacking sources
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
building
installing
/build/dist /build
tensorflow-1.10.1-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.
builder for '/nix/store/z5gvmddm4zgq3h8snad3dhmfdagq7ycy-python3.6-tensorflow-1.10.1.drv' failed with exit code 1
error: build of '/nix/store/ycsnndig14bqx42cn9rm8jlqhicm7w6y-python2.7-tensorflow-1.10.1.drv', '/nix/store/z5gvmddm4zgq3h8snad3dhmfdagq7ycy-python3.6-tensorflow-1.10.1.drv' failed

@jyp
Copy link
Contributor Author

jyp commented Aug 30, 2018

@xeji note that tensorboard is a dependency of tensorflow (weird I know).
@FRidh right, please let me know if I'm not doing things as they should be done.

@orivej
Copy link
Contributor

orivej commented Sep 25, 2018

@jyp Could you rebase onto the current master? Since #47353 Protobuf 3.6 is the default.

@jyp
Copy link
Contributor Author

jyp commented Sep 26, 2018

@orivej Done, thanks for the heads up.
@FRidh Please re-consider merging this :)

@globin
Copy link
Member

globin commented Sep 26, 2018

Could you remove the old hashes file, otherwise looks fine.

@jyp
Copy link
Contributor Author

jyp commented Sep 28, 2018

@globin I have removed the old hashes. Also, tensorflow 1.11 was released, so I have updated to that.

@jyp jyp changed the title tensorflow: 1.9.0 -> 1.10.1 tensorflow: 1.9.0 -> 1.11.0 Sep 28, 2018
@globin
Copy link
Member

globin commented Oct 15, 2018

@GrahamcOfBorg build python3.pkgs.tensorflow python2.pkgs.tensorflow

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Attempted: python3.pkgs.tensorflow, python2.pkgs.tensorflow

Partial log (click to expand)

unpacking sources
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
building
installing
/build/dist /build
tensorflow-1.11.0-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.
builder for '/nix/store/cijd4yqir9n8v9rl4vlc6j8jyj6nz6pm-python3.6-tensorflow-1.11.0.drv' failed with exit code 1
error: build of '/nix/store/cijd4yqir9n8v9rl4vlc6j8jyj6nz6pm-python3.6-tensorflow-1.11.0.drv', '/nix/store/sdwzgiz6zyfxrs2i6pqaghzp2hwd8r1w-python2.7-tensorflow-1.11.0.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python3.pkgs.tensorflow

The following builds were skipped because they don't evaluate on x86_64-darwin: python2.pkgs.tensorflow

Partial log (click to expand)

strip is /nix/store/53nysl8bqwxihwzs1pgwka20nf8mbvlp-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/lib  /nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin
patching script interpreter paths in /nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0
/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/lib/python3.6/site-packages/tensorflow/aux-bin/tflite_convert: interpreter directive changed from "/usr/bin/env python" to "/nix/store/8rw1vq2k50g7jgniskzs51qjb47w8nhz-python3-3.6.6/bin/python"
wrapping `/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin/freeze_graph'...
wrapping `/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin/saved_model_cli'...
wrapping `/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin/tflite_convert'...
wrapping `/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin/toco'...
wrapping `/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0/bin/toco_from_protos'...
/nix/store/0mhr5rrfa8n3kan0san1crjr3ijj7jmg-python3.6-tensorflow-1.11.0

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python3.pkgs.tensorflow, python2.pkgs.tensorflow

Partial log (click to expand)

patching script interpreter paths in /nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0
/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/lib/python3.6/site-packages/tensorflow/aux-bin/tflite_convert: interpreter directive changed from "/usr/bin/env python" to "/nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/bin/python"
checking for references to /build in /nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0...
wrapping `/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/bin/freeze_graph'...
wrapping `/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/bin/toco'...
wrapping `/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/bin/saved_model_cli'...
wrapping `/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/bin/tflite_convert'...
wrapping `/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0/bin/toco_from_protos'...
/nix/store/6ckfdl9l0vny7mnsq6my57dp0i0pkclr-python3.6-tensorflow-1.11.0
/nix/store/3n27wdjsqiaj26n340acm55zykf55pz5-python2.7-tensorflow-1.11.0

@globin globin merged commit a8a3928 into NixOS:master Oct 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants