Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libtensorflow: 1.9 -> 1.14.0 #63208

Closed
wants to merge 1 commit into from
Closed

libtensorflow: 1.9 -> 1.14.0 #63208

wants to merge 1 commit into from

Conversation

yorickvP
Copy link
Contributor

This also changes it to a from-source build.

Motivation for this change

Update, also convert to built-from-source package.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

This still needs testing everywhere. The linux build without cuda compiled one time. I can't test on darwin.


cc: @basvandijk

@basvandijk
Copy link
Member

@GrahamcOfBorg build libtensorflow

@FRidh
Copy link
Member

FRidh commented Jun 16, 2019

Should close #47025 when this is merged.

@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 17, 2019

I'm working on upgrading the haskell package: tensorflow/haskell#244

Copy link
Member

@Profpatsch Profpatsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not extremely excited about having to maintain multiple versions of bazel.

Can you please put the renaming file action into a separate commit, so that I can see the differences in generic.nix?

@yorickvP
Copy link
Contributor Author

I've updated to 1.14.0-rc1, which does compile with bazel 0.26.1. Not ideal but probably better than including another bazel version.

@yorickvP
Copy link
Contributor Author

I'm working on cuda support now.

@yorickvP
Copy link
Contributor Author

yorickvP commented Jun 18, 2019

Cuda support compiles now. Feel free to merge.

@Profpatsch
Copy link
Member

cc @basvandijk

@timokau
Copy link
Member

timokau commented Jun 24, 2019

I wish I had seen this before starting to work on #63616! Lots of good work here. We should merge the libtensorflow derivation with the tensorflow wheel build. Probably best to build both at the same time and then split it, to save build time.

This also changes it to a from-source build.
@yorickvP
Copy link
Contributor Author

I updated this to 1.14.0 final. @timokau that seems to have some good changes. Using system libraries might help a lot with our dynamic linking problems! Any ideas about how to do the multiple build outputs if this is built for multiple python versions?

@timokau
Copy link
Member

timokau commented Jul 1, 2019

Its probably not easy, maybe not even desirable to do this with multiple outputs. It would be possible to build multiple different bazel targets and put them in different outputs. That would add some complexity however and would always force everyone who wants to build one to build all of them.

@FRidh FRidh changed the title libtensorflow: 1.9 -> 1.13.1 libtensorflow: 1.9 -> 1.14 Jul 11, 2019
@FRidh FRidh changed the title libtensorflow: 1.9 -> 1.14 libtensorflow: 1.9 -> 1.14.0 Jul 11, 2019
@yorickvP
Copy link
Contributor Author

Superseded by #64716

@yorickvP yorickvP closed this Jul 14, 2019
abbradar added a commit to abbradar/nixpkgs that referenced this pull request Jul 14, 2019
This merges work done by yorickvP and timokau in NixOS#63208 and NixOS#63616 respectively.
Now the derivation builds both libtensorflow and the Python package and puts them into
different outputs.

Quite a bit of improvements were done on the top, including:

* Use official tag revision as source, not a branch;
* Use all system libraries possible (before only one was actually used);
* Move various environment variables to the derivation itself from hooks;
* Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
* And more that I forgot!
abbradar added a commit to abbradar/nixpkgs that referenced this pull request Jul 31, 2019
This merges work done by yorickvP and timokau in NixOS#63208 and NixOS#63616 respectively.
Now the derivation builds both libtensorflow and the Python package and puts them into
different outputs.

Quite a bit of improvements were done on the top, including:

* Use official tag revision as source, not a branch;
* Use all system libraries possible (before only one was actually used);
* Move various environment variables to the derivation itself from hooks;
* Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
* And more that I forgot!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants