Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3.tensorflow_2: 2.1.0 -> 2.3.0 #94323

Closed
wants to merge 2 commits into from
Closed

python3.tensorflow_2: 2.1.0 -> 2.3.0 #94323

wants to merge 2 commits into from

Conversation

raboof
Copy link
Member

@raboof raboof commented Jul 31, 2020 •
edited

Motivation for this change

Make tensorflow available on python3 (the previous version was disabled for python 3.8 and later)

WIP: the build currently fails because it specifically requires Bazel 3.1.0, while nixpkgs ships Bazel 3.3.1.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested a review from abbradar July 31, 2020 11:20
@raboof raboof force-pushed the python3.tensorflow_2-update-to-2.3.0 branch 2 times, most recently from b9cb33e to 939c8ed Compare July 31, 2020 20:49
@raboof
bazel: add 3.1.0
fd61ef4 
Because specifically this version is required for tensorflow 2.3.0.

There's a lot of duplication with bazel_3 here. In this case duplication
seemed like the 'lesser evil' compared to making the build more generic,
since the latter seems like it would increase complexity quite heavily.
@raboof
python3.tensorflow_2: 2.1.0 -> 2.3.0
a853b05
@raboof raboof force-pushed the python3.tensorflow_2-update-to-2.3.0 branch from 939c8ed to a853b05 Compare August 1, 2020 09:14
@raboof
Copy link
Member Author

raboof commented Aug 1, 2020

The build currently fails because it tries to download rules_cc during the build:

Repository rule tf_http_archive defined at:
  /build/source/third_party/repo.bzl:134:19: in <toplevel>
Loading: 0 packages loaded
    currently loading: tensorflow/tools/pip_package ... (2 packages)
    Fetching @rules_cc; fetching
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_cc/archive/01d4a48911d5e7591ecb1c06d3b8af47fe872371.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: storage.googleapis.com
Loading: 0 packages loaded
    currently loading: tensorflow/tools/pip_package ... (2 packages)
WARNING: Download from https://github.com/bazelbuild/rules_cc/archive/01d4a48911d5e7591ecb1c06d3b8af47fe872371.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com
Loading: 0 packages loaded
    currently loading: tensorflow/tools/pip_package ... (2 packages)
ERROR: An error occurred during the fetch of repository 'rules_cc':
   java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_cc/archive/01d4a48911d5e7591ecb1c06d3b8af47fe872371.zip, https://github.com/bazelbuild/rules_cc/archive/01d4a48911d5e7591ecb1c06d3b8af47fe872371.zip] to /build/output/external/rules_cc/01d4a48911d5e7591ecb1c06d3b8af47fe872371.zip: Unknown host: github.com
Loading: 0 packages loaded

@Wulfsta
Copy link
Member

Wulfsta commented Aug 19, 2020 •
edited

Does adding

bazel-build = buildBazelPackage {
    ...
    removeRulesCC = false;
    ...
}

fix the issue?

I did a bit of work trying to get this compiling with a ROCm backend here, there might be a handful of useful things in that derivation.

@matthewbauer
Copy link
Member

I have a working version now in #95824 ,could you try that out?

@Wulfsta
Copy link
Member

Wulfsta commented Aug 20, 2020

I don't have a Nvidia card to test with at the moment, but I might be able to try a build later.

@raboof
Copy link
Member Author

raboof commented Aug 21, 2020

closing in favour of #95824

@raboof raboof closed this Aug 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abbradar abbradar Awaiting requested review from abbradar

@Profpatsch Profpatsch Awaiting requested review from Profpatsch Profpatsch will be requested when the pull request is marked ready for review Profpatsch is a code owner

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@raboof @Wulfsta @matthewbauer