Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel_latest: 2.1.0 -> 3.0.0 #85356

Closed
wants to merge 1 commit into from
Closed

bazel_latest: 2.1.0 -> 3.0.0 #85356

wants to merge 1 commit into from

Conversation

uri-canva
Copy link
Contributor

@uri-canva uri-canva commented Apr 16, 2020

Motivation for this change

Upgrade to latest version of Bazel.

Things done

Renamed bazel_2 to bazel_latest, upgraded.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@uri-canva
Copy link
Contributor Author

Not sure what this is about, is it a preexisting issue on master?

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3
shrinking /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3/bin/ibazel
cannot find section .dynamic
strip is /nix/store/sq2b0dqlq243mqn4ql5h36xmpplyy20k-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3/bin
patching script interpreter paths in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3
checking for references to /build/ in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3...
cannot find section .dynamic
error: build of '/nix/store/i33llps78bk1145jmkjx5vlmdg0fmamr-bazel-test-shebangs.drv' failed

@edef1c
Copy link
Member

edef1c commented Apr 16, 2020

This removes Bazel 2, which I don't think is what we want. I think a separate commit introducing Bazel 3 (bazel_3: init at 3.00) is in order, without changing the default.

@jonringer
Copy link
Contributor

correct, please leave the individual bazel version, then bazel_latest should just be aliased to whatever happens to be latest

@jonringer
Copy link
Contributor

don't want to end up with tensorflow broken for another 4 months

@uri-canva
Copy link
Contributor Author

Tensorflow uses bazel_0_26 and bazel_0_29, and I saw a PR for a version of tensorflow that uses bazel_1, is there one that uses bazel_2? Rather than adding a new bazel derivation for each bazel version regardless of whether other derivations are using them or not, wouldn't it make sense to only add versions that are used? So rather than leave bazel_2 and add bazel_3 now, then bazel_4 etc, we can keep bazel_latest rolling, and add bazel_2 when we update tensorflow to a version that needs bazel_2, bazel_3 when we need that etc. And when we remove the last package that is using bazel_0_26 we can remove that.

That way we don't add more and more bazel derivations with no clear way of reducing the number of them.

@uri-canva
Copy link
Contributor Author

Bazel releases a new major at most every 3 months, according to:

https://docs.google.com/document/d/1NCKLVwYMMp7Wjpb4-49FuVCWmQci6i4WZQVvm3u-WcI/edit#heading=h.t52pudpvuk1s

Bazel 1.0.0: 2019-10-10 https://github.com/bazelbuild/bazel/releases/tag/1.0.0
Bazel 2.0.0: 2019-12-19 https://github.com/bazelbuild/bazel/releases/tag/2.0.0
Bazel 3.0.0: 2020-04-06 https://github.com/bazelbuild/bazel/releases/tag/3.0.0

This is why I think there's too many version to keep them all regardless of whether another derivation is using them or not.

@jonringer
Copy link
Contributor

I can definitely see the use case where a future version of tensorflow requires bazel_2.

@timokau
Copy link
Member

timokau commented Apr 17, 2020

One possible policy would be to always keep

  • The current release
  • The last major release
  • All releases that are currently in use

Keeping the last major release will make it possible to use a semi-recent version of bazel in packages like tensorflow which always lag behind a bit.

@bhipple
Copy link
Contributor

bhipple commented Apr 18, 2020

Tensorflow 2.1.0 (latest stable release) requires bazel 0.29; they're currently doing the release candidates for 2.2.0, which looks like it's going to be on bazel 3.0:
tensorflow/tensorflow@2f65c24

So it might be OK to drop bazel 2.0.

@uri-canva
Copy link
Contributor Author

Rebased on current master to see if the issue is still there.

ping @mboes @kalbasit @Profpatsch

@uri-canva
Copy link
Contributor Author

I'm still getting the same error, but I haven't figured out how to reproduce it locally.

@uri-canva uri-canva closed this Apr 28, 2020
@uri-canva uri-canva deleted the bazel branch April 28, 2020 05:51
@Profpatsch
Copy link
Member

We can revive bazel versions posthumously if some package upgrades to an older version.

@bhipple
Copy link
Contributor

bhipple commented Apr 28, 2020

Agreed, I'd say delete old bazel versions, unless it's actually being used by something that can't upgrade to a newer one.

@avdv avdv mentioned this pull request May 13, 2020
10 tasks
@Profpatsch
Copy link
Member

Huh @uri-canva, what’s the reason this PR was closed?

@uri-canva
Copy link
Contributor Author

I never managed to figure out the reason behind this error on ofborg:

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3
shrinking /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3/bin/ibazel
cannot find section .dynamic
strip is /nix/store/sq2b0dqlq243mqn4ql5h36xmpplyy20k-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3/bin
patching script interpreter paths in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3
checking for references to /build/ in /nix/store/qfsz9zn71803bhwqzr8fmhkcbfp58762-bazel-watcher-0.10.3...
cannot find section .dynamic
error: build of '/nix/store/i33llps78bk1145jmkjx5vlmdg0fmamr-bazel-test-shebangs.drv' failed

I switched to using bazelisk from nix and instead of bazel, and letting bazelisk get bazel, so I closed this PR so other people wouldn't get blocked on it and could start their own attempts to upgrade.

@mjlbach
Copy link
Contributor

mjlbach commented May 16, 2020

FYI Tensorflow 2.2 build with bazel 2.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants