Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel: 1.0 -> 1.1 #71612

Merged
merged 2 commits into from Oct 24, 2019
Merged

bazel: 1.0 -> 1.1 #71612

merged 2 commits into from Oct 24, 2019

Conversation

danjuv
Copy link
Contributor

@danjuv danjuv commented Oct 22, 2019

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @

@FRidh
Copy link
Member

FRidh commented Oct 22, 2019

A new minor release already, that's fast.

@flokli
Copy link
Contributor

flokli commented Oct 22, 2019

@GrahamcOfBorg build bazel.tests

@danjuv
Copy link
Contributor Author

danjuv commented Oct 23, 2019

Anyone have any ideas around why x86_64-darwin is failing with
error: build of '/nix/store/bl0bhqc50bjwlipq77zk6lygc5bgwdmy-bazel-test-shebangs.drv' failed?

@Profpatsch
Copy link
Member

Let’s see what it says when we only run that one.

@GrahamcOfBorg build bazel.tests.shebang

Btw, we have a million of these warnings:

/nix/store/rlwl76pcfzh9pmjx0mynpdb2spm49qmj-bash-4.4-p23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.ISO-8859-1): No such file or directory

Maybe we should add the locale archive. But I wonder why ISO-8859-1 anyway. Maybe that’s a bug.

@flokli
Copy link
Contributor

flokli commented Oct 23, 2019

@danjuv bazel tests were already broken on darwin before that - see #70994.

Fixing these tests on darwin should probably be done in a followup PR.

@Profpatsch
Copy link
Member

Bazel extracts itself to a cache directory every time it’s run (yes, that is as silly as it sounds).

For the tests we have an optimization in place, which puts that cache directory in a derivation, so that bazel doesn’t extract itself for every single test.

There is some magic, so we have a sanity-check in place in the setup code:

# bazel wants to extract itself into $install_dir/install every time it runs,
# so let’s do that only once.
extracted = bazelPkg:
let install_dir =
# `install_base` field printed by `bazel info`, minus the hash.
# yes, this path is kinda magic. Sorry.
"$HOME/.cache/bazel/_bazel_nixbld";
in runLocal "bazel-extracted-homedir" { passthru.install_dir = install_dir; } ''
export HOME=$(mktemp -d)
touch WORKSPACE # yeah, everything sucks
install_base="$(${bazelPkg}/bin/bazel info | grep install_base)"
# assert it’s actually below install_dir
[[ "$install_base" =~ ${install_dir} ]] \
|| (echo "oh no! $install_base but we are \
trying to copy ${install_dir} to $out instead!"; exit 1)
cp -R ${install_dir} $out
'';

This sanity-check is getting broken now. This probably means one of the following:

  • new bazel changed their install_base on darwin
  • nix changed something with how tmpdir and sandboxes work on darwin
  • the ofborg builder changed its darwin configuration to do more or less sandboxing

Since macOS’s sandboxing is highly primitive, nix builds are kind of leaky there.

I see two ways forward:

  • special case the extraction drv to only fire on non-darwin
  • add a TODO and merge with failing tests on darwin (we already have an issue for broken bazel tests on darwin)

@uri-canva
Copy link
Contributor

@flokli The preexisting failure you mention is a different one. I can't find bazel.tests on hydra, so I'm not sure if this failure is preexisting or not, if I try to build it on master I get:

$   nix-build -A bazel.tests
...
building '/nix/store/hw5gc3k552qxlxsnfhwa7wv0miy9vj7v-bazel-extracted-homedir.drv'...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Writing tracer profile to '/private/var/tmp/_bazel_uri/f0a8ba7d540c97e00b10579823ad40f8/command.profile.gz'
oh no! install_base: /var/tmp/_bazel_uri/install/657cd4cb9e53a7fefb922907cea40492 but we are trying to copy /private/var/folders/0d/f45mrlw51yj0fn50bs44xych0000gn/T/nix-build-bazel-extracted-homedir.drv-0/tmp.ggAbP92PJ0/.cache/bazel/_bazel_nixbld to /nix/store/q6mkbl93x51iqrs02kglg1q3kzs92006-bazel-extracted-homedir instead!
builder for '/nix/store/hw5gc3k552qxlxsnfhwa7wv0miy9vj7v-bazel-extracted-homedir.drv' failed with exit code 1
cannot build derivation '/nix/store/v2mdj058ii3h9vkcwbcpqs4zq6nzb2q6-bazel-test-shebangs.drv': 1 dependencies couldn't be built
error: build of '/nix/store/2bism00fj4rvff5x10w853kvf95cic7r-bazel-test-cpp.drv', '/nix/store/3hrq5v09yh89f8ng8pgaw12dqj0dxmdj-bazel-test-bash-tools.drv', '/nix/store/6fci9c1hjkgi26yi2apk2ncwqxxdy802-bazel-test-cpp.drv', '/nix/store/72ipr99sdanvflnk2dvfkjwyyh31yp3y-bazel-test-protocol-buffers.drv', '/nix/store/b58zm3zcag37g1bq0xnf8dlnr6ys0jfh-bazel-test-protocol-buffers.drv', '/nix/store/ccyzm62y83n8yw5kqpm5x316px361xm3-bazel-test-cpp.drv', '/nix/store/inqpjj71b2mx67l8cnwv20h9v3syx6pb-bazel-test-bash-tools.drv', '/nix/store/q5dal7hn6fw798hhk3c0d3n3v80is2av-bazel-test-builtin-rules.drv', '/nix/store/v2mdj058ii3h9vkcwbcpqs4zq6nzb2q6-bazel-test-shebangs.drv', '/nix/store/wkwk1jxay1hh3phpc4j5xyc1az9kfqpg-bazel-test-builtin-rules.drv', '/nix/store/zbw8xp6jip2h150cxwll12hw7bfanay7-bazel-test-cpp.drv' failed

@flokli
Copy link
Contributor

flokli commented Oct 23, 2019

@uri-canva I'm not sure if hydra is building bazel.tests, but #70994 was opened because it already was failing on master (according to ofborg's CI).

It might be this PR introduces another breakage on top of the existing one, since tests were already broken on macos, I don't think we should block other bazel PRs if tests on x86_64-linux and aarch64-linux still pass.

Of course, #70994 should be fixed, but at least @Profpatsch and I don't have a MacOS around to interactively debug this - if you could fine some time to dig into it, that would be very much appreciated.

@Profpatsch
Copy link
Member

Profpatsch commented Oct 24, 2019

Ah, found it. We were already skipping the extraction step on darwin, but the new shebang test runs it manually.

Adding commit …

@Profpatsch
Copy link
Member

@GrahamcOfBorg build bazel.tests

1 similar comment
@flokli
Copy link
Contributor

flokli commented Oct 24, 2019

@GrahamcOfBorg build bazel.tests

@uri-canva
Copy link
Contributor

Finally passed 🎉

@flokli flokli merged commit a484c07 into NixOS:master Oct 24, 2019
@aherrmann-da aherrmann-da mentioned this pull request Oct 28, 2019
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants