bazel: Set a sensible strict action environment. #43530

mboes · 2018-07-14T15:59:07Z

Bazel either reuses the PATH from the client, or sets a hardcoded
one. The former mode in problematic for build hermeticity. But the
latter is crippled on NixOS, because the hardcoded value is
/bin:/usr/bin. So we set the hardcoded value to match what
customBash provides. This has the effect of aligning the
environments for ctx.actions.run and ctx.actions.run_shell, which
were previously distinct (bug).

cc @uri-canva

Profpatsch · 2018-07-16T11:38:10Z

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-07-16T11:38:48Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-16T11:40:57Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

./tools/test/collect_coverage.sh: interpreter directive changed from "/bin/bash -x" to "/nix/store/6ylxcy8rr5ssyqfwk7k8hmjywl6c5jvw-bash/bin/bash -x"
./tools/test/test-setup.sh: interpreter directive changed from "/bin/bash" to "/nix/store/6ylxcy8rr5ssyqfwk7k8hmjywl6c5jvw-bash/bin/bash"
configuring
no configure script, doing nothing
building
Building Bazel from scratch....../usr/bin/xcrun clang -fobjc-arc -framework CoreServices -framework Foundation -o /tmp/.bazel-30001/bazel_KBMOqhi6/archive/_embedded_binaries/xcode-locator tools/osx/xcode_locator.m
xcode-select: error: no developer tools were found at '/Applications/Xcode.app', and no install could be requested (perhaps no UI is present), please install manually from 'developer.apple.com'.
/nix/store/7amnc2662p390bgzj6c0snylj19vxmj2-stdenv-darwin/setup: line 1291: ./output/bazel: No such file or directory
builder for '/nix/store/5vyr0l003nfqpkqrnjjb31k0n731qd4z-bazel-0.13.0.drv' failed with exit code 127
�[31;1merror:�[0m build of '/nix/store/5vyr0l003nfqpkqrnjjb31k0n731qd4z-bazel-0.13.0.drv' failed

GrahamcOfBorg · 2018-07-16T12:08:34Z

Failure on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

src/main/cpp/blaze_util_posix.cc: In function 'uint64_t blaze::AcquireLock(const string&, bool, bool, blaze::BlazeLock*)':
src/main/cpp/blaze_util_posix.cc:667:3: warning: ignoring return value of 'int ftruncate(int, __off_t)', declared with attribute warn_unused_result [-Wunused-result]
   (void) ftruncate(lockfd, 0);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main/cpp/blaze_util_posix.cc: At global scope:
src/main/cpp/blaze_util_posix.cc:404:15: warning: 'std::__cxx11::string blaze::RunProgram(const string&, const std::vector<std::__cxx11::basic_string<char> >&)' defined but not used [-Wunused-function]
 static string RunProgram(const string& exe,
               ^~~~~~~~~~
building of ‘/nix/store/hbcq8a1nkqi509skcjdpck4di0cyn9n8-bazel-0.13.0.drv’ timed out after 1800 seconds
error: build of ‘/nix/store/hbcq8a1nkqi509skcjdpck4di0cyn9n8-bazel-0.13.0.drv’ failed

Profpatsch · 2018-07-18T12:54:56Z

Rebased to master.

@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-07-18T12:55:22Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: bazel

Partial log (click to expand)

Cannot nix-instantiate `bazel' because:
GC Warning: Failed to expand heap by 10737418240 bytes
Can't start up: not enough memory

Profpatsch · 2018-07-18T13:44:37Z

Retriggering because of memory failure
@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-07-18T13:44:57Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: bazel

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-07-18T13:45:17Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: bazel

Partial log (click to expand)

Cannot nix-instantiate `bazel' because:
GC Warning: Failed to expand heap by 10737418240 bytes
Can't start up: not enough memory

GrahamcOfBorg · 2018-07-18T13:45:24Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

building '/nix/store/b1x1yflwkw15hwvnypqwbj6fjlnc4r1d-bazel-0.15.1.drv'...
unpacking sources
unpacking source archive /nix/store/62081xrk8ipgcj9a0xm8dvcpk657sjfg-bazel-0.15.1-dist.zip
source root is .
setting SOURCE_DATE_EPOCH to timestamp 1531925814 of file ./.sandbox.sb
warning: file ./.sandbox.sb may be generated; SOURCE_DATE_EPOCH may be non-deterministic
patching sources
substitute(): ERROR: file 'src/main/java/com/google/devtools/build/lib/bazel/rules/BazelConfiguration.java' does not exist
builder for '/nix/store/b1x1yflwkw15hwvnypqwbj6fjlnc4r1d-bazel-0.15.1.drv' failed with exit code 1
�[31;1merror:�[0m build of '/nix/store/b1x1yflwkw15hwvnypqwbj6fjlnc4r1d-bazel-0.15.1.drv' failed

GrahamcOfBorg · 2018-07-18T13:55:14Z

Success on aarch64-linux (full log)

Attempted: hello

Partial log (click to expand)

/nix/store/l51wb4n37sslmchg9ncb8v0mcgqxwfz1-hello-2.10

Profpatsch · 2018-07-18T15:07:46Z

once more
@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-07-18T15:09:26Z

Failure on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

copying path '/nix/store/1vmpmfr713fpp8pwvpfqvibhy2hz0dmw-openjdk-8u172b11' from 'https://cache.nixos.org'...
building '/nix/store/f78ywnwf78ihvncn6hd4s9iyl76crs8n-bazel-0.15.1.drv'...
unpacking sources
unpacking source archive /nix/store/62081xrk8ipgcj9a0xm8dvcpk657sjfg-bazel-0.15.1-dist.zip
source root is .
setting SOURCE_DATE_EPOCH to timestamp 315532800 of file ./tools/zip/BUILD.tools
patching sources
substitute(): ERROR: file 'src/main/java/com/google/devtools/build/lib/bazel/rules/BazelConfiguration.java' does not exist
builder for '/nix/store/f78ywnwf78ihvncn6hd4s9iyl76crs8n-bazel-0.15.1.drv' failed with exit code 1
error: build of '/nix/store/f78ywnwf78ihvncn6hd4s9iyl76crs8n-bazel-0.15.1.drv' failed

Profpatsch · 2018-07-18T16:31:05Z

Ah, the string moved to BazelRuleClassProvider.java in the same folder.

Bazel either reuses the `PATH` from the client, or sets a hardcoded one. The former mode in problematic for build hermeticity. But the latter is crippled on NixOS, because the hardcoded value is `/bin:/usr/bin`. So we set the hardcoded value to match what `customBash` provides. This has the effect of aligning the environments for `ctx.actions.run` and `ctx.actions.run_shell`, which were previously distinct (bug).

Profpatsch · 2018-07-18T16:54:45Z

One more time!
@GrahamcOfBorg build bazel

GrahamcOfBorg · 2018-07-18T16:57:57Z

Failure on x86_64-darwin (full log)

Attempted: bazel

Partial log (click to expand)

./tools/test/collect_coverage.sh: interpreter directive changed from "/bin/bash -x" to "/nix/store/4b394rnkm139ykzzhgl453g7pwj2wh8v-bash/bin/bash -x"
./tools/test/test-setup.sh: interpreter directive changed from "/bin/bash" to "/nix/store/4b394rnkm139ykzzhgl453g7pwj2wh8v-bash/bin/bash"
configuring
no configure script, doing nothing
building
Building Bazel from scratch....../usr/bin/xcrun clang -fobjc-arc -framework CoreServices -framework Foundation -o /tmp/.bazel-30001/bazel_WbI8Um1G/archive/_embedded_binaries/xcode-locator tools/osx/xcode_locator.m
xcode-select: error: no developer tools were found at '/Applications/Xcode.app', and no install could be requested (perhaps no UI is present), please install manually from 'developer.apple.com'.
/nix/store/znciimv8frnpfdhqpwc2nmxfgrjvxs1b-stdenv-darwin/setup: line 1291: ./output/bazel: No such file or directory
builder for '/nix/store/pdx5dpdqg9j9vzmbv79zs353jjvgb1bl-bazel-0.15.1.drv' failed with exit code 127
�[31;1merror:�[0m build of '/nix/store/pdx5dpdqg9j9vzmbv79zs353jjvgb1bl-bazel-0.15.1.drv' failed

GrahamcOfBorg · 2018-07-18T17:47:35Z

Success on x86_64-linux (full log)

Attempted: bazel

Partial log (click to expand)

//examples/java-native/src/test/java/com/example/myproject:hello         PASSED in 0.8s

Executed 2 out of 2 tests: 2 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
INFO: Build completed successfully, 22 total actions
installing
post-installation fixup
patching script interpreter paths in /nix/store/0xpl3l0mwqshx5hccha77fq88irg6vay-bazel-0.15.1
checking for references to /tmp/.bazel-1000 in /nix/store/0xpl3l0mwqshx5hccha77fq88irg6vay-bazel-0.15.1...
/nix/store/0xpl3l0mwqshx5hccha77fq88irg6vay-bazel-0.15.1

Profpatsch · 2018-07-19T07:50:57Z

Finally! \o/

mboes force-pushed the bazel-action-env branch from 399ef1a to e4f1cd4 Compare July 14, 2018 16:09

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jul 14, 2018

mboes force-pushed the bazel-action-env branch from e4f1cd4 to a51a777 Compare July 14, 2018 16:35

bazel: make more tools available in customBash.

e2fa03c

Profpatsch force-pushed the bazel-action-env branch from a51a777 to 887bfc9 Compare July 18, 2018 12:54

NixOS deleted a comment from GrahamcOfBorg Jul 18, 2018

Profpatsch force-pushed the bazel-action-env branch from 887bfc9 to dd6c83f Compare July 18, 2018 16:54

NixOS deleted a comment from GrahamcOfBorg Jul 18, 2018

Profpatsch merged commit c27f686 into NixOS:master Jul 19, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bazel: Set a sensible strict action environment. #43530

bazel: Set a sensible strict action environment. #43530

mboes commented Jul 14, 2018

Profpatsch commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 19, 2018

bazel: Set a sensible strict action environment. #43530

bazel: Set a sensible strict action environment. #43530

Conversation

mboes commented Jul 14, 2018

Profpatsch commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

GrahamcOfBorg commented Jul 16, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

Profpatsch commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

GrahamcOfBorg commented Jul 18, 2018

Profpatsch commented Jul 19, 2018