Add gzip and gnutar to default bazel-bash tools. #54459

robinp · 2019-01-22T10:19:11Z

Motivation for this change

These are often used by rules, mostly due to Bazel's one-output restriction, where multi-output results are tarred up or archived in other way. Also, these are harmless tools to include.

Continuation of #50765.

Things done

CC @Profpatsch

ghost · 2019-01-22T10:22:56Z

LGTM

robinp · 2019-01-22T10:23:20Z

I tried to exec nix-shell -p nix-review --command "nix-review pr 54459" but apparently you need NixOS for it, or at least root-builder support which I don't have yet.

ghost · 2019-01-22T10:28:09Z

I'm on NixOS, I'll give it a try. Any commands I should execute in the build to test it out?

robinp · 2019-01-22T12:28:28Z

@ivegotasthma Good you ask, I added a test, also cleaned up the other test a bit. Could you take a look on the changes, as well try the local test command mentioned? I don't expect it to behave differently, but who knows.

By the way, why are these tests in passthru? Are they executed by some integration tool?

ghost · 2019-01-22T12:44:53Z

@robinp I'll tell you when I've tested it. I'm still building bazel.

ghost · 2019-01-22T13:36:20Z

When I was building bazel I got:

[asthma@dev:~/work/nixpkgs]$ nix-review pr --token "tokenhere" 54459
$ git fetch --force https://github.com/NixOS/nixpkgs master:refs/nix-review/0 pull/54459/head:refs/nix-review/1
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 33 (delta 22), reused 27 (delta 22), pack-reused 4
Unpacking objects: 100% (33/33), done.
From https://github.com/NixOS/nixpkgs
   d202e8ca527..4b6020faf58  master               -> refs/nix-review/0
   9059c3e78bf..a58ef9b132a  refs/pull/54459/head -> refs/nix-review/1
$ git worktree add /home/asthma/.cache/nix-review/pr-54459/nixpkgs 4b6020faf58c88a6a1e610280b54f8ac79b1c165
Preparing worktree (detached HEAD 4b6020faf58)
Checking out files: 100% (18059/18059), done.
HEAD is now at 4b6020faf58 tdesktop: 1.5.4 -> 1.5.8
$ git merge --no-commit a58ef9b132ac17d18c993b4742ee8beed075940a
Automatic merge went well; stopped before committing as requested
$ nix build --no-link --keep-going --max-jobs 4 --option build-use-sandbox true -f /home/asthma/.cache/nix-review/pr-54459/build.nix
builder for '/nix/store/51nvccjq2vzsq334x5sgpfli8v6npixd-bazel-deps-2018-11-01.drv' failed with exit code 1; last 10 log lines:
  + git clone git://github.com/bazelbuild/rules_scala /build/output/external/io_bazel_rules_scala
  Cloning into '/build/output/external/io_bazel_rules_scala'...
  fatal: unable to look up github.com (port 9418) (Name or service not known)
  + git clone git://github.com/bazelbuild/rules_scala /build/output/external/io_bazel_rules_scala
  Cloning into '/build/output/external/io_bazel_rules_scala'...
  fatal: unable to look up github.com (port 9418) (Name or service not known)
  INFO: Elapsed time: 66.512s
  INFO: 0 processes.
  FAILED: Build did NOT complete successfully (0 packages loaded)
  FAILED: Build did NOT complete successfully (0 packages loaded)
cannot build derivation '/nix/store/qwnd2l30sjpblxw223ih8c994j1z2k8x-env.drv': 1 dependencies couldn't be built
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/qwnd2l30sjpblxw223ih8c994j1z2k8x-env.drv' failed
https://github.com/NixOS/nixpkgs/pull/54459
1 package failed to build:
bazel-deps

2 package were build:
bazel bazel-watcher

$ nix-shell /home/asthma/.cache/nix-review/pr-54459/shell.nix

[nix-shell:~/.cache/nix-review/pr-54459]$

It happened multiple times, but it could be a local issue.

I ran the tests with nix-build . -A bazel.tests in the nixpkgs root.

[nix-shell:~/.cache/nix-review/pr-54459/nixpkgs]$ nix-build . -A bazel.tests
these derivations will be built:
  /nix/store/4c3hnkx48ch9bnwdpsbl08k1v50ckjm0-bin.py.drv
  /nix/store/4k37xwsfxylk2av8a4h0pxzdp5j9zlcg-lib.py.drv
  /nix/store/kyr20jcrsnx0y79fk8ckinfxmv3ba7vp-WORKSPACE.drv
  /nix/store/w8gzfsdcs1aq3gclg7qnrkimx7rxrhwj-BUILD.drv
  /nix/store/0lrbfgf2wsyipfazxp422yddhcmlfvpl-our_workspace.drv
  /nix/store/2pa5m6qzcz98n6hm1xjpz19f6cski5j3-bazel-test-builtin-rules.drv
  /nix/store/5ps4qgxqi849dqpq1xshj1a5pfnm9x85-BUILD.drv
  /nix/store/q03gd2c7kzyh3g0l6afh837nycx556ga-input.txt.drv
  /nix/store/fm3bnhv729x789v0dck94p3jbfhp6d7q-our_workspace.drv
  /nix/store/i2qw2ysax5px5rmwqq0k7hh0fxci88gx-bazel-test-bash-tools.drv
building '/nix/store/5ps4qgxqi849dqpq1xshj1a5pfnm9x85-BUILD.drv'...
building '/nix/store/w8gzfsdcs1aq3gclg7qnrkimx7rxrhwj-BUILD.drv'...
building '/nix/store/kyr20jcrsnx0y79fk8ckinfxmv3ba7vp-WORKSPACE.drv'...
building '/nix/store/4c3hnkx48ch9bnwdpsbl08k1v50ckjm0-bin.py.drv'...
building '/nix/store/q03gd2c7kzyh3g0l6afh837nycx556ga-input.txt.drv'...
building '/nix/store/4k37xwsfxylk2av8a4h0pxzdp5j9zlcg-lib.py.drv'...
building '/nix/store/fm3bnhv729x789v0dck94p3jbfhp6d7q-our_workspace.drv'...
building '/nix/store/0lrbfgf2wsyipfazxp422yddhcmlfvpl-our_workspace.drv'...
building '/nix/store/i2qw2ysax5px5rmwqq0k7hh0fxci88gx-bazel-test-bash-tools.drv'...
building '/nix/store/2pa5m6qzcz98n6hm1xjpz19f6cski5j3-bazel-test-builtin-rules.drv'...
Extracting Bazel installation...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: d39b132f-5045-4e5c-9136-4fc60ac42bb9
Loading:
Loading: 0 packages loaded
Loading: 1 packages loaded
Analyzing: target //:tool_usage (1 packages loaded, 0 targets configured)
INFO: Analysed target //:tool_usage (3 packages loaded, 18 targets configured).
INFO: Found 1 target...
[0 / 2] [-----] BazelWorkspaceStatusAction stable-status.txt ... (2 actions, 0 running)
[0 / 2] [-----] BazelWorkspaceStatusAction stable-status.txt ... (2 actions, 0 running)
Target //:tool_usage up-to-date:
  bazel-genfiles/output.txt
INFO: Elapsed time: 94.437s, Critical Path: 0.32s
INFO: 1 process: 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
Testing content
OK
INFO: Invocation ID: 4307dc25-a5b7-4899-86fe-ec7fd429942d
Loading:
Loading: 0 packages loaded
Analyzing: 2 targets (1 packages loaded, 0 targets configured)
Analyzing: 2 targets (7 packages loaded, 34 targets configured)
Analyzing: 2 targets (7 packages loaded, 34 targets configured)
INFO: Analysed 2 targets (19 packages loaded, 157 targets configured).
Building: no action
INFO: Found 1 target and 1 test target...
[0 / 4] [-----] Creating source manifest for //python:bin
INFO: Elapsed time: 100.061s, Critical Path: 0.55s
INFO: 1 process: 1 linux-sandbox.
INFO: Build completed successfully, 5 total actions
//python:bin                                                             PASSED in 0.5s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
INFO: Build completed successfully, 5 total actions
/nix/store/l0rf02b5xqhinp5yqjlrq77j6b90djgc-bazel-test-bash-tools
/nix/store/zafinhza0z8x7iw7bzw6rl67kyhmajy3-bazel-test-builtin-rules

[nix-shell:~/.cache/nix-review/pr-54459/nixpkgs]$ exit

Tests pass fine, and it LGTM.

By the way, why are these tests in passthru? Are they executed by some integration tool?

I'm not sure why they're there or how they're executed. Lets ask the people who wrote them. @NinjaTrappeur @Profpatsch

picnoir · 2019-01-22T13:51:17Z

By the way, why are these tests in passthru? Are they executed by some integration tool?

I'm not sure why they're there or how they're executed. Lets ask the people who wrote them.

See #44439 .

To the best of my understanding (Profpatsch wrote the test), they are evaluated by ofborg ( #53988 (comment) ).

robinp · 2019-01-22T14:10:45Z

@ivegotasthma thanks, the issue

+ git clone git://github.com/bazelbuild/rules_scala /build/output/external/io_bazel_rules_scala
  Cloning into '/build/output/external/io_bazel_rules_scala'...
  fatal: unable to look up github.com (port 9418) (Name or service not known)

seems unrelated.

@NinjaTrappeur thanks, will try that invocation!

@GrahamcOfBorg build bazel.tests

robinp · 2019-01-22T14:26:51Z

@GrahamcOfBorg build bazel.tests

… by rules, mostly due to Bazel's one-output rule.

robinp · 2019-01-23T18:55:58Z

@Profpatsch gentle ping.

Profpatsch · 2019-01-26T20:54:22Z

To the best of my understanding (Profpatsch wrote the test), they are evaluated by ofborg ( #53988 (comment) ).

No, not yet. The plan is to add some support to ofborg. But I haven’t had the time yet.

@GrahamcOfBorg build bazel.tests

^ @robinp I think you need to be a member for that.

Profpatsch · 2019-01-26T21:22:30Z

Looking good, thanks!

robinp requested a review from Profpatsch as a code owner January 22, 2019 10:19

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jan 22, 2019

robinp added 2 commits January 23, 2019 11:30

Add gzip and gnutar to default bazel-bash tools. These are often used…

85e93b9

… by rules, mostly due to Bazel's one-output rule.

Add test and docs.

41a3ef6

robinp force-pushed the bazeltools branch from a58ef9b to 41a3ef6 Compare January 23, 2019 10:36

Profpatsch merged commit 566539f into NixOS:master Jan 26, 2019

robinp deleted the bazeltools branch January 26, 2019 22:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add gzip and gnutar to default bazel-bash tools. #54459

Add gzip and gnutar to default bazel-bash tools. #54459

robinp commented Jan 22, 2019

ghost commented Jan 22, 2019

robinp commented Jan 22, 2019

ghost commented Jan 22, 2019

robinp commented Jan 22, 2019

ghost commented Jan 22, 2019

ghost commented Jan 22, 2019 •

edited by ghost

picnoir commented Jan 22, 2019

robinp commented Jan 22, 2019

robinp commented Jan 22, 2019

robinp commented Jan 23, 2019

Profpatsch commented Jan 26, 2019 •

edited

Profpatsch commented Jan 26, 2019

Add gzip and gnutar to default bazel-bash tools. #54459

Add gzip and gnutar to default bazel-bash tools. #54459

Conversation

robinp commented Jan 22, 2019

Motivation for this change

Things done

ghost commented Jan 22, 2019

robinp commented Jan 22, 2019

ghost commented Jan 22, 2019

robinp commented Jan 22, 2019

ghost commented Jan 22, 2019

ghost commented Jan 22, 2019 • edited by ghost

picnoir commented Jan 22, 2019

robinp commented Jan 22, 2019

robinp commented Jan 22, 2019

robinp commented Jan 23, 2019

Profpatsch commented Jan 26, 2019 • edited

Profpatsch commented Jan 26, 2019

ghost commented Jan 22, 2019 •

edited by ghost

Profpatsch commented Jan 26, 2019 •

edited