Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

symlinkJoin: fix "bash: Argument list too long" error (release-19.09) #80420

Conversation

basvandijk
Copy link
Member

@basvandijk basvandijk commented Feb 18, 2020

Motivation for this change

Cherry-pick of #80348 on release-19.09.

When the paths argument is too big symlinkJoin will fail with:

while setting up the build environment: executing '/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash': Argument list too long
Things done

Cherry picked from commit 4e0a6bc which passes paths as a file instead of as an environment variable.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

When the `paths` argument is too big `symlinkJoin` will fail with:

```
while setting up the build environment: executing '/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash': Argument list too long
```

This is fixed by passing `paths` as a file instead of as an
environment variable.

(cherry picked from commit 4e0a6bc)
@basvandijk basvandijk changed the title symlinkJoin: fix "bash: Argument list too long" error symlinkJoin: fix "bash: Argument list too long" error (release-19.09) Feb 18, 2020
@basvandijk
Copy link
Member Author

The NixOS boot test is using symlinkJoin so let's test it:

@GrahamcOfBorg test boot

in runCommand name args
''
mkdir -p $out
for i in $paths; do
for i in $(cat $pathsPath); do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could overflow as well if you have tons of paths.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better would be:

Suggested change
for i in $(cat $pathsPath); do
cat $pathsPath | while read i; do

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I now see that this is already in master. I guess it makes sense for now.

@vcunat
Copy link
Member

vcunat commented Aug 16, 2020

I expect 19.09 isn't worth the effort anymore.

@vcunat vcunat closed this Aug 16, 2020
@peterbecich
Copy link
Contributor

Appears similar to #97475

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants