Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixUnstable: pre20200721_ff314f1 -> pre20200829_f156513 #95742

Merged
merged 1 commit into from Aug 30, 2020
Merged

nixUnstable: pre20200721_ff314f1 -> pre20200829_f156513 #95742

merged 1 commit into from Aug 30, 2020

Conversation

kalekseev
Copy link
Contributor

@kalekseev kalekseev commented Aug 18, 2020 •
edited

Motivation for this change

Flakes are very slow in current version on mac os NixOS/nix#3874

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested a review from edolstra August 18, 2020 10:54
@Ma27
Copy link
Member

Ma27 commented Aug 18, 2020

The tests seem broken btw:

nix> ran test tests/recursive.sh... [FAIL]
nix> common.sh: line 94: unshare: command not found
nix> ++ uname
nix> + [[ Linux != Linux ]]
nix> + clearStore
nix> + echo 'clearing store...'
nix> clearing store...
nix> + chmod -R +w /build/nix-test/recursive/store
nix> + rm -rf /build/nix-test/recursive/store
nix> + mkdir /build/nix-test/recursive/store
nix> + rm -rf /build/nix-test/recursive/var/nix
nix> + mkdir /build/nix-test/recursive/var/nix
nix> + nix-store --init
nix> + clearProfiles
nix> + profiles=/build/nix-test/recursive/var/nix/profiles
nix> + rm -rf /build/nix-test/recursive/var/nix/profiles
nix> + rm -f /build/nix-test/recursive/result
nix> ++ nix add-to-store ./recursive.sh
nix> warning: you don't have Internet access; disabling some network-dependent features
nix> + export unreachable=/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh
nix> + unreachable=/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh
nix> + nix --experimental-features 'nix-command recursive-nix' build -o /build/nix-test/recursive/result -L --impure --expr '
nix> with import ./config.nix;
nix> with import <nix/config.nix>;
nix> mkDerivation {
nix> name = "recursive";
nix> dummy = builtins.toFile "dummy" "bla bla";
nix> SHELL = shell;
nix> # Note: this is a string without context.
nix> unreachable = builtins.getEnv "unreachable";
nix> requiredSystemFeatures = [ "recursive-nix" ];
nix> buildCommand = '\'''\''
nix> mkdir $out
nix> PATH=${nixBinDir}:$PATH
nix> opts="--experimental-features nix-command"
nix> # Check that we can query/build paths in our input closure.
nix> nix $opts path-info $dummy
nix> nix $opts build $dummy
nix> # Make sure we cannot query/build paths not in out input closure.
nix> [[ -e $unreachable ]]
nix> (! nix $opts path-info $unreachable)
nix> (! nix $opts build $unreachable)
nix> # Add something to the store.
nix> echo foobar > foobar
nix> foobar=$(nix $opts add-to-store ./foobar)
nix> nix $opts path-info $foobar
nix> nix $opts build $foobar
nix> # Add it to our closure.
nix> ln -s $foobar $out/foobar
nix> [[ $(nix $opts path-info --all | wc -l) -eq 3 ]]
nix> # Build a derivation.
nix> nix $opts build -L --impure --expr '\''
nix> derivation {
nix> name = "inner1";
nix> builder = builtins.getEnv "SHELL";
nix> system = builtins.getEnv "system";
nix> fnord = builtins.toFile "fnord" "fnord";
nix> args = [ "-c" "echo $fnord blaat > $out" ];
nix> }
nix> '\''
nix> [[ $(nix $opts path-info --json ./result) =~ fnord ]]
nix> ln -s $(nix $opts path-info ./result) $out/inner1
nix> '\'''\'';
nix> }
nix> '
nix> warning: you don't have Internet access; disabling some network-dependent features
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> recursive> /build/nix-test/recursive/store/bxgxg5zcfy4l291y9x943qg0p6ylrs65-dummy
nix> 2 operations
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 4 operations
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- InvalidPath --- nix
nix> recursive> path '/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh' is not valid
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- Error --- nix
nix> recursive> error: --- InvalidPath --- nix
nix> recursive> cannot build unknown path '/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh' in recursive Nix
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- Error --- nix
nix> recursive> error: --- Error --- nix
nix> recursive> invalid operation 120
nix> error: --- Error --- nix
nix> builder for '/build/nix-test/recursive/store/a3kzhq1bri96pviigkqfa4rw9n4v8sni-recursive.drv' failed with exit code 1

@kalekseev
Copy link
Contributor Author

I tested only on mac os, and that test is skipped on it, I thought that if nix passed tests on github actions it should pass on linux, will look at it when I get access to my linux box.

@Ma27
Copy link
Member

Ma27 commented Aug 18, 2020

Hmm, when I build it locally (with nix develop in my git checkout and with nix-build) it's not reproducible anymore. I assume this is because the nix-daemon on my remote builder is too old (i.e. Nix 2.3).

@edolstra not sure what we want to do about this tbh.

@kalekseev
Copy link
Contributor Author

Hmm, when I build it locally (with nix develop in my git checkout and with nix-build) it's not reproducible anymore. I assume this is because the nix-daemon on my remote builder is too old (i.e. Nix 2.3).

@edolstra not sure what we want to do about this tbh.

Tested on Ubuntu: tests pass, binaries work in all my use cases.

@Ma27
Copy link
Member

Ma27 commented Aug 19, 2020

Tested on Ubuntu: tests pass, binaries work in all my use cases.

Well I assume you already have a Nix 2.4/3.0 running as nix-daemon? The remote builder I used for my first (and only unsuccessful) attempt still runs Nix 2.3 because I prefer a stable release on my servers.

This is just a guess, but seems reasonable looking at the "invalid operation 120" which seems related to the worker-protocol which is (IIRC) also used in the nix-daemon.

I think that this can become a problem as a lot of users still use Nix 2.3 for good reasons and will run into the same issues if my theory is right. That's why I'd like to get feedback from @edolstra on this first before merging this anywhere.

@edolstra
Copy link
Member

We're looking into this here: NixOS/nix#3943

@kalekseev kalekseev changed the title nixUnstable: pre20200721_ff314f1 -> pre20200817_e849b19 nixUnstable: pre20200721_ff314f1 -> pre20200829_f156513 Aug 30, 2020
@kalekseev
Copy link
Contributor Author

Hi @Ma27, NixOS/nix#3943 was fixed, so I updated pr with latest commit, could you test it please, I tested on macOS and ubuntu with nix2.4/nix2.3.

@Ma27 Ma27 merged commit ba6ac14 into NixOS:master Aug 30, 2020
@Ma27
Copy link
Member

Ma27 commented Aug 30, 2020

@edolstra should we backport this as well?

@Ma27 Ma27 mentioned this pull request Sep 3, 2020
Merged
10 tasks
This was referenced Feb 27, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edolstra edolstra Awaiting requested review from edolstra

Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@kalekseev @Ma27 @edolstra