Comparing changes
Open a pull request
base repository: NixOS/nix
base: 258da08de7f0
head repository: NixOS/nix
compare: b10c9dd35817
- 18 commits
- 30 files changed
- 3 contributors
Commits on Dec 9, 2020
-
Revert "Re-query for the derivation outputs in the post-build-hook"
This reverts commit 1b1e076. Using `queryPartialDerivationOutputMap` assumes that the derivation exists locally which isn't the case for remote builders.
-
Test the post-build-hook with remote builders
Regression test for #4245
-
Store the final drv outputs in memory when building remotely
The `DerivationGoal` has a variable storing the “final” derivation output paths that is used (amongst other things) to fill the environment for the post build hook. However this variable wasn't set when the build-hook is used, causing a crash when both hooks are used together. Fix this by setting this variable (from the informations in the db) after a run of the post build hook.
-
libstore/openStore: fix stores with IPv6 addresses
In `nixStable` (2.3.7 to be precise) it's possible to connect to stores using an IPv6 address: nix ping-store --store ssh://root@2001:db8::1 This is also useful for `nixops(1)` where you could specify an IPv6 address in `deployment.targetHost`. However, this behavior is broken on `nixUnstable` and fails with the following error: $ nix store ping --store ssh://root@2001:db8::1 don't know how to open Nix store 'ssh://root@2001:db8::1' This happened because `openStore` from `libstore` uses the `parseURL` function from `libfetchers` which expects a valid URL as defined in RFC2732. However, this is unsupported by `ssh(1)`: $ nix store ping --store 'ssh://root@[2001:db8::1]' cannot connect to 'root@[2001:db8::1]' This patch now allows both ways of specifying a store (`root@2001:db8::1`) and also `root@[2001:db8::1]` since the latter one is useful to pass query parameters to the remote store. In order to achieve this, the following changes were made: * The URL regex from `url-parts.hh` now allows an IPv6 address in the form `2001:db8::1` and also `[2001:db8::1]`. * In `libstore`, a new function named `extractConnStr` ensures that a proper URL is passed to e.g. `ssh(1)`: * If a URL looks like either `[2001:db8::1]` or `root@[2001:db8::1]`, the brackets will be removed using a regex. No additional validation is done here as only strings parsed by `parseURL` are expected. * In any other case, the string will be left untouched. * The rules above only apply for `LegacySSHStore` and `SSHStore` (a.k.a `ssh://` and `ssh-ng://`). Unresolved questions: * I'm not really sure whether we want to allow both variants of IPv6 addresses in the URL parser. However it should be noted that both seem to be possible according to RFC2732: > This document incudes an update to the generic syntax for Uniform > Resource Identifiers defined in RFC 2396 [URL]. It defines a syntax > for IPv6 addresses and allows the use of "[" and "]" within a URI > explicitly for this reserved purpose. * Currently, it's not supported to specify a port number behind the hostname, however it seems as this is not really supported by the URL parser. Hence, this is probably out of scope here.
-
Merge pull request #4319 from Ma27/store-v6-addr
libstore/openStore: fix stores with IPv6 addresses
-
Merge pull request #4342 from tweag/fix-remote-build-hook
fix remote build hook
-
Use no substituers by default in the tests
Otherwise https://cache.nixos.org is chosen by default, causing the OSX testsuite to hang inside the sandbox. (In a way, this is probably rugging an actual bug under the carpet as Nix should be able to gracefully timeout in such a case, but that's beyond mac OSX-fu)
-
Merge pull request #4343 from tweag/fix-osx-ci
Use no substituers by default in the tests
Commits on Dec 10, 2020
-
This is like syslog's LOG_NOTICE: "normal, but significant, condition".
-
Commits on Dec 11, 2020
-
Fix the
nix
command with CA derivationsPrevents a crash because most `nix` subcommands assumed that derivations know their output path, which isn't the case for CA derivations
-
Merge pull request #4350 from NixOS/ca/fix-build-with-nix-command
Fix the `nix` command with CA derivations
-
Store metadata about drv outputs realisations
For each known realisation, store: - its output - its output path This comes with a set of needed changes: - New `realisations` module declaring the types needed for describing these mappings - New `Store::registerDrvOutput` method registering all the needed informations about a derivation output (also replaces `LocalStore::linkDeriverToPath`) - new `Store::queryRealisation` method to retrieve the informations for a derivations This introcudes some redundancy on the remote-store side between `wopQueryDerivationOutputMap` and `wopQueryRealisation`. However we might need to keep both (regardless of backwards compat) because we sometimes need to get some infos for all the outputs of a derivation (where `wopQueryDerivationOutputMap` is handy), but all the stores can't implement it − because listing all the outputs of a derivation isn't really possible for binary caches where the server doesn't allow to list a directory.
-
Rework the db schema for derivation outputs
Add a new table for tracking the derivation output mappings. We used to hijack the `DerivationOutputs` table for that, but (despite its name), it isn't a really good fit: - Its entries depend on the drv being a valid path, making it play badly with garbage collection and preventing us to copy a drv output without copying the whole drv closure too; - It dosen't guaranty that the output path exists; By using a different table, we can experiment with a different schema better suited for tracking the output mappings of CA derivations. (incidentally, this also fixes #4138)
-
Register the outputs of non-resolved derivations
When building a (non fully-input-addressed) derivation, we first resolve it to a `BasicDerivation` that depends directly on the output paths of its dependencies (rather than on the derivation outputs) − this is what enables early cutoff. Currently we register the realisation of the resolved derivation, so that subsequent calls to `nix-build` can directly reuse this without rebuilding the derivation. However, we don't register the unresolved derivation, meaning that we have to re-do the resolution step each time. This in turn means that we must either keep all the build inputs or keep the knowledge of their output path (without the content of the path itself). The former is very costly in disk-space (it amounts to making every build-input a runtime dependency), and the latter comes with a big set of challenges in presence of non-determinism that makes this a too heavy-handed solution for the time being (though we might switch to it eventually if we can find a way to solve its issues as it's a rather tempting design). To avoid these issues, we choose a third way which is to also register the realisation of the unresolved derivation − so we can completely sidestep the resolving phase, which makes these behave as classical input-addressed derivations.
-
Don't crash nix-build when not all outputs are realised
Change the `nix-build` logic for linking/printing the output paths to allow for some outputs to be missing. This might happen when the toplevel derivation didn't have to be built, either because all the required outputs were already there, or because they have all been substituted.
-
Test the garbage collection of CA derivations
Simple test to ensure that `nix-build && nix-collect-garbage && nix-build -j0` works as it should
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff 258da08de7f0...b10c9dd35817