Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nix
base: 95787d9e5caf^
Choose a base ref
...
head repository: NixOS/nix
compare: 98139878335f
Choose a head ref
  • 13 commits
  • 22 files changed
  • 1 contributor

Commits on Dec 15, 2020

  1. Use the fs accessor for readInvalidDerivation

    Extend `FSAccessor::readFile` to allow not checking that the path is a
    valid one, and rewrite `readInvalidDerivation` using this extended
    `readFile`.
    
    Several places in the code use `readInvalidDerivation`, either because
    they need to read a derivation that has been written in the store but
    not registered yet, or more generally to prevent a deadlock because
    `readDerivation` tries to lock the state, so can't be called from a
    place where the lock is already held.
    However, `readInvalidDerivation` implicitely assumes that the store is a
    `LocalFSStore`, which isn't always the case.
    
    The concrete motivation for this is that it's required for `nix copy
    --from someBinaryCache` to work, which is tremendously useful for the
    tests.
    thufschmitt committed Dec 15, 2020
    Copy the full SHA
    7080321 View commit details
    Browse the repository at this point in the history
  2. Register the outputs of non-resolved derivations

    When building a (non fully-input-addressed) derivation, we first resolve it to
    a `BasicDerivation` that depends directly on the output paths of its
    dependencies (rather than on the derivation outputs) − this is what enables
    early cutoff.
    
    Currently we register the realisation of the resolved derivation, so that
    subsequent calls to `nix-build` can directly reuse this without rebuilding the
    derivation.  However, we don't register the unresolved derivation, meaning that
    we have to re-do the resolution step each time. This in turn means that we must
    either keep all the build inputs or keep the knowledge of their output path
    (without the content of the path itself). The former is very costly in
    disk-space (it amounts to making every build-input a runtime dependency), and
    the latter comes with a big set of challenges in presence of non-determinism
    that makes this a too heavy-handed solution for the time being (though we might
    switch to it eventually if we can find a way to solve its issues as it's a
    rather tempting design).
    
    To avoid these issues, we choose a third way which is to also register the
    realisation of the unresolved derivation − so we can completely sidestep the
    resolving phase, which makes these behave as classical input-addressed
    derivations.
    thufschmitt committed Dec 15, 2020
    Copy the full SHA
    95787d9 View commit details
    Browse the repository at this point in the history
  3. Don't crash nix-build when not all outputs are realised

    Change the `nix-build` logic for linking/printing the output paths to allow for
    some outputs to be missing. This might happen when the toplevel
    derivation didn't have to be built, either because all the required
    outputs were already there, or because they have all been substituted.
    thufschmitt committed Dec 15, 2020
    Copy the full SHA
    71ae102 View commit details
    Browse the repository at this point in the history
  4. Test the garbage collection of CA derivations

    Simple test to ensure that `nix-build && nix-collect-garbage &&
    nix-build -j0` works as it should
    thufschmitt committed Dec 15, 2020
    Copy the full SHA
    3510d7c View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2020

  1. Better detect when buildPaths would be a no-op

    `buildPaths` can be called even for stores where it's not defined in case it's
    bound to be a no-op.
    The “no-op detection” mechanism was only detecting the case wher `buildPaths`
    was called on a set of (non-drv) paths that were already present on the store.
    
    This commit extends this mechanism to also detect the case where `buildPaths`
    is called on a set of derivation outputs which are already built on the store.
    
    This only works with the ca-derivations flag. It could be possible to
    extend this to also work without it, but it would add quite a bit of
    complexity, and it's not used without it anyways.
    thufschmitt committed Dec 16, 2020
    Copy the full SHA
    6e89927 View commit details
    Browse the repository at this point in the history
  2. Add a new Cmd type working on RealisedPaths

    Where a `RealisedPath` is a store path with its history, meaning either
    an opaque path for stuff that has been directly added to the store, or a
    `Realisation` for stuff that has been built by a derivation
    thufschmitt committed Dec 16, 2020
    Copy the full SHA
    f2da8c7 View commit details
    Browse the repository at this point in the history
  3. Use RealisedPaths in copyPaths

    That way we can copy the realisations too (in addition to the store
    paths themselves)
    thufschmitt committed Dec 16, 2020
    Copy the full SHA
    2ba6f74 View commit details
    Browse the repository at this point in the history
  4. Fix BinaryCacheStore::registerDrvOutput

    Was crashing because coercing a json document into a string is only
    valid if the json is a string, otherwise we need to call `.dump()`
    thufschmitt committed Dec 16, 2020
    Copy the full SHA
    962b82e View commit details
    Browse the repository at this point in the history
  5. Copy the full SHA
    d9226f5 View commit details
    Browse the repository at this point in the history
  6. Copy the full SHA
    ae2cf3b View commit details
    Browse the repository at this point in the history
  7. Copy the full SHA
    a634b3d View commit details
    Browse the repository at this point in the history
  8. Copy the full SHA
    9e974e3 View commit details
    Browse the repository at this point in the history
  9. Copy the full SHA
    9813987 View commit details
    Browse the repository at this point in the history