Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy drv outputs between stores #4220

Closed
wants to merge 15 commits into from
Closed

Conversation

thufschmitt
Copy link
Member

Partial (cleaner) rewrite of #4174. This PR doesn't try to get substitution to work, just to be able to manually copy drv outputs closures between stores (which is already a big deal).

The exact schema of the local db and the remote caches isn't really set yet, but we're hopefully approaching it. The rational behind most of the changes is (lengthily) explained in https://hackmd.io/@Xs46ZZwNQtOhEkXnSehjeA/rkDU1BTOv

For each “known” derivation output, store:
- its output id
- its output path
- its dependencies
  (this bit isn't implemented for `LocalFSStore` yet and we recompute them
  on-the-fly instead)

This comes with a set of needed changes:

- New `drv-output-info` module declaring the types needed for describing
  these dependencies
- New `Store::registerDrvOutput` method registering all the needed informations
  about a derivation output (also replaces `LocalStore::linkDeriverToPath`)
- new `Store::queryDrvOutputInfo` method to retrieve the informations for a
  derivations

This introcudes some redundancy on the remote-store side between
`wopQueryDerivationOutputMap` and `wopQueryDrvOutputInfo`.
However we might need to keep both (regardless of backwards compat)
because we sometimes need to get some infos for all the outputs of a
derivation (where `wopQueryDerivationOutputMap` is handy), but all the
stores can't implement it − because listing all the outputs of a
derivation isn't really possible for binary caches where the server
doesn't allow to list a directory.
Copying a derivation output is equivalent to copy the output path,
except that it will also link the output to the path (with
`linkDeriverToPath` on the remote store)
Commands that accept store paths or installables as input can now take a
derivation path with outputs. Like when the input is a nix expr, the
derivation will be realised if needed.
Generalises `StorePathsCommand` to allow manipulating `Buildables`
rather than just store paths.
…d path

Fix NixOS#4138 and allows copying drv outputs without having to copy the
whole drv closure
That way we can `nix copy --from` a store that doesn't have the derivation but
just its outputs
This isn't really satisfying. It'll probably be nicer to have two separate
sql tables for the path references and the drvoutput references in the end
@thufschmitt
Copy link
Member Author

Closing in favor of #4225

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant