Build ca derivations remotely #4477
Conversation
thufschmitt
added
ca-derivations
thufschmitt
force-pushed
the
ca/build-remote
branch
from
fe5f5f8
to
f961c07
Compare
|
Rebased on top of master to fix some conflicts |
thufschmitt
force-pushed
the
ca/build-remote
branch
from
f961c07
to
a85bbcc
Compare
src/libstore/build/entry-points.cc
Outdated
| // XXX: Should use something like | ||
| // `queryPartialDerivationOutputMap`, but we can't do that | ||
| // as it assumes a registered derivation |
There was a problem hiding this comment.
There is a DerivationGoal::queryPartialDerivationOutputMap for this purpose that you might be able to somehow use?
There was a problem hiding this comment.
Mh I might indeed. Will need to update it a bit to handle non-static output paths, but I think it's a good thing anyways. Thanks :)
There was a problem hiding this comment.
At a second though it can't, because queryPartialDerivationOutputMap only returns the output paths and not the full realisations. So we'll have to update it first.
I've updated the comment to reflect this
thufschmitt
force-pushed
the
ca/build-remote
branch
from
a85bbcc
to
e2d5e02
Compare
| @@ -1,5 +0,0 @@ | |||
| source common.sh | |||
There was a problem hiding this comment.
This shouldn't be deleted (but fixed in a separate PR)
thufschmitt
force-pushed
the
ca/build-remote
branch
2 times, most recently
from
86701db
to
c9d3276
Compare
|
Rebased on top of latest master |
- Pass it the name of the outputs rather than their output paths (as these don't exist for ca derivations) - Get the built output paths from the remote builder - Register the new received realisations
To allow it to build ca derivations remotely
To allow it to build ca derivations remotely
To allow it to build ca derivations remotely
Otherwise they don't get registered, triggering an assertion failure at some point later
It's possible that all the paths are already there, but just not associated to the current drv output
There was already some logic for that, but it didn't handle the case of content-addressed outputs, so extend it a bit for that
thufschmitt
force-pushed
the
ca/build-remote
branch
from
c9d3276
to
c32168c
Compare
src/build-remote/build-remote.cc
Outdated
| if (!store->isValidPath(store->parseStorePath(path))) missing.insert(store->parseStorePath(path)); | ||
| std::set<Realisation> missingRealisations; | ||
| StorePathSet missingPaths; | ||
| if (settings.isExperimentalFeatureEnabled("ca-derivations")) { |
There was a problem hiding this comment.
Maybe this should check whether drv is a CA derivation? Since we don't have to do the CA-related stuff (missingRealisations etc.) if it isn't.
src/libstore/build/entry-points.cc
Outdated
| outputId, | ||
| Realisation{ outputId, *staticOutput.second} | ||
| ); | ||
| if (settings.isExperimentalFeatureEnabled("ca-derivations")) { |
There was a problem hiding this comment.
Likewise this should probably be conditional on drv being a CA derivation instead.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/tweag-nix-dev-update-8/11758/1 |
Allow using the build hook to build ca derivations remotely.
Required extending the remote build protocol(s) a bit to send back the newly built outputs.