Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way to get all the outputs of a derivation with their label #3678

Merged
merged 1 commit into from Jul 1, 2020

Conversation

thufschmitt
Copy link
Member

Generalize queryDerivationOutputNames and queryDerivationOutputs by adding a queryDerivationOutputMap that returns the map outputName=>outputPath.

This is needed by #3528 as it needs a way to get the path of a specific derivation output, which (afaik) isn't possible with the current API

(not that this is not equivalent to merging the results of queryDerivationOutputs and queryDerivationOutputNames as sets don't preserve the order, so we would end up with an incorrect mapping).

return outputPaths;
}

StringSet Store::queryDerivationOutputNames(const StorePath & path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment saying these are obsolete / deprecated ?

@@ -61,6 +61,7 @@ struct StorePath : rust::Value<3 * sizeof(void *) + 24, ffi_StorePath_drop>

typedef std::set<StorePath> StorePathSet;
typedef std::vector<StorePath> StorePaths;
typedef std::map<string, StorePath> StorePathMap;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might call this OutputPathMap instead, as StorePathMap sounds to me like std::map<StorePath, ???>.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that makes sense. Changed in 53edff2

@@ -308,8 +308,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,

case wopQueryReferences:
case wopQueryReferrers:
case wopQueryValidDerivers:
case wopQueryDerivationOutputs: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what the policy is on supporting on protocol versions? I don't!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The daemon should support older clients. So we can't remove APIs like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops sorry, I'll restore them

@edolstra
Copy link
Member

Do we actually need these functions? The outputs of a derivation are also available by doing readDerivation(drvPath). IIRC, the only reason why we have LocalStore::queryDerivationOutputs() was to make the garbage collector more efficient (since getting this stuff from SQLite is faster than reading a lot of .drvs). queryDerivationOutputNames() doesn't look like it was ever necessary from a performance perspective...

@edolstra
Copy link
Member

As a cleanup I've removed queryDerivationOutputNames() (045b072).

@Ericson2314
Copy link
Member

I agree having 3 of them is unnecessary, but this new on seams good to me? In #3523 (comment) you talk about not wanting to rely on derivations existing on disk as much. Avoiding readDerivation with these functions would seem to me in service of that goal.

@thufschmitt
Copy link
Member Author

Do we actually need these functions? The outputs of a derivation are also available by doing readDerivation(drvPath). IIRC, the only reason why we have LocalStore::queryDerivationOutputs() was to make the garbage collector more efficient (since getting this stuff from SQLite is faster than reading a lot of .drvs). queryDerivationOutputNames() doesn't look like it was ever necessary from a performance perspective...

Atm yes, but that won't be true anymore in a CA setting where the output paths don't match what's in the drv (which is the reason why I needed this in the first place)

@thufschmitt
Copy link
Member Author

I've rebased it on top of master. No idea what the OSX failure is. It seems totally unrelated

@Ericson2314
Copy link
Member

Yes, we currently fetch the latest nixos-20.03-small, but I think that might be a bit too fast non-deterministism, as it opens the door to all these issues on Darwin deps.

Generalize `queryDerivationOutputNames` and `queryDerivationOutputs` by
adding a `queryDerivationOutputMap` that returns the map
`outputName=>outputPath`

(not that this is not equivalent to merging the results of
`queryDerivationOutputs` and `queryDerivationOutputNames` as sets don't
preserve the order, so we would end up with an incorrect mapping).

squash! Add a way to get all the outputs of a derivation with their label

Rename StorePathMap to OutputPathMap
@edolstra edolstra merged commit 86a4aba into NixOS:master Jul 1, 2020
@edolstra edolstra deleted the remote-query-outputs branch July 1, 2020 13:32
@edolstra
Copy link
Member

edolstra commented Jul 1, 2020

Thanks, merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants