queryDerivationOutputMap no longer assumes all outputs have a mapping
#3859
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This assumption is broken by CA derivations. Making a PR now to do the breaking daemon change as soon as possible (if it is already too late, we can bump protocol intead).
Ericson2314
commented
Jul 24, 2020
src/libstore/build.cc
Outdated
| { | ||
| if (!goal.isAllowed(path)) | ||
| throw InvalidPath("cannot query output map for unknown path '%s' in recursive Nix", printStorePath(path)); | ||
| return next->queryDerivationOutputMap(path); |
There was a problem hiding this comment.
Doesn't matter until we actually implement CA derivations, but we should censor returned output paths too.
Ericson2314
commented
Jul 24, 2020
src/libstore/worker-protocol.hh
Outdated
Comment on lines
73
to
123
| /* To guide overloading */ | ||
| template<typename T> | ||
| struct Proxy {}; | ||
|
|
||
| template<typename T> | ||
| std::map<std::string, T> read(const Store & store, Source & from, Proxy<std::map<std::string, T>> _) | ||
| { | ||
| std::map<string, T> resMap; | ||
| auto size = (size_t)readInt(from); | ||
| while (size--) { | ||
| auto thisKey = readString(from); | ||
| resMap.insert_or_assign(std::move(thisKey), read(store, from, Proxy<T> {})); | ||
| } | ||
| return resMap; | ||
| } | ||
|
|
||
| template<typename T> | ||
| void write(const Store & store, Sink & out, const std::map<string, T> & resMap) | ||
| { | ||
| out << resMap.size(); | ||
| for (auto & i : resMap) { | ||
| out << i.first; | ||
| write(store, out, i.second); | ||
| } | ||
| } | ||
|
|
||
| template<typename T> | ||
| std::optional<T> read(const Store & store, Source & from, Proxy<std::optional<T>> _) | ||
| { | ||
| auto tag = readNum<uint8_t>(from); | ||
| switch (tag) { | ||
| case 0: | ||
| return std::nullopt; | ||
| case 1: | ||
| return read(store, from, Proxy<T> {}); | ||
| default: | ||
| throw Error("got an invalid tag bit for std::optional: %#04x", tag); | ||
| } | ||
| } | ||
|
|
||
| template<typename T> | ||
| void write(const Store & store, Sink & out, const std::optional<T> & optVal) | ||
| { | ||
| out << (optVal ? 1 : 0); | ||
| if (optVal) | ||
| write(store, out, *optVal); | ||
| } | ||
|
|
||
| StorePath read(const Store & store, Source & from, Proxy<StorePath> _); | ||
|
|
||
| void write(const Store & store, Sink & out, const StorePath & storePath); |
There was a problem hiding this comment.
While the generic machinery is a bit overkill for just one problem, I would like to keep this stuff to make the daemon protocol less annoying to work with in the future.
Ericson2314
force-pushed
the
drv-outputs-map-allow-missing
branch
from
2f2ae99
to
fbeb869
Compare
Sorry, Haskell.
We had to predeclare our template functions
While I am cautious to break parametricity, I think it's OK in this cases---we're not about to try to do some crazy polymorphic protocol anytime soon.
|
OK, this is now backwards compatible, and thus is ready to be merged. |
edolstra
reviewed
Aug 20, 2020
| template<> | ||
| std::optional<StorePath> read(const Store & store, Source & from, Phantom<std::optional<StorePath>> _) | ||
| { | ||
| auto s = readString(from); |
There was a problem hiding this comment.
Suggested change
| auto s = readString(from); | |
| auto s = readString(from); |
tab -> spaces
edolstra
reviewed
Aug 20, 2020
src/libstore/store-api.hh
Outdated
| { unsupported("queryDerivationOutputMap"); } | ||
|
|
||
| /* Query the mapping outputName=>outputPath for the given derivation. | ||
| Assume every output has a mapping and throw an exception otherwise. */ | ||
| OutputPathMap queryDerivationOutputMapAssumeTotal(const StorePath & path); |
There was a problem hiding this comment.
Suggested change
| OutputPathMap queryDerivationOutputMapAssumeTotal(const StorePath & path); | |
| OutputPathMap queryDerivationOutputMap(const StorePath & path); |
edolstra
reviewed
Aug 20, 2020
src/libstore/store-api.hh
Outdated
| /* Query the mapping outputName => outputPath for the given derivation. All | ||
| outputs are mentioned so ones mising the mapping are mapped to | ||
| `std::nullopt`. */ | ||
| virtual std::map<std::string, std::optional<StorePath>> queryDerivationOutputMap(const StorePath & path) |
There was a problem hiding this comment.
Suggested change
| virtual std::map<std::string, std::optional<StorePath>> queryDerivationOutputMap(const StorePath & path) | |
| virtual std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path) |
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap` - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This assumption is broken by CA derivations. Making a PR now to do the
breaking daemon change as soon as possible (if it is already too late,
we can bump protocol instead).
CC @regnat