Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new Cmd type working on RealisedPaths #4372

Merged
merged 5 commits into from Feb 5, 2021

Conversation

thufschmitt
Copy link
Member

Where a RealisedPath is a store path with its history, meaning either
an opaque path for stuff that has been directly added to the store, or a
Realisation for stuff that has been built by a derivation

This is a low-level refactoring that doesn't bring anything by itself
(except a few dozen extra lines of code :/ ), but raising the
abstraction level a bit is important on a number of levels:

  • Commands like nix build have to query for the realisations after the
    build is finished which is fragile (see
    27905f1 for example). Having them
    oprate directly at the realisation level would avoid that
  • Others like nix copy currently operate directly on (built) store
    paths, but need a bit more information as they will need to register
    the realisations on the remote side

@thufschmitt thufschmitt added this to the ca-derivations-mvp milestone Dec 16, 2020
@thufschmitt thufschmitt added the ca-derivations Derivations with content addressed outputs label Dec 16, 2020
/**
* A store path with all the history of how it went into the store
*/
struct RealisedPath {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is all this C++ worth it? In derivation.hh we just wrapped a variant and called it a day. Other places we typedefed the variant. It would be good to be consistent whatever route we choose, I suppose.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I eventually removed because it was a lot of boilerplate for nearly nothing

/**
* A store path with all the history of how it went into the store
*/
struct RealisedPath {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a separate note, I think it would be good to make a plan for reducing into fewer types this, StorePathWithOutputs, Buildable, and perhaps even the separation of inputsDrv vs inputSrcs. We've had too many similar data types for too long, but only recently have we been bringing them closer together.

@thufschmitt thufschmitt force-pushed the ca/drvoutputs-commands branch 2 times, most recently from 3e1d847 to 9fe4bf8 Compare January 28, 2021 08:38
Where a `RealisedPath` is a store path with its history, meaning either
an opaque path for stuff that has been directly added to the store, or a
`Realisation` for stuff that has been built by a derivation

This is a low-level refactoring that doesn't bring anything by itself
(except a few dozen extra lines of code :/ ), but raising the
abstraction level a bit is important on a number of levels:

- Commands like `nix build` have to query for the realisations after the
  build is finished which is fragile (see
  27905f1 for example). Having them
  oprate directly at the realisation level would avoid that
- Others like `nix copy` currently operate directly on (built) store
  paths, but need a bit more information as they will need to register
  the realisations on the remote side
src/libcmd/installables.cc Outdated Show resolved Hide resolved
src/libcmd/installables.cc Outdated Show resolved Hide resolved
src/libstore/realisation.cc Outdated Show resolved Hide resolved
* }
* ```
*/
#define GENERATE_ONE_CMP(COMPARATOR, MY_TYPE, FIELDS...) \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this into a separate file in libutil?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I guess that can be useful elsewhere.

Done in 3436874


/**
* Syntactic sugar to run `std::visit` on the raw value:
* path.visit(blah) == std::visit(blah, path.raw)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I wouldn't mind getting rid of the use of std::visit, which often seems unnecessarily verbose/complicated because of the use of closures compared to

if (auto realisation = variant.get_if<Realisation>)
  do something 
else if (auto realisation = variant.get_if<OpaquePath>)
  do something else

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know in the general case (it's indeed very verbose and easily hard to read, but at the same time the exhaustiveness check is a killer feature…). But in this precise case, there was only one single call to visit, so it's definitely not worth it indeed.

Fixed in 4f84b2c

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@regnat For some reason that commit isn't showing up in this PR, did you push it?

Théophane Hufschmitt and others added 4 commits February 4, 2021 14:47
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Despite being an ugly hack, it can probably be useful in a couple extra
places
In addition to being some ugly template trickery, it was also totally
useless as it was used in only one place where I could replace it by
just a few extra characters
@edolstra edolstra merged commit d7c27f2 into NixOS:master Feb 5, 2021
@edolstra edolstra deleted the ca/drvoutputs-commands branch February 5, 2021 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ca-derivations Derivations with content addressed outputs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants