Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

top-level: Duplicate overlaying unless stdenvOverrides comes last #34332

Merged
merged 1 commit into from Jan 31, 2018

Conversation

twhitehead
Copy link
Contributor

The stdenvOverrides overlay is used to bring packages forward during bootstrapping via stdenv.overrides. These packages have already had the overlays applied to them in the previous boostrapping stage. If stdenvOverrides is not last in the overlays stack, all remaining overlays will windup being applied again to these packages.

closes #34086

Motivation for this change

See #34086 and above description.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

The stdenvOverrides overlay is used to bring packages forward during
bootstrapping via stdenv.overrides.  These packages have already had
the overlays applied to them in the previous boostrapping stage.  If
stdenvOverrides is not last in the overlays stack, all remaining
overlays will windup being applied again to these packages.

closes NixOS#34086
Copy link
Member

@Ericson2314 Ericson2314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great sluething! I'll wait for @nbp's final approval but looks good to me.

Copy link
Member

@nbp nbp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#34086 (comment):

Possibly stdenvOverrides should always be at the end of overlays list in stage.nix. If it is only used to bring packages forward, this could be valid as previous packages will already have the overlays applied to them. I don't know if this is correct, but I'll submit a pull request.

If this work, this means that stdenvOverrides do not take into account the previous overlays applied in the super stages, which in it-self should probably be considered as a bug.

Also, do we want to even apply overlays to stdenv packages? From my understanding, doing so implies that we would be recompiling everything.

I will approve this commit as this work-around an issue, but be aware that this should not be the definitive solution.

The best solution would be to move stdenvOverrides into all-packages.nix The fact that we override the previous stages should also move to Nixpkgs, by making stages as a form of nesting.

{
  stdenvLinux = {
    prevStage = {
      prevStage = {
      };
    };
  }
}

@twhitehead
Copy link
Contributor Author

twhitehead commented Jan 31, 2018

Not sure I fully understand the "move stdenvOverrides into all-packages.nix" comment. A couple of comments that may be relevant though

  1. The stdenv override varies by boot stage so at least what is overridden needs to be declared in the stdenv stage files (i.e., I don't think it can be entirely moved into all-packages.nix).
  2. There is an existing mechanism to only apply an override to the last stage. This is done via the allowCustoOverrides attribute. Currently only the configOverrides uses this.

I would also note that we would appreciate not having the overlay functionality too artificially limited as we are looking to use it to overlay (potentially quite deeply) HPC packages and tweaks onto the standard nix package set

@Ericson2314 Ericson2314 merged commit fefa9ef into NixOS:master Jan 31, 2018
@Ericson2314
Copy link
Member

Merged as a good stop gap. But let's continue discussing how to make things better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

overrideAttrs and overlays don't play well together (overrides get invoked twice)
4 participants