Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

stdenv-darwin: now with 50% less LLVM! #87583

Merged
merged 1 commit into from May 15, 2020
Merged

stdenv-darwin: now with 50% less LLVM! #87583

merged 1 commit into from May 15, 2020

Conversation

Gaelan
Copy link
Contributor

@Gaelan Gaelan commented May 11, 2020

Motivation for this change

On Darwin, stdenv was building LLVM twice, with the only difference being that one used a stage 3 libxml2 and the other used one from stage 4. Unless I'm missing something (which is quite possible!), there's no reason to do this, and we can save a bunch of build time by just using the one LLVM.

I tested this by building stdenv and using it to build GNU hello, and also checked to make sure bootstrap-tools didn't end up in stdenv's closure.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label May 11, 2020
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020

cc @LnL7

@matthewbauer
Copy link
Member

Nice work!

@cole-h
Copy link
Member

cole-h commented May 11, 2020 •
edited

This should probably be rebased against staging.

@ofborg eval

@FRidh FRidh added this to WIP in Staging via automation May 11, 2020
@Gaelan Gaelan changed the base branch from master to staging May 11, 2020 17:23
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020

Now against staging.

LnL7
LnL7 approved these changes May 11, 2020
View reviewed changes
Copy link
Member

@LnL7 LnL7 left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal here is to build all of the stdenv tooling from scratch without any references to the tools used to bootstrap it. Secondly some of the overrides are there to reduce the tree needed to do this, by default certain packages like sqlite would be included by python/cmake while they are not actually needed for example.

Assuming libxml2 from stage 3 and any other dependencies of that first llvm already don't have references anymore, then there shouldn't be any reason to build another one. If this is the case I wonder if we could even get rid of a stage.

Could you double check this?

$ nix-store -qR $(nix-build -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/vvk6w4jdwm5m5r1f6mlvqjx4p1lpmik8-bootstrap-stage4-binutils-wrapper-
/nix/store/xkl1gvhj841qq3m0abxlg9w06dpx11hh-bootstrap-stage4-clang-wrapper-
/nix/store/1rkmy85jsvqq9bnbcz4fwm246n4h9r3a-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build -A stdenv) | grep bootstrap

Staging automation moved this from WIP to Ready May 11, 2020
@Gaelan
Copy link
Contributor Author

Gaelan commented May 11, 2020 

$ git checkout single-llvm
Already on 'single-llvm'
Your branch is up to date with 'Gaelan/single-llvm'.

$ nix-store -qR $(nix-build . -A stdenv.__bootPackages.stdenv) | grep bootstrap
/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools
/nix/store/9aa2rfiacjka8kvrg17ryyb6lhk5n62x-bootstrap-stage4-binutils-wrapper-
/nix/store/pm9jimjpvjpq0v9jsyswz5h35mx2bxi7-bootstrap-stage4-clang-wrapper-
/nix/store/rn1r46nyp4qfjhjbyysl91l0g1rv0q2l-bootstrap-stage4-stdenv-darwin

$ nix-store -qR $(nix-build . -A stdenv) | grep bootstrap

@Gaelan Gaelan mentioned this pull request May 14, 2020
Closed
@Gaelan
Copy link
Contributor Author

Gaelan commented May 15, 2020

Is anything blocking this at this point?

@matthewbauer matthewbauer merged commit fb35227 into NixOS:staging May 15, 2020
Staging automation moved this from Ready to Done May 15, 2020
@matthewbauer
Copy link
Member

Nope- merged to staging. If you could be on alert for any breakages from this that would be good though. Occasionally staging-next will get stuck because we don’t know why something broke.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LnL7 LnL7 LnL7 approved these changes

Assignees
No one assigned
Labels
6.topic: darwin Running or building packages on Darwin 6.topic: stdenv Standard environment 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-darwin-stdenv 10.rebuild-linux: 0
Projects
Staging
  
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@Gaelan @matthewbauer @cole-h @LnL7