Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildRustPackage: add support for parallel build #95622

Merged
merged 1 commit into from Aug 19, 2020
Merged

buildRustPackage: add support for parallel build #95622

merged 1 commit into from Aug 19, 2020

Conversation

marsam
Copy link
Contributor

@marsam marsam commented Aug 16, 2020

Motivation for this change

Fixes #95595
cc: @bennofs

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@marsam marsam requested a review from andir as a code owner August 16, 2020 21:29
@marsam marsam force-pushed the buildrustpackage-parallel-build branch from 9bf0c03 to 920fc28 Compare August 16, 2020 21:32
@marsam
Copy link
Contributor Author

marsam commented Aug 16, 2020

@GrahamcOfBorg build fd ripgrep

@bennofs
Copy link
Contributor

bennofs commented Aug 16, 2020

I think we should pass --jobs 1 if enableParallelBuilding is false, otherwise cargo will default to just use as many jobs as there a cpus available.

@marsam marsam changed the base branch from master to staging August 17, 2020 01:36
@marsam marsam force-pushed the buildrustpackage-parallel-build branch from 920fc28 to b52808f Compare August 17, 2020 01:38
@jonringer
Copy link
Contributor

ping @bennofs

I think we should merge as is. If there are some issues with reproducibility or correctness, then we can introduce the enableParallelBuilding switch.

@Mic92
Copy link
Member

Mic92 commented Aug 19, 2020

It was parallel before now it is just parallel in a controlled way.

@bennofs
Copy link
Contributor

bennofs commented Aug 19, 2020

My comment was referring to the version of this PR that still had the extra enableParallelBuilding flag. I'm totally fine with the PR as it is now.

@Mic92 Mic92 merged commit 542f4e4 into NixOS:staging Aug 19, 2020
@jonringer jonringer mentioned this pull request Aug 19, 2020
Closed
@zowoq zowoq mentioned this pull request Sep 7, 2020
Merged
10 tasks
@dhess
Copy link
Contributor

dhess commented Nov 20, 2020

Does this fix actually work? Rust builds bring my iMac Pro to a grinding halt and seem to ignore max-jobs and cores.

@jonringer
Copy link
Contributor

Does this fix actually work? Rust builds bring my iMac Pro to a grinding halt and seem to ignore max-jobs and cores.

how are you invoking the build? home-manager, or nix tooling?

@dhess
Copy link
Contributor

dhess commented Nov 21, 2020 •
edited

Just a regular nix-build. It appears to be running about 2 x max-jobs x cores jobs processes.

@jonringer
Copy link
Contributor

nix-build --cores 2 -A ... should set the NIX_BUILD_CORES within a build

@marsam marsam deleted the buildrustpackage-parallel-build branch May 31, 2023 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zowoq zowoq zowoq left review comments

@Mic92 Mic92 Mic92 approved these changes

@bennofs bennofs bennofs approved these changes

@jonringer jonringer jonringer approved these changes

@andir andir Awaiting requested review from andir

Assignees
No one assigned
Labels
6.topic: rust 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 501-1000 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ 10.rebuild-linux: 1001-2500
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

buildRustPackage: honor NIX_BUILD_CORES
6 participants
@marsam @bennofs @jonringer @Mic92 @dhess @zowoq