Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel: enableParallelBuilding now effective #31887

Closed
wants to merge 2 commits into from
Closed

Conversation

teto
Copy link
Member

@teto teto commented Nov 21, 2017

Motivation for this change

I need to build kernels faster

Things done

Improves error message.
If enableParallelBuilding, set NIX_BUILD_CORES to 0 so that the builder guesses the correct value to use.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@orivej
Copy link
Contributor

orivej commented Nov 21, 2017

Why don't you set Nix build cores to zero in your Nix config?

@teto
Copy link
Member Author

teto commented Nov 21, 2017

I haven't set it but previously it was using only -j1. Do you mean that if nix.buildCores is not set in configuration.nix, nixos assumes -j1 ?
I never paid attention before but it got really annoying with the kernel recompilations.

Seems confirmed with

if [ -z "${NIX_BUILD_CORES:-}" ]; then
  NIX_BUILD_CORES="1"
elif [ "$NIX_BUILD_CORES" -le 0 ]; then
  NIX_BUILD_CORES=$(nproc 2>/dev/null || true)
  if expr >/dev/null 2>&1 "$NIX_BUILD_CORES" : "^[0-9][0-9]*$"; then
    :
  else
    NIX_BUILD_CORES="1"
  fi
fi

I didn't check further because the pb looked similar to #25208
Let s try with buildCores ;)

@teto
Copy link
Member Author

teto commented Nov 21, 2017

I just tried with nix.buildCores set to 4 on nixos-unstable within a nix-shell and I don't know why but it dosn't work I get during the buildPhase:
build flags: -j1 -l1 SHELL=/nix/store/4ada72n7785wwazv42fhsnxjvilaa3aj-bash-4.4-p12/bin/bash O=\$\(buildRoot\) ARCH=x86_64 KBUILD_BUILD_VERSION=1-NixOS bzImage vmlinux modules KBUILD_BUILD_TIMESTAMP=jeu.\ janv.\ \ 1\ 00:00:01\ UTC\ 1970 KBUILD_BUILD_TIMESTAMP=jeu.\ janv.\ \ 1\ 00:00:01\ UTC\ 1970

EDIT: I guess it has to export NIX_BUILD_CORES else it will default to one.

@orivej
Copy link
Contributor

orivej commented Nov 21, 2017

You have to perform nixos-rebuild switch after setting nix.buildCores in the configuration.nix: Nix tools read the value from /etc/nix/nix.conf.

@teto
Copy link
Member Author

teto commented Nov 21, 2017

I had done it.

@orivej
Copy link
Contributor

orivej commented Nov 21, 2017

Please confirm that /etc/nix/nix.conf contains build-cores = 0. nix-shell '<nixpkgs>' -A linux --run 'echo $NIX_BUILD_CORES' should print the actual number of cores on the machine (without this PR). NIX_BUILD_CORES is set by Nix: https://nixos.org/nix/manual/#part-command-ref

@teto
Copy link
Member Author

teto commented Nov 21, 2017

arf seems like I had interrupted the rebuild because of a wrong channel my bad sorry. It works. I'll ditch the first commit, Is the 2nd commit of interest ?

@orivej
Copy link
Contributor

orivej commented Nov 21, 2017

I'll ditch the first commit, Is the 2nd commit of interest ?

Yes, please do.

@vcunat vcunat closed this in 28f9262 Nov 21, 2017
@vcunat
Copy link
Member

vcunat commented Nov 21, 2017

Done manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants