Cross-compiled NixOS #36187

shlevy · 2018-03-01T20:02:14Z

This includes a slew of fixes to get NixOS cross-compiling. Highlights:

Cross-building systemd
Cross-building perl modules (306d5cd)
Cross-compile safe ldd (897b7c7)
pkgs.runtimeShell (e70f61f) and fixing NixOS scripts to use it (fec5434)
The top-level switch to turn it on (3448794)

No need to cross-compile pure data...

This involved: * Installing miniperl as $dev/bin/perl * Setting miniperl to take INC from lib/perl5/{site_perl/,}cross_perl/${version} as well as lib/perl5/{site_perl/,}/${version}/${runtimeArch}, in that order. miniperl taking from runtimeArch is not really correct, but it works in some pure-perl cases (e.g. Config.pm) and can be overridden with the cross_perl variant. * Installing perl-cross's stubs into $dev/lib/perl5/cross_perl/${version} * Patching MakeMaker.pm to gracefully degrade (very slightly) if B.pm can't be loaded, which it can't in cross-compilation. * Passing the right build-time and runtime perls to Makefile.PL

Also disable it when noXlibs in NixOS.

This is like stdenv.shell{,Package}, but for the runtime system. The majority of uses of stdenv.shell probably want this instead.

Ericson2314

I did something between a read and skim, and it looks all good so far!

Ericson2314 · 2018-03-01T20:29:55Z

nixos/modules/misc/nixpkgs.nix

@@ -130,6 +130,18 @@ in
      '';
    };

+    crossSystem = mkOption {
+      type = types.nullOr types.attrs;


Don't let me forget about this when I return to #34444

Can we have a similar value for localSystem? O:).

Yes if system before is new in 18.03, it should be changed to local system instead. And the description of that config should in that case take into account crossSystem: "for which NixOS should be built" isn't generally true, should be "on which".

I'd be happy to do this in a follow-up PR though.

thoughtpolice · 2018-03-01T20:40:17Z

FWIW I just took a quick look at this as well and it's all pretty rote stuff, so I'm 👍 as well and excited to see these pieces making it in!

Ericson2314

OK reviewed all the nix stuff more thoroughly (didn't dive into every longer bash blob.)

Ericson2314 · 2018-03-01T21:03:53Z

pkgs/servers/dns/bind/default.nix

@@ -24,6 +27,10 @@ stdenv.mkDerivation rec {

  STD_CDEFINES = [ "-DDIG_SIGCHASE=1" ]; # support +sigchase

+  BUILD_CC = "cc";


This also surprises me, I think the setup hook should also do this.

https://gitlab.isc.org/isc-projects/bind9/blob/master/configure.in#L5125-5127

https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/cc-wrapper/setup-hook.sh#L145

Hmm, will try again without these a bit later

Ericson2314 · 2018-03-01T21:04:04Z

pkgs/servers/dns/bind/default.nix

@@ -13,6 +14,8 @@ stdenv.mkDerivation rec {
    sha256 = "10iwkghl5g50b7wc17bsb9wa0dh2gd57bjlk6ynixhywz6dhx1r9";
  };

+  preConfigure = "export AR=${stdenv.cc}/bin/${stdenv.cc.targetPrefix}ar";


What's this about? Is the setup hook not working?

https://gitlab.isc.org/isc-projects/bind9/blob/master/configure.in#L119

https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/bintools-wrapper/setup-hook.sh#L75

Permalink for posterity:

nixpkgs/pkgs/build-support/bintools-wrapper/setup-hook.sh

Lines 74 to 84 in 806edaa

for cmd in \

ar as ld nm objcopy objdump readelf ranlib strip strings size windres

do

if

PATH=$_PATH type -p "@targetPrefix@${cmd}" > /dev/null

then

upper_case="$(echo "$cmd" | tr "[:lower:]" "[:upper:]")"

export "${role_pre}${upper_case}=@targetPrefix@${cmd}";

export "${upper_case}${role_post}=@targetPrefix@${cmd}";

fi

done

(Sorry for the necro)

Ericson2314 · 2018-03-01T21:04:32Z

pkgs/tools/networking/dhcpcd/default.nix

@@ -25,6 +25,10 @@ stdenv.mkDerivation rec {
  # Check that the udev plugin got built.
  postInstall = stdenv.lib.optional (udev != null) "[ -e $out/lib/dhcpcd/dev/udev.so ]";

+  postFixup = ''


Add TODO for fixed patched shebangs?

shlevy · 2018-03-01T23:33:56Z

@Ericson2314 Fixed

Ericson2314 · 2018-03-02T00:00:09Z

nixos/modules/system/boot/stage-1.nix

+
+    add_needed $1
+
+    while [ ''${#left[@]} -ne 0 ]; do


I sometimes like while (( ''${#left[@]} )); do

Ericson2314

Total approval now (don't really care about my one bash nit). Thanks again for doing this! Really nice whenever I see my cross work so bearing fruit.

bgamari · 2018-03-02T01:57:37Z

Wow, great work!

bjornfor · 2018-03-02T07:47:08Z

@shlevy: Can you show the Nix expression(s) you used to build the system in the screenshot?

shlevy · 2018-03-02T10:20:31Z

@bjornfor still have quite a bit of cleanup to do before "official" announcement, but https://github.com/shlevy/nixos-riscv-bootstrap/tree/589d1b3bb10909a268ada5d3542dbc844d086a92 (note that that repo has very little risc-v specific in it now and will be made more generic)

bjornfor · 2018-03-02T17:14:58Z

Thanks!

dezgeg · 2018-03-03T20:48:33Z

nixos/doc/manual/default.nix

@@ -139,7 +139,7 @@ let

  manual-combined = runCommand "nixos-manual-combined"
    { inherit sources;
-      buildInputs = [ libxml2 libxslt ];
+      nativeBuildInputs = [ buildPackages.libxml2 buildPackages.libxslt ];


Why does this need both nativeBuildInputs and buildPackages? Isn`t

nativeBuildInputs = [ libxml2 libxslt ];

enough?

@dezgeg No, because the magic autoresolution for nativeBuildInputs essentially relies on being inside of a callPackaged function (this is an oversimplification, but I don't think it's misleading in this context. @Ericson2314 ?). Because we simply have a top-level with pkgs in this file, we don't get that benefit.

Note that we are working toward infrastructure that doesn't rely on the "splicing" hack that causes this behavior.

Ugh, I see.

Yeah that's all correct.

oxij · 2018-09-04T21:08:32Z

@shlevy `pkgs.runtimeShell` vs. `stdenv.shell`. I'm confused. Why make another one? Do you expect people to override `pkgs.runtimeShellPackage`? With what?

Ericson2314 · 2018-09-04T21:26:33Z

@oxij stdenv.shell is the shell used at build time. That need not be the same as the shell used by the newly-built NixOS, and indeed must not be for cross-compiled NixOS.

oxij · 2018-09-04T22:10:24Z

@oxij `stdenv.shell` is the shell used at *build* time. That need not be the same as the shell used by the newly-built NixOS, and indeed must not be for cross-compiled NixOS.

Why not simply `buildPackages.stdenv.shell` vs. `hostPackages.stdenv.shell` then?

Ericson2314 · 2018-09-04T22:32:56Z

stdenv is not like the other packages.

buildPackages.stdenv.cc is usually buildPackages.buildPackages.gcc
buildPackages.stdenv.shell is usually buildPackages.buildPackages.bash
stdenv.cc is usually buildPackages.cc
stdenv.shell is usually buildPackages.bash
targetPackages.stdenv.cc is usually gcc
targetPackages.stdenv.shell is usually bash

But referring the successive (rather than previous) stages as in targetPackages.stdenv.shell is rarely a good idea. The simplest reason is that many stages could be bootstrapped from the same previous stage in principle, so there ought not to be a canonical next stage.

Ericson2314 · 2018-09-04T22:34:07Z

Also the notion of a "run-time shell" is arguably more of a NixOS one and nixpkgs one. You can see the only use in pkgs is to get around a patchShebangs bug. It would be nice to remove it from pkgs/ entirely.

oxij · 2018-09-04T23:13:34Z

Ugh, wait.

You can see the only use in `pkgs` is to get around a `patchShebangs` bug.

Right, let's consider that ``` find $out -type f -print0 | xargs --null sed -i 's|${stdenv.shellPackage}|${runtimeShellPackage}|' ``` from `pkgs/tools/networking/dhcpcd/default.nix`. I do understand that `stdenv.shellPackage` is magical here, but am I correct to assume that `stdenv.shellPackage = buildPackages.bash`, but `runtimeShellPackage = hostPackages.bash` (or `targetPackages.bash`?) in the above use case? Can't we simply replace that `runtimeShellPackage` with `pkgs.bash` (which is its definition) and get rid of it? Also note that while you're right w.r.t. the only use of `runtimeShellPackage`, `grep -r runtimeShell` produces a whole lot of results. And I'm confused when one is supposed to use `${pkgs.bash}/bin/bash` and when `pkgs.runtimeShell`. Are they not aliases? `pkgs.runtimeShell` is defined as an alias to `${pkgs.bash}/bin/bash`. Is there any magic here? Why do we even need `pkgs.runtimeShellPackage` and `pkgs.runtimeShell`? Most packages refer to `bash` via `${pkgs.bash}/bin/bash`, are they wrong in doing that?

Ericson2314 · 2018-09-04T23:18:31Z

I don't see any problems with referring to bash directly. I can't imagine the indirection buys us much. (If we ever get NixOS/nix#1080, that might be another matter, but we don't have it.)

shlevy added 30 commits February 28, 2018 15:01

utillinux: Fix cross-compiling when building with systemd

085bfce

bind: Fix cross-compilation

a1e219e

libaio: Add generic cross-compilation patch

a5bb01b

boost: Fix cross-compilation

1f27013

iputils: Fix cross compile

0221112

sudo: Fix cross compilation

5815c7d

strace: Fix cross-compilation

d61a22f

nixos: kexec: Disable on non-keexecable systems.

a929301

nixos: udev: Generate hwdb with build-native udevadm.

c0e814a

fuse: Fix cross-compilation

ed5bd8a

nixos: Take glibcLocales from buildPackages.

aebb024

No need to cross-compile pure data...

nixos: stage-1: Don't try to test binaries when cross-compiling.

b28cb72

nixos: initrd-kmod-blacklist-ubuntu: Fix cross-compilation.

cc81202

makeModulesClosure: Fix cross-compilation

6a32291

nixos: keymap: Fix cross-compilation.

8e65205

perlPackages.DBI: Stub out dbd_postamble for cross-compilation.

dd6c926

perlPackages.JSON: Fix cross-compilation.

b0d6c67

nixos: sudo: Use build-time visudo for syntax check.

fe406d6

make-dbus-conf: Fix cross-compilation.

e51a76c

tzdata: Fix cross-compilation

942855c

Merge remote-tracking branch 'origin/master' into cross-nixos

4d5be58

Merge branch 'staging' into cross-nixos

f66da15

gobjectIntrospection: Enable building without X11 support.

c54730d

Also disable it when noXlibs in NixOS.

groff: Fix cross-compilation

bcb1bb5

nixos: manual: Fix cross-compilation.

20072d7

nixos: Fix initrd dependency detection when cross-compiling.

897b7c7

stage-2-init: Use the host bash as SHELL

948e290

Add top-level aliases for the runtime shell and its package.

e70f61f

This is like stdenv.shell{,Package}, but for the runtime system. The majority of uses of stdenv.shell probably want this instead.

nixos: Move uses of stdenv.shell to runtimeShell.

fec5434

GrahamcOfBorg added 8.has: documentation 8.has: module (update) 10.rebuild-darwin-stdenv 8.has: package (new) 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ labels Mar 1, 2018

Ericson2314 approved these changes Mar 1, 2018

View reviewed changes

Ericson2314 reviewed Mar 1, 2018

View reviewed changes

shlevy added 2 commits March 1, 2018 16:11

dhcpcd: Add TODO to cleanup after patchShebangs fix

600b08c

bind: Remove unnecessary environment defines.

60c8c02

Ericson2314 reviewed Mar 2, 2018

View reviewed changes

Ericson2314 approved these changes Mar 2, 2018

View reviewed changes

shlevy merged commit 60c8c02 into NixOS:staging Mar 2, 2018

shlevy deleted the cross-nixos branch March 2, 2018 00:31

dezgeg reviewed Mar 3, 2018

View reviewed changes

malte-v mentioned this pull request Oct 25, 2020

NixOS on RISC-V #101651

Closed

		@@ -24,6 +27,10 @@ stdenv.mkDerivation rec {

		STD_CDEFINES = [ "-DDIG_SIGCHASE=1" ]; # support +sigchase

		BUILD_CC = "cc";

	for cmd in \
	ar as ld nm objcopy objdump readelf ranlib strip strings size windres
	do
	if
	PATH=$_PATH type -p "@targetPrefix@${cmd}" > /dev/null
	then
	upper_case="$(echo "$cmd" \| tr "[:lower:]" "[:upper:]")"
	export "${role_pre}${upper_case}=@targetPrefix@${cmd}";
	export "${upper_case}${role_post}=@targetPrefix@${cmd}";
	fi
	done

Cross-compiled NixOS #36187

Cross-compiled NixOS #36187

Conversation

shlevy commented Mar 1, 2018 • edited

Ericson2314 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ericson2314 Mar 1, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thoughtpolice commented Mar 1, 2018

Ericson2314 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shlevy commented Mar 1, 2018

Choose a reason for hiding this comment

Ericson2314 left a comment

Choose a reason for hiding this comment

bgamari commented Mar 2, 2018

bjornfor commented Mar 2, 2018

shlevy commented Mar 2, 2018

bjornfor commented Mar 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oxij commented Sep 4, 2018 via email

Ericson2314 commented Sep 4, 2018

oxij commented Sep 4, 2018 via email

Ericson2314 commented Sep 4, 2018

Ericson2314 commented Sep 4, 2018

oxij commented Sep 4, 2018 via email

Ericson2314 commented Sep 4, 2018 • edited

shlevy commented Mar 1, 2018 •

edited

Ericson2314 Mar 1, 2018 •

edited

Ericson2314 commented Sep 4, 2018 •

edited