linux bootstrap tools: Use same derivation whether cross compiling or not #26883

Ericson2314 · 2017-06-26T21:16:13Z

Motivation for this change

~~I'm hoping this doesn't constitute a mass rebuild, but I'm not really sure.~~ It doesn't.

Things done

Tested at https://hydra.nixos.org/jobset/nixpkgs/ericson2314-cross-trunk. The aarch64-linux stdenv is actually tested now!

Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

CC @Dridus

mention-bot · 2017-06-26T21:16:16Z

@Ericson2314, thanks for your PR! By analyzing the history of the files in this pull request, we identified @dezgeg, @viric and @ambrop72 to be potential reviewers.

dezgeg

nix-build ./pkgs/top-level/release-cross.nix -A bootstrapTools.aarch64 gives me:

error: syntax error, unexpected $end, expecting ')', at /home/tmtynkky/opt/nixpkgs/pkgs/stdenv/linux/make-bootstrap-tools-cross.nix:23:2

Ericson2314 · 2017-06-27T04:20:32Z

Well, that's awkward. Sorry @dezgeg, I guess I was multitasking on too many PRs today. Fixed now.

dezgeg · 2017-06-27T19:01:39Z

For some reason it's trying to build a non-native nukeReferences, which doesn't sound right:

these derivations will be built:                                                                                   
  /nix/store/nyn46sb87pwi9cw5f50pfaz83i3dygwh-perl-5.24.1-aarch64-unknown-linux-gnu.drv
  /nix/store/0xl7hvyqlikl363dgys57qxmzhaxz9gx-nuke-references-aarch64-unknown-linux-gnu.drv
  /nix/store/57pnqsddsd9984kp85jaqw0w72zz6s4d-gcc-5.4.0-aarch64-unknown-linux-gnu.drv
...

Ericson2314 · 2017-06-27T20:11:03Z

@dezgeg Ah, that is because without call-package, the derivations aren't spliced with buildDrv and crossDrv so I need to use buildPackages.foobar.

Also, I realized from the fact that glibc wasn't cached that I also had was comparing the wrong target platforms too. The stdenv targets the platform it runs on, so target == host always. My target != host check should instead be host != build checks.

Ericson2314 · 2017-06-28T04:44:00Z

Pushed again with a way to make the cross file even smaller. (!)

dezgeg · 2017-06-30T00:58:58Z

Next up seems to be:

  CCLD     isl_polyhedron_minimize
/nix/store/kaq3yynl3256pwx583nnzckyx3qz1029-aarch64-unknown-linux-gnu-binutils-2.28/bin/aarch64-unknown-linux-gnu-ld: warning: libgmp.so.10, needed by ./.libs/libisl.so, not found (try using -rpath or -rpath-link)
./.libs/libisl.so: undefined reference to `__gmpz_get_si'
...
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:1280: isl_polyhedron_detect_equalities] Error 1
make[2]: Leaving directory '/tmp/nix-build-isl-0.17.1-aarch64-unknown-linux-gnu.drv-0/isl-0.17.1'
make[1]: *** [Makefile:1536: all-recursive] Error 1
make[1]: Leaving directory '/tmp/nix-build-isl-0.17.1-aarch64-unknown-linux-gnu.drv-0/isl-0.17.1'
make: *** [Makefile:1140: all] Error 2
builder for ‘/nix/store/679nzcjahsm7f1q3l648r4wy42nyh6p6-isl-0.17.1-aarch64-unknown-linux-gnu.drv’ failed with exit code 2
error: build of ‘/nix/store/679nzcjahsm7f1q3l648r4wy42nyh6p6-isl-0.17.1-aarch64-unknown-linux-gnu.drv’ failed

Ericson2314 · 2017-06-30T11:14:23Z

@dezgeg Hmm, is that a regression of this PR though?

Ericson2314 · 2017-07-06T15:45:33Z

@dezgeg, to be clear, I'm sure it isn't. I think it would be best to merge this and then fix?

dezgeg · 2017-07-07T04:13:17Z

I'm not sure since it's impossible to verify that the end result works.

Ericson2314 · 2017-07-07T17:17:38Z

Fair enough, I'll probably let this sit until after #26805, which is a better foundation to see what's wrong with gcc.

Ericson2314 · 2018-01-01T23:55:51Z

pkgs/stdenv/linux/make-bootstrap-tools.nix

@@ -176,11 +189,14 @@ rec {
    bootstrapTools = runCommand "bootstrap-tools.tar.xz" {} "cp ${build}/on-server/bootstrap-tools.tar.xz $out";
  };

-  bootstrapTools = import ./bootstrap-tools { inherit system bootstrapFiles; };
+  bootstrapTools = import ./bootstrap-tools {
+    inherit (hostPlatform) system;


This will now be built on the platform the stdenv's host platform. For native, this makes so difference, but for cross it does mean we would have permanently enqueued derivations, unless we add builders for those platforms.

Added an assertion to prevent those jobs from appearing. Easy to undo if we want them later.

Ericson2314 · 2018-01-01T23:57:00Z

pkgs/stdenv/linux/make-bootstrap-tools.nix


  test = derivation {
    name = "test-bootstrap-tools";
-    inherit system;
+    inherit (hostPlatform) system;


Same sort of change as above #26883 (comment)

Ericson2314 · 2018-01-01T23:57:43Z

pkgs/stdenv/linux/make-bootstrap-tools.nix

            fi
        done

        nuke-refs $out/bin/*
        nuke-refs $out/lib/*
        nuke-refs $out/libexec/gcc/*/*/*
+        nuke-refs $out/lib/gcc/*/*/*


This was only recently needed by cross, for reasons unknown. Should be harmless for native?

Ericson2314 · 2018-01-02T16:24:28Z

pkgs/top-level/release-cross.nix

@@ -123,6 +123,8 @@ in
  bootstrapTools = let
    tools = import ../stdenv/linux/make-bootstrap-tools-cross.nix { system = "x86_64-linux"; };
    maintainers = [ lib.maintainers.dezgeg ];
-    mkBootstrapToolsJob = drv: hydraJob' (lib.addMetaAttrs { inherit maintainers; } drv);
+    mkBootstrapToolsJob = drv:
+      assert lib.elem drv.system (supportedSystems ++ [ "aarch64-linux" ]);


I'll eventually make supportedSystems include that, but I need to do a few thinks with release-lib first.

… not

Ericson2314 · 2018-01-02T23:31:34Z

Merging to unblock other things. It's not a mass rebuild so easy to address anything brought up after the fact.

Ericson2314 requested a review from dezgeg June 26, 2017 21:16

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from 12d45b8 to 74c3238 Compare June 26, 2017 21:37

0xABAB approved these changes Jun 26, 2017

View reviewed changes

Ericson2314 added the 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on label Jun 26, 2017

dezgeg suggested changes Jun 27, 2017

View reviewed changes

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from 74c3238 to 029cb3a Compare June 27, 2017 04:19

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from 029cb3a to be7699d Compare June 27, 2017 20:06

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from be7699d to 773b4ea Compare June 28, 2017 04:42

Ericson2314 force-pushed the linux-make-bootstrap-tools branch 2 times, most recently from 5710ccd to 27349be Compare June 29, 2017 02:53

Ericson2314 force-pushed the linux-make-bootstrap-tools branch 2 times, most recently from 561a45c to 3134a13 Compare July 5, 2017 18:21

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from 3134a13 to b6a9d9c Compare July 6, 2017 16:08

Ericson2314 modified the milestone: 17.09 Aug 16, 2017

Ericson2314 added this to After big PR in Cross compilation Aug 16, 2017

Ericson2314 modified the milestone: 17.09 Aug 20, 2017

Ericson2314 requested a review from zimbatm as a code owner January 1, 2018 23:26

GrahamcOfBorg added 10.rebuild-darwin-stdenv 10.rebuild-linux-stdenv 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ labels Jan 1, 2018

Ericson2314 changed the base branch from master to staging January 1, 2018 23:43

Ericson2314 requested review from dezgeg and vcunat and removed request for zimbatm, peti, ttuegel, edolstra and FRidh January 1, 2018 23:43

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from aadf0f1 to d9dc989 Compare January 1, 2018 23:51

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 0 and removed 10.rebuild-darwin-stdenv 10.rebuild-linux-stdenv 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ labels Jan 1, 2018

Ericson2314 commented Jan 2, 2018

View reviewed changes

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from d9dc989 to 434f96c Compare January 2, 2018 16:12

Ericson2314 commented Jan 2, 2018

View reviewed changes

linux bootstrap tools: Use same derivation whether cross compiling or…

ab651d2

… not

Ericson2314 changed the base branch from staging to master January 2, 2018 18:53

Ericson2314 force-pushed the linux-make-bootstrap-tools branch from 434f96c to ab651d2 Compare January 2, 2018 18:53

Ericson2314 merged commit 0bc1642 into NixOS:master Jan 2, 2018

Ericson2314 deleted the linux-make-bootstrap-tools branch January 3, 2018 00:37

Ericson2314 mentioned this pull request Jan 3, 2018

wiki: Bootstrapping NixOS on ARM #13290

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

linux bootstrap tools: Use same derivation whether cross compiling or not #26883

linux bootstrap tools: Use same derivation whether cross compiling or not #26883

Ericson2314 commented Jun 26, 2017 •

edited

mention-bot commented Jun 26, 2017

dezgeg left a comment

Ericson2314 commented Jun 27, 2017

dezgeg commented Jun 27, 2017

Ericson2314 commented Jun 27, 2017

Ericson2314 commented Jun 28, 2017

dezgeg commented Jun 30, 2017

Ericson2314 commented Jun 30, 2017 •

edited

Ericson2314 commented Jul 6, 2017

dezgeg commented Jul 7, 2017

Ericson2314 commented Jul 7, 2017

Ericson2314 Jan 1, 2018 •

edited

Ericson2314 Jan 2, 2018

Ericson2314 Jan 1, 2018 •

edited

Ericson2314 Jan 1, 2018

Ericson2314 Jan 2, 2018

Ericson2314 commented Jan 2, 2018

linux bootstrap tools: Use same derivation whether cross compiling or not #26883

linux bootstrap tools: Use same derivation whether cross compiling or not #26883

Conversation

Ericson2314 commented Jun 26, 2017 • edited

Motivation for this change

Things done

mention-bot commented Jun 26, 2017

dezgeg left a comment

Choose a reason for hiding this comment

Ericson2314 commented Jun 27, 2017

dezgeg commented Jun 27, 2017

Ericson2314 commented Jun 27, 2017

Ericson2314 commented Jun 28, 2017

dezgeg commented Jun 30, 2017

Ericson2314 commented Jun 30, 2017 • edited

Ericson2314 commented Jul 6, 2017

dezgeg commented Jul 7, 2017

Ericson2314 commented Jul 7, 2017

Ericson2314 Jan 1, 2018 • edited

Choose a reason for hiding this comment

Ericson2314 Jan 2, 2018

Choose a reason for hiding this comment

Ericson2314 Jan 1, 2018 • edited

Choose a reason for hiding this comment

Ericson2314 Jan 1, 2018

Choose a reason for hiding this comment

Ericson2314 Jan 2, 2018

Choose a reason for hiding this comment

Ericson2314 commented Jan 2, 2018

Ericson2314 commented Jun 26, 2017 •

edited

Ericson2314 commented Jun 30, 2017 •

edited

Ericson2314 Jan 1, 2018 •

edited

Ericson2314 Jan 1, 2018 •

edited