stdenv: update ARM bootstrap tarballs #82051

grahamc · 2020-03-08T12:52:19Z

Motivation for this change

follow-up to #79793

Things done

grahamc · 2020-03-08T13:27:40Z

ehh....

hash mismatch in fixed-output derivation '/nix/store/crk7rqpclwd6c282lkzjpakk76iv8p7y-7a7a0266d4687eb1b44496d4d185b9ffa9b1c6260c5df1c40a4999c612312bbd':
  wanted: sha256:1g9b649cd6a91b2g2p8c4v3b3agzp62x3m4n8jsb2zk8sik04yks
  got:    sha256:18qc6w2yykh7nvhjklsqb2zb3fjh4p9r22nvmgj32jr1mjflcsjn

thefloweringash · 2020-03-08T13:38:40Z

The executable flag in fetchurl changes the hash algorithm to recursive to include the executable metadata. see fetchurl/default.nix.

thefloweringash · 2020-03-08T13:40:53Z

pkgs/stdenv/linux/bootstrap-files/armv5tel.nix

-    url = https://hydra.nixos.org/build/112609163/download/2/busybox;
-    sha256 = "0dc5471dc6a5f69ad98eb7445f51a61e88aa5792d7a677025bf012bdb513b763";
+    # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv5tel.dist/latest
+    # from build: https://hydra.nixos.org/build/113692702


The build url in the comment is the same for all three architectures, and points to an armv7l build.

hah... uh .... :)

thefloweringash · 2020-03-08T14:00:35Z

The name of the busybox derivation appears to be significant. If I run this after fixing the hash, I get

$ nix-build . -A hello --argstr system armv7l-linux
[ ... ]
building '/nix/store/cff954wzdw9r66kwi3gipihmw8rb1ffs-bootstrap-tools.drv' on 'ssh://alex@alex7'...
crk7rqpclwd6c282lkzjpakk76iv8p7y-7a7a0266d4687eb1b44496d4d185b9ffa9b1c6260c5df1c40a4999c612312bbd: applet not found
builder for '/nix/store/cff954wzdw9r66kwi3gipihmw8rb1ffs-bootstrap-tools.drv' failed with exit code 127

Adding name = "busybox" is enough to make it start building.

thefloweringash · 2020-03-08T14:12:20Z

My suggested changes:

diff --git a/pkgs/stdenv/linux/bootstrap-files/armv5tel.nix b/pkgs/stdenv/linux/bootstrap-files/armv5tel.nix
index e8da070c397..18a94fb1b40 100644
--- a/pkgs/stdenv/linux/bootstrap-files/armv5tel.nix
+++ b/pkgs/stdenv/linux/bootstrap-files/armv5tel.nix
@@ -3,14 +3,15 @@
   # infrastructure team to mirror the job.
   busybox = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv5tel.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081916
+    name = "busybox";
     url = https://tarballs.nixos.org/sha256/3091d8e22778d023d51c50526ac4911b617db38b7f092fa3332c1d3ff5bd53b7;
-    sha256 = "3091d8e22778d023d51c50526ac4911b617db38b7f092fa3332c1d3ff5bd53b7";
+    sha256 = "0qxp2fsvs4phbc17g9npj9bsm20ylr8myi5pivcrmxm5qqflgi8d";
     executable = true;
   };
   bootstrapTools = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv5tel.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081916
     url = https://tarballs.nixos.org/sha256/16c307ce6d22166f2da506744caf90fcb529e3cbd003da57bcd8c1c3812ebfd7;
     sha256 = "16c307ce6d22166f2da506744caf90fcb529e3cbd003da57bcd8c1c3812ebfd7";
   };
diff --git a/pkgs/stdenv/linux/bootstrap-files/armv6l.nix b/pkgs/stdenv/linux/bootstrap-files/armv6l.nix
index 8541091ded1..eac1fa32b90 100644
--- a/pkgs/stdenv/linux/bootstrap-files/armv6l.nix
+++ b/pkgs/stdenv/linux/bootstrap-files/armv6l.nix
@@ -3,14 +3,15 @@
   # infrastructure team to mirror the job.
   busybox = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv6l.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081910
+    name = "busybox";
     url = https://tarballs.nixos.org/sha256/d96c7656bf43e9ab24949a0b7f6c7c4a4daf1f89ce10ea6e86aa645cbff65bce;
-    sha256 = "d96c7656bf43e9ab24949a0b7f6c7c4a4daf1f89ce10ea6e86aa645cbff65bce";
+    sha256 = "1q02537cq56wlaxbz3s3kj5vmh6jbm27jhvga6b4m4jycz5sxxp6";
     executable = true;
   };
   bootstrapTools = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv6l.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081910
     url = https://tarballs.nixos.org/sha256/787ce488369504c29c57e36dd25eaf84e8cc8423d9f3da7359daa0023aaafb99;
     sha256 = "787ce488369504c29c57e36dd25eaf84e8cc8423d9f3da7359daa0023aaafb99";
   };
diff --git a/pkgs/stdenv/linux/bootstrap-files/armv7l.nix b/pkgs/stdenv/linux/bootstrap-files/armv7l.nix
index c788b1e436c..be69dd4ca69 100644
--- a/pkgs/stdenv/linux/bootstrap-files/armv7l.nix
+++ b/pkgs/stdenv/linux/bootstrap-files/armv7l.nix
@@ -3,14 +3,15 @@
   # infrastructure team to mirror the job.
   busybox = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv7l.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081942
+    name = "busybox";
     url = https://tarballs.nixos.org/sha256/7a7a0266d4687eb1b44496d4d185b9ffa9b1c6260c5df1c40a4999c612312bbd;
-    sha256 = "7a7a0266d4687eb1b44496d4d185b9ffa9b1c6260c5df1c40a4999c612312bbd";
+    sha256 = "18qc6w2yykh7nvhjklsqb2zb3fjh4p9r22nvmgj32jr1mjflcsjn";
     executable = true;
   };
   bootstrapTools = import <nix/fetchurl.nix> {
     # from job: https://hydra.nixos.org/job/nixpkgs/cross-trunk/bootstrapTools.armv7l.dist/latest
-    # from build: https://hydra.nixos.org/build/113692702
+    # from build: https://hydra.nixos.org/build/114081942
     url = https://tarballs.nixos.org/sha256/c727c21ebb863f6e8e88cfef73f4ddb534ae942df1a0f9d2e1d1825b6f843a86;
     sha256 = "c727c21ebb863f6e8e88cfef73f4ddb534ae942df1a0f9d2e1d1825b6f843a86";
   };

lopsided98 · 2020-03-08T16:41:53Z

I applied @thefloweringash's changes and rebased on staging (to get the unpack-boostrap-tools fix), and I am building stdenv for armv7l and armv6l to make sure this still works.

thefloweringash · 2020-03-08T16:45:25Z

I've successfully built up to hello for armv7l. I don't think you need to rebase, since 05fc2a3 is an ancestor of this commit.

lopsided98 · 2020-03-08T16:47:29Z

Oh I see; I assumed staging hadn't been merged yet. I'll build it for armv6l then.

thefloweringash · 2020-03-08T18:54:01Z

I've verified the tarballs contents match their hash and correspond to the named builds. ✅

Without name = "busybox" I'm getting the "applet not found" error.

vcunat · 2020-03-08T19:25:05Z

Nitpick: is the URL style intentionally different from the other stdenv-bootstrap tarballs? Those have e.g.

http://tarballs.nixos.org/stdenv-linux/x86_64/c5aabb0d603e2c1ea05f5a93b3be82437f5ebf31/bootstrap-tools.tar.xz;

(where the hash is for nixpkgs git commit, not for contents)

grahamc · 2020-03-08T19:57:01Z

Not different on purpose, but I'm feeling too tired today to follow the (new-to-me) directions properly. Can you give me specific instructions? We need to do that before we merge this.

vcunat · 2020-03-08T20:23:18Z

I have no idea about reasons for the style... so your hashedMirrors style might even be preferable AFAIK. (Assuming it's also never garbage-collected, right?) When I was (involved in) updating the tarballs because of the last glibc bump, I just discovered how the URL was composed and kept to the style.

If we were to follow the style currently used for x86_64, I believe the URLs would be:

http://tarballs.nixos.org/stdenv-linux/armv5tel/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/busybox
http://tarballs.nixos.org/stdenv-linux/armv5tel/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/bootstrap-tools.tar.xz
http://tarballs.nixos.org/stdenv-linux/armv6el/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/busybox
http://tarballs.nixos.org/stdenv-linux/armv6el/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/bootstrap-tools.tar.xz
http://tarballs.nixos.org/stdenv-linux/armv7el/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/busybox
http://tarballs.nixos.org/stdenv-linux/armv7el/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778/bootstrap-tools.tar.xz

I'm not sure why we typically use http://tarballs.nixos.org. I'd expect https:// to be preferable nowadays, but I don't feel strongly about this case.

lopsided98 · 2020-03-08T21:59:56Z

hello built successfully for armv6l.

vcunat · 2020-03-10T12:42:11Z

I see there's now a Hydra machine and a jobset for armv7l (!) That verifies bootstrap in there, but apparently much was failed due to out-of-memory killer.

misuzu · 2020-03-13T17:31:17Z

I have successfully built pkgs.nix using this branch on top of master on my armv7 device so it's seems to be good to go.

grahamc · 2020-03-16T22:14:38Z

Updated to use the new URLs from @vcunat (thank you for the hard-to-mistake instructions :).)

cole-h

I'm still new here, so maybe there's a valid reason you're not following RFC 45. If there is, feel free to disregard this review (though I would like to hear an explanation so I can keep it in mind for the future).

EDIT: LGTM

pkgs/stdenv/linux/bootstrap-files/armv5tel.nix

pkgs/stdenv/linux/bootstrap-files/armv6l.nix

pkgs/stdenv/linux/bootstrap-files/armv7l.nix

grahamc · 2020-03-16T23:01:52Z

I need reviews on two things:

First, validate that for each URL, the string between stdenv-linux/ and /0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778 matches the name of the .nix file it is contained. Example: stdenv-linux/armv5tel/0eb0ddc4dbe3cd5415c6b6e657538eb809fc3778 is in armv5tel.nix.

Second, clone the repo and: nix-build pkgs/stdenv/linux/bootstrap-files/armv{5tel,6l,7l}.nix then nix-build pkgs/stdenv/linux/bootstrap-files/armv{5tel,6l,7l}.nix --check

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>

misuzu · 2020-03-16T23:06:33Z

I can confirm that URL's are correct.

After running nix-build pkgs/stdenv/linux/bootstrap-files/armv{5tel,6l,7l}.nix:

% nix-build pkgs/stdenv/linux/bootstrap-files/armv{5tel,6l,7l}.nix --check
checking outputs of '/nix/store/4pqc9wnk7bl58ldsrp19bag989apycyk-bootstrap-tools.tar.xz.drv'...
checking outputs of '/nix/store/5s7769ikm3q8xj9lqnimxmykmv5skzd1-bootstrap-tools.tar.xz.drv'...
warning: rewriting hashes in '/nix/store/07z7gmwq8hp426lz3jqffkgab2jxi90l-bootstrap-tools.tar.xz'; cross fingers
checking outputs of '/nix/store/df6rmkjx6ibmnciidb1xjb8cvd2s6vsa-bootstrap-tools.tar.xz.drv'...
warning: rewriting hashes in '/nix/store/3m5jx9hphpvgyk1y4xl6briydlr79ry4-bootstrap-tools.tar.xz'; cross fingers
checking outputs of '/nix/store/3f0x83pm1czzzh3xa53sh4k6hvhsrk1r-busybox.drv'...
warning: rewriting hashes in '/nix/store/rs65pfdrqvzw6h65i2jwdxkksvp0m459-busybox'; cross fingers
checking outputs of '/nix/store/cn5zp12vp0k2bm97k6ykr84mkw43bvlh-busybox.drv'...
warning: rewriting hashes in '/nix/store/mg8nzxs7mmg6ff47fgv8i61g6m818r8g-busybox'; cross fingers
checking outputs of '/nix/store/cw8afs6r5dnbvqwf7bnc8w86pd4x756b-busybox.drv'...
warning: rewriting hashes in '/nix/store/imsbql435j383ha6q6nlwn5bmqdnaqhq-busybox'; cross fingers
warning: rewriting hashes in '/nix/store/iwd9w34aif01zrs1bxz5kcdkawkv7i0a-bootstrap-tools.tar.xz'; cross fingers
/nix/store/07z7gmwq8hp426lz3jqffkgab2jxi90l-bootstrap-tools.tar.xz
/nix/store/rs65pfdrqvzw6h65i2jwdxkksvp0m459-busybox
/nix/store/3m5jx9hphpvgyk1y4xl6briydlr79ry4-bootstrap-tools.tar.xz
/nix/store/mg8nzxs7mmg6ff47fgv8i61g6m818r8g-busybox
/nix/store/iwd9w34aif01zrs1bxz5kcdkawkv7i0a-bootstrap-tools.tar.xz
/nix/store/imsbql435j383ha6q6nlwn5bmqdnaqhq-busybox

vcunat · 2020-03-17T06:59:18Z

I believe this MR should be picked to release-20.03. Any opinions? @NixOS/nixos-release-managers New glibc is in there as well, so the bootstrap is surely broken in there.

lopsided98 · 2020-03-17T15:02:38Z

If this is backported, make sure #79793 is as well.

expipiplus1 · 2020-06-16T14:52:37Z

It would be great to have this backported

stdenv: update ARM bootstrap tarballs

Combination of NixOS#79793 and NixOS#82051

ofborg bot added 6.topic: stdenv Standard environment 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Mar 8, 2020

grahamc requested review from lopsided98, thefloweringash and samueldr March 8, 2020 13:12

thefloweringash reviewed Mar 8, 2020

View reviewed changes

grahamc force-pushed the arm-bootstrap branch from 16bd48a to bbb1a4e Compare March 8, 2020 18:41

grahamc force-pushed the arm-bootstrap branch from bbb1a4e to 7232895 Compare March 8, 2020 18:58

This comment has been minimized.

Sign in to view

vcunat mentioned this pull request Mar 8, 2020

Unable to build glibc on armv7l #81972

Closed

veprbl linked an issue Mar 8, 2020 that may be closed by this pull request

Unable to build glibc on armv7l #81972

Closed

armv{5tel,6l,7l}: update bootstrap tools after #80497

11b54ad

grahamc force-pushed the arm-bootstrap branch from 7232895 to 11b54ad Compare March 16, 2020 22:12

cole-h reviewed Mar 16, 2020

View reviewed changes

armv{5tel,6l,7l}: bootstrap-files: quote URLs for rfc-45

6b361c3

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>

grahamc merged commit 5e57bea into master Mar 16, 2020

grahamc deleted the arm-bootstrap branch March 16, 2020 23:36

sarcasticadmin pushed a commit to sarcasticadmin/nixpkgs that referenced this pull request Aug 3, 2020

Merge pull request NixOS#82051 from NixOS/arm-bootstrap

639ec83

stdenv: update ARM bootstrap tarballs

andrew-d added a commit to andrew-d/nixpkgs that referenced this pull request Nov 19, 2020

Update bootstrap tools for ARM

bd5c1a9

Combination of NixOS#79793 and NixOS#82051

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stdenv: update ARM bootstrap tarballs #82051

stdenv: update ARM bootstrap tarballs #82051

grahamc commented Mar 8, 2020

grahamc commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

thefloweringash Mar 8, 2020

grahamc Mar 8, 2020

thefloweringash commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

lopsided98 commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

lopsided98 commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

This comment has been minimized.

vcunat commented Mar 8, 2020 •

edited

grahamc commented Mar 8, 2020

vcunat commented Mar 8, 2020 •

edited

lopsided98 commented Mar 8, 2020

vcunat commented Mar 10, 2020

misuzu commented Mar 13, 2020

grahamc commented Mar 16, 2020

cole-h left a comment •

edited

grahamc commented Mar 16, 2020

misuzu commented Mar 16, 2020 •

edited

vcunat commented Mar 17, 2020

lopsided98 commented Mar 17, 2020

expipiplus1 commented Jun 16, 2020

stdenv: update ARM bootstrap tarballs #82051

stdenv: update ARM bootstrap tarballs #82051

Conversation

grahamc commented Mar 8, 2020

Motivation for this change

Things done

grahamc commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

thefloweringash Mar 8, 2020

Choose a reason for hiding this comment

grahamc Mar 8, 2020

Choose a reason for hiding this comment

thefloweringash commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

lopsided98 commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

lopsided98 commented Mar 8, 2020

thefloweringash commented Mar 8, 2020

This comment has been minimized.

vcunat commented Mar 8, 2020 • edited

grahamc commented Mar 8, 2020

vcunat commented Mar 8, 2020 • edited

lopsided98 commented Mar 8, 2020

vcunat commented Mar 10, 2020

misuzu commented Mar 13, 2020

grahamc commented Mar 16, 2020

cole-h left a comment • edited

Choose a reason for hiding this comment

grahamc commented Mar 16, 2020

misuzu commented Mar 16, 2020 • edited

vcunat commented Mar 17, 2020

lopsided98 commented Mar 17, 2020

expipiplus1 commented Jun 16, 2020

vcunat commented Mar 8, 2020 •

edited

vcunat commented Mar 8, 2020 •

edited

cole-h left a comment •

edited

misuzu commented Mar 16, 2020 •

edited