Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

binutils: 2.31.1 -> 2.33.1 #78204

Closed
wants to merge 1 commit into from
Closed

Conversation

guibou
Copy link
Contributor

@guibou guibou commented Jan 21, 2020

Motivation for this change

Bump of binutils to a more recent version. It closes #78197 . Please review with care because I have NO idea what I'm doing. This will trigger a mass rebuild and mostly everything in nixpkgs depend on this.

I'm opening this PR as a proof of motivation to work on this issue, but really, feel free to tell me that I totally missed something and that binutils is stuck at 2.31.1 on nixpkgs for good reasons.

Things done

I bumped the package and ensured patchs applies. This is a mass rebuild, so I did not even tried to wait for the end of the build on my laptop. At least, patches are correctly applied (after fixup) and the build is running.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

- I've removed the stack of patch linked to
https://sourceware.org/bugzilla/show_bug.cgi?id=23428 . The associated
issue says it is closed and targeted for 2.32.

- I've ugraded the "no_plugin" patch. The logic changed in
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=41f37a6fb71f2a3de388108f5cdfca9cbe6e9d51
and I tried to keep the same logic by disabling everything.

It closes NixOS#78197
@tomberek
Copy link
Contributor

tomberek commented Feb 2, 2020

I get a few failures with coreutils:

tests/misc/mknod.sh
tests/cp/parent-perm-race.sh
tests/mkdir/perm.sh

seems to be the same ones as grahamcofborg gets (x86_64-linux, NixOS)

@FRidh FRidh added this to the 20.09 milestone Feb 16, 2020
@lovesegfault
Copy link
Member

I get some test failures with this. Here's the full build log

@lovesegfault
Copy link
Member

Also: binutils-2.34 has since been released.

@lovesegfault
Copy link
Member

Here's the patch to apply to this PR to get 2.34:

diff --git a/pkgs/development/tools/misc/binutils/default.nix b/pkgs/development/tools/misc/binutils/default.nix
index e1ff92d00f4..ebbddbb2190 100644
--- a/pkgs/development/tools/misc/binutils/default.nix
+++ b/pkgs/development/tools/misc/binutils/default.nix
@@ -8,15 +8,13 @@
 , bison ? null
 , flex
 , texinfo
+, perl
 }:
 
 let
   reuseLibs = enableShared && withAllTargets;
 
-  # Remove gold-symbol-visibility patch when updating, the proper fix
-  # is now upstream.
-  # https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=330b90b5ffbbc20c5de6ae6c7f60c40fab2e7a4f;hp=99181ccac0fc7d82e7dabb05dc7466e91f1645d3
-  version = "2.33.1";
+  version = "2.34";
   basename = "binutils";
   # The targetPrefix prepended to binary names to allow multiple binuntils on the
   # PATH to both be usable.
@@ -31,7 +29,7 @@ let
   # HACK to ensure that we preserve source from bootstrap binutils to not rebuild LLVM
   normal-src = stdenv.__bootPackages.binutils-unwrapped.src or (fetchurl {
     url = "mirror://gnu/binutils/${basename}-${version}.tar.bz2";
-    sha256 = "1cmd0riv37bqy9mwbg6n3523qgr8b3bbm5kwj19sjrasl4yq9d0c";
+    sha256 = "1rin1f5c7wm4n3piky6xilcrpf2s0n3dd5vqq8irrxkcic3i1w49";
   });
 in
 
@@ -64,10 +62,6 @@ stdenv.mkDerivation {
     # cross-compiling.
     ./always-search-rpath.patch
 
-  ] ++ lib.optionals (!stdenv.targetPlatform.isVc4)
-  [
-    # https://sourceware.org/bugzilla/show_bug.cgi?id=22868
-    ./gold-symbol-visibility.patch
   ] ++ lib.optional stdenv.targetPlatform.isiOS ./support-ios.patch;
 
   outputs = [ "out" "info" "man" ];
@@ -75,9 +69,11 @@ stdenv.mkDerivation {
   depsBuildBuild = [ buildPackages.stdenv.cc ];
   nativeBuildInputs = [
     bison
+    perl
+    texinfo
   ] ++ (lib.optionals stdenv.targetPlatform.isiOS [
     autoreconfHook
-  ]) ++ lib.optionals stdenv.targetPlatform.isVc4 [ texinfo flex ];
+  ]) ++ lib.optionals stdenv.targetPlatform.isVc4 [ flex ];
   buildInputs = [ zlib gettext ];
 
   inherit noSysDirs;

@lovesegfault
Copy link
Member

lovesegfault commented Apr 23, 2020

2.34 fails in the same way (build log)

Here are the salient points:

coreutils> FAIL: tests/misc/mknod
coreutils> ======================
coreutils> mknod: cannot set permissions of 'f1': Function not implemented
coreutils> mkfifo: cannot set permissions of 'f2': Function not implemented
coreutils> mkdir: cannot change permissions of 'f3': Function not implemented
coreutils> FAIL tests/misc/mknod.sh (exit status: 1)
coreutils> FAIL: tests/cp/parent-perm-race
coreutils> ===============================
coreutils> cp: preserving permissions for 'd/ownership': Function not implemented
coreutils> FAIL tests/cp/parent-perm-race.sh (exit status: 1)
coreutils> FAIL: tests/mkdir/perm
coreutils> ======================
coreutils> mkdir: cannot change permissions of 'parent/sub': Function not implemented
coreutils> parent/sub: expected d-wx--x-w-, got d-wx----w-
coreutils> mkdir: cannot change permissions of 'parent/sub': Function not implemented
coreutils> parent/sub: expected d-wx--x-w-, got d-wx----w-
coreutils> FAIL tests/mkdir/perm.sh (exit status: 1)

@lovesegfault
Copy link
Member

Checking whether coreutils 8.32 fixes the issue.

@lovesegfault
Copy link
Member

coreutils 8.32 fails in the exact same way. Unclear to me why this is happening.

@lovesegfault
Copy link
Member

Could this have something to do with how the fs where these tests run is mounted? cc. @andi @flokli.

@flokli
Copy link
Contributor

flokli commented Apr 24, 2020

@lovesegfault We disable various syscalls that might produce files which we can't serialize into the nix-store, see https://github.com/NixOS/nix/blob/master/src/libstore/build.cc#L3001-L3028.

In the case of coreutils' tests/cp/parent-perm-race, it tries to chmod g+s a directory, which is disallowed by the seccomp configured inside the first part of the linked code.

For now, I propose disabling these problematic tests, if they're all because of these restrictions. I have to admit, I haven't looked into the other failing tests :-).

Back to "how can we actually enable these tests": Nix runs builds as unprivileged users, and sources are chown'ed to the build user. So we might be able to at least lift the setuid/setgid binary restrictions, if we check the outputs after the build to not contain these bits.

The reasoning behind is that build processes should be fine to create setuid/setgid binaries at will, as long as they are not persisted in the outputs. Assuming the testsuite puts these artifacts in a temporary directory, or cleans up afterwards, there shouldn't be setuid/setgid binaries in the output.

I assume we already walk over all the files once after the build, so checking for no leftover setuid/setgid bits and bailing out if we see something shouldn't be that costly.

Edit: Contains related information, but is not what we'd actually want here: NixOS/nix#2522

@lovesegfault lovesegfault mentioned this pull request Apr 24, 2020
10 tasks
@flokli
Copy link
Contributor

flokli commented Apr 24, 2020

closing in favor of #85951.

@flokli flokli closed this Apr 24, 2020
Staging automation moved this from WIP to Done Apr 24, 2020
@lovesegfault lovesegfault mentioned this pull request May 5, 2020
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

5 participants