Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump gcc to gcc10 and binutils to 2.34 #89793

Closed
wants to merge 26 commits into from
Closed

Conversation

luc65r
Copy link
Contributor

@luc65r luc65r commented Jun 8, 2020

Motivation for this change

I wanted to have a try at updating binutils (I took the work of @lovesegfault #86954), and as all packages needed to be recompiled, I tried to make gcc10 default.

Things done

  • Fix ld: multiple definition of (made gcc10 use -fcommon by default)
  • Fix packages to work with -fnocommon
    • gdbm
  • Fix python's _findLib_ld
  • Fix libomxil-bellagio build
  • Fix game-music-emu build (updated it)
  • Fix rEFInd build (TODO: build it with gcc10)
  • Fix dhcp build
  • Fix pythonPackages.cffi (TODO: make tests pass)
  • Fix pythonPackages.numpy (ImportError: liblapack.so.3: ELF load command address/offset not properly aligned)
  • Fix systemd

Make clean fixes for

  • rEFInd
  • pythonPackages.cffi
  • blas and lapack
  • libopus
  • qt512
  • ncmpcpp
  • libreoffice
  • range-v3
  • zimg

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@FRidh
Copy link
Member

FRidh commented Jun 8, 2020

We were at binutils 2.34 one or twice before but it got reverted.

@FRidh FRidh marked this pull request as draft June 8, 2020 12:29
@luc65r
Copy link
Contributor Author

luc65r commented Jun 8, 2020

@FRidh We need to make sure that most of the things work before merging.
Is it fine to do multiple things in one PR? I don't want to compile my system 2 times, it takes already long enough.

@FRidh
Copy link
Member

FRidh commented Jun 8, 2020

Please look at the past work that has been done on 2.34. This involves more work than you probably imagine right now.

@luc65r
Copy link
Contributor Author

luc65r commented Jun 8, 2020

Yes, I saw that, I have a lot of free time (not a lot of skills), and I hope people will help.

@lovesegfault
Copy link
Member

While I appreciate the effort to try and bump binutils, I have a few comments:

  1. It's not great that you chose to rewrite the previous patches and submit them without any authorship of the work that was done before to try and get this to land.
  2. Not sure the right approach is to go about reverting upstream commits. I recommend you go ask their mailing list so that, if not now in the future, we can avoid needing to maintain downstream patches.
  3. If you need someone to test things that requires lots of rebuilds ping me on IRC.

@luc65r
Copy link
Contributor Author

luc65r commented Jun 8, 2020

1. It's not great that you chose to rewrite the previous patches and submit them without any authorship of the work that was done before to try and get this to land.

Sorry, I was stupid.

2. Not sure the right approach is to go about reverting upstream commits. I recommend you go ask their mailing list so that, if not now in the future, we can avoid needing to maintain downstream patches.

You are right, should I try to patch python instead?

Also I'm a bit stuck with numpy (who would have thought...).

@ofborg ofborg bot requested review from domenkozar and LnL7 June 8, 2020 23:05
@luc65r
Copy link
Contributor Author

luc65r commented Jun 9, 2020

That's not really a fix.

It isn't, but I do not have the technical abilities to make a proper fix, I don't understand the install phases of lapack and blas.
It enables me to try fixing other packages which depends on numpy.

@luc65r luc65r force-pushed the staging branch 2 times, most recently from 40c966c to c07bec6 Compare June 12, 2020 18:52
@luc65r
Copy link
Contributor Author

luc65r commented Jun 13, 2020

I'm really stuck on systemd...

This involves more work than you probably imagine right now.

I never thought systemd could brake so badly xD

It fails to start a lot a services like journald, logind or udevd with error code 127.
It seems related to binutils, as it has the same behavior compiled with gcc9 and gcc10.

Here is a strace systemctl restart systemd-journald on a minimal VM. I don't know why it is failing.

@fabianhjr
Copy link
Member

Thanks @FRidh, @luc65r, et al. for working on this. :3

Will take a look at next-staging tomorrow and help stabilize it.

@FRidh
Copy link
Member

FRidh commented Dec 28, 2020

Several aarch64 builds fail with is referenced by DSO followed by bin/ld: final link failed: bad value

ar: `u' modifier ignored since `D' is the default (see `U')
  CXXLD    libprotoc.la
libtool: warning: '/nix/store/spps8qis3wmbr5051g64l1lzv82d7zsc-gcc-10.2.0-lib/lib/libstdc++.la' seems to be moved
ar: `u' modifier ignored since `D' is the default (see `U')
  CXXLD    protoc
/nix/store/d3a24znms2blwv2fvqcrmrhwqq08f425-binutils-2.34/bin/ld: .libs/protoc: hidden symbol `__aarch64_cas8_rel' in /nix/store/9zvbxwpzfp9k6xl9px67y0l2m8xy2d1w-gcc-10.2.0/lib/gcc/aarch64-unknown-linux-gnu/10.2.0/libgcc.a(cas_8_3.o) is referenced by DSO
/nix/store/d3a24znms2blwv2fvqcrmrhwqq08f425-binutils-2.34/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:3892: protoc] Error 1
make[2]: Leaving directory '/build/source/src'
make[1]: *** [Makefile:1874: all-recursive] Error 1
make[1]: Leaving directory '/build/source'
make: *** [Makefile:1781: all] Error 2
builder for '/nix/store/cl5fmbdqxnkvmfw21s5wwz2lrqpc72lq-protobuf-3.14.0.drv' failed with exit code 2

@luc65r
Copy link
Contributor Author

luc65r commented Dec 28, 2020

I didn't test on aarch64 and darwin at all, we might have to force gcc9 on them for now.

@vcunat
Copy link
Member

vcunat commented Dec 28, 2020

Darwin uses clang as default.

@flokli
Copy link
Contributor

flokli commented Dec 28, 2020

I can reproduce the build error with nix-build -A protobuf on a aarch64-linux box.

I can't reproduce it when cross-compiling from x86_64.

@kira-bruneau
Copy link
Contributor

kira-bruneau commented Dec 28, 2020

Thanks everyone for working on this!

I noticed two small issues building my system with 4008fd3:

I noticed that stripping was re-enabled on wine, so I will also confirm if binutils 2.34 fixes the original issue.

EDIT: strip with binutils 2.34 now keeps the original DOS stub instead of resetting it, so Wine builtin DLL & Wine placeholder DLL markers are no longer corrupted!

@FRidh
Copy link
Member

FRidh commented Dec 29, 2020

cc @NixOS/exotic-platform-maintainers for assistance in fixing up aarch64. See #107783.

@FRidh
Copy link
Member

FRidh commented Dec 29, 2020

possibly related aarch64 fix https://bugzilla.redhat.com/show_bug.cgi?id=1830472#c1

On aarch64 -moutline-atomics has been turned on by default, and those symbols are solely in libgcc.a, not in libgcc_s.so.*.
So, libgcc_s.so linker script is needed instead of libgcc_s.so symlink like on some other arches, gcc-10.0.1-0.14.fc33 with a fix is building in koji.

found via https://www.spinics.net/lists/fedora-devel/msg270592.html

@vcunat
Copy link
Member

vcunat commented Dec 29, 2020

Sounds it could be "the same problem". Their patch: https://src.fedoraproject.org/rpms/gcc/c/95507e8b68

@FRidh
Copy link
Member

FRidh commented Jan 1, 2021

They remove libgcc_s.so and then replace it with

GROUP ( /%{_lib}/libgcc_s.so.1 libgcc.a )' > $FULLPATH/libgcc_s.so

If I am correct libgcc_s.so is actually a statically linked file despite the .so, and this line here merges the two libraries into a new one, which is I think also what they meant with the message I quoted.

We have libgcc_s.so in the glibc store path, not in the gcc path. It is pulled in by the wrapper in the file /nix/store/y78inja64xaaq6bx33nxn2h1lgw0jswk-gcc-wrapper-10.2.0/nix-support/libc-crt1-cflags which contains

-B/nix/store/rqrklqsvw4ydpcg5kdcvn506fhcbqxk2-glibc-2.32-10/lib/

This is defined in pkgs/build-support/cc-wrapper/default.nix

echo "-B${libc_lib}${libc.libdir or "/lib/"}" >> $out/nix-support/libc-crt1-cflags

Note also the comment

# The "-B${libc_lib}/lib/" flag is a quick hack

@luc65r
Copy link
Contributor Author

luc65r commented Jan 4, 2021

The tests for pkgsi686Linux.libopus pass with xnorpx/opus@6e2209d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

None yet