glibc: don't use bootstrap libgcc_s #36948
Conversation
|
No attempt on x86_64-darwin The following builds were skipped because they don't evaluate on x86_64-darwin: glibc No log is available. |
|
Success on aarch64-linux (full log) Attempted: glibc Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: glibc Partial log (click to expand)
|
|
Yep, |
|
build log of the failure, for the curious: https://gist.github.com/dtzWill/9aea1fb5695788fcb0a63c96f32d36d4 This is the error we currently find w/trying to use clang as a cross-compiler, which is what started me looking into all of this :). cc #36867 |
|
@dtzWill if you are at it, could you maybe also teach clang to find c++ stdlib headers without compiler wrappers on linux? This would fix all tooling that uses libclang. I tried to fix it in https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L758 but this only affects clang from a build directory. |
|
that one looks also interesting: https://github.com/llvm-mirror/clang/blob/master/lib/Driver/ToolChains/Linux.cpp#L804 |
|
Yeah we need to get change GCC at some point so the compiler itself and runtime libraries are built separately. That will make the one problem caused by this go away! |
|
Hmm, I suppose we can afford this now that we have |
|
Hmm, I think this issue is currently making some builds fail for 18.03, esp. ones that are required for the channel to move forward and actually come into existence (spidermonkey_38 for i686 in particular is required by tests that are required for 18.03, if I read hydra correctly). Would it make sense to merge this PR into (cc ZHF #36453) |
|
Eh, better not do such experiments directly on 18.03 anymore. After it's confirmed that this works good, we can cherry-pick it in 18.03. We may have a separate temporary jobset if you think there's a risk of mass breakage, but I suppose it won't be that risky. |
|
Plus 18.03's tested job already succeeded so as I understand it this shouldn't block channel updates (I can't see the channel being created yet but I suppose that's a different issue). |
|
Perhaps a new jobset would be appropriate then? |
|
@pbogdan oh indeed, I was misreading the jobs tab as the tested job, because I didn't understand why 18.03 hadn't been created yet -- my reasoning was completely wrong, then, sorry for the noise :) |
Ericson2314
added
the
6.topic: portability
|
I added the 18.09 milestone because I definitely want this to happen by then. Not trying to say it shouldn't also happen for 18.03 :). |
|
For reference, the 18.03 channels are there now, for about a day. The machine bumping them wasn't immediately re-deployed with the new configuration, by mistake. |
|
Can we still do this? A quick hack for LLVM in cc-wrapper should be easy enough. |
|
Agreed, and we can probably improve the libclang situation while we're at it. Definitely want this to happen, just not sure when :) |
|
Doing this would make everything that needs on |
|
We soon will for compiler-rt! #39743 |
|
Unexpected error: command failed with exit code 1 on aarch64-linux (full log) Attempted: pkgsi686Linux.dejagnu Partial log (click to expand)
|
|
Timed out, unknown build status on x86_64-linux (full log) Attempted: pkgsi686Linux.dejagnu Partial log (click to expand)
|
The underlying problem with libgcc is worked around. (cherry picked from commit afea12f)
dtzWill
force-pushed
the
fix/glibc-libgcc_s
branch
from
38b17f5
to
ff30d7b
Compare
|
No attempt on x86_64-darwin (full log) The following builds were skipped because they don't evaluate on x86_64-darwin: glibc Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: glibc Partial log (click to expand)
|
|
Success on x86_64-linux (full log) Attempted: glibc Partial log (click to expand)
|
matthewbauer
force-pushed
the
fix/glibc-libgcc_s
branch
2 times, most recently
from
8d7d962
to
e26c8aa
Compare
matthewbauer
force-pushed
the
fix/glibc-libgcc_s
branch
from
e26c8aa
to
2ed8ee4
Compare
|
FWIW |
|
|
|
@dtzWill is there any way to avoid these issues? I still like getting rid of references to libgcc... But it may need to wait for another time? |
|
Ok let's revert for now. ideally we can get this fixed at a later point. 319ebef is the revert. |
|
Aww can't reopen this? 😿 Suppose can always make a new one if someone finds time to tackle the issues. Not sure I prefer the current state (subtle issues everywhere) over breaking things explicitly and forcing the issue, but that's just me O:). Probably not best to do that with a release looming, I admit... :) |
Let's try this again. See NixOS#36947 and NixOS#36948 for history
Let's try this again. See NixOS#36947 and NixOS#36948 for history
|
I figured out the root cause of these segfaults. |
#### Immediate Benefits
- Allow `gcc11` on `aarch64`
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more [static `lib{mpfr,mpc,gmp,isl}.a`
hack](https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380)
- *Zero* additional `gcc` builds (stage1+stage2+stageCompare)
- The `gcc` derivation builds `gcc` once instead of three times.
- The libraries that are linked into the final `pkgs.gcc` (`mpfr`,
`mpc`, `gmp`, `isl`, `glibc`) are built by
`stdenv.__bootPkgs.gcc` rather than by the `bootstrapFiles`. No
more Frankenstein compiler!
- stageCompare runs **concurrently** with (not in series with)
with `stdenv`'s dependees.
- Many other `stdenv` hacks eliminated.
- `gcc` and `clang` share the same codepath for more of
`cc-wrapper`.
- Makes the cross and native codepaths much more similar --
another step towards "cross by default".
Note that *all* the changes in this PR are controlled by flags; no
old codepaths need to be removed until/if we're completely certain
that this is the right way to go.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- There will be an "avalanche of simplification" when we set
`enableGccExternalBootstrap=true` and run dead code elimination.
It's really quite a huge amount of code that goes away.
Native-gcc has its own special codepath in so many places, while
cross-gcc and clang work the same way (and are much simpler).
- This should allow each of the libraries that ship with `gcc`
(`lib{backtrace,atomic,cc1,decnumber,ffi,gomp,iberty,offloadatomic,quadmath,sanitizer,ssp,stdc++-v3,vtv}`)
to be built in separate (one-liner) derivations which `inherit
src;` from `gcc`.
- Building `libstdc++-v3` in a separate derivation will eliminate
a lot of accidental-reference-to-the-`bootstrapFiles` landmines.
#### Incorporates
- NixOS#209054
- NixOS#210004
- NixOS#36948 (unreverted)
- NixOS#210325
- NixOS#210118
- NixOS#210132
- NixOS#210109
#### Closes
- Closes NixOS#208412
- Closes NixOS#108111
- Closes NixOS#108305
- Closes NixOS#201254
#### Build history
- First successful builds (stage1/stage2):
- powerpc64le-linux at 9c7e9ef
- x86_64-linux at 9c7e9ef
- aarch64-linux at 4d5bc7d
- First successful comparisons (stageCompare):
- at 81949cf
- [aarch64-linux][aarch64-compare-ofborg]
- [x86\_64-linux][amd64-compare-ofborg]
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @Ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. External
bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds. Now that gcc is written in C++, it depends on
`libstdc++`, builds a copy of `libstdc++`, and builds auxiliary
products (like `libplugin`) which depend on `libstdc++`. @trofi
developed two important techniques for keeping this straight: the
use of a [nonexistent sysroot] and moving the `bootstrapFiles`'
`libstdc++` into a [versioned directory]. Without these two
discoveries, external bootstrap would be impossible, because the
final gcc would still have references to the `bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: NixOS#210004
[versioned directory]: NixOS#209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: NixOS#210112 (comment)
[crisis]: NixOS#108305
[foreign]: NixOS#170857 (comment)
Fixes #36947
build-use-sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)Huge rebuild, have not tested everything obviously :).
Wanted to see what broke and was surprised to find
that things didn't immediately and completely break.
I'm curious if clang works after this, we'll see!