Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glibc: 2.27 -> 2.30 WIP #66528

Merged
merged 60 commits into from Feb 5, 2020
Merged

glibc: 2.27 -> 2.30 WIP #66528

merged 60 commits into from Feb 5, 2020

Conversation

lblasc
Copy link
Contributor

@lblasc lblasc commented Aug 12, 2019

Motivation for this change

Bump to latest glibc.

Started to fix core packages, managed to build and test rustc on x86_64.

  • can someone please point me to latest bootstrap files for i686 and x86_64 on tarballs.nixos.org so I can remove my private mirror from pkgs/stdenv/linux/bootstrap-files/* ? @grahamc @Mic92
  • hydra jobset for this would be great
  • test darwin
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @edolstra

@jonringer
Copy link
Contributor

You're a madman, and i support you on your journey ❤️

However, such a universal package may cause a lot more heart-ache than it's worth. (don't want you to get burnt out)

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2019

cc @LnL7

@vcunat
Copy link
Member

vcunat commented Feb 5, 2020

Thanks for the fast reaction. I guess I have to ping @edolstra, as I don't know about anyone else.

@vcunat
Copy link
Member

vcunat commented Feb 5, 2020

@Ma27

I talked with a few people at FOSDEM how we could improve the situation with bootstrap files (e.g. by using stuff from a binary cache by default), but this is something we should take care of after the branchoff.

There seems to be some misunderstanding. Bootstrap files are substituted from cache.nixos.org (if present); see e.g. log from busybox a couple posts above.

  • The regular Hydra job as of now doesn't produce these as separate files, so it's the same files but nested deeper in other /nix/store paths and can't be directly re-used. Still, it's not difficult to nix build $file and then nix-prefetch-url file://$file
  • Users with different location of nix store can't use the official binary cache (the hash parts are changed), so it's good to have working fixed-output fetches for them to bootstrap from. Hence tarballs.nixos.org
  • These fetches also serve as backup in case the files got garbage-collected from the cache, though I suspect in our case that's just a theoretical possibility for now.

Maybe I forgot some point. We certainly had at least one GitHub discussion about the (non-)utility of tarballs.nixos.org.

as a workaround to fix build after updating stdenv bootstrap
(in the followup commit).  Interested parties can improve this later.
From Hydra's binaries for c5aabb0 (19.09).
This time I re-tested full stdenv bootstrap on these three platforms.
NixOS#66528 (comment)
@edolstra
Copy link
Member

edolstra commented Feb 5, 2020

@vcunat Thanks, I've copied the files.

I suppose the diff of the glibc-upgrade branch is a bit cleaner
without including these unnecessary changes.
@vcunat vcunat self-assigned this Feb 5, 2020
vcunat added a commit that referenced this pull request Feb 5, 2020
Includes update of stdenv bootstap tools (for three main platforms)
and many package fixes with new glibc.
@vcunat vcunat merged commit 5ca088f into NixOS:staging Feb 5, 2020
Staging automation moved this from WIP to Done Feb 5, 2020
@vcunat
Copy link
Member

vcunat commented Feb 5, 2020

Confirmed, except for the optional one (but that one doesn't matter).

@edolstra
Copy link
Member

edolstra commented Feb 5, 2020

No, I copied it as well. Note that you have a typo in the URL (http://tarballs.nixos/org...).

@lblasc
Copy link
Contributor Author

lblasc commented Feb 5, 2020

@vcunat @Ma27 it was a pleasure working with you guys! I've started this as a learning experience to investigate how NixOS bootstrapping works, I'm very pleased to have contribution which rebuilds the whole dependency tree 🔥 on three major archs 😁

@vcunat
Copy link
Member

vcunat commented Feb 5, 2020

Oh, right, typo; pushed as cf24265. First x86_64 binaries should start compiling today; the rest when staging-next get rotated.

@lblasc: it's a bit unlucky that it was a difficult case and took us so much time – about half a year. Usually changes aren't so cumbersome, even mass rebuilds get merged basically every week.

@vcunat
Copy link
Member

vcunat commented Feb 5, 2020

Repeated note: 32-bit ARMs are quite certainly broken by this PR. /cc @dezgeg (again) Regenerating bootstrap tools for them might suffice to fix them, but I don't have a good way of testing the builds natively.

I suspect cross-building these platforms might keep working, except I now see that this PR broke evaluation of (all?) cross stuff 🤕

@Ma27
Copy link
Member

Ma27 commented Feb 5, 2020

I suspect cross-building these platforms might keep working, except I now see that this PR broke evaluation of (all?) cross stuff

Can you please point me to a Hydra eval where I can see the errors?

@FRidh FRidh mentioned this pull request Feb 5, 2020
10 tasks
bjornfor added a commit that referenced this pull request Feb 10, 2020
Fixes this build error:

  In file included from src/helper/options.c:38:
  /nix/store/dl4h1p847f2rsrsfvlmm6cxxx7q21kxj-glibc-2.30-dev/include/sys/sysctl.h:21:2: error: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Werror=cpp]
     21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed."
        |  ^~~~~~~
  cc1: all warnings being treated as errors

Fixes: 48a997c ("Merge #66528: glibc: 2.27 -> 2.30 (into staging)")
@jcumming
Copy link
Contributor

Uh-oh. These bootstrap tools no work on SLES 11-SP4 3.0.101-108.87-default kernels:

Unpacking the bootstrap tools...
Patching the bootstrap tools...
FATAL: kernel too old

@Ma27
Copy link
Member

Ma27 commented Feb 11, 2020

Just to properly understand your issue: you're using Nix on Sles11sp4 and our new bootstrap tools break the setup, right?

However I'd like to point out that I'm pretty much against putting additional effort in fixing stuff for unmaintained versions of other distros (well unless you're paying for LTSS, but that's another story), but that's just my opinion :)

@vcunat
Copy link
Member

vcunat commented Feb 11, 2020

@jcumming: did it even work before this PR? I've never looked into this for SLES/SLED. We haven't changed this part for quite a long time:

"--enable-kernel=3.2.0" # can't get below with glibc >= 2.26

We have special patch to allow RHEL 6 -like kernels. That reminds me I last verified the real compatibility with that on glibc-2.26-131... so even though you don't get this nice refusal, some parts might not work on those kernels anymore. Well, today I suspect we might even drop RHEL 6 support 🤷‍♀️ I personally don't feel it's important anymore.

@vcunat
Copy link
Member

vcunat commented Feb 11, 2020

To be clear, it isn't that much about bootstap... but about running anything built with our libc (i.e. with our stdenv).

@vcunat
Copy link
Member

vcunat commented Feb 11, 2020

Actually, I think previous service packs with 2.6.32 kernel would not give you the error message, by co-incidence with RHEL 6 :-)

@infinisil infinisil mentioned this pull request Feb 21, 2020
1 task
@vbgl vbgl mentioned this pull request Feb 25, 2020
10 tasks
@Ma27 Ma27 mentioned this pull request Feb 2, 2021
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

None yet