Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

glibc: fix /etc/ld-nix.so.preload sharing #96289

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

glibc: fix /etc/ld-nix.so.preload sharing #96289

wants to merge 1 commit into from
+19 −20

Conversation

ju1m
Copy link
Contributor

@ju1m ju1m commented Aug 25, 2020

Untested by me yet (require too much rebuilds for my computer), but ready for reviews.

Motivation for this change

Sharing /etc/ld-nix.so.preload accross programs requiring different versions of glibc can break everything, including upgrading NixOS, as experienced there: https://discourse.nixos.org/t/nixos-rebuild-fails-because-bash-is-broken-dependency-issue-in-libpthread-so-0-undefined-symbol-nanosleep-nocancel/8494

For example, when using environment.memoryAllocator.provider = "scudo", as done in nixos/modules/profiles/hardened.nix, one could get a /etc/ld-nix.so.preload loading glibc-2.30:

# cat /etc/ld-nix.so.preload
/nix/store/in0cn8csqlx9al9ih5xj5j7gkw11ladd-malloc-provider-scudo/lib/libclang_rt.scudo-x86_64.so

# ldd /nix/store/in0cn8csqlx9al9ih5xj5j7gkw11ladd-malloc-provider-scudo/lib/libclang_rt.scudo-x86_64.so
        linux-vdso.so.1 (0x0000694ebde48000)
        libgcc_s.so.1 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libgcc_s.so.1 (0x0000694ebd54d000)
        libc.so.6 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libc.so.6 (0x0000694ebd38e000)
        libdl.so.2 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libdl.so.2 (0x0000694ebd389000)
        librt.so.1 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/librt.so.1 (0x0000694ebd37f000)
        libpthread.so.0 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libpthread.so.0 (0x0000694ebd35e000)
        libstdc++.so.6 => /nix/store/rmn2n2sycqviyccnhg85zangw1qpidx0-gcc-9.3.0-lib/lib/libstdc++.so.6 (0x0000694ebd17d000)
        /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib64/ld-linux-x86-64.so.2 (0x0000694ebde49000)
        libm.so.6 => /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libm.so.6 (0x0000694ebd03b000)

And then a nixos-rebuild switch would break when upgrading to a NixOS version using a bash built with glibc-2.31:

$ /nix/store/6737cq9nvp4k5r70qcgf61004r0l2g3v-bash-4.4-p23/bin/bash
/nix/store/6737cq9nvp4k5r70qcgf61004r0l2g3v-bash-4.4-p23/bin/bash: symbol lookup error: /nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30/lib/libpthread.so.0: undefined symbol: __nanosleep_nocancel, version GLIBC_PRIVATE
Things done
  • Rename /etc/ld-nix.so.preload to /etc/ld-nix.${pkgs.glibc.src.outputHash}.so.preload
  • Fix incoming apparmor profile in apparmor: fix and improve the service #93457 . This patch being likely to me merged before the broader apparmor one, I'll fix it in the apparmor one if the present one is merged.
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-rebuild-fails-because-bash-is-broken-dependency-issue-in-libpthread-so-0-undefined-symbol-nanosleep-nocancel/8494/4

bqv
bqv reviewed Aug 25, 2020
View reviewed changes
nixos/modules/config/malloc.nix
environment.etc."ld-nix.so.preload".text = ''
# See comment in pkgs/development/libraries/glibc/common.nix
# for an explanation about the name of this file.
environment.etc."ld-nix.${pkgs.glibc.src.outputHash}.so.preload".text = ''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps just change target, so this attr has a predictable name?

@ofborg ofborg bot requested a review from edolstra August 25, 2020 16:28
vcunat
vcunat reviewed Aug 26, 2020
View reviewed changes
Copy link
Member

@vcunat vcunat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good in principle for this particular use case. I'm not at all sure about others.

If you want to replace (part of) glibc, you may want to do it based on its hash. If you want to replace (part of) some other library, you might want to do it based hash of that library.

Moreover, it depends on the nature of the preload – I've seen many preloads that are just thin API wrappers that are OK to use with basically any version of the library (e.g. any POSIX libc).

@ju1m ju1m mentioned this pull request Nov 16, 2020
Closed
@stale
Copy link

stale bot commented Jul 21, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 21, 2021
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
@wegank wegank marked this pull request as draft March 25, 2024 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bqv bqv bqv left review comments

@vcunat vcunat vcunat left review comments

@edolstra edolstra Awaiting requested review from edolstra

Assignees
No one assigned
Labels
2.status: merge conflict 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 11-100 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ 10.rebuild-linux-stdenv
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@ju1m @nixos-discourse @bqv @vcunat @ryantm @SuperSandro2000 @wegank