By analyzing the blame information on this pull request, we identified @edolstra, @nckx and @urv to be potential reviewers

The bogus Travis failure should be fixed now.

That is one (IMO too) huge diff... any chance of splitting this megacommit into smaller pieces? I count roughly 10 commits worth of changes.

I split up the commit into separate commits for each package. I also made a modification to the ld.so.conf, so it's populated using $NIX_PROFILES instead of hard-coding the /nix folders. This should make it work when nix is installed under a custom prefix. Please let me know if there is anything I can fix or improve. Thanks!

The ld.so.conf thing is probably not going to be well received. Instead what could work is to patch glibc to search for the NSS modules in say, /run/nss-modules and then have a NixOS module to populate that if SSSD is enabled. This is similar to how OpenGL drivers are handled on NixOS.

Is there some way this could work for nix when installed as a prefix on
another distro? Could the search path be somewhere in /nix/var instead of
/run? If a nix installed program such as perl or python is running in
another distro thats set up with sss in the nsswitch conf file, then nix's
glibc should still be able to find the nix-installed libnss_sss.so file.
On Apr 16, 2016 8:34 PM, "Tuomas Tynkkynen" notifications@github.com
wrote:

The ld.so.conf thing is probably not going to be well received. Instead
what could work is to patch glibc to search for the NSS modules in say,
/run/nss-modules and then have a NixOS module to populate that if SSSD is
enabled. This is similar to how OpenGL drivers are handled on NixOS.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#14697 (comment)

What is the rationale for not wanting to use ld.so.conf in this way? To me it seemed like the ideal way to add to the library search path without having to patch glibc, since this search capability is already built into glibc.

This PR as it is does not create the {glibc.out}/etc/ld.so.conf file. I think it's because builder.sh defines its own postInstall, overriding the one in glibc/default.nix.

Also note that $NIX_PROFILES is empty during build, at least when using sandboxing, and anyway, you can't anticipate user home locations during build, if that's what you intended. There is $NIX_STORE variable, so I suppose $NIX_STORE/../var/nix/profiles could be used, but searching such potentially large folders for libraries is bound to lead to problems.

Looking at po4a, please don't use the old texLive anymore. See texlive documentation: http://nixos.org/nixpkgs/manual/#sec-language-texlive

@vcunat thanks for your input! I'm working on fixing my patch with your suggestions. I'm going to go ahead and rebuild everything from scratch to make sure my glibc changes work, which could take awhile.

searching such potentially large folders for libraries is bound to lead to problems.

Can you elaborate on this? glibc on other linux systems searches /lib and /usr/lib for libnss_* libraries, do you think /nix/var/nix/profiles/system/sw/lib and /nix/var/nix/profiles/default/lib could have a significantly larger amount of files to search through?

Okay, so I think I see another problem: the ld.so.cache file is caching the paths to each individual library, not the folders themselves. So if a new library is added to one of the profile folders, the ld.so.cache would have to be updated, which is of course impossible....

Damn, I guess the only way to do this is to patch glibc to search for other folders. What would be the performance implications of this? The ld.so.cache mechanism seems to be used to speed up dynamic library lookups. Telling glibc to search other directories that aren't cached in ld.so.cache could cause a performance hit, but I'm not sure how significant that would be. I'll have to think about this some more...

Is there some way to get glibc to depend on sssd, so I can copy the libnss_sss.so file into the glibc folder, without causing infinite recursion? sssd depends on glibc, but glibc needs libnss_sss.so from sssd...

searching such potentially large folders for libraries is bound to lead to problems.

Can you elaborate on this? glibc on other linux systems searches /lib and /usr/lib for libnss_* libraries, [...]

The problem is that AFAIK adding stuff into ld.so.cache makes it look into the folders when loading any libraries. With nix we almost always run against exactly the same versions of libraries that we build against. It's not good to allow changing all linkage inadvertently by a nix-env command.

Is there some way to get glibc to depend on sssd

That's standard bootstrapping problem. It should be doable by changes to http://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/linux/default.nix, but it would certainly seem better to have a solution that doesn't involve a mass rebuild when switching sssd on/off.

I anticipate a couple options:

1. libGL-like solution, i.e. let NixOS extend $LD_LIBRARY_PATH by a directory with libnss plugins symlinked into place.
2. Patch glibc to add a special environment variable containing a directory or list of full paths or something like that.

But one should consider:

• security implications – any user can override such variables;
• what will happen in case of version mismatch, i.e. when the plugins are loaded by a glibc version completely different from the one it built against.

I removed the texLive import and fixed a parallel build issue with jade.

I also removed the glibc commit since it's currently broken. I'm not sure how to proceed since changing something as fundamental as glibc needs more guidance and supervision. It might make more sense to submit the glibc changes as a separate PR.

Would it be possible to get sssd merged without the glibc changes? It wouldn't work properly until a solution to the glibc problem is found, but at least people could make use of use it with some hacks, e.g. manually copying the .so files into the glibc/lib folder.

1. libGL-like solution, i.e. let NixOS extend $LD_LIBRARY_PATH by a directory with libnss plugins symlinked into place.

I played around with this, and its workable (on NixOS) without recompiling glibc. The main issue here is that creating a systemd --user session is apparently done by systemd by first dropping privileges and then running the user daemon in the environment specified by the user@ unit. For users that don't exist on the local system, the first step fails unless PID 1 itself is able to find the nss module. So I had to put the LD_LIBRARY_PATH export directly into the stage 2 init script.

I could try and create a PR from this if you think its a viable solution.

security implications – any user can override such variables;

Isn't that always the case? On any distribution, users can override LD_LIBRARY_PATH and make programs load different versions of nss modules. So all security issues that could occur in NixOS using this approach could also occur anywhere else...

So I had to put the LD_LIBRARY_PATH export directly into the stage 2 init script.

@cruegge, how did you do this? I tried adding this to my configuration.nix:

  boot.postBootCommands = ''
    export LD_LIBRARY_PATH=${pkgs.sssd}/lib
  '';

The environment variable doesn't seem to be inherited by systemd. How did you modify the stage2 init script to make this work? Thanks!

Also, how does your solution deal with prefix-installed Nix running on another distro? In that case, we don't have control over the init scripts, so I don't know how we could set the LD_LIBRARY_PATH.

@benwbooth there seems to be a failure building this against the current nix master on a prefixed nix install

$ nix-env -i sssd -f $NIXPKGS
...
mv -f /tmp/nix-build-glibc-locales-2.23.drv-0/build/sysd-versionsT /tmp/nix-build-glibc-locales-2.23.drv-0/build/sysd-versions
(while read which lib number setname; do \
   eval seen_$which=1; \
   test x"$which" = xDEFAULT || continue; \
   case $number in \
     [0-9]*) echo "$lib.so-version=.$number"; \
             echo "all-sonames+=$lib=$lib.so\$($lib.so-version)";;\
     *)      echo "$lib.so-version=$number"; \
             echo "all-sonames+=$lib=\$($lib.so-version)";;\
   esac; \
 done; \
 echo soversions.mk-done = t;) < /tmp/nix-build-glibc-locales-2.23.drv-0/build/soversions.i > /tmp/nix-build-glibc-locales-2.23.drv-0/build/soversions.mkT; exit 0
mv -f /tmp/nix-build-glibc-locales-2.23.drv-0/build/soversions.mkT /tmp/nix-build-glibc-locales-2.23.drv-0/build/soversions.mk
gawk -v buildlist="iconvprogs iconvdata ldconfig lddlibc4 libmemusage libSegFault libpcprofile librpcsvc locale_programs memusagestat nonlib nscd extramodules libnldbl" -f scripts/gen-libc-modules.awk /tmp/nix-build-glibc-locales-2.23.drv-0/build/soversions.i > /tmp/nix-build-glibc-locales-2.23.drv-0/build/libc-modules.T
/bin/sh scripts/move-if-change /tmp/nix-build-glibc-locales-2.23.drv-0/build/libc-modules.T /tmp/nix-build-glibc-locales-2.23.drv-0/build/libc-modules.h
touch /tmp/nix-build-glibc-locales-2.23.drv-0/build/libc-modules.stmp
for dir in /tmp/nix-build-glibc-locales-2.23.drv-0/build sysdeps/unix/sysv/linux/x86_64/64 sysdeps/unix/sysv/linux/x86_64 sysdeps/unix/sysv/linux/x86 sysdeps/unix/sysv/linux/wordsize-64 sysdeps/x86_64/nptl sysdeps/unix/sysv/linux sysdeps/nptl sysdeps/pthread sysdeps/gnu sysdeps/unix/inet sysdeps/unix/sysv sysdeps/unix/x86_64 sysdeps/unix sysdeps/posix sysdeps/x86_64/64 sysdeps/x86_64/fpu/multiarch sysdeps/x86_64/fpu sysdeps/x86/fpu sysdeps/x86_64/multiarch sysdeps/x86_64 sysdeps/x86 sysdeps/ieee754/ldbl-96 sysdeps/ieee754/dbl-64/wordsize-64 sysdeps/ieee754/dbl-64 sysdeps/ieee754/flt-32 sysdeps/wordsize-64 sysdeps/ieee754 sysdeps/generic ; do \
  test -f $dir/syscalls.list && \
  { sysdirs='sysdeps/unix/sysv/linux/x86_64/64 sysdeps/unix/sysv/linux/x86_64 sysdeps/unix/sysv/linux/x86 sysdeps/unix/sysv/linux/wordsize-64 sysdeps/x86_64/nptl sysdeps/unix/sysv/linux sysdeps/nptl sysdeps/pthread sysdeps/gnu sysdeps/unix/inet sysdeps/unix/sysv sysdeps/unix/x86_64 sysdeps/unix sysdeps/posix sysdeps/x86_64/64 sysdeps/x86_64/fpu/multiarch sysdeps/x86_64/fpu sysdeps/x86/fpu sysdeps/x86_64/multiarch sysdeps/x86_64 sysdeps/x86 sysdeps/ieee754/ldbl-96 sysdeps/ieee754/dbl-64/wordsize-64 sysdeps/ieee754/dbl-64 sysdeps/ieee754/flt-32 sysdeps/wordsize-64 sysdeps/ieee754 sysdeps/generic' \
    asm_CPP='gcc -c     -Iinclude   -I/tmp/nix-build-glibc-locales-2.23.drv-0/build  -Isysdeps/unix/sysv/linux/x86_64/64  -Isysdeps/unix/sysv/linux/x86_64  -Isysdeps/unix/sysv/linux/x86  -Isysdeps/unix/sysv/linux/wordsize-64  -Isysdeps/x86_64/nptl  -Isysdeps/unix/sysv/linux/include -Isysdeps/unix/sysv/linux  -Isysdeps/nptl  -Isysdeps/pthread  -Isysdeps/gnu  -Isysdeps/unix/inet  -Isysdeps/unix/sysv  -Isysdeps/unix/x86_64  -Isysdeps/unix  -Isysdeps/posix  -Isysdeps/x86_64/64  -Isysdeps/x86_64/fpu/multiarch  -Isysdeps/x86_64/fpu  -Isysdeps/x86/fpu/include -Isysdeps/x86/fpu  -Isysdeps/x86_64/multiarch  -Isysdeps/x86_64  -Isysdeps/x86  -Isysdeps/ieee754/ldbl-96  -Isysdeps/ieee754/dbl-64/wordsize-64  -Isysdeps/ieee754/dbl-64  -Isysdeps/ieee754/flt-32  -Isysdeps/wordsize-64  -Isysdeps/ieee754  -Isysdeps/generic   -Ilibio -I. -nostdinc -isystem /scratch/gopinatr/nix/store/pc3y1mmd5547k7ri0hqhcm5f3w24z9zh-gcc-5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0/include -isystem /scratch/gopinatr/nix/store/pc3y1mmd5547k7ri0hqhcm5f3w24z9zh-gcc-5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0/include-fixed -isystem no  -D_LIBC_REENTRANT -include /tmp/nix-build-glibc-locales-2.23.drv-0/build/libc-modules.h -DMODULE_NAME=libc -include include/libc-symbols.h       -DASSEMBLER  -g -Werror=undef -Wa,--noexecstack   -E -x assembler-with-cpp' \
    /bin/sh sysdeps/unix/make-syscalls.sh $dir || exit 1; }; \
  test $dir = sysdeps/unix && break; \
done > /tmp/nix-build-glibc-locales-2.23.drv-0/build/sysd-syscallsT
mv -f /tmp/nix-build-glibc-locales-2.23.drv-0/build/sysd-syscallsT /tmp/nix-build-glibc-locales-2.23.drv-0/build/sysd-syscalls
make  -C localedata install-locales
make[2]: Entering directory '/tmp/nix-build-glibc-locales-2.23.drv-0/glibc-2.23/localedata'
.././scripts/mkinstalldirs /scratch/gopinatr/nix/store/j84kwchcycbsl8yr23g64z51zsdf3c3w-glibc-locales-2.23/lib/locale
mkdir -p -- /scratch/gopinatr/nix/store/j84kwchcycbsl8yr23g64z51zsdf3c3w-glibc-locales-2.23/lib/locale
aa_DJ.UTF-8...cannot create temporary file: /tmp/nix-build-glibc-locales-2.23.drv-0/scratch/gopinatr/nix/store/0wfgakp0ssbkr57jdaa23a55s9v6ldf2-glibc-2.23/lib/locale/locale-archive.eeZI74: No such file or directory
Makefile:205: recipe for target 'install-aa_DJ.UTF-8/UTF-8' failed
make[2]: *** [install-aa_DJ.UTF-8/UTF-8] Error 1
make[2]: Leaving directory '/tmp/nix-build-glibc-locales-2.23.drv-0/glibc-2.23/localedata'
Makefile:421: recipe for target 'localedata/install-locales' failed
make[1]: *** [localedata/install-locales] Error 2
make[1]: Leaving directory '/tmp/nix-build-glibc-locales-2.23.drv-0/glibc-2.23'
Makefile:9: recipe for target 'localedata/install-locales' failed
make: *** [localedata/install-locales] Error 2

This seems specific to a prefixed install /tmp/nix-build-glibc-locales-2.23.drv-0/scratch/gopinatr/nix/store/0wfgakp0ssbkr57jdaa23a55s9v6ldf2-glibc-2.23/lib/locale/locale-archive.eeZI74: No such file or directory Makefile:205: recipe for target 'install-aa_DJ.UTF-8/UTF-8' failed

@vrthra Looks like a bug in glibc package. You should submit an issue for that package. I haven't tried a prefixed nix install so I'm not sure if I can help with this.

Adding a solution posted at #554 here so that it will gain visibility.

I noticed that not all packages require the libnss_sss plugin. Only a few packages do (git, redo, python, etc) that uses getpwuid. So why not create a second package glibc-with-plugins with a copy of the glibc content and update it with libnss_sss (and any other plugins)? Then the few packages that require libnss_sss plugin can depend on the glibc-with-plugins instead of standard glibc, and we wont cause the world to be rebuilt. More importantly, whether to depend on glibc or glibc-with-plugins can be tackled on a package by package basis

Would this be acceptable?

I would really like to get sssd into NixOS, I need it for FreeIPA support. What needs to be done to to finish this? Can I help somehow?

So there seems to be no way around patching glibc, and skimming the code, it does generally not look too complicated to do. The open problem is how to make things work on non-NixOS systems. A possible issue here is that it should be made sure that user-supplied nss modules do not get loaded by setuid programs. Therefore, either

• loading nss modules adheres to the secure-execution mode described in ld.so(8), which might break setuid wrappers on NixOS, or
• the module search path in /nix/var is made root-writeable, which would be no real advantage over using /run/nss-modules, or
• a combination of both, like having a per-user plugin path that respects secure-execution, plus a system-wide root-only path.

One could also abuse the fact that there are usually (IIUC) no setuid binaries using Nix-installed libraries on non-NixOS systems, and make the plugin path root-writeable only on NixOS systems, but that somehow feels like a dirty solution.

Moreover, the problem is not really relevant for per-user Nix installations, since then you won't have setuid binaries linking to your stuff unless you already have some way to become root, so the question is in how many cases these issues actually occur. To put it differently: why would there be a system-wide Nix installation, but admins not willing to add some additional logic to set up the proper site-specific nss modules (if only we came up with a sufficiently simple way to do that)?

Also, none of this addresses possible versioning issues. A simple way might be to just have multiple plugin directories, like /run/nss-modules/<unique-id-of-underlying-glibc>.

I rebased the original changes onto channels/nixos-16.09, fixed all conflicts and a minor issue with libxml2's header files. Should I be creating a new pull request so we can proceed with this? The branch is located at https://github.com/e-user/nixpkgs/commits/sssd-1.13.3.

So next would be patching glibc to look for NSS modules in /run/nss-modules/<own-unique-id>/..?

Update: I was able to produce a patch that makes glibc look for NSS modules in a specific additional location first, but only if AT_SECURE is not set (by checking __libc_enable_secure). I tested this by running getent vs. su.
I'll figure out how to create per-user directories in /run/nss-modules and update the patch accordingly, next.

For getent passwd $USER, my version of glibc first attempts to open

/run/nss-modules/4iszg1mhyd5pimxm06x28xyq1nv60swr-glibc-2.24/per-user/1000/libnss_files.so.2
/run/nss-modules/4iszg1mhyd5pimxm06x28xyq1nv60swr-glibc-2.24/system/libnss_files.so.2

If getent has the setuid flag set, it only looks for

/run/nss-modules/4iszg1mhyd5pimxm06x28xyq1nv60swr-glibc-2.24/system/libnss_files.so.2

4iszg1mhyd5pimxm06x28xyq1nv60swr-glibc-2.24 is identical with the store path of the same version of glibc built. Since resolving the user name of the calling process would cause infinite recursion, I'm only using getuid () for the per-user paths.
I had to switch to using getauxval (AT_SECURE) as __libc_enable_secure does only seem to get set under different circumstances.

Next up is making SSSD install its module into the new location and testing it. I'm still wondering whether I should create a new pull request..?

@e-user I think it would be helpful to create new pull request with your changes.

deprecated by #21150