glibc: Add an environment variable for specifying an NSS search path #111194

edolstra · 2021-01-29T19:35:41Z

With this, you can set $NIX_GLIBC_NSS_PATH to a colon-separated list of directories in which glibc will look for NSS modules. In environments where nscd cannot be used, this allows dealing with non-standard NSS modules in a less sledgehammer-y way than setting $LD_LIBRARY_PATH (which affects everything, and in particular could cause ABI incompatibilities).

The default value is

/nix/run/glibc-nss-path/${out-hash}:/nix/run/glibc-nss-path

where out-hash is the hash part of glibc.out (e.g. /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag). Thus,
glibc will look for an NSS module named libnss_mymachines.so.2 at

/nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag/libnss_mymachines.so.2

and

/nix/run/glibc-nss-path/libnss_mymachines.so.2

These are tried after the default search locations (i.e. $LD_LIBRARY_PATH and glibc.out/lib) so they don't override any current behaviour.

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Motivation for this change

This does for NSS modules what /run/opengl-driver does for our OpenGL support. In particular it's intended for non-NixOS, non-nscd environments that require non-standard NSS modules.

Maybe instead of NIX_GLIBC_NSS_PATH we could use a generic environment variable as suggested in https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585 (and then look for out-hash/nss and nss relative to the directories in that variable).

Things done

With this, you can set $NIX_GLIBC_NSS_PATH to a colon-separated list of directories in which glibc will look for NSS modules. In environments where nscd cannot be used, this allows dealing with non-standard NSS modules in a less sledgehammer-y way than setting $LD_LIBRARY_PATH (which affects everything, and in particular could cause ABI incompatibilities). The default value is /nix/run/glibc-nss-path/${out-hash}:/nix/run/glibc-nss-path where out-hash is the hash part of glibc.out (e.g. /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag). Thus, glibc will look for an NSS module named libnss_mymachines.so.2 at /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag/libnss_mymachines.so.2 and /nix/run/glibc-nss-path/libnss_mymachines.so.2 These are tried *after* the default search locations (i.e. $LD_LIBRARY_PATH and glibc.out/lib) so they don't override any current behaviour. The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

ghost · 2021-06-08T07:36:31Z

This would really help me, what is needed to make this happen?

ghost · 2021-06-09T06:43:59Z

I was able to make the google-oslogin test work without nscd with a version of this.

The default value is

/nix/run/glibc-nss-path/${out-hash}:/nix/run/glibc-nss-path

where out-hash is the hash part of glibc.out (e.g. /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag).

This creates problems for extra-utils, since it will be interpreted as a reference by Nix but nuke-refs can't remove it.

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Hmm, I think it might be better to keep it in line with /run/opengl-driver for now, and maybe change both over to /nix/run/... later. Is this even a problem non-NixOS systems?

r-burns · 2021-09-17T02:02:23Z

Looks like the patch doesn't apply anymore after glibc moved some stuff around: bminor/glibc@f8847d8.

Moving the fallback module loading logic after this line seems to work:
https://github.com/bminor/glibc/blob/f8847d83e17774ed5e9c0f75ef693680b91bcae4/nss/nss_module.c#L135

flokli · 2021-09-21T13:58:52Z

#138178 re-rolls parts of this, by loading nss modules from /run/nss-modules. I didn't add the complicated "multiple paths, separated by colons, read from an environment variable".

We can probably get along with a static set of paths to look in.

In particular it's intended for non-NixOS, non-nscd environments that require non-standard NSS modules.

I'd propose adding a second path in a followup to #138178, that looks in /usr/lib, which is where other distros put their NSS modules.

I don't think we should use /nix/run or anything like that on non-NixOS, as NSS modules are provided by the distribution, and already installed in /usr/lib.

Using that approach, nix-built binaries running on non-NixOS, without nscd installed will just use whatever NSS modules are configured by the other distro itself, without the need to also manually install them through Nix.

r-burns · 2021-09-21T17:19:20Z

I'd propose adding a second path in a followup to #138178, that looks in /usr/lib, which is where other distros put their NSS modules.

Not necessarily, they might be in e.g. /lib/x86_64-linux-gnu.

I would say to stick to your guns and just do /run/nss-modules. Whatever solution non-NixOS users are using for /run/opengl-drivers/lib would apply here, whether that's manually creating the directory and symlinking the distro's files there, or mounting it with nix-user-chroot.

SuperSandro2000 · 2021-09-21T18:23:26Z

I would say to stick to your guns and just do /run/nss-modules. Whatever solution non-NixOS users are using for /run/opengl-drivers/lib would apply here, whether that's manually creating the directory and symlinking the distro's files there, or mounting it with nix-user-chroot.

No should do be doing that. /run is a tmpfs and it is way easier to just use nixGL which collects env variables which point to the correct shared objects which in the end is similar to this PR.

Alternatively you can also just wrap the final binary and set the correct env variables. I am doing that to get mpv hardware acceleration working with home-manager on Debian.

Mic92 · 2021-09-22T16:30:53Z

pkgs/development/libraries/glibc/nss-path.patch

+		  size_t shlib_pathlen = (end - pos) + 1 + strlen (shlib_name) + 1;
+		  if (shlib_pathlen < sizeof(shlib_path))
+		    {
+		      __stpcpy (__stpcpy (__stpncpy (shlib_path, pos, end - pos), "/"), shlib_name);


Is memcpy not better here given that we already calulate the string length?

Alternative would be good old snprintf...

I'm just following the surrounding code here, didn't want to deviate from the coding style.

Mic92 · 2021-09-22T16:31:27Z

pkgs/development/libraries/glibc/nss-path.patch

+	      const char *end = __strchrnul(pos, ':');
+	      if (pos != end)
+		{
+		  char shlib_path[1024];


How about:

Suggested change

+ char shlib_path[1024];

+ char shlib_path[PATH_MAX];

Not that it PATH_MAX is not a lie..

r-burns · 2021-09-22T18:58:53Z

Okay, #138178 (comment) explains the per-glibc out-hash path very well. I noticed that the new self-reference in libc.so causes issues when building a NixOS VM. I think it could cause issues for bootstrap-tools as well. So I guess we need to either truncate/rot13 the hash to thwart Nix's dependency detection, or nuke the hash in places like this.

tomberek · 2021-10-07T08:09:12Z

Maybe instead of NIX_GLIBC_NSS_PATH we could use a generic environment variable as suggested in https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585 (and then look for out-hash/nss and nss relative to the directories in that variable).

As this need keeps coming up, it makes sense to have a single well-understood escape hatch that is common for each use-case. I'd rather avoid the proliferation of NIX_SOMETHING_PATH env vars for each thing. A single NIX_PIVOT_PATH=/some/path with the ${out-hash} mechanism that can be utilized as a search path for locales, opengl, nss, certificates seems good. A system admin on non-NixOS can add this to /etc/environment as needed. The tooling to create these dirs is a good barrier to limit its accidental use. (ref: https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585)

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Random thought about the default value: With the generic pivot approach one can imagine a default re-using the specialisation mechanism at /nix/var/nix/profiles/system/specialisation/HASH (pivots sound like a specialisation for a specific package). A bit less hacky is to use a new directory in that profile at /nix/var/nix/profiles/system/pivot/HASH. So NixOS can declare these if desired, or can do ln -s /run $out/pivot/HASH. Non-NixOS can create this special system profile and subdirs using any profile generation tooling with Nix, or it can be manually symlink'd in /nix directory by adventurous sysadmins. Example for NixOS:

  system.extraSystemBuilderCmds = with builtins; let
    glibcPath = elemAt (split "-" (baseNameOf pkgs.glibc.outPath)) 0;
  in ''
      mkdir $out/pivot
      ln -s ${pkgs.nss_ldap}/lib $out/pivot/${glibcPath}
    '';
};

This would have the default for NIX_PIVOT_PATH be /nix/var/nix/profiles/system/pivot.

Security considerations?

This overall issue may need an RFC?

infinisil · 2021-10-12T16:21:48Z

Just a small note, I think a better name for such an env var is NIX_IMPURE_RUNTIME_DIR or something along those lines.

stale · 2022-04-16T03:15:50Z

I marked this as stale due to inactivity. → More info

flokli · 2022-11-18T17:23:49Z

Just a small note, I think a better name for such an env var is NIX_IMPURE_RUNTIME_DIR or something along those lines.

Thie explicitly only affects NSS module loading. I don't think that env var name is very helpful in conveying the message.

flokli · 2022-11-18T17:25:41Z

As written in https://flokli.de/posts/2022-11-18-nsncd/, I still think we should get something around the lines of this patch into glibc, either here in nixpkgs, or upstream.

Izorkin · 2023-01-04T20:30:47Z

There is some progress?

edolstra marked this pull request as draft January 29, 2021 19:36

ofborg bot added 6.topic: nixos 8.has: module (update) 10.rebuild-linux-stdenv labels Jan 29, 2021

ofborg bot requested a review from Ma27 January 29, 2021 20:44

ofborg bot added 11.by: package-maintainer 10.rebuild-darwin: 11-100 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Jan 29, 2021

edolstra mentioned this pull request May 26, 2021

glibc should be patched to look for /etc/protocols in /nix/store #124401

Closed

r-burns mentioned this pull request Sep 17, 2021

Provide NSS modules globally, make nscd unnecessary #138178

Closed

12 tasks

bjornfor mentioned this pull request Sep 19, 2021

phoronix-test-suite: run missing hooks: preInstall, postInstall #138444

Merged

12 tasks

Mic92 reviewed Sep 22, 2021

View reviewed changes

flokli mentioned this pull request Jan 26, 2022

Provide NSS modules globally, make nscd unnecessary (v2) #155655

Closed

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Apr 16, 2022

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 18, 2022

bjornfor mentioned this pull request Nov 24, 2022

Username lookup (getpwuid) fails when LDAP or Systemd Homed are used NixOS/nix#7341

Open

Izorkin mentioned this pull request Dec 24, 2022

glibc: allow load libnss tcb module #207547

Closed

13 tasks

bjornfor mentioned this pull request Apr 26, 2023

Nix NSS name service (libnss_8.so) is incompatible with Debian/Ubuntu #229486

Closed

bjornfor mentioned this pull request May 21, 2023

Missing libnss_sss.so.2 where needed. #230110

Open

This was referenced Jun 22, 2023

timesyncd: disable NSCD when DNSSEC validation is disabled systemd/systemd#28110

Closed

systemd: disable NSCD when DNSSEC validation is disabled in timesyncd #239201

Merged

wegank added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: merge conflict labels Mar 19, 2024

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

glibc: Add an environment variable for specifying an NSS search path #111194

glibc: Add an environment variable for specifying an NSS search path #111194

edolstra commented Jan 29, 2021 •

edited

ghost commented Jun 8, 2021

ghost commented Jun 9, 2021

r-burns commented Sep 17, 2021

flokli commented Sep 21, 2021

r-burns commented Sep 21, 2021

SuperSandro2000 commented Sep 21, 2021 •

edited

Mic92 Sep 22, 2021

Mic92 Sep 22, 2021

edolstra Oct 5, 2021

Mic92 Sep 22, 2021

r-burns commented Sep 22, 2021 •

edited

tomberek commented Oct 7, 2021

infinisil commented Oct 12, 2021

stale bot commented Apr 16, 2022

flokli commented Nov 18, 2022

flokli commented Nov 18, 2022

Izorkin commented Jan 4, 2023

glibc: Add an environment variable for specifying an NSS search path #111194

Are you sure you want to change the base?

glibc: Add an environment variable for specifying an NSS search path #111194

Conversation

edolstra commented Jan 29, 2021 • edited

Motivation for this change

Things done

ghost commented Jun 8, 2021

ghost commented Jun 9, 2021

r-burns commented Sep 17, 2021

flokli commented Sep 21, 2021

r-burns commented Sep 21, 2021

SuperSandro2000 commented Sep 21, 2021 • edited

Mic92 Sep 22, 2021

Choose a reason for hiding this comment

Mic92 Sep 22, 2021

Choose a reason for hiding this comment

edolstra Oct 5, 2021

Choose a reason for hiding this comment

Mic92 Sep 22, 2021

Choose a reason for hiding this comment

r-burns commented Sep 22, 2021 • edited

tomberek commented Oct 7, 2021

infinisil commented Oct 12, 2021

stale bot commented Apr 16, 2022

flokli commented Nov 18, 2022

flokli commented Nov 18, 2022

Izorkin commented Jan 4, 2023

edolstra commented Jan 29, 2021 •

edited

SuperSandro2000 commented Sep 21, 2021 •

edited

r-burns commented Sep 22, 2021 •

edited