Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] nixos/lsb.nix: init #78798

Closed
wants to merge 3 commits into from
Closed

Conversation

matthewbauer
Copy link
Member

Add an LSB module to run binaries built for other distros. Two
sub-options are available:

  • enableDesktop: add extra desktop libraries like gtk & qt
  • support32Bit: support 32-bit binaries
Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Add an LSB module to run binaries built for other distros. Two
sub-options are available:

  - enableDesktop: add extra desktop libraries like gtk & qt
  - support32Bit: support 32-bit binaries
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-make-nixos-so-easy-that-people-can-be-productive-up-front-without-having-to-first-learn-the-nix-language/5625/20

@matthewbauer matthewbauer changed the title [wi[] nixos/lsb.nix: init [wip] nixos/lsb.nix: init Jan 30, 2020
@grahamc
Copy link
Member

grahamc commented Jan 30, 2020

Is every package here required by LSB? I think if there are any packages in this list which aren't defined as required by LSB, this module is doomed to become a "well this one might be useful..." megalist of software without a reasoned criteria for adding (or rejecting) something.

edit gosh, this module gives me a serious case of queasy stomach. I would much rather find a different way to support those users than a module which breaks NixOS's story around upgrades and version dependencies.

nixos/modules/config/lsb.nix Outdated Show resolved Hide resolved
};
in lib.mkIf config.environment.lsb.enable ({
environment.sessionVariables.LD_LIBRARY_PATH =
"${base-libs64}/lib${lib.optionalString config.environment.lsb.support32Bit ":${base-libs32}/lib"}";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this be a list so that it can be extended?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it should be a list. I forgot that was allowed in sessionVariables.

@infinisil
Copy link
Member

It's an interesting module and I can see how it can be useful for special cases. However this goes very much against the purity aspect of Nix. With such a module I can imagine such situations:

  • Somebody with that option enabled packages a downloaded binary for nixpkgs, but they don't realize it uses these impure paths because it just works, resulting in it not working for everybody else
  • Package repositories saying that people need to enable this option to have their package work on NixOS
  • Troubleshooters saying people should enable this option to "fix" it, resulting in less people helping to fix impurities

As a NixOS user, the fact that /lib doesn't even exist is a thing I can say with pride, going against the global packages on almost every other distro. But with this option it kind of feels like giving up because it's not worth the effort to patch stuff. Next thing somebody adds to this module is a symlink /{,usr/}bin -> /run/current-system/sw/bin, and who knows what follows from that. I don't think going in this direction is a good idea.

Instead we should continue to push upstreams to not hardcode their paths, to make things configurable, maybe we can auto-patch things or use things like buildFHSUserEnv or bubblewrap to rewrite paths. That's the direction I'd like to go in.

So yeah I'm against having such a module in NixOS, but I think it would fit nicely into NUR

@zimbatm
Copy link
Member

zimbatm commented Jan 30, 2020

So to recap the current objection, what is happening right now is that we have an install base of NixOS users that run programs and discover missing runtime dependencies by using those packages. This only works when the runtime dependencies are sourced from absolute paths, if they are fetched from PATH we rely on an even smaller subset of NixOS users that don't have the runtime dependency installed. The whole discovery process is quite asynchronous and means we get regressions in nixpkgs. It's a strategy that works but it much less compelling than what we have for build dependencies where the sandbox basically guarantees that all dependencies are mapped and is fixed synchronously.

On the other side we have users whose system usability is impaired because they can't just download binaries from the Internet like they would be able to on any other distro. Fixing those issues rarely an easy task, which means that they basically need to be nix experts to use NixOS. Binaries can be shipped in many forms. As a developer I might get them in a JAR file, npm package, python wheel, ... We could solve a lot of that pain with that NixOS module.

I don't see those two things as necessarily mutually exclusive. What we really need is to find a better solution to map missing runtime dependencies so that they get discovered earlier.

@edolstra
Copy link
Member

Closing this for the reasons @infinisil stated. It's fundamentally incompatible with the NixOS approach. If we bring back /lib etc., then what really is the point of running NixOS rather than a conventional Linux distribution? I realize this module can be very useful as a quick fix for getting 3rd party binaries to work, but it should be provided in NUR or as a flake. If it's in the NixOS base distribution it implies that this is an endorsed mechanism which it really shouldn't be.

Also, the option enableDesktop ("add extra desktop libraries like gtk & qt") is extremely open-ended: there is no way that this module would not quickly acquire dozens of other options to manage the set of impure packages.

@edolstra edolstra closed this Jan 30, 2020
@matthewbauer
Copy link
Member Author

I'm a little bit surprised by how strong the objection is. It's opt-in and undoing it is only a simple unset LD_LIBRARY_PATH away. The difference between setting LD_LIBRARY_PATH and PATH is no different - they both introduce runtime impurities and allow what works on one machine to not work on another. If you really don't like this, you should also really not like all of the commands that NixOS provides in PATH by default (with no option for removal even!). At least with this module it is pretty standardized what you get and binary distributors can actually predictably get things working. PATH ends up being a mess of random binaries.

I definitely get the objection though, and at least agree that it shouldn't go in NixOS. I think it does makes sense to maintain this out of tree, which is actually pretty easy. enableDesktop probably should be rethought though, and maybe removed altogether. It was the minimum necessary libraries for me to get Electron + Dwarf Fortress working, but ended up being a huge number of packages, some of which really should be distributed with those libraries.

@jtojnar
Copy link
Contributor

jtojnar commented Jan 30, 2020

LD_LIBRARY_PATH is much worse than PATH since it overrules the libraries applications were linked against (through DT_RUNPATH ELF header). And we do criticize setting global PATH for dependencies: wrapper or, better, patch is preferred for that whenever possible.

@matthewbauer
Copy link
Member Author

matthewbauer commented Jan 30, 2020

LD_LIBRARY_PATH is much worse than PATH since it overrules the libraries applications were linked against (through DT_RUNPATH ELF header).

Yeah, maybe this isn't such a good idea :) I guess I need a LD_LIBRARY_PATH_AFTER for this to work like I want it to.

@danbst
Copy link
Contributor

danbst commented Feb 2, 2020

@edolstra

then what really is the point of running NixOS rather than a conventional Linux distribution?

I disagree. There is point in running NixOS even when lsb.enable option exists. We only have to enforce that PR contributor and PR reviewer both have lsb.enable = false && nix.sandbox = true;.

This option is perfectly fine for end-users as long as contributors have pure systems.

@danbst
Copy link
Contributor

danbst commented Feb 2, 2020

@matthewbauer I understand you won't continue work on this PR, but I'll throw in my suggestion.

I think list of packages included under enviornment.lsb.enable should be autogenerated from webpage, so it is clear it is not "arbitrary" or something is missing, or something is added ad-hoc.

Also, I'm waiting for options environment.ubuntu."19.10".enable, environment.debian.enable, environment.gentoo.enable which will masquerade NixOS as another distribution. environment.lsb.enable is first step here :)

And yes, it should be done more like in #69057

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/add-bin-bash-to-avoid-unnecessary-pain/5673/26

edolstra added a commit that referenced this pull request Feb 2, 2020
This reverts commit af665d8, see
#78798 (comment) for
the reasons in a similar PR.
@alyssais
Copy link
Member

alyssais commented Feb 2, 2020

I disagree. There is point in running NixOS even when lsb.enable option exists. We only have to enforce that PR contributor and PR reviewer both have lsb.enable = false && nix.sandbox = true;.

This option is perfectly fine for end-users as long as contributors have pure systems.

If you don’t want a pure system, there are plenty of other distributions that will serve your needs much better than NixOS.

Your suggestion means that in order for somebody to contribute to NixOS, they have to break installations of software that otherwise work for them. That’s going to be great for attracting new contributors! Part of the reason NixOS is so successful is that it’s so easy to go from being an end-user to a contributor.

Plus, if contributors aren’t allowed to use an option, who’s going to maintain it?

@danbst
Copy link
Contributor

danbst commented Feb 3, 2020

@alyssais I still like NixOS. On landing page https://nixos.org/nixos/ you don't see "purity" as an advantage. All of the "declarative", "reliable" and "devops-friendly" will still stay even with LSB/FHS stuff.

Disabling LSB by default is understandable, but forbidding it is too opinionated.

@alyssais
Copy link
Member

alyssais commented Feb 3, 2020

On landing page https://nixos.org/nixos/ you don't see "purity" as an advantage.

It literally says “The Purely Functional Linux distribution” and all the benefits listed are consequences of purity. It is not “too opinionated” to break purity when the whole point of the project is to create a pure system.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/add-bin-bash-to-avoid-unnecessary-pain/5673/29

@7c6f434c
Copy link
Member

7c6f434c commented Feb 4, 2020 via email

@7c6f434c
Copy link
Member

7c6f434c commented Feb 4, 2020 via email

@zimbatm
Copy link
Member

zimbatm commented Feb 4, 2020

This discussion reminds me a bit of Debian and Ubuntu where Debian took a principled approach, did a lot of amazing ground work. Then Ubuntu came along, added a graphical installer, added all the proprietary graphical drivers, picked a default desktop environment, created a great default user experience and became very popular because of it. I don't know if there are necessarily lessons to take from that except that NixBuntu will probably happen at some point. It's our choice if we want to make it part of the main project or have it as a (friendly) fork.

jpgu-epam pushed a commit to jpgu-epam/nixpkgs that referenced this pull request Feb 4, 2020
This reverts commit af665d8, see
NixOS#78798 (comment) for
the reasons in a similar PR.
@balsoft
Copy link
Member

balsoft commented May 5, 2020

For anyone interested, I've created https://github.com/balsoft/nixos-fhs-compat which implements this functionality as well as basic FHS support (yes, yes, linking stuff to /bin and /usr/bin etc). It's available as a flake and intended for use in nixos-containers and virtual machines specifically tailored for running stubborn executables and scripts without patchelfing them. I'm going to add a separate module that generates a container that does this and a simple command to run something in that container soon.

@matthewbauer
Copy link
Member Author

For anyone interested, I've created https://github.com/balsoft/nixos-fhs-compat which implements this functionality as well as basic FHS support (yes, yes, linking stuff to /bin and /usr/bin etc). It's available as a flake and intended for use in nixos-containers and virtual machines specifically tailored for running stubborn executables and scripts without patchelfing them. I'm going to add a separate module that generates a container that does this and a simple command to run something in that container soon.

Nice! I was meaning to make something like this.

The one piece that is still missing from all of this is a patch to glibc to add support for something like LD_FALLBACK_LIBRARY_PATH that is only used when a library is not in DT_RUNPATH.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-flakes-and-kiss/10602/6

@jkachmar
Copy link
Contributor

I'd just like to register my objection that this is the issue provided to justify the reversion of af665d8 (cf. 26aba55 commit message).

@JojOatXGME
Copy link

JojOatXGME commented Jul 17, 2021

I just made some research to find possible solutions for NixOS/nix-idea#30. I have to say that I am not quite satisfied with the argumentation for reverting config.environment.ld-linux (26aba55). I think there is one question: Do we want to allow executables to work if they are build outside Nix? If the answer is “no”, we should remove bin/sh and /usr/bin/env. Otherwise, we should support the most common interpreters for ELF files.

If I understand the ELF specification correctly, it is currently impossible to build an ELF executable outside the Nix store with dynamic linking that supports NixOS. (ELF does not support any dynamic resolution of the interpreter.) I guess you could even argue that this makes these interpreters even more important than /bin/sh.

Anyway, I agree that having these binaries at global paths is not ideal. I personally think that the concepts of nix-ld or envfs could be a good compromise. Both of them avoid the problem of /bin/sh, that it is always available (EDIT: in a specific version) regardless of the environment.

PS: Why does #69057 not exist anymore? I have found a bunch of references to this issue, which obviously do not work anymore.

@samueldr
Copy link
Member

samueldr commented Jul 17, 2021

PS: Why does #69057 not exist anymore? I have found a bunch of references to this issue, which obviously do not work anymore.

There are some users who got "disappeared" from GitHub. And when that happens project information gets disappeared too, which in turn means we're left with well gaslit holes in our history :(.

@infinisil
Copy link
Member

we should remove bin/sh and /usr/bin/env

I was going to argue that /usr/bin/env makes sense to stay because many people rely on that for script shebangs. But then I realized that such Nix-external shell scripts are no different than Nix-external binaries, but we allow the former while disallowing the latter.

@JojOatXGME
Copy link

we should remove bin/sh and /usr/bin/env

I was going to argue that /usr/bin/env makes sense to stay because many people rely on that for script shebangs. But then I realized that such Nix-external shell scripts are no different than Nix-external binaries, but we allow the former while disallowing the latter.

I guess you could argue that binaries are build by a compiler, while shell scripts are written by hand. If you develop a binary on your local machine, you don't have to set the interpreter manually. If you write a shell script, you have to manually specify the path to bash. So it is a bigger annoyance for scripts then for binaries if you develop them locally. 🤔

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-make-nixos-so-easy-that-people-can-be-productive-up-front-without-having-to-first-learn-the-nix-language/5625/51

@infinisil infinisil added the significant Novel ideas, large API changes, notable refactorings, issues with RFC potential, etc. label Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 significant Novel ideas, large API changes, notable refactorings, issues with RFC potential, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet