I'm not going to try to block this, but IMO things failing when they use /bin/sh is a good thing.

But yes, if we're going to try to keep /bin/sh supported there's no reason not to just use bash.

@shlevy I think your approach of having no /bin/sh should be eventually the solution, but takes more time. We would need a hydra jobset, a long checklist and a army of contributors for that. Nothing we should pull of before a release.

With this change, could it still happen that a package builds in the sandbox on my machine but sometimes fails on Hydra? Whatever solution we choose should produce consistent results.

@xeji Yes, it could still happen. Your machine could have nix built with a different version of nixpkgs, or outside of nixpkgs altogether, or hydra's could. But it's less likely with this fix.

@Mic92 If we get a commitment to move to a no-bin-sh world, I'll gladly stand behind this as an interim solution 😀 I already had patches to glibc etc...

Of course, killing /bin/sh is also a cat-and-mouse, at some point we will find some programming languages have standard library assuming existence of both /bin/sh and /usr/bin/env…

Should we put patchShebangs over source into default unpack phase? I am not sure what is a proper fix for system() given setuid binaries (and relative ease of proper escaping, so a setuid binary using system() might not be completely broken yet).

@7c6f434c system and popen are already in the "don't use in setuid binaries" list IMO 😄 .

By the way, I consider /usr/bin/env simply a userspace patch for the kernel, which should obviously support PATH lookup in shebangs. IMO the proper end-goal, without modifying the kernel, would be sandboxes with no /bin or /usr, but NixOS having statically linked /usr/bin/env that a) verifies it was called from a shebang script and b) simply calls execvp

@7c6f434c how about using sh ./configure instead stdenv?
Mhm. sometimes systems are not using sh in configure. Bad idea.
Maybe patchShebangs ./configure as a starter?

@Mic92 sometimes the builds use more scripts; so maybe better safe than annoyed?

@shlevy We-ell, I can answer that yes, fusermount, fusermount3, su and unix_chkpwd do not use system(), and there is no reason to have any other SUID binaries. In general, though, Nix is a system to beat reason into bad software, assuming all software can be made completely good is a bit too optimistic — and, after all, system() is not that hard to use safely.

Re: kernel: will binfmt_misc handler on #! work? If yes, it is not kernel's job to support PATH lookup, as it is obviously a policy decision.

What’s the problem with supporting both /bin/sh (the POSIX-compliant shell) and /bin/bash (extended featureset) by default? By just linking from sh to bash, which changes its feature-set according to name.

@Profpatsch Even more impurity. We've already had real issues stemming from the existence of /bin/sh in and out of sandbox...

+1 this (or whatever) as stop gap. and no /bin impurities for 18.09.

If there is going to be a /bin/sh it needs to be as POSIX-compliant as we can make it-- for the sake of "purity"... even if that's an iterative process that breaks things along the way. I have no idea how close our busybox shell is to that goal, or how close this replacement is. But for example I don't think we want to promise /bin/sh is a particular bash version or something.

Are there any known problems with our current /bin/sh? AFAIK all problems these days are from folks using machines/builders with a copy of Nix that didn't have the /bin/sh fixes.

This PR doesn't make migration pains go away, but I believe is intended to prevent/limit such pain in the future should we find something lacking in /bin/sh.

Regarding static linking bash, two quick thoughts:

• Patch here fixes static linking w/o bash-malloc: https://lists.gnu.org/archive/html/bug-bash/2016-10/msg00006.html (I've used this myself quite a bit, although note it does require an autoreconfHook or so)
• How about...using musl?! 😁

Bash config:

• IIRC bash has some sort of "--be-more-posix" flag? Just checked, yep it's --enable-strict-posix-default. Don't have experience with it but at least sounds like something we want :).
• --enable-minimal-config a minimal sh-like configuration seems interesting
• IMHO consider also disabling the heck out of bash's dlopen-to-provide-new-builtins functionality. Adding these to your configureFlags are a start IIRC:

      "ac_cv_func_dlopen=no"
      "ac_cv_func_dlclose=no"
      "ac_cv_func_dlsym=no"

I'm sorry about the busybox-specific attribute name, I did comment about it in my PR 😇 although I didn't come up with a better one :).
The idea behind putting that in nixpkgs was to make it such that building nixpkgs with a Nix built from that Nixpkgs will work. There is a bit of a bootstrap problem but I don't think that will be a problem in practice. AFAICT all recent problems have been due to violations of this, and perhaps fixing that should be the priority instead?

Any news on that? Apparently busybox sh is also not fully posix-compliant: #37082 (comment)

Thanks for the many comments, all.

So the intention of this was to restore the previous behaviour that has been in place for the past few NixOS releases using Nix <= 1.11. That is, the proposals of having per-drv /bin/sh or removing /bin/sh totally from the sandbox are out of scope here.

Are there any known problems with our current /bin/sh? AFAIK all problems these days are from folks using machines/builders with a copy of Nix that didn't have the /bin/sh fixes.

Actually you are right, I checked again the two PRs that did some patchShebangs changes and they were for $(( and command, so there were still some Hydra builders using the old versions, sigh... At least the Linux build slaves seem to be now fixed.

Still, I conceptually dislike using the busybox shell here. For one it's Linux-only. So if we ever grow a way to have a non-host /bin/sh in e.g. Darwin or even other Unices we'd be forced to have a different /bin/sh implementation there. And from a purity point of view I prefer a bug-for-bug / unspecified-behaviour compatible /bin/sh so the chance of old Nixpkgs versions from stopping working is as small as possible.

If there is going to be a /bin/sh it needs to be as POSIX-compliant as we can make it-- for the sake of "purity"... even if that's an iterative process that breaks things along the way.

I personally don't see a reason why, given that Bash is open-source and ported to practically everywhere. And again in the current model of /bin/sh, it's also about potentially breaking existing .drvs.

I will look into the configure/compile flags more tomorrow, thanks for the tips.

What is the status of this pull request?

Closing due to lack of activity, feel free re-open this if needed.