Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

profiles/hardened: Do not limit user namespaces if Nix's sandbox builds are enabled #48207

Closed
wants to merge 1 commit into from

Conversation

DIzFer
Copy link
Contributor

@DIzFer DIzFer commented Oct 11, 2018

Motivation for this change

This is one of those small niche fixes I've been too lazy to commit for a while...

Nix apparently uses user namespaces when configured to build in a sandbox. This causes it to error out with an ENOSPC whenever you try to build after enabling the hardened profile, which cannot be circumvented in any other way than setting the user namespace count to a number high enough for the build to finish.

This commit gives priority to the nix.useSandbox setting: If it is enabled, it won't change the user namespace limit (so as to let it be whatever the kernel default is).

As I said, I've been sitting on this for a while, and I am unaware if this has been fixed some other way. I can't seem to find any issues, open or closed, related to this particular symptom.

An alternative fix I can think of is to have the nix-daemon (which AFAICS runs as root) or nixos-rebuild (which definitely does) to raise the limit itself before attempting a rebuild. This would require a way to tell how much does the number need to be raised (since setting it to 1 will let the first rebuild happen, but not the next one). I assume there is some way of getting the necessary info and implementing it, I've just not looked at it further.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@DIzFer DIzFer changed the title Do not limit user namespaces if Nix's sandbox builds are enabled profiles/hardened: Do not limit user namespaces if Nix's sandbox builds are enabled Oct 11, 2018
@fpletz fpletz requested a review from joachifm October 11, 2018 13:50
fpletz
fpletz previously approved these changes Oct 11, 2018
Copy link
Member

@fpletz fpletz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either this or disable sandbox builds in the hardened profile. I'd prefer this solution though.

@joachifm
Copy link
Contributor

This is certainly more convenient. Implicitly enabling userns, however, somewhat goes against the idea of the hardened profile (at least as I think of it) which is to default to paranoid settings even if it breaks stuff. In that sense, the failures you're seeing are very much a feature and not a bug :)

My preference is to instead notify the user that these options are incompatible (and perhaps reify userns as an option in its own right, to not require knowledge of the sysctl knob). Doing so would enhance UX somewhat, by promoting the run-error to build-error at least.

Now, if greater convenience gets more users to run hardened that'd be great, which speaks in favour of this PR. I worry, though, that not everyone will realize that Nix sandboxing permits features that have historically been quite problematic from a security perspective.
(IMO, security sensitive machines should not be building stuff at all, sandbox or no, but this might be an idiosyncratic view ...)

Anyway, I'm happy to defer to user desires, these are just my 2c.

@joachifm
Copy link
Contributor

joachifm commented Oct 11, 2018

If we had unprivileged_userns_clone that might be a reasonable compromise, but iirc that is yet to be upstreamed (if it ever will be).
EDIT: that is, if Nix does not depend on unprivileged userns specifically.

@DIzFer
Copy link
Contributor Author

DIzFer commented Oct 11, 2018

Hmm, regarding the first point, I guess printing an error pointing the user to either disable sandbox builds or re-enable userns would be sort-of acceptable. Except I'm pretty sure the sandbox builds are the default now, so then a vanilla config in which you only enable the hardened profile errors out... Which is not good user experience IMO.

I'm thinking, what about keeping the "if sandbox = true then userns stays enabled" but then also print a warning (instead of an error) telling the user to disable sandbox or manually lower the userns limit? This way a build is not stopped, and the user still gets notified that the combination of these two options is probably not what they want.

I type all that because I haven't the slightest idea what you mean in your second message, so I don't really know if there's a better solution there :(

@fpletz fpletz dismissed their stale review October 11, 2018 23:10

An error or warning would indeed be better.

@fpletz
Copy link
Member

fpletz commented Oct 11, 2018

@DIzFer IMHO you could just add an assertion about the state of both options and print a message that either user namespaces should be enabled or nix sandboxing has to be disabled. It would probably be good to be a bit verbose about it and mention in the message which options to set.

@joachifm has a point here, implicitly enabling user namespaces might be a bad for current users of the hardened profile. For instance, if you use nixops to deploy your machines, the target machines won't ever have to run a nix build so sandboxing could've still been enabled without running into trouble. Enabling user namespaces for those users without notifying them is probably not a good idea and a warning can easily be overlooked. (Happened to me a few times.)

Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! It would be nice if Nix could gracefully downgrade to non-sandboxed in this case though.

@DIzFer
Copy link
Contributor Author

DIzFer commented Oct 14, 2018

I believe the point @fpletz makes about a warning that allows the build to continue is on point in this case. This is one of those circumstances in which user-friendliness is at the opposite end to security...

I think part of the issue at hand is that the discoverability of the "profiles" feature is poor. Had it a section in the manual, for example, or at least I couldn't find it in a couple of minutes. If you search for "hardened nixos" the first and only truly relevant result is the first one... a link to the source file.

If the discovery mechanism for the profiles were in a better place, we could just update the relevant section in the docs with a note about how these two are incompatible, and that for true hardening one has to disable sandboxing too. Add a paragraph to the release notes, and most users should be fine. In this scenario, silently and painlessly falling back to not disabling userns is fine IMO.

In our current situation, the only documentation appears to be in the comments in the hardening profile. A responsible user might read them, but I didn't, just saw "hardened" in the name, or saw it mentioned when grsec dissapeared, and blindly applied it to my system, ending with it broken.

Given that since 18.09 nix.useSandbox defaults to true, and the aforementioned lack of documentation outside comments, I propose making this actually stop the build and making the user aware that they must either disable nix sandboxing, or reenable userns explicitly, therefore avoiding making a decision with possibly dangerous security implications for a user that is worried enough to be enabling the hardened profile. I will work on this today.

After that, I would like to add a section to the documentation about profiles in general, and with a small summary about each one. But that's for another PR, and another unnecessarily long thinking-aloud session :)

@joachifm joachifm mentioned this pull request Oct 14, 2018
9 tasks
@joachifm
Copy link
Contributor

@DIzFer I'm closing this as the incompatibility is now a build error. I think adding docs for profiles is still worth pursuing, however. Please feel free to ping me if you want to work on that, I have a rough draft for a section on the hardened profile.

@joachifm joachifm closed this Oct 15, 2018
@DIzFer DIzFer deleted the fix-hardened-sandbox-builds branch October 16, 2018 08:56
@DIzFer
Copy link
Contributor Author

DIzFer commented Oct 16, 2018

Please do pass it along. I was familiarising myself with the manual build process as I type.

@joachifm
Copy link
Contributor

@DIzFer I've added a tracking issue for documenting profiles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants