Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve RPM and DEB packages for multi-user Nix #1141

Closed
wants to merge 15 commits into from

Conversation

abbradar
Copy link
Member

@abbradar abbradar commented Nov 28, 2016

This is an effort to make installation of Nix on Debian and Red Hat-based distributions much easier experience. As an end result, you can now install .deb or .rpm built by Hydra and have store created, a minimal nix.conf, build users for Nix daemon created and other changes so you can just enable/start nix-daemon, relogin (to apply a profile script) and use multi-user Nix. Also this fixes package dependencies and makes other improvements.

One of the bigger changes is move from checkinstall to full-featured Debian packaging. Checkinstall has several limitations that are not nice to work with, particularly its configuration files detection and inability to detect /nix/store directory creation. Debian packaging is much more complex than I thought, but this gives us ability to fully control all aspects of package.

Also there is a general change that affects overall Nix -- I've replaced underscore in Nix unstable version with dot. This is because Debian explicitly forbids usage of underscores in versions. I'm not sure if this is a good idea -- thoughts?

This was tested in both Docker containers (it was my first encounter with it and it appears to be very useful for such development) and full-fledged VMs (for daemon testing) for Fedora 24 and Ubuntu 16.04. Test plan:

  1. Install a package;
  2. Enable/start daemon (distro-specific, for systemd: systemctl enable nix-daemon; systemctl start nix-daemon);
  3. Re-login;
  4. Multi-user Nix should work.

@abbradar
Copy link
Member Author

You also need NixOS/nixpkgs#20767 for this to work.

Copy link
Member

@matthewbauer matthewbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually just working on something pretty similar to this. My main hope was to get the Nix installer to detect when you are root and automatically switch to that when available.

See https://github.com/matthewbauer/nix/tree/multi-user-install.

scripts/nix.conf Outdated
# Default Nix configuration file.
binary-caches = https://cache.nixos.org/
signed-binary-caches = *
binary-cache-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should build-users-group be set to nixbld? I think that's necessary for NIX_REMOTE=daemon to work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the default, so no need to set it explicitly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to install an empty configuration file, but it must not duplicate Nix's built-in defaults. So all the settings here should be removed.

Also, binary-caches has been renamed to substituters.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, fair enough -- let's remove it altogether then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the second thought I've left it with an advice to read nix.conf manpage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it may seem insane to do so from the perspective of an individual user, it's not uncommon for many enterprise environments to prefer mirroring and building everything from source in-house for security / auditing / reproducibility-without-external-internet, among other reasons. Of course, it's certainly feasible for them to maintain a patched fork of nix, but nevertheless it'd be nice if the cache options were entirely in the configuration file instead of hard-coded in C++. There also a few users who get confused when a nix.conf with 0 keys and 0 binary caches still substitutes from cache.nixos.org.

Just throwing this out there as food for thought; I'd imagine it's outside the scope of this particular PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is out of scope yeah, we can include nix.conf.example with commented options explaining what each does and what the defaults are.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file still needs to be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -1,6 +1,6 @@
if [ -n "$HOME" ] && [ -n "$USER" ]; then
__savedpath="$PATH"
export PATH=@coreutils@
export PATH=@coreutils@:/usr/bin:/bin
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be necessary. It will restore to __savedpath at the end.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we want coreutils to be accessible here, and autodetect its path in configure by searching for some of them. However, on Ubuntu part of coreutils is in /bin and part is in /usr/bin -- so we detect /bin and then fail to find some other binaries. This fallback solves the problem (but I'm open to better ways).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I thought coreutils was a Nix directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was (is) confused about this too. Can you say something in the commit message about why /bin etc. must be added to PATH?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bjornfor Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The better way would be to have configure look for the path of the other coreutils (any of them will do).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit stuck how to fix this completely -- in various places in code we use @coreutils@ to get their paths but on Debian they are semi-randomly distributed between /bin and /usr/bin. For example, what should coreutils in config.nix be equal to?

if [ "$USER" != "root" ] && [ -e "@localstatedir@/nix/daemon-socket/socket" ]; then
export NIX_REMOTE=daemon
else
export NIX_REMOTE=
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably should just get rid of the else branch. Sometimes you may want to have NIX_REMOTE set on root.

release.nix Outdated
make installcheck
'';

installPhase = ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nitpick, but this seems like this could be split into patch, install, and build phases. Can we maybe even move this logic back into "debBuild"?

Copy link
Member Author

@abbradar abbradar Nov 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to move this back, because debBuild uses completely separate infrastructure (checkinstall). It's like stdenv which assumes configure/make/make install -- we usually implement other ways by hooks or phase overrides like this one.

I'll separate phases, thanks!

release.nix Outdated

installPhase = ''
# Hack to set desired init system.
sed -i 's,--with-init-system=systemd,--with-init-system=${initSystem},g' debian/rules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could alternatively have a make rule in debian/rules like:

INIT_SYSTEM ?= systemd

Then we can just run export INIT_SYSTEM = initSystem here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better, thanks.

nix.spec.in Outdated
systemctl enable nix-daemon.socket nix-daemon.service
systemctl start nix-daemon.socket
%endif
%(cat debian/postinst)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make "postinst" more generic? Maybe put it into ./scripts/ as "install-groups-linux.sh.in" so we can use it elsewhere? (like in install-nix-from-closure.sh)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's split it, yes.


* Initial release.

-- Eelco Dolstra <eelco.dolstra@logicblox.com> Sun, 13 Dec 2009 00:11:46 +0300
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this date be the date of the initial release or the data of PACKAGE_VERSION release?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right -- I've used this date as a placeholder to test that it builds and then forgot about it.

@matthewbauer
Copy link
Member

Another question: do users of Nix still need to have . $HOME/.nix-profile/etc/profile.d/nix.sh in their profile or is there a way to get it setup for everyone?

@abbradar
Copy link
Member Author

Hm, I think this is not needed now -- /etc/profile.d/nix.sh does everything that's needed.

@abbradar
Copy link
Member Author

abbradar commented Dec 3, 2016

I'm not sure where to take date from. We can either set it to some pre-defined meaningful date, or somehow derive it from git commit information or wall clock (I don't like either of those because of nondeterminism/reliance on git)...

@abbradar abbradar force-pushed the stable branch 2 times, most recently from 27ce24a to 815c061 Compare December 7, 2016 08:54
@abbradar
Copy link
Member Author

abbradar commented Dec 7, 2016

I've settled for timestamp of commit b07ba22 as the datetime in changelog. Also resolved merge conflicts. This is ready for another round of review/merge.

@abbradar
Copy link
Member Author

@matthewbauer Thank you for the review!

FWIW we use it in our company for some time -- seem to work okay for CentOS (there are additional manual steps for sandboxing but this can prove difficult to automate without NixOS/nixpkgs#4998).

@abbradar
Copy link
Member Author

abbradar commented Jan 13, 2017

I've pushed new version which also fixes Debian packages while making things simpler and less hacky. Notice that this introduces new command to nix-store -- --print-store-dir (which, as the title says, prints the store dir -- usually /nix/store).

EDIT: by Debian I meant not DEB (Ubuntu already works) but Debian 8 specifically.

@abbradar
Copy link
Member Author

abbradar commented Feb 6, 2017

I've resolved conflicts again but I don't want to keep resolving them once a month while awaiting a review. I hope it's okay to just leave it as is -- if it comes out of sync again please ping me and I'll resolve it!

@abbradar
Copy link
Member Author

Ping?

@therealpxc
Copy link

How many more ought to examine this before it will be considered mergeable?

I can't offer that since I'm not a core contributor or whatever, but I'd like to try this thing out. It feels painful to even ask, with how long this has been sitting here, but would you mind resolving the current merge conflicts so that I can play with this, @abbradar ?

@abbradar
Copy link
Member Author

@therealpxc You are welcome; I've rebased this (added some new fixes, too). I haven't mostly tested this though (.deb builds fine but .rpm fails with a virtual machine memory error, which I assume is my host machine problem).

@therealpxc
Copy link

Thanks!

I'll see if I can get the RPM building on my machine at home today and let you know if I run into any issues. :-)

@therealpxc
Copy link

therealpxc commented Jun 22, 2017

So the RPM does indeed build. I've been unable to test it, however, because I don't have a Fedora 25 machine around. On openSUSE 42.2, the RPM is uninstallable because of the dependency on libcrypto.so.10. The machine I really need this for running RHEL 6 anyway, though (I don't know what point release).

Do you happen to know if a similar strategy to the one we're using to build Nix for Fedora 25 can be used for RHEL? Could you share the installation procedure for CentOS? You mentioned above that there were some quirks.

@abbradar
Copy link
Member Author

@therealpxc Actually we are using Nix in production on CentOS 7 servers (which would correspond to RHEL 7). For this you'd need to use stable Nix which doesn't use C++14. Hopefully you can also build it for RHEL6. I have a branch with my patches rebased atop Nix 1.11 for that; I can find it and publish for you later.

@therealpxc
Copy link

That would be wonderful. Thanks for the clarification on the state of things.

@therealpxc
Copy link

@abbradar I tried building based on this branch for RHEL6 (6.1 and 6.5) and it complained about missing libseccomp-devel. Is this due to the version of Nix being too new?

@abbradar
Copy link
Member Author

@therealpxc Sorry, I have forgotten about this and then had a vacation.

I've rebased my 1.11 branch to include new changes and tried to build it for CentOS 6.5. Unfortunately it doesn't have C++11-compliant compiler... It seems you'd need to build it by yourself on your target system using a backported GCC (I'm sure there are some prebuilt in the wild, not so sure about their authenticity).

FWIW here is my branch: https://github.com/abbradar/nix/tree/stable-1.11

@therealpxc
Copy link

Thanks for coming back to this! I was wondering when and how it would be polite to poke you about this. :-)

I'm already using Nix on that CentOS system in a pivot_root environment, so I can probably even use that Nix installation, whose main defect is slowness, to get sufficiently recent GCC working and build it that way.

I'm excited to give this a try ASAP. I'd also like to try to get RPMs working for recent openSUSE and do some general testing for you on this PR to help nudge it forward.

Thanks again!

@abbradar
Copy link
Member Author

abbradar commented Feb 21, 2018

I've tested this in Ubuntu 17.10, Fedora 27 and Debian 9 VMs and it seems to work correctly completely with systemd unit. Pushed a Debian cleanup (extraDebRequires is no longer needed because runtime dependencies are specified in debian/control and partially derived by Debian package build system).

@dtzWill
Copy link
Member

dtzWill commented Feb 22, 2018

Super excited about this!

Still going through but quick thoughts:

@Mic92
Copy link
Member

Mic92 commented Feb 24, 2018

Today I tested this on debian:

$ nix-build release.nix -A deb_debian9x86_64 --arg nixpkgs ~/git/nixpkgs

Installation worked out, but build users are not added at the moment by the package.

groupadd -r nixbld
for n in $(seq 1 10); do useradd -c "Nix build user $n" \
    -d /var/empty -g nixbld -G nixbld -M -N -r -s "$(which nologin)" \
    nixbld$n; done

It's installed by default; also, NSS flavour doesn't support certificate
verification by default.
It's shown on Ubuntu in an error message on session start, as if something gone
wrong.
On Ubuntu coreutils is split between /bin and /usr/bin. Our ./configure script
then detects only part of this path, leaving some utils inaccessible. The most
simple fix is adding both of those paths as a fallback.
On Debian underscores in versions are prohibited.
This is to avoid installing unnecessary scripts.
checkinstall doesn't let us specify which files are actually configuration
files. Also, this allows for greater control of package fields.

This also adds installation hooks which add and remove build users and set
access rights. RPM .spec file now just includes them.
Add libsodium as a requirement and other build tools. Drop Perl dependencies.
This way bootstrap.sh fully handles bootstrapping.
Dist files shouldn't be cleaned by `make clean`.
C++14 macro is not supported everywhere and we hardcode needed flag anyway, so
let's just drop this.
Build with Brotli support on Fedora (where the library is available).
This is required by Nix daemon when testing.
@abbradar
Copy link
Member Author

abbradar commented Feb 24, 2018

I've rebased this against master again.

@dtzWill Sadly Brotli is available only on Fedora for now. As for needed binaries I'd prefer for #1345 to be finished first so that configure.ac scripts discover and use those tools.

@Mic92 That's strange, I have it working. Do you have NixOS/nixpkgs#21531 applied in your fork?

@Mic92
Copy link
Member

Mic92 commented Feb 26, 2018

@abbradar no, just this pull request. I was supposed to get something else working, so I have not looked into the root cause yet.

@abbradar
Copy link
Member Author

abbradar commented Mar 6, 2018

@edolstra anything else with this? I so much hope I don't have to rebase it for another several months ~_~

@dezgeg
Copy link
Contributor

dezgeg commented Apr 19, 2018

I think this approach of using rpmbuild and dpkg-buildpackage / debhelper sounds maintenance-heavy since we now have three build scripts to keep in sync. So I've created an alternative approach which uses the Nix-build version of Nix packaged with fpm: https://github.com/dezgeg/nix-fpm-multiuser

No pull request just yet but I guess the ML post works for any comments on that approach: https://groups.google.com/forum/#!topic/nix-devel/kIxMWOwX4m8

@Mic92
Copy link
Member

Mic92 commented Apr 26, 2018

@dezgeg I move the discussion out of this pull request here: dezgeg/nix-fpm-multiuser#3

@abbradar
Copy link
Member Author

abbradar commented Aug 9, 2019

I like fpm-multiuser approach more and I've used it in production for some time. Let's close this.

@abbradar abbradar closed this Aug 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet