Multi user darwin installer #1453

grahamc · 2017-07-09T17:23:11Z

By Commit:

First commit makes iterating on the installers much faster, as it is able to cache the already built Nix. It includes a pattern I've begun to use, that if a dir contains a filed called .nix-ignore-dir, the directory is ignored. I can remove this commit if you don't like it.

Second commit runs shellcheck on the existing installer file, and fixes numerous concerns it brought up. It seems a kindness to make sure these scripts operate as we expect.

Third commit extends the default installer to call a separate, darwin-specific installer to install a multi-user Nix, with the nix-daemon. I have now tested this process dozens of times on four different macs of my own, @LnL7's laptop, @copumpkin, @imalsogreg, and I think some others too. @wmertens is about to try it. It seems to work very well. I'm tracking this progress here: https://gitlab.com/grahamc/mac-nix-multi-user/issues/2

I have built this and uploaded the results:

curl -o install.tar.bz2 http://gsc.io/nix.tar.bz2 && echo "1587e440bac9974733ec7182b22aeb7f883aa262db02486d05949979a8dec177  install.tar.bz2" | shasum -c && tar -xf ./install.tar.bz2 && cd nix-* && ./install

or if you want to run the tests and try them over and over:

test.sh:

#!/bin/sh

set -x

if [ -f /Library/LaunchDaemons/org.nixos.nix-daemon.plist ]; then
    sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist
    sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist
fi

if [ -f /etc/profile.backup-before-nix ]; then
    sudo mv /etc/profile.backup-before-nix /etc/profile
fi

if [ -f /etc/bashrc.backup-before-nix ]; then
    sudo mv /etc/bashrc.backup-before-nix /etc/bashrc
fi


if [ -f /etc/zshrc.backup-before-nix ]; then
    sudo mv /etc/zshrc.backup-before-nix /etc/zshrc
fi


for i in $(seq 1 $(sysctl -n hw.ncpu)); do
    sudo /usr/bin/dscl . -delete "/Users/nixbld$i"
done
sudo /usr/bin/dscl . -delete "/Groups/nixbld"


sudo rm -rf /etc/nix /nix /var/root/.nix-profile /var/root/.nix-defexpr /var/root/.nix-channels "$HOME/.nix-profile" "$HOME/.nix-defexpr" "$HOME"/.nix-channels"

mkdir temp

(
    cd temp
    curl -o install.tar.bz2 http://gsc.io/nix.tar.bz2
    echo "1587e440bac9974733ec7182b22aeb7f883aa262db02486d05949979a8dec177  install.tar.bz2" | shasum -c \
        && tar -xf ./install.tar.bz2 \
        && cd nix-* \
        && (
            cd nix*
            (
                echo n
                echo y
                echo y
            ) | ./install
            ret=$?

            if [ $ret -eq 0 ]; then
                echo "Success!"
            else
                echo "Failure :("
            fi
        )
);

rm -rf temp

Closes #1061

cc @shlevy @matthewbauer @domenkozar

matthewbauer · 2017-07-09T18:30:00Z

👍

You should take a look at #1141 though. It mostly deals with Linux packaging but some of the code is similar. Ideally we could get a multi-user script for Linux too.

grahamc · 2017-07-09T18:31:55Z

Sounds good. I don't want to bog this down with general Linux support. I do want to get daemon support for the general Linux deployment case, but darwin is the easiest to start with I think.

grahamc · 2017-07-09T18:55:28Z

With this latest commit we have a chicken and the egg problem which is fixed by first merging #1454:

Nix is installed with the correct profile (which came from this build)
Nix then replaces itself with one from nixpkgs which doesn't have this new profile, now nix doesn't work

Step 2 is run due to #1449

The only difference between the instructions posted above and these here, are the profile was moved in to nix itself.

veprbl · 2017-07-09T19:52:56Z

Would it make sense to save .nixignore filename for future use in files with .gitignore-like syntax?

grahamc · 2017-07-09T19:56:16Z

Fixed, @veprbl.

shlevy · 2017-07-09T20:04:57Z

release.nix

@@ -158,6 +158,9 @@ let
          substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
            --subst-var-by nix ${toplevel} \
            --subst-var-by cacert ${cacert}
+
+          shellcheck -e SC1090 $TMPDIR/install


What does SC1090 mean?

SC1090: it doesn't know what . $var/file means or resolves to, so it can't follow to /file and figure out if your code is valid.
SC1091: similar to above, it can't source the file and statically check it.
SC2002: "Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead." except if I did sudo ... < thefile it complains that sudo doesn't affect pipes.

shlevy · 2017-07-09T20:08:23Z

Merged #1454. Overall this LGTM, let's give a few days for @edolstra to take a look and if there are no objections ping me and I'll merge.

grahamc · 2017-07-09T20:10:29Z

I'll open PRs to apply this and 1454 to master, too. It is important that we release the updated nix version and it go out through to the nixpkgs-unstable channel prior to the installer being updated on nixos.org. Maybe this should be two separate version releases? .12 and then .13?

shlevy · 2017-07-09T20:14:38Z

Maybe patch releases? .12 and .12.1?

grahamc · 2017-07-09T20:25:46Z

These PRs are already to 1.11 though, so I meant 1.11.12 then 1.11.13. (as opposed to 1.11.12, 1.11.12.1) what do you think?

shlevy · 2017-07-10T01:03:24Z

Right, I meant the thing you put in parens. Don't really care though :)

grahamc · 2017-07-10T01:04:08Z

I don't either :)

puffnfresh · 2017-07-10T01:06:59Z

Just tested the install script and it looks good 👍

grahamc · 2017-07-10T01:55:08Z

I reordered a few elements to handle asking for sudo sooner, and it makes the UX not as nice in some ways. I'm still tweaking the words and ordering of the output, but the fundamentals are here and appear to work (almost) every time. Done

edolstra · 2017-07-10T14:25:58Z

Thanks @grahamc! A few comments/questions:

It's a bit unfortunate to add a huge (700+ line) shell script. We just got rid of all Perl code, and shell is a much worse language than Perl. I'm a bit scared about the maintainability of this, in particular because...
There is no test. This means we would have to test manually on macOS before every release (but since I'm lazy, we'd probably just end up releasing with an untested installer). However, testing on Hydra is probably infeasible due to the lack of an equivalent of the Linux VM test functions.
IMHO the installer should support upgrading an existing installation, rather than requiring the user to nuke everything. It seems more work not to support upgrades (since Nix takes care of any schema migration). E.g. it would simplify the installer a lot if it doesn't need to handle things like backing up profiles.
ui_confirm makes the installer interactive, which makes the script hard to use for unattended installs (like installing all Macs in the build farm). A --confirm flag would be nice, or auto-confirm if !isatty().
For reproducibility, I'm not sure if the "bootstrapping" (the auto-upgrade via Nixpkgs) is desirable. On the one hand it's convenient, but on the other hand, if I run the Nix 1.N installer, I don't expect to end up with 1.M. (Especially in a scripted scenario, e.g. curl https://nixos.org/releases/nix/nix-1.11.11/nix-1.11.11-x86_64-darwin.tar.bz2; tar xf ...; ./nix-1.11.11/install should not install a different version than 1.11.11.)
Instead of separately running each command under sudo, wouldn't it be easier to just re-exec the installer under sudo?
I don't like the first commit (filtering some files). It seems ad hoc to me. Depending on what you're working on, there are all sorts of files that you might want to pin to prevent a rebuild. It also only works because release.nix is inconsistent in using the nix input (e.g. it uses ./scripts/install-from-closure.sh rather than nix + "/scripts/install-from-closure.sh"). The way I usually deal with this is to temporarily add something like this to release.nix:

build.x86_64-linux = builtins.storePath /nix/store/...some previous build...;

grahamc · 2017-07-10T14:42:52Z

It's a bit unfortunate to add a huge (700+ line) shell script. We just got rid of all Perl code, and shell is a much worse language than Perl. I'm a bit scared about the maintainability of this, in particular because...

I don't disagree, but I take quite a bit of solace in the fact that almost all the code is either UI text output, or checking for problems before starting. This was an important goal to me, to make the experience friendly and help prevent as many problems at the outset as possible.

There is no test. This means we would have to test manually on macOS before every release (but since I'm lazy, we'd probably just end up releasing with an untested installer). However, testing on Hydra is probably infeasible due to the lack of an equivalent of the Linux VM test functions.

I was researching this last night. Travis-CI has macOS builders with root. If that works for you, I would be happy to make a thorough test suite for this code, using the travis-ci system. Unfortunately I don't think we can make this work in Hydra.

IMHO the installer should support upgrading an existing installation, rather than requiring the user to nuke everything. It seems more work not to support upgrades (since Nix takes care of any schema migration). E.g. it would simplify the installer a lot if it doesn't need to handle things like backing up profiles.

I disagree, because of the difficulties in the state transition. Being able to start clean and do the install the same way every time seems valuable. This removes any impurities caused by the single user model, for example NPM and Python tooling modifying the nix store. I think Nix's promise of reproduciblity makes this a safe and acceptable requirement for a user. Out of the many people I've had test this code, none of them balked at the idea of having to erase their store.

Since the current code is only adding a small bit of code to the profiles now, we could remove the code performing backups.

ui_confirm makes the installer interactive, which makes the script hard to use for unattended installs (like installing all Macs in the build farm). A --confirm flag would be nice, or auto-confirm if !isatty().

I'd be totally happy to add a --confirm flag, though I'd be hesitant to have it enable --confirm automatically on!isatty() as subtle changes in calling the script can accidentally trigger that (I believe I had this issue with curl | bash for example.)

For reproducibility, I'm not sure if the "bootstrapping" (the auto-upgrade via Nixpkgs) is desirable. On the one hand it's convenient, but on the other hand, if I run the Nix 1.N installer, I don't expect to end up with 1.M. (Especially in a scripted scenario, e.g. curl https://nixos.org/releases/nix/nix-1.11.11/nix-1.11.11-x86_64-darwin.tar.bz2; tar xf ...; ./nix-1.11.11/install should not install a different version than 1.11.11.)

I definitely agree, I don't like that either. I'm afraid of distributing configs with signed binary caches disabled, though. What would it take to patch and fix #1449? Fixing this would eliminate the need for this step, and allow users to benefit from signed binary caches.

Instead of separately running each command under sudo, wouldn't it be easier to just re-exec the installer under sudo?

We could do that. I thought about that, but liked the verbosity of the output, showing to the user exactly what was going on. Switching to a reexec (or asking them to run it under sudo directly) would make it hard to retain the verbosity, however I recognize that the verbosity may not be desirable. We could remove it.

I don't like the first commit (filtering some files). It seems ad hoc to me.

Okay, I will remove that commit before this merges :)

edolstra · 2017-07-10T19:04:07Z

Thanks for the reply. Sounds good to me.

Not super keen about depending on multiple CIs, but using Travis to test it is an option. (See #1152.) How does Travis give builders root? Using a VM?

The libsodium issue is easily fixed, we could even do a new maintenance release for that.

Stuff like a confirm flag can be added later if there is demand.

grahamc · 2017-07-10T22:57:59Z

It is possible to run macOS in Qemu: http://www.contrib.andrew.cmu.edu/%7Esomlo/OSXKVM/ and as far as I know the EULA says you can run 1 copy of macOS on 1 piece of Apple hardware.

It'd be a bit of a waste to have a whole, capable Apple machine not running full-bore on other x86_64-darwin Hydra jobs. If we had an older Apple machine it could run Linux, have a mandatory-feature / required-feature of emulates-macos with a max-jobs=1.

If we got macOS-in-QEMU working, I'm not sure it is something we'd want in Nix's tree or Nixpkgs' tree ... it seems a bit tenuous, but maybe?

I am not a lawyer.

grahamc · 2017-07-10T23:41:22Z

As of ef2999e darwin will have libsodium, (also closing #1449) which leads to the next commit: dd9a975 which removes the second nix install step, restoring the purity of the installer.

NOTE: Because of this change, this PR doesn't need to be released in two steps: a single release will do! 💯

grahamc · 2017-07-11T10:44:13Z

This uses the version I built last night:

curl -o install.tar.bz2 http://gsc.io/nix-v2.tar.bz2 && echo "33e60aa51a2afe901f8aa824a339516ca53fd54abcd08203ddbaca147da3c49c  install.tar.bz2" | shasum -c && tar -xf ./install.tar.bz2 && cd nix-* && ./install

Ericson2314 · 2017-07-11T18:05:57Z

scripts/install-nix-from-closure.sh

+    fi
+
+    printf '\e[1;31mSwitching to the Multi-User Darwin Installer\e[0m\n'
+    "$self/install-darwin-multi-user"


exec for that beautiful tail-call? :D

Ericson2314 · 2017-07-11T18:29:29Z

scripts/install-darwin-multi-user.sh

+
+    echo "I am executing:"
+    echo ""
+    echo "    $ sudo $cmd"


I'd consider using printf %q and not pass the command as a single string, lest there be any ambiguities.

…r root-owned files instead of making a scary warning.

grahamc · 2017-07-12T15:46:49Z

I deleted the first commit, and have added Travis tests. Still waiting on Travis to say if it passes.

grahamc · 2017-07-12T16:03:42Z

If https://travis-ci.org/grahamc/nix/builds/252864812 builds, it should be good to merge. I believe I have addressed all feedback.

Ericson2314 · 2017-07-12T20:11:00Z

Err @grahamc for the _sudo printing I mean something like https://github.com/NixOS/nixpkgs/pull/27336/files#diff-7a1764865f30c6cb4c22c2654ebe9a38R728, e.g.:

printf "   $ sudo"
printf " %q" "${cmd[@]}"
echo

(printf will repeat the format string until all arguments are consumed.)
then

_sudo foo 'bar baz' ''

will print

  $ sudo foo bar\ baz ''

grahamc · 2017-07-12T21:08:11Z

except it is a single string due to the '$*': https://github.com/NixOS/nix/pull/1453/files#diff-2935a4d9174dece2a973aef5f0e25c97R213

grahamc · 2017-07-13T01:44:42Z

This PR passed on Travis: https://travis-ci.org/grahamc/nix/builds/253029042 and also closes #1459.

grahamc · 2017-07-13T09:58:43Z

Final build here:

curl -o install.tar.bz2 http://gsc.io/nix-v3.tar.bz2 && echo "1a6e12495734b9772cae6fd308ee2b9d09d14e8959604e348244b4e7ec1d60d2  install.tar.bz2" | shasum -c && tar -xf ./install.tar.bz2 && cd nix-* && ./install

copumpkin · 2017-07-13T12:50:09Z

Joy!

domenkozar · 2017-07-13T17:00:32Z

FYI: this broke curl http://nixos.org/nix/install | sh since we're stealing stdin.

Need to change our short installation api or fix the script to not use stdin when it's taken.

domenkozar · 2017-07-13T18:38:28Z

Furthermore, for example on travis-ci, we cache /nix/store and it's restored on each build before everything else. Now the installer complains since nix is already "installed".

matthewbauer · 2017-07-13T23:03:54Z

@grahamc Can we get an opt-out option in install-nix-from-closure.sh? It seems to me there are legitimate reasons to prefer single user install.

Similarly, some sort of "quiet" option and a "noninteractive" option would be helpful when integrating this with CIs.

grahamc · 2017-07-13T23:35:58Z

@domenkozar I can't find where we cache /nix/store in Travis, where is that?

Working on solutions here: 1.11-maintenance...grahamc:fix-stdin-errors

if there is no TTY, it doesn't prompt (assumes yes)
If Travis sets PINCH_ME_IM_SILLY=x it doesn't check anything before installing, and allows /nix/store to remain during the installation (although @LnL7 isn't sure how or why this works, he thinks Nix will delete the unknown paths)
Reduce output -- I've stopped outputting the sudo description if there is no TTY. This greatly reduces output.

Opt-out? Hmm.. I'm not sure. I don't love nix without the daemon, and would rather we push everyone to the daemon. Do you have any specific example use cases that justifies the over-head of continuing to support single-user nix on macOS? Obviously advanced users will be able to go back to a single-user installation manually, but that is a different case as they're more prepared to handle the down sides.

grahamc changed the title ~~Multi user darwin~~ Multi user darwin installer Jul 9, 2017

grahamc mentioned this pull request Jul 9, 2017

Add nix profile suitable for using the daemon #1454

Merged

grahamc force-pushed the multi-user-darwin branch from d9199c3 to e832b9f Compare July 9, 2017 19:55

shlevy reviewed Jul 9, 2017

View reviewed changes

grahamc force-pushed the multi-user-darwin branch from e832b9f to 98ac2a8 Compare July 9, 2017 20:47

Ericson2314 reviewed Jul 11, 2017

View reviewed changes

grahamc added 4 commits July 12, 2017 11:44

Shellcheck the existing installer

799f5ad

Switch to a fancy multi-user installer on Darwin

2189781

multi-user install: move the profile in to the nix etc/profiles.d output

3ebd25a

Prompt for sudo before validating assumptions, and check ourselves fo…

262a08c

…r root-owned files instead of making a scary warning.

grahamc added 2 commits July 12, 2017 11:45

Address feedback around printf & exec

2442c46

Test the installer

302e820

grahamc force-pushed the multi-user-darwin branch from 9246e06 to 302e820 Compare July 12, 2017 15:45

chmod

163d931

Run nix-build inside a fresh bash login

c4f349d

grahamc mentioned this pull request Jul 12, 2017

Build tarball per system #1459

Closed

release: don't build libseccomp if we're on darwin

a31347d

grahamc force-pushed the multi-user-darwin branch from 93f315a to a31347d Compare July 12, 2017 23:23

grahamc added 3 commits July 12, 2017 20:29

Clean up nix hints from the old insstaller

b2917c8

Only clean if the file exists

85acfcd

Cleanup and more specificity around set -e

c821267

grahamc force-pushed the multi-user-darwin branch from fdd17b6 to c821267 Compare July 13, 2017 01:44

edolstra merged commit e135db7 into NixOS:1.11-maintenance Jul 13, 2017

grahamc deleted the multi-user-darwin branch July 13, 2017 11:29

grahamc restored the multi-user-darwin branch July 13, 2017 11:29

domenkozar mentioned this pull request Jul 13, 2017

nixos.org/nix/install doesn't build with libsodium #1449

Closed

domenkozar mentioned this pull request Jul 13, 2017

Switch Mac installer to set up multi-user installation of Nix #1061

Closed

matthewbauer mentioned this pull request Jul 13, 2017

Nix: fix macOS install by using 'yes' travis-ci/travis-build#1112

Merged

grahamc mentioned this pull request Jul 14, 2017

Fix stdin errors #1463

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi user darwin installer #1453

Multi user darwin installer #1453

grahamc commented Jul 9, 2017 •

edited

Loading

matthewbauer commented Jul 9, 2017

grahamc commented Jul 9, 2017

grahamc commented Jul 9, 2017

veprbl commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy Jul 9, 2017

grahamc Jul 9, 2017

shlevy commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy commented Jul 10, 2017

grahamc commented Jul 10, 2017

puffnfresh commented Jul 10, 2017

grahamc commented Jul 10, 2017 •

edited

Loading

edolstra commented Jul 10, 2017

grahamc commented Jul 10, 2017 •

edited

Loading

edolstra commented Jul 10, 2017

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 11, 2017 •

edited

Loading

Ericson2314 Jul 11, 2017

Ericson2314 Jul 11, 2017

grahamc commented Jul 12, 2017

grahamc commented Jul 12, 2017

Ericson2314 commented Jul 12, 2017 •

edited

Loading

grahamc commented Jul 12, 2017

grahamc commented Jul 13, 2017

grahamc commented Jul 13, 2017

copumpkin commented Jul 13, 2017

domenkozar commented Jul 13, 2017

domenkozar commented Jul 13, 2017

matthewbauer commented Jul 13, 2017

grahamc commented Jul 13, 2017 •

edited

Loading

Multi user darwin installer #1453

Multi user darwin installer #1453

Conversation

grahamc commented Jul 9, 2017 • edited Loading

matthewbauer commented Jul 9, 2017

grahamc commented Jul 9, 2017

grahamc commented Jul 9, 2017

veprbl commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy Jul 9, 2017

Choose a reason for hiding this comment

grahamc Jul 9, 2017

Choose a reason for hiding this comment

shlevy commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy commented Jul 9, 2017

grahamc commented Jul 9, 2017

shlevy commented Jul 10, 2017

grahamc commented Jul 10, 2017

puffnfresh commented Jul 10, 2017

grahamc commented Jul 10, 2017 • edited Loading

edolstra commented Jul 10, 2017

grahamc commented Jul 10, 2017 • edited Loading

edolstra commented Jul 10, 2017

grahamc commented Jul 10, 2017 • edited Loading

grahamc commented Jul 10, 2017 • edited Loading

grahamc commented Jul 11, 2017 • edited Loading

Ericson2314 Jul 11, 2017

Choose a reason for hiding this comment

Ericson2314 Jul 11, 2017

Choose a reason for hiding this comment

grahamc commented Jul 12, 2017

grahamc commented Jul 12, 2017

Ericson2314 commented Jul 12, 2017 • edited Loading

grahamc commented Jul 12, 2017

grahamc commented Jul 13, 2017

grahamc commented Jul 13, 2017

copumpkin commented Jul 13, 2017

domenkozar commented Jul 13, 2017

domenkozar commented Jul 13, 2017

matthewbauer commented Jul 13, 2017

grahamc commented Jul 13, 2017 • edited Loading

grahamc commented Jul 9, 2017 •

edited

Loading

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 10, 2017 •

edited

Loading

grahamc commented Jul 11, 2017 •

edited

Loading

Ericson2314 commented Jul 12, 2017 •

edited

Loading

grahamc commented Jul 13, 2017 •

edited

Loading