Wait, this is viable? From following #2925, I thought the consensus was that nixpkgs currently has impurities in it that preclude using a (symlink to a) non-/nix store path. Did I misunderstand, or is the worry just overblown and this will work for "most people"?

There's a big difference between changing NIX_STORE_DIR and it's physical location. With a symlinked store it's possible for build results to leak the canonical location. While this isn't great, it's is only causes problems when sharing caches between hosts that use different locations. So if all macs use the same target there's no issue. It's a tradeoff, but I feel like it's better to keep the installer as simple as possible.

If we avoid this on the hydra infrastructure that build the official caches (at least for now) I think it won't be a problem anybody runs into in practice.

I am a bit concerned that what I download from Hydra wouldn't match what I build locally, e.g. someone already said llvm (or clang?) seems to do a realpath on the build location and will embed the symlink destination.

As long as apfs.util works, can we not use it for the volume approach?

Another potential benefit of the volume approach that's been mentioned is the fact that we can make it case-sensitive.

While this isn't great, it's is only causes problems when sharing caches between hosts that use different locations. So if all macs use the same target there's no issue.

Ahh, gotcha. I think I understand now. And as @lilyball says, this would mean that packages built with a "real" store path of /nix might be different to ones we build on a Mac with the symlinked store. But your point is, while they might not match, everything will still work?

Long term this wouldn't be a problem. I'm more concerned about the complexity of managing a separate volume and what could change/break there in the future.

cc @edolstra I tested this and it works, it would be great to have 2.3.2 release out once it's reviewed and merged.

So worst case, if the official macOS builders stay on pre-10.15 for awhile and keep building packages for a physical /nix store location, it just means that Macs that symlink their physical store location to /nix will miss the Hydra cache occasionally, correct?

In any case, I think this should be /opt/nix, because it follows the Linux FHS and therefore there's a better chance (though, still, admittedly slim) that Nix on Linux could adopt it someday, as well.

Are the Hydra macOS builders guaranteed to run on the OS that we use as the minimum macOS target? If we ever run a 10.15 Hydra macOS builder while we still support pre-10.15, that means binary caches for packages that "see through" the symlink will then break on the older OSes.

I like this solution a lot. However, since I experimented with a symlinked store before, we may need to extend the purity checks in the ld wrapper. I had problems, where the absolute path of the store would sneak into a build process. ld would then complain, that a library from outside the store was linked against, while in fact it was located within the store.

Unfortunately, I do not remember what I was building back then. So this is more of a warning than a call for action right now.

Aside from the obvious problems regarding purity, it's worth noting here that I've so far encountered a few inconsistencies in nix from trying to switch to a symlink'd store path:

• builtins.storePath appears to just not work at all?
• nix-build /nix/store/something.drv gets confused
  • (need to check if nix-store -r has the same problem?)

@mroi Things like that I what I was most afraid of, would be great if you had an example.

@mroi Things like that I what I was most afraid of, would be great if you had an example.

I honestly do not remember. Sorry. If I find the time, I will try some things I usually build myself.

Note that we'll also need to fix #2407 in order to support macos + multi-user mode.

EDIT: already done in d4e51aa#diff-3f9ecc4aa9722ed5597a741e4cb91d3e

Looks good besides a few small things. I think it would be good to get this in a Nix release pretty soon as we lose lots of potential users without a well-documented Catalina installer.

Tried on a fresh macOS using the following steps:

# on a pre-10.15 Nix install
$ git clone https://github.com/LnL7/nix
$ cd nix
$ git checkout darwin-10.15-install
$ nix build -f release.nix binaryTarball
$ cp result/*.tar.xz ~/nix.tar.xz
# Copy ~/nix.tar.xz to your Catalina macOS and open a terminal there
$ tar xf nix.tar.xz
$ ./nix-*/create-volume.sh
$ ./nix-*/install.sh --daemon

@lilyball Given that bash is ancient on macOS I intentionally kept this POSIX sh.

The manual and error messages should be much nicer now, uploaded the updated manual here. Since I rebased to fix some merge conflicts it should be tested again but other then that I think it's done.

Thanks @abathur!

The manual and error messages should be much nicer now, uploaded the updated manual here. Since I rebased to fix some merge conflicts it should be tested again but other then that I think it's done.

Thanks @abathur!

• @LnL7 tested the installer on Catalina + pre-T2 + FileVault and it correctly prompted to pre-create the volume.
• I tested Catalina + T2 + (without FileVault & with FileVault). In both cases:
  • a bare call to the installer correctly pointed me to the documentation and prompted me to re-run with the --darwin-use-unencrypted-nix-store-volume flag
  • re-running with the flag correctly created the volume and installed Nix
• I also tested Mojave + pre-T2 + (without FileVault & with FileVault, for good measure) and in both cases it cleanly installed on a bare call to the installer.

I'm going to merge this, given that people have been reporting for months that it works (and now that documentation has been written) it seems like not much will improve until we get this live.

So either I'm missing something or the instructions in the manual definitely don't work. It complains about asking for the flag about --daemon | --no-daemon and I cannot seem to supply either argument. It isn't threading the flags through to the installer correctly. What's going on here?

@ProofOfKeags How are you building/obtaining the install script? What exact command are you using?

$ sh <(curl https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume as stated in the manual link

EDIT: after digging it seems apparent that whatever install script I'm fetching does not even include the option for the --darwin... I gather this hasn't been pushed out to the main host?

A new version of Nix hasn't been released yet.

Edit: I don't know exactly what the release/deploy cycle for Nix looks like, but (AFAIK) if you watch the repo for a new release, the command you quoted should work at approximately that time :)

Until it's released, my macOS system bootstrap has been doing roughly (I'm adapting it for you--I last ran it before any of this work was merged...) this:

curl https://raw.githubusercontent.com/NixOS/nix/master/scripts/create-darwin-volume.sh | bash
curl https://nixos.org/nix/install | sh

(create-darwin-volume.sh is what the --darwin-use-unencrypted-nix-store-volume flag will cause the main installer to run; it needs to be run before the main install)

@ProofOfKeags Aside from this mishap: You may be one of the first to try following the manual update from scratch--any feedback? Do you have questions it didn't answer?

The guide seemed fairly straight forward, I think. But given that the first command didn't work and I didn't bother trying to go through the rest since it'll work when it is released, I can't say for sure whether the instructions for the rest of it will work. I actually have half a mind right now to partition my mac again and just use NixOS raw, because MacOS seems to be increasingly hostile towards developers.

@abathur just for confirmation I was successfully able to remove my single user-install and reinstall a multi-user installation on Catalina + T2 + Filevault earlier today running those same commands

@abathur I can confirm that I could successfully set up nix on Catalina (with T2 chip) with the commands

curl https://raw.githubusercontent.com/NixOS/nix/master/scripts/create-darwin-volume.sh | bash
curl https://nixos.org/nix/install | sh