Support High Sierra on Darwin #27318

copumpkin · 2017-07-12T02:00:16Z

Motivation for this change

An upcoming macOS 10.13 release breaks the current nixpkgs. This should fix it. Please test on 10.11, 10.12, and even 10.10 (which is a best effort, but should still probably work).

Should fix NixOS/nix#1414 NixOS/nix#1451 once this flows through to the Nix build.

This could be done without a linux mass-rebuild, but I figured I'd just send to staging since that lets me use lib.optional rather than null in derivation attributes.

Things done

Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- macOS (10.13)
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

The main changes are in libSystem, which lost the coretls component in 10.13 and some hardening changes that quietly crash any program that uses %n in a non-constant format string, so we've needed to patch a lot of programs that use gnulib.

mention-bot · 2017-07-12T02:00:19Z

@copumpkin, thanks for your PR! By analyzing the history of the files in this pull request, we identified @pikajude, @Ericson2314, @LnL7 and @copumpkin to be potential reviewers.

copumpkin · 2017-07-12T02:02:47Z

Note that testing this on 10.13 is still pretty tricky until we have a working Nix build based on this branch.

domenkozar · 2017-07-12T12:24:20Z

@copumpkin do you have 10.13? We could prepare Nix installer using this branch.

copumpkin · 2017-07-12T12:51:07Z

Yeah, running in a VM. How do I prepare an installer?

domenkozar · 2017-07-12T13:03:59Z

In Nix repo:

$ nix-build release.nix -A binaryTarball.x86_64-darwin --arg nixpkgs ./foo

Where foo is the checkout of this branch.

Then download nixos.org/nix/install and replace the location of the tarball with the result built and hosted somewhere :)

LnL7 · 2017-07-12T21:30:57Z

Everything looks good on 10.11

copumpkin · 2017-07-13T02:31:51Z

Thanks @LnL7!

Okay, so anyone who wants to test it on 10.13 can use this magic installer link hosted on my very reputable Dropbox account and gist 😉

curl https://gist.githubusercontent.com/copumpkin/8373ae6c89236989ae4296771ba5aa14/raw/684cd2edb9b250d24f8737a17a7295b1142e6651/nix-install.sh | sh

This will install Nix 1.12 not because it needs 1.12 but because that's what @domenkozar's PR did and I was too lazy to cherry-pick it over to 1.11 maintenance 😄

Also, the default channel it downloads will be wrong (it'll point to expressions that don't support 10.13 and download binary cache binaries that crash), so after you install it you should point your nixpkgs against the tree in this PR.

P.S: I also had to patch out the weird rpath stuff in aws-sdk-cpp that I mention in here, or the install tarball was 170MB (vs. 13MB here)

jhenahan · 2017-07-14T04:43:00Z

What's the right thing to do to point nix-channel at the tree in this PR? I gave it a shot but I think I still ended up with an old stdenv-darwin.

copumpkin · 2017-07-14T05:01:18Z

@jhenahan I think you should be able to add https://github.com/copumpkin/nixpkgs/archive/darwin-high-sierra.tar.gz as your nixpkgs channel! If that doesn't work, you can just git clone, check out this revision, then repoint /.nix-defexpr at that clone.

jhenahan · 2017-07-14T19:10:23Z

@copumpkin That did seem to work. Everything's rebuilding right now, but things seem to be working so far. Thanks. 👍

copumpkin · 2017-07-18T14:50:13Z

Okay, if there are no objections or further testers, I'm going to merge this in a few hours.

domenkozar · 2017-07-18T14:53:59Z

@copumpkin note that in order to use this for next Nix release, we need to backport it to 17.03 branch. Otherwise it will have to wait until NixOS 17.09.

another reason for #25479

copumpkin · 2017-07-18T19:04:49Z

@domenkozar good point, but I don't have time right now to try it out on 17.03. If anyone wants to try cherry-picking and making a PR for that I'd appreciate it. I'd still like to merge this to master today if nobody objects

oforero · 2017-08-01T04:19:34Z

Hello,

I am trying to move to Nix from homebrew, I was able to install nix with the link provided above, but I am unable to install stack because I get the same issues with libcoretls.

What I have tried:

Clone the repository mentioned above, created a symlink at /.nix-defexpr pointing to the clone, checkout the darwin-high-sierra branch
removed the nix-channel

Trying to install stack fails with what it seems to be the same issue related to the installer not working.

nix-env -i stack
building path(s) ‘/nix/store/l04jzymp73pi0s9pyzfcyzxraxzld5ir-stdenv-darwin-boot-0’
dyld: Library not loaded: /usr/lib/system/libsystem_coretls.dylib
Referenced from: /nix/store/49g980lqdsy92rji8bw3qlj7qk24a82f-bootstrap-tools/lib/libSystem.B.dylib
Reason: image not found
builder for ‘/nix/store/dy6c63dza03ycfkjlc563ldsvrxjysrj-stdenv-darwin-boot-0.drv’ failed due to signal 6 (Abort trap: 6)

PS: Another thing I noticed is that it tries to install stack 1.4 instead of 1.5

copumpkin · 2017-08-01T04:32:36Z

@oforero is your clone on the staging branch? I think the key change hasn't made it to master yet. That'll affect everything else. The Stack 1.4 vs. 1.5 thing just depends on what version we have in nixpkgs, which I don't keep an eye on.

Feel free to swing by ##nix-darwin on freenode IRC if you want more help (but probably not right now since most of us are asleep 😄)

jhenahan · 2017-08-01T04:38:16Z

bd2e91e3a2 ought to be in master, according to git log. I got the same complaint about libsystem_coretls when I tried just changing the nixpkgs channel to unstable. EDIT: Ah, I see, the build is failing so it hasn't actually made it to unstable, yet.

oforero · 2017-08-01T07:41:44Z

I cloned the official repo instead of @copumpkin's fork. And switch the branch to staging.

Now it proceeds a lot more and then fails in a different way:

Undefined symbols for architecture x86_64:
  "_host_zone_info", referenced from:
     -reexported_symbols_list command line option
ld: symbol(s) not found for architecture x86_64

I guess it is the price to pay for being in Beta. I think I have to spend some more time learning the basics of nix.
Is there some very good examples of how to create/upgrade a package definition? for example what if I wanted to give a shot at having GCC 7.1 in Nix, the latest one I can find is 5.4

I see that there are derivations for many versions, however, `nix-env -qa` only see version 5.4, I would appreciate a pointer to a document that explains how the package derivations are found and how to install specific ones.

domenkozar · 2017-08-11T11:03:30Z

Sierra release is on 19th September, so we can wait for 17.09 branch, fix bugs there and make another Nix release.

domenkozar · 2017-09-02T10:10:04Z

@edolstra can we switch to 17.09 branch for building Nix 1.11 maintenance branch?

edolstra · 2017-09-04T15:03:50Z

I switched the master branch to 17.09. It doesn't build on Darwin: https://hydra.nixos.org/eval/1389529

copumpkin · 2017-09-04T15:15:47Z

It looks like @NeQuissimus upgraded coreutils (in 17.09 and staging) which broke the patch we apply for High Sierra compatibility (by making it unnecessary, presumably). I'm out and about right now but if someone just wants to just kill that darwin patch on staging and 17.09, I'd expect that to fix it.

NeQuissimus · 2017-09-04T15:33:15Z

Looks like we merged from staging too eagerly :) I do not have a Darwin machine handy, so somebody else will need to take a look at it :(

domenkozar · 2017-09-12T10:51:04Z

there is a binary tarball now in nix-store -r /nix/store/nnga7p9ivvi333q9y3llsncvyybm8qgq-nix-binary-tarball-1.11.15/ as part of https://hydra.nixos.org/eval/1391723# - can someone test it on high sierra?

Fix Darwin stdenv to work on 10.13

0419452

The main changes are in libSystem, which lost the coretls component in 10.13 and some hardening changes that quietly crash any program that uses %n in a non-constant format string, so we've needed to patch a lot of programs that use gnulib.

copumpkin requested review from grahamc, LnL7 and edolstra July 12, 2017 02:00

copumpkin requested a review from domenkozar July 12, 2017 02:03

LnL7 approved these changes Jul 12, 2017

View reviewed changes

This was referenced Jul 13, 2017

nix fails to install under macosx 10.13 (High Sierra) beta NixOS/nix#1414

Closed

[High sierra] dyld: Library not loaded NixOS/nix#1451

Closed

domenkozar added the 6.topic: darwin label Jul 18, 2017

copumpkin merged commit bd2e91e into NixOS:staging Jul 18, 2017

jhenahan mentioned this pull request Sep 22, 2017

Tracking High Sierra support reflex-frp/reflex-platform#147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support High Sierra on Darwin #27318

Support High Sierra on Darwin #27318

copumpkin commented Jul 12, 2017

mention-bot commented Jul 12, 2017

copumpkin commented Jul 12, 2017

domenkozar commented Jul 12, 2017

copumpkin commented Jul 12, 2017

domenkozar commented Jul 12, 2017 •

edited

Loading

LnL7 commented Jul 12, 2017

copumpkin commented Jul 13, 2017

jhenahan commented Jul 14, 2017 •

edited

Loading

copumpkin commented Jul 14, 2017

jhenahan commented Jul 14, 2017

copumpkin commented Jul 18, 2017

domenkozar commented Jul 18, 2017 •

edited

Loading

copumpkin commented Jul 18, 2017

oforero commented Aug 1, 2017 •

edited

Loading

copumpkin commented Aug 1, 2017

jhenahan commented Aug 1, 2017 •

edited

Loading

oforero commented Aug 1, 2017 •

edited

Loading

domenkozar commented Aug 11, 2017

domenkozar commented Sep 2, 2017

edolstra commented Sep 4, 2017

copumpkin commented Sep 4, 2017 •

edited

Loading

NeQuissimus commented Sep 4, 2017

domenkozar commented Sep 12, 2017

Support High Sierra on Darwin #27318

Support High Sierra on Darwin #27318

Conversation

copumpkin commented Jul 12, 2017

Motivation for this change

Things done

mention-bot commented Jul 12, 2017

copumpkin commented Jul 12, 2017

domenkozar commented Jul 12, 2017

copumpkin commented Jul 12, 2017

domenkozar commented Jul 12, 2017 • edited Loading

LnL7 commented Jul 12, 2017

copumpkin commented Jul 13, 2017

jhenahan commented Jul 14, 2017 • edited Loading

copumpkin commented Jul 14, 2017

jhenahan commented Jul 14, 2017

copumpkin commented Jul 18, 2017

domenkozar commented Jul 18, 2017 • edited Loading

copumpkin commented Jul 18, 2017

oforero commented Aug 1, 2017 • edited Loading

copumpkin commented Aug 1, 2017

jhenahan commented Aug 1, 2017 • edited Loading

oforero commented Aug 1, 2017 • edited Loading

domenkozar commented Aug 11, 2017

domenkozar commented Sep 2, 2017

edolstra commented Sep 4, 2017

copumpkin commented Sep 4, 2017 • edited Loading

NeQuissimus commented Sep 4, 2017

domenkozar commented Sep 12, 2017

domenkozar commented Jul 12, 2017 •

edited

Loading

jhenahan commented Jul 14, 2017 •

edited

Loading

domenkozar commented Jul 18, 2017 •

edited

Loading

oforero commented Aug 1, 2017 •

edited

Loading

jhenahan commented Aug 1, 2017 •

edited

Loading

oforero commented Aug 1, 2017 •

edited

Loading

copumpkin commented Sep 4, 2017 •

edited

Loading