darwin: generate apple packages preparing for update macos sdk #108590

holymonson · 2021-01-06T15:43:17Z

Motivation for this change

macos sdk is outdated for a long time (#101229), this PR provides scaffolds and candidates for future updating.
Except for normalizing package name as ${pname}-${version}-${sdkName}, in essence this PR doesn't update any packages. We may want a smooth update process divided into small tasks instead of a large package commit.

Things done

Generate package sources and rewrite some function names.
Will cause mass rebuild due to some derivation names normalized, but it should not cause any actual change in compiling.

cc @matthewbauer

holymonson · 2021-01-07T10:41:13Z

Success building all apple packages and the depending bootstrap packages, except #108651.

add cc @siraben @SuperSandro2000

SuperSandro2000 · 2021-01-07T10:51:07Z

I saw this already earlier. If I understand it correctly this should make the update process easier but does not actually change any packages?

Also please change the commit message and PR title to something starting with darwin: or darwin.appke_sdk.

holymonson · 2021-01-07T11:44:07Z

If I understand it correctly this should make the update process easier but does not actually change any packages?

Yes.

We hesitate to update apple_sdk is mainly because we make up a header-only Libsystem, which is required for bootstrap. But for many other apple packages in apple-source-releases, there are few packages depending on them individual, and could be updated without a mass impact. For example, network_cmds is marked as insecure because of openssl-1.0.2u since 2019, we could keep it up-to-date instead of sticking to osx-10.11.6.

My plan is to take down those light-depended packages one by one, the PR could make it easier.

nixos-discourse · 2021-01-15T14:39:18Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/445

holymonson · 2021-01-17T05:28:37Z

@veprbl could you review this?

veprbl

This is a bit hard to review as orthogonal changes are implemented in one commit, but here is what I saw

pkgs/os-specific/darwin/apple-source-releases/default.nix

pkgs/os-specific/darwin/apple-source-releases/generate-sdk-packages.sh

veprbl · 2021-01-17T16:28:23Z

pkgs/os-specific/darwin/apple-source-releases/default.nix

  };
+
+  packages = developerToolsPackages // macosPackages // stubPackages;


Suggested change

packages = developerToolsPackages // macosPackages // stubPackages;

packages = developerToolsPackages // macosPackages // stubPackages;

Do I understand it correctly that developerToolsPackages // macosPackages is a dead code right now?

Yes, right now it is. Next step if we remove a line in stubPackages, then the package will be updated.

It would have been more clean to first regenerate the current set with your framework to prove that it works and then do the update separately.

Hard to do that because you see versions containing multiple versions of sdk and even some of them are fake, I would rather keep it what it is. And I won't update all package in a time, instead, I'm going to create PR later one by one and test to ensure they work. At least in this PR the redundant framework won't take prior than the original stubPackages.

Well, my thought was that we could still hold on including definitions developerToolsPackages and macosPackages until at least one of the updated frameworks is used. But if you are confident that it will work out the way you've designed, then we could try.

pkgs/os-specific/darwin/apple-source-releases/default.nix

holymonson · 2021-01-19T18:54:05Z

@veprbl I have rewrite some function name back and add new function with suffix _ , hope this would make it easier to review.

pkgs/os-specific/darwin/apple-source-releases/developer-tools-11.3.1.nix

pkgs/os-specific/darwin/apple-source-releases/macos-11.0.1.nix

pkgs/os-specific/darwin/apple-source-releases/default.nix

holymonson · 2021-01-21T07:13:57Z

gentle ping @matthewbauer

siraben

LGTM

bobrik · 2021-02-05T06:38:35Z

pkgs/os-specific/darwin/apple-source-releases/default.nix

  };
+
+  packages = developerToolsPackages_11_3_1 // macosPackages_11_0_1 // stubPackages;


Is the order here correct? Should stub packages take precedence over everything else?

I have #105026 locally and it tries to install libutil-47.30.1-osx-10.12.6, which doesn't seem right (and it also fails, because 10.12 doesn't know anything about aarch64).

cc @thefloweringash

Is the order here correct? Should stub packages take precedence over everything else?

stubPackages does take precedence, the later overwrites the formers.

I have #105026 locally and it tries to install libutil-47.30.1-osx-10.12.6, which doesn't seem right (and it also fails, because 10.12 doesn't know anything about aarch64).

libutil-47.30.1-osx-10.12.6 is in stubPackages, we haven't updated libutil yet, so it still installs the old 10.12 one, that is exactly expected. I'm a bit confused of which version is you actually want.

ofborg bot added 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin-stdenv labels Jan 6, 2021

ofborg bot requested review from shlevy, gridaphobe, copumpkin, matthewbauer and LnL7 January 6, 2021 15:53

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 1-10 labels Jan 6, 2021

holymonson force-pushed the apple_package_version branch from 0ffa9d3 to 8301dfa Compare January 7, 2021 11:57

holymonson changed the title ~~generate apple packages preparing for update macos sdk~~ darwin: generate apple packages preparing for update macos sdk Jan 7, 2021

holymonson mentioned this pull request Jan 11, 2021

darwin.bootstrap_cmds: update to 121 #109052

Merged

10 tasks

siraben requested review from Mic92 and removed request for Mic92 January 15, 2021 15:22

veprbl reviewed Jan 17, 2021

View reviewed changes

veprbl added this to WIP in Staging via automation Jan 17, 2021

SuperSandro2000 added the 2.status: merge conflict label Jan 18, 2021

holymonson mentioned this pull request Jan 19, 2021

darwin: check apple headers #109368

Merged

10 tasks

holymonson force-pushed the apple_package_version branch from 8301dfa to 8a7c84d Compare January 19, 2021 18:47

holymonson requested a review from veprbl January 19, 2021 18:54

holymonson force-pushed the apple_package_version branch from 8a7c84d to be92725 Compare January 19, 2021 18:55

ofborg bot added 8.has: clean-up and removed 2.status: merge conflict labels Jan 19, 2021

ofborg bot added 8.has: package (new) 10.rebuild-linux: 1 labels Jan 19, 2021

veprbl reviewed Jan 20, 2021

View reviewed changes

holymonson force-pushed the apple_package_version branch 2 times, most recently from b5c5507 to 77096f4 Compare January 20, 2021 07:13

darwin: generate apple packages preparing for update macos sdk

9d5cece

holymonson force-pushed the apple_package_version branch from 77096f4 to 9d5cece Compare January 20, 2021 07:19

holymonson requested a review from veprbl January 20, 2021 07:24

veprbl approved these changes Jan 22, 2021

View reviewed changes

Staging automation moved this from WIP to Ready Jan 22, 2021

siraben approved these changes Jan 23, 2021

View reviewed changes

veprbl merged commit 963286d into NixOS:staging Jan 23, 2021

Staging automation moved this from Ready to Done Jan 23, 2021

bobrik reviewed Feb 5, 2021

View reviewed changes

bobrik mentioned this pull request Feb 6, 2021

Native support for Apple Silicon #105026

Merged

21 tasks

toonn mentioned this pull request Mar 14, 2021

Status of macOS #116341

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

darwin: generate apple packages preparing for update macos sdk #108590

darwin: generate apple packages preparing for update macos sdk #108590

holymonson commented Jan 6, 2021

holymonson commented Jan 7, 2021

SuperSandro2000 commented Jan 7, 2021

holymonson commented Jan 7, 2021

nixos-discourse commented Jan 15, 2021

holymonson commented Jan 17, 2021

veprbl left a comment

veprbl Jan 17, 2021

holymonson Jan 17, 2021

veprbl Jan 17, 2021

holymonson Jan 19, 2021

veprbl Jan 20, 2021

holymonson commented Jan 19, 2021

holymonson commented Jan 21, 2021

siraben left a comment

bobrik Feb 5, 2021 •

edited

holymonson Feb 5, 2021

		};

		packages = developerToolsPackages // macosPackages // stubPackages;

		};

		packages = developerToolsPackages_11_3_1 // macosPackages_11_0_1 // stubPackages;

darwin: generate apple packages preparing for update macos sdk #108590

darwin: generate apple packages preparing for update macos sdk #108590

Conversation

holymonson commented Jan 6, 2021

Motivation for this change

Things done

holymonson commented Jan 7, 2021

SuperSandro2000 commented Jan 7, 2021

holymonson commented Jan 7, 2021

nixos-discourse commented Jan 15, 2021

holymonson commented Jan 17, 2021

veprbl left a comment

Choose a reason for hiding this comment

veprbl Jan 17, 2021

Choose a reason for hiding this comment

holymonson Jan 17, 2021

Choose a reason for hiding this comment

veprbl Jan 17, 2021

Choose a reason for hiding this comment

holymonson Jan 19, 2021

Choose a reason for hiding this comment

veprbl Jan 20, 2021

Choose a reason for hiding this comment

holymonson commented Jan 19, 2021

holymonson commented Jan 21, 2021

siraben left a comment

Choose a reason for hiding this comment

bobrik Feb 5, 2021 • edited

Choose a reason for hiding this comment

holymonson Feb 5, 2021

Choose a reason for hiding this comment

bobrik Feb 5, 2021 •

edited