Upstream PLIST handling #42669

matthewbauer · 2018-06-27T19:39:26Z

Motivation for this change

Need to make sure this is okay with @LnL7 first, but this is the plist generator from nix-darwin. Hopefully there is no problem with upstreaming it to Nixpkgs.

@3noch has some improvements for it that are needed by reflex-platform. I want to work with him to get this included in Nixpkgs as soon as possible (LnL7/nix-darwin#78 still needs
some changes).

@LnL7

/cc @LnL7 @3noch

This is a good way to test that plist works. Sorry for the large diff. There are a bunch of cleanups in here that needed to be done. Should make it possible to use in cross compilation.

matthewbauer · 2018-06-27T19:39:46Z

@GrahamcOfBorg build darwin.adv_cmds

GrahamcOfBorg · 2018-06-27T19:39:55Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: darwin.adv_cmds

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-06-27T19:40:20Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: darwin.adv_cmds

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-06-27T19:40:33Z

Success on x86_64-darwin (full log)

Attempted: darwin.adv_cmds

Partial log (click to expand)

    /nix/store/1nzpp0fbbgbrm36v2sjxkjcpy9zjbw82-nixpkgs.xctoolchain/usr/bin/dsymutil /private/tmp/nix-build-adv_cmds-osx-10.11.6.drv-0/adv_cmds-163/Products/Release/whois -o /private/tmp/nix-build-adv_cmds-osx-10.11.6.drv-0/adv_cmds-163/Products/Release/whois.dSYM

** BUILD SUCCEEDED **
installing
post-installation fixup
gzipping man pages under /nix/store/n7bgk5p42545y5l9vkbgbp8r0p4gkcsw-adv_cmds-osx-10.11.6/share/man/
strip is /nix/store/7ddbq63v97nk8gkbf7gcsfmby37h6gbl-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/n7bgk5p42545y5l9vkbgbp8r0p4gkcsw-adv_cmds-osx-10.11.6/bin
patching script interpreter paths in /nix/store/n7bgk5p42545y5l9vkbgbp8r0p4gkcsw-adv_cmds-osx-10.11.6
/nix/store/n7bgk5p42545y5l9vkbgbp8r0p4gkcsw-adv_cmds-osx-10.11.6

matthewbauer · 2018-06-27T19:40:53Z

Anyone know of a package that uses xcbuild & works in linux?

GrahamcOfBorg · 2018-06-27T20:01:11Z

Success on x86_64-darwin (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-27T20:01:36Z

Success on aarch64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

building '/nix/store/aagc2wppbsxg7bsjxxqbdfgzv97hhnw1-MacOSX.platform.drv'...
building '/nix/store/j4fbrvihslh5v9jwdl0f60jrs1j3cc6n-xcbuild-wrapper-0.1.2-pre.drv'...
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre
strip is /nix/store/0pjsgkxz0rp5baycq5sp2s72lrr5q9sg-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre/bin
patching script interpreter paths in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre
checking for references to /build in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre...
/nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-27T20:01:56Z

Success on x86_64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

building '/nix/store/h9kz2xgz2y3bb1bz355m7wgfj3pna2p6-MacOSX.platform.drv'...
building '/nix/store/vp9kiynxbhlrfr94xlr9zydrfkycdgbf-xcbuild-wrapper-0.1.2-pre.drv'...
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre/bin
patching script interpreter paths in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre
checking for references to /build in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre...
/nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-27T21:30:58Z

Success on x86_64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-27T21:31:20Z

Success on aarch64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-27T21:31:20Z

Success on x86_64-darwin (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre

Profpatsch

That’s all I have on reading through the generators.nix code.

In general, the generator seems to have some duplication, so maybe it can be cleaned up a bit.

As for the indentation trick, that’s pretty smart, I once had the same problem and arrived at a different solution, that feels more combinator-like, but was a bit brittle: https://gitlab.techcultivation.org/sangha/sangha-deployment/blob/master/containers/helpers/write-script-argparse/build-script.nix#L97
(No criticism, just for the fun of the problem).

Profpatsch · 2018-06-27T21:45:49Z

lib/generators.nix

@@ -175,4 +175,52 @@ rec {
                         else "<λ:{${showFnas}}>"
    else abort "toPretty: should never happen (v = ${v})";

+  # PLIST handling
+
+  toPLIST = x: ''


toX functions should all take an attrset before the input value, which is an attribute set of options (possibly added in the future). So

toPlist = {}: x: ''

I think plist is normally written with small letters (going by the wiki article), so I’d camel case.

The next step is thinking about all parts of the generator and moving everything that is not fixed by the generated format into an option. For example mkINI doesn’t specify how to format names in section headers (mainly what escaping is used, mkSectionName) and does not fix how key-value-pairs look, only the fact that they exist (the formatting of them is relayed to mkKeyValueDefault, another generator).

Profpatsch · 2018-06-27T21:50:49Z

lib/generators.nix

+    if isString x then pprStr ind x else
+    if isList x then pprList ind x else
+    if isAttrs x then pprAttrs ind x else
+    throw "invalid plist type";


The error message should mirror the other pattern matching error messages in this module, namely:

else abort "toPretty: should never happen (v = ${v})";

and

let err = t: v: abort ("generators.mkValueStringDefault: " + "${t} not supported: ${toPretty {} v}");

And now that I see them, they can you edit factor out the error function from the second and make all say:

"generators.${name}: this value is not supported: ${toPretty {} v}"

? :)

Profpatsch · 2018-06-27T21:50:59Z

lib/generators.nix

+
+  attrFilter = name: value: name != "_module" && value != null;
+
+  pprAttr = ind: x: libStr.concatStringsSep "\n" (lib.flatten (lib.mapAttrsToList (name: value: lib.optional (attrFilter name value) [


Line length

Profpatsch · 2018-06-27T21:51:31Z

lib/generators.nix

+  '' + pprExpr "" x
+     + "\n</plist>";
+
+  pprExpr = ind: x: with builtins;


The pprX definitions should go into a let definition at the start of toPlist, and the ppr prefixes can be removed.

LnL7

Like I mentioned on the nix-darwin pr, I'm fine with moving these to nixpkgs. Didn't realise there are some places where it could be used already.

LnL7 · 2018-06-27T20:13:19Z

lib/generators.nix

+    if isAttrs x then pprAttrs ind x else
+    throw "invalid plist type";
+
+  pprLiteral = ind: x: ind + x;


I wouldn't expose these. None of those helper functions are useful except for maybe a version of toPLIST without the header, but it should probably have a better name in that case.

matthewbauer · 2018-06-28T15:10:26Z

Ok is all caps PLIST acceptable? We can do toPlist as well if that's preferred.

GrahamcOfBorg · 2018-06-28T15:34:56Z

Success on x86_64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T15:35:02Z

Success on x86_64-darwin (full log)

Attempted: xcbuild

Partial log (click to expand)

copying path '/nix/store/va9b2y3kvyjdhx48jwz624kxbs0hs68g-gperf-3.1' from 'https://cache.nixos.org'...
copying path '/nix/store/j73q6p96fps8b0xwv7kjic9jglrafb6d-unifdef-2.6' from 'https://cache.nixos.org'...
building '/nix/store/4wnyzai7l4cmjhpi1xgd833wh45m5dxi-nixpkgs.xctoolchain.drv'...
building '/nix/store/ihiffqf9xpsq0dbkpaz8xpak91sd8xfn-xcbuild-wrapper-0.1.2-pre.drv'...
installing
post-installation fixup
strip is /nix/store/7ddbq63v97nk8gkbf7gcsfmby37h6gbl-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre/bin
patching script interpreter paths in /nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre
/nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T15:35:28Z

Success on aarch64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

building '/nix/store/aagc2wppbsxg7bsjxxqbdfgzv97hhnw1-MacOSX.platform.drv'...
building '/nix/store/j4fbrvihslh5v9jwdl0f60jrs1j3cc6n-xcbuild-wrapper-0.1.2-pre.drv'...
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre
strip is /nix/store/0pjsgkxz0rp5baycq5sp2s72lrr5q9sg-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre/bin
patching script interpreter paths in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre
checking for references to /build in /nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre...
/nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre

Profpatsch · 2018-06-28T17:30:52Z

lib/generators.nix

@@ -175,4 +175,51 @@ rec {
                         else "<λ:{${showFnas}}>"
    else abort "toPretty: should never happen (v = ${v})";

+  # PLIST handling
+  toPLIST = {}: v: let
+    pprExpr = ind: x: with builtins;


The ppr prefix is superfluous.

Profpatsch · 2018-06-28T17:32:55Z

lib/generators.nix

@@ -175,4 +175,51 @@ rec {
                         else "<λ:{${showFnas}}>"
    else abort "toPretty: should never happen (v = ${v})";

+  # PLIST handling
+  toPLIST = {}: v: let


I’m torn. When I search for it, everyone else doesn’t write PLIST, but plist. So I’d prefer camel case in this case.

GrahamcOfBorg · 2018-06-28T19:17:02Z

Success on x86_64-darwin (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T19:17:11Z

Success on aarch64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T19:17:48Z

Success on x86_64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

building '/nix/store/20qmi4wxf2vy95mwacl0qfmjvwdj77kw-nixpkgs.xctoolchain.drv'...
building '/nix/store/79900axaq56xlzh5cyhlnvn9gw2zzy3v-xcbuild-wrapper-0.1.2-pre.drv'...
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre/bin
patching script interpreter paths in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre
checking for references to /build in /nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre...
/nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre

Address PR comments Refactors - Rename toPLIST -> toPlist

matthewbauer · 2018-06-28T19:26:10Z

Ok darwin.adv_cmds still builds so I will merge for now. Will definitely accept refactors though. Also we need to make sure nix-darwin can use this successfully without issues.

GrahamcOfBorg · 2018-06-28T19:41:39Z

Success on x86_64-darwin (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/mk32zacn7i3lz9anb2xji35r0xkgv43i-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T19:41:53Z

Success on aarch64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/y22l4jf6bj44ly61kzr3dqwh02y92v2g-xcbuild-wrapper-0.1.2-pre

GrahamcOfBorg · 2018-06-28T19:43:03Z

Success on x86_64-linux (full log)

Attempted: xcbuild

Partial log (click to expand)

/nix/store/5ib31pwkvwykhybg2rc235ha6jix4irw-xcbuild-wrapper-0.1.2-pre

matthewbauer added 2 commits June 27, 2018 15:35

generators: add PLIST handling

3210dd3

/cc @LnL7 @3noch

xcbuild: rework to use lib.PLIST

c672e37

This is a good way to test that plist works. Sorry for the large diff. There are a bunch of cleanups in here that needed to be done. Should make it possible to use in cross compilation.

matthewbauer requested review from copumpkin, LnL7 and Profpatsch June 27, 2018 19:39

matthewbauer requested review from edolstra and nbp as code owners June 27, 2018 19:39

GrahamcOfBorg added 10.rebuild-darwin: 501+ 10.rebuild-linux: 1-10 labels Jun 27, 2018

matthewbauer added 2 commits June 27, 2018 16:45

unixtools: add killall

bceb29f

treewide: use killall

1614140

GrahamcOfBorg added the 8.has: package (new) label Jun 27, 2018

Profpatsch requested changes Jun 27, 2018

View reviewed changes

LnL7 reviewed Jun 28, 2018

View reviewed changes

generators: refactor toPLIST

d361371

Profpatsch requested changes Jun 28, 2018

View reviewed changes

Profpatsch approved these changes Jun 28, 2018

View reviewed changes

generators: refactor toPlist

337b589

Address PR comments Refactors - Rename toPLIST -> toPlist

matthewbauer force-pushed the upstream-plist branch from d98bc39 to 337b589 Compare June 28, 2018 19:24

matthewbauer merged commit a44d33a into NixOS:master Jun 28, 2018

matthewbauer deleted the upstream-plist branch June 28, 2018 19:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upstream PLIST handling #42669

Upstream PLIST handling #42669

matthewbauer commented Jun 27, 2018

matthewbauer commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

matthewbauer commented Jun 27, 2018 •

edited

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

Profpatsch left a comment •

edited

Profpatsch Jun 27, 2018

Profpatsch Jun 27, 2018

Profpatsch Jun 27, 2018

Profpatsch Jun 27, 2018

LnL7 left a comment

LnL7 Jun 27, 2018

matthewbauer commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

Profpatsch Jun 28, 2018

Profpatsch Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

matthewbauer commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018


		attrFilter = name: value: name != "_module" && value != null;

		pprAttr = ind: x: libStr.concatStringsSep "\n" (lib.flatten (lib.mapAttrsToList (name: value: lib.optional (attrFilter name value) [

Upstream PLIST handling #42669

Upstream PLIST handling #42669

Conversation

matthewbauer commented Jun 27, 2018

Motivation for this change

matthewbauer commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

matthewbauer commented Jun 27, 2018 • edited

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

GrahamcOfBorg commented Jun 27, 2018

Profpatsch left a comment • edited

Choose a reason for hiding this comment

Profpatsch Jun 27, 2018

Choose a reason for hiding this comment

Profpatsch Jun 27, 2018

Choose a reason for hiding this comment

Profpatsch Jun 27, 2018

Choose a reason for hiding this comment

Profpatsch Jun 27, 2018

Choose a reason for hiding this comment

LnL7 left a comment

Choose a reason for hiding this comment

LnL7 Jun 27, 2018

Choose a reason for hiding this comment

matthewbauer commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

Profpatsch Jun 28, 2018

Choose a reason for hiding this comment

Profpatsch Jun 28, 2018

Choose a reason for hiding this comment

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

matthewbauer commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

GrahamcOfBorg commented Jun 28, 2018

matthewbauer commented Jun 27, 2018 •

edited

Profpatsch left a comment •

edited