Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS closure-size reduction #39940

Merged
merged 5 commits into from May 6, 2018
Merged

macOS closure-size reduction #39940

merged 5 commits into from May 6, 2018

Conversation

matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented May 3, 2018

These are a couple of changes to reduce closure size. Based on the output of nix-store -qR $(nix-build -A nix) | xargs du -sch | sort -h:

68K     /nix/store/3l6nwsgsp4jyii72qmjzg550myc12kkm-bzip2-1.0.6.0.1-bin
80K     /nix/store/brsvr29c2w07qf6yjrq4n5k0fma6461d-bzip2-1.0.6.0.1
88K     /nix/store/7l324p6dmgxvv5dyi89l3hljbk1l6zhz-nix-2.0.1-man
100K    /nix/store/27jk2cnq6x7ls12cdn4awmzgfx281hyg-zlib-1.2.11
144K    /nix/store/ig0dykdy3vhkj7fj5sy0avyc68sj759i-xz-5.2.3-bin
152K    /nix/store/8s3dnrs0nvn7r2x07p6557xvjilm7qz8-xz-5.2.3
152K    /nix/store/bgg5gwlp86lqrcrxybvpw18sbc34i4v1-nghttp2-1.24.0-lib
168K    /nix/store/a2p84va4w4g7cjqqx1icmfdqiqqnfcmp-libssh2-1.8.0
176K    /nix/store/q41xv3gbh3ayfaxjj6hzwkszz7kilyks-gzip-1.9
212K    /nix/store/jhm3ari7cfrznp6knfiqq6873zw760bm-boehm-gc-7.6.4
564K    /nix/store/slsk698k0z9c8gqqdjnmivahqnch9pd4-curl-7.59.0
604K    /nix/store/m8z5wglaz4pdrrjxynq2q50r1bhfxb32-libsodium-1.0.16
644K    /nix/store/60ww1ja7av5lnm584qray3wnckrn1c4p-gmp-6.1.2
688K    /nix/store/l3g88gagk2a2y7grzp9w6jkj4ymzpqdd-libc++abi-5.0.1
808K    /nix/store/877d5gk4mzgw8q22nzwx40g502f5ri5b-libatomic_ops-7.6.2
1.1M    /nix/store/4cq7zlsqbs3l2lf7bb3vr03xxf0v2czz-bash-4.4-p12
1.1M    /nix/store/h90rp179q22r68cwmm7jjl4sphvdh9ym-libiconv-osx-10.11.6
1.5M    /nix/store/nvn4c4g2ydlrr8mgb8b6f79qi26w48w2-sqlite-3.21.0
1.6M    /nix/store/vn99garaqrg51xlb0qq4zzdrnmddsyph-brotli-1.0.3-lib
1.7M    /nix/store/2k9v6cha9r6ikqn9iyp6frqlc8kl1l5k-libkrb5-1.15.2
1.7M    /nix/store/dpi4c08jkfqbpsp2y8fd1wx9pjfciza6-coreutils-8.29
2.5M    /nix/store/5zddplqzjz4cvnirdnnvlv48y2m6kjz4-CF-osx-10.10.5
2.6M    /nix/store/qqp1h7sksdlwz2w4kg5z0k92wxbdx7wm-gnutar-1.30
3.5M    /nix/store/6jgixnd9szvwidp9zbnr61kx0xsm0b5p-openssl-1.0.2o
3.7M    /nix/store/j9vqw6z1g563pr2vj79ff9wqcr77v7q1-aws-sdk-cpp-1.3.22
6.2M    /nix/store/db6vf9amv937fqmmfzayi1sqa5xrhj0f-nix-2.0.1
6.8M    /nix/store/ggbd4h41rnvgsvjyyqd0y73jnmqzvp8r-libc++-5.0.1
9.1M    /nix/store/cmm30wfd981rx7bfm98mnamlgabpdps3-gettext-0.19.8
9.3M    /nix/store/46qj01lcjz48wb4hc7mvn3mhb3syda7h-Libsystem-osx-10.11.6
31M     /nix/store/pbn1p52nckwckwx0320hsj8vk3vnbkrs-ICU-osx-10.10.5
88M     total

should reduce:

  • ICU
  • libSystem
  • gettext

(also I wonder if aws-sdk-cpp should be disabled by default? it's huge & not necessary for regular Nix users)

Note - I don't have the time to build this right now. I think they are pretty straightforward changes though & hopefully ofborg will succeed.

This is a huge directory & not referenced anywhere.
adds 2 outputs for libSystem:

- out
- dev

should shrink closure sizes
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gettext

The following builds were skipped because they don't evaluate on x86_64-linux: darwin.ICU

Partial log (click to expand)

/nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc/share/doc/gettext/examples/hello-tcl-tk/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/n21jinak0jj40sfqnkbbrzv04anhvwa6-bash-4.4-p19/bin/sh"
/nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc/share/doc/gettext/examples/hello-perl/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/n21jinak0jj40sfqnkbbrzv04anhvwa6-bash-4.4-p19/bin/sh"
/nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc/share/doc/gettext/examples/hello-perl/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/n21jinak0jj40sfqnkbbrzv04anhvwa6-bash-4.4-p19/bin/sh"
/nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc/share/doc/gettext/examples/hello-php/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/n21jinak0jj40sfqnkbbrzv04anhvwa6-bash-4.4-p19/bin/sh"
/nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc/share/doc/gettext/examples/hello-php/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/n21jinak0jj40sfqnkbbrzv04anhvwa6-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/mpvmw5xj8flz1z3yx1qfihg9w2b4bma9-gettext-0.19.8-doc...
shrinking RPATHs of ELF executables and libraries in /nix/store/f6zr9csa6xg3vqjdq1x13mj7hgn42l76-gettext-0.19.8-info
strip is /nix/store/dk3mznwzzg2szs9j822821lqvih9n394-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/f6zr9csa6xg3vqjdq1x13mj7hgn42l76-gettext-0.19.8-info
checking for references to /build in /nix/store/f6zr9csa6xg3vqjdq1x13mj7hgn42l76-gettext-0.19.8-info...

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: darwin.ICU, gettext

Partial log (click to expand)

cannot build derivation '/nix/store/lapwccz09d3qihp0cc46g31da8fs6l6m-perl-5.24.3.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/w41wms11iw8v5rybsrjb0awanfbs9fdx-autoconf-2.69.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/1mgbq1r7x38ql8xiz0wys8jllgrx783l-gettext-0.19.8.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/f4qpzqp5w2yczg3z2b03cs598hq6y3zy-automake-1.16.1.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/9pj7hvbyqq7bkm8rs1kxb02bsp61gigd-perl-gettext-1.07.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/nf382bgxa7zwfvrvnjp739kyvgks9ilx-help2man-1.47.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/zr1j64aqjjbv62c7ys4lk1yglcdgwj9j-libtool-2.4.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/8r99c400kp0klj3b6lv34klim1y10wcw-cctools-port-895.drv': 7 dependencies couldn't be built
cannot build derivation '/nix/store/7kszlq9j40x6bzqfs3m3dc7p3y1fz2fz-ICU-osx-10.10.5.drv': 3 dependencies couldn't be built
error: build of '/nix/store/1mgbq1r7x38ql8xiz0wys8jllgrx783l-gettext-0.19.8.drv', '/nix/store/7kszlq9j40x6bzqfs3m3dc7p3y1fz2fz-ICU-osx-10.10.5.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gettext

The following builds were skipped because they don't evaluate on aarch64-linux: darwin.ICU

Partial log (click to expand)

/nix/store/dg9n8k74w3x2rvbgg8jjb32j79xdv60x-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c-gnome/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/9zjrdnwcab54f7azxx0gafwwxb71x1gr-bash-4.4-p19/bin/sh"
/nix/store/dg9n8k74w3x2rvbgg8jjb32j79xdv60x-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c-gnome/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/9zjrdnwcab54f7azxx0gafwwxb71x1gr-bash-4.4-p19/bin/sh"
/nix/store/dg9n8k74w3x2rvbgg8jjb32j79xdv60x-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/9zjrdnwcab54f7azxx0gafwwxb71x1gr-bash-4.4-p19/bin/sh"
/nix/store/dg9n8k74w3x2rvbgg8jjb32j79xdv60x-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/9zjrdnwcab54f7azxx0gafwwxb71x1gr-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/dg9n8k74w3x2rvbgg8jjb32j79xdv60x-gettext-0.19.8-doc...
shrinking RPATHs of ELF executables and libraries in /nix/store/2q8f3hgbyal6b1c4yr6szin65aj726cr-gettext-0.19.8-info
strip is /nix/store/ly95c6g0fijn16hhvzz8y3a0ap3qdyh1-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/2q8f3hgbyal6b1c4yr6szin65aj726cr-gettext-0.19.8-info
checking for references to /build in /nix/store/2q8f3hgbyal6b1c4yr6szin65aj726cr-gettext-0.19.8-info...
/nix/store/q2hfprm39kj6pd9341v4dlqc0d0frr78-gettext-0.19.8

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: darwin.ICU, gettext

Partial log (click to expand)

cannot build derivation '/nix/store/ir63hjch8x19nhh8q9rmbrhw27s5vs9q-perl-5.24.3.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/3w9pndkc9a1wcir9v00yq19j4zdkri2q-autoconf-2.69.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/592jxq02jbpd0z34hvza0jz88phzmhs9-gettext-0.19.8.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/gjbgia5l5v4ygql9kdw7jxl4a8p80jzf-automake-1.16.1.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/rpfjzdp1m27w2d53jyvxziif8lq020ah-perl-gettext-1.07.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/ffn56aff5han9ca4qjv3vybqcq7z6v0m-help2man-1.47.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/k7jxi9wxkmqms0p1zl69ran86vgms1fn-libtool-2.4.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/x0hjbsynbmrjh1x5zn02kh45g6wl4k58-cctools-port-895.drv': 7 dependencies couldn't be built
cannot build derivation '/nix/store/rxz5wrapgcyababa7ys29v5yw4ji88df-ICU-osx-10.10.5.drv': 3 dependencies couldn't be built
error: build of '/nix/store/592jxq02jbpd0z34hvza0jz88phzmhs9-gettext-0.19.8.drv', '/nix/store/rxz5wrapgcyababa7ys29v5yw4ji88df-ICU-osx-10.10.5.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: gettext

The following builds were skipped because they don't evaluate on x86_64-linux: darwin.ICU

Partial log (click to expand)

cannot build derivation '/nix/store/74w3l5jb2y8axcc6xrc29qkjpdxa1paf-lzip-1.20.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/lcrs2a2n8yrfkhwasjh00rfsc6w1f39f-binutils-wrapper-2.30.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/blhw3qns1zkkbvbzcxz7kfkymnylgi5m-diffutils-3.6.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/p5hivm9iiybrifmsjjf70l7l5fp0l4wf-ed-1.14.2.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/8cbsx3djjfpjhmj1pksc8kjp0j8kiwpw-findutils-4.6.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/j08fpr1zky8hcx83zsx5rvmgiwy9zw9j-gcc-wrapper-7.3.0.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/98sidgvmb6p08qxfg2nvksn8zzbfnmvn-patch-2.7.6.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/ir29y382nss1wax6j1dgky5y039naqwc-stdenv-linux.drv': 24 dependencies couldn't be built
cannot build derivation '/nix/store/1vcz1wsczm02pqdzi0hficiqx4v7algh-gettext-0.19.8.drv': 3 dependencies couldn't be built
error: build of '/nix/store/1vcz1wsczm02pqdzi0hficiqx4v7algh-gettext-0.19.8.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gettext

The following builds were skipped because they don't evaluate on aarch64-linux: darwin.ICU

Partial log (click to expand)

/nix/store/vm8hpkm745hq44jclmis86r1j4n56450-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c-gnome/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/f0s8jwm3ks566k5dlyr5nmi5j1ghnji1-bash-4.4-p19/bin/sh"
/nix/store/vm8hpkm745hq44jclmis86r1j4n56450-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c-gnome/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/f0s8jwm3ks566k5dlyr5nmi5j1ghnji1-bash-4.4-p19/bin/sh"
/nix/store/vm8hpkm745hq44jclmis86r1j4n56450-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/f0s8jwm3ks566k5dlyr5nmi5j1ghnji1-bash-4.4-p19/bin/sh"
/nix/store/vm8hpkm745hq44jclmis86r1j4n56450-gettext-0.19.8-doc/share/doc/gettext/examples/hello-c/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/f0s8jwm3ks566k5dlyr5nmi5j1ghnji1-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/vm8hpkm745hq44jclmis86r1j4n56450-gettext-0.19.8-doc...
shrinking RPATHs of ELF executables and libraries in /nix/store/0rw3w6b0yx0k52vjbsn92r8dibzz4pvy-gettext-0.19.8-info
strip is /nix/store/gfgsq9kswkp9779rj84nmql5qxkj6rxp-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/0rw3w6b0yx0k52vjbsn92r8dibzz4pvy-gettext-0.19.8-info
checking for references to /build in /nix/store/0rw3w6b0yx0k52vjbsn92r8dibzz4pvy-gettext-0.19.8-info...
/nix/store/z5pdzwf6s1cj8vfqsrb112n2ccrkih45-gettext-0.19.8

@@ -3,10 +3,13 @@
removefile, libresolv, Libnotify, libplatform, libpthread, mDNSResponder, launchd, libutil, version }:

appleDerivation rec {
phases = [ "unpackPhase" "installPhase" ];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should get rid of these in the other apple stuff, it can be pretty confusing when stuff like multiple outputs don't work if you don't think of this immediately.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! I think they can all be either removed or replaced with dontBuild = true & dontConfigure = true. I'm never sure whether they're actually needed though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was all clueless @copumpkin right at the beginning of his Nix career not knowing what he's doing. And yes, he occasionally speaks of himself in the third person.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's good to know! Occasionally it's necessary, for instance when a Makefile exists, but I'll just remove all of them & see what breaks.

@GrahamcOfBorg GrahamcOfBorg added the 6.topic: stdenv Standard environment label May 3, 2018
@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gettext, stdenv

The following builds were skipped because they don't evaluate on aarch64-linux: darwin.ICU

Partial log (click to expand)

  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.


/nix/store/z5pdzwf6s1cj8vfqsrb112n2ccrkih45-gettext-0.19.8
/nix/store/hx7dnwzz7m3llssvnkvjl3nidly30pwd-stdenv-linux

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gettext, stdenv

The following builds were skipped because they don't evaluate on x86_64-linux: darwin.ICU

Partial log (click to expand)

/nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc/share/doc/gettext/examples/hello-tcl-tk/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/cp5bk9wqg0s8y5w3xic1wqzbsjh0wf41-bash-4.4-p19/bin/sh"
/nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc/share/doc/gettext/examples/hello-perl/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/cp5bk9wqg0s8y5w3xic1wqzbsjh0wf41-bash-4.4-p19/bin/sh"
/nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc/share/doc/gettext/examples/hello-perl/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/cp5bk9wqg0s8y5w3xic1wqzbsjh0wf41-bash-4.4-p19/bin/sh"
/nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc/share/doc/gettext/examples/hello-php/autogen.sh: interpreter directive changed from "/bin/sh" to "/nix/store/cp5bk9wqg0s8y5w3xic1wqzbsjh0wf41-bash-4.4-p19/bin/sh"
/nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc/share/doc/gettext/examples/hello-php/autoclean.sh: interpreter directive changed from "/bin/sh" to "/nix/store/cp5bk9wqg0s8y5w3xic1wqzbsjh0wf41-bash-4.4-p19/bin/sh"
checking for references to /build in /nix/store/arfps9lfd220wmrfsc0bfxnfvwi26a4m-gettext-0.19.8-doc...
shrinking RPATHs of ELF executables and libraries in /nix/store/bd4133nrszq50pd0s5k06ygrcwxla3k7-gettext-0.19.8-info
strip is /nix/store/p8gfvvsf6fc12rzbcmsacgim10nr57di-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/bd4133nrszq50pd0s5k06ygrcwxla3k7-gettext-0.19.8-info
checking for references to /build in /nix/store/bd4133nrszq50pd0s5k06ygrcwxla3k7-gettext-0.19.8-info...

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: darwin.ICU, gettext, stdenv

Partial log (click to expand)

cannot build derivation '/nix/store/sismm6b5kwa9jf5f9vaslm6c18bd9xdl-perl-5.24.3.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/0qkl01g1my1x9ydrr9s11a0v1xna0vn6-autoconf-2.69.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/1cnyrg9ki6nq2vfgcd0x0s79fsywlssv-gettext-0.19.8.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/gwh4yic4bqh0y4114xidq61jsw9p0rhg-automake-1.16.1.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/xaih0fhci8ji2c0mqqirgh2ligqp8dym-perl-gettext-1.07.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/9sy3y4dvzgr8kgpgd3kwmh7cw5grzrwk-help2man-1.47.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/09ak1qrmig6m7933wpdjwfysbdrbfxba-libtool-2.4.6.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/z9saq6ic1iiqhzc2n5rcrq7jxdz4vkz7-cctools-port-895.drv': 7 dependencies couldn't be built
cannot build derivation '/nix/store/qz2p1r8k0wacmqmf3hn80ila1nsxy6cv-ICU-osx-10.10.5.drv': 3 dependencies couldn't be built
error: build of '/nix/store/1cnyrg9ki6nq2vfgcd0x0s79fsywlssv-gettext-0.19.8.drv', '/nix/store/qz2p1r8k0wacmqmf3hn80ila1nsxy6cv-ICU-osx-10.10.5.drv', '/nix/store/xxywm51y4r56crmzcy0rkyplkn3idx9b-stdenv-darwin.drv' failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants