linbox: Add darwin support #45015

jbaum98 · 2018-08-14T06:32:24Z

Motivation for this change

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

LnL7 · 2018-08-14T06:41:24Z

pkgs/development/libraries/linbox/default.nix

    homepage = http://linalg.org/;
  };
-}
+} // (if stdenv.isDarwin then {


No need to make this conditional, add a comment instead.

mpickering · 2018-08-14T18:39:02Z

@GrahamcOfBorg build linbox

GrahamcOfBorg · 2018-08-14T18:39:27Z

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: linbox

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-08-14T18:42:17Z

Success on x86_64-linux (full log)

Attempted: linbox

Partial log (click to expand)

make[2]: Entering directory '/build/source/examples/data'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/build/source/examples/data'
make[2]: Entering directory '/build/source/examples'
make[2]: Nothing to be done for 'check-am'.
make[2]: Leaving directory '/build/source/examples'
make[1]: Leaving directory '/build/source/examples'
make[1]: Entering directory '/build/source'
make[1]: Leaving directory '/build/source'
/nix/store/6mhrf02f0mjybdxnlzw6s8bpw3ga2704-linbox-1.5.2

GrahamcOfBorg · 2018-08-14T18:51:07Z

Success on aarch64-linux (full log)

Attempted: linbox

Partial log (click to expand)

make[2]: Entering directory '/build/source/examples/data'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/build/source/examples/data'
make[2]: Entering directory '/build/source/examples'
make[2]: Nothing to be done for 'check-am'.
make[2]: Leaving directory '/build/source/examples'
make[1]: Leaving directory '/build/source/examples'
make[1]: Entering directory '/build/source'
make[1]: Leaving directory '/build/source'
/nix/store/hj4ffhyy5y65s2ch8a80xwy8qxv4nrkc-linbox-1.5.2

mpickering · 2018-08-14T18:55:27Z

This depends on #45001

timokau · 2018-08-14T20:11:23Z

pkgs/development/libraries/linbox/default.nix

+
+  # needed for darwin
+  doInstallCheck = true;
+  installCheckTarget = "check";


Same as #45001, please make the reason clear in the comment and set doCheck = false. Maybe you should report the issue upstream.

You can then add a link to the upstream issue to the comment.

See #44248, this is a pretty common problem for packages that test their own libraries before they are installed.

That is unfortunate. So it is a nix problem, not a linbox problem? Shouldn't that be solveable through DYLD_LIBRARY_PATH in a general manner?

I personally think that this is a much better test, messing with LD_LIBRARY_PATH or rpaths like what happens on linux provides no guarantee that the libraries actually work properly once installed.

But this doesn't either: It still checks the built binary, not the installed one.

If a package has proper install checks and we someday default doInstallCheck = true, they wouldn't be executed in this case. Also it requires a bunch of manual changes.

In general I agree that install checks are more useful than build checks for packaging.

No this is the opposite: it's testing against the installed binary even though the tests want to test against the built binary. Even though the Makefile explicitly passes flags to point the tests at the built binary, Nix must be overriding them to point it at the installed binary because in general it wants everything to point there. So I think doInstallCheck is fine.

Did you test that by removing the build results before running the tests? I would be surprised if nix did anything behind the scenes to make it use the installed binary. I would think that the built binary just looks for the installed library.

Anyways, thanks for adding support! Although I don't entirely like the InstallCheck result, I think with the comment it is a worthy tradeoff for darwin support.

timokau · 2018-08-15T09:58:24Z

@GrahamcOfBorg eval
@GrahamcOfBorg build linbox

GrahamcOfBorg · 2018-08-15T09:59:05Z

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: linbox

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

timokau · 2018-08-15T10:05:13Z

Depends on #45013

GrahamcOfBorg · 2018-08-15T10:20:15Z

Success on x86_64-linux (full log)

Attempted: linbox

Partial log (click to expand)

make[2]: Entering directory '/build/source/examples/data'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/build/source/examples/data'
make[2]: Entering directory '/build/source/examples'
make[2]: Nothing to be done for 'check-am'.
make[2]: Leaving directory '/build/source/examples'
make[1]: Leaving directory '/build/source/examples'
make[1]: Entering directory '/build/source'
make[1]: Leaving directory '/build/source'
/nix/store/i5snakcn4c4qarqrdh02vmgsf2zlhh9m-linbox-1.5.2

GrahamcOfBorg · 2018-08-15T10:58:48Z

Timed out, unknown build status on aarch64-linux (full log)

Attempted: linbox

Partial log (click to expand)

libtool: link: g++ -O2 -Wall -g -I.. -I/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/include -fabi-version=6 -fopenmp -o test-solve test-solve.o -fopenmp  -L/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib /nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib/libgivaro.so -lm /nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib/libgmpxx.so /nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib/libgmp.so /nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64/libstdc++.so -lopenblas -fopenmp -Wl,-rpath -Wl,/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib -Wl,-rpath -Wl,/nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib -Wl,-rpath -Wl,/nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64 -Wl,-rpath -Wl,/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib -Wl,-rpath -Wl,/nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib -Wl,-rpath -Wl,/nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64
PASS: test-solve
g++ -DHAVE_CONFIG_H -I. -I..    -O2  -Wall -g  -I..  -I/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/include -fabi-version=6 -fopenmp -c -o test-fgemv.o test-fgemv.C
/nix/store/7q3ayirslrcva28wava6zpjcflcz1h3b-bash-4.4-p23/bin/bash ../libtool  --tag=CXX   --mode=link g++ -O2  -Wall -g  -I..  -I/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/include -fabi-version=6 -fopenmp   -o test-fgemv test-fgemv.o  -L/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib -lgivaro -lgmp -lgmpxx  -fopenmp -lopenblas
libtool: link: g++ -O2 -Wall -g -I.. -I/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/include -fabi-version=6 -fopenmp -o test-fgemv test-fgemv.o -fopenmp  -L/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib /nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib/libgivaro.so -lm /nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib/libgmpxx.so /nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib/libgmp.so /nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64/libstdc++.so -lopenblas -fopenmp -Wl,-rpath -Wl,/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib -Wl,-rpath -Wl,/nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib -Wl,-rpath -Wl,/nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64 -Wl,-rpath -Wl,/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/lib -Wl,-rpath -Wl,/nix/store/qmaa42qqjq97n5n9j0p43ps97rb2qaiz-gmp-6.1.2/lib -Wl,-rpath -Wl,/nix/store/ri3iwhv860fq7nkcyc8aqj586idj87p2-gcc-7.3.0-lib/lib/../lib64
PASS: test-fgemv
g++ -DHAVE_CONFIG_H -I. -I..    -O2  -Wall -g  -I..  -I/nix/store/7xhkxgq1nnq9rmqwjlqw8yqfcs0g4z1z-givaro-4.0.4/include -fabi-version=6 -fopenmp -c -o regression-check.o regression-check.C
building of '/nix/store/dz69pjwjyigxpqkwa51qr8cgln0f7nhv-fflas-ffpack-2.3.2.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/6aw3qsc5i7yy79hp6zrbcx8chpg1jw1x-linbox-1.5.2.drv': 1 dependencies couldn't be built
error: build of '/nix/store/6aw3qsc5i7yy79hp6zrbcx8chpg1jw1x-linbox-1.5.2.drv' failed

jbaum98 · 2018-08-17T03:08:06Z

Depends on #45013.

timokau · 2018-08-19T14:27:38Z

Please rebase and remove the wip tag if appropriate.

Out of curiosity: Do you know how many more sage dependencies need fixing?

timokau · 2018-08-19T14:28:13Z

Also best leave a comment when you're done, github is really bad at notifications with stuff other than comments.

jbaum98 · 2018-08-19T14:39:06Z

@timokau All done.

timokau · 2018-08-19T14:48:44Z

@GrahamcOfBorg build linbox

GrahamcOfBorg · 2018-08-19T14:48:52Z

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: linbox

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

timokau · 2018-08-19T14:49:53Z

Does this really suffer the same blas issue as fflas_ffpack or do you just avoid the dependency on two different versions of blas?

GrahamcOfBorg · 2018-08-19T14:51:52Z

Success on x86_64-linux (full log)

Attempted: linbox

Partial log (click to expand)

make[1]: Leaving directory '/build/source'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2
shrinking /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/lib/liblinbox.so.0.0.0
gzipping man pages under /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/share/man/
strip is /nix/store/553rihc190vsyy8b22iqcq25a6489h8y-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/lib  /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/bin
patching script interpreter paths in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2
/nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/bin/linbox-config: interpreter directive changed from " /bin/bash" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/bash"
checking for references to /build in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2...

timokau · 2018-08-19T14:52:58Z

looks like you forgot to update platforms.

jbaum98 · 2018-08-19T14:57:04Z

This does suffer from the same issue, building without this commit but with the fflas-ffpack commit fails tests.

I don't know how many more sage dependencies need fixing, I just keep rebuilding and fixing what fails.

GrahamcOfBorg · 2018-08-19T15:02:08Z

Success on aarch64-linux (full log)

Attempted: linbox

Partial log (click to expand)

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2
shrinking /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2/lib/liblinbox.so.0.0.0
gzipping man pages under /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2/share/man/
strip is /nix/store/4md2i310zklkkl5j41yw70gcwgn4kav5-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2/lib  /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2/bin
patching script interpreter paths in /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2
/nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2/bin/linbox-config: interpreter directive changed from " /bin/bash" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/bash"
checking for references to /build in /nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2...
/nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2

timokau · 2018-08-19T15:44:25Z

Okay, interesting. Maybe its just because of the indirect mixing of the two blas versions, maybe because the blas issue is more widespread and not an fflas_ffpack issue after all.

Okay, excited to see sage build on darwin :) When you fix the platform list this should be ready for merge.

jbaum98 · 2018-08-19T18:03:59Z

@timokau Updated the platform list.

timokau · 2018-08-19T18:21:38Z

@GrahamcOfBorg build linbox

GrahamcOfBorg · 2018-08-19T18:24:19Z

Success on aarch64-linux (full log)

Attempted: linbox

Partial log (click to expand)

/nix/store/qp4hm51i8jd0q4dy6zd85h9qr6qc88lw-linbox-1.5.2

GrahamcOfBorg · 2018-08-19T18:28:48Z

Success on x86_64-darwin (full log)

Attempted: linbox

Partial log (click to expand)

 /nix/store/vc4q7zqpxx15ckxshdxikf7c6vq57c5k-coreutils-8.29/bin/mkdir -p '/nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/lib/pkgconfig'
 /nix/store/vc4q7zqpxx15ckxshdxikf7c6vq57c5k-coreutils-8.29/bin/install -c -m 644 linbox.pc '/nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/lib/pkgconfig'
make[2]: Leaving directory '/private/tmp/nix-build-linbox-1.5.2.drv-0/source'
make[1]: Leaving directory '/private/tmp/nix-build-linbox-1.5.2.drv-0/source'
post-installation fixup
gzipping man pages under /nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/share/man/
strip is /nix/store/bg39mxc401qqxgjasxfgardi6xl2ikmc-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/lib  /nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/bin
patching script interpreter paths in /nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2
/nix/store/1cvivvn9sskw5jn4wi9ksr84bkyqsh0z-linbox-1.5.2/bin/linbox-config: interpreter directive changed from " /bin/bash" to "/nix/store/avxlkpsr36xpnsy0byq2c4vxk6sw157a-bash-4.4-p23/bin/bash"

GrahamcOfBorg · 2018-08-19T18:40:40Z

Success on x86_64-linux (full log)

Attempted: linbox

Partial log (click to expand)

make[1]: Leaving directory '/build/source'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2
shrinking /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/lib/liblinbox.so.0.0.0
gzipping man pages under /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/share/man/
strip is /nix/store/553rihc190vsyy8b22iqcq25a6489h8y-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/lib  /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/bin
patching script interpreter paths in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2
/nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2/bin/linbox-config: interpreter directive changed from " /bin/bash" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/bash"
checking for references to /build in /nix/store/jljsy1nd38xffi0601jw7h9smy4jhx27-linbox-1.5.2...

timokau · 2018-08-19T18:41:58Z

Thanks!

GrahamcOfBorg added 6.topic: darwin Running or building packages on Darwin 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Aug 14, 2018

LnL7 reviewed Aug 14, 2018

View reviewed changes

jbaum98 force-pushed the linxbox_darwin branch from 5b08abd to 7e3c476 Compare August 14, 2018 14:22

timokau reviewed Aug 14, 2018

View reviewed changes

jbaum98 force-pushed the linxbox_darwin branch from 7e3c476 to b1eefe2 Compare August 14, 2018 23:52

jbaum98 force-pushed the linxbox_darwin branch from b1eefe2 to 30743bf Compare August 17, 2018 03:06

jbaum98 force-pushed the linxbox_darwin branch from 30743bf to afbe19d Compare August 17, 2018 19:23

jbaum98 changed the title ~~linbox: Add darwin support~~ [WIP] linbox: Add darwin support Aug 17, 2018

jbaum98 force-pushed the linxbox_darwin branch from afbe19d to 08c0928 Compare August 18, 2018 20:54

jbaum98 changed the title ~~[WIP] linbox: Add darwin support~~ linbox: Add darwin support Aug 19, 2018

jbaum98 force-pushed the linxbox_darwin branch from 08c0928 to fe23b1a Compare August 19, 2018 14:38

GrahamcOfBorg added 10.rebuild-darwin: 0 and removed 10.rebuild-darwin: 1-10 labels Aug 19, 2018

linbox: Add darwin support

b99b7c5

jbaum98 force-pushed the linxbox_darwin branch from fe23b1a to b99b7c5 Compare August 19, 2018 17:56

GrahamcOfBorg added 10.rebuild-darwin: 1-10 and removed 10.rebuild-darwin: 0 labels Aug 19, 2018

timokau merged commit 2eed806 into NixOS:master Aug 19, 2018

linbox: Add darwin support #45015

linbox: Add darwin support #45015

Conversation

jbaum98 commented Aug 14, 2018

Motivation for this change

Things done

Choose a reason for hiding this comment

mpickering commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

GrahamcOfBorg commented Aug 14, 2018

mpickering commented Aug 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LnL7 Aug 14, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timokau commented Aug 15, 2018

GrahamcOfBorg commented Aug 15, 2018

timokau commented Aug 15, 2018

GrahamcOfBorg commented Aug 15, 2018

GrahamcOfBorg commented Aug 15, 2018

jbaum98 commented Aug 17, 2018

timokau commented Aug 19, 2018

timokau commented Aug 19, 2018

jbaum98 commented Aug 19, 2018

timokau commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

timokau commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

timokau commented Aug 19, 2018

jbaum98 commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

timokau commented Aug 19, 2018

jbaum98 commented Aug 19, 2018

timokau commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

timokau commented Aug 19, 2018

LnL7 Aug 14, 2018 •

edited