slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604 #42536

bhipple · 2018-06-24T20:52:11Z

This commit also swaps to use the official repository's github release tags
instead of their download site, which only keeps the most recent version with no
historical archives.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

This commit also swaps to use the official repository's github release tags instead of their download site, which only keeps the most recent version with no historical archives.

bhipple · 2018-06-24T20:53:20Z

@GrahamcOfBorg build slurm pythonPackages.pyslurm

bhipple · 2018-06-24T20:53:53Z

CC maintainers @veprbl @jagajaga @markuskowa

bhipple · 2018-06-24T20:55:42Z

BTW it looks like pyslurm isn't running its test cases in the nix build with setup.py. It looks like upstream ships a ./pyslurm/scripts/run_tests.sh that we could maybe invoke instead of the standard test run?

EDIT: Give me a minute, I'll see if I can get it running with a custom checkPhase.

GrahamcOfBorg · 2018-06-24T20:57:15Z

Success on x86_64-linux (full log)

Attempted: slurm, pythonPackages.pyslurm

Partial log (click to expand)

   slurmdb_tres_rec_t __pyx_v_acct_tres_rec;
                      ^~~~~~~~~~~~~~~~~~~~~
gcc -pthread -shared -lgcc_s build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o -L -L/slurm -L/nix/store/d3aa78jf7pm9lk6ad67x4f21a068dxn6-python-2.7.15/lib -Wl,-R/ -Wl,-R/slurm -lslurmdb -lslurm -lpython2.7 -o /build/source/pyslurm/pyslurm.so

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7
/nix/store/3nhzkx90vyi2jvc5v3anag04p8vlg44b-python2.7-pyslurm-20180604

bhipple · 2018-06-24T20:57:16Z

pkgs/servers/computing/slurm/default.nix

@@ -8,11 +8,16 @@

 stdenv.mkDerivation rec {
  name = "slurm-${version}";
-  version = "17.11.5";
+  version = "17.11.7";



The current slurm build doesn't work if you try to rebuild from source, since the upstream tarball has disappeared after schedmd swapped it to the latest release.

bhipple · 2018-06-24T21:20:43Z

pkgs/development/python-modules/pyslurm/default.nix

+  checkInputs = [ nose ];
+  checkPhase = ''
+    nosetests -v
+  '';


When we're not running tests, do we prefer to leave the config showing how you would run tests if you had a system with all the pre-requisites setup, or do we prefer to delete it all to keep the default.nix lean?

Delete of course. Nobody is going to maintain that.

You could run such a test, for example, in nixos/tests/slurm (assuming that the test suite is included in the pyslurm installation).

GrahamcOfBorg · 2018-06-24T21:21:58Z

Success on x86_64-linux (full log)

Attempted: slurm

Partial log (click to expand)

/nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7

GrahamcOfBorg · 2018-06-24T21:22:15Z

Success on aarch64-linux (full log)

Attempted: slurm

Partial log (click to expand)

/nix/store/bj6xwprq26l0b4mwyx96yam0i9xl1af0-slurm-17.11.7

GrahamcOfBorg · 2018-06-24T21:52:06Z

Success on aarch64-linux (full log)

Attempted: slurm

Partial log (click to expand)

/nix/store/bj6xwprq26l0b4mwyx96yam0i9xl1af0-slurm-17.11.7

GrahamcOfBorg · 2018-06-24T21:52:07Z

Success on x86_64-linux (full log)

Attempted: slurm

Partial log (click to expand)

/nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7

GrahamcOfBorg · 2018-06-24T21:53:52Z

Failure on aarch64-linux (full log)

Attempted: slurm, pythonPackages.pyslurm

Partial log (click to expand)

In file included from /nix/store/cxp5iq3352qr87353d134yiaccfq7a35-python2.7-numpy-1.14.5/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1816:0,
                 from /nix/store/cxp5iq3352qr87353d134yiaccfq7a35-python2.7-numpy-1.14.5/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                 from /nix/store/cxp5iq3352qr87353d134yiaccfq7a35-python2.7-numpy-1.14.5/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from memoryviewattrs.cpp:581:
/nix/store/cxp5iq3352qr87353d134yiaccfq7a35-python2.7-numpy-1.14.5/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^~~~~~~
building of '/nix/store/q33v4a671f67g92k5dd8yfc721a9mxxc-python2.7-Cython-0.28.3.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/7qci1ja5livz2ddzv9026byb97p1gkr3-python2.7-pyslurm-20180604.drv': 1 dependencies couldn't be built
error: build of '/nix/store/7qci1ja5livz2ddzv9026byb97p1gkr3-python2.7-pyslurm-20180604.drv' failed

GrahamcOfBorg · 2018-06-24T22:29:29Z

Success on aarch64-linux (full log)

Attempted: slurm

Partial log (click to expand)

/nix/store/bj6xwprq26l0b4mwyx96yam0i9xl1af0-slurm-17.11.7

GrahamcOfBorg · 2018-06-24T22:31:34Z

Success on x86_64-linux (full log)

Attempted: slurm

Partial log (click to expand)

gzipping man pages under /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/share/man/
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/lib  /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/bin  /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/sbin
patching script interpreter paths in /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7
checking for references to /build in /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7...
moving /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/sbin/* to /nix/store/mh9pkscw7lpxhw0wxjj8ncxln11grra3-slurm-17.11.7/bin
shrinking RPATHs of ELF executables and libraries in /nix/store/g2pgqw6prnrvdhyamjqlppx2h9sl2277-slurm-17.11.7-dev
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/g2pgqw6prnrvdhyamjqlppx2h9sl2277-slurm-17.11.7-dev
checking for references to /build in /nix/store/g2pgqw6prnrvdhyamjqlppx2h9sl2277-slurm-17.11.7-dev...

xeji · 2018-06-25T12:19:22Z

@GrahamcOfBorg test slurm

GrahamcOfBorg · 2018-06-25T12:26:53Z

Failure on x86_64-linux (full log)

Attempted: tests.slurm

Partial log (click to expand)

cleaning up
killing submit (pid 593)
killing node3 (pid 605)
killing control (pid 617)
killing node1 (pid 629)
killing node2 (pid 641)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/yvs0hvx1hn8s8h3j9vhyw681hgvyb1jg-vm-test-run-slurm.drv' failed with exit code 255
error: build of '/nix/store/yvs0hvx1hn8s8h3j9vhyw681hgvyb1jg-vm-test-run-slurm.drv' failed

GrahamcOfBorg · 2018-06-25T12:41:42Z

Success on aarch64-linux (full log)

Attempted: tests.slurm

Partial log (click to expand)

test script finished in 28.41s
cleaning up
killing control (pid 627)
killing node2 (pid 640)
killing submit (pid 653)
killing node3 (pid 666)
killing node1 (pid 679)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/grpcl1kl3fm09pn02nsh7133xid2gf1s-vm-test-run-slurm

xeji · 2018-06-25T12:57:41Z

Test failure on x86_64-linux was a sporadic error unrelated to this PR. Test succeeds locally.

slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604

e3b00d1

This commit also swaps to use the official repository's github release tags instead of their download site, which only keeps the most recent version with no historical archives.

bhipple requested a review from FRidh as a code owner June 24, 2018 20:52

bhipple commented Jun 24, 2018

View reviewed changes

GrahamcOfBorg added the 6.topic: python label Jun 24, 2018

Document why we don't run tests

c536478

bhipple commented Jun 24, 2018

View reviewed changes

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jun 24, 2018

Remove dead testing code

f797bd3

xeji mentioned this pull request Jun 24, 2018

slurm: 17.11.5 -> 17.11.6 #40638

Closed

xeji merged commit 39f357a into NixOS:master Jun 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604 #42536

slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604 #42536

bhipple commented Jun 24, 2018 •

edited

bhipple commented Jun 24, 2018

bhipple commented Jun 24, 2018

bhipple commented Jun 24, 2018 •

edited

GrahamcOfBorg commented Jun 24, 2018

bhipple Jun 24, 2018

bhipple Jun 24, 2018

veprbl Jun 24, 2018

markuskowa Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

xeji commented Jun 25, 2018

GrahamcOfBorg commented Jun 25, 2018

GrahamcOfBorg commented Jun 25, 2018

xeji commented Jun 25, 2018

slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604 #42536

slurm: 17.11.5 -> 17.11.7, pyslurm: 20180427 -> 20180604 #42536

Conversation

bhipple commented Jun 24, 2018 • edited

Things done

bhipple commented Jun 24, 2018

bhipple commented Jun 24, 2018

bhipple commented Jun 24, 2018 • edited

GrahamcOfBorg commented Jun 24, 2018

bhipple Jun 24, 2018

Choose a reason for hiding this comment

bhipple Jun 24, 2018

Choose a reason for hiding this comment

veprbl Jun 24, 2018

Choose a reason for hiding this comment

markuskowa Jun 24, 2018

Choose a reason for hiding this comment

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

GrahamcOfBorg commented Jun 24, 2018

xeji commented Jun 25, 2018

GrahamcOfBorg commented Jun 25, 2018

GrahamcOfBorg commented Jun 25, 2018

xeji commented Jun 25, 2018

bhipple commented Jun 24, 2018 •

edited

bhipple commented Jun 24, 2018 •

edited