lammps: update, hoomd-blue, dl_poly: init #46667

costrouc · 2018-09-14T18:42:37Z

Motivation for this change

Adding molecular dynamics codes that our group uses at UTK. Hoomd-blue is the most interesting addition. While I have not added GPU support it is mpi enabled and a fun code to work with. I do not have enough experience packaging with GPUs yet (maybe will be packaged at a future date). Scales to 10K GPUs according to benchmarks.

Would like to request @markuskowa for review.

Things done

lammps: patch_2Aug2018 -> stable_22Aug2018
dl-poly-classic: init 1.10
hoomd-blue: init at 2.3.4

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix

markuskowa · 2018-09-14T20:35:31Z

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix

+    inherit components mpi;
+  };
+
+  buildInputs = [ cmake stdenv pkgconfig ]


pkgconfig and cmake should be in nativeBuildInputs. No need to put stdenv to build inputs.
The line should then read nativeBuildInputs = [ cmake pkgconfig ];
mpi goes into buildInputs.

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix

markuskowa · 2018-09-14T20:39:35Z

pkgs/applications/science/molecular-dynamics/lammps/default.nix

-    platforms = stdenv.lib.platforms.linux;
-    maintainers = with lib.maintainers; [ costrouc ];
+    license = licenses.gpl2;
+    platforms = [ "x86_64-linux" ];


The previous version actual did build on aarch64. I guess we could leave it as platforms.linux and see if it builds on arm.

markuskowa · 2018-09-14T20:41:56Z

pkgs/applications/science/molecular-dynamics/lammps/default.nix


+  installPhase = ''
    mkdir -p $out/bin


A single mkdir line should also work here: mkdir -p $out/bin $out/include $out/lib.

markuskowa · 2018-09-14T20:44:36Z

As general comment (for the future): One could break that PR up into two or three PRs (at least the update as separate PR).

costrouc · 2018-09-14T22:00:56Z

@markuskowa the changes have been made. The reason why I am responding to the conversations on the code review in a single comment. Is that when I rebase with git (to keep it to only a few commits). Github completely looses track of where you made the comment. Is there a way around this?

What is the purpose of components?

Components in hoomd-blue are a set of features to include with the installation. The components are included within the git repository. Here is the documenation for one of the componenets https://hoomd-blue.readthedocs.io/en/stable/package-hpmc.html.

markuskowa · 2018-09-14T22:14:17Z

@GrahamcOfBorg build lammps lammps-mpi dl-poly-classic-mpi pythonPackages36.hoomd-blue pythonPackages36.hoomd-blue-mpi

markuskowa · 2018-09-14T22:14:45Z

pkgs/top-level/all-packages.nix

+    mpi = openmpi;
+  };
+
+  # hoomd-blue in python-packages.nix


That line can be deleted?

markuskowa · 2018-09-14T22:21:59Z

Thanks for the explanation. I think I understand what the purpose of the components is. Do you use the information propagated with passthru in some special way?
Regarding the loss of context for the comments after a rebase: I guess you lose track. I don't know of any solution for that issue. The comments at least remain with at least a few lines of context above. So that at least leaves the possibility to reply point by point.

GrahamcOfBorg · 2018-09-14T22:42:02Z

Success on x86_64-linux (full log)

Attempted: lammps, lammps-mpi, dl-poly-classic-mpi

The following builds were skipped because they don't evaluate on x86_64-linux: pythonPackages36.hoomd-blue, pythonPackages36.hoomd-blue-mpi

Partial log (click to expand)

'liblammps_mpi.so' -> '/nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/lib/liblammps_mpi.so'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018
shrinking /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/bin/lmp_mpi
shrinking /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/lib/liblammps_mpi.so
shrinking /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/lib/liblammps.so
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/lib  /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018/bin
patching script interpreter paths in /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018
checking for references to /build in /nix/store/nkyix7zi7y7pckqwkjrc8abj1jlh4j7j-lammps-stable_22Aug2018...

GrahamcOfBorg · 2018-09-14T23:14:16Z

Success on aarch64-linux (full log)

Attempted: lammps

The following builds were skipped because they don't evaluate on aarch64-linux: lammps-mpi, dl-poly-classic-mpi, pythonPackages36.hoomd-blue, pythonPackages36.hoomd-blue-mpi

Partial log (click to expand)

'liblammps_serial.so' -> '/nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/lib/liblammps_serial.so'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018
shrinking /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/lib/liblammps_serial.so
shrinking /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/lib/liblammps.so
shrinking /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/bin/lmp_serial
strip is /nix/store/y4ymnvgxygpq05h03kyzbj572zmh6zla-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/lib  /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018/bin
patching script interpreter paths in /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018
checking for references to /build in /nix/store/i6qv9rmbqbykr7hb7cb6q4i06fh8gam8-lammps-stable_22Aug2018...

markuskowa · 2018-09-15T09:00:41Z

@GrahamcOfBorg build python36Packages.hoomd-blue, python36Packages.hoomd-blue-mpi

GrahamcOfBorg · 2018-09-15T09:01:11Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: python36Packages.hoomd-blue,, python36Packages.hoomd-blue-mpi

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-09-15T09:10:09Z

Failure on x86_64-linux (full log)

Attempted: python36Packages.hoomd-blue-mpi

The following builds were skipped because they don't evaluate on x86_64-linux: python36Packages.hoomd-blue,

Partial log (click to expand)

        316 - mpcd-core-streaming_method (SEGFAULT)
        317 - mpcd-core-cell_communicator-mpi (SEGFAULT)
        318 - mpcd-core-cell_list-mpi (SEGFAULT)
        319 - mpcd-core-cell_thermo_compute-mpi (SEGFAULT)
        320 - example_plugin-test_example.py-cpu (Failed)
        321 - example_plugin-test_example.py-mpi-cpu (SEGFAULT)
Errors while running CTest
make: *** [Makefile:98: test] Error 8
builder for '/nix/store/1l864gdq3xbidfwm0jvfh6851jw5aicl-hoomd-blue-2.3.4.drv' failed with exit code 2
error: build of '/nix/store/1l864gdq3xbidfwm0jvfh6851jw5aicl-hoomd-blue-2.3.4.drv' failed

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix

pkgs/top-level/all-packages.nix

FRidh · 2018-09-18T16:57:22Z

pkgs/top-level/python-packages.nix

@@ -322,6 +322,12 @@ in {

  habanero = callPackage ../development/python-modules/habanero { };

+  hoomd-blue = callPackage ../applications/science/molecular-dynamics/hoomd-blue { };
+
+  hoomd-blue-mpi = callPackage ../applications/science/molecular-dynamics/hoomd-blue {


Do we need this attribute when one can also just override?

costrouc · 2018-09-18T20:52:40Z

All requested changes have been made. I have tested that hoomd works properly with a few of the example scripts. https://nbviewer.jupyter.org/github/joaander/hoomd-examples/blob/master/index.ipynb

costrouc · 2018-10-14T12:33:57Z

Rebased. @markuskowa with your approval this is ready for a merge. We will need to test rebuilding of python36Packages.hoomd-blue and python36Packages.hoomd-blue-mpi. I have verified that both build on my machine with sandbox. Hoomd tests were turned off but I can vouch that the program does run (with and without mpi) and was able to run several examples without errors.

markuskowa · 2018-10-14T21:19:35Z

I guess it needs another rebase.

only mpi version provided

nix derivation was cleaned significantly to follow nixpkgs better

costrouc · 2018-10-15T12:46:57Z

Wow a merge conflict in less than 8 hours, that's some good luck. Fixed now.

costrouc · 2018-10-15T19:29:43Z

@GrahamcOfBorg build python36Packages.hoomd-blue

GrahamcOfBorg · 2018-10-15T19:30:00Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: python36Packages.hoomd-blue

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-10-15T19:44:47Z

Success on x86_64-linux (full log)

Attempted: python36Packages.hoomd-blue

Partial log (click to expand)

shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/libquickhull.so
shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/metal/_metal.cpython-36m-x86_64-linux-gnu.so
shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/_hoomd.cpython-36m-x86_64-linux-gnu.so
shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/hpmc/_hpmc.cpython-36m-x86_64-linux-gnu.so
shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/deprecated/_deprecated.cpython-36m-x86_64-linux-gnu.so
shrinking /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib/python3.6/site-packages/hoomd/example_plugin/_example_plugin.cpython-36m-x86_64-linux-gnu.so
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4/lib
patching script interpreter paths in /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4
checking for references to /build in /nix/store/8ygidmr77xmbyzq8q8b00kmva1nf2n74-hoomd-blue-2.3.4...

markuskowa · 2018-10-16T07:01:50Z

@FRidh Are you OK with the changes?

costrouc · 2018-10-24T20:11:48Z

Bump. Ready for merge following your approval @FRidh. I have verified all of the packages to build properly.

pkgs/top-level/python-packages.nix

costrouc · 2018-10-30T18:22:58Z

@FRidh changes have been made and ready for merge.

GrahamcOfBorg · 2018-10-30T18:45:43Z

Success on x86_64-linux (full log)

Attempted: lammps

Partial log (click to expand)

'liblammps_serial.so' -> '/nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/lib/liblammps_serial.so'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018
shrinking /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/bin/lmp_serial
shrinking /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/lib/liblammps_serial.so
shrinking /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/lib/liblammps.so
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/lib  /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018/bin
patching script interpreter paths in /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018
checking for references to /build in /nix/store/s6wj48aicqr5a1cmnr1chfqp8hbqrf3w-lammps-stable_22Aug2018...

GrahamcOfBorg · 2018-10-30T18:57:58Z

Success on aarch64-linux (full log)

Attempted: lammps

Partial log (click to expand)

'liblammps_serial.so' -> '/nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/lib/liblammps_serial.so'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018
shrinking /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/lib/liblammps_serial.so
shrinking /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/lib/liblammps.so
shrinking /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/bin/lmp_serial
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/lib  /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018/bin
patching script interpreter paths in /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018
checking for references to /build in /nix/store/zmi2rjp72dm2g7735a2nfsv6kjh3hrjq-lammps-stable_22Aug2018...

costrouc · 2018-11-12T19:04:49Z

Bump. Ready for merge.

costrouc requested a review from FRidh as a code owner September 14, 2018 18:42

costrouc force-pushed the costrouc/lammps-update branch from a696517 to 15a1c6a Compare September 14, 2018 18:45

GrahamcOfBorg added the 6.topic: python label Sep 14, 2018

costrouc force-pushed the costrouc/lammps-update branch from 15a1c6a to e37b316 Compare September 14, 2018 19:13

GrahamcOfBorg added 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Sep 14, 2018

markuskowa reviewed Sep 14, 2018

View reviewed changes

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix Outdated Show resolved Hide resolved

markuskowa reviewed Sep 14, 2018

View reviewed changes

pkgs/applications/science/molecular-dynamics/hoomd-blue/default.nix Outdated Show resolved Hide resolved

markuskowa reviewed Sep 14, 2018

View reviewed changes

costrouc force-pushed the costrouc/lammps-update branch from e37b316 to 268b7ac Compare September 14, 2018 21:55

markuskowa reviewed Sep 14, 2018

View reviewed changes

FRidh requested changes Sep 18, 2018

View reviewed changes

costrouc force-pushed the costrouc/lammps-update branch from 268b7ac to 758c612 Compare September 18, 2018 20:51

costrouc force-pushed the costrouc/lammps-update branch from 758c612 to 2031018 Compare October 14, 2018 12:23

costrouc added 2 commits October 15, 2018 08:43

dl-poly-classic: init 1.10

7717a2b

only mpi version provided

lammps: patch_2Aug2018 -> stable_22Aug2018

397e15f

nix derivation was cleaned significantly to follow nixpkgs better

costrouc force-pushed the costrouc/lammps-update branch from 2031018 to 77bab99 Compare October 15, 2018 12:45

FRidh requested changes Oct 25, 2018

View reviewed changes

pkgs/top-level/python-packages.nix Outdated Show resolved Hide resolved

pkgs/top-level/python-packages.nix Outdated Show resolved Hide resolved

hoomd-blue: init at 2.3.4

ae66a92

costrouc force-pushed the costrouc/lammps-update branch from 77bab99 to ae66a92 Compare October 30, 2018 18:20

markuskowa merged commit 2823b5e into NixOS:master Nov 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lammps: update, hoomd-blue, dl_poly: init #46667

lammps: update, hoomd-blue, dl_poly: init #46667

costrouc commented Sep 14, 2018

markuskowa Sep 14, 2018

markuskowa Sep 14, 2018

markuskowa Sep 14, 2018

markuskowa commented Sep 14, 2018

costrouc commented Sep 14, 2018 •

edited

markuskowa commented Sep 14, 2018

markuskowa Sep 14, 2018

markuskowa commented Sep 14, 2018

GrahamcOfBorg commented Sep 14, 2018

GrahamcOfBorg commented Sep 14, 2018

markuskowa commented Sep 15, 2018

GrahamcOfBorg commented Sep 15, 2018

GrahamcOfBorg commented Sep 15, 2018

FRidh Sep 18, 2018

costrouc commented Sep 18, 2018

costrouc commented Oct 14, 2018

markuskowa commented Oct 14, 2018

costrouc commented Oct 15, 2018

costrouc commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

markuskowa commented Oct 16, 2018

costrouc commented Oct 24, 2018

costrouc commented Oct 30, 2018

GrahamcOfBorg commented Oct 30, 2018

GrahamcOfBorg commented Oct 30, 2018

costrouc commented Nov 12, 2018

lammps: update, hoomd-blue, dl_poly: init #46667

lammps: update, hoomd-blue, dl_poly: init #46667

Conversation

costrouc commented Sep 14, 2018

Motivation for this change

Things done

markuskowa Sep 14, 2018

Choose a reason for hiding this comment

markuskowa Sep 14, 2018

Choose a reason for hiding this comment

markuskowa Sep 14, 2018

Choose a reason for hiding this comment

markuskowa commented Sep 14, 2018

costrouc commented Sep 14, 2018 • edited

markuskowa commented Sep 14, 2018

markuskowa Sep 14, 2018

Choose a reason for hiding this comment

markuskowa commented Sep 14, 2018

GrahamcOfBorg commented Sep 14, 2018

GrahamcOfBorg commented Sep 14, 2018

markuskowa commented Sep 15, 2018

GrahamcOfBorg commented Sep 15, 2018

GrahamcOfBorg commented Sep 15, 2018

FRidh Sep 18, 2018

Choose a reason for hiding this comment

costrouc commented Sep 18, 2018

costrouc commented Oct 14, 2018

markuskowa commented Oct 14, 2018

costrouc commented Oct 15, 2018

costrouc commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

markuskowa commented Oct 16, 2018

costrouc commented Oct 24, 2018

costrouc commented Oct 30, 2018

GrahamcOfBorg commented Oct 30, 2018

GrahamcOfBorg commented Oct 30, 2018

costrouc commented Nov 12, 2018

costrouc commented Sep 14, 2018 •

edited