pythonPackages.nipype: 0.10.0 -> 0.14.0 #33916

ashgillman · 2018-01-16T03:34:07Z

Motivation for this change

Many new nipype features in interim, including py3 support

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Mic92 · 2018-01-16T07:43:03Z

pkgs/top-level/python-packages.nix

@@ -6196,6 +6196,26 @@ in {
    };
  };

+  prov = buildPythonPackage rec {


Please do not add new packages directly to this file, but store them in python-modules and only reference them here as described in the header of this file.

Thanks, can do

Mic92 · 2018-01-16T08:54:21Z

pkgs/development/python-modules/prov/default.nix

+    pydotplus
+    rdflib
+  ];
+  doCheck = false; # takes forever!


Please also add a meta attributes here.

Mic92 · 2018-01-16T08:54:47Z

pkgs/development/python-modules/pydotplus/default.nix

+
+  meta = {
+    homepage = https://code.google.com/p/pydot/;
+    description = "An improved version of the old pydot project that provides a Python Interface to Graphviz’s Dot language";


License is missing; do you want to maintain this package?

Mic92 · 2018-01-16T08:56:07Z

pkgs/top-level/python-packages.nix

    };

-    # Tests fail due to getcwd returning ENOENT???
+    # Fails in testing, "no such file or directory: which"


Have you tried adding the which package?

I didn't realise which was a command :s
which which gives which: built in shell command

Adding which actually does help, but still fails :(

in zsh it is a builtin.

FRidh · 2018-01-16T10:35:58Z

pkgs/top-level/python-packages.nix

@@ -11316,17 +11320,18 @@ in {
  });

  nibabel = buildPythonPackage rec {


We are moving Python expressions out of pkgs/top-level/python-packages.nix into pkgs/development/python-modules/<module>/default.nix.

Please move the expression there, and call it from pkgs/top-level/python-packages.nix using callPackage ../development/python-modules/<package> { };.

FRidh · 2018-01-16T10:36:12Z

pkgs/top-level/python-packages.nix

@@ -11387,34 +11392,42 @@ in {
  };

  nipype = buildPythonPackage rec {


We are moving Python expressions out of pkgs/top-level/python-packages.nix into pkgs/development/python-modules/<module>/default.nix.

Please move the expression there, and call it from pkgs/top-level/python-packages.nix using callPackage ../development/python-modules/<package> { };.

FRidh · 2018-01-16T10:36:22Z

pkgs/top-level/python-packages.nix

@@ -15606,11 +15619,11 @@ in {


  rdflib = buildPythonPackage (rec {


We are moving Python expressions out of pkgs/top-level/python-packages.nix into pkgs/development/python-modules/<module>/default.nix.

Please move the expression there, and call it from pkgs/top-level/python-packages.nix using callPackage ../development/python-modules/<package> { };.

FRidh · 2018-01-16T10:36:28Z

pkgs/top-level/python-packages.nix

@@ -17851,11 +17864,11 @@ in {

  traits = buildPythonPackage rec {


We are moving Python expressions out of pkgs/top-level/python-packages.nix into pkgs/development/python-modules/<module>/default.nix.

Please move the expression there, and call it from pkgs/top-level/python-packages.nix using callPackage ../development/python-modules/<package> { };.

FRidh · 2018-01-16T10:36:44Z

pkgs/development/python-modules/prov/default.nix

+}:
+
+buildPythonPackage rec {
+  name = "${pname}-${version}";


FRidh · 2018-01-16T10:36:49Z

pkgs/development/python-modules/pydotplus/default.nix

+}:
+
+buildPythonPackage rec {
+  name = "${pname}-${version}";


FRidh · 2018-01-16T10:36:53Z

pkgs/development/python-modules/xvfbwrapper/default.nix

+}:
+
+buildPythonPackage rec {
+  name = "${pname}-${version}";


Not sure what you mean sorry. There should be no name attr? https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/python.md recommends this

That example should indeed be updated. buildPython* functions fill in name when given pname and version.

FRidh · 2018-01-16T10:37:30Z

pkgs/development/python-modules/prov/default.nix

+    pydotplus
+    rdflib
+  ];
+  doCheck = false; # takes forever!


What is "forever"? Test suites of most scientific python packages take some time.

Looking into this now (I wrote some of these expressions a number of months ago).

Ran 956 tests in 3594.012s FAILED (failures=22, expected failures=17) Test failed: <unittest.runner.TextTestResult run=956 errors=0 failures=22>

So about an hour (total build time was about an hour and 10).

I am investigating whether the errors are present in 1.5.1 before reporting upstream. (NB: Nipype currently requires exactly 1.5.0, this is being dealt with)

FRidh · 2018-01-16T16:36:24Z

pkgs/development/python-modules/nibabel/default.nix

+}:
+
+buildPythonPackage rec {
+  name = "${pname}-${version}";


FRidh · 2018-01-16T16:36:47Z

pkgs/development/python-modules/nipype/default.nix

+
+  doCheck = false; # fails with TypeError: None is not callable
+  checkInputs = [ which ];
+  buildInputs = [ pytest mock ];


checkInputs

I should move which back to buildInputs?

Sorry, I am not aware of a few of these conventions. Are they documented somewhere?

no, he was asking you to move pytest and mock to checkInputs. checkInputs is a buildPythonPackage specific attribute that contains dependencies only needed during testing. Does this make things clear?

Ah, I see. However, the build fails in installPhase without pytest and mock so these are needed as buildInputs. I will add a comment

FRidh · 2018-01-16T16:37:46Z

pkgs/development/python-modules/xvfbwrapper/default.nix

+}:
+
+buildPythonPackage rec {
+  name = "${pname}-${version}";


That example should indeed be updated. buildPython* functions fill in name when given pname and version.

ashgillman · 2018-01-17T00:59:01Z

I believe all changes have been made

ashgillman · 2018-01-17T02:13:44Z

NB: prov still has testing disabled due to errors that were fixed in 5.0.1. Also these tests take an hour to run. Nipype requires 5.0.0.

FRidh · 2018-01-20T19:26:45Z

These changes have landed in staging through #34077.

ashgillman added 2 commits January 16, 2018 13:27

pythonPackages.rdflib: 4.1.2 -> 4.2.2

cc72c60

pythonPackages.traits: 4.5.0 -> 4.6.0

ab59e9d

ashgillman requested a review from FRidh as a code owner January 16, 2018 03:34

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Jan 16, 2018

Mic92 reviewed Jan 16, 2018

View reviewed changes

ashgillman added 5 commits January 16, 2018 18:37

pythonPackages.xvfbwrapper: init at 0.2.9

1b69f23

pythonPackages.nibabel: 2.0.2 -> 2.2.0

7b0e038

pythonPackages.pydotplus: init at 2.0.2

0133d1e

pythonPackages.prov: init at 1.5.0

2c9b615

pythonPackages.nipype: 0.10.0 -> 0.14.0

f34d818

ashgillman force-pushed the update-nipype branch from 15585b2 to f34d818 Compare January 16, 2018 08:38

Mic92 reviewed Jan 16, 2018

View reviewed changes

ashgillman added 3 commits January 16, 2018 19:31

pythonPackages.prov: Add meta

d272782

pythonPackages.pydotplus: Update homepage, add license and maintainers.

4b0d9b3

pythonPackages.nipype: Check phase requires which, but still fails

6f9e372

FRidh requested changes Jan 16, 2018

View reviewed changes

FRidh added the 6.topic: python label Jan 16, 2018

ashgillman added 2 commits January 16, 2018 23:28

pythonPackages.nibabel: Move to separate file.

265e9be

pythonPackages.nipype: To separate file.

5a6148f

FRidh requested changes Jan 16, 2018

View reviewed changes

ashgillman added 4 commits January 17, 2018 10:04

pythonPackages.{nibabel,nipype,prov,pydotplus,xvfbwrapper}: Remove name

56cd80d

pythonPackages.prov: More descriptive reason for skipping check phase

4f6283d

pythonPackages.nipype: Fix bad definition for py27.

c3c6559

pythonPackages.nipype: Document reason for buildInputs.

04c960c

FRidh self-assigned this Jan 17, 2018

FRidh mentioned this pull request Jan 20, 2018

Python: package updates and fixes #34077

Merged

8 tasks

FRidh closed this Jan 20, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pythonPackages.nipype: 0.10.0 -> 0.14.0 #33916

pythonPackages.nipype: 0.10.0 -> 0.14.0 #33916

ashgillman commented Jan 16, 2018

Mic92 Jan 16, 2018 •

edited

ashgillman Jan 16, 2018

Mic92 Jan 16, 2018 •

edited

Mic92 Jan 16, 2018

Mic92 Jan 16, 2018

ashgillman Jan 16, 2018

Mic92 Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

ashgillman Jan 16, 2018

FRidh Jan 16, 2018

ashgillman Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

ashgillman Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

ashgillman Jan 16, 2018

ashgillman Jan 16, 2018

FRidh Jan 16, 2018

FRidh Jan 16, 2018

ashgillman Jan 17, 2018

Mic92 Jan 17, 2018

ashgillman Jan 17, 2018

FRidh Jan 16, 2018

ashgillman commented Jan 17, 2018

ashgillman commented Jan 17, 2018

FRidh commented Jan 20, 2018

		@@ -11316,17 +11320,18 @@ in {
		});

		nibabel = buildPythonPackage rec {

		@@ -11387,34 +11392,42 @@ in {
		};

		nipype = buildPythonPackage rec {

		@@ -15606,11 +15619,11 @@ in {


		rdflib = buildPythonPackage (rec {

		@@ -17851,11 +17864,11 @@ in {

		traits = buildPythonPackage rec {

pythonPackages.nipype: 0.10.0 -> 0.14.0 #33916

pythonPackages.nipype: 0.10.0 -> 0.14.0 #33916

Conversation

ashgillman commented Jan 16, 2018

Motivation for this change

Things done

Mic92 Jan 16, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mic92 Jan 16, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashgillman commented Jan 17, 2018

ashgillman commented Jan 17, 2018

FRidh commented Jan 20, 2018

Mic92 Jan 16, 2018 •

edited

Mic92 Jan 16, 2018 •

edited