pythonPackages.patator: init at 0.7 #34642

y0no · 2018-02-05T21:46:29Z

Motivation for this change

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

dotlambda

name is not necessary for buildPython* anymore. Please remove it

dotlambda · 2018-02-05T22:52:04Z

pkgs/development/python-modules/patator/default.nix

+	dnspython
+	IPy
+	pysnmp
+	pyasn1


indentation

Thanks, I fix this in next commit =)

dotlambda · 2018-02-05T22:53:53Z

pkgs/top-level/python-packages.nix

-  };
+  IPy = callPackage ../development/python-modules/IPy { };
+  # Alias to remove 
+  ipy = callPackage ../development/python-modules/IPy { };


What's wrong with the old name? Most attributes in Nixpkgs are lowercase.

Don't know what's better between the real name (as a lots of packages), or all in lowercase. I think, in pypi you can have two different packages with the same name except but different capitalized letters.

No, that's not true. The attributes are preferably the normalized names, so ipy in this case.
https://www.python.org/dev/peps/pep-0503/#normalized-names

dotlambda · 2018-02-05T22:55:12Z

pkgs/development/python-modules/cx_oracle/default.nix

+  };
+
+  checkPhase = ''
+    LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/oracle/12.2/client64/lib


If we don't do that, cx_Oracle don't find Oracle libs :/

You are referring to a path outside the store. That is incorrect. Furthermore, paths to libraries are better hardcoded. Also, something should run in the checkPhase.

Okay, Is there a good way to handle the path?

dotlambda · 2018-02-05T22:57:33Z

pkgs/development/python-modules/patator/default.nix

+    sha256 = "335e432e6cc591437e316ba8c1da935484ca39fc79e595ccf60ccd9166e965f1";
+  };
+
+  propagatedBuildInputs = with python3Packages; [


Please add the dependencies as arguments instead of python3Packages.

Why not. What's the difference ?

FRidh · 2018-02-06T08:29:27Z

pkgs/development/python-modules/IPy/default.nix

+buildPythonPackage rec {
+  pname = "IPy";
+  version = "0.83";
+  name = "${pname}-${version}";


The name attribute is added by buildPython* and should therefore be removed.

FRidh · 2018-02-06T08:29:36Z

pkgs/development/python-modules/ajpy/default.nix

+buildPythonPackage rec {
+  pname = "ajpy";
+  version = "0.0.2";
+  name = "${pname}-${version}";


The name attribute is added by buildPython* and should therefore be removed.

FRidh · 2018-02-06T08:29:41Z

pkgs/development/python-modules/cx_oracle/default.nix

+buildPythonPackage rec {
+  pname = "cx_Oracle";
+  version = "6.1";
+  name = "${pname}-${version}";


The name attribute is added by buildPython* and should therefore be removed.

FRidh · 2018-02-06T08:29:51Z

pkgs/development/python-modules/cx_oracle/default.nix

+  };
+
+  checkPhase = ''
+    LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/oracle/12.2/client64/lib


You are referring to a path outside the store. That is incorrect. Furthermore, paths to libraries are better hardcoded. Also, something should run in the checkPhase.

FRidh · 2018-02-06T08:30:41Z

pkgs/development/python-modules/mysqlclient/default.nix

+buildPythonPackage rec {
+  pname = "mysqlclient";
+  version = "1.3.12";
+  name = "${pname}-${version}";


The name attribute is added by buildPython* and should therefore be removed.

FRidh · 2018-02-06T08:31:30Z

pkgs/development/python-modules/patator/default.nix

+buildPythonPackage rec {
+  pname = "patator";
+  version = "0.7";
+  name = "${pname}-${version}";


The name attribute is added by buildPython* and should therefore be removed.

FRidh · 2018-02-06T08:32:10Z

pkgs/development/python-modules/patator/default.nix

@@ -0,0 +1,35 @@
+{ stdenv, buildPythonPackage, isPy3k, python3Packages, fetchPypi }:


Don't pass in the package set; instead pass in individual requirements.

FRidh · 2018-02-06T08:33:26Z

pkgs/top-level/python-packages.nix

-  };
+  IPy = callPackage ../development/python-modules/IPy { };
+  # Alias to remove 
+  ipy = callPackage ../development/python-modules/IPy { };


No, that's not true. The attributes are preferably the normalized names, so ipy in this case.
https://www.python.org/dev/peps/pep-0503/#normalized-names

y0no · 2018-02-06T09:46:25Z

Hope the changes requested are well fixed.

dotlambda · 2018-02-06T09:49:52Z

You're still not running any checks in the checkPhase.
Also, please squash the commits.

y0no · 2018-02-06T10:04:37Z

Woops you are right. In fact, it seems that cx_Oracle tests need an Oracle database running in background. I think it will be better to pass the test if we don't want an error

dotlambda · 2018-02-06T10:06:55Z

Yes, if it needs a database, then you can set doCheck = false. But don't forget to add a comment with the reason.

dotlambda · 2018-02-06T10:13:40Z

Now, it would be nice if you replaced your 11 commits by 5, one for each package.
And please name your commits according to CONTRIBUTING.md, e.g. pythonPackages.mysqlclient: init at 1.3.12.

y0no · 2018-02-06T10:38:17Z

Ok, I think it's good now.

dotlambda · 2018-02-06T11:56:51Z

Very nice. Thank you! I'll try to build them...

@GrahamcOfBorg build python2.pkgs.ipy python3.pkgs.ipy
@GrahamcOfBorg build python2.pkgs.ajpy python3.pkgs.ajpy
@GrahamcOfBorg build python2.pkgs.cx_oracle python3.pkgs.cx_oracle
@GrahamcOfBorg build python2.pkgs.mysqlclient python3.pkgs.mysqlclient
@GrahamcOfBorg build python3.pkgs.patator

GrahamcOfBorg · 2018-02-06T11:57:00Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

Package ‘oracle-instantclient-12.2.0.1.0’ in /var/lib/gc-of-borg/nix-test-rs-2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-2/pkgs/development/libraries/oracle-instantclient/default.nix:80 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-06T11:57:04Z

Success on x86_64-linux (full log)

Partial log (click to expand)

reading manifest file 'ajpy.egg-info/SOURCES.txt'
writing manifest file 'ajpy.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/71brd1hj1qv93dn3cnl6lgh07cphsfhs-python2.7-ajpy-0.0.2
/nix/store/sc49lckzp43v71qw1pkqqdmzygdk5s4b-python3.6-ajpy-0.0.2

GrahamcOfBorg · 2018-02-06T11:57:08Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

Package ‘oracle-instantclient-12.2.0.1.0’ in /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/rbvermaa-spot/pkgs/development/libraries/oracle-instantclient/default.nix:80 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-06T11:57:09Z

Success on x86_64-linux (full log)

Partial log (click to expand)

running install tests
make: Nothing to be done for 'test'.
make: Nothing to be done for '/nix/store/dbmw100gg8cpyg07hc1cf188ad1h6mbr-python2.7-IPy-0.83'.
patching script interpreter paths in /nix/store/2rnp1b00f6d28bvq4kg1nh5jzaxng4zj-python3.6-IPy-0.83
checking for references to /tmp/nix-build-python3.6-IPy-0.83.drv-0 in /nix/store/2rnp1b00f6d28bvq4kg1nh5jzaxng4zj-python3.6-IPy-0.83...
running install tests
make: Nothing to be done for 'test'.
make: Nothing to be done for '/nix/store/2rnp1b00f6d28bvq4kg1nh5jzaxng4zj-python3.6-IPy-0.83'.
/nix/store/dbmw100gg8cpyg07hc1cf188ad1h6mbr-python2.7-IPy-0.83
/nix/store/2rnp1b00f6d28bvq4kg1nh5jzaxng4zj-python3.6-IPy-0.83

GrahamcOfBorg · 2018-02-06T11:57:13Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

Package ‘oracle-instantclient-12.2.0.1.0’ in /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-zoidberg/pkgs/development/libraries/oracle-instantclient/default.nix:80 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-06T11:57:16Z

Success on aarch64-linux (full log)

Partial log (click to expand)

shrinking RPATHs of ELF executables and libraries in /nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83
strip is /nix/store/xmpjypwjmp2qi1chs5kr0hacnh161ls4-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83/lib
patching script interpreter paths in /nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83
checking for references to /build in /nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83...
running install tests
make: Nothing to be done for 'test'.
make: Nothing to be done for '/nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83'.
/nix/store/3zfxf3fhlpk531140g3lmvix1w7ryspd-python2.7-IPy-0.83
/nix/store/g3m1xicpgibmggzz90si1f2w994sw18f-python3.6-IPy-0.83

GrahamcOfBorg · 2018-02-06T11:57:18Z

Success on x86_64-linux (full log)

Partial log (click to expand)

writing manifest file 'mysqlclient.egg-info/SOURCES.txt'
running build_ext
copying build/lib.linux-x86_64-3.6/_mysql.cpython-36m-x86_64-linux-gnu.so -> 

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/kav2cz1fqvyfff1ljbzzy38qm12khp1b-python2.7-mysqlclient-1.3.12
/nix/store/6va306zj54kfj4196pli6lwgf3k50dy2-python3.6-mysqlclient-1.3.12

GrahamcOfBorg · 2018-02-06T11:57:19Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

writing top-level names to ajpy.egg-info/top_level.txt
reading manifest file 'ajpy.egg-info/SOURCES.txt'
writing manifest file 'ajpy.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
error: build of '/nix/store/4ibpyxf8c0c6ml9hnypa723lgsrcxayx-python2.7-ajpy-0.0.2.drv' failed

GrahamcOfBorg · 2018-02-06T11:57:23Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

Package ‘oracle-instantclient-12.2.0.1.0’ in /var/lib/gc-of-borg/nix-test-rs-3/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-3/pkgs/development/libraries/oracle-instantclient/default.nix:80 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-06T11:57:25Z

Success on aarch64-linux (full log)

Partial log (click to expand)

writing manifest file 'mysqlclient.egg-info/SOURCES.txt'
running build_ext
copying build/lib.linux-aarch64-3.6/_mysql.cpython-36m-aarch64-linux-gnu.so ->

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
/nix/store/g6mknw0053ila77sfid4vcb4qvb891m0-python2.7-mysqlclient-1.3.12
/nix/store/j0mp03yghzw1dhwdw71x8qnnyqxnqy5m-python3.6-mysqlclient-1.3.12

dotlambda · 2018-02-06T12:05:07Z

Ran 0 tests in 0.000s

Do ajpy and mysqlclient provide tests? If so, you should specify the appropriate checkPhase. Otherwise, please set doCheck = false and add a comment that there are no tests.

Regarding ipy, the tests don't seem to run there as well. Do you have an idea why?
Maybe use nosetests in the checkPhase? See https://github.com/autocracy/python-ipy/blob/master/.travis.yml
Another idea would be to use make -B.

y0no · 2018-02-06T14:02:45Z

Ajpy, patator doesn't seems to got tests.
Concerning mysqlclient, it's like cx_Oracle, we need a database to run the tests.
Finally, I have fixed the tests for IPy. As you mentionned, the tests are run with nose.

FRidh · 2018-02-09T15:23:57Z

A dependency can't be built:

nix build python3.pkgs.cx_oracle -f .
builder for '/nix/store/6k80zw21b9chd00lqya0vfj03y8s91xw-oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm.drv' failed to produce output path '/nix/store/3ly42sshg74lqk1vw2fksagq6zd3sz0g-oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm'
cannot build derivation '/nix/store/1g1bbai4y39zx3b4y26c2waip8ymygdf-oracle-instantclient-12.2.0.1.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/pqy6859y8xs0gkd5758faz28aff6300d-python3.6-cx_Oracle-6.1.drv': 1 dependencies couldn't be built
[0 built (1 failed)]
error: build of '/nix/store/pqy6859y8xs0gkd5758faz28aff6300d-python3.6-cx_Oracle-6.1.drv' failed

y0no · 2018-02-09T15:42:39Z

Can this error be made because of oracle-instantclient license ? It correctly build on my laptop when oracle-instantclient is installed (package that I don't handle)

FRidh · 2018-02-09T16:28:54Z

Right, it needs prefetching. That important part was not visible in the nix build output. You can ignore it.

y0no · 2018-02-09T16:44:27Z

So do I have to do something else to get the package merged ?

dotlambda · 2018-02-22T10:23:14Z

@FRidh This looks fine, doesn't it? I did however not test if it works.

dotlambda · 2018-02-25T09:52:33Z

ping @FRidh

dotlambda · 2018-03-18T10:04:18Z

@y0no Can you rebase on master?
@FRidh Any objections against merging?

y0no requested a review from FRidh as a code owner February 5, 2018 21:46

GrahamcOfBorg added 6.topic: python 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100 labels Feb 5, 2018

dotlambda requested changes Feb 5, 2018

View reviewed changes

FRidh requested changes Feb 6, 2018

View reviewed changes

y0no force-pushed the add_patator branch 2 times, most recently from 42846ea to 537afb8 Compare February 6, 2018 10:37

y0no changed the title ~~patator: init at 0.7~~ pythonPackages.patator: init at 0.7 Feb 6, 2018

FRidh approved these changes Mar 18, 2018

View reviewed changes

dotlambda force-pushed the add_patator branch from 0dc21c4 to f82762f Compare March 19, 2018 10:39

y0no added 5 commits March 19, 2018 11:40

pythonPackages.ipy: 0.74 -> 0.83

c9bbf07

pythonPackages.ajpy: init at 0.0.2

a15544d

pythonPackages.cx_oracle: init at 6.1

c936651

pythonPackages.mysqlclient: init at 1.3.12

38719d1

pythonPackages.patator: init at 0.7

0725d66

dotlambda approved these changes Mar 19, 2018

View reviewed changes

dotlambda force-pushed the add_patator branch from f82762f to 0725d66 Compare March 19, 2018 10:42

GrahamcOfBorg added the 8.has: package (new) label Mar 19, 2018

dotlambda merged commit def2a25 into NixOS:master Mar 19, 2018

		@@ -0,0 +1,35 @@
		{ stdenv, buildPythonPackage, isPy3k, python3Packages, fetchPypi }:

pythonPackages.patator: init at 0.7 #34642

pythonPackages.patator: init at 0.7 #34642

Conversation

y0no commented Feb 5, 2018

Motivation for this change

Things done

dotlambda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

y0no commented Feb 6, 2018

dotlambda commented Feb 6, 2018

y0no commented Feb 6, 2018

dotlambda commented Feb 6, 2018

dotlambda commented Feb 6, 2018

y0no commented Feb 6, 2018

dotlambda commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

GrahamcOfBorg commented Feb 6, 2018

dotlambda commented Feb 6, 2018

y0no commented Feb 6, 2018

FRidh commented Feb 9, 2018

y0no commented Feb 9, 2018

FRidh commented Feb 9, 2018

y0no commented Feb 9, 2018

dotlambda commented Feb 22, 2018

dotlambda commented Feb 25, 2018

dotlambda commented Mar 18, 2018