pythonPackages.uproot: init at 2.8.23. #39120

ktf · 2018-04-18T19:49:03Z

Motivation for this change

Package uproot is missing from the distribution.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

FRidh · 2018-04-18T19:53:34Z

maintainers/maintainer-list.nix

@@ -1976,6 +1976,11 @@
    github = "kristoff3r";
    name = "Kristoffer Søholm";
  };
+  ktf = {


separate commit

FRidh · 2018-04-18T19:53:57Z

pkgs/development/python-modules/uproot/default.nix

+{lib, fetchFromGitHub, buildPythonPackage, numpy }:
+
+buildPythonPackage rec {
+  name = "uproot-${version}";


The name attribute is added by buildPython* when pname is added, and should therefore be removed.

FRidh · 2018-04-18T19:54:39Z

pkgs/development/python-modules/uproot/default.nix

+
+  meta = with lib; {
+    homepage = "https://github.com/scikit-hep/uproot";
+    description = "ROOT I/O in pure Python and Numpy.";


no dot at the end

FRidh · 2018-04-18T19:54:59Z

@GrahamcOfBorg build python3.pkgs.uproot

GrahamcOfBorg · 2018-04-18T19:55:16Z

No attempt on x86_64-darwin (full log)

The following builds were skipped because they don't evaluate on x86_64-darwin: python3.pkgs.uproot

Partial log (click to expand)

Cannot nix-instantiate `python3.pkgs.uproot' because:
error: attribute 'uproot' in selection path 'python3.pkgs.uproot' not found

GrahamcOfBorg · 2018-04-18T19:55:26Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: python3.pkgs.uproot

Partial log (click to expand)

Cannot nix-instantiate `python3.pkgs.uproot' because:
�[31;1merror:�[0m attribute 'uproot' in selection path 'python3.pkgs.uproot' not found

ktf · 2018-04-18T20:07:53Z

I think I've addressed all the comments. Can you please check again?

FRidh · 2018-04-19T06:05:44Z

The expression needs to be called from python-packages.nix.

ktf · 2018-04-19T07:44:19Z

done. This is my first real contribution, sorry for being sloppy...

FRidh · 2018-04-19T07:56:07Z

@GrahamcOfBorg build python3.pkgs.uproot

GrahamcOfBorg · 2018-04-19T07:56:44Z

Success on aarch64-linux (full log)

Attempted: python3.pkgs.uproot

Partial log (click to expand)

Installing collected packages: uproot
Successfully installed uproot-2.8.16
/build/source
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/nq9lpyzs5zzg5hy3i60v20xsn60mhr6z-python3.6-uproot-2.8.16
strip is /nix/store/j7d4mr0ikv974ig7yzhknpsq288js4bs-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/nq9lpyzs5zzg5hy3i60v20xsn60mhr6z-python3.6-uproot-2.8.16/lib
patching script interpreter paths in /nix/store/nq9lpyzs5zzg5hy3i60v20xsn60mhr6z-python3.6-uproot-2.8.16
checking for references to /build in /nix/store/nq9lpyzs5zzg5hy3i60v20xsn60mhr6z-python3.6-uproot-2.8.16...
/nix/store/nq9lpyzs5zzg5hy3i60v20xsn60mhr6z-python3.6-uproot-2.8.16

GrahamcOfBorg · 2018-04-19T07:57:03Z

Success on x86_64-darwin (full log)

Attempted: python3.pkgs.uproot

Partial log (click to expand)

Processing ./uproot-2.8.16-py3-none-any.whl
Requirement already satisfied: numpy in /nix/store/8jcsbs0m9hklh01mmh2ydnhw1mrgsy6i-python3.6-numpy-1.14.2/lib/python3.6/site-packages (from uproot==2.8.16)
Installing collected packages: uproot
Successfully installed uproot-2.8.16
/private/tmp/nix-build-python3.6-uproot-2.8.16.drv-0/source
post-installation fixup
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/0k4xajqzgiy4rahzi5bz69dsg1mq9xv2-python3.6-uproot-2.8.16/lib
patching script interpreter paths in /nix/store/0k4xajqzgiy4rahzi5bz69dsg1mq9xv2-python3.6-uproot-2.8.16
/nix/store/0k4xajqzgiy4rahzi5bz69dsg1mq9xv2-python3.6-uproot-2.8.16

GrahamcOfBorg · 2018-04-19T13:55:44Z

Success on x86_64-linux (full log)

Attempted: python3.pkgs.uproot

Partial log (click to expand)

Installing collected packages: uproot
Successfully installed uproot-2.8.16
/build/source
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/l7q8azscxx9y6j3x2rwmd07f3dzz4x1m-python3.6-uproot-2.8.16
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/l7q8azscxx9y6j3x2rwmd07f3dzz4x1m-python3.6-uproot-2.8.16/lib
patching script interpreter paths in /nix/store/l7q8azscxx9y6j3x2rwmd07f3dzz4x1m-python3.6-uproot-2.8.16
checking for references to /build in /nix/store/l7q8azscxx9y6j3x2rwmd07f3dzz4x1m-python3.6-uproot-2.8.16...
/nix/store/l7q8azscxx9y6j3x2rwmd07f3dzz4x1m-python3.6-uproot-2.8.16

dotlambda

Did you intentionally not include the dependencies listed here?

dotlambda · 2018-04-20T10:25:45Z

pkgs/development/python-modules/uproot/default.nix

+    sha256 = "07f6h7zcvjc4ynqydp12cbq8c6bb8964s1i65cc25py4mg46f4my";
+  };
+
+  # Why does it propagate packages that are used for testing?


numpy is listed in install_requires, you're sure it isn't a runtime dependency?

You can't do anything sensible in uproot without Numpy. It's an absolute requirement. I hope the setup.py reflects that.

Many of the other libraries that uproot might use are optional, particularly compression codecs that aren't found in all ROOT files.

Yes, setup.py reflects this. But the comment

Why does it propagate packages that are used for testing?

doesn't make sense in that case.

dotlambda · 2018-04-20T10:26:58Z

pkgs/development/python-modules/uproot/default.nix

+buildPythonPackage rec {
+  pname = "uproot";
+  version = "2.8.16";
+  src = fetchFromGitHub {


Please use fetchPypi instead: https://pypi.org/project/uproot/. This allows easier automatic updating.

dotlambda · 2018-04-20T10:27:07Z

pkgs/development/python-modules/uproot/default.nix

+  doCheck = false;
+
+  meta = with lib; {
+    homepage = "https://github.com/scikit-hep/uproot";


dotlambda · 2018-04-20T10:29:59Z

pkgs/development/python-modules/uproot/default.nix

+    numpy
+  ];
+
+  #Checks are failing due to missing TTY, which won't exist.


The tests run just fine for me. Can you please check again?

ktf · 2018-04-26T08:12:48Z

I've done the requested changes.

@dotlambda I've tried adding lz4 and lama, however they seem not to be picked up when building. I am using an overlay with the following to test:

uproot = import ../nixpkgs/pkgs/development/python-modules/uproot {
     inherit (self) lib;
     inherit (super.python27.pkgs) fetchPypi;
     inherit (super.python27.pkgs) buildPythonPackage numpy lz4;
     inherit (super) lzma;
};

and I of course added lz4 and lzma to the propagatedBuildInputs. Am I doing something trivially wrong?

jpivarski · 2018-04-26T11:15:50Z

If it's Python 3, there might not be an lzma package because that's in the standard library now.

ktf · 2018-04-26T12:15:27Z

@jpivarski in my test it's python2.7. I suspect it's something nix related, actually.

dotlambda · 2018-04-26T12:36:16Z

pkgs/development/python-modules/uproot/default.nix

+buildPythonPackage rec {
+  pname = "uproot";
+  version = "2.8.18";
+  name = "${pname}-${version}";


drop this line

you mean the name one? If I do I get an error. Am I using a too old version of buildPythonPackage?

buildPython* has been setting name automatically for quite some time. Which error are you getting?

dotlambda · 2018-04-26T12:47:35Z

By looking at the code, e.g. https://github.com/scikit-hep/uproot/blob/47b981e6ce0b5913a763d379eae1c0337a898a01/tests/test_compression.py#L34-L41, it should just work when adding the lz4 (and lzma in the case of python 2) python packages to propagatedBuildInputs.

dotlambda · 2018-04-26T13:01:23Z

pkgs/development/python-modules/uproot/default.nix

+  meta = with lib; {
+    homepage = https://github.com/scikit-hep/uproot;
+    description = "ROOT I/O in pure Python and Numpy";
+    license = with licenses; [ bsd3 ];


just licenses.bsd3

ktf · 2018-05-15T20:51:51Z

I think I've addressed the comments. lz4 / lzma does not work (I get an import error) but that could be something to followup upstream and I'd rather add in a subsequent PR.

dotlambda · 2018-05-15T21:30:39Z

@ktf Now what's left to do is naming your second commit (and your PR) pythonPackages.uproot: init at <version>.

@GrahamcOfBorg build python2.pkgs.uproot python3.pkgs.uproot

GrahamcOfBorg · 2018-05-15T21:31:39Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.uproot, python3.pkgs.uproot

Partial log (click to expand)

test_issue74 (tests.test_issues.TestIssues) ... ok
test_issue76 (tests.test_issues.TestIssues) ... ok
test_issue79 (tests.test_issues.TestIssues) ... ok

----------------------------------------------------------------------
Ran 74 tests in 7.619s

OK
/nix/store/pff88w2kvarhm3ysj796g7i7m0cy2l2c-python2.7-uproot-2.8.23
/nix/store/bbf4pavmbjm0ghzsdf7311mqkgbzgyd8-python3.6-uproot-2.8.23

GrahamcOfBorg · 2018-05-15T21:32:08Z

Success on aarch64-linux (full log)

Attempted: python2.pkgs.uproot, python3.pkgs.uproot

Partial log (click to expand)

test_irregular_array (tests.test_cache.TestCache) ... ok
test_regular_array (tests.test_cache.TestCache) ... ok
test_strings_array (tests.test_cache.TestCache) ... ok

----------------------------------------------------------------------
Ran 74 tests in 41.814s

OK
/nix/store/h8f1f3vclgm2i3773byd07im6brdw1g4-python2.7-uproot-2.8.23
/nix/store/2an977hpilsm2bh1qvwnh75lqyj1389j-python3.6-uproot-2.8.23

ktf · 2018-05-16T07:38:58Z

@dotlambda done.

dotlambda · 2018-05-16T08:19:24Z

Thanks a lot!

ktf requested a review from FRidh as a code owner April 18, 2018 19:49

GrahamcOfBorg added the 6.topic: python label Apr 18, 2018

FRidh requested changes Apr 18, 2018

View reviewed changes

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Apr 18, 2018

ktf force-pushed the add-uproot branch from 094d1fe to 5ac4459 Compare April 18, 2018 20:06

ktf force-pushed the add-uproot branch from 5ac4459 to 4fb7666 Compare April 19, 2018 07:43

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Apr 19, 2018

dotlambda requested changes Apr 20, 2018

View reviewed changes

dotlambda reviewed Apr 20, 2018

View reviewed changes

ktf mentioned this pull request Apr 20, 2018

Error running tests scikit-hep/uproot3#73

Closed

ktf force-pushed the add-uproot branch from 4fb7666 to c46da15 Compare April 26, 2018 08:09

dotlambda reviewed Apr 26, 2018

View reviewed changes

Add myself to the maintainers list

0d32d93

ktf force-pushed the add-uproot branch from c46da15 to b735dff Compare May 15, 2018 20:49

ktf changed the title ~~Add uproot python package~~ pythonPackages.uproot: init at 2.8.23. May 15, 2018

pythonPackages.uproot: init at 2.8.23

54cf5d9

ktf force-pushed the add-uproot branch from b735dff to 54cf5d9 Compare May 16, 2018 07:37

dotlambda merged commit 40edf5e into NixOS:master May 16, 2018

ktf deleted the add-uproot branch May 16, 2018 09:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pythonPackages.uproot: init at 2.8.23. #39120

pythonPackages.uproot: init at 2.8.23. #39120

ktf commented Apr 18, 2018

FRidh Apr 18, 2018

FRidh Apr 18, 2018

FRidh Apr 18, 2018

FRidh commented Apr 18, 2018

GrahamcOfBorg commented Apr 18, 2018

GrahamcOfBorg commented Apr 18, 2018

ktf commented Apr 18, 2018

FRidh commented Apr 19, 2018

ktf commented Apr 19, 2018

FRidh commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

dotlambda left a comment

dotlambda Apr 20, 2018

jpivarski Apr 20, 2018

dotlambda Apr 20, 2018

dotlambda Apr 20, 2018

dotlambda Apr 20, 2018

dotlambda Apr 20, 2018

ktf commented Apr 26, 2018

jpivarski commented Apr 26, 2018

ktf commented Apr 26, 2018

dotlambda Apr 26, 2018

ktf Apr 26, 2018

dotlambda Apr 26, 2018

dotlambda commented Apr 26, 2018

dotlambda Apr 26, 2018

ktf commented May 15, 2018

dotlambda commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

ktf commented May 16, 2018

dotlambda commented May 16, 2018

pythonPackages.uproot: init at 2.8.23. #39120

pythonPackages.uproot: init at 2.8.23. #39120

Conversation

ktf commented Apr 18, 2018

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FRidh commented Apr 18, 2018

GrahamcOfBorg commented Apr 18, 2018

GrahamcOfBorg commented Apr 18, 2018

ktf commented Apr 18, 2018

FRidh commented Apr 19, 2018

ktf commented Apr 19, 2018

FRidh commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

GrahamcOfBorg commented Apr 19, 2018

dotlambda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ktf commented Apr 26, 2018

jpivarski commented Apr 26, 2018

ktf commented Apr 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Apr 26, 2018

Choose a reason for hiding this comment

ktf commented May 15, 2018

dotlambda commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

GrahamcOfBorg commented May 15, 2018

ktf commented May 16, 2018

dotlambda commented May 16, 2018