pythonPackages.imgaug: init at 0.2.6 #43720

CMCDragonkai · 2018-07-18T06:02:30Z

Motivation for this change

Added pythonPackages.imgaug. Should work on python 3.6 and python 2.

One thing I didn't understand is why nix_run_setup test would result in a segmentation fault. The current imgaug.tar.gz does not bundle any tests in the source archive so there are no tests to run anyway. But can there be a better error than just segmentation fault?

Also while imgaug doesn't require dask, the scikit-image when installed by setup.py appears to require dask now.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

FRidh · 2018-07-18T07:52:40Z

One thing I didn't understand is why nix_run_setup test would result in a segmentation fault. The current imgaug.tar.gz does not bundle any tests in the source archive so there are no tests to run anyway. But can there be a better error than just segmentation fault?

Looks like a regression in Python 3 (maybe setuptools) since it does not happen with Python 2 and the Nixpkgs code is exactly the same for both.

FRidh · 2018-07-18T07:53:20Z

pkgs/development/python-modules/imgaug/default.nix

+    dask
+  ];
+
+  doCheck = false;


Include the motivation for disabling tests as a comment.

This is due to segmentation fault. And there being no tests?

Yes, but you can't know that when looking at the expression.

FRidh · 2018-07-18T08:06:46Z

pkgs/development/python-modules/imgaug/default.nix

+  meta = with stdenv.lib; {
+    homepage = https://github.com/aleju/imgaug;
+    description = "Image augmentation for machine learning experiments";
+    license = licenses.mit;


maintainership

This is done!

CMCDragonkai · 2018-07-18T08:17:34Z

Added everything. Except reason for no tests.

CMCDragonkai · 2018-07-18T13:00:10Z

Added the comment about the segmentation fault for the test.

CMCDragonkai · 2018-07-20T03:12:32Z

I've moved dask to buildInputs instead of propagatedBuildInputs here as well.

CMCDragonkai · 2018-07-20T03:52:33Z

Regarding dask: #43723 (comment)

FRidh · 2018-07-21T09:06:39Z

pkgs/development/python-modules/imgaug/default.nix

+    sha256 = "1wy8ydkqq0jrwxwdv04q89n3gwsr9pjaspsbw26ipg5a5lnhb9c2";
+  };
+
+  buildInputs = [ dask ];


as I mentioned in another PR, this needs to be removed

CMCDragonkai · 2018-07-22T04:16:39Z

@FRidh Just did it.

CMCDragonkai · 2018-07-30T04:40:38Z

@FRidh Is this ready to be merged?

dotlambda · 2018-07-31T22:07:20Z

@CMCDragonkai There's a merge conflict. Please resolve that.

@GrahamcOfBorg build python2.pkgs.imgaug python3.pkgs.imgaug

dotlambda · 2018-07-31T22:07:45Z

pkgs/development/python-modules/imgaug/default.nix

+    six
+  ];
+
+  # disable tests due to python 3 segmentation fault


If there are no tests, that's what the comment should say.

@dotlambda imgaug has tests, it just doesn't work due to python 3 segmentation fault. #43720 (comment)

The current imgaug.tar.gz does not bundle any tests in the source archive so there are no tests to run anyway.

If I understand it correctly, there are tests but not in the tarball. The comment could read something like No tests in PyPI tarball.
The segfault likely happens because the tests are missing.

GrahamcOfBorg · 2018-07-31T22:07:48Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

Merge failed

GrahamcOfBorg · 2018-07-31T22:07:52Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

Merge failed

CMCDragonkai · 2018-08-02T04:33:36Z

@dotlambda Merge conflicts resolved by rebasing PR on master.

CMCDragonkai · 2018-08-03T10:51:34Z

@dotlambda Comment changed.

dotlambda · 2018-08-03T11:58:38Z

@GrahamcOfBorg eval
@GrahamcOfBorg build python2.pkgs.imgaug python3.pkgs.imgaug

GrahamcOfBorg · 2018-08-03T12:00:07Z

Success on x86_64-linux (full log)

Attempted: python2.pkgs.imgaug, python3.pkgs.imgaug

Partial log (click to expand)

Successfully installed imgaug-0.2.6
/build/imgaug-0.2.6
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/fx958h5000w6whs85rzpwhhlpcqs4icf-python3.6-imgaug-0.2.6
strip is /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/fx958h5000w6whs85rzpwhhlpcqs4icf-python3.6-imgaug-0.2.6/lib
patching script interpreter paths in /nix/store/fx958h5000w6whs85rzpwhhlpcqs4icf-python3.6-imgaug-0.2.6
checking for references to /build in /nix/store/fx958h5000w6whs85rzpwhhlpcqs4icf-python3.6-imgaug-0.2.6...
/nix/store/0phyzwsr7wb2kaf1c7714qw8rgs4mcsw-python2.7-imgaug-0.2.6
/nix/store/fx958h5000w6whs85rzpwhhlpcqs4icf-python3.6-imgaug-0.2.6

GrahamcOfBorg · 2018-08-03T12:58:47Z

Failure on aarch64-linux (full log)

Attempted: python2.pkgs.imgaug, python3.pkgs.imgaug

Partial log (click to expand)

    return self._engine.get_loc(key)

-- Docs: http://doc.pytest.org/en/latest/warnings.html
 5 failed, 16889 passed, 1036 skipped, 11894 deselected, 37 xfailed, 2 xpassed, 6 warnings in 1520.07 seconds
builder for '/nix/store/v9g5l67xxd9vna2qljywd6kzkyihzkd7-python3.6-pandas-0.23.3.drv' failed with exit code 1
cannot build derivation '/nix/store/f4h17md0advik19jayq3izawhipgyc2i-python3.6-partd-0.3.8.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/iw12nkmh5n4r9klsqyd5z96aqn10z86x-python3.6-dask-0.18.2.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/6lzmz31fhxwdiysm7dza8h25691whp77-python3.6-scikit-image-0.14.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/g9iabmwclxy7lsn7c9h2i186aibznyv7-python3.6-imgaug-0.2.6.drv': 1 dependencies couldn't be built
error: build of '/nix/store/5bi6ca2gbwal42k2rahnl86611q89lpn-python2.7-imgaug-0.2.6.drv', '/nix/store/g9iabmwclxy7lsn7c9h2i186aibznyv7-python3.6-imgaug-0.2.6.drv' failed

CMCDragonkai requested a review from FRidh as a code owner July 18, 2018 06:02

FRidh added the 6.topic: python label Jul 18, 2018

FRidh requested changes Jul 18, 2018

View reviewed changes

FRidh reviewed Jul 18, 2018

View reviewed changes

CMCDragonkai force-pushed the imgaug branch from ef2f91c to fa5004a Compare July 18, 2018 08:16

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jul 18, 2018

CMCDragonkai force-pushed the imgaug branch from fa5004a to e3a9050 Compare July 18, 2018 12:59

CMCDragonkai force-pushed the imgaug branch from e3a9050 to 228cefc Compare July 20, 2018 03:11

FRidh reviewed Jul 21, 2018

View reviewed changes

CMCDragonkai force-pushed the imgaug branch from 228cefc to ceed957 Compare July 22, 2018 04:16

dotlambda reviewed Jul 31, 2018

View reviewed changes

CMCDragonkai force-pushed the imgaug branch from ceed957 to 3b0dc2f Compare August 2, 2018 04:32

pythonPackages.imgaug: init at 0.2.6

cd38c55

CMCDragonkai force-pushed the imgaug branch from 3b0dc2f to cd38c55 Compare August 3, 2018 10:51

dotlambda merged commit 83cc1e3 into NixOS:master Aug 3, 2018

CMCDragonkai deleted the imgaug branch August 8, 2018 11:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pythonPackages.imgaug: init at 0.2.6 #43720

pythonPackages.imgaug: init at 0.2.6 #43720

CMCDragonkai commented Jul 18, 2018 •

edited

FRidh commented Jul 18, 2018 •

edited

FRidh Jul 18, 2018

CMCDragonkai Jul 18, 2018

FRidh Jul 18, 2018

FRidh Jul 18, 2018

CMCDragonkai Aug 2, 2018

CMCDragonkai commented Jul 18, 2018 •

edited

CMCDragonkai commented Jul 18, 2018

CMCDragonkai commented Jul 20, 2018

CMCDragonkai commented Jul 20, 2018

FRidh Jul 21, 2018

CMCDragonkai commented Jul 22, 2018

CMCDragonkai commented Jul 30, 2018

dotlambda commented Jul 31, 2018

dotlambda Jul 31, 2018

CMCDragonkai Aug 2, 2018

dotlambda Aug 2, 2018 •

edited

GrahamcOfBorg commented Jul 31, 2018

GrahamcOfBorg commented Jul 31, 2018

CMCDragonkai commented Aug 2, 2018

CMCDragonkai commented Aug 3, 2018

dotlambda commented Aug 3, 2018

GrahamcOfBorg commented Aug 3, 2018

GrahamcOfBorg commented Aug 3, 2018

pythonPackages.imgaug: init at 0.2.6 #43720

pythonPackages.imgaug: init at 0.2.6 #43720

Conversation

CMCDragonkai commented Jul 18, 2018 • edited

Motivation for this change

Things done

FRidh commented Jul 18, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CMCDragonkai commented Jul 18, 2018 • edited

CMCDragonkai commented Jul 18, 2018

CMCDragonkai commented Jul 20, 2018

CMCDragonkai commented Jul 20, 2018

Choose a reason for hiding this comment

CMCDragonkai commented Jul 22, 2018

CMCDragonkai commented Jul 30, 2018

dotlambda commented Jul 31, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda Aug 2, 2018 • edited

Choose a reason for hiding this comment

GrahamcOfBorg commented Jul 31, 2018

GrahamcOfBorg commented Jul 31, 2018

CMCDragonkai commented Aug 2, 2018

CMCDragonkai commented Aug 3, 2018

dotlambda commented Aug 3, 2018

GrahamcOfBorg commented Aug 3, 2018

GrahamcOfBorg commented Aug 3, 2018

CMCDragonkai commented Jul 18, 2018 •

edited

FRidh commented Jul 18, 2018 •

edited

CMCDragonkai commented Jul 18, 2018 •

edited

dotlambda Aug 2, 2018 •

edited