Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3packages.stytra: init at 0.8.26 #72353

Merged
merged 6 commits into from Dec 16, 2019
Merged

python3packages.stytra: init at 0.8.26 #72353

merged 6 commits into from Dec 16, 2019

Conversation

tbenst
Copy link
Contributor

@tbenst tbenst commented Oct 31, 2019 •
edited

WIP - wanted to get feedback on help as this is my first time packing using wheel.

It seems like propogatedBuildInputs is not behaving the same way--if I only have a package here I get failures like:

ERROR: Could not find a version that satisfies the requirement pyqtgraph>=0.10.0 (from stytra==0.8.26) (from versions: none)
ERROR: No matching distribution found for pyqtgraph>=0.10.0 (from stytra==0.8.26)
builder for '/nix/store/brscmbwx2nzr56f8wdh3n8vnjq2ih58p-python3.7-stytra-0.8.26.drv' failed with exit code 1

As a result, I'm including in both buildInputs, and propogatedBuildInputs, which feels wrong. It does build, but when I try open a shell the following happens:

> python
>>> import stytra
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/prr2jha833ix3qa30w0cbfgcr1zxh75s-python3-3.7.5-env/lib/python3.7/site-packages/stytra/__init__.py", line 3, in <module>
    from stytra.experiments import VisualExperiment
  File "/nix/store/prr2jha833ix3qa30w0cbfgcr1zxh75s-python3-3.7.5-env/lib/python3.7/site-packages/stytra/experiments/__init__.py", line 5, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

This seems very odd, as in any other derivation I test, I can import from propogatedBuildInputs in python shell. To debug, I switched back to using the tar.gz file, and found an unexpected error:

these derivations will be built:
  /nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv
building '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv'...
collision between `/nix/store/10b0xgqaqacaxiwlvcklp9nbl20y5wcx-python3.7-sip-4.19.18/bin/sip' and `/nix/store/nqa4rbzhi6aj0ry54dwc82r2900zlnmr-python3.7-PyQt5.sip-4.19.18/bin/sip'
builder for '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv' failed with exit code 255
error: build of '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv' failed

Any thoughts or tips? Thanks!

Motivation for this change

Add module for closed-loop experiments for larval zebrafish.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@tbenst tbenst changed the title <!-- Nixpkgs has a lot of new incoming Pull Requests, but not enough people to review this constant stream. Even if you aren't a committer, we would appreciate reviews of other PRs, especially simple ones like package updates. Just testing the relevant package/service and leaving a comment saying what you tested, how you tested it and whether it worked would be great. List of open PRs: <https://github.com/NixOS/nixpkgs/pulls>, for more about reviewing contributions: <https://hydra.nixos.org/job/nixpkgs/trunk/manual/latest/download/1/nixpkgs/manual.html#chap-reviewing-contributions>. Reviewing isn't mandatory, but it would help out a lot and reduce the average time-to-merge for all of us. Thanks a lot if you do! --> ###### Motivation for this change python3packages.stytra: init at 0.8.26 Oct 31, 2019
jonringer
jonringer reviewed Oct 31, 2019
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see you declaring the pyqtgraph package in the propagated build inputs....

python packages should go into propagatedBuildInputs

pkgs/development/python-modules/stytra/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/stytra/default.nix Outdated Show resolved Hide resolved
@tbenst
Copy link
Contributor Author

tbenst commented Oct 31, 2019 •
edited

I don't see you declaring the pyqtgraph package in the propagated build inputs....
python packages should go into propagatedBuildInputs

It's there (via commonInputs). If pyqtgraph is only in propogatedBuildInputs, I get the error above. It goes away if I add to buildInputs, too, as is currently done in the code

Edit: for posterity, if I use format = "wheel" all hell breaks loose, but easy enough with setuptools

@tbenst tbenst force-pushed the stytra branch 2 times, most recently from 78d383d to 8d1a549 Compare October 31, 2019 21:01
@tbenst
Copy link
Contributor Author

tbenst commented Oct 31, 2019 •
edited

This is blocked until #72378 is merged, but reviews in the meantime are welcome!

Edit: forgot to mention that the package is running according to my tests locally:

> nix-shell -I nixpkgs=. -p 'python3.buildEnv.override { extraLibs = [ python3Packages.stytra ]; ignoreCollisions = true; }'
$ python -m stytra.examples.looming_exp
[success!]

jonringer
jonringer reviewed Nov 2, 2019
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove all instances of pulling down wheels when a source distribution is available, the only way a wheel should be allowed is if there no way to pull down source through pypi or the repo

pkgs/development/python-modules/arrayqueues/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/arrayqueues/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/colorspacious/default.nix Outdated Show resolved Hide resolved
pkgs/development/python-modules/colorspacious/default.nix Outdated Show resolved Hide resolved
@tbenst
Copy link
Contributor Author

tbenst commented Dec 12, 2019

Thanks @jonringer. This is ready for final review

jonringer
jonringer suggested changes Dec 15, 2019
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please squash the fixup to adhere to contributing guidelines

git reset HEAD^
git add pkgs/
git commit --amend --no-edit
git push <fork> <branch> --force

@tbenst
Copy link
Contributor Author

tbenst commented Dec 15, 2019

@jonringer done

jonringer
jonringer approved these changes Dec 16, 2019
View reviewed changes
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff LGTM
commits LGTM

https://github.com/NixOS/nixpkgs/pull/72353
12 package were built:
python27Packages.colorspacious python27Packages.flammkuchen python37Packages.arrayqueues python37Packages.colorspacious python37Packages.flammkuchen python37Packages.lightparam python37Packages.qimage2ndarray python37Packages.stytra python38Packages.arrayqueues python38Packages.colorspacious python38Packages.lightparam python38Packages.qimage2ndarray

@jonringer jonringer merged commit 60829ec into NixOS:master Dec 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonringer jonringer jonringer approved these changes

@FRidh FRidh Awaiting requested review from FRidh

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@tbenst @jonringer