Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python37Packages.distributed: 1.25.2 -> 1.25.3 #55212

Merged
merged 2 commits into from Feb 7, 2019
Merged

python37Packages.distributed: 1.25.2 -> 1.25.3 #55212

merged 2 commits into from Feb 7, 2019

Conversation

r-ryantm
Copy link
Contributor

@r-ryantm r-ryantm commented Feb 4, 2019

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/python3.7-distributed/versions.

meta.description for python37Packages.distributed is: '"Distributed computation in Python."'.

Checks done (click to expand)
  • built on NixOS
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-ssh-wrapped passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-ssh passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-submit-wrapped passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-submit passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-remote-wrapped passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-remote passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-scheduler-wrapped passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-scheduler passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-worker-wrapped passed the binary check.
  • /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-worker passed the binary check.
  • Warning: no invocation of /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-mpi-wrapped had a zero exit code or showed the expected version
  • Warning: no invocation of /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/dask-mpi had a zero exit code or showed the expected version
  • 10 of 12 passed binary check by having a zero exit code.
  • 0 of 12 passed binary check by having the new version present in output.
  • found 1.25.3 with grep in /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3
  • directory tree listing: https://gist.github.com/7dbafb36ef6c48a882e2e64ee55f0d17
  • du listing: https://gist.github.com/0631de32e27535276996775c2faad817
Rebuild report (if merged into master) (click to expand)

22 total rebuild path(s)

6 package rebuild(s)

6 x86_64-linux rebuild(s)
6 i686-linux rebuild(s)
6 x86_64-darwin rebuild(s)
4 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
python27Packages.dask-jobqueue
python27Packages.dask-xgboost
python27Packages.distributed
python37Packages.dask-jobqueue
python37Packages.dask-xgboost
python37Packages.distributed

Instructions to test this update (click to expand)

Either download from Cachix:

nix-store -r /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)

Or, build yourself:

nix-build -A python37Packages.distributed https://github.com/r-ryantm/nixpkgs/archive/8107b31f86c52b483336e949a7de211a3ee4b92f.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:

ls -la /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3
ls -la /nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin

cc @teh @costrouc for testing.

@r-ryantm r-ryantm requested a review from FRidh as a code owner February 4, 2019 16:05
@Ma27
Copy link
Member

Ma27 commented Feb 7, 2019

This breaks with errors like this:

Traceback (most recent call last):
  File "/nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/bin/.dask-mpi-wrapped", line 8, in <module>
    from distributed.cli.dask_mpi import go
  File "/nix/store/wz4rlcw5dkgzmx1xihjkw10skbf0f3lh-python3.7-distributed-1.25.3/lib/python3.7/site-packages/distributed/cli/dask_mpi.py", line 4, in <module>
    from mpi4py import MPI
ModuleNotFoundError: No module named 'mpi4py'

I'll push a fix.

@Ma27
pythonPackages.distributed: add missing dependencies
9dd0b54 
Added `mpi4py` and `bokeh` to get the executables working.

Also simplified the `lib.optional` expression as Python 3.2 and 3.4
aren't supported anymore. Rather than referencing unsupported Python 3.x
versions it's far more obvious now to only use `futures` and
`singledispatch` if Python2 is used.
@Ma27 Ma27 force-pushed the auto-update/python3.7-distributed branch from ae14cd3 to 9dd0b54 Compare February 7, 2019 20:30
@costrouc
Copy link
Member

costrouc commented Feb 7, 2019 •
edited

@Ma27 dask distributed should not be packaged with mpi4py since it does not depend on it and is not mentioned in the requirements. If a user wants to have mpi4py with dask they need to install https://github.com/dask/dask-mpi. It looks like this package does not exist yet in nixpkgs

I agree that bokeh is nice to have included with distributed since the dashboard is helpful.

costrouc added a commit to costrouc/nixpkgs that referenced this pull request Feb 7, 2019
@costrouc
pythonPackages.dask-mpi: init at 1.0.2
3415614 
Address issue with PR NixOS#55212
@costrouc costrouc mentioned this pull request Feb 7, 2019
Merged
10 tasks
@costrouc
Copy link
Member

costrouc commented Feb 7, 2019 •
edited

I have added dask mpi in the PR referenced above. Still no need for mpi4py in inputs

@Ma27
Copy link
Member

Ma27 commented Feb 7, 2019 •
edited

As posted above, some of the executables are failing without it. What would you suggest instead? Installing mpi4py globally isn't sufficient.

@costrouc
Copy link
Member

costrouc commented Feb 7, 2019

Yes I agree that it is right to include mpi4py since it is indeed needed and separately from dask-mpi. I opened an issue that they add it as an extra dependency dask/distributed#2516

@Ma27
Copy link
Member

Ma27 commented Feb 7, 2019

Ok great! I'll run nix-review to ensure that this doesn't introduce more breaking packages and merge then :)

@Ma27
Copy link
Member

Ma27 commented Feb 7, 2019

nix-review passes, as dask-mpi is still a part of pythonPackages.distributed the current change should be fine. As soon as they start splitting the project, we can fix the derivation here as well.

@Ma27 Ma27 merged commit 9649145 into NixOS:master Feb 7, 2019
@r-ryantm r-ryantm deleted the auto-update/python3.7-distributed branch February 15, 2019 02:59
FRidh pushed a commit that referenced this pull request Feb 23, 2019
@costrouc @FRidh
pythonPackages.dask-mpi: init at 1.0.2
34beeb7 
Address issue with PR #55212
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@FRidh FRidh Awaiting requested review from FRidh

Assignees
No one assigned
Labels
6.topic: python 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@r-ryantm @Ma27 @costrouc @GrahamcOfBorg