Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pyspark #36529

Merged
merged 2 commits into from Mar 17, 2018
Merged

Add pyspark #36529

merged 2 commits into from Mar 17, 2018

Conversation

shlevy
Copy link
Member

@shlevy shlevy commented Mar 8, 2018

No description provided.

@shlevy shlevy requested a review from FRidh as a code owner March 8, 2018 15:23
FRidh
FRidh requested changes Mar 8, 2018
View reviewed changes
pkgs/development/python-modules/py4j/default.nix Outdated

version = "0.10.6";

src = fetchurl {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetchPypi with extension = "zip";

pkgs/development/python-modules/py4j/default.nix Outdated
meta = with stdenv.lib; {
description = "Py4J enables Python programs running in a Python interpreter to dynamically access Java objects in a Java Virtual Machine. Methods are called as if the Java objects resided in the Python interpreter and Java collections can be accessed through standard Python collection methods. Py4J also enables Java programs to call back Python objects.";
homepage = https://www.py4j.org/;
license = licenses.bsd;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainer?

pkgs/development/python-modules/pyspark/default.nix Outdated
pname = "pyspark";
version = "2.3.0";

src = fetchurl {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetchPypi

@shlevy
Copy link
Member Author

shlevy commented Mar 8, 2018

@FRidh Sorry about this, I guess python2nix needs updating... Fix incoming.

@shlevy shlevy requested a review from FRidh March 8, 2018 16:00
@shlevy
Copy link
Member Author

shlevy commented Mar 8, 2018

@FRidh Fixed.

@shlevy
Copy link
Member Author

shlevy commented Mar 17, 2018

@GrahamcOfBorg build pythonPackages.pyspark

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: pythonPackages.pyspark

Partial log (click to expand)

/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/pyspark: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/sparkR: interpreter directive changed from "/bin/bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/docker-image-tool.sh: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-submit: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/beeline: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-class: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/run-example: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/find-spark-home: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/8hdxjv3m14jgdnawi5l0mz2h59qkk2g9-bash-4.4-p19/bin/bash"
wrapping `/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0/bin/find_spark_home.py'...
/nix/store/jhzpnwfln32x1alplnls4xx0vznpldiy-python2.7-pyspark-2.3.0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: pythonPackages.pyspark

Partial log (click to expand)

/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/pyspark: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/run-example: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-class: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-shell: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-sql: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-submit: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/sparkR: interpreter directive changed from "/bin/bash" to "/nix/store/3gg2p6n4kv2f0lsxd41f5iz1ivkbzyzr-bash-4.4-p19/bin/bash"
checking for references to /build in /nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0...
wrapping `/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0/bin/find_spark_home.py'...
/nix/store/slpqdah8k1sp06gnydymgknqn7i0pbm0-python2.7-pyspark-2.3.0

@primeos primeos merged commit 2963d76 into NixOS:master Mar 17, 2018
@shlevy shlevy deleted the pyspark branch March 17, 2018 14:41
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: pythonPackages.pyspark

Partial log (click to expand)

/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/beeline: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-sql: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/find-spark-home: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-shell: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/spark-submit: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/sparkR: interpreter directive changed from "/bin/bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/lib/python2.7/site-packages/pyspark/bin/run-example: interpreter directive changed from "/usr/bin/env bash" to "/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
checking for references to /build in /nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0...
wrapping `/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0/bin/find_spark_home.py'...
/nix/store/55j9ry57v71qjhv99bcrx61x4gf455ng-python2.7-pyspark-2.3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@FRidh FRidh FRidh approved these changes

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@shlevy @GrahamcOfBorg @FRidh @primeos