Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spark: 1.6.0 -> 1.6.2 #19705

Closed
wants to merge 1 commit into from
Closed

spark: 1.6.0 -> 1.6.2 #19705

wants to merge 1 commit into from

Conversation

cko
Copy link
Member

@cko cko commented Oct 19, 2016

Motivation for this change
Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
    • OS X
    • Linux
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

spark-shell still has some problems due to a hard coded path to /bin/ls in org.apache.hadoop.util.Shell.class - it was also there in the previous version.

spark-shell still has some problems due to a hard coded path to /bin/ls in org.apache.hadoop.util.Shell.class
@jgeerds
Copy link
Member

jgeerds commented Oct 20, 2016

What about the temporary broken mesos support?

@cko
Copy link
Member Author

cko commented Oct 20, 2016

The mesos package was set broken a couple of weeks ago (#19040) as version 0.28.2 has a security issue. There are some efforts to package mesos version 1.0.1 (#19064), but it's not fully done yet.
The spark package doesn't build with mesos being broken, even if the dependency is just optional.

@Mic92
Copy link
Member

Mic92 commented Oct 22, 2016

apache spark 2.0.1 is out

@cko
Copy link
Member Author

cko commented Nov 8, 2016

The 1.6 branch is stilled developed, so I don't see the need to abandon it yet.
btw I submitted a patch for Hadoop for the /bin/ls issue (apache/hadoop@f768955), though it will probably need a couple of months untils it's part of Spark.

@joachifm
Copy link
Contributor

@cko mesos has been updated

@cstrahan
Copy link
Contributor

Just a heads up - I'm in the process of bumping Mesos to 1.1.0 and ironing out the last couple bugs: #21416

@cko cko closed this Mar 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants