Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: NixOS/nixpkgs
base: cc5a3cb54701
Choose a base ref
...
head repository: NixOS/nixpkgs
compare: cde89e3ecf8f
Choose a head ref
  • 1 commit
  • 1 file changed
  • 1 contributor

Commits on Nov 24, 2020

  1. python3Packages.pyspark: Fix out of date postPatch 

    Pyspark switched to pinning py4j==0.10.9 with v3.0.0 - see this commit:
https://github.com/apache/spark/\
   commit/fc4e56a54c15e20baf085e6061d3d83f5ce1185d

This meant that since the bump to pyspark v3.0.0 - in this commit:
https://github.com/NixOS/nixpkgs/\
   commit/5181547ae6624b462919a806c4d0888e6e4630f4 -
the patch was no longer matching on the 'py4j==0.10.7' string that was
working previously.

The failing patch went unnoticed previously because the version of py4j
pinned by pyspark>=3.0.0 was the same as the py4j provided by nixpkgs.

However, a recent PR (#101636) bumped the version of py4j to 0.10.9.1 in
this commit:
https://github.com/NixOS/nixpkgs/\
   commit/43a91282d66223c5cb978d53fbe1033f56dd7f2b
which caused the version pinned by pyspark to no longer match the
version provided by nixpkgs. FWIW, @jonringer flagged this issue on
another PR that tried to bump py4j: #100623.

My solution here was to upgrade the patch's target string to match the
version found in pyspark's current setup.py.
    @jskrzypek
    jskrzypek authored and Jonathan Ringer committed Nov 24, 2020
    Copy the full SHA
    cde89e3 View commit details
    Browse the repository at this point in the history