Skip to content

pythonPackages.zstd: init at 1.3.5.1 #49339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 1, 2018
Merged

pythonPackages.zstd: init at 1.3.5.1 #49339

merged 1 commit into from
Nov 1, 2018
+53 −0

Conversation

eadwu
Copy link
Member

@eadwu eadwu commented Oct 28, 2018

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

Sorry, something went wrong.

@eadwu eadwu requested a review from FRidh as a code owner October 28, 2018 13:13
@veprbl
Copy link
Member

veprbl commented Oct 28, 2018 •
edited
Loading

Perhaps it would make sense to use zstd already packaged in nixpkgs. Something along the lines of

{
setupPyBuildFlags = [
"--external"
"--include-dirs=${zstd}/include"
"--libraries=zstd"
"--library-dirs=${zstd}/lib"
];
}

@eadwu
Copy link
Member Author

eadwu commented Oct 29, 2018 •
edited
Loading

How should the src attribute be set? Just a random placeholder file/folder?

@eadwu eadwu force-pushed the init/pythonPackages.zstd branch from cebf13e to e5f8aa7 Compare October 29, 2018 23:46
@eadwu eadwu force-pushed the init/pythonPackages.zstd branch from e5f8aa7 to e674e1b Compare October 30, 2018 00:21
@veprbl
Copy link
Member

veprbl commented Oct 30, 2018

@eadwu May I suggest a following change

diff --git a/pkgs/development/python-modules/zstd/default.nix b/pkgs/development/python-modules/zstd/default.nix
index 5d1485c2720..785dd9d3946 100644
--- a/pkgs/development/python-modules/zstd/default.nix
+++ b/pkgs/development/python-modules/zstd/default.nix
@@ -1,4 +1,4 @@
-{ stdenv, buildPythonPackage, fetchpatch, fetchFromGitHub }:
+{ stdenv, buildPythonPackage, fetchpatch, fetchFromGitHub, pkgconfig, pytest, zstd }:
 
 buildPythonPackage rec {
   pname = "zstd";
@@ -8,18 +8,33 @@ buildPythonPackage rec {
     owner = "sergey-dryabzhinsky";
     repo = "python-zstd";
     rev = "v${version}";
-    sha256 = "0sy1ccqsh3bilzs7f6qkgqq9r734kgp8410vyv2x5vvzx3azp45d";
-    fetchSubmodules = true;
+    sha256 = "08n1vz4zavas4cgzpdfcbpy33lnv39xxhq5mgj0zv3xi03ypc1rl";
   };
 
-  patches = [
-    # make sure tests directory is included
-    (fetchpatch {
-      url = "https://github.com/sergey-dryabzhinsky/python-zstd/commit/cbdfbe2477b43e2d527fa4ee5dc960134a051be4.patch";
-      sha256 = "18ws563nmvx5rnpm1lyzi2379rgbsc614wp2cd5m40k3j21g0sfq";
-    })
+  postPatch = ''
+    substituteInPlace setup.py \
+      --replace "/usr/bin/pkg-config" "${pkgconfig}/bin/pkg-config"
+  '';
+
+  nativeBuildInputs = [ pkgconfig ];
+  buildInputs = [ zstd ];
+
+  setupPyBuildFlags = [
+    "--external"
+    "--include-dirs=${zstd}/include"
+    "--libraries=zstd"
+    "--library-dirs=${zstd}/lib"
   ];
 
+  # Running tests via setup.py triggers an attempt to recompile with the vendored zstd
+  ZSTD_EXTERNAL = 1;
+  VERSION = zstd.version;
+  PKG_VERSION = version;
+  checkInputs = [ pytest ];
+  checkPhase = ''
+    pytest
+  '';
+
   meta = with stdenv.lib; {
     description = "Simple python bindings to Yann Collet ZSTD compression library";
     homepage = "https://github.com/sergey-dryabzhinsky/python-zstd";
diff --git a/pkgs/top-level/python-packages.nix b/pkgs/top-level/python-packages.nix
index 2e99065a727..b2554dc53bf 100644
--- a/pkgs/top-level/python-packages.nix
+++ b/pkgs/top-level/python-packages.nix
@@ -10917,7 +10917,9 @@ EOF
 
   todoist = callPackage ../development/python-modules/todoist { };
 
-  zstd = callPackage ../development/python-modules/zstd { };
+  zstd = callPackage ../development/python-modules/zstd {
+    inherit (pkgs) pkgconfig zstd;
+  };
 
   zxcvbn-python = callPackage ../development/python-modules/zxcvbn-python { };

You don't need the https://github.com/sergey-dryabzhinsky/python-zstd/commit/cbdfbe2477b43e2d527fa4ee5dc960134a051be4.patch . You switched to fetching from github and that was enough to get the tests directory, it is just not available from Pypi.

@eadwu eadwu force-pushed the init/pythonPackages.zstd branch from e674e1b to ccd983c Compare October 30, 2018 20:28
@eadwu
pythonPackages.zstd: init at 1.3.5.1

Verified

This commit was signed with the committer’s verified signature.
eadwu Edmund Wu
GPG key ID: 43C9E53FFCF51DEC
Verified
Learn about vigilant mode
5983aef
@eadwu eadwu force-pushed the init/pythonPackages.zstd branch from ccd983c to 5983aef Compare November 1, 2018 01:34
@veprbl
Copy link
Member

veprbl commented Nov 1, 2018

@GrahamcOfBorg build python2Packages.zstd python3Packages.zstd

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: python2Packages.zstd, python3Packages.zstd

Partial log (click to expand)

rootdir: /build/source, inifile:
collected 16 items

tests/test_compress.py ........s..ss                                     [ 81%]
tests/test_version.py s..                                                [100%]

===================== 12 passed, 4 skipped in 0.08 seconds =====================
pytestcachePhase
/nix/store/0n0q3qal28h8nx6vd5jxydqiphk88ax3-python2.7-zstd-1.3.5.1
/nix/store/1l02m8hpn43gigbj4jakszc6fcf9qvj4-python3.6-zstd-1.3.5.1

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: python2Packages.zstd, python3Packages.zstd

Partial log (click to expand)

rootdir: /build/source, inifile:
collected 16 items

tests/test_compress.py ........s..ss                                     [ 81%]
tests/test_version.py s..                                                [100%]

===================== 12 passed, 4 skipped in 0.07 seconds =====================
pytestcachePhase
/nix/store/8i3ry1bylrrp2ny82blq3dq2cjlxqvya-python2.7-zstd-1.3.5.1
/nix/store/dzxsqlvpji5n18g1g8vq8wmdj1q6gylh-python3.6-zstd-1.3.5.1

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: python2Packages.zstd, python3Packages.zstd

Partial log (click to expand)

collected 16 items

pytestcachePhase
tests/test_compress.py ........s..ss                                     [ 81%]
tests/test_version.py s..                                                [100%]

===================== 12 passed, 4 skipped in 0.11 seconds =====================
pytestcachePhase
/nix/store/y4mwby2lkva9i1bxwi9y6cj4j33bysr9-python2.7-zstd-1.3.5.1
/nix/store/gap9fwnky5za6z7kym7blkx3x8ykkhkf-python3.6-zstd-1.3.5.1

veprbl
veprbl approved these changes Nov 1, 2018
View reviewed changes
Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@dotlambda dotlambda merged commit 0111c7d into NixOS:master Nov 1, 2018
@eadwu eadwu deleted the init/pythonPackages.zstd branch November 17, 2020 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dotlambda dotlambda

@veprbl veprbl

@FRidh FRidh

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@eadwu @veprbl @GrahamcOfBorg @dotlambda