pythonPackages update: asn1ate and asn2quickder #34161

leenaars · 2018-01-22T21:01:58Z

Motivation for this change

Upgrade to latest released version of these asn1 tools.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

dotlambda · 2018-01-22T22:44:19Z

pkgs/development/python-modules/asn1ate/default.nix

@@ -3,13 +3,12 @@

 buildPythonPackage rec {
  pname = "asn1ate";
-  date = "20160810";
-  version = "unstable-${date}";
+  version= "0.6";
  name = "${pname}-${version}";


You can delete the name.

Isn't that mandatory? I mean, in the sense that this creates the final name for the package?

Not for Python. buildPythonPackage will automatically derive it from pname and version.

Done. Thanks for helping out!

dotlambda · 2018-01-22T22:46:05Z

pkgs/development/tools/asn2quickder/default.nix


-stdenv.mkDerivation rec {
+python2Packages.buildPythonPackage rec {
  pname = "asn2quickder";
  name = "${pname}-${version}";


Leave out name here as well.

dotlambda · 2018-01-22T22:47:19Z

pkgs/development/tools/asn2quickder/default.nix


-stdenv.mkDerivation rec {
+python2Packages.buildPythonPackage rec {


Better use python2Packages.callPackage and take buildPythonPackage as well as all dependencies as arguments.

I don't understand what you mean by that? I thought his was a very common way of doing this.
Do you have a recommended example where this is done elsewhere in nixpkgs.
I grepped but could not find something that seems to fit here ...

In all-packages.nix, one can use python2Packages.callPackage. And in default.nix the first line should look something like

{ stdenv, fetchFromGitHub, python2Packages, makeWrapper, cmake, pytestrunner, pytest, six, pyparsing }:

Actually, this should be buildPythonApplication, since it's not a library.

@dotlambda: When I do that, for some reason it tries to create a wheel and cannot find six. So it doesn't build anymore.

It builds perfectly fine as is. Would it not be okay to leave it as is? I only did a version bump...

{ stdenv, buildPythonApplication, fetchFromGitHub, makeWrapper, cmake, pytestrunner, pytest, six, pyparsing }: buildPythonApplication rec { pname = "asn2quickder"; version = "1.2-5"; src = fetchFromGitHub { sha256 = "0qwd3c0q7j8d22frqxrgqjnji36slnm8c2px6iyrvpd85xamza44"; rev = "version-${version}"; owner = "vanrein"; repo = "quick-der"; }; buildInputs = [ makeWrapper cmake pytestrunner pytest six ]; propagatedBuildInputs = [ pyparsing ]; meta = with stdenv.lib; { description = "An ASN.1 compiler with a backend for Quick DER"; homepage = https://github.com/vanrein/asn2quickder; license = licenses.bsd3; platforms = platforms.linux; maintainers = with maintainers; [ leenaars ]; }; }

By the way, some of those dependencies are nativeBuildInputs.

Thanks. Done.

leenaars · 2018-01-23T07:21:56Z

@dotlambda: I removed the 'name' but it turns out then you get a real ugly prefix, e.g. python2.7-asn1ate-0.6 and python2.7-asn2quickder-1.2-5 instead of asn1ate-0.6. So I reverted back to what I had.

dotlambda · 2018-01-23T07:38:03Z

Actually, I think this should be called quick-der now.
We can add asn2quickder as an alias.
But then, it doesn't make sense to use buildPythonApplication, since quick-der is much more than a python executable.
I don't have enough experience to know how to package C and Python libraries at the same time. Maybe @FRidh can help.

leenaars · 2018-01-23T07:46:09Z

I don't know either, my solution was to keep them apart. That at least I understand and works... I have a separate quickder on my system which I will submit soon.

I don't think this should cost anyone a lot of time at this stage. When we would run into any issues later on, this could be fixed then. This is very useful but not mainstream technical tooling...

leenaars · 2018-01-23T18:47:56Z

All suggestions have been processed, should be ready to merge.

Mic92 · 2018-01-24T06:35:31Z

@GrahamcOfBorg build quickder

GrahamcOfBorg

Failure for system: x86_64-linux

Skipped ASN.1 header file generation for rfc2578.asn1
rm -f quick-der
ln -s . quick-der
Skipped compilation of skipped header files
install: cannot stat 'rfc3280.h': No such file or directory
make[1]: *** [Makefile:39: install] Error 1
make[1]: Leaving directory '/tmp/nix-build-quickder-1.0-RC2.drv-0/source/rfc'
make: *** [Makefile:20: install] Error 2
builder for ‘/nix/store/c82bfpjpiizfmxg90qymz9gkm5xb4rkl-quickder-1.0-RC2.drv’ failed with exit code 2
error: build of ‘/nix/store/c82bfpjpiizfmxg90qymz9gkm5xb4rkl-quickder-1.0-RC2.drv’ failed

GrahamcOfBorg

Failure for system: x86_64-darwin

Package ‘quickder-1.0-RC2’ in /tmp/nix-ofborg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/libraries/quickder/default.nix:32 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg

Failure for system: aarch64-linux

Skipped ASN.1 header file generation for rfc2578.asn1
rm -f quick-der
ln -s . quick-der
Skipped compilation of skipped header files
install: cannot stat 'rfc3280.h': No such file or directory
make[1]: *** [Makefile:39: install] Error 1
make[1]: Leaving directory '/build/source/rfc'
make: *** [Makefile:20: install] Error 2
builder for '/nix/store/aqy0wjncd0f4g5qzzjh2iwqqhjrxw3nm-quickder-1.0-RC2.drv' failed with exit code 2
error: build of '/nix/store/aqy0wjncd0f4g5qzzjh2iwqqhjrxw3nm-quickder-1.0-RC2.drv' failed

Mic92 · 2018-01-24T17:06:23Z

Was quickder broken before?

FRidh · 2018-02-11T08:43:10Z

asn2quickder builds on master but not with this commit. Some fixing is needed.

leenaars · 2018-02-12T18:37:20Z

I updated quickder to the latest release, everything should work now.
I also rebased to the current HEAD, and it merges correctly.

FRidh · 2018-02-18T16:06:32Z

@GrahamcOfBorg build quickder

GrahamcOfBorg · 2018-02-18T16:07:10Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

make[1]: *** Waiting for unfinished jobs....
[ 97%] Built target rfc4531_asn1_h
[ 98%] Build Python script rfc5805.py from ASN.1 spec
[ 98%] Built target rfc4120_asn1_h
[ 98%] Built target rfc5805_asn1_h
[ 98%] Built target kxover_asn1_h
[ 98%] Built target RemotePKCS11_asn1_h
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/8amsviz8pz2gjyd6mcad0k3wa62c4dpp-quickder-1.2-6.drv' failed with exit code 2
error: build of '/nix/store/8amsviz8pz2gjyd6mcad0k3wa62c4dpp-quickder-1.2-6.drv' failed

FRidh · 2018-02-18T16:07:35Z

@leenaars quickder still seems to be failing.

GrahamcOfBorg · 2018-02-18T16:08:12Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

[100%] Built target rfc5805_asn1_h
[100%] Built target rfc4120_asn1_h
[100%] Built target rfc4373_asn1_h
[100%] Built target rfc2315_asn1_h
[100%] Built target rfc7292_asn1_h
[100%] Built target kxover_asn1_h
[100%] Built target RemotePKCS11_asn1_h
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/gwnyic3yjxqpz85bm572vcajgri02jzz-quickder-1.2-6.drv' failed with exit code 2
error: build of '/nix/store/gwnyic3yjxqpz85bm572vcajgri02jzz-quickder-1.2-6.drv' failed

GrahamcOfBorg · 2018-02-18T16:21:48Z

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Package ‘quickder-1.2-6’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/libraries/quickder/default.nix:51 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

leenaars · 2018-02-18T21:02:54Z

The failure is in both cases a script that is called which fails on:
/usr/bin/env: bad interpreter: No such file or directory

A little digging reveals that a chrooted system does not have /usr/bin/env by design.
(#6227)...

I used patchShebangs to fix the issue.

leenaars · 2018-02-18T21:04:45Z

@GrahamcOfBorg build quickder

Mic92 · 2018-02-20T11:17:28Z

@GrahamcOfBorg build quickder

GrahamcOfBorg · 2018-02-20T11:17:42Z

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Package ‘quickder-1.2-6’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/libraries/quickder/default.nix:51 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-02-20T11:18:31Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

Scanning dependencies of target asn1-spec-modules
[ 97%] Built target asn1-spec-modules
Scanning dependencies of target kxover_asn1_md
[ 98%] Build markdown text file kxover.md from ASN.1 spec
/nix/store/fcxliihmhm2ak9z4890gk0qw63zsbrcx-bash-4.4-p12/bin/bash: /tmp/nix-build-quickder-1.2-6.drv-0/source/python/scripts/asn1literate: /usr/bin/env: bad interpreter: No such file or directory
make[2]: *** [arpa2/CMakeFiles/kxover_asn1_md.dir/build.make:61: arpa2/doc/kxover.md] Error 126
make[1]: *** [CMakeFiles/Makefile2:1167: arpa2/CMakeFiles/kxover_asn1_md.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
builder for ‘/nix/store/np8fkjzrp1lay1avgsw0a0yyclrzk61l-quickder-1.2-6.drv’ failed with exit code 2
error: build of ‘/nix/store/np8fkjzrp1lay1avgsw0a0yyclrzk61l-quickder-1.2-6.drv’ failed

GrahamcOfBorg · 2018-02-20T11:18:43Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

[100%] Built target rfc4531_asn1_h
[100%] Built target rfc4120_asn1_h
[100%] Built target rfc4373_asn1_h
[100%] Built target rfc2315_asn1_h
[100%] Built target rfc7292_asn1_h
[100%] Built target kxover_asn1_h
[100%] Built target RemotePKCS11_asn1_h
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/10lbn5321ihfqcjzgwbrn6zyflf84l7j-quickder-1.2-6.drv' failed with exit code 2
�[31;1merror:�[0m build of '/nix/store/10lbn5321ihfqcjzgwbrn6zyflf84l7j-quickder-1.2-6.drv' failed

Mic92 · 2018-02-20T11:24:25Z

@GrahamcOfBorg build quickder

GrahamcOfBorg · 2018-02-20T11:39:29Z

Success on x86_64-linux (full log)

Partial log (click to expand)

-- Installing: /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6/share/doc/quick-der/kxover.md
-- Installing: /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6/share/doc/quick-der/RemotePKCS11.md
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6
shrinking /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6/lib/libquickder.so
strip is /nix/store/adidfx4pa7vmvby0gjqqmiwg2x49yr27-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6/lib 
patching script interpreter paths in /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6
checking for references to /tmp/nix-build-quickder-1.2-6.drv-0 in /nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6...
/nix/store/hhc46r9wnmz16hshafgqmf6b11ynap4k-quickder-1.2-6

GrahamcOfBorg · 2018-02-20T12:09:56Z

Success on aarch64-linux (full log)

Partial log (click to expand)

-- Installing: /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6/share/doc/quick-der/kxover.md
-- Installing: /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6/share/doc/quick-der/RemotePKCS11.md
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6
shrinking /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6/lib/libquickder.so
strip is /nix/store/skd6ix5ipkyhxzq7naylj4digawakl4j-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6/lib
patching script interpreter paths in /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6
checking for references to /build in /nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6...
/nix/store/jc4vv1dkhnlljnn00gmqlil55gbj3xz2-quickder-1.2-6

GrahamcOfBorg · 2018-02-20T12:26:36Z

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Package ‘quickder-1.2-6’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/libraries/quickder/default.nix:53 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

leenaars requested a review from FRidh as a code owner January 22, 2018 21:01

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Jan 22, 2018

dotlambda reviewed Jan 22, 2018

View reviewed changes

leenaars force-pushed the asn2quickder branch 2 times, most recently from 1122567 to 1d4ca59 Compare January 23, 2018 07:18

leenaars force-pushed the asn2quickder branch from 1d4ca59 to 852706c Compare January 23, 2018 18:43

GrahamcOfBorg reviewed Jan 24, 2018

View reviewed changes

FRidh added 6.topic: python 2.status: work-in-progress labels Feb 11, 2018

leenaars force-pushed the asn2quickder branch from 852706c to 44b2b15 Compare February 12, 2018 16:44

GrahamcOfBorg added the 8.has: package (new) label Feb 12, 2018

leenaars force-pushed the asn2quickder branch from 44b2b15 to 5496150 Compare February 12, 2018 16:54

leenaars added 2 commits February 12, 2018 19:43

arpa2cm: init -> 0.5

ae6887d

pythonPackages.asn1ate: unstable -> 0.6

32efe66

leenaars force-pushed the asn2quickder branch from 5496150 to c473eff Compare February 12, 2018 18:43

leenaars force-pushed the asn2quickder branch from c473eff to 2b90592 Compare February 18, 2018 21:00

leenaars added 2 commits February 18, 2018 22:01

asn2quickder: 0.7RC1 -> 1.2-6

4f53b77

quickder: 1.0-RC2 -> 1.2-6

1afb012

leenaars force-pushed the asn2quickder branch from 2b90592 to 1afb012 Compare February 18, 2018 21:02

quickder: fix python shebangs

5859716

Mic92 merged commit 825ff64 into NixOS:master Feb 20, 2018


		stdenv.mkDerivation rec {
		python2Packages.buildPythonPackage rec {

pythonPackages update: asn1ate and asn2quickder #34161

pythonPackages update: asn1ate and asn2quickder #34161

Conversation

leenaars commented Jan 22, 2018

Motivation for this change

Things done

Choose a reason for hiding this comment

leenaars Jan 23, 2018 • edited

Choose a reason for hiding this comment

dotlambda Jan 23, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leenaars commented Jan 23, 2018

dotlambda commented Jan 23, 2018 • edited

leenaars commented Jan 23, 2018

leenaars commented Jan 23, 2018

Mic92 commented Jan 24, 2018

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

GrahamcOfBorg left a comment

Choose a reason for hiding this comment

Mic92 commented Jan 24, 2018

FRidh commented Feb 11, 2018

leenaars commented Feb 12, 2018 • edited

FRidh commented Feb 18, 2018

GrahamcOfBorg commented Feb 18, 2018

FRidh commented Feb 18, 2018

GrahamcOfBorg commented Feb 18, 2018

GrahamcOfBorg commented Feb 18, 2018

leenaars commented Feb 18, 2018 • edited

leenaars commented Feb 18, 2018

Mic92 commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

Mic92 commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

GrahamcOfBorg commented Feb 20, 2018

leenaars Jan 23, 2018 •

edited

dotlambda Jan 23, 2018 •

edited

dotlambda commented Jan 23, 2018 •

edited

leenaars commented Feb 12, 2018 •

edited

leenaars commented Feb 18, 2018 •

edited