python3Packages.gpgme: enable python3 support #46304

teto · 2018-09-07T09:02:26Z

when calling python3Packages.gpgme, it was still picking python2.
Changed withPython into pythonSupport since it's the convention.

I had to enable ncurses too because of this error when configuring:

configure:19858: checking for Python include path
configure:19874: result: -I/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/include/python3.6m
configure:19881: checking for Python library path
configure:19958: result: -L/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib -lpython3.6m
configure:19965: checking for Python site-packages path
configure:19971: result: /nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib/python3.6/site-packages
configure:19978: checking python extra libraries
configure:19985: result: -lpthread -ldl -lcrypt -lncurses -lutil -lm
configure:19992: checking python extra linking flags
configure:19999: result: -Xlinker -export-dynamic
configure:20006: checking consistency of all components of python development environment
configure:20032: gcc -o conftest -g -O2  -I/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/include/python3.6m  conftest.c  -L/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib -lpython3.6m -Xlinker -export-dynamic -lpthread -ldl -lcrypt -lncurses -lutil -lm >&5
/nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/ld: cannot find -lncurses
collect2: error: ld returned 1 exit status
configure:20032: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "gpgme"
| #define PACKAGE_TARNAME "gpgme"
| #define PACKAGE_VERSION "1.11.1"
| #define PACKAGE_STRING "gpgme 1.11.1"
| #define PACKAGE_BUGREPORT "http://bugs.gnupg.org"
| #define PACKAGE_URL ""
| #define PACKAGE "gpgme"
| #define VERSION "1.11.1"
| #define STDC_HEADERS 1
| #define HAVE_SYS_TYPES_H 1
| #define HAVE_SYS_STAT_H 1
| #define HAVE_STDLIB_H 1
| #define HAVE_STRING_H 1
| #define HAVE_MEMORY_H 1
| #define HAVE_STRINGS_H 1
| #define HAVE_INTTYPES_H 1
| #define HAVE_STDINT_H 1
| #define HAVE_UNISTD_H 1
| #define __EXTENSIONS__ 1
| #define _ALL_SOURCE 1
| #define _GNU_SOURCE 1
| #define _POSIX_PTHREAD_SEMANTICS 1
| #define _TANDEM_SOURCE 1
| #define PACKAGE "gpgme"
| #define VERSION "1.11.1"
| #define HAVE_DLFCN_H 1
| #define LT_OBJDIR ".libs/"
| #define FIXED_SEARCH_PATH "/nix/store/cf1fwlgilrv4z45qw5zvqkkvb1bv6zyv-gnupg-2.2.9/bin"
| /* end confdefs.h.  */
| 
| 		#include <Python.h>
| int
| main ()
| {
| Py_Initialize();
|   ;
|   return 0;
| }
| 
configure:20049: result: no
configure:20062: WARNING:
  Could not link test program to Python. Maybe the main Python library has been
  installed in some non-standard library path. If so, pass it to configure,
  via the LDFLAGS environment variable.
  Example: ./configure LDFLAGS="-L/usr/non-standard-path/python/lib"
  ============================================================================
   You probably have to install the development version of the Python package
   for your distribution.  The exact name of this package varies among them.
  ============================================================================

Motivation for this change

Need python3 bindings for my mail agent "alot"

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

teto · 2018-09-25T16:10:19Z

@FRidh should I put this into staging ?

dtzWill · 2018-10-10T04:32:02Z

LGTM! I actually did largely the same thing to get gpgme working w/py3 for use with recent alot:

dtzWill@70c4b8f

(commit not intended for upstream inclusion as-is, but even so it's what I've been using and works for me--yours looks the same but better ;))

teto · 2018-10-15T07:47:09Z

@GrahamcOfBorg build python3Packages.gpgme

GrahamcOfBorg · 2018-10-15T07:48:37Z

Failure on aarch64-linux (full log)

Attempted: python3Packages.gpgme

Partial log (click to expand)

   You probably have to install the development version of the Python package
   for your distribution.  The exact name of this package varies among them.
  ============================================================================

configure: error:
***
*** Please install the python development packages.
***
builder for '/nix/store/7y8pxhs5r15h4fp23zjrndr6ka9mi8kh-gpgme-1.12.0.drv' failed with exit code 1
error: build of '/nix/store/7y8pxhs5r15h4fp23zjrndr6ka9mi8kh-gpgme-1.12.0.drv' failed

GrahamcOfBorg · 2018-10-15T07:48:45Z

Failure on x86_64-linux (full log)

Attempted: python3Packages.gpgme

Partial log (click to expand)

   You probably have to install the development version of the Python package
   for your distribution.  The exact name of this package varies among them.
  ============================================================================

configure: error:
***
*** Please install the python development packages.
***
builder for '/nix/store/xvviaf9nfjbxy49z5qm9329z1kwz7cvs-gpgme-1.12.0.drv' failed with exit code 1
error: build of '/nix/store/xvviaf9nfjbxy49z5qm9329z1kwz7cvs-gpgme-1.12.0.drv' failed

teto · 2018-10-18T07:12:00Z

Needed to test master version of alot with python3 support @Fuuzetsu @primeos

primeos · 2018-10-18T12:00:39Z

Needed to test master version of alot with python3 support @Fuuzetsu @primeos

What do you mean with this comment? According to GrahamcOfBorg gpgme doesn't build with Python(3) support, that should be fixed first. Also: I'm happy to help with gpgme but I'm neither using nor maintaining alot.

teto · 2018-10-18T12:32:12Z

@GrahamcOfBorg build python3Packages.gpgme

GrahamcOfBorg · 2018-10-18T12:35:15Z

Success on aarch64-linux (full log)

Attempted: python3Packages.gpgme

Partial log (click to expand)

shrinking /nix/store/72wfnshv009a5m211l0zmgfmlsrpss6q-gpgme-1.12.0-dev/bin/gpgme-tool
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/72wfnshv009a5m211l0zmgfmlsrpss6q-gpgme-1.12.0-dev/bin
patching script interpreter paths in /nix/store/72wfnshv009a5m211l0zmgfmlsrpss6q-gpgme-1.12.0-dev
/nix/store/72wfnshv009a5m211l0zmgfmlsrpss6q-gpgme-1.12.0-dev/bin/gpgme-config: interpreter directive changed from "/bin/sh" to "/nix/store/dsyc1z7ck08ga7l0b1jcxx35wj69qcii-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/72wfnshv009a5m211l0zmgfmlsrpss6q-gpgme-1.12.0-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/b3x9nskcp09jjg51fxcgp49gkdnxqi1z-gpgme-1.12.0-info
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/b3x9nskcp09jjg51fxcgp49gkdnxqi1z-gpgme-1.12.0-info
checking for references to /build in /nix/store/b3x9nskcp09jjg51fxcgp49gkdnxqi1z-gpgme-1.12.0-info...

GrahamcOfBorg · 2018-10-18T12:42:11Z

Success on x86_64-linux (full log)

Attempted: python3Packages.gpgme

Partial log (click to expand)

shrinking /nix/store/p265lww9xk8lzg0fbjlvwg936w67bny9-gpgme-1.12.0-dev/bin/gpgme-json
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/p265lww9xk8lzg0fbjlvwg936w67bny9-gpgme-1.12.0-dev/bin
patching script interpreter paths in /nix/store/p265lww9xk8lzg0fbjlvwg936w67bny9-gpgme-1.12.0-dev
/nix/store/p265lww9xk8lzg0fbjlvwg936w67bny9-gpgme-1.12.0-dev/bin/gpgme-config: interpreter directive changed from "/bin/sh" to "/nix/store/r47p5pzx52m3n34vdgqpk5rvqgm0m24m-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/p265lww9xk8lzg0fbjlvwg936w67bny9-gpgme-1.12.0-dev...
shrinking RPATHs of ELF executables and libraries in /nix/store/lbrvq1bw203d0939c2n16x2ns6gaja13-gpgme-1.12.0-info
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/lbrvq1bw203d0939c2n16x2ns6gaja13-gpgme-1.12.0-info
checking for references to /build in /nix/store/lbrvq1bw203d0939c2n16x2ns6gaja13-gpgme-1.12.0-info...

teto · 2018-10-18T12:45:14Z

@primeos true my comment was a bit cryptic. I just looked for the gpgme maintainers to see if I could do anything to merge this. GrahamcOfBorg didn't trigger on my force push but it now compiles (and works with the mail client alot)

primeos · 2018-10-18T22:08:17Z

pkgs/development/libraries/gpgme/default.nix

 , autoreconfHook
 , git
 , texinfo
 , qtbase ? null
-, withPython ? false, swig2 ? null, python ? null
+, pythonSupport ? false, swig2 ? null, python ? null


I'm not sure about this change. It obviously makes sense to stick to conventions but I'm also against backward incompatible changes that could break setups (I don't know how many users override withPython). I also wonder if this convention is documented somewhere (didn't find anything inside the manuals).

But pythonSupport is definitely used more often:

$ git grep withPython | wc -l 31 $ git grep pythonSupport | wc -l 89

I remember a PR to improve python docs where I suggested to add it but can't find it again. Another suggestion is 39e9de1#r29184438 .
I am for changing the parameter name as my main grip with nixpkgs is its lack of consistency (across languages mainly).
People needing the python support must have been using pythonPackages.gpgme so it should impact 0 or few people.

I am not sure quite why we need ncurses except for this command line done by gpgme.

configure:20006: checking consistency of all components of python development environment configure:20032: gcc -o conftest -g -O2 -I/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/include/python3.6m conftest.c -L/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib -lpython3.6m -Xlinker -export-dynamic -lpthread -ldl -lcrypt -lncurses -lutil -lm >&5 /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/ld: cannot find -lncurses

anyway I request ncurses only when python is used.

Ok, thanks for elaborating. You're right, due to pythonPackages.gpgme this shouldn't cause much problems :)

primeos · 2018-10-18T22:19:20Z

pkgs/development/libraries/gpgme/default.nix

@@ -28,8 +28,8 @@ stdenv.mkDerivation rec {
    [ libgpgerror glib libassuan pth ]
    ++ lib.optional (qtbase != null) qtbase;

-  nativeBuildInputs = [ file pkgconfig gnupg autoreconfHook git texinfo ]
-  ++ lib.optionals withPython [ python swig2 which ];
+  nativeBuildInputs = [ file pkgconfig gnupg autoreconfHook git texinfo ncurses ]


Can you move ncurses to the next line (only add it for Python support) or is there a reason to always add it (I couldn't find a dependency on ncurses in the source code)?

when calling python3Packages.gpgme, it was still picking python2. Changed withPython into pythonSupport since it's the convention. I had to enable ncurses too because of this error when configuring: configure:19978: checking python extra libraries configure:19985: result: -lpthread -ldl -lcrypt -lncurses -lutil -lm configure:19992: checking python extra linking flags configure:19999: result: -Xlinker -export-dynamic configure:20006: checking consistency of all components of python development environment configure:20032: gcc -o conftest -g -O2 -I/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/include/python3.6m conftest.c -L/nix/store/hy65mn4wjswqih75gfr6g4q3xgqdm325-python3-3.6.6/lib -lpython3.6m -Xlinker -export-dynamic -lpthread -ldl -lcrypt -lncurses -lutil -lm >&5 /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/ld: cannot find -lncurses collect2: error: ld returned 1 exit status configure:20032: $? = 1

primeos

Thanks, all rebuilds still succeed :)

teto requested a review from FRidh as a code owner September 7, 2018 09:02

GrahamcOfBorg added 6.topic: python 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Sep 7, 2018

teto mentioned this pull request Oct 10, 2018

gpgme: 1.11.1 -> 1.12.0 #48129

Merged

9 tasks

teto changed the title ~~pythonPackages.gpgme: enable python3 support~~ python3Packages.gpgme: enable python3 support Oct 15, 2018

teto force-pushed the gpgme_python3 branch from 1ee9cf7 to fdfc920 Compare October 15, 2018 07:47

teto changed the title ~~python3Packages.gpgme: enable python3 support~~ python3Packages.gpgme: enable python3 support [wip] Oct 15, 2018

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Oct 15, 2018

teto force-pushed the gpgme_python3 branch from fdfc920 to 75571c8 Compare October 18, 2018 06:42

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 and removed 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Oct 18, 2018

teto changed the title ~~python3Packages.gpgme: enable python3 support [wip]~~ python3Packages.gpgme: enable python3 support Oct 18, 2018

primeos requested changes Oct 18, 2018

View reviewed changes

teto force-pushed the gpgme_python3 branch from 75571c8 to a336f2b Compare October 19, 2018 05:19

teto force-pushed the gpgme_python3 branch from a336f2b to 3902b3f Compare October 19, 2018 05:20

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Oct 19, 2018

primeos approved these changes Oct 19, 2018

View reviewed changes

primeos merged commit 02b5f4f into NixOS:master Oct 19, 2018

teto deleted the gpgme_python3 branch May 13, 2020 09:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python3Packages.gpgme: enable python3 support #46304

python3Packages.gpgme: enable python3 support #46304

teto commented Sep 7, 2018 •

edited

teto commented Sep 25, 2018

dtzWill commented Oct 10, 2018

teto commented Oct 15, 2018 •

edited

GrahamcOfBorg commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

teto commented Oct 18, 2018

primeos commented Oct 18, 2018

teto commented Oct 18, 2018

GrahamcOfBorg commented Oct 18, 2018

GrahamcOfBorg commented Oct 18, 2018

teto commented Oct 18, 2018

primeos Oct 18, 2018

teto Oct 19, 2018

primeos Oct 19, 2018

primeos Oct 18, 2018

primeos left a comment

python3Packages.gpgme: enable python3 support #46304

python3Packages.gpgme: enable python3 support #46304

Conversation

teto commented Sep 7, 2018 • edited

Motivation for this change

Things done

teto commented Sep 25, 2018

dtzWill commented Oct 10, 2018

teto commented Oct 15, 2018 • edited

GrahamcOfBorg commented Oct 15, 2018

GrahamcOfBorg commented Oct 15, 2018

teto commented Oct 18, 2018

primeos commented Oct 18, 2018

teto commented Oct 18, 2018

GrahamcOfBorg commented Oct 18, 2018

GrahamcOfBorg commented Oct 18, 2018

teto commented Oct 18, 2018

primeos Oct 18, 2018

Choose a reason for hiding this comment

teto Oct 19, 2018

Choose a reason for hiding this comment

primeos Oct 19, 2018

Choose a reason for hiding this comment

primeos Oct 18, 2018

Choose a reason for hiding this comment

primeos left a comment

Choose a reason for hiding this comment

teto commented Sep 7, 2018 •

edited

teto commented Oct 15, 2018 •

edited