pybind11: init at 2.2.2 #35027

yuriaisaka · 2018-02-16T06:14:18Z

Motivation for this change

pybind11 ( https://github.com/pybind/pybind11 ) is a lightweight header-only library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code.

This package is handy as well as is used at least in a couple of well-known deep learning frameworks (alas used often as git submodules).

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

FRidh · 2018-02-16T07:27:04Z

This is a header-only library so the buildInputs and flags are not needed.

yuriaisaka · 2018-02-16T08:58:18Z

@FRidh Thanks very much for the advise!

The upstream CMakeLists.txt appears to be organized in a way that the standard mkdir build && cd build && cmake .. && make sequence fails when cmake cannot find a python interpreter.

Also, it by default (i.e. without "-DPYBIND11_TEST=0" to cmakeFlags) tries to build test binaries, and cmake fails if it cannot find pytest. And we need numpy etc. in order to build/run all the tests.

But then I've tentatively set doCheck = false as some tests seem to be failing (haven't dug in to details), so indeed catch pytest eigen numpy are unnecessary at this moment, even as nativeBuildInputs.

So how's about something like 57b0610 for now?

FRidh · 2018-02-16T09:08:22Z

But why go through the make process at all? These are headers-only. When not considering the tests, all that needs to be done is move the include folder to $out. Or am I missing something?

yuriaisaka · 2018-02-16T09:47:19Z

Yes, I do think moving the include folder to $out should work just fine in the nix world.

But I thought (1) one would want to run upstream tests and also (2) have .cmake files (pybind11Config.cmake etc.) installed.

For me the 2nd point is more important, and then I can (thanks to nix updating CMAKE_PREFIX_PATH properly) use/create CMake projects that use pybind11 through find_package(pybind11). I think it is useful when I have to communicate with people who are not using nix, but I'm happy to know if there's better "nixy" way to achive that.

knedlsepp · 2018-02-16T10:31:28Z

Using this as a private overlay for some time now, I also got:

propagatedBuildInputs = [
  eigen
  python
  pythonPackages.numpy
  pythonPackages.pytest
  pythonPackages.scipy
];

TBH I don't recall why I did it that way. :-/ maybe I'll be able to look into this in the next days.

yuriaisaka · 2018-02-16T11:33:12Z

Yes, I do think moving the include folder to $out should work just fine in the nix world.

Sorry, I think my statement above was wrong.

pybind11 can be used in two ways: as a tool to embed python runtime to C++, and as a tool to write python modules using C++. The easiest way to achieve the latter is to use the CMake function pybind11_add_module() provided by pybind11, so I guess in this case,

(2) have .cmake files (pybind11Config.cmake etc.) installed.

is a bit more important than being nice to the world outside nix..

sjmackenzie · 2018-04-20T19:42:46Z

any reason why this isn't merged?

pandaman64 · 2018-05-12T13:51:06Z

I need this!

xeji · 2018-08-01T19:10:27Z

@GrahamcOfBorg build pybind11

GrahamcOfBorg · 2018-08-01T19:10:45Z

Success on x86_64-linux (full log)

Attempted: pybind11

Partial log (click to expand)

-- Installing: /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2/share/cmake/pybind11/pybind11ConfigVersion.cmake
-- Installing: /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2/share/cmake/pybind11/FindPythonLibsNew.cmake
-- Installing: /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2/share/cmake/pybind11/pybind11Tools.cmake
-- Installing: /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2/share/cmake/pybind11/pybind11Targets.cmake
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2
strip is /nix/store/1hi76hr87bd1y1q1qjk0lv8nmcjip1c8-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2
checking for references to /build in /nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2...
/nix/store/9dv1may05s8w0x03kc6xzvch6ls5zhxz-pybind-2.2.2

GrahamcOfBorg · 2018-08-01T19:10:52Z

Success on x86_64-darwin (full log)

Attempted: pybind11

Partial log (click to expand)

-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/include/pybind11/complex.h
-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/share/cmake/pybind11/pybind11Config.cmake
-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/share/cmake/pybind11/pybind11ConfigVersion.cmake
-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/share/cmake/pybind11/FindPythonLibsNew.cmake
-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/share/cmake/pybind11/pybind11Tools.cmake
-- Installing: /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2/share/cmake/pybind11/pybind11Targets.cmake
post-installation fixup
strip is /nix/store/qfxcr8c4fg7lkybrny9n2sb77bmippdx-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2
/nix/store/9yp6crr8r0rw60b611r6cp142a5qrj05-pybind-2.2.2

GrahamcOfBorg · 2018-08-01T19:11:15Z

Success on aarch64-linux (full log)

Attempted: pybind11

Partial log (click to expand)

-- Installing: /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2/share/cmake/pybind11/pybind11ConfigVersion.cmake
-- Installing: /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2/share/cmake/pybind11/FindPythonLibsNew.cmake
-- Installing: /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2/share/cmake/pybind11/pybind11Tools.cmake
-- Installing: /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2/share/cmake/pybind11/pybind11Targets.cmake
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2
strip is /nix/store/zrs21zqcchgyabjf4xfimncdq16njizc-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2
checking for references to /build in /nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2...
/nix/store/ksmvmml8b29fkz8r90ncp52rhb0lypgg-pybind-2.2.2

knedlsepp · 2018-08-01T20:36:27Z

Haven't come around it yet, but it seems to me there's now a python style and a cmake based style to install this. Both yielding a little different results. We might need to reinvestigate.

jluttine · 2019-01-06T13:09:59Z

I have problems building pyopencl that has pybind11 as a dependency: pybind11 is not found. Perhaps it's because it's not installed as a Python package at the moment? I opened an issue: #53497

GrahamcOfBorg added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Feb 16, 2018

Yuri Aisaka added 3 commits February 16, 2018 17:54

pybind11: init at 2.2.2

236d4f9

fix license

4c1eef5

remove test dependencies

57b0610

yuriaisaka force-pushed the pr-pybind11-init branch from a01f9b5 to 57b0610 Compare February 16, 2018 08:57

xeji merged commit 9caf7ec into NixOS:master Aug 1, 2018

jluttine mentioned this pull request Jan 6, 2019

pybind11 not installed correctly #53497

Closed

dotlambda mentioned this pull request Jan 29, 2019

pythonPackages.pybind11: init at 2.2.4 #54888

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pybind11: init at 2.2.2 #35027

pybind11: init at 2.2.2 #35027

yuriaisaka commented Feb 16, 2018

FRidh commented Feb 16, 2018

yuriaisaka commented Feb 16, 2018 •

edited

FRidh commented Feb 16, 2018

yuriaisaka commented Feb 16, 2018 •

edited

knedlsepp commented Feb 16, 2018 •

edited

yuriaisaka commented Feb 16, 2018

sjmackenzie commented Apr 20, 2018

pandaman64 commented May 12, 2018

xeji commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

knedlsepp commented Aug 1, 2018

jluttine commented Jan 6, 2019

pybind11: init at 2.2.2 #35027

pybind11: init at 2.2.2 #35027

Conversation

yuriaisaka commented Feb 16, 2018

Motivation for this change

Things done

FRidh commented Feb 16, 2018

yuriaisaka commented Feb 16, 2018 • edited

FRidh commented Feb 16, 2018

yuriaisaka commented Feb 16, 2018 • edited

knedlsepp commented Feb 16, 2018 • edited

yuriaisaka commented Feb 16, 2018

sjmackenzie commented Apr 20, 2018

pandaman64 commented May 12, 2018

xeji commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

GrahamcOfBorg commented Aug 1, 2018

knedlsepp commented Aug 1, 2018

jluttine commented Jan 6, 2019

yuriaisaka commented Feb 16, 2018 •

edited

yuriaisaka commented Feb 16, 2018 •

edited

knedlsepp commented Feb 16, 2018 •

edited