libvirt: darwin/linux fix dlopen("libjansson.so.4") #45324

teto · 2018-08-19T08:54:24Z

Motivation for this change

Commit 59027e2 broke libvirt
Revert "libvirt: fix dlopen("libjansson.so.4")"

broke libvirt/nixops on linux :/ I am not sure what's the proper fix but meanwhile could we wrap the change in an if darwin ?

cf
#44900

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Determined the impact on package closure size (by running nix path-info -S before and after)
Fits CONTRIBUTING.md.

GrahamcOfBorg · 2018-08-19T09:14:39Z

Success on x86_64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

patching script interpreter paths in /nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/var/lib/rc.d/init.d/virtlockd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/var/lib/rc.d/init.d/libvirtd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/var/lib/rc.d/init.d/virtlogd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/var/lib/rc.d/init.d/libvirt-guests: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0...
moving /nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/sbin/* to /nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0/bin
/nix/store/jcr9jlp7bxzmpvlmb3zf2wz7lcgvyb44-libvirt-4.6.0

GrahamcOfBorg · 2018-08-19T09:25:20Z

Success on aarch64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

patching script interpreter paths in /nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/var/lib/rc.d/init.d/libvirt-guests: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/var/lib/rc.d/init.d/virtlockd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/var/lib/rc.d/init.d/libvirtd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/var/lib/rc.d/init.d/virtlogd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0...
moving /nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/sbin/* to /nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0/bin
/nix/store/37cfpicx7axzs3c1y7lsnz69dx8p85jv-libvirt-4.6.0

teto · 2018-08-19T09:26:30Z

@pikajude what do you think ?

LnL7 · 2018-08-19T11:12:19Z

#44900 (comment)

teto · 2018-08-19T11:39:09Z

@LnL7 Does that mean replace .so with targetPlatform.extensions.sharedLibrary ?

LnL7 · 2018-08-19T12:17:42Z

I don't understand what the problem is, the previous thread wasn't exactly productive.

But if you look at the build output (FYI. you don't need darwin for this) you'll see that there's a dylib instead of so, which is pretty common.

$ curl https://cache.nixos.org/fvzch4km3hqpz05752ndj99hqymk75qp.ls | brotli -d | jq .
...
          "libjansson.4.dylib"
          "libjansson.dylib"
          "libjansson.la"


$ nix-store -r /nix/store/fvzch4km3hqpz05752ndj99hqymk75qp-jansson-2.11 \
  && ls /nix/store/fvzch4km3hqpz05752ndj99hqymk75qp-jansson-2.11/lib
libjansson.4.dylib libjansson.dylib   libjansson.la      pkgconfig

So what's wrong with this?

     patchShebangs . # fixes /usr/bin/python references'
+    substituteInPlace src/util/virjsoncompat.c --replace \
+        '"libjansson.so.4"' '"${jansson}/lib/libjansson${targetPlatform.extensions.sharedLibrary}"'
   '';

teto · 2018-08-19T12:51:30Z

The commit #44900 was purely reverted so I assumed libjansson was not mandatory for darwin else I don't see the point in the revert it just broke libvirt on both platforms. Also don't assume everyone is familiar with every bit of nixpkgs it's the first time I come across ${targetPlatform.extensions.sharedLibrary} hence my question, I am just trying to help here (as you are).

Commit 59027e2 broke libvirt broke libvirt/nixops on linux. Use ${targetPlatform.extensions.sharedLibrary} to accomodate both linux and darwin.

GrahamcOfBorg · 2018-08-19T14:01:00Z

Success on x86_64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

patching script interpreter paths in /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/libvirtd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/virtlockd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/libvirt-guests: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/virtlogd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0...
moving /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/sbin/* to /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/bin
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0

teto · 2018-08-19T14:02:55Z

I pushed your fix, it is fine on linux. Hope some macos user can test it too

GrahamcOfBorg · 2018-08-19T14:07:09Z

Success on aarch64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

patching script interpreter paths in /nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/var/lib/rc.d/init.d/libvirt-guests: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/var/lib/rc.d/init.d/virtlockd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/var/lib/rc.d/init.d/libvirtd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/var/lib/rc.d/init.d/virtlogd: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/x8npv3gr1rq1pf6gqpcdb9b68374pfd6-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0...
moving /nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/sbin/* to /nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0/bin
/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0

flokli · 2018-08-19T19:05:03Z

@teto was able to build it on darwin, start libvirtd, and it kept running.
I was also able to connect via virsh to a remote system, and could instruct libvirt commands there - so nixopsing should work too after this is merged.

Note: I could connect to the locally started libvirtd as well, but wasn't able to define any domains - libvirt kept complaining it couldn't find capabilities for domaintype=qemu - unsure if running qemu with darwin as host ever worked, so this should be fine for now.

LnL7 · 2018-08-19T19:05:43Z

@teto I wasn't talking about the sharedLibrary helper specifically, if ... then "dylib" else "so", while not as nice will do the same thing. I'm just a bit confused why there's so much discussion about something that almost looks easier to fix then revert.

Just disabling the fix on darwin wouldn't break nixops anymore, but I assume (since it builds on darwin) you could run into the same problem there.

LnL7 · 2018-08-19T19:06:26Z

@GrahamcOfBorg build libvirt

GrahamcOfBorg · 2018-08-19T19:06:50Z

Success on aarch64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

/nix/store/24y5k8gynpdxi0rk5qrf4wrvf59l43zy-libvirt-4.6.0

GrahamcOfBorg · 2018-08-19T19:12:59Z

Success on x86_64-linux (full log)

Attempted: libvirt

Partial log (click to expand)

patching script interpreter paths in /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/libvirt-guests: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/virtlockd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/libvirtd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/var/lib/rc.d/init.d/virtlogd: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/79b6s2wql94lwk8dyib2jc4fq8ixl13f-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0...
moving /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/sbin/* to /nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0/bin
/nix/store/id6684w4qx8xblvwgisyi97b8rk4b5bq-libvirt-4.6.0

GrahamcOfBorg · 2018-08-19T19:13:27Z

Success on x86_64-darwin (full log)

Attempted: libvirt

Partial log (click to expand)

Removing /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/share/gtk-doc
rmdir: failed to remove '/nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/share': Directory not empty
gzipping man pages under /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/share/man/
strip is /nix/store/bg39mxc401qqxgjasxfgardi6xl2ikmc-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/lib  /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/libexec  /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/bin  /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/sbin
patching script interpreter paths in /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0
/nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/bin/virt-xml-validate: interpreter directive changed from "/bin/sh" to "/nix/store/avxlkpsr36xpnsy0byq2c4vxk6sw157a-bash-4.4-p23/bin/sh"
/nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/libexec/libvirt-guests.sh: interpreter directive changed from "/bin/sh" to "/nix/store/avxlkpsr36xpnsy0byq2c4vxk6sw157a-bash-4.4-p23/bin/sh"
moving /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/sbin/* to /nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0/bin
/nix/store/jqijf49nycl31xcjs42rnsirpphdhdwb-libvirt-4.6.0

GrahamcOfBorg added 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 0 10.rebuild-linux: 11-100 labels Aug 19, 2018

libvirt: fix dlopen("libjansson.so.4")

929a53f

Commit 59027e2 broke libvirt broke libvirt/nixops on linux. Use ${targetPlatform.extensions.sharedLibrary} to accomodate both linux and darwin.

teto force-pushed the libvirt_fix2 branch from 8da303e to 929a53f Compare August 19, 2018 13:33

GrahamcOfBorg added 10.rebuild-darwin: 1-10 and removed 10.rebuild-darwin: 0 labels Aug 19, 2018

LnL7 approved these changes Aug 19, 2018

View reviewed changes

LnL7 merged commit 9857994 into NixOS:master Aug 19, 2018

KiaraGrouwstra mentioned this pull request Aug 20, 2018

rebuild yields libjansson error KiaraGrouwstra/nix-config#34

Closed

LnL7 mentioned this pull request Aug 21, 2018

libvirt: add upstream patch to revert jansson #45409

Closed

9 tasks

teto deleted the libvirt_fix2 branch May 13, 2020 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libvirt: darwin/linux fix dlopen("libjansson.so.4") #45324

libvirt: darwin/linux fix dlopen("libjansson.so.4") #45324

teto commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

teto commented Aug 19, 2018 •

edited

LnL7 commented Aug 19, 2018

teto commented Aug 19, 2018

LnL7 commented Aug 19, 2018

teto commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

teto commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

flokli commented Aug 19, 2018

LnL7 commented Aug 19, 2018

LnL7 commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

libvirt: darwin/linux fix dlopen("libjansson.so.4") #45324

libvirt: darwin/linux fix dlopen("libjansson.so.4") #45324

Conversation

teto commented Aug 19, 2018

Motivation for this change

Things done

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

teto commented Aug 19, 2018 • edited

LnL7 commented Aug 19, 2018

teto commented Aug 19, 2018

LnL7 commented Aug 19, 2018

teto commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

teto commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

flokli commented Aug 19, 2018

LnL7 commented Aug 19, 2018

LnL7 commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

GrahamcOfBorg commented Aug 19, 2018

teto commented Aug 19, 2018 •

edited