cargo-vendor: 0.1.13 -> 0.1.23 #57017

akru · 2019-03-07T11:24:08Z

Motivation for this change

Upgrade cargo-vendor to 0.1.23 is required for support build edition 2018 crates.

Things done

akru · 2019-03-07T11:49:08Z

andir · 2019-03-07T14:40:29Z

pkgs/build-support/rust/default-crate-overrides.nix

@@ -14,7 +14,7 @@ in
    buildInputs = [ openssl zlib curl ]
      ++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation libiconv ];
    # TODO: buildRustCrate seems to use incorrect default inference
-    crateBin = [ {  name = "cargo"; path = "src/bin/cargo.rs"; } ];
+    crateBin = [ {  name = "cargo"; path = "src/bin/cargo/main.rs"; } ];


You can remove the path attribute. Since fc5e595 we are doing the right thing(tm) within buildRustCrate.

You are right, fixed

akru · 2019-03-07T15:00:15Z

Unfortunately with big crate deps list (like a parity) cargo-vendor has segfault in libcurl.so. WIP

andir · 2019-03-07T15:07:28Z

I am only seeing

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/glsy9j0x5j23sq1rqnazs1jvd9hd7mxl-source
source root is source
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "installPhase" }
installing
    Updating git repository `https://github.com/cheme/heapsize.git`
    Updating crates.io index
    Updating git repository `https://github.com/paritytech/rust-secp256k1`
    Updating git repository `https://github.com/paritytech/app-dirs-rs`
    Updating git repository `https://github.com/paritytech/bn`
    Updating git repository `https://github.com/paritytech/jsonrpc.git`
    Updating git repository `https://github.com/paritytech/hidapi-rs`
    Updating git repository `https://github.com/paritytech/libusb-rs`
    Updating git repository `https://github.com/paritytech/trezor-sys`
    Updating git repository `https://github.com/paritytech/rust-ctrlc.git`
    Updating git repository `https://github.com/paritytech/libusb-sys`
    Updating git repository `https://github.com/tomusdrw/ws-rs`
 Downloading crates ...
  Downloaded digest v0.7.6
  Downloaded crossbeam-deque v0.5.2
Traceback (most recent call last):
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 42, in <module>
    main()
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 17, in main
    assert list(data.keys()) == ["source"]
AssertionError

When running nix-build -I nixpkgs=. -A parity.cargoDeps --check on this PR.
Is that what you are seeing?

EDIT: Now I see the segfault in dmesg: cargo-vendor[31122]: segfault at 576d ip 00007ffff7f93ac5 sp 00007ffffffee100 error 4 in libcurl.so.4.5.0[7ffff7f4e000+61000]

akru · 2019-03-07T15:09:22Z

Traceback (most recent call last):
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 42, in <module>
    main()
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 17, in main
    assert list(data.keys()) == ["source"]
AssertionError

@andir This assertion is you catch because cargo vendor program fails with segmentation fault. You can see it in dmesg log.

Looks like:

[ 1277.594384] cargo-vendor[18115]: segfault at 318 ip 00007ffff7f93ac5 sp 00007ffffffedfe0 error 4 in libcurl.so.4.5.0[7ffff7f4e000+61000]

dtzWill · 2019-03-08T19:54:11Z

FWIW this seems to be working for me? If I'm the only one NOT seeing segfaults I can investigate further... I am building on top of staging's rust bump to 1.33-- no clue if that's relevant :).

andir · 2019-03-08T20:06:05Z

@dtzWill how did you test it? I had to run --check on the cargoDeps attribute to actually re-run the whole cargo-vendor thing.

dtzWill · 2019-03-08T20:53:46Z

On Fri, 08 Mar 2019 12:06:09 -0800, Andreas Rammhold ***@***.***> wrote: @dtzWill how did you test it? I had to run `--check` on the `cargoDeps` attribute to actually re-run the whole cargo-vendor thing.

Yep, with `--check` using the command for `parity` you posted. I also used it in packaging up a newer (git) version of `fractal` \o/ which was what led me to discovering this PR :). Let's see what behavior this has for other folks, maybe?

…

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #57017 (comment) part: text/html

andir · 2019-03-08T21:08:47Z

@GrahamcOfBorg build parity.cargoDeps

Does ofBorg also have --check?

andir · 2019-03-14T11:06:46Z

@dtzWill I tried to again on a newly installed NixOS packet.net machine. Cloned nixpkgs and ran nix-build -I nixpkgs=. '<nixpkgs>' -A parity.cargoDeps --check. Segfaults like on all my other machines.

What differences are there between our systems?

L-as · 2019-03-14T17:06:05Z

FWIW I'm not seeing any failures with the command andir posted.

flokli · 2019-03-14T23:54:49Z

I was able to reproduce this after a second try:

⇒  nix-build -I nixpkgs=. -A parity.cargoDeps --check
checking outputs of '/nix/store/fvqwb3q1yf8kv2h4ld8v0ij92v51g7lw-parity-2.2.9-vendor.drv'...
unpacking sources
unpacking source archive /nix/store/glsy9j0x5j23sq1rqnazs1jvd9hd7mxl-source
source root is source
patching sources
installing
    Updating git repository `https://github.com/cheme/heapsize.git`
    Updating crates.io index
    Updating git repository `https://github.com/paritytech/rust-secp256k1`
    Updating git repository `https://github.com/paritytech/app-dirs-rs`
    Updating git repository `https://github.com/paritytech/bn`
    Updating git repository `https://github.com/paritytech/jsonrpc.git`
    Updating git repository `https://github.com/paritytech/hidapi-rs`
    Updating git repository `https://github.com/paritytech/libusb-rs`
    Updating git repository `https://github.com/paritytech/trezor-sys`
    Updating git repository `https://github.com/paritytech/rust-ctrlc.git`
    Updating git repository `https://github.com/paritytech/libusb-sys`
    Updating git repository `https://github.com/tomusdrw/ws-rs`
 Downloading crates ...
  Downloaded url v1.7.1
  Downloaded indexmap v1.0.2
Traceback (most recent call last):
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 42, in <module>
    main()
  File "/nix/store/7qnming9jgiqb6hk2pqfmgp4dn60vs5r-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 17, in main
    assert list(data.keys()) == ["source"]
AssertionError
builder for '/nix/store/fvqwb3q1yf8kv2h4ld8v0ij92v51g7lw-parity-2.2.9-vendor.drv' failed with exit code 1
error: build of '/nix/store/fvqwb3q1yf8kv2h4ld8v0ij92v51g7lw-parity-2.2.9-vendor.drv' failed

dmesg:

[333635.128291] cargo-vendor[4457]: segfault at 0 ip 00007ffff77c430e sp 00007ffffffed820 error 4 in libc-2.27.so[7ffff7778000+13d000]
[333635.128296] Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 57 41 56 41 55 41 54 55 53 48 89 f3 48 0f af da 48 83 ec 08 48 85 db 0f 84 22 01 00 00 <8b> 01 49 89 fc 49 89 f2 49 89 c9 25 00 80 00 00 75 59 4c 8b 81 88

dtzWill · 2019-03-15T04:09:29Z

To be clear, I've been using this on top of my normal branch and not the exact tree here, and there are many reasons I'd be seeing different behavior-- but MOSTLY the bits that turn out well I send upstream so I don't think I'm sitting on anything relevant.

The most obvious difference is I'm using the rust 1.33 updates from our staging branch-- and I don't know what changed in 1.33 but "fixing things that cause crashes" sounds plausible enough to check it out ;).

When my builders have a chance I'll try the tree here and maybe on a few machines... heisenbug is lame!

akru · 2019-04-18T10:37:15Z

As I see it fixed in latest cargo-vendor.

akru · 2019-04-18T10:51:20Z

@GrahamcOfBorg build cargo-vendor

sondr3 · 2019-04-18T15:38:24Z

This would be really nice to have, just ran into issues with the old version of cargo-vendor with lto = "thin", fingers crossed this works out 😄

lilyball · 2019-04-20T06:06:46Z

What's the current status of this? It looks like I can't update ffsend right now due to the current cargo-vendor not understanding a dependency of the form clip = { version = "0.5", optional = true, package = "clipboard" } (e.g. where the actual package name doesn't match the dependency name), and this update fixes that.

Incidentally, I can't build this on macOS. I tried merging it into current master and building, and rust_cargo-0.35.0 fails to build, saying it can't find the Security framework (which according to the .drv is not listed as a buildInput).

lilyball · 2019-04-20T06:12:10Z

pkgs/build-support/rust/default-crate-overrides.nix

@@ -13,15 +13,15 @@ in
  cargo = attrs: {
    buildInputs = [ openssl zlib curl ]
      ++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation libiconv ];


It looks like my macOS build error can be fixed with

Suggested change

++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation libiconv ];

++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation Security libiconv ];

Of course, then I get the same error for cargo-vendor itself, so the same fix needs to be applied there too.

lilyball · 2019-04-20T06:13:07Z

pkgs/build-support/rust/default-crate-overrides.nix

@@ -13,15 +13,15 @@ in
  cargo = attrs: {
    buildInputs = [ openssl zlib curl ]
      ++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation libiconv ];
-    # TODO: buildRustCrate seems to use incorrect default inference
-    crateBin = [ {  name = "cargo"; path = "src/bin/cargo.rs"; } ];
  };

  cargo-vendor = attrs: {
    buildInputs = [ openssl zlib curl ];


Suggested change

buildInputs = [ openssl zlib curl ];

buildInputs = [ openssl zlib curl ]

++ stdenv.lib.optionals stdenv.isDarwin [ Security ];

andir · 2019-04-20T09:19:59Z

I've been trying to reproduce the segfault we did observe with this rebased onto currentmaster. After 6 rebuilds there hasn't been a single segfault. Currently trying to reproduce it on another machine but that also succeeded for a few times. I hope this isn't due to the host being on nixos-19.03 now...

@lilyball can you apply the mentioned fix to cargo to figure out if this package (and the users) work on master for Darwin? I do not own any of that hardware :/

lilyball · 2019-04-20T19:52:24Z

@andir I already applied the two fixes I suggested and successfully built cargo-vendor-0.1.23 (and ffsend-0.2.45) on Darwin.

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

andir · 2019-04-21T10:13:38Z

@lilyball does this look alright now?

lilyball · 2019-04-21T20:48:51Z

@andir Yes, it builds just fine on Darwin now.

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

GrahamcOfBorg added the 6.topic: rust label Mar 7, 2019

akru force-pushed the cargo-vendor-update branch from 9e62121 to ce373b8 Compare March 7, 2019 11:48

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Mar 7, 2019

akru changed the title ~~cargo-vendor: 0.1.13 -> 0.1.23~~ WIP cargo-vendor: 0.1.13 -> 0.1.23 Mar 7, 2019

andir reviewed Mar 7, 2019

View reviewed changes

akru force-pushed the cargo-vendor-update branch from ce373b8 to 0a4fcdb Compare March 7, 2019 15:05

akru mentioned this pull request Mar 7, 2019

NixOS build: libcurl segfault alexcrichton/cargo-vendor#222

Closed

dywedir mentioned this pull request Mar 10, 2019

cargo-vendor: 0.1.13 -> 0.1.23 #57299

Closed

10 tasks

infinisil added the 8.has: package (update) label Mar 30, 2019

timokau added the 2.status: work-in-progress label Apr 7, 2019

timokau changed the title ~~WIP cargo-vendor: 0.1.13 -> 0.1.23~~ [WIP] cargo-vendor: 0.1.13 -> 0.1.23 Apr 7, 2019

akru changed the title ~~[WIP] cargo-vendor: 0.1.13 -> 0.1.23~~ cargo-vendor: 0.1.13 -> 0.1.23 Apr 18, 2019

akru force-pushed the cargo-vendor-update branch from 0a4fcdb to 26435b7 Compare April 18, 2019 10:35

akru mentioned this pull request Apr 18, 2019

parity: 2.3 -> 2.4 #59826

Merged

10 tasks

GrahamcOfBorg added 10.rebuild-darwin: 1-10 and removed 10.rebuild-darwin: 0 labels Apr 18, 2019

sondr3 mentioned this pull request Apr 19, 2019

cargo-outdated: init at unstable-2019-04-13 #59893

Merged

10 tasks

lilyball reviewed Apr 20, 2019

View reviewed changes

lilyball mentioned this pull request Apr 20, 2019

ffsend: 0.2.42 -> 0.2.45 #59918

Merged

10 tasks

akru and others added 3 commits April 21, 2019 12:11

cargo-vendor: 0.1.13 -> 0.1.23

a4902a3

cargo: fix build on Darwin

2e2f7cb

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

cargo-vendor: fix build on Darwin

1bb989c

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

andir force-pushed the cargo-vendor-update branch from 26435b7 to 1bb989c Compare April 21, 2019 10:13

andir merged commit ee3b96c into NixOS:master Apr 21, 2019

c0bw3b pushed a commit to c0bw3b/nixpkgs that referenced this pull request Apr 22, 2019

cargo: fix build on Darwin

3f84342

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

c0bw3b pushed a commit to c0bw3b/nixpkgs that referenced this pull request Apr 22, 2019

cargo-vendor: fix build on Darwin

1a260d5

This is supposedly fixing the build of the cargo crate on Drawin [1]. [1] NixOS#57017 (review)

evanjs mentioned this pull request May 1, 2019

Error parsing manifest with lto = "thin" on NixOS alexcrichton/cargo-vendor#267

Closed

matthewbauer mentioned this pull request May 1, 2019

buildRustPackage: cargo-vendor upgrade broke cargoSha256 hashes #60668

Closed

anderslundstedt mentioned this pull request Aug 7, 2019

spotifyd: update cargoSha256 hash #66233

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cargo-vendor: 0.1.13 -> 0.1.23 #57017

cargo-vendor: 0.1.13 -> 0.1.23 #57017

akru commented Mar 7, 2019 •

edited

akru commented Mar 7, 2019

andir Mar 7, 2019

akru Mar 7, 2019

akru commented Mar 7, 2019

andir commented Mar 7, 2019 •

edited

akru commented Mar 7, 2019 •

edited

dtzWill commented Mar 8, 2019

andir commented Mar 8, 2019

dtzWill commented Mar 8, 2019 via email

andir commented Mar 8, 2019

andir commented Mar 14, 2019

L-as commented Mar 14, 2019

flokli commented Mar 14, 2019

dtzWill commented Mar 15, 2019

akru commented Apr 18, 2019

akru commented Apr 18, 2019

sondr3 commented Apr 18, 2019

lilyball commented Apr 20, 2019 •

edited

lilyball Apr 20, 2019

lilyball Apr 20, 2019

andir commented Apr 20, 2019

lilyball commented Apr 20, 2019 •

edited

andir commented Apr 21, 2019

lilyball commented Apr 21, 2019

	++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation libiconv ];
	++ stdenv.lib.optionals stdenv.isDarwin [ CoreFoundation Security libiconv ];

	buildInputs = [ openssl zlib curl ];
	buildInputs = [ openssl zlib curl ]
	++ stdenv.lib.optionals stdenv.isDarwin [ Security ];

cargo-vendor: 0.1.13 -> 0.1.23 #57017

cargo-vendor: 0.1.13 -> 0.1.23 #57017

Conversation

akru commented Mar 7, 2019 • edited

Motivation for this change

Things done

akru commented Mar 7, 2019

andir Mar 7, 2019

Choose a reason for hiding this comment

akru Mar 7, 2019

Choose a reason for hiding this comment

akru commented Mar 7, 2019

andir commented Mar 7, 2019 • edited

akru commented Mar 7, 2019 • edited

dtzWill commented Mar 8, 2019

andir commented Mar 8, 2019

dtzWill commented Mar 8, 2019 via email

andir commented Mar 8, 2019

andir commented Mar 14, 2019

L-as commented Mar 14, 2019

flokli commented Mar 14, 2019

dtzWill commented Mar 15, 2019

akru commented Apr 18, 2019

akru commented Apr 18, 2019

sondr3 commented Apr 18, 2019

lilyball commented Apr 20, 2019 • edited

lilyball Apr 20, 2019

Choose a reason for hiding this comment

lilyball Apr 20, 2019

Choose a reason for hiding this comment

andir commented Apr 20, 2019

lilyball commented Apr 20, 2019 • edited

andir commented Apr 21, 2019

lilyball commented Apr 21, 2019

akru commented Mar 7, 2019 •

edited

andir commented Mar 7, 2019 •

edited

akru commented Mar 7, 2019 •

edited

lilyball commented Apr 20, 2019 •

edited

lilyball commented Apr 20, 2019 •

edited