buildRustPackage: introduce --no-merge-sources #48274
Conversation
Give the option to use cargo-vendor with the flag --no-merge-sources within the buildRustPackage infrastructure.
ljli
changed the title
|
I tried using this to build a project with duplicate dependencies (relibc), but after trying to build the vendored dependencies, the actual output directory doesn't exist. |
|
I had tested it with |
|
The commit id you reference in my repo is not of this PR but of a previous WIP. |
|
🤦♂️ Now I'm just running into problems with not being able to find |
|
I'm not sure I can follow you, do you attribute any problems to this PR? If so, please give complete reproduction steps. |
|
No. It's a separate problem. |
| @@ -42,7 +43,12 @@ stdenv.mkDerivation { | |||
| ${cargoUpdateHook} | |||
|
|
|||
| mkdir -p $out | |||
| cargo vendor $out | cargo-vendor-normalise > config | |||
| ${if flagNoMergeSources | |||
There was a problem hiding this comment.
Ideally this would not be a flag at all. Because it sounds to me like something we want to have by default.
There was a problem hiding this comment.
Afaik there is no good reason to merge the packages into a single source. Probably ~80 packages need to get their hash updated in nixpkgs and out-of-tree uses might break, if we use the flag unconditionally, but we could just run cargo-vendor without the flag and if the duplicate version error, occurs re-run it with the flag, that way everything should stay the same.
There was a problem hiding this comment.
Would re-running cause cargo-vendor to download all dependencies twice? Otherwise this might be not a bad idea.
| cargo vendor $out | cargo-vendor-normalise > config | ||
| ${if flagNoMergeSources | ||
| then '' | ||
| cargo vendor --no-merge-sources $out > config |
There was a problem hiding this comment.
Is this incompatible with our normalize script?
There was a problem hiding this comment.
Currently it is incompatible. Is there any reason to believe the formatting details will change, other than that they don't guarantee it won't? Did we actually observe this? It shouldn't be a problem to adapt the script though.
There was a problem hiding this comment.
We talked to the author of cargo-vendor and there is no guarantee that the output will stay the same after updating cargo-vendor. By having the script we can make sure it will or we can create workarounds.
|
Is this going in? Getting this error now. |
|
Closing because it has become outdated. cargo-vendor was integrated into Cargo 1.37 and removed the flag --no-merge-sources rust-lang/cargo#6869 |
The update for cargo-vendor is needed to get access to the
--no-merge-sourcesflag.The other commit just exposes the flag for the builder, with the necessary changes to make it work.
Should close #30742.
The core problem in my understanding is that in the rust packaging world a crate can be referenced
by it's source, name and version. Sources are for example, the
crates.ioregistry or a git repository containing a rust crate.cargo-vendordownloads a dependency closure for a crate to a local directory and while doing so tries to merge it into a single source. Since different sources can contain a crate with the same name and version and need not agree about what it is, this can not always work. This is the source of thefound duplicate versionerror some people encounter when trying to package a crate.--no-merge-sourcesjust keeps crates apart depending on the source. For backwards compatibility reasons (i think) this isn't the default incargo-vendorand unless we want to fix the hashes for all rust packages usingbuildRustPackageinnixpkgswe probably do not want to make it the default either.I choose the names like
cargoVendorNoMergeSourcesrather mechanical and am not opinionated about them.Can be used like this: