dont yet merge i'm still building locally so that everything will works

paul, aszlig and me are about to removing cargo from the nixos toolchain and create a cargo2nix tool next week. would be nice if this patch is in then as it looks promising, .... especially if it builds.

@qknight similar to #24991 ?

@qknight what's the design of cargo2nix going to be? will it fully replace cargo and call rustc manually or will it just generate derivations that then use cargo?

@Mic92 @bennofs @sjmackenzie @seitz
the plan is this: cargo creates a cargo.lock file. we write a simple rust application, called cargo2nix, which generates a cargo.lock.nix file from it.

cargo2nix internal

1. cargo2nix additionally has a small database, a nix attribute set, where we map additional system dependencies as sqlite for packages like rusqlite which requires libsqlite3-sys ^0.8 which assumes sqlite-dev to be present.

2. later on: cargo.lock.nix file is commited into nixpkgs alongside the respective packaged project using it. in comparison, this is what we do for yarn2nix for our nixcloud-frontend in the javascript world.

note: cargo is then only used for generating the cargo.lock.nix file and during project build never called again.

references

what has been created in https://github.com/NixOS/nixpkgs/pull/24991/files looks similar to https://github.com/nixcloud/nixcrates/blob/master/default.nix .

we are going build something like buildRustPackage which builds all single dependencies, listed in cargo.lock.nix first, then links them all with the project object into the final ELF binary. historically this would be done by cargo and is a constant source of pain in nixos. in nixcrates, for reference, we already engineered the part where the build happens without any cargo involvement. the nixcrates drawback is that semver is ignored and we are duplicating the code of dependency calculation which would be done by cargo.

i strongly encourage to use the dependency calculator of cargo and not to duplicated the code as we did in nixcrates because it avoids to have two sources of errors when building targets using cargo, on say ubuntu, and using nixcrates with nix.

todo

yet we have to check if cargo.lock already holds all the information we need to build a cargo2nix tool with. a first glimples looked promising. however, if that does not hold true, we would extend cargo with a plugin which would ouput what we required for making the cargo2nix tool.

@qknight what is the motiviation for not using the cargo tool for building?

motivation for dropping cargo

@bennofs one original motivation for nixcrates was to be able to describe dependencies in nix like this:

buildInputs = with allCrates; [ tar filetime libc xattr ]; 

one is also able to pick a specific version of a crate using nixcrates.

post mortem

turns out that cargo2nix wouldn't help in this regard and it is good that you asked this question. i have to think about this much more. one problem with cargo2nix, as with all these tools, like yarn2nix, is that we can't use nix to evaluate them but have to do that manually.

maybe we can do something with https://github.com/shlevy/nix-exec to be able to call cargo2nix from a shell and then extend the dependencies. will report back when i gave this more thought.

@qknight I really like the approach, how this is handled for ruby modules in nixpkgs.

@qknight actually the original motivation was a mix of being able to describe dependencies in such a brief manner but also to facilitate incremental recompilation which is achieved by "builds all single dependencies" then "then links them all with the project object into the final ELF binary". Primarily the incremental compilation.

@sjmackenzie thanks for bringing me back on track!

I'm not sure it's possible to achieve the succinct dependency declaration, it requires a Cargo.toml + Cargo.lock + Cargo.lock.nix.

In an attempt to rescue the succinct dependency declaration the default.nix could call the Cargo.lock.nix file, this way users are only exposed to this:

{ agent, edges, mods, pkgs }:

agent {
  src = ./.;
  edges = with edges; [ PrimBool ];
  mods = with mods.rs; [ ./Cargo.lock.nix ];
  osdeps = with pkgs; [];
}

Though the urgency of a functional dependency calculator is more important than aesthetics atm/imho.

i'm closing this since i will focus on things in current master since it will becomes stable soon