Introduce "wrapCommand" builder #43074
Conversation
This makes it possible to use some of the setup hooks in trivial builders.
This is a useful way to wrap a command without having to use a full mkDerivation. There are a couple of options that can be passed to control it.
Usually we want to compile .java files directly but ocassionaly this is not possible. The most common reason is the code is closed source. Any way, the .jar format is about as cross-platform as you can get so it makes sense to support it. Adding the wrapCommand simplifies these derivations greatly.
Ruby related packages that can now use wrapCommand.
|
Success on x86_64-darwin (full log) Attempted: apktool, emv, grepm, nix-generate-from-cpan, nixpkgs-lint Partial log (click to expand)
|
|
Failure on x86_64-linux (full log) Attempted: apktool, emv, grepm, nix-generate-from-cpan, nixpkgs-lint Partial log (click to expand)
|
|
Success on aarch64-linux (full log) Attempted: emv, grepm, nix-generate-from-cpan, nixpkgs-lint The following builds were skipped because they don't evaluate on aarch64-linux: apktool Partial log (click to expand)
|
There was a problem hiding this comment.
Very nice!
I just wonder about the patchShebangs thing, can you comment on that?
I also looked at whether this change may break other nixpkgs configs that are not in the tree, I don't think so, since trivial-builders is augmented and all-pkgs and the build env are the result of merging, so that seems to be ok.
I didn't check all the individual wrapCommands (yet) – so a tentative 👍 from me.
A great idea in any case!
| allowSubstitutes = false; | ||
| }) '' | ||
| local -a args="($makeWrapperArgs)" | ||
| makeWrapper ${executable} $out/bin/${name} ''${args[@]} |
There was a problem hiding this comment.
You should put quotes around these arguments, or run some quote-for-shell function. Academic, maybe.
There was a problem hiding this comment.
Come to think of it, shouldn't ${args[@]} also be in double quotes (I'm assuming that will work like "$@" then)?
Otherwise you can't pass arguments with spaces to makeWrapper. But that would mean you also have to go split all the arguments you have in all the makeWrapperArgs into the individual words.
There was a problem hiding this comment.
Yeah this is definitely good to do before merging - this is more of a proof of concept for now.
| '' | ||
| mkdir -p $out/bin | ||
| cp ${./nix-generate-from-cpan.pl} $out/bin/nix-generate-from-cpan | ||
| patchShebangs $out/bin/nix-generate-from-cpan |
There was a problem hiding this comment.
So now patchShebangs is no longer being done? Is that intentional? I'm thinking of the case where people want to run the wrapped program directly for any reason – maybe this should be part of makeCommand?
There was a problem hiding this comment.
Yeah it may be wanted it some cases. In this case perl is used as the interpreter so the shebang is kind of unused. What's nice about this is it means we can reuse the original path of the file - so fixed output can keep their paths.
For this one anyway it has the correct shebang (#!/usr/bin/env perl will work everywhere!) So it seemed reasonable to avoid the patchShebangs.
The problem with patchShebangs is that it doesn't differentiate between "nativeBuildInputs" & "buildInputs" - so you can wind up with references from nativeBuildInputs in your output paths if you're not careful. It needs to be reworked to be compatible with cross compilation & strictDeps.
There was a problem hiding this comment.
#!/usr/bin/env perlwill work everywhere!
There is no /usr/bin/env in Nix sandbox. (I have not read the context of this PR so I don't know if this is important or not.)
|
Instead of being able to wrap a single command, I think it would be nice to have instead a mapping and with the possibility of using glob to apply wrapper arguments to a selection of executables. |
adds two new builders:
- buildTree
- wrapExecutable
These each can be used to run wrapCommand.
buildTree can be used to generate an output path from an attribute
list of directories. For instance:
buildTree "test" {
"/bin" = "${lib.getBin libxml2}/bin";
"/lib" = "${lib.getLib libxml2}/lib";
"/share/man" = "${lib.getOutput "man" libxml2}/share/man";
"/share/doc" = "${lib.getOutput "doc" libxml2}/share/doc";
} {}
Puts the multiple output derivation libxml2 back into one output.
wrapExecutable will use makeWrapper to generate a single output
wrapped version of the old executable. For instance:
wrapExecutable "bash" {
executable = "";
makeWrapperArgs = [
"--prefix" "PATH" ":"
(lib.makeSearchPathOutput "bin" "bin" [ a b c d e f ])
[])
}
Will generate wrapped bash with a, b, c, d, e, & f in the PATH.
|
|
||
| # Build a tree from an attribute mapping of paths to files. | ||
| buildTree = name: tree: args: runCommand name (args // { |
There was a problem hiding this comment.
This is linkFarm but then with a more convenient tree argument?
| # Pointless to do this on a remote machine. | ||
| preferLocalBuild = true; | ||
| allowSubstitutes = false; | ||
| }) '' | ||
| local -a args="($makeWrapperArgs)" | ||
| makeWrapper ${executable} $out/bin/${name} ''${args[@]} | ||
| ''; | ||
| makeWrapper ${executable} $out ''${args[@]} |
Motivation for this change
"wrapCommand" is an abstraction of somethings fairly common in Nixpkgs where single binary is produced from a single source file. This happens frequently when using something like .jar or .phar files as well as in some of the Ruby things.
This should lead to less code & maintenance.
wrapCommand needs to use makeWrapper to work, so I had to move it from all-packages.nix into a new setup-hooks/default.nix file that will be used pulled in from stage.nix.