lib: introduce binPath #67392

zimbatm · 2019-08-24T14:40:16Z

A lot of derivations have a main executable. For example pkgs.curl has
bin/curl, pkgs.bash has bin/bash. Writing those can be repetitive:

"${lib.getBin pkgs.bash}/bin/bash"

Introduce a new lib.binPath function that can be used to infer a
default binary path for all packages.

"${lib.binPath pkgs.bash}"

By default the binary is at $out/bin/$pname. For the package where that
doesn't match it's possible to provide a meta.binPath attribute that
contains the relative path to the binary.

Motivation for this change

Things done

Notify maintainers

cc @

zimbatm · 2019-08-24T14:44:27Z

hmm, maybe only use builtins.parseDrvName to infer the binary name.

nix-repl> pkgs.lib.pkgBin pkgs.terraform.full 
"/nix/store/44gjizfr1v5lf9yhq9jvhvynsk9j8g5i-terraform-0.11.14/bin/terraform-with-plugins"

zimbatm · 2019-08-24T14:46:51Z

One idea is that if nixpkgs is consistently mapped, it will make it easy to build single-program containers with docker / firejail / ...

zimbatm · 2019-08-24T14:52:15Z

It would be nice to also be able to annotate which packages don't have a default executable so they can fail during evaluation.

zimbatm · 2019-08-24T15:02:16Z

renamed lib.pkgBin to lib.binPath so that there is only one term to learn.

gloaming · 2019-08-25T19:59:18Z

binPath sounds like it would be bin/... That could be confusing for someone seeing it for the first time. MaybemainBin so it does what it says?

gloaming · 2019-08-25T20:01:36Z

hmm, maybe only use builtins.parseDrvName to infer the binary name.
nix-repl> pkgs.lib.pkgBin pkgs.terraform.full 
"/nix/store/44gjizfr1v5lf9yhq9jvhvynsk9j8g5i-terraform-0.11.14/bin/terraform-with-plugins"

When wouldn't they return the same thing?

nix-repl> builtins.parseDrvName terraform.full.name
{ name = "terraform-with-plugins"; version = "0.11.14"; }

gloaming · 2019-08-25T20:07:02Z

I was actually thinking about this recently in relation to NixOS modules :)

One concern I had is that a user might override a package with a new pname, and calls to the package would then fail. I don't see a way to solve this without adding the attribute to meta from mkDerivation, which would be a bit annoying.

However, I'm not sure to what extent anyone actually cares about being able to override the pname.

danbst · 2019-08-26T13:37:50Z

pkgs/build-support/trivial-builders.nix

@@ -63,6 +63,9 @@ rec {
        # Pointless to do this on a remote machine.
        preferLocalBuild = true;
        allowSubstitutes = false;
+        meta = {
+          binPath = destination;


maybe only for executables?

While working on this PR I discovered that there is already an executables (plural) attribute. I have no idea what it's for.

$ rg "executables = \[" pkgs/os-specific/linux/autofs/default.nix 45: executables = [ "automount" ]; pkgs/tools/text/patchutils/generic.nix 23: executables = [ "combinediff" "dehtmldiff" "editdiff" "espdiff" pkgs/tools/backup/dirvish/default.nix 13: executables = [ "dirvish" "dirvish-runall" "dirvish-expire" "dirvish-locate" ];

summoning @winpat who added dirvish

quick review suggests those are used only for build process, so probably not a subject for this PR

@zimbatm What @danbst said.

danbst

Good! Can you also add some real usages inside NixOS or Nixpkgs? Though I understand it's main purpose is to be used externally (by update bots most likely)

danbst · 2019-08-26T14:04:52Z

loosely related NixOS/nix#1049

danbst · 2019-08-26T14:08:57Z

also, loosely related https://github.com/NixOS/nixpkgs/pull/861/files. The idea is to force create /bin/foo symlinks if package attribute name is called foo, to minimize surprises.

With this PR, it could be inversed (as a NixOS module) - autocreate "/bin" symlinks based on pname and meta.binPath.

zimbatm · 2019-08-26T14:33:45Z

Good! Can you also add some real usages inside NixOS or Nixpkgs? Though I understand it's main purpose is to be used externally (by update bots most likely)

I will convert the code inside of nixpkgs once the naming has stabilized. Most calls to "${lib.getBin thepackage}/bin/thepackage" can be simplified to lib.mainBIn thepackage.

Once the API is stable I also think that tools like nix-bundle and dockerTools.buildImage can be adapted to work 99% of the time. nix-bundle nixpkgs.hello => get a distributable image

domenkozar · 2019-08-26T14:41:03Z

I think the name could be more descriptive like lib.getDefaultExecutable or maybe just lib.getExecutable
This function is going to fail a lot of times due to the missing file, it's seems like a slippery slope. It would be better to build another derivation that asserts the path really exists and is executable. A bit more development work with less overhead is to extend stdenv to assert drv.defaultExecutable exists at build time, so then only those packages that define that attribute can use the shortcut.

gloaming · 2019-08-26T20:59:25Z

This could potentially benefit NixOS modules, see NixOS/rfcs#42 (comment)

gloaming · 2019-08-26T21:04:49Z

I think the name could be more descriptive like lib.getDefaultExecutable or maybe just lib.getExecutable

Ehh, it's a bit long...

This function is going to fail a lot of times due to the missing file, it's seems like a slippery slope.

Well, the function can't fail, but I get what you mean. I'm not sure that it's much worse than doing it manually, but I guess it is akin to a partial function in some sense.

extend stdenv to assert drv.defaultExecutable exists at build time, so then only those packages that define that attribute can use the shortcut.

👍 I like this. Then we can, for example, require ExecStart to be defined iff there is no default binary for the options.package.

FRidh

I think packages should instead declare that variable in their passthru. E.g., for Python we have
python3.interpreter which is the path to the Python executable (python3.executable).

Don't we also somewhere have an attribute set which a bunch of these executables?

matthewbauer · 2019-11-03T17:24:30Z

Once the API is stable I also think that tools like nix-bundle and dockerTools.buildImage can be adapted to work 99% of the time. nix-bundle nixpkgs.hello => get a distributable image

Yeah! That would be really nice. I had an old issue about this a while back:

NixOS/nix#2154

@edolstra wants to use the "defaultApp" in Flakes to accomplish this (NixOS/nix#2909), but we may be able to have some way to put this on plain derivations.

zimbatm · 2019-11-03T22:39:39Z

I'm not sure how to move this forward. @domenkozar has a good point that bad paths wouldn't be detected automatically. On the other hand adding a derivation per derivation is also a lot of overhead.

A lot of derivations have a *main* executable. For example pkgs.curl has bin/curl, pkgs.bash has bin/bash. Writing those can be repetitive: "${lib.getBin pkgs.bash}/bin/bash" Introduce a new `lib.binPath` function that can be used to infer a default binary path for all packages. "${lib.binPath pkgs.bash}" By default the binary is at $out/bin/$pname. For the package where that doesn't match it's possible to provide a `meta.binPath` attribute that contains the relative path to the binary.

Support the lib.binPath interface nix-repl> pkgs = import ./. {} nix-repl> pkg = pkgs.writeScript "foo" "hohoho" nix-repl> pkgs.lib.pkgBin pkg "/nix/store/v9xjci3rz9ck1amyfndh6wjz2f4d5v1l-foo" nix-repl> pkg2 = pkgs.writeScriptBin "foo" "hahaha" nix-repl> pkgs.lib.pkgBin pkg2 "/nix/store/0zp2hj619874rh2g1v8n62958f6scp6f-foo/bin/foo"

zimbatm · 2020-03-28T10:08:32Z

flakes are introducing a similar idea so maybe it's worth resurrecting this.

{ drv, name ? drv.pname or drv.name, exePath ? "/bin/${name}" }:
{
   type = "app";
   program = "${drv}${exePath}";
}

zimbatm · 2020-08-16T19:09:10Z

replaced with #95615 which mimics the heuristic of nix run

zimbatm requested review from edolstra and nbp as code owners August 24, 2019 14:40

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Aug 24, 2019

zimbatm force-pushed the lib-pkgBin branch from 0981208 to 6000ed1 Compare August 24, 2019 15:01

zimbatm requested review from Ericson2314 and matthewbauer as code owners August 25, 2019 22:22

ofborg bot added the 6.topic: stdenv Standard environment label Aug 25, 2019

gloaming mentioned this pull request Aug 26, 2019

[RFC 0042] NixOS settings options NixOS/rfcs#42

Merged

danbst reviewed Aug 26, 2019

View reviewed changes

danbst approved these changes Aug 26, 2019

View reviewed changes

tazjin mentioned this pull request Sep 6, 2019

Use upcoming 'pkgBin' feature from nixpkgs to set entrypoints tazjin/nixery#51

Open

FRidh requested changes Nov 3, 2019

View reviewed changes

zimbatm added 2 commits January 12, 2020 11:44

zimbatm added 5 commits January 12, 2020 11:45

vscode: add meta.binPath

91f02df

ldns: add meta.binPath

455a6e3

ripgrep: add meta.binPath

95c9d18

timewarrior: add meta.binPath

9334fda

webfs: add meta.binPath

35da0fa

zimbatm force-pushed the lib-pkgBin branch from 73e3dce to 35da0fa Compare January 12, 2020 10:45

zimbatm requested a review from infinisil as a code owner January 12, 2020 10:45

zimbatm changed the title ~~lib: introduce pkgBin~~ lib: introduce binPath Jan 12, 2020

zimbatm mentioned this pull request Aug 16, 2020

lib: add toApp #95615

Closed

10 tasks

zimbatm closed this Aug 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib: introduce binPath #67392

lib: introduce binPath #67392

zimbatm commented Aug 24, 2019 •

edited

zimbatm commented Aug 24, 2019 •

edited

zimbatm commented Aug 24, 2019

zimbatm commented Aug 24, 2019

zimbatm commented Aug 24, 2019

gloaming commented Aug 25, 2019

gloaming commented Aug 25, 2019

gloaming commented Aug 25, 2019 •

edited

danbst Aug 26, 2019

zimbatm Aug 26, 2019

zimbatm Aug 26, 2019

danbst Aug 26, 2019

winpat Aug 26, 2019

danbst left a comment

danbst commented Aug 26, 2019

danbst commented Aug 26, 2019

zimbatm commented Aug 26, 2019

domenkozar commented Aug 26, 2019 •

edited

gloaming commented Aug 26, 2019

gloaming commented Aug 26, 2019

FRidh left a comment

matthewbauer commented Nov 3, 2019

zimbatm commented Nov 3, 2019

zimbatm commented Mar 28, 2020 •

edited

zimbatm commented Aug 16, 2020

lib: introduce binPath #67392

lib: introduce binPath #67392

Conversation

zimbatm commented Aug 24, 2019 • edited

Motivation for this change

Things done

Notify maintainers

zimbatm commented Aug 24, 2019 • edited

zimbatm commented Aug 24, 2019

zimbatm commented Aug 24, 2019

zimbatm commented Aug 24, 2019

gloaming commented Aug 25, 2019

gloaming commented Aug 25, 2019

gloaming commented Aug 25, 2019 • edited

danbst Aug 26, 2019

Choose a reason for hiding this comment

zimbatm Aug 26, 2019

Choose a reason for hiding this comment

zimbatm Aug 26, 2019

Choose a reason for hiding this comment

danbst Aug 26, 2019

Choose a reason for hiding this comment

winpat Aug 26, 2019

Choose a reason for hiding this comment

danbst left a comment

Choose a reason for hiding this comment

danbst commented Aug 26, 2019

danbst commented Aug 26, 2019

zimbatm commented Aug 26, 2019

domenkozar commented Aug 26, 2019 • edited

gloaming commented Aug 26, 2019

gloaming commented Aug 26, 2019

FRidh left a comment

Choose a reason for hiding this comment

matthewbauer commented Nov 3, 2019

zimbatm commented Nov 3, 2019

zimbatm commented Mar 28, 2020 • edited

zimbatm commented Aug 16, 2020

zimbatm commented Aug 24, 2019 •

edited

zimbatm commented Aug 24, 2019 •

edited

gloaming commented Aug 25, 2019 •

edited

domenkozar commented Aug 26, 2019 •

edited

zimbatm commented Mar 28, 2020 •

edited