document nix-env bug relating to multiple output installation #76794
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ofborg
bot
added
6.topic: stdenv
|
Thank you for your contributions. This has been automatically marked as stale because it has had no activity for 180 days. If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity. Here are suggestions that might help resolve this more quickly:
|
stale
bot
added
the
2.status: stale
|
still valid |
stale
bot
removed
the
2.status: stale
lilyball
reviewed
Jul 27, 2020
|
@lilyball, I don't remember much about this PR. I skimmed through your
recommendations, and they make sense. At this point, you know more about
the topic than I do, so you'd probably be the most appropriate person to
take this forward.
…On Mon, Jul 27, 2020, 4:26 PM Lily Ballard ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> @@ -27,34 +27,59 @@
<title>Installing a split package</title>
<para>
- When installing a package via <varname>systemPackages</varname> or <command>nix-env</command> you have several options:
+ When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L289">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs.
Linking to a particular line of the master ref risks the reference going
stale. You can press y on GitHub to canonicalize the current URL, which
will change it to a blob reference so the line number will always be
accurate, at the cost of the linked file not necessarily reflecting what's
in master anymore. We should probably canonicalize this.
Note that since this PR was submitted, the referenced line number is in
fact no longer accurate. The new correspond line number is 310, though we
can go ahead and select the whole range too.
⬇️ Suggested change
- When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L289">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs.
+ When installing a package with multiple outputs, the package's <varname>meta.outputsToInstall</varname> attribute determines which outputs are actually installed. <varname>meta.outputsToInstall</varname> is a list whose <link xlink:href="https://github.com/NixOS/nixpkgs/blob/f1680774340d5443a1409c3421ced84ac1163ba9/pkgs/stdenv/generic/make-derivation.nix#L310-L320">default installs binaries and the associated man pages</link>. The following sections describe ways to install different outputs.
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> </para>
- <itemizedlist>
- <listitem>
- <para>
- You can install particular outputs explicitly, as each is available in the Nix language as an attribute of the package. The <varname>outputs</varname> attribute contains a list of output names.
This sentence seems useful, especially since this is the introduction to
split packages
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> - </listitem>
- <listitem>
- <para>
- You can let it use the default outputs. These are handled by <varname>meta.outputsToInstall</varname> attribute that contains a list of output names.
- </para>
+ <section xml:id="sec-multiple-outputs-installing-nixos">
+ <title>Selecting outputs to install via NixOS</title>
+
+ <para>
+ NixOS provides two ways to select the outputs to install for packages listed in <varname>environment.systemPackages</varname>:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages.
⬇️ Suggested change
- The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages.
+ The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install <literal>info</literal> documentation or debug symbols for all packages.
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> + <section xml:id="sec-multiple-outputs-installing-nixos">
+ <title>Selecting outputs to install via NixOS</title>
+
+ <para>
+ NixOS provides two ways to select the outputs to install for packages listed in <varname>environment.systemPackages</varname>:
+ </para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ The configuration option <varname>environment.extraOutputsToInstall</varname> is appended to each package's <varname>meta.outputsToInstall</varname> attribute to determine the outputs to install. It can for example be used to install for <literal>info</literal> documentation or debug symbols for all packages.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The outputs can be listed as packages in <varname>environment.systemPackages</varname>. For example, the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package can be installed by including <varname>coreutils</varname> and <varname>coreutils.info</varname> in <varname>environment.systemPackages</varname>.
The fact that outputs are vended as attributes on the package hasn't been
explained yet. That happens in the next section *Using a split package*.
We may need to pull that first sentence from that section up to this
section (not inside this list item, but in the intro paragraph, perhaps
right after explaining the outputs attribute).
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> <para>
- NixOS provides configuration option <varname>environment.extraOutputsToInstall</varname> that allows adding extra outputs of <varname>environment.systemPackages</varname> atop the default ones. It's mainly meant for documentation and debug symbols, and it's also modified by specific options.
+ installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstal</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>.
⬇️ Suggested change
- installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstal</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>.
+ installs the <literal>"out"</literal> output (<varname>coreutils.meta.outputsToInstall</varname> is <literal>[ "out" ]</literal>) instead of the requested <literal>"info"</literal>.
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> + The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay installs the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package:
+ </para>
+
+<programlisting>self: super:
+{
+ coreutils = super.coreutils.overrideAttrs (oldAttrs: {
+ meta = oldAttrs.meta // { outputsToInstall = [ "out" "info" ]; };
+ });
+}
It would be nice to change this to showing how to add info instead of
overriding the existing outputs. coreutils doesn't currently have a man
output (which is a shame, I'd like to be able to get the GNU coreutils
manpages since darwin has the FreeBSD ones), but many other packages do so
we shouldn't accidentally instruct the user to throw that away.
⬇️ Suggested change
- The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay installs the <literal>"out"</literal> and <literal>"info"</literal> outputs for the <varname>coreutils</varname> package:
- </para>
-
-<programlisting>self: super:
-{
- coreutils = super.coreutils.overrideAttrs (oldAttrs: {
- meta = oldAttrs.meta // { outputsToInstall = [ "out" "info" ]; };
- });
-}
+ The only recourse to select an output with <command>nix-env</command> is to override the package's <varname>meta.outputsToInstall</varname>, using the functions described in <xref linkend="chap-overrides" />. For example, the following overlay adds the <literal>"info"</literal> output for the <varname>coreutils</varname> package:
+ </para>
+
+<programlisting>self: super:
+{
+ coreutils = super.coreutils.overrideAttrs (oldAttrs: {
+ meta = oldAttrs.meta // { outputsToInstall = oldAttrs.meta.outputsToInstall or [ "out" ] ++ [ "info" ]; };
+ });
+}
------------------------------
In pkgs/stdenv/generic/make-derivation.nix
<#76794 (comment)>:
> @@ -287,7 +287,8 @@ in rec {
name = attrs.name or "${attrs.pname}-${attrs.version}";
# If the packager hasn't specified `outputsToInstall`, choose a default,
- # which is the name of `p.bin or p.out or p`;
+ # which is the name of `p.bin or p.out or p` along with `p.man` when
+ # present;
# if he has specified it, it will be overridden below in `// meta`.
Since you're editing this, can we remove the unnecessarily gendered
pronoun here?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#76794 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC5LGX77T3EDDUNTBEQRUTR5XPFTANCNFSM4KB4LVKQ>
.
|
|
@lilyball I have attempted to address all your concerns. Let me know if you need anything more. |
infinisil
reviewed
Aug 22, 2020
| <para> | ||
| TODO: more about tweaking the attribute, etc. | ||
| <command>nix-env</command> silenty disregards the outputs selected by the user, and instead installs the outputs from <varname>meta.outputsToInstall</varname>. For example, |
There was a problem hiding this comment.
This is already mentioned in the paragraph above. Can probably remove the warning section.
|
I see what you are saying, but I think this warrants a warning box. This is
a very confusing bug from a user perspective and having a self-contained
warning box describing the problem can save time for the reader. If we want
to remove the warning box, the section needs to be tweaked because the
warning box contains the example that is used in the following paragraph.
On a related note, any idea on what can be done to push this PR past the
finish line? This bug has been known for a long time, there was a TODO in
the doc, I offered some content almost 8 months ago and I cannot seem to
get traction to get this merged in.
…On Sat, Aug 22, 2020 at 10:50 AM Silvan Mosberger ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In doc/stdenv/multiple-output.xml
<#76794 (comment)>:
> <para>
- TODO: more about tweaking the attribute, etc.
+ <command>nix-env</command> silenty disregards the outputs selected by the user, and instead installs the outputs from <varname>meta.outputsToInstall</varname>. For example,
This is already mentioned in the paragraph above. Can probably remove the
warning section.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#76794 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAC5LGTA4DMMHGK5OHZUDTLSB7LKTANCNFSM4KB4LVKQ>
.
|
Alright I see. I guess it's fine, even though the warning box contains part of what was said just before it.
That's what I'm here for :). I can merge this PR after the commits are squashed! |
dudebout
force-pushed
the
document-nix-env-multiple-output-install-bug
branch
from
1b6a8fd
to
e3be3a8
Compare
|
@infinisil I have squashed the commits |
dudebout
force-pushed
the
document-nix-env-multiple-output-install-bug
branch
from
e3be3a8
to
611258f
Compare
|
And re-squashed with trailing whitespaces removed |
fricklerhandwerk
added a commit
to fricklerhandwerk/nixpkgs
that referenced
this pull request
Sep 18, 2023
The text was originally added [0] following an apparently incomplete research on how everything plays together. In fact, Nix propagates `outputs` to the corresponding nested derivations, and there is some messy behavior in Nixpkgs that only seems to propagate `meta.outputsToInstall` in `buildEnv`[1]. This change moves the hints on how to use NixOS specifics to NixOS module documentation (which is hopefully easier to find through search.nixos.org), describes the default behavior in Nixpkgs (updating a the link to the source), and removes the confusing mention of `nix-env`. the last of them should not be there to begin with. we don't want beginners to use `nix-env`, as this is known to run them into trouble eventually. [0]: NixOS#76794 [1]: https://github.com/NixOS/nixpkgs/blob/1774d07242995050d2d8a91cb4da0855eac2e472/pkgs/build-support/buildenv/default.nix#L66
fricklerhandwerk
added a commit
to fricklerhandwerk/nixpkgs
that referenced
this pull request
Sep 18, 2023
The text was originally added [0] following an apparently incomplete research on how everything plays together. In fact, Nix propagates `outputs` to the corresponding nested derivations, and there is some messy behavior in Nixpkgs that only seems to propagate `meta.outputsToInstall` in `buildEnv`[1]. This change moves the hints on how to use NixOS specifics to NixOS module documentation (which is hopefully easier to find through search.nixos.org), describes the default behavior in Nixpkgs (updating a the link to the source), and removes the confusing mention of `nix-env`. the last of them should not be there to begin with. we don't want beginners to use `nix-env`, as this is known to run them into trouble eventually. [0]: NixOS#76794 [1]: https://github.com/NixOS/nixpkgs/blob/1774d07242995050d2d8a91cb4da0855eac2e472/pkgs/build-support/buildenv/default.nix#L66
fricklerhandwerk
added a commit
to fricklerhandwerk/nixpkgs
that referenced
this pull request
Sep 18, 2023
The text was originally added [0] following an apparently incomplete research on how everything plays together. In fact, Nix propagates `outputs` to the corresponding nested derivations, and there is some messy behavior in Nixpkgs that only seems to propagate `meta.outputsToInstall` in `buildEnv`[1]. This change moves the hints on how to use NixOS specifics to NixOS module documentation (which is hopefully easier to find through search.nixos.org), describes the default behavior in Nixpkgs (updating a the link to the source), and removes the confusing mention of `nix-env`. the last of them should not be there to begin with. we don't want beginners to use `nix-env`, as this is known to run them into trouble eventually. [0]: NixOS#76794 [1]: https://github.com/NixOS/nixpkgs/blob/1774d07242995050d2d8a91cb4da0855eac2e472/pkgs/build-support/buildenv/default.nix#L66
fricklerhandwerk
added a commit
that referenced
this pull request
Sep 25, 2023
…255947) The text was originally added [0] following an apparently incomplete research on how everything plays together. In fact, Nix propagates `outputs` to the corresponding nested derivations, and there is some messy behavior in Nixpkgs that only seems to propagate `meta.outputsToInstall` in `buildEnv`[1]. This change moves the hints on how to use NixOS specifics to NixOS module documentation (which is hopefully easier to find through search.nixos.org), describes the default behavior in Nixpkgs (updating a the link to the source), and removes the confusing mention of `nix-env`. the last of them should not be there to begin with. we don't want beginners to use `nix-env`, as this is known to run them into trouble eventually. [0]: #76794 [1]: https://github.com/NixOS/nixpkgs/blob/1774d07242995050d2d8a91cb4da0855eac2e472/pkgs/build-support/buildenv/default.nix#L66
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Document the issue and workaround discussed in https://discourse.nixos.org/t/cant-find-a-way-to-install-gnumake-info-files-to-user-environment/3534 @doronbehar @lilyball @lheckemann
cc @dtzWill @matthewbauer as you both worked in that space in #35884
I encountered the issue trying to install netcat from nix-env, which is not easy anymore since it is provided via the
ncoutput of libressl which is not part ofoutputsToInstall(#14708, #39634)