nixos: doc: include all modules in manual generation #56020

oxij · 2019-02-18T22:36:39Z

What? Why?

An edited version of #47177. See commit messages below.

`git log`

nixos: doc: include all modules in manual generation

Before this change man 5 configuration.nix would only show options of modules in
the baseModules set, which consists only of the list of modules in
nixos/modules/module-list.nix

With this change applied all modules included in configuration.nix file will
be used instead.

This makes configurations with custom modules self-documenting. It also means
that importing non-baseModules modules like gce.nix or azure.nix
will make their documentation available in man 5 configuration.nix.

This was originally implemented in nixos-manual: Include all modules in documentation generation #47177, edited for more configurability and
rebased onto master by @oxij.
nixos: doc: increase maxdepth for xsltproc

See manual fails to build on master #37903 (comment)
for details. With the previous patch and some custom modules included in
configuration.nix the above bug is very easy to trigger.

This is a simplest workaround I have. A proper solution would look like
manual fails to build on master #37903 (comment).

`nix-instantiate` environment

Host OS: Linux 4.9, SLNOS 19.03
Nix: nix-env (Nix) 2.1.3
Multi-user: yes
Sandbox: yes
NIXPKGS_CONFIG:

{
  checkMeta = true;
  doCheckByDefault = true;
}

`nix-env -qaP` diffs

On x86_64-linux:
- Updated (2):
  - tests.nixos-functions.nixos-test
  - tests.nixos-functions.nixosTest-test
On aarch64-linux: ditto
On x86_64-darwin: noop

/cc original author of #47177 @arianvp, documentation aficionados @grahamc @vcunat

arianvp · 2019-02-19T09:01:46Z

Thanks for moving this forward!

danbst · 2019-02-20T08:30:04Z

nixos/lib/eval-config.nix

@@ -51,7 +51,7 @@ in rec {
  # system configuration.
  inherit (lib.evalModules {
    inherit prefix check;
-    modules = modules ++ extraModules ++ baseModules ++ [ pkgsModule ];
+    modules = baseModules ++ extraModules ++ [ pkgsModule ] ++ modules;


what is the rational here? AFAIK modules don't depend on their order.

danbst · 2019-02-20T08:33:56Z

nixos/modules/misc/documentation.nix


 with lib;

 let

  cfg = config.documentation;

+  manualModules = baseModules ++ optionals cfg.nixos.includeAllModules (extraModules ++ modules);


This ignores pkgsModule module. Why ignore that?

oxij · 2019-02-20T10:53:28Z

- modules = modules ++ extraModules ++ baseModules ++ [ pkgsModule ]; + modules = baseModules ++ extraModules ++ [ pkgsModule ] ++ modules; what is the rational here? AFAIK `modules` don't depend on their order.

OCD. This way the order is the same everywhere. This order also makes sense semantically.

+ manualModules = baseModules ++ optionals cfg.nixos.includeAllModules (extraModules ++ modules); This ignores `pkgsModule` module. Why ignore that?

`pkgsModule` is an autogenerated thing defined at the top of `nixos/lib/eval-config.nix`, it has no options/documentation. When and if `nixpkgs.config` will get structured then this will need to be updated.

danbst

OCD

OK :)

I'd also like things like documentation only for custom modules, documentation only about used options, documentation about only changed options, but this is very out of scope in this context.

infinisil · 2019-02-21T17:08:16Z

Can you add a release note for this? I think this very much warrants one

oxij · 2019-02-21T20:00:09Z

Done.

danbst · 2019-02-21T21:03:59Z

I've tried building my system with this patch. Turns out, lots of my custom options are not documented, so I end up in:

error: Option `nixpkgs.latestPackages' has no description.
(use '--show-trace' to show detailed location information)

Fix was easy, add description = ""; to all options, but still it required fixes. Perhaps gradual enable is needed - first add option with false as default, then set it by default true in nixos-generate-config, then change default after a few releases.

oxij · 2019-02-22T00:08:27Z

Perhaps gradual enable is needed - first add option with `false` as default, then set it by default true in `nixos-generate-config`, then change default after a few releases.

Well, since adding descriptions to your custom options is a backwards-compatible change I think this breakage is of okay kind. But for convenience sake, I think, a better approach is to just change that "has no description" error into a warning. The new commit does that.

infinisil · 2019-03-01T18:53:04Z

Testing this on my setup uncovered that nixops' option descriptions apparently don't all evaluate:

error: attribute 'bootstrap' missing, at /nix/store/nsnycm9acirddfdwj0mwiqx88ilwp4mi-nixops-1.6.1pre2728_8ed39f9/share/nix/nixops/gce.nix:343:19

This should be fixed in nixops and nixpkgs' nixopsUnstable should be updated before this can be merged.

danbst · 2019-03-05T02:23:47Z

Change default to false (with respected release notes changes) and I'll merge. It can even be backported to 19.03.

Because this can fail configuration evaluation, we should warn users first. They update to 19.03, see some red warnings in configuration and know, that option declarations must be fixed before 19.09, because 19.09 will make this behavior default.

This is just an idea. Making default false will make life easier for maintainers.

@oxij

Before this change `man 5 configuration.nix` would only show options of modules in the `baseModules` set, which consists only of the list of modules in `nixos/modules/module-list.nix` With this change applied and `documentation.nixos.includeAllModules` option enabled all modules included in `configuration.nix` file will be used instead. This makes configurations with custom modules self-documenting. It also means that importing non-`baseModules` modules like `gce.nix` or `azure.nix` will make their documentation available in `man 5 configuration.nix`. `documentation.nixos.includeAllModules` is currently set to `false` by default as enabling it usually uncovers bugs and prevents evaluation. It should be set to `true` in a release or two. This was originally implemented in NixOS#47177, edited for more configurability, documented and rebased onto master by @oxij.

See NixOS#37903 (comment) for details. With the previous patch and some custom modules included in `configuration.nix` the above bug is very easy to trigger. This is a simplest workaround I have. A proper solution would look like NixOS#37903 (comment).

…out descriptions For convenience, it's not like not having a description is deadly or something.

oxij · 2019-03-05T09:43:27Z

I see, I agree that if this uncovers bugs in third-party things then we should wait with setting it to `true` by default. Rebased onto master, changed the default and documentation. This now adds release notes to 19.09. It's not like this is urgent or anything so enabling this by default in 20.03 should be fine, IMHO.

danbst · 2019-03-05T10:20:14Z

thanks @oxij and @arianvp !

oxij requested review from infinisil and nbp as code owners February 18, 2019 22:36

GrahamcOfBorg added 6.topic: nixos 8.has: documentation 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Feb 18, 2019

danbst reviewed Feb 20, 2019

View reviewed changes

danbst approved these changes Feb 20, 2019

View reviewed changes

oxij force-pushed the pull/47177-nixos-doc-include-all-modules-edited branch from b99203c to 6ccd848 Compare February 21, 2019 19:58

GrahamcOfBorg added the 8.has: changelog label Feb 21, 2019

oxij requested a review from edolstra as a code owner February 22, 2019 00:07

oxij force-pushed the pull/47177-nixos-doc-include-all-modules-edited branch 2 times, most recently from 2c5ce91 to ee229ba Compare March 5, 2019 09:35

arianvp and others added 3 commits March 5, 2019 09:41

lib: optionAttrSetToDocList: warn instead of throwing on options with…

a53b3ba

…out descriptions For convenience, it's not like not having a description is deadly or something.

oxij force-pushed the pull/47177-nixos-doc-include-all-modules-edited branch from ee229ba to a53b3ba Compare March 5, 2019 09:42

danbst merged commit 1f409d0 into NixOS:master Mar 5, 2019

oxij deleted the pull/47177-nixos-doc-include-all-modules-edited branch August 12, 2023 09:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos: doc: include all modules in manual generation #56020

nixos: doc: include all modules in manual generation #56020

oxij commented Feb 18, 2019

arianvp commented Feb 19, 2019

danbst Feb 20, 2019

danbst Feb 20, 2019

oxij commented Feb 20, 2019 via email

danbst left a comment

infinisil commented Feb 21, 2019

oxij commented Feb 21, 2019 via email

danbst commented Feb 21, 2019

oxij commented Feb 22, 2019 via email

infinisil commented Mar 1, 2019

danbst commented Mar 5, 2019

oxij commented Mar 5, 2019 via email

danbst commented Mar 5, 2019

nixos: doc: include all modules in manual generation #56020

nixos: doc: include all modules in manual generation #56020

Conversation

oxij commented Feb 18, 2019

What? Why?

git log

nix-instantiate environment

nix-env -qaP diffs

arianvp commented Feb 19, 2019

danbst Feb 20, 2019

Choose a reason for hiding this comment

danbst Feb 20, 2019

Choose a reason for hiding this comment

oxij commented Feb 20, 2019 via email

danbst left a comment

Choose a reason for hiding this comment

infinisil commented Feb 21, 2019

oxij commented Feb 21, 2019 via email

danbst commented Feb 21, 2019

oxij commented Feb 22, 2019 via email

infinisil commented Mar 1, 2019

danbst commented Mar 5, 2019

oxij commented Mar 5, 2019 via email

danbst commented Mar 5, 2019

`git log`

`nix-instantiate` environment

`nix-env -qaP` diffs