New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
lib/types: add the selectorFunction type #44601
Conversation
This is used to type options like: services.xserver.windowManager.xmonad.extraPackages that specify functions that take an attribute set containing packages / plugins and return a list containing a selection of the values in this set. The reason we need a dedicated type for this is to define the correct merge behaviour. Without the selectorFunction type merging multiple selector function option definitions results in an evaluation error. The selectorFunction merges definitions by returning a new selector function that applies the selector functions of all the definitions to the given input and concatenates the results.
Ping @Profpatsch @rycee This looks good to me, neat! |
Isn't this just a less generic version of `functionTo` that existed before 4ff1ab5?
In https://github.com/NixOS/nixpkgs/pull/11866/files#r48413164 @nbp writes on the topic:
no function should be allowed in the module system!
Function should be forbidden by default [...] because they are impossible to document properly, and to introspect statically.
I disagree, IMHO function (X → Monoid) is as good a Monoid as any other and hence `types` should provide such an instance. While I agree that overusing `functionTo` in a module system isn't good, that type is pretty useful.
In short, we revived it SLNOS by reverting the above commit. This PR shows that there are uses even in pure NixOS. I think you should just revert 4ff1ab5 instead.
|
No, we should avoid having function as NixOS option. While this might make our life simpler to think in terms of functions, not everybody has a degree in functional programming, and we should not require as much from our users. If you disagree, you are free to have your own modules outside of NixOS, or to use one of the following suggestions: For all the uses cases mentioned here, a simpler alternative is to add an overlay to add the set of packages out of which compatible packages can be extracted, such as Another option, is to create an attribute set which contains the packages to be used as argument, like we do with Sorry, if this answer is not what you had expected, but until I see these alternative fail (i-e not being as expressive or more), I will consider this answer to be final. |
@oxij is right that this is a special case of (one sensible?) function instance for Monoid, that is “a function which takes some
In our case You can easily achieve that generality in your definition as well, by giving Update: lol, @nbp closes this PR as being too confusing for potential users while I’m proposing to make it more general first. |
@Profpatsch Honestly, if we have to explain category theory to understand NixOS, then this project is a failure. I am glad that people can have satisfaction in the sanity of the idea using these. However, this does mean that we should live up to everything which can be expressed in this system, especially if this leads to a documentation hell. Today, I think that functions are a documentation hell to explain to someone who has only manipulated |
Let me preemptively remind you that I mean no personal offense. I see the argument, I disagree with the argument, I dump by thoughts about the argument. Such things are much easier to do via another anonymous peer review system, but NixOS uses GitHub which insists on contributors having names with all the attached personality-related anthropological baggage.
Let me also remind you that the service layer implementation of the module system is one of the three main disagreements between the current NixOS and SLNOS (with the other two being use flags and systemd), so I'm obviously biased.
@nbp
For all the uses cases mentioned here, a simpler alternative is to add an overlay ...
By that same logic, instead of providing `boot.kernelPackages` option we should make users override `pkgs.linuxPackages` with overlays. Such a design reverses the normal relation where nixpkgs provides and NixOS modules use. What I hear you saying is that you want to introduce NixOS-module-specific overlays to nixpkgs for every module that manages an overridable lists of packages. That's way worse than `functionTo`, see below.
Another option, is to create an attribute set which contains the packages to be used as argument
So, instead of
```nix
whateverPackages = p: with p; [ a b c ];
```
users would have to write
```nix
whateverPackagesSet = pkgs.whateverPackagesSet_version;
whateverPackages = with config.whateverPackagesSet; [ a b c ];
```
Isn't that just an ugly encoding of the same function to monoid by tying-the-knot over `config.whateverPackageSet` with a module system itself?
Yes, it is. And your previous solution does the same by tying-the-knot over `pkgs` argument of the module system with overlays.
I also feel like this encoding even apparently looks more complicated than the original. You had several tiiiiny local functions that produced lists, but to hide the fact that they are functions you now have to encode their arguments with huuuuuge fixed points over the whole module system (or via overlays over the `pkgs` argument of the module system) and set/override some seemingly unrelated things.
Also note that neither solution works for multi-instance services (which NixOS doesn't implement ATM, I know, but see below), in both cases you won't be able to give names to your overlays/system.build-like options beforehand.
... but until I see these alternative fail ...
Define "fail". These alternatives fail modularity and KISS-principle checks: they encode the same monoids, but with worse encodings.
To paraphrase Ben Franklin: "Those who would give up Sane Semantics to purchase a little Apparent Simplicity, will have neither good Semantics nor actual Simplicity."
... if we have to explain category theory to understand NixOS ...
1) Neither functions nor monoids are "Category theory".
2) To "understand" NixOS module system one would need to understand both of them anyway (one might reason in an equivalent system, but if one understands how module system works one most certainly understands both already).
3) "Understand" != "Use", to use `functionTo` you don't need to understand what it does. Most users won't even be aware that it exists, even when using the options in question, as witnessed by the current state of having said options without anyone complaining (unless they try to assign them from several places and the module system which has no types for them starts to panic).
I think that functions are a documentation hell to explain to someone who has only manipulated `apt-get install` commands so far
That's not an argument.
1) You don't need to explain, just give examples, newbies copy-paste and modify ready-made examples.
2) Also note that you can't plainly explain how your solutions work either, the user has to set/override some seemingly unrelated things (overlays, some random variables) for them to work properly.
"Hey, so why do I have to use `config.` before `whateverPackageSet` in the assignment to `whateverPackages` to make it work?"
"Because that is not an assignment and `config.` is a dark secret art of the module system you'll never understand before you understand functions, fixed points, and mkMerge attrset monoid anyway. So, please go learn some functional programming first."
Btw, this reminds me that you criticized monoidal services design with a similar "grand-mother" argument (https://nixos.org/nix-dev/2014-December/015370.html). Do you, like, hate monoids or something?
|
@nbp regarding your first suggestion:
The problem with this approach is that you make the user responsible for choosing the correct package set which is error prone. For example say a users defines the following configuration: { pkgs, ... }: {
services.postgresql.package = pkgs.postgresql10;
services.postgresql.extraPlugins = [ pkgs.postgis ];
} This will crash the PostgreSQL server because the { pkgs, ... }: {
services.postgresql.package = pkgs.postgresql10;
services.postgresql.extraPlugins = [
(pkgs.postgis.override { postgresql = pkgs.postgresql10; })
];
} We shouldn't require our users to go to this trouble. In #38698 @thoughtpolice proposes to fix this by having users specify the following: { pkgs, ... }: {
services.postgresql.packages = pkgs.postgresql10Packages;
services.postgresql.plugins = ps: [ ps.postgis ];
} The postgresql module will then take care of applying the plugins function to the choosen postgresql package set. This approach is more safe and it will become extensible by typing the option with Difficulty argumentRegarding your argument that functions are too hard to use for regular NixOS users, aren't we already requiring that users understand functions by having them write configurations like: { pkgs, ... }: { ... } Also having good documentation helps a lot with overcoming this problem. IMHO @thoughtpolice did a good job with this in the documentation of AlternativeA potential alternative which doesn't use functions can look like this: { pkgs, ... }: {
services.postgresql.packages = pkgs.postgresql10Packages;
services.postgresql.plugins = [ "postgis" ];
} So instead of using a function that selects a list of attributes we use a list of strings instead. The strings will denote the names of attributes which the module will then select out of the correct package set. In one way this will be safer since we can't accidentally return an attribute from a set different than the input set. Note that we make this error in the xmonad test. I do wonder what kind of error messages users will get when they misspell an attribute name: @nbp @thoughtpolice do you see any problems with this approach? |
@basvandijk I understand the problem. Thanks for explaining. { pkgs, ... }: {
services.postgresql.packages = pkgs.postgresql10Packages;
services.postgresql.plugins.postgis = true;
} Then we can add a module assertion that all of the attributes provided that way are listed in the list of associated packages. I think that we can provide a better error message than what Nix does by default when looking up attribute sets. Thus, not using |
@nbp I like your suggestion. An additional advantage of having an explicit option for each plugin is that we can use it to add additional options per plugin. So we could use a scheme like for example: { pkgs, ... }: {
services.postgresql.packages = pkgs.postgresql10Packages;
services.postgresql.plugins.postgis = {
enable = true;
... additional postgis options ...
};
} @thoughtpolice would this work for you? |
It probably would, and I think this change can basically be isolated within the module, but there are some things to think about. I guess this is the only way you can make better error messages work (because you can throw an error based on To be clear I'm not, like, put off by that position -- I just do not find it convincing. (It was already bad, the current I don't think the ability to specify per-plugin options is super useful right now, but maybe in the future some other code could make use of it. It might be most useful for overrides (i.e. "I want to apply this patch to my Postgis plugin"), but probably not for most configuration ("I need to set |
@thoughtpolice bloating up the module system should not be an issue. What would be an issue is users not being to figure out that NixOS support the feature they want because of the lack of documentation. For example that you can easily create mediawiki instances: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/web-servers/apache-httpd/mediawiki.nix
Do not blame the module system for not being modular by the fact that NixOS does not have the features you are looking for. I bet one can write a |
We can discuss whether having function options in NixOS is a good thing all day long. But fact is, we currently do have such options, and right now they all throw an obscure error when you assign them multiple times. Because we can't just remove all those options, we need to fix this. And the best way to fix this is to have such a type as in this PR. So I propose to add a warning such as # Function types should only be used when it really makes sense above the type and merge this PR. I've just ran into the same problem with @Jomik on IRC because of this. |
I don't get "serializability" argument. As of now we can't get plain JSON out of configuration
We can't serialize derivations, so need As for selector function, I'd like to have 2 variants:
Actually, I like nixsap style more:
PS: this was already started by @nbp 8 years ago nixpkgs/nixos/modules/misc/nixpkgs.nix Lines 17 to 43 in db70173
|
@danbst regarding:
We could also reconsider #44626 which provides
|
@basvandijk that's fine. What I've proposed is allow dependent types (muahaha) in module system. Then, if we do something like this:
we can infer argument type from Added benefit is that we can introspect that set, for example, in future interactive configurator:
|
I recently implemented something like this ad hoc in #109035. I believe it should be in |
Pinging @infinisil, since he has the best overview of the types setup right now. |
See #110707 |
I marked this as stale due to inactivity. → More info |
This has been implemented with #110707! |
This came out of #38698 (comment).
This is used to type options like:
services.xserver.windowManager.xmonad.extraPackages
that specify functions that take an attribute set containing packages / plugins and return a list containing a selection of the values in this set.Motivation for this change
The reason we need a dedicated type for this is to define the correct merge behaviour. Without the
selectorFunction
type merging multiple selector function option definitions results in an evaluation error.The
selectorFunction
merges definitions by returning a new selector function that applies the selector functions of all the definitions to the given input and concatenates the results.Bikeshedding
Instead of naming it
selectorFunction
we could name itselector
which is shorter. However I opted for the former because it makes it immediately clear the value should be a function.Things done
sandbox
innix.conf
on non-NixOS)nix-shell -p nox --run "nox-review wip"
./result/bin/
)nix path-info -S
before and after)