@qknight, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edolstra, @bjornfor and @offlinehacker to be potential reviewers.

👎 on adding vendored code, this seems to imply that the nixpkgs/NixOS maintainers are responsible for maintenance, I'd prefer keeping code like this in seperate repositories, you can always just import those open-source modules from your configuration e.g nixops repository

I share the same opinion. I don't want to discourage your contribution. The module is nice and useful for people, but I think this is out of scope of nixpkgs. For the ease of deployment it is possible to add modules from external resources. Musnix does this for example.

We had an discussion in IRC on how to load external modules and also other examples were provided:

In nix 1.12 there will be builtins.fetchgit:

imports = [ (fetchgit .../nixcloud.git) ];

An alternative is this project https://github.com/xtruder/nix-profiles.

@Mic92 @globin
this is open source code, everyone can do with it what he wants. i'm only saying that we take care of this code which is usually done by the meta attribute and is best practice also.

this abstraction makes mainly sense in nixpkgs as with the tests we can check if it is working after changes, thus less breakages when developers change things. this is the whole point of hydra and makes bug tracking much easier.

currently nixcloud.email won't even show up in the nixos options serach as it is not in the services search field.

I agree with @Mic92 and @globin. It makes more sense to setup a vendor repository and give some instructions on how to add your vendor repository for people that want to use the additions. I could see allowing something like this into nixpkgs resulting in similar PR's with lots of edge cases of how a system should be setup from the point of view of the vendor. The problem then becomes maintaining that mess of extra code.

@disassembler this abstraction makes it so much easier for novice users to setup a mailserver with all there is required and this is really a complex problem. more advanced users can copy parts of this abstraction and create one of their own.

it is really important for us to have this in nixpkgs and we are sure both sides will hugely benefit from this code as well as all the other email related things we already did and have in nixpkgs.

@disassembler @Mic92 @globin
is the problem here the nixcloud.email namespace or the actual implementation? we also have services.gitlab which basically is a rather complex service as it combines many different services in nixpkgs already and is closely related to gitlab. i think that the gitlab service in nixpkgs is a wonderful thing as it makes use of the service easy, teaches how to unse bundler and to combine different nixos modules into a new abstraction.

i think the same is true for this abstraction. maybe we can rename the abstraction into services.nixcloudemail or similar. not sure about the name but we should not name it services.email as it makes it very unclear what such a service covers and what not.

@qknight I'd be willing to pool efforts on an external module solution if this does not get merged. See https://github.com/r-raymond/nixos-mailserver

Getting email right is extremely hard, and the solution will not be in improving the separate pieces as these were not designed as a whole but are scattered everywhere. As an accomplished user, I find having a reproducible integrated solution in NixOS that embodies all best practices would be an extremely valuable building block. I just got a mail from someone asking me literally today.

Using a namespace other than services.email does makes sens, so that people can educate themselves of the risk of this being abandoned at some point - but it is open source, so anyone can pick it up. I'd love this to be part of the NixOS experience of users as soon as possible in the most easy shape and form, because this is such a basic need of everyone. Look forward as well to see more features added by @qknight and @r-raymond, and willing to betatest and contribute.

I also think a mail server is hard to abstract away, our WIP mail server takes quite a different approach in some parts and I'm not sure very opinionated configs belong into nixpkgs. We have stuff like this for our monitoring, which fetches all configs from nixops resources.machines and then generates our monitoring from that. (https://github.com/mayflower/nixos-modules/blob/master/monitoring/default.nix) That is something I cannot imagine being integrated into nixpkgs either and this seems similar to me.

But we might want to establish some best practices on how to easily integrate this kind of extra modules into your own config. We are currently doing this, but needs updating the hash on change, but by that makes sure that the modules are compatible to the deployment always:

$ cat mf-nixops/profile/base.nix 
{ ... }:
let
  # pkgs as an argument creates an infinite recursion
  pkgs = import <nixpkgs> {};
  modules = pkgs.fetchFromGitHub {
    owner = "mayflower";
    repo = "nixos-modules";
    rev = "a60aeba056b13ab9787fb0a417398c509f9a23b7";
    sha256 = "1ldxqhmgc8vd5yn1jix38q2m6hgx2nvkrhlpyn6vwfpzvci6s62h";
  };
in {
  imports = 
    # for development:
    #  (import <nixos-modules/module-list.nix>);
    (import "${modules}/module-list.nix");
}

@globin that is interesting, where is the source code for your mailserver abstraction? i'd like to have a look at that as well.

@qknight not ready yet, but will ping you when we open source it

since there is no futher discussion and more devs dislike to put a generalized email abstraction into nixpkgs we will create our own overlay as globin suggested.

thanks for the discussion.

@r-raymond we will be releasing this in nixcloud-webservices from now on.

@qknight could you link to the repo when it's ready?

@globin using a third party repository with disabledModules over using a nixpkgs branch is IMHO tricky to maintain as nixpkgs updates for 'replaced modules' won't show up in git diff. do you have a good workflow for that as well?

another workflow would be to use a third party repository which is importet and a nixpkgs branch with patches in it. but i can't imagine someone would want to use that since the overhead in debugging is overwhelming. that is also the reason why we as the the nixos community merged nixpkgs and nixos into one repository.

@eqyiel we are about to release our email source code among a few other cool things but we have to find a 'easy to use' solution for the problem stated above first.

@r-raymond @leenaars

what is your oppinion on this, should we fork nixpkgs or use disabledModules with overlays and compare 'forked' services implementation differences manually? i basically think that the overlays approach as appealing advantages but maintaining differences in implementation will be very tricky.

@qknight The approach I have taken so far is too use the stock implementations of nixpkgs, and if necessary work around things by creating systemd modules to handle other stuff (e.g. create dkim certs etc). Is your solution heavily relying on modifying nixpkgs?

@r-raymond we started nixcloud.email with 4 patches to nixpkgs and are down to 1 patch (nixcloud.email not counted). so yes, we need to modify other services in order to react to implementation detail adaptions.

originally we modified:

• postfix
• spamassassin
• dovecot
• opendkim

from which only a patch to opendkim is still open as a PR. yes, we rely heavily on that functionality being able to modify services in nixpkgs i'd say.

do you see a way around this besides forking nixpkgs and maintaining many patches on top of nixpkgs?

@qknight Sorry for answering so late, I'm kind of swamped right now. Hopefully I'll find more time in the beginning of the next year. Anyway, I don't see a way around this if you insist on modifying the services, but I'm not sure I'm up for maintaining a fork of nixpkgs. My hope was that doing this kind of thing should be easier with nix (as compared to people doing this for debian, ubuntu, ... (see e.g. iredmail, mail-in-a-box)), not harder.

I'd vote for getting as many patches as possible to be merged by nixpkgs and working around the one's that can't as described in my earlier post.

@r-raymond we are now using lib.disabledModules and aszlig wrote a feature for us so we can hash a nixpkgs module and keep track of that hash so we can see if upstream changes happened.

that said, we don't aim on getting nixcloud.email into nixpkgs anymore. that is also true for a bunch of other extensions we are about to release soon. oh and we don't have to use a nixpkgs fork but instead are using the imports mechanism only.

@r-raymond nixcloud.email is now finally released, have a look at https://github.com/nixcloud/nixcloud-webservices and https://lastlog.de/blog/posts/nixcloud-webservices.html