sympa: init at 6.2.52 + NixOS module #65397

mmilata · 2019-07-25T23:02:45Z

Motivation for this change

Adds Sympa mailing list manager and its dependencies to nixpkgs, with NixOS module.

Things done

(joint effort w/ @sorki)

dasJ

Sorry for the high amount of comments, but it's also a huge PR :/

cc @Scriptkiddi maybe you can have a look here?

maintainers/maintainer-list.nix

dasJ · 2019-07-28T17:26:59Z

nixos/modules/misc/ids.nix

@@ -340,6 +340,7 @@
      cockroachdb = 313;
      zoneminder = 314;
      paperless = 315;
+      sympa = 316;


I don't think sympa needs to be added here. Just create a user with isSystemUser = true and it will get some free UID. If you want to go wild, you can try systemd's DynamicUser ;)

Hmm, DynamicUser looks good though it's not obvious if it works with the kind of setup w/ multiple unit files and multiple daemon processes we have here.

nixos/modules/misc/ids.nix

nixos/modules/services/mail/sympa.nix

dasJ · 2019-07-28T17:40:12Z

pkgs/servers/mail/sympa/default.nix

+stdenv.mkDerivation rec {
+  name = "${pname}-${version}";
+
+  src = fetchurl {


please use fetchFromGitHub instead

~~Working on this.~~ Is building from git generally preferred to building from release tarballs?

pkgs/servers/mail/sympa/default.nix

dasJ · 2019-07-28T17:42:27Z

pkgs/servers/mail/sympa/default.nix

+    for i in chown chgrp chmod; do
+      echo '#!${stdenv.shell}' >> "$TMP/bin/$i"
+      chmod +x "$TMP/bin/$i"
+      PATH="$TMP/bin:$PATH"


Move this one line lower. Also can't you just omit that?

Oops. Omitting causes the build to fail in install phase:

make install-exec-hook make[4]: Entering directory '/build/sympa-6.2.44/src/cgi' chown sympa /nix/store/2yv66g71p9bmy7i1gxry4gvp25wl203y-sympa-6.2.44/bin/wwsympa-wrapper.fcgi chown: invalid user: 'sympa' make[4]: [Makefile:783: install-exec-hook] Error 1 (ignored) chgrp sympa /nix/store/2yv66g71p9bmy7i1gxry4gvp25wl203y-sympa-6.2.44/bin/wwsympa-wrapper.fcgi chgrp: invalid group: 'sympa' make[4]: [Makefile:784: install-exec-hook] Error 1 (ignored) chmod 6755 /nix/store/2yv66g71p9bmy7i1gxry4gvp25wl203y-sympa-6.2.44/bin/wwsympa-wrapper.fcgi chmod: changing permissions of '/nix/store/2yv66g71p9bmy7i1gxry4gvp25wl203y-sympa-6.2.44/bin/wwsympa-wrapper.fcgi': Operation not permitted

Scriptkiddi

Hey first of really nice that you are packaging sympa, I know the config is not something that is fun to read.
I would add an option to add files and folders to the dataDir because that covers pretty much all the other options sympa has, like plugins, list templates and list families, but thats something that can be done after this pr

nixos/modules/services/mail/sympa.nix

mmilata

@dasJ @Scriptkiddi thank you for your feedback! I tried to address most of the issues and marked them as resolved. Still working on ~~fetchFromGitHub and~~ structural settings.

Sorry for the high amount of comments, but it's also a huge PR :/

Don't worry, that's expected in order to achieve quality;) Would splitting this PR e.g. into one w/ package and another with NixOS module help?

I would add an option to add files and folders to the dataDir because that covers pretty much all the other options sympa has

Something like environment.etc? Is there other module that has option like this for me to ~~copy it~~ study?

mmilata · 2019-07-30T00:52:40Z

@infinisil I've refactored the module to use structural settings, PTAL.

infinisil · 2019-07-30T02:20:59Z

Forgot to push?

mmilata · 2019-07-30T08:59:55Z

Yep ... pushed now.

Scriptkiddi · 2019-07-30T09:14:12Z

something like it, the Gams package is doing it.

mmilata · 2019-07-30T18:06:12Z

Regarding the rest of the configuration, it doesn't seem feasible to model the structure of all the config files, so we'd have to have something like environment.etc but for /var/lib/sympa. The other problem is that some (most?) of these files are editable from the Sympa web UI.

I'd propose to leave the rest of the configuration (i.e. other than sympa.conf and robot.conf) unmanaged by NixOS as it still allows you to:

install an empty configuration and edit it through the web UI
manually import the configuration (and archives) from a backup

Scriptkiddi · 2019-07-31T08:01:48Z

Works for me, most of the files are not editable from the web ui but I'm fine with just adding them under /var/lib/sympa

nixos/modules/services/mail/sympa.nix

aanderse · 2019-08-01T21:20:16Z

nixos/modules/services/mail/sympa.nix

+    };
+
+    database = {
+      type = mkOption {


A big 👍 for using services.mysql.ensureDatabases, services.mysql.ensureUsers, services.postgresql.ensureDatabases, and services.postgresql.ensureUsers. I highly suggest adding a database.createLocally option which defaults to true so users don't have to do any additional steps to provision a database. A decent example can be seen here:

nixpkgs/nixos/modules/services/monitoring/zabbix-server.nix

Lines 205 to 224 in c4ce832

services.mysql = optionalAttrs mysqlLocal {

enable = true;

package = mkDefault pkgs.mariadb;

ensureDatabases = [ cfg.database.name ];

ensureUsers = [

{ name = cfg.database.user;

ensurePermissions = { "${cfg.database.name}.*" = "ALL PRIVILEGES"; };

}

];

};

services.postgresql = optionalAttrs pgsqlLocal {

enable = true;

ensureDatabases = [ cfg.database.name ];

ensureUsers = [

{ name = cfg.database.user;

ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; };

}

];

};

Please let me know if anything isn't clear from the example or you have any questions as I'm always happy to help (especially when it comes to utilizing ensureDatabases and ensureUsers.

pkgs/servers/mail/sympa/default.nix

aanderse · 2019-08-01T22:28:31Z

pkgs/servers/mail/sympa/default.nix

+  ];
+  nativeBuildInputs = [ autoreconfHook ];
+  buildInputs = [ perlPackages.perl makeWrapper ];
+  propagatedBuildInputs = with perlPackages; [


Wow! Are all of these direct dependencies of every program? This could probably benefit from using perl.withPackages like so:

prog1Env = perl.withPackages (p: with p; [ ArchiveZip CGI DateTime ... ]); prog2Env = perl.withPackages (p: with p; [ DBI FGCI ]);

Note you can explicitly list dependencies for each program and a 'better' perl environment will be built. From there you can pass prog1Env and prog2Env into buildInputs and patchShebangs can take care of the perl binary.

Don't know ... I'm not familiar with Sympa internals (nor can I read Perl very well) but it seems that the programs mostly depend on Sympa library/package which in turn depends on these packages.

Is there a way to determine which program depends on which package that doesn't involve reviewing the whole source code?

Sorry I suppose I shouldn't have bothered mentioning about multiple perl environments.

The point you should takeaway is that you can avoid PERL5LIB if you want to by creating a dedicated (custom) perl environment using perl.withPackages, if you want to.

I'm not sure I understand how that works. Do I define a custom perlEnv and then use it instead of perlPackages.perl in buildInputs and use patchShebangs instead of makeWrapper? How do I pass the environment to patchShebangs? Will propagatedBuildInputs be empty? Got an example I can look at?

#64646 is a simple example which should help.

Modified the package to use perl.withPackages, please take a look if this is what you meant.

mmilata · 2019-08-06T01:02:44Z

@infinisil @aanderse thank you for the comments, I think I addressed most of them, please take a look!

pkgs/servers/mail/sympa/default.nix

aanderse · 2019-08-06T01:32:19Z

pkgs/servers/mail/sympa/default.nix

+  ];
+  nativeBuildInputs = [ autoreconfHook ];
+  buildInputs = [ perlPackages.perl makeWrapper ];
+  propagatedBuildInputs = with perlPackages; [


Sorry I suppose I shouldn't have bothered mentioning about multiple perl environments.

The point you should takeaway is that you can avoid PERL5LIB if you want to by creating a dedicated (custom) perl environment using perl.withPackages, if you want to.

mmilata · 2019-09-04T01:01:02Z

Changes:

remove dataDir parameter from sympa package
convert sympa package to using perl.withPackages
make unix user and group not configurable
bring back forking spawn-fcgi with configurable number of children
add settingsFile config option that allows you to customize files under /var/lib/sympa in a way similar to environment.etc

@aanderse @infinisil @Scriptkiddi PTAL & please advise how to proceed with this pull request.

nixos-discourse · 2019-09-28T18:13:05Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review-may-2019/3032/62

mmilata · 2019-10-08T23:38:58Z

Changes:

upgrade sympa package to 6.2.48
use tmpfiles instead of RuntimeDirectory because /run/sympa/wwsympa.sock needs to be owned by nginx
explicitly tell sympa to use /run/mysqld/mysqld.sock for connecting to mysql on localhost because 6.2.48 looks for it under /tmp for some reason

ajs124 · 2019-10-09T08:56:35Z

The mysql.sock thing should be solved by #70010.

mmilata · 2020-02-04T15:01:14Z

FYI I'd still like to get this merged but it needs some more work from my side which will hopefully happen soon.

infinisil · 2020-02-04T22:10:51Z

Other than fixing the merge conflicts, what's left?

mmilata · 2020-02-07T18:50:58Z

I think bumping packages to current versions would be nice to have (e.g. sympa 6.2.52).

mmilata · 2020-02-07T21:56:59Z

Changes:

fix merge conflicts
sympa: 6.2.48 -> 6.2.52
convert test to python

infinisil · 2020-02-10T00:23:34Z

Cool! Let's merge this for now so it gets into 20.03 (https://discourse.nixos.org/t/nixos-20-03-feature-freeze/5655/13). We can fix any potential issues later (hopefully before 20.03 is released).

mmilata · 2020-02-10T02:15:00Z

@infinisil thanks!!!

mmilata requested a review from infinisil as a code owner July 25, 2019 23:02

ofborg bot added 6.topic: nixos 8.has: module (update) 8.has: package (new) 11.by: package-maintainer 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Jul 25, 2019

mmilata mentioned this pull request Jul 25, 2019

Sympa setup - lists.otevrenamesta.cz otevrenamesta/otevrenamesta-cz-configuration#2

Merged

dasJ requested changes Jul 28, 2019

View reviewed changes

Scriptkiddi reviewed Jul 29, 2019

View reviewed changes

nixos/modules/services/mail/sympa.nix Outdated Show resolved Hide resolved

nixos/modules/services/mail/sympa.nix Outdated Show resolved Hide resolved

infinisil reviewed Jul 29, 2019

View reviewed changes

nixos/modules/services/mail/sympa.nix Outdated Show resolved Hide resolved

mmilata force-pushed the sympa branch from 7e21eb7 to ccce783 Compare July 29, 2019 16:41

mmilata commented Jul 29, 2019

View reviewed changes

mmilata force-pushed the sympa branch from ccce783 to d377369 Compare July 29, 2019 21:17

mmilata force-pushed the sympa branch from d377369 to 2e7e296 Compare July 30, 2019 08:54

infinisil requested changes Aug 1, 2019

View reviewed changes

aanderse requested changes Aug 1, 2019

View reviewed changes

mmilata force-pushed the sympa branch 2 times, most recently from 96398dc to 39e6ad2 Compare August 6, 2019 00:34

aanderse reviewed Aug 6, 2019

View reviewed changes

mmilata force-pushed the sympa branch from 39e6ad2 to 46ee52b Compare September 4, 2019 00:50

ofborg bot added the 8.has: documentation label Sep 4, 2019

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Sep 8, 2019

sorki approved these changes Sep 18, 2019

View reviewed changes

mmilata force-pushed the sympa branch from 755b01f to 4a75b73 Compare October 8, 2019 23:33

mmilata changed the title ~~sympa: init at 6.2.44 + NixOS module~~ sympa: init at 6.2.48 + NixOS module Oct 8, 2019

ofborg bot requested a review from sorki October 8, 2019 23:46

sorki approved these changes Feb 4, 2020

View reviewed changes

sorki and others added 5 commits February 7, 2020 22:19

perlPackages.MIMELiteHTML: init at 1.24

3652efd

perlPackages.HTMLStripScriptsParser: init at 1.03

02493a6

perlPackages.HTMLStripScripts: init at 1.06

80098e9

sympa: init at 6.2.52

e39d7fa

nixos/sympa: init module

097ab90

mmilata force-pushed the sympa branch from 4a75b73 to 097ab90 Compare February 7, 2020 21:55

mmilata changed the title ~~sympa: init at 6.2.48 + NixOS module~~ sympa: init at 6.2.52 + NixOS module Feb 7, 2020

ofborg bot requested a review from sorki February 7, 2020 22:13

infinisil merged commit b9d7f1f into NixOS:master Feb 10, 2020

mmilata mentioned this pull request Mar 24, 2020

nixos/sympa: fix outgoing emails, update package version #83258

Merged

10 tasks

mmilata mentioned this pull request Apr 3, 2020

[20.03] nixos/sympa: fix outgoing emails, update package version #84202

Merged

10 tasks

Janik-Haag added the 12. first-time contribution label Jun 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sympa: init at 6.2.52 + NixOS module #65397

sympa: init at 6.2.52 + NixOS module #65397

mmilata commented Jul 25, 2019 •

edited

dasJ left a comment

dasJ Jul 28, 2019

mmilata Jul 29, 2019

dasJ Jul 28, 2019

mmilata Jul 29, 2019 •

edited

dasJ Jul 28, 2019

mmilata Jul 29, 2019

Scriptkiddi left a comment

mmilata left a comment •

edited

mmilata commented Jul 30, 2019

infinisil commented Jul 30, 2019

mmilata commented Jul 30, 2019

Scriptkiddi commented Jul 30, 2019

mmilata commented Jul 30, 2019 •

edited

Scriptkiddi commented Jul 31, 2019

aanderse Aug 1, 2019

aanderse Aug 1, 2019

mmilata Aug 6, 2019

aanderse Aug 6, 2019

mmilata Aug 6, 2019

aanderse Aug 6, 2019

mmilata Sep 4, 2019

mmilata commented Aug 6, 2019

aanderse Aug 6, 2019

mmilata commented Sep 4, 2019

nixos-discourse commented Sep 28, 2019

mmilata commented Oct 8, 2019

ajs124 commented Oct 9, 2019

mmilata commented Feb 4, 2020

infinisil commented Feb 4, 2020

mmilata commented Feb 7, 2020

mmilata commented Feb 7, 2020

infinisil commented Feb 10, 2020

mmilata commented Feb 10, 2020

	services.mysql = optionalAttrs mysqlLocal {
	enable = true;
	package = mkDefault pkgs.mariadb;
	ensureDatabases = [ cfg.database.name ];
	ensureUsers = [
	{ name = cfg.database.user;
	ensurePermissions = { "${cfg.database.name}.*" = "ALL PRIVILEGES"; };
	}
	];
	};

	services.postgresql = optionalAttrs pgsqlLocal {
	enable = true;
	ensureDatabases = [ cfg.database.name ];
	ensureUsers = [
	{ name = cfg.database.user;
	ensurePermissions = { "DATABASE ${cfg.database.name}" = "ALL PRIVILEGES"; };
	}
	];
	};

sympa: init at 6.2.52 + NixOS module #65397

sympa: init at 6.2.52 + NixOS module #65397

Conversation

mmilata commented Jul 25, 2019 • edited

Motivation for this change

Things done

dasJ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmilata Jul 29, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Scriptkiddi left a comment

Choose a reason for hiding this comment

mmilata left a comment • edited

Choose a reason for hiding this comment

mmilata commented Jul 30, 2019

infinisil commented Jul 30, 2019

mmilata commented Jul 30, 2019

Scriptkiddi commented Jul 30, 2019

mmilata commented Jul 30, 2019 • edited

Scriptkiddi commented Jul 31, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmilata commented Aug 6, 2019

Choose a reason for hiding this comment

mmilata commented Sep 4, 2019

nixos-discourse commented Sep 28, 2019

mmilata commented Oct 8, 2019

ajs124 commented Oct 9, 2019

mmilata commented Feb 4, 2020

infinisil commented Feb 4, 2020

mmilata commented Feb 7, 2020

mmilata commented Feb 7, 2020

infinisil commented Feb 10, 2020

mmilata commented Feb 10, 2020

mmilata commented Jul 25, 2019 •

edited

mmilata Jul 29, 2019 •

edited

mmilata left a comment •

edited

mmilata commented Jul 30, 2019 •

edited