nixos/rspamd: worker configuration, socket activation and tests #34562

griff · 2018-02-03T19:16:44Z

Motivation for this change

These two commits add first tests for existing rspamd module and then updated the module so that you have more control over workers configuration and add support for socket activation.

Care has been taken for the updated module to stay compatible with existing configurations with tests to verify that this is the case.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

andir

Looks good overall. Please have a look at the missing description.

andir · 2018-02-05T15:07:48Z

nixos/modules/services/mail/rspamd.nix

+        description = "The type of this worker";
+      };
+      bindSockets = mkOption {
+        type = types.listOf (types.either types.str (types.submodule bindSocketOpts));


you are missing a description here. That is also why @GrahamcOfBorg is reporting a build failure:

Option `services.rspamd.workers.<name>.bindSockets' has no description. ``

Should be fixed now

andir · 2018-02-05T15:09:32Z

@GrahamcOfBorg test rspamd

GrahamcOfBorg · 2018-02-05T18:33:13Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

error: attribute ‘x86_64-linux’ in selection path ‘tests.rspamd.x86_64-linux’ not found

GrahamcOfBorg · 2018-02-05T18:33:14Z

Success on aarch64-linux (full log)

Partial log (click to expand)

griff · 2018-02-05T19:12:34Z

The test errors seem to be because rspamd has subtests so the selection path should be tests.rspamd.bindports.x86_64-linux.

With a selection path for each of bindports, deprecated, ipv4only, ipv4only-socketActivated, simple, simple-socketActivated and socketActivation

andir · 2018-02-07T09:30:49Z

@GrahamcOfBorg test rspamd.bindports rspamd.deprecated rspamd.ipv4only rspamd.ipv4only-socketActivated rspamd.simple rspamd.simple-socketActivated rspamd.socketActivated

GrahamcOfBorg · 2018-02-07T09:31:10Z

Failure on x86_64-linux (full log)

Partial log (click to expand)

trace: warning: The option `services.rspamd.bindUISocket' defined in `<unknown-file>' has been renamed to `services.rspamd.workers.controller.bindSockets'.
trace: warning: The option `services.rspamd.bindSocket' defined in `<unknown-file>' has been renamed to `services.rspamd.workers.normal.bindSockets'.
error: attribute ‘socketActivated’ in selection path ‘tests.rspamd.socketActivated.x86_64-linux’ not found

GrahamcOfBorg · 2018-02-07T09:32:18Z

Failure on aarch64-linux (full log)

Partial log (click to expand)

trace: warning: The option `services.rspamd.bindUISocket' defined in `<unknown-file>' has been renamed to `services.rspamd.workers.controller.bindSockets'.
trace: warning: The option `services.rspamd.bindSocket' defined in `<unknown-file>' has been renamed to `services.rspamd.workers.normal.bindSockets'.
error: attribute 'socketActivated' in selection path 'tests.rspamd.socketActivated' not found

griff · 2018-02-09T05:27:19Z

@andir Ups I had inconsistently named the tests. Could I trouble you with redoing the @GrahamcOfBorg test command since the tests are now named consistently.

andir · 2018-02-09T11:02:04Z

@GrahamcOfBorg test rspamd.bindports rspamd.deprecated rspamd.ipv4only rspamd.ipv4only-socketActivated rspamd.simple rspamd.simple-socketActivated rspamd.socketActivated

GrahamcOfBorg · 2018-02-09T11:03:52Z

Success on x86_64-linux (full log)

Partial log (click to expand)

machine# [   26.373297] rspamd[765]: <4pr1kh>; map; read_map_file: /nix/store/dddvgkwkx31ic2q3wdfx18xsfnqpvy0b-rspamd-1.6.5/etc/rspamd/2tld.inc: read map dat, 8213 bytes
/nix/store/dccvpliw89gfhr5v5qypafz7039p46b5-vm-test-run-rspamd-socketActivated
machine# [   26.375293] rspamd[765]: <4pr1kh>; map; rspamd_map_file_read_callback: rereading map file /var/lib/rspamd/2tld.inc.local
machine: exit status 0
machine# [   26.379489] rspamd[765]: <4pr1kh>; map; read_map_file: /var/lib/rspamd/2tld.inc.local: map file is unavailable for reading
test script finished in 28.00s
cleaning up
killing machine (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/tmp/nix-build-vm-test-run-rspamd-ipv4only-socketActivated.drv-0/vde1.ctl': Directory not empty

GrahamcOfBorg · 2018-02-09T11:15:22Z

Success on aarch64-linux (full log)

Partial log (click to expand)

killing machine (pid 627)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/fz5chxdw0bwpbqlzp7liwhh3zx1vkxm7-vm-test-run-rspamd-bindports
/nix/store/6n40vxv5jxg8ppmwi6mgspxyl7w0ljii-vm-test-run-rspamd-deprecated
/nix/store/hfxlhva4hrh2vh75f71hsf8ygqk0dk4b-vm-test-run-rspamd-ipv4only
/nix/store/7zks4mmam1f6p65m9n5jz39wbabnw5pf-vm-test-run-rspamd-ipv4only-socketActivated
/nix/store/n5cpyx9zy07x8rx7plr7s18m7dlmj5b1-vm-test-run-rspamd-simple
/nix/store/abxjhj5zszrk3amfsgmxz738y879c00n-vm-test-run-rspamd-simple-socketActivated
/nix/store/2q6ckm8c512np2051fqz6fmpgs5gd156-vm-test-run-rspamd-socketActivated

andir · 2018-02-09T11:35:55Z

nixos/modules/services/mail/rspamd.nix

  };
+  imports = [
+    (mkRenamedOptionModule [ "services" "rspamd" "bindSocket" ] [ "services" "rspamd" "workers" "normal" "bindSockets" ])


Should we add some deprecation deadlin from the beginning? Like saying "with 18.09 this will be removed!" ?

Thinking about this more you probably also want to write a brief changelog entry about "breaking" changes in regards to rspamd ;-)

Well since this does a rename and is backwards compatible it isn't really a breaking change. It works exactly like before when using the old options it just gives you a warning as well.

I also wouldn't add a deadline for removal since having the old option supported has at the moment no downsides and no other usage of mkRenamedOptionModule has such a deadline.

I know.. It was just my personal opinion that we should probably not carry everything for the remaining time of this project. :-)

nixos/tests: add basic tests for services.rspamd

27ee0b9

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) labels Feb 3, 2018

griff mentioned this pull request Feb 4, 2018

Upstream from my fork r-raymond/nixos-mailserver#61

Open

andir requested changes Feb 5, 2018

View reviewed changes

griff force-pushed the rspamd-workers branch from 9621e5b to cca43d0 Compare February 5, 2018 17:28

GrahamcOfBorg added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Feb 5, 2018

nixos/rspamd: options for worker configuration and socket activation

908fc5e

griff force-pushed the rspamd-workers branch from cca43d0 to 908fc5e Compare February 9, 2018 05:19

andir reviewed Feb 9, 2018

View reviewed changes

andir approved these changes Feb 9, 2018

View reviewed changes

andir merged commit 04051ee into NixOS:master Feb 9, 2018

griff deleted the rspamd-workers branch February 9, 2018 13:50

This was referenced Feb 24, 2018

rspamd has no extraConfig and no way to configure workers #25437

Closed

nixos/rspamd: Preserve runtime directory when using socket activation #35690

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/rspamd: worker configuration, socket activation and tests #34562

nixos/rspamd: worker configuration, socket activation and tests #34562

griff commented Feb 3, 2018

andir left a comment

andir Feb 5, 2018

griff Feb 5, 2018

andir commented Feb 5, 2018 •

edited

GrahamcOfBorg commented Feb 5, 2018

GrahamcOfBorg commented Feb 5, 2018

griff commented Feb 5, 2018 •

edited

andir commented Feb 7, 2018

GrahamcOfBorg commented Feb 7, 2018

GrahamcOfBorg commented Feb 7, 2018

griff commented Feb 9, 2018

andir commented Feb 9, 2018

GrahamcOfBorg commented Feb 9, 2018

GrahamcOfBorg commented Feb 9, 2018

andir Feb 9, 2018

andir Feb 9, 2018

griff Feb 9, 2018

andir Feb 9, 2018

nixos/rspamd: worker configuration, socket activation and tests #34562

nixos/rspamd: worker configuration, socket activation and tests #34562

Conversation

griff commented Feb 3, 2018

Motivation for this change

Things done

andir left a comment

Choose a reason for hiding this comment

andir Feb 5, 2018

Choose a reason for hiding this comment

griff Feb 5, 2018

Choose a reason for hiding this comment

andir commented Feb 5, 2018 • edited

GrahamcOfBorg commented Feb 5, 2018

GrahamcOfBorg commented Feb 5, 2018

griff commented Feb 5, 2018 • edited

andir commented Feb 7, 2018

GrahamcOfBorg commented Feb 7, 2018

GrahamcOfBorg commented Feb 7, 2018

griff commented Feb 9, 2018

andir commented Feb 9, 2018

GrahamcOfBorg commented Feb 9, 2018

GrahamcOfBorg commented Feb 9, 2018

andir Feb 9, 2018

Choose a reason for hiding this comment

andir Feb 9, 2018

Choose a reason for hiding this comment

griff Feb 9, 2018

Choose a reason for hiding this comment

andir Feb 9, 2018

Choose a reason for hiding this comment

andir commented Feb 5, 2018 •

edited

griff commented Feb 5, 2018 •

edited