Update public-inbox to 1.8.0 and add systemd services #104457

ju1m · 2020-11-21T05:42:49Z

Works for me and is ready for review.

Motivation for this change

Add a public-inbox NixOS service.

Things done

Update to latest release of public-inbox (1.8.0), removing old Nixpkgs patches (merged since).
Work upon @alyssais 's nixos/public-inbox: init #77450 using RFC0042 and freeform modules.
Enable InlineC.
Secure the systemd services :

# systemd-analyze security | grep public-inbox
public-inbox-httpd.service                0.6 SAFE      😀    
public-inbox-imapd.service                0.6 SAFE      😀
public-inbox-nntpd.service                0.6 SAFE      😀

Support enabling systemd-confinement for the services.
Add myself as maintainer to both the package and the service.
Use generators.toGitINI and introduce a formats.gitIni.
Add nixos/tests/public-inbox.nix.
Add option services.public-inbox.postfix.enable for configuring postfix using a dedicated transport, and support recipientDelimiter. No longer use a .forward file.
Adding a release note (not sure if I should).

Ping @alyssais, @catern, @spacekookie

nixos-discourse · 2020-11-25T06:49:05Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/385

nixos/modules/services/mail/public-inbox.nix

infinisil · 2020-11-28T21:51:38Z

Looking pretty good overall!

matthiasbeyer · 2021-04-11T16:23:31Z

@ju1m any plans on updating this PR so we can merge it?

Would be nice to use this to provide an archive for nixpkgs-dev.

alyssais

I'm very sorry for taking so long to get to this. I should have more time available for it now. I'm also happy to help, or to take back over, if you'd prefer.

pkgs/servers/mail/public-inbox/default.nix

alyssais · 2021-04-15T09:48:57Z

pkgs/servers/mail/public-inbox/default.nix

+          git
+          /* for InlineC */
+          gnumake
+          stdenv.cc


This will break if cross-compiled. Is there no way to compile InlineC stuff up front at build time?

Well, according to public-inbox-tuning, recent public-inbox generates and compiles C code at runtime to increase performance:

Process spawning Our optional use of Inline::C speeds up subprocess spawning from large daemon processes. To enable Inline::C, either set the "PERL_INLINE_DIRECTORY" environment variable to point to a writable directory, or create "~/.cache/public-inbox/inline-c" for any user(s) running public-inbox processes. More (optional) Inline::C use will be introduced in the future to lower memory use and improve scalability.

I've little experience with cross-compiling using Nixpkgs, so I don't know when or how it will break, nor how to address that concern. Should I use something else than stdenv.cc to get the target's compiler? Should I put InlineC usage behind a package option?

Ah, I see -- Inline::C is different to XS, in that the code is generated at runtime. So we can't compile at ahead of time and will need a C compiler. In that case, the usual approach I've seen here (that is cross-friendly) is to wrap with gcc instead of stdenv.cc.

Inline::C is cool and all, but...
Has anyone mentioned to upstream that the lack of support for ahead of time compilation/the dependency on a compiler at runtime, is fairly annoying for packaging? I'm sure other package managers would also prefer to not add a dependency on gcc to public-inbox, I don't think that's just us. (And the public-inbox maintainer is, I think, very responsive to such concerns)

Inline::C is optional

pkgs/servers/mail/public-inbox/default.nix

alyssais

It's going to take me a little while to review this because it's so big and so different to my version, so this isn't going to be my last round of comments. But we're getting there. :)

nixos/modules/services/mail/public-inbox.nix

alyssais · 2021-04-19T20:15:38Z

nixos/modules/services/mail/public-inbox.nix

+          to be used as a reverse proxy, eg. in nginx:
+          <code>locations."/inbox".proxyPass = "http://unix:''${config.services.public-inbox.http.port}:/inbox";</code>
+          Set to null and use <code>systemd.sockets.public-inbox-httpd.listenStreams</code>
+          if you need a more advanced listening.


I don't think the extra complexity of taking two different kinds of values here is worth the small convenience of being able to open the firewall automatically.

I'm not sure to understand what you want, isn't openFirewall working well enough?

nixos/modules/services/mail/public-inbox.nix

ju1m · 2021-04-23T06:15:22Z

Thanks for all your feedbacks.
I've uploaded some changes in a "release early" spirit because of the recent regain in interest for this PR, but before taking time to test them nor finishing to address all the concerns raised. I'll make another pass in a few days.

nixos/modules/services/mail/public-inbox.nix

ju1m · 2022-05-02T06:39:32Z

Fix git clone inboxes by reallowing @timer syscalls

ju1m · 2022-05-02T23:37:13Z

Enable confinement. It was not that hard after all, but obviously I may very well have overlooked some corner cases (eg. missing BindPaths=/BindReadOnlyPaths= for some directory in /run).
Ping @Gaelan.

aanderse · 2022-05-03T06:08:42Z

Currently there isn't a single module within NixOS that uses confinement. Merging this PR as-is would set a precedent. I'm not saying that is a good or bad precedent, but I'm saying it is a precedent people should understand and agree to. Given how overzealous this community is about hardening services I see it as almost a certainty that countless modules will start using confinement as soon as developers are aware of the change this PR introduces.

So what are all of the implications of using confinement? Should you remove confinement from this PR, merge, then take this discussion elsewhere and depending on the outcome of that conversation start a new PR to add confinement back?

nixos-discourse · 2022-05-04T01:01:12Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-policy-regarding-systemd-confinement/18976/1

ju1m · 2022-05-04T01:28:13Z

Revert confinement.enable = true, but keep supporting its enabling by the user.
Started a discussion about the policy to follow regarding systemd-confinement.

pkgs/servers/mail/public-inbox/default.nix

ju1m · 2022-05-11T11:47:44Z

Reenable on all platforms but disable tests on Darwin

pkgs/servers/mail/public-inbox/default.nix

Add support for enabling confinement but does not enable it by default yet because so far no module within NixOS uses confinement hence that would set a precedent.

ju1m · 2022-05-11T23:59:01Z

Use stdenv.cc.cc instead of gcc directly.

infinisil · 2022-05-12T18:47:49Z

This PR has had enough feedback, imo it looks good enough for a merge, any further improvements can be done with other PRs. Thanks for the hard work @ju1m!

ofborg bot added 6.topic: nixos 8.has: module (update) labels Nov 21, 2020

ju1m force-pushed the public-inbox branch 5 times, most recently from 9f79ab7 to 67c9564 Compare November 25, 2020 06:42

ju1m force-pushed the public-inbox branch from 67c9564 to b48fb74 Compare November 26, 2020 06:19

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 labels Nov 26, 2020

infinisil requested changes Nov 28, 2020

View reviewed changes

teto mentioned this pull request Dec 21, 2020

nixos/public-inbox: init #77450

Closed

10 tasks

alyssais reviewed Apr 15, 2021

View reviewed changes

alyssais requested changes Apr 19, 2021

View reviewed changes

ju1m force-pushed the public-inbox branch from b48fb74 to f74b0bb Compare April 23, 2021 05:31

ju1m changed the title ~~Update public-inbox to 1.6.0 and add systemd services~~ Update public-inbox to 1.6.1 and add systemd services Apr 23, 2021

ju1m force-pushed the public-inbox branch from f74b0bb to 9311f68 Compare April 23, 2021 05:46

SuperSandro2000 requested a review from aanderse April 23, 2021 23:17

infinisil reviewed May 3, 2021

View reviewed changes

nixos/modules/services/mail/public-inbox.nix Outdated Show resolved Hide resolved

infinisil requested changes May 3, 2021

View reviewed changes

nixos/modules/services/mail/public-inbox.nix Outdated Show resolved Hide resolved

nixos/modules/services/mail/public-inbox.nix Outdated Show resolved Hide resolved

ju1m force-pushed the public-inbox branch 2 times, most recently from 00d31cb to 9b6cab8 Compare May 11, 2021 03:09

ju1m requested a review from alyssais May 11, 2021 03:27

Gaelan reviewed May 19, 2021

View reviewed changes

nixos/modules/services/mail/public-inbox.nix Outdated Show resolved Hide resolved

ju1m force-pushed the public-inbox branch from 9b6cab8 to 8e62add Compare May 20, 2021 14:25

ofborg bot removed the 10.rebuild-darwin: 1-10 label Apr 17, 2022

ju1m force-pushed the public-inbox branch from b8790c1 to d93522a Compare May 2, 2022 06:38

ju1m force-pushed the public-inbox branch from 0b17190 to 1ca5dc6 Compare May 3, 2022 23:22

Artturin added the 12.approvals: 2 label May 4, 2022

ju1m changed the title ~~Update public-inbox to 1.7.0 and add systemd services~~ Update public-inbox to 1.8.0 and add systemd services May 6, 2022

SuperSandro2000 reviewed May 9, 2022

View reviewed changes

pkgs/servers/mail/public-inbox/default.nix Outdated Show resolved Hide resolved

ju1m force-pushed the public-inbox branch from ca5e6d7 to f9f41e3 Compare May 11, 2022 11:45

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 and removed 10.rebuild-darwin: 0 labels May 11, 2022

SuperSandro2000 reviewed May 11, 2022

View reviewed changes

pkgs/servers/mail/public-inbox/default.nix Outdated Show resolved Hide resolved

ju1m added 7 commits May 12, 2022 01:56

public-inbox: 1.2.0 -> 1.6.1

9b6223d

nixos/public-inbox: init

8514800

nixos/public-inbox: add tests

0e29044

public-inbox: 1.6.1 -> 1.7.0

6cf2f46

public-inbox: 1.7.0 -> unstable-2022-04-05

8b2b5be

nixos/public-inbox: support enabling confinement

c646d37

Add support for enabling confinement but does not enable it by default yet because so far no module within NixOS uses confinement hence that would set a precedent.

public-inbox: unstable-2022-04-05 -> 1.8.0

fd2616c

ju1m force-pushed the public-inbox branch from f9f41e3 to fd2616c Compare May 11, 2022 23:58

infinisil merged commit fd50826 into NixOS:master May 12, 2022

ju1m deleted the public-inbox branch December 26, 2022 20:21

infinisil mentioned this pull request Oct 10, 2023

move question on impurities to FAQs section NixOS/nix.dev#757

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update public-inbox to 1.8.0 and add systemd services #104457

Update public-inbox to 1.8.0 and add systemd services #104457

ju1m commented Nov 21, 2020 •

edited

nixos-discourse commented Nov 25, 2020

infinisil commented Nov 28, 2020

matthiasbeyer commented Apr 11, 2021

alyssais left a comment

alyssais Apr 15, 2021

ju1m Apr 15, 2021

alyssais Apr 16, 2021

catern Apr 17, 2021

ju1m May 10, 2021

alyssais left a comment

alyssais Apr 19, 2021

ju1m May 10, 2021

ju1m commented Apr 23, 2021

ju1m commented May 2, 2022

ju1m commented May 2, 2022

aanderse commented May 3, 2022

nixos-discourse commented May 4, 2022

ju1m commented May 4, 2022

ju1m commented May 11, 2022

ju1m commented May 11, 2022

infinisil commented May 12, 2022

Update public-inbox to 1.8.0 and add systemd services #104457

Update public-inbox to 1.8.0 and add systemd services #104457

Conversation

ju1m commented Nov 21, 2020 • edited

Motivation for this change

Things done

nixos-discourse commented Nov 25, 2020

infinisil commented Nov 28, 2020

matthiasbeyer commented Apr 11, 2021

alyssais left a comment

Choose a reason for hiding this comment

alyssais Apr 15, 2021

Choose a reason for hiding this comment

ju1m Apr 15, 2021

Choose a reason for hiding this comment

alyssais Apr 16, 2021

Choose a reason for hiding this comment

catern Apr 17, 2021

Choose a reason for hiding this comment

ju1m May 10, 2021

Choose a reason for hiding this comment

alyssais left a comment

Choose a reason for hiding this comment

alyssais Apr 19, 2021

Choose a reason for hiding this comment

ju1m May 10, 2021

Choose a reason for hiding this comment

ju1m commented Apr 23, 2021

ju1m commented May 2, 2022

ju1m commented May 2, 2022

aanderse commented May 3, 2022

nixos-discourse commented May 4, 2022

ju1m commented May 4, 2022

ju1m commented May 11, 2022

ju1m commented May 11, 2022

infinisil commented May 12, 2022

ju1m commented Nov 21, 2020 •

edited