New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nixos/modules/system/boot/networkd: enable socket activation #89064
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I propose cherry-picking over the timesyncd and resolved commits from #88158.
@@ -1179,6 +1179,7 @@ in | |||
|
|||
systemd.additionalUpstreamSystemUnits = [ | |||
"systemd-networkd.service" "systemd-networkd-wait-online.service" | |||
"systemd-networkd.socket" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please put each list item in its own newline and sort.
This PR unlike mine is missing the dbus parts. Would be good to cherry pick that too |
Yeah, I mentioned that in my review |
I will happily address all the review comments but I would like you both to reconsider merging these two PRs into one. They both solve different issues and thus should both be accepted. We should try to avoid scope creep and keep things separated as they make logical sense (not purely by looking at the Diff but looking at the actual reasoning). In contrast to #88158 this issue is not about early boot netlink messages or DBUS activation. It does something similar but with a different goal. |
How about we merge this and rebase me on top of it instead of putting the
burden on you to integrate my changes?
I'm happy to do that
…On Mon, Jun 1, 2020, 12:00 Andreas Rammhold ***@***.***> wrote:
I will happily address all the review comments but I would like you both
to reconsider merging these two PRs into one.
They both solve different issues and thus should *both* be accepted. We
should try to avoid scope creep and keep things separated as they make
logical sense (not purely by looking at the Diff but looking at the actual
reasoning).
In contrast to #88158 <#88158> this
issue is *not* about early boot netlink messages or DBUS activation. It
does something similar but with a different goal.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#89064 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEZNI4LKSTZGNPH5KGS7MTRUN35JANCNFSM4NMTW3RQ>
.
|
Alright, I reopened #88158. Please address the other review comments, they're still valid. |
If I haven't missed anything, it seems like all the config options set here were already set in #88158, so all that's left should be the release notes, right? |
d7e241d
to
d4f35ed
Compare
d4f35ed
to
783728d
Compare
I rebased this on top of latest master. Indeed the same config changes were already done in #88158 - your PR made networkd socket-activated to be able to change the socket buffer size, #88158 did it to buffer netlink messages in early boot - so this PR only contains release notes explaining the change itself, and some guidelines on when an increase of the buffer size is needed. |
783728d
to
f44c7c7
Compare
With this release <literal>systemd-networkd</literal> (when enabled through <xref linkend="opt-networking.useNetworkd"/>) | ||
has it's netlink socket created through a <literal>systemd.socket</literal> unit. This gives us control over | ||
socket buffer sizes and other parameters. For larger setups where networkd has to create a lot of (virtual) | ||
devices the default buffer size (currently 128MB) is not enough. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Were these 128MB also the buffer size when networkd wasn't socket-activated by systemd?
Since cd1deda systemd-networkd has it's netlink socket created via a systemd.socket unit. One might think that this doesn't make much sense since networkd is just going to create it's own socket on startup anyway. The difference here is that we have configuration-time control over things like socket buffer sizes vs compile-time constants. For larger setups where networkd has to create a lot of (virtual) devices the default buffer size of currently 128MB is not enough. A good example is a machine with >100 virtual interfaces (e.g., wireguard tunnels, VLANs, …) that all have to be brought up during startup. The receive buffer size will spike due to all the generated message from the new interfaces. Eventually some of the message will be dropped since there is not enough (permitted) buffer space available. By having networkd start through / with a netlink socket created by systemd we can configure the `ReceiveBufferSize` parameter in the socket options without recompiling networkd. Since the actual memory requirements depend on hardware, timing, exact configurations etc. it isn't currently possible to infer a good default from within the NixOS module system. Administrators are advised to monitor the logs of systemd-networkd for `rtnl: kernel receive buffer overrun` spam and increase the memory as required. Note: Increasing the ReceiveBufferSize doesn't allocate any memory. It just increases the upper bound on the kernel side. The memory allocation depends on the amount of messages that are queued on the kernel side of the netlink socket.
f44c7c7
to
55c09a8
Compare
Rebased on latest master. |
Motivation for this change
With this change systemd-networkd has it's netlink socket created via a
systemd.socket unit. One might think that this doesn't make much sense
since networkd is just going to create it's own socket on startup
anyway. The difference here is that we have configuration-time control
over things like socket buffer sizes vs compile-time constants.
For larger setups where networkd has to create a lot of (soft) devices
the default buffer size of currently 128MB is not enough.
A good example is a machine with >100 soft interfaces (e.g., wireguard
tunnels, VLANs, …) that all have to be brought up during startup. The
receive buffer size will spike due to all the generated message from the
new interfaces. Eventually some of the message will be dropped since
there is not enough (permitted) buffer space available.
By having networkd start through / with a netlink socket created by
systemd we can configure the
ReceiveBufferSize
parameter in the socketoptions without recompiling networkd.
Since the actual memory requirements depend on hardware, timing, exact
configurations etc. it isn't currently possible to infer a good default
from within the NixOS module system. Administrators are advised to
monitor the logs of systemd-networkd for
rtnl: kernel receive buffer overrun
spam and increase the memory as required.Note: Increasing the ReceiveBufferSize doesn't allocate any memory. It
just increases the upper bound on the kernel side. The memory allocation
depends on the amount of messages that are queued on the kernel side of
the netlink socket.
Things done
sandbox
innix.conf
on non-NixOS linux)