Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

firewall: move rpfilter to mangle.PREROUTING to fix conntrack #110197

Merged
merged 1 commit into from Oct 14, 2022

Conversation

milahu
Copy link
Contributor

@milahu milahu commented Jan 20, 2021

fix #51258

make conntrack work: rpfilter must be in the mangle table, after the conntrack stage

see the netfilter flowchart: input path, network layer

--> raw.prerouting --> conntrack --> mangle.prerouting -->
Motivation for this change

this patch is needed to make VPN work (wireguard protocol, configured with wg-quick, but probably also with openvpn protocol)
otherwise the VPN server is not reachable (response packets are dropped)

workarounds:

  • run ip route add $vpn_server_ip via $local_gateway_ip, for example ip route add 1.2.3.4 via 192.168.0.1. this works, because this explcit route has a higher priority in the wg-quick routing config, so the return-path is found correctly
  • in configuration.nix set networking.firewall.checkReversePath = false; to disable rpfilter completely (security risk!)

not tested yet, nixos-rebuild switch fails to compile (for other reasons than this patch)

thanks to another from #wireguard at freenode.net for finding the bug

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-there-a-way-to-extend-the-default-list-of-modules-imported-via-an-overlay-of-some-sort/5282/6

@milahu
Copy link
Contributor Author

milahu commented Mar 29, 2021

successfully tested with module overlay

before patch, with networking.firewall.checkReversePath = true; (default config)

# drop the new rules
iptables -t mangle -D PREROUTING -j nixos-fw-rpfilter
iptables -t mangle -F nixos-fw-rpfilter
iptables -t mangle -X nixos-fw-rpfilter

nixos-rebuild switch

# table mangle
iptables -t mangle -L | grep rpfilter

# table raw
iptables -t raw -L | grep rpfilter
nixos-fw-rpfilter  all  --  anywhere             anywhere            
Chain nixos-fw-rpfilter (1 references)
RETURN     all  --  anywhere             anywhere             rpfilter validmark

add patch in module overlay

mkdir -p /etc/nixos/extra-modules/networking
cd /etc/nixos/extra-modules/networking
# https://github.com/NixOS/nixpkgs/pull/110197
wget https://github.com/NixOS/nixpkgs/raw/15e75e8ff564ae8e6b8349b537098c0c80fed0d9/nixos/modules/services/networking/firewall.nix
wget https://github.com/NixOS/nixpkgs/raw/master/nixos/modules/services/networking/helpers.nix
cd /etc/nixos/extra-modules
# https://github.com/Infinisil/system/pull/4
wget https://github.com/Infinisil/system/raw/838b30ea319e0334bd542048dd3a9a8554c6e843/config/new-modules/default.nix

/etc/nixos/configuration.nix

{
  imports = [
      ./hardware-configuration.nix
      ./extra-modules # module overlays: https://discourse.nixos.org/t/5282
    ];
}

/etc/nixos/extra-modules/default.nix

{
  ignorePaths = [
    "networking/helpers.nix"
  ];
}

/etc/nixos/extra-modules/networking/firewall.nix

{
  disabledModules = [ "services/networking/firewall.nix" ]; # FIXME only needed in module overlay
  ###### interface

after patch

# drop the old rules
iptables -t raw -D PREROUTING -j nixos-fw-rpfilter
iptables -t raw -F nixos-fw-rpfilter
iptables -t raw -X nixos-fw-rpfilter

nixos-rebuild switch

# table mangle
iptables -t mangle -L | grep rpfilter
nixos-fw-rpfilter  all  --  anywhere             anywhere            
Chain nixos-fw-rpfilter (1 references)
RETURN     all  --  anywhere             anywhere             rpfilter validmark

# table raw
iptables -t raw -L | grep rpfilter

as expected:

make conntrack work: rpfilter must be in the mangle table, after the conntrack stage

@stale
Copy link

stale bot commented Sep 26, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 26, 2021
Copy link
Member

@RealityAnomaly RealityAnomaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, would be nice to get this in.

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 9, 2022
@milahu
Copy link
Contributor Author

milahu commented Jan 9, 2022

thanks for your review

i also wanted to show a notice to change config

{
networking.firewall.checkReversePath = false; # -> show notice
networking.firewall.checkReversePathNoticePR110197 = false; # -> hide notice

sample notice:

please remove checkReversePath=false from your config. in most cases, this is a workaround for wireguard, which is not needed any longer. if you really want to disable rpfilter, set checkReversePathNoticePR110197=false; to hide this message

@RealityAnomaly
Copy link
Member

thanks for your review

i also wanted to show a notice to change config

{
networking.firewall.checkReversePath = false; # -> show notice
networking.firewall.checkReversePathNoticePR110197 = false; # -> hide notice

sample notice:

please remove checkReversePath=false from your config. in most cases, this is a workaround for wireguard, which is not needed any longer. if you really want to disable rpfilter, set checkReversePathNoticePR110197=false; to hide this message

Not sure this is a good idea. There are many legitimate reasons for disabling rpfilter, such as when running a router.

@milahu
Copy link
Contributor Author

milahu commented Jan 10, 2022

rp_filter is "default on" for better security

its just a warning in the console, similar to option x was renamed

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/844

@zx2c4
Copy link
Contributor

zx2c4 commented Oct 4, 2022

Looks like somebody ran into this in the #wireguard IRC channel today. Any reason it's still pending?

@ncfavier
Copy link
Member

ncfavier commented Oct 4, 2022

It's me, I'm somebody. I'll probably have a few comments on this, let me just test it first.

Copy link
Member

@ncfavier ncfavier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on both client and server, works fine. Restarting wg-quick reinstalls the --restore-mark rule correctly in the first position since -I is used.

We should probably also remove

# This is forced to false for now because the default "--validmark" rpfilter we apply on reverse path filtering
# breaks the wg-quick routing because wireguard packets leave with a fwmark from wireguard.
networking.firewall.checkReversePath = false;

since this PR fixes the issue.

nixos/modules/services/networking/firewall.nix Outdated Show resolved Hide resolved
@ncfavier
Copy link
Member

ncfavier commented Oct 4, 2022

See firewalld/firewalld#603 for a similar issue fixed by moving the rpfilter check to the mangle table.

@milahu
Copy link
Contributor Author

milahu commented Oct 4, 2022

fixed 2 comments, rebased

@ncfavier
Copy link
Member

ncfavier commented Oct 4, 2022

We should probably also remove

# This is forced to false for now because the default "--validmark" rpfilter we apply on reverse path filtering
# breaks the wg-quick routing because wireguard packets leave with a fwmark from wireguard.
networking.firewall.checkReversePath = false;

since this PR fixes the issue.

I was suggesting doing this in this PR

@ncfavier
Copy link
Member

ncfavier commented Oct 4, 2022

Last nit: please prefix the commit message with nixos/firewall:

@ncfavier
Copy link
Member

ncfavier commented Oct 4, 2022

(and maybe succinctly explain why this is needed in the commit description)

fix wireguard (wg-quick)

netfilter packet flow:
raw.prerouting -> conntrack -> mangle.prerouting

rpfilter must be after conntrack
otherwise response packets are dropped
@milahu
Copy link
Contributor Author

milahu commented Oct 5, 2022

See firewalld/firewalld#603 for a similar issue fixed by moving the rpfilter check to the mangle table.

also adds --validmark to rpfilter
but this is not needed to fix my use case (wireguard)

https://ipset.netfilter.org/iptables-extensions.man.html

rpfilter

Performs a reverse path filter test on a packet. If a reply to the packet would be sent via the same interface that the packet arrived on, the packet will match. [...]

--validmark
Also use the packets' nfmark value when performing the reverse path route lookup.

@ncfavier
Copy link
Member

ncfavier commented Oct 5, 2022

We already have --validmark (in fact this would not work without --validmark because then the fwmark wouldn't be taken into account at all).

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/626

@ncfavier
Copy link
Member

ncfavier commented Oct 6, 2022

@bobby285271 why remove the approvals label?

@bobby285271
Copy link
Member

bobby285271 commented Oct 6, 2022

I have a bot that runs https://github.com/nix-community/label-approved regularly and it removes approval labels in some cases, maybe we disable that behavior? /cc @Artturin

@ncfavier
Copy link
Member

ncfavier commented Oct 6, 2022

Anyway, please merge 😅

@milahu
Copy link
Contributor Author

milahu commented Oct 10, 2022

ping @SuperSandro2000 @NickCao @nrdxp

guys, can we please merge this trivial patch?

@SuperSandro2000
Copy link
Member

Sorry, I am not very confident with networking.

@nrdxp nrdxp merged commit 912a3de into NixOS:master Oct 14, 2022
@Artturin
Copy link
Member

I have a bot that runs https://github.com/nix-community/label-approved regularly and it removes approval labels in some cases, maybe we disable that behavior? /cc @Artturin

No, it was requested by @zowoq

@Artturin I'm assuming the labels are from a bot? Having it label PRs as approved when the approval is outdated (the PR has new commits or force-pushed) is misleading.

#172542 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wireguard: unable to route all traffic through interface
10 participants