Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/sshguard: create ipsets before starting, and clean up after stopping. #65995

Merged
merged 1 commit into from
Aug 19, 2019
Merged

nixos/sshguard: create ipsets before starting, and clean up after stopping. #65995

merged 1 commit into from
Aug 19, 2019

Conversation

danderson
Copy link
Contributor

The fix for #62874 introduced a race condition on startup: the postStart
commands that configure the firewall run concurrently with sshguard's
creation of the ipsets that the rules depend on. Unfortunately iptables
fails hard when referencing an ipset that doesn't exist, so this causes
non-deterministic crashlooping until sshguard wins the race.

This change fixes that race condition by always creating the ipset and
reconfiguring the firewall before starting sshguard, so that the order
of operations is always deterministic.

This change also cleans up the ipsets on sshguard shutdown, so that
removing sshguard from a running system doesn't leave state behind.

Motivation for this change

Fixes #65985.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @sargon

Sorry, something went wrong.

…pping.

The fix for #62874 introduced a race condition on startup: the postStart
commands that configure the firewall run concurrently with sshguard's
creation of the ipsets that the rules depend on. Unfortunately iptables
fails hard when referencing an ipset that doesn't exist, so this causes
non-deterministic crashlooping until sshguard wins the race.

This change fixes that race condition by always creating the ipset and
reconfiguring the firewall before starting sshguard, so that the order
of operations is always deterministic.

This change also cleans up the ipsets on sshguard shutdown, so that
removing sshguard from a running system doesn't leave state behind.

Fixes #65985.
@danderson
Copy link
Contributor Author

This should also be backported to 19.03 ideally, because right now sshguard sometimes fails to start (if the race condition hits too many times, systemd stops retrying).

@ofborg ofborg bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Aug 4, 2019
@mmahut
Copy link
Member

mmahut commented Aug 19, 2019

Thank you for it, I've tested it in a new VM, looks legit. Do you mind cherry picking this to 19.03?

@mmahut mmahut merged commit d7b3d2d into NixOS:master Aug 19, 2019
@wahjava
Copy link
Contributor

wahjava commented Oct 6, 2019

@danderson --- Sorry about the bug, and thank you for fixing.

@Janik-Haag Janik-Haag added the 12. first-time contribution This PR is the author's first one; please be gentle! label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sshguard has a startup race condition
4 participants