Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consul.passthru.tests: Fix failure on current consul versions, add more tests #90701

Merged
merged 8 commits into from Jun 26, 2020

Conversation

nh2
Copy link
Contributor

@nh2 nh2 commented Jun 18, 2020

Motivation for this change

Fixes #90613. Based on feedback in upstream issue hashicorp/consul#8118.

See commit messages for details.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Fits CONTRIBUTING.md.

nh2 added 8 commits June 18, 2020 02:22
Done by setting `autopilot.min_quorum = 3`.

Techncially, this would have been required to keep the test correct since
Consul's "autopilot" "Dead Server Cleanup" was enabled by default (I believe
that was in Consul 0.8). Practically, the issue only occurred with our NixOS
test with releases >= `1.7.0-beta2` (see NixOS#90613). The setting itself is
available since Consul 1.6.2.

However, this setting was not documented clearly enough for anybody to notice,
and only the upstream issue hashicorp/consul#8118
I filed brought that to light.

As explained there, the test could also have been made pass by applying the
more correct rolling reboot procedure

    -m.wait_until_succeeds("[ $(consul members | grep -o alive | wc -l) == 5 ]")
    +m.wait_until_succeeds(
    +    "[ $(consul operator raft list-peers | grep true | wc -l) == 3 ]"
    +)

but we also intend to test that Consul can regain consensus even if
the quorum gets temporarily broken.
For better naming and commentary.
@nh2
Copy link
Contributor Author

nh2 commented Jun 26, 2020

@GrahamcOfBorg test

@nh2
Copy link
Contributor Author

nh2 commented Jun 26, 2020

@GrahamcOfBorg build consul.passthru.tests

@GrahamcOfBorg test consul

@nh2 nh2 merged commit 5c5f7a2 into NixOS:master Jun 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

consul VM tests are failing with lost consensus
1 participant