Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixosTests.networking: Port tests to python #75721

Merged
merged 1 commit into from Dec 19, 2019

Conversation

kampka
Copy link
Contributor

@kampka kampka commented Dec 15, 2019

Motivation for this change

#72828

Note that the networkd version of the networking test suite does not succeed.
This is not an issue with the port but was already the case in the Perl version of the tests.
The issue is that networkd has issues setting up some of the network configurations.
Since I am not very familiar with the networkd setup, I'd like to address that in a different PR.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

cc @tfc @rnhmjoj

Copy link
Contributor

@rnhmjoj rnhmjoj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the nitpicks it looks good. Thank you for your work!

nixos/tests/networking.nix Outdated Show resolved Hide resolved
nixos/tests/networking.nix Outdated Show resolved Hide resolved
nixos/tests/networking.nix Outdated Show resolved Hide resolved
@rnhmjoj
Copy link
Contributor

rnhmjoj commented Dec 15, 2019

It looks like only nixosTests.networking.networkd.virtual is failing.

cc @Ma27 @flokli

@rnhmjoj
Copy link
Contributor

rnhmjoj commented Dec 16, 2019

This is probably the issue:

machine# [    5.747061] systemd-networkd[525]: /etc/systemd/network/40-tap0.network:27: Invalid Gateway='', ignoring assignment: Invalid argument
machine# [    5.754528] systemd-networkd[525]: /etc/systemd/network/40-tun0.network:27: Invalid Gateway='', ignoring assignment: Invalid argument

nixos/tests/networking.nix Outdated Show resolved Hide resolved
$router->waitUntilSucceeds("ping -c 1 192.168.3.1");
$client->waitUntilSucceeds("ping -c 1 192.168.3.1");
router.wait_until_succeeds("ping -c 1 192.168.3.1")
client.wait_until_succeeds("ping -c 1 192.168.3.1")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not:

vlan1_ips = [  "192.168.1.1", " 192.168.1.2", " 192.168.1.3",  "192.168.1.10" ]
vlan2_ips = [  "192.168.2.1", " 192.168.2.2" ]
gateway_ip = "192.168.3.1"

def ping_until_succeeds(machine, addr):
    machine.wait_until_succeeds(f"ping -c 1 {addr}")

with subtest("Test VLAN 1"):
    for machine in client, router:
        for ip in vlan1_ips:
            ping_until_succeeds(machine, ip)

with subtest("Test VLAN 2"):
     for machine in client, router:
         for ip in vlan2_ips:
            ping_until_succeeds(machine, ip)

with subtest("Test default gateway"):
    ping_until_succeeds(router, gateway_ip)
    ping_until_succeeds(client, gateway_ip)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nixos/tests/networking.nix Outdated Show resolved Hide resolved
router.wait_until_succeeds("ping -c 1 192.168.2.1")
router.wait_until_succeeds("ping -c 1 192.168.2.2")
router.wait_until_succeeds("ping -c 1 fd00:1234:5678:2::1")
router.wait_until_succeeds("ping -c 1 fd00:1234:5678:2::2")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code is rather repetitive and could live with some lists of ips, loops and well-named subprocedures, as in the example before

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll stick to #75721 (comment) :)

router.wait_until_succeeds("ping -c 1 192.168.1.1")
router.wait_until_succeeds("ping -c 1 192.168.1.2")
router.wait_until_succeeds("ping -c 1 192.168.1.3")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about:

import itertools

def ping_until_succeeds(machine, ip):
    machine.wait_until_succeeds(f"ping -c 1 {ip}")


machines = [client1, client2, router]
ips = ["192.168.1.1", "192.168.1.2", "192.168.1.3" ]

with subtest("Test bridging"):
    for i in itertools.product(machines, ips):
        ping_until_succeeds(*i)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I prefer simple, repetetive code in tests as opposed to more complex solutions.
At the moment you can read the test without knowing much about how python works.
Your example makes the code more compact / elegant, but also harder to understand.
I can refactor it if you want, but I don't think we should strive for idiomatic python in these tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't see it as a hard requirement. I am fine with either solutions, i just like code to be compact which also reduces the risk for unseen typos

nixos/tests/networking.nix Outdated Show resolved Hide resolved
client_with_privacy.wait_until_succeeds(
"ip addr show dev eth1 | grep -q 'fd00:1234:5678:1:'"
)
client.wait_until_succeeds("ip addr show dev eth1 | grep -q 'fd00:1234:5678:1:'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here, the output of the identical command could again be stored in a variable and then be asserted against with the ip addresses.
Also, the ip addresses could be put into variables with descriptive names, to make this nicer to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see wait_until_succeeds above.

nixos/tests/networking.nix Outdated Show resolved Hide resolved
@Ma27
Copy link
Member

Ma27 commented Dec 16, 2019

machine# [ 5.747061] systemd-networkd[525]: /etc/systemd/network/40-tap0.network:27: Invalid Gateway='', ignoring assignment: Invalid argument
machine# [ 5.754528] systemd-networkd[525]: /etc/systemd/network/40-tun0.network:27: Invalid Gateway='', ignoring assignment: Invalid argument

This is "just" a warning which happens if useDHCP = true, but no Gateway is set for networking.interfaces.<interface>. Should be fixed in #75193 though.

@rnhmjoj
Copy link
Contributor

rnhmjoj commented Dec 16, 2019

@Ma27 Thank you, I saw the commit but after I left the comment.

@kampka
Copy link
Contributor Author

kampka commented Dec 16, 2019

@GrahamcOfBorg test networking.scripted

nixos/tests/networking.nix Outdated Show resolved Hide resolved
Copy link
Contributor

@rnhmjoj rnhmjoj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks all good to me.
All the tests (except networkd.virtual) are passing and the error reporting, if I force a failure, is working as expected.
Some tests could be rewritter more concisely but I too agree with @kampka that's better to keep things explicit and use fewer abstractions in tests.

@Ma27 Ma27 requested a review from flokli December 19, 2019 11:50
@flokli flokli merged commit dee7a7f into NixOS:master Dec 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants