Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/networking-interfaces: make static routes configurable #27688

Merged
merged 6 commits into from Feb 19, 2018

Conversation

rnhmjoj
Copy link
Contributor

@rnhmjoj rnhmjoj commented Jul 27, 2017

Motivation for this change

#27380

Things done
  • Test via nixos/tests/networking.nix (added a specific test)
  • Add proper error handling
  • Handle route options
  • Document changes
  • Fits CONTRIBUTING.md.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 27, 2017

That's strange. Wasn't net-tools deprecated in favor of iproute2? It should at least cover every use case.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 27, 2017

networking.localCommands?

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 27, 2017

First disadvantage of networking.localCommands

It works for me: the service is restarted when switching configuration.

sudo nixos-rebuild test
building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/s7i31b3662wz6w6bdzgvkxmbisanmwhq-unit-script.drv
  /nix/store/q4dwrsxs02wq5hh2mq2vzjrjjljhsy33-unit-network-local-commands.service.drv
  /nix/store/h9g94ch8hcspb1xbs9vc3jyy1ginfv24-system-units.drv
  /nix/store/hic1dx0jkw2nc27hxp77wfjirlswvr3w-etc.drv
  /nix/store/p84zvmg9qm4xqi6w5m7aqd4slmmwfpj6-nixos-system-wigfrid-17.09pre111388.00512470ec.drv
building path(s) ‘/nix/store/1kq8q6aknx7rb240drr2v67n03jgmzgy-unit-script’
building path(s) ‘/nix/store/mgj9rqqs4rxizficaqz6dgqv5zj3gvxg-unit-network-local-commands.service’
building path(s) ‘/nix/store/r9h74dlv3qdxfhdks1mspqxvcp893598-system-units’
building path(s) ‘/nix/store/3pqd81nsx28k0gl3ihw1wy0z630cpnyq-etc’
building path(s) ‘/nix/store/jxz4wm0f24nwfx0vhkmx1fgvx5sskgqd-nixos-system-wigfrid-17.09pre111388.00512470ec’
stopping the following units: network-local-commands.service
activating the configuration...
setting up /etc...
setting up tmpfiles
starting the following units: network-local-commands.service

Also, same work 3 years ago: #3929

Thanks. I'll try to adapt those changes to the new network-interfaces.

@rnhmjoj rnhmjoj force-pushed the routes branch 3 times, most recently from 834f54f to e5972ab Compare July 27, 2017 22:39
@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 27, 2017

I have tested IPv4/6 addresses/routes. Everything is working except the stop script fails to remove IPv4 routes for some reason.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 28, 2017

ip route del fails with RTNETLINK answers: No such process

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 28, 2017

I'm trying to write a test but I know exactly zero about perl.

import src/nixpkgs/nixos/tests/make-test.nix

{
  name = "routes";
  machine = {
    networking.useDHCP = false;
    networking.interfaces."eth0" = {
      ip4 = [ { address = "192.168.1.2"; prefixLength = 24; } ];
      ip6 = [ { address = "2001:1470:fffd:2097::"; prefixLength = 64; } ];
      ipv6Routes = [
        { address = "fdfd:b3f0::"; prefixLength = 48; }
        { address = "2001:1470:fffd:2098::"; prefixLength = 64; nextHop = "fdfd:b3f0::1"; }
      ];
      ipv4Routes = [
        { address = "10.0.0.0"; prefixLength = 16; }
        { address = "192.168.2.0"; prefixLength = 24; nextHop = "192.168.1.1"; }
      ];
    };
    virtualisation.vlans = [ ];
  };

  testScript = ''
    my $ipv4Table = <<'END';
    10.0.0.0/16 scope link 
    192.168.1.0/24 proto kernel scope link src 192.168.1.2 
    192.168.2.0/24 via 192.168.1.1 
    END

    my $ipv6Table = <<'END';
    2001:1470:fffd:2097::/64 proto kernel metric 256 pref medium
    2001:1470:fffd:2098::/64 via fdfd:b3f0::1 metric 1024 pref medium
    fdfd:b3f0::/48 metric 1024 pref medium
    END

    $machine->start;
    $machine->waitForUnit("network.target");
    $machine->succeed("ip -4 route list dev eth0 | head -n3") eq "$ipv4Table";
    $machine->succeed("ip -6 route list dev eth0 | head -n3") eq "$ipv6Table";
    $machine->succeed("systemctl stop network-addresses-eth0");
    $machine->succeed("ip -4 route list dev eth0 | head -n-3") eq "";
    $machine->succeed("ip -6 route list dev eth0 | head -n-3") eq "";
  '';
}

How do I make an assertion?

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 28, 2017

Thank you.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 28, 2017

I prefer this way because it's consistent with interfaces.<name>.ip{4,6} and you have a check on the prefix length. I could add a <route>.options instead of three different ones.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 29, 2017

I think it's complete now.

@globin
Copy link
Member

globin commented Jul 29, 2017

What happened here? Who were you talking to? 😲

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Jul 29, 2017

No idea, the messages just disappeared today. It was @volth

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Aug 3, 2017

Updates? What happened to the messages?

@globin globin added this to the 17.09 milestone Aug 11, 2017
@fpletz fpletz self-assigned this Aug 28, 2017
@fpletz fpletz self-requested a review August 28, 2017 17:49
Copy link
Member

@fpletz fpletz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in principle, but I think we need some more options on the top level that can't be added with options. For instance, I use other types than unicast routes quite often. I'll push some more code for options into your branch shortly.

'';
};

nextHop = mkOption {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling this option nextHop is IMHO a bit misleading because there is a nexthop option in ip route that is used for multipath routes yet this option only sets via.

@@ -1059,6 +1117,9 @@ in
'' + optionalString (i.mtu != null) ''
echo "setting MTU to ${toString i.mtu}..."
ip link set "${i.name}" mtu "${toString i.mtu}"
'' + ''
echo -n "bringing up interface... "
ip link set "${i.name}" up && echo "done" || (echo "failed"; exit 1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you recall why this is necessary? It should be fine but just interested. :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I meant the bringing up as a whole but didn't see that this was removed from the addresses service. This is fine! 👍

@@ -189,6 +223,30 @@ let
'';
};

ipv4Routes = mkOption {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really like the extra v in ipv[46]Routes because the address options are just named ip[46]. Even without the v, the interface looks inconsistent to me because it is not clear that ip[46] is only for configuring addresses. No better idea how we should structure the interfaceOpts either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, what about ip{4,6}.{routes,addresses}?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

};

options = mkOption {
type = types.str;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be more appropriate to use an attrset of strings here. That also makes it more easily to override.

@fpletz
Copy link
Member

fpletz commented Aug 28, 2017

I'm wondering if we also need global routes that are not bound to interfaces.

For instance, I'm adding unreachable routes to RFC1918 subnets on my routers to no traffic to them except explicitly defined via the default gateway (which is always an ISP in my case).

@rnhmjoj rnhmjoj force-pushed the routes branch 2 times, most recently from 5f9a127 to fa6de1b Compare September 12, 2017 16:01
@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Sep 12, 2017

Any chance to get this into 17.09?

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Dec 2, 2017

I started working on

  1. renaming ip{4,6} -> ipv{4,6}.addresses
  2. renaming ipv{4,6}Routes -> ipv{4,6}.routes
  3. removing ip{Address,prefixLength}

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Feb 17, 2018

Fixed another conflict.

@oxij
Copy link
Member

oxij commented Feb 18, 2018

Just want to express my thanks for your continuous effort of rebasing this.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Feb 19, 2018

Thank you, also thank you for trying to get the maintainers' attention on this.

@7c6f434c
Copy link
Member

@GrahamcOfBorg test networkingProxy networking.networkd networking.scripted nfs4

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Partial log (click to expand)

cannot build derivation '/nix/store/gmd87q3diy277lajrczx8yly9nwlcc8d-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/hz6sc6xwvvda58wiwxa1lh6bkg75640g-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/vs14ca32s2abfkbs81p83naz24j9846p-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/xxlwrqfsqfhbphbdvl0j6hs35kzs95w8-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/yb3bf7fqky0sg90xgadx8nbp0512qmya-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/z6gyca4ngsxb2g8md6sncwb1igk8dhfb-nixos-test-driver-networking-proxy.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/k9fbkggl1w55f3ilr01iy1v0nzs44b3k-nixos-test-driver-nfs.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/l6yhr2nqzr8qpmw6r2gj9121mrsm1x4r-vm-test-run-networking-proxy.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/k05m4hws5qznf56b7di137zbw2llbwa2-vm-test-run-nfs.drv': 1 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/k05m4hws5qznf56b7di137zbw2llbwa2-vm-test-run-nfs.drv', '/nix/store/l6yhr2nqzr8qpmw6r2gj9121mrsm1x4r-vm-test-run-networking-proxy.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Partial log (click to expand)

client2: running command: sync
client2: exit status 0
test script finished in 242.64s
cleaning up
killing server (pid 695)
killing client2 (pid 625)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/zw6fr3jfjsrw12bzmsmbawljrpbadm1x-vm-test-run-networking-proxy
/nix/store/br73l3c9j5kjf0h8xj38f7i89zp289cf-vm-test-run-nfs

@FRidh FRidh modified the milestones: 17.09, 18.03 Feb 19, 2018
@7c6f434c
Copy link
Member

Observations: and old PR, useful cleanup, looks fine to me, unchanged tests still pass, no objections even though multiple people have read the code, enthusiastic support from @oxij , commit logs in the relevant directory do not make me want to mention someone…

@7c6f434c 7c6f434c merged commit 10b3f7d into NixOS:master Feb 19, 2018
@oxij
Copy link
Member

oxij commented Feb 19, 2018 via email

@dtzWill
Copy link
Member

dtzWill commented Feb 19, 2018

Hopefully it's not hard to fix, but this breaks NixOS configurations using virtualbox, giving error:

The option value `networking.interfaces.vboxnet0.ipv4.addresses' in `/home/will/nixos/nixos/modules/virtualisation/virtualbox-host.nix' is not of type `list of submodules'.

@GrahamcOfBorg test virtualbox.headless

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Partial log (click to expand)

�[31;1merror:�[0m attribute 'headless' in selection path 'tests.virtualbox.headless' not found

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Partial log (click to expand)

while evaluating the attribute 'ExecStart' at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/nixos/modules/services/networking/dhcpcd.nix:179:13:
while evaluating the attribute 'text' of the derivation 'dhcpcd.conf' at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/nixos/modules/services/networking/dhcpcd.nix:19:30, called from undefined position:
while evaluating the attribute 'ipv4.addresses' at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/attrsets.nix:199:44:
while evaluating anonymous function at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/modules.nix:75:45, called from /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/attrsets.nix:199:52:
while evaluating the attribute 'value' at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/modules.nix:312:9:
while evaluating the option `networking.interfaces.vboxnet0.ipv4.addresses':
while evaluating the attribute 'mergedValue' at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/modules.nix:339:5:
while evaluating anonymous function at /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/modules.nix:339:32, called from /home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/lib/modules.nix:339:19:
The option value `networking.interfaces.vboxnet0.ipv4.addresses' in `/home/borg/borg-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/7c6f434c-buildbox/nixos/modules/virtualisation/virtualbox-host.nix' is not of type `list of submodules'.

@oxij
Copy link
Member

oxij commented Feb 19, 2018

Does

diff --git a/nixos/modules/virtualisation/virtualbox-host.nix b/nixos/modules/virtualisation/virtualbox-host.nix
index 91a46ac852b..47ae59bb38a 100644
--- a/nixos/modules/virtualisation/virtualbox-host.nix
+++ b/nixos/modules/virtualisation/virtualbox-host.nix
@@ -117,16 +117,16 @@ in
               VBoxManage hostonlyif create
               cat /tmp/VBoxSVC.log >&2
             fi
           '';
         postStop =
           ''
             VBoxManage hostonlyif remove vboxnet0
           '';
       };
 
-    networking.interfaces.vboxnet0.ipv4.addresses = { address = "192.168.56.1"; prefixLength = 24; };
+    networking.interfaces.vboxnet0.ipv4.addresses = [ { address = "192.168.56.1"; prefixLength = 24; } ];
     # Make sure NetworkManager won't assume this interface being up
     # means we have internet access.
     networking.networkmanager.unmanaged = ["vboxnet0"];
   })]);
 }

fix it?

oxij added a commit to oxij/nixpkgs that referenced this pull request Feb 19, 2018
dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request Feb 19, 2018
@oxij
Copy link
Member

oxij commented Feb 19, 2018

See #35185.

@rnhmjoj
Copy link
Contributor Author

rnhmjoj commented Feb 19, 2018

Thank you for the fix.

@oxij
Copy link
Member

oxij commented Mar 30, 2018 via email

@rnhmjoj rnhmjoj deleted the routes branch February 23, 2019 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants