Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos-container: allow setting custom local and host address #60029

Merged
merged 1 commit into from Jun 6, 2019

Conversation

Ma27
Copy link
Member

@Ma27 Ma27 commented Apr 22, 2019

Motivation for this change

I have a nixops network where I deploy containers using the container
backend which uses nixos-container intenrally to deploy several
containers to a certain host.

During that time I removed and added new containers and while trying to
deploy those to a different host I realized that it isn't guaranteed
that each container gets the same IP address which is a problem as some
parts of the deployment need to know which container is using which IP
(i.e. to configure port forwarding on the host).

With this change you can specify the container's IP like this (and don't
have to use the arbitrarily used 10.233.0.0/16 subnet):

$ nixos-container create test --config-file test-container.nix \
    --local-address 10.235.1.2 --host-address 10.235.1.1
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

I have a nixops network where I deploy containers using the `container`
backend which uses `nixos-container` intenrally to deploy several
containers to a certain host.

During that time I removed and added new containers and while trying to
deploy those to a different host I realized that it isn't guaranteed
that each container gets the same IP address which is a problem as some
parts of the deployment need to know which container is using which IP
(i.e. to configure port forwarding on the host).

With this change you can specify the container's IP like this (and don't
have to use the arbitrarily used 10.233.0.0/16 subnet):

```
$ nixos-container create test --config-file test-container.nix \
    --local-address 10.235.1.2 --host-address 10.235.1.1
```
@nixos-discourse
Copy link

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review-may-2019/3032/9

Copy link
Contributor

@danbst danbst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally all NixOS container options should be allowed to specify on command line. This is small step towards that, hence 👍

I'm not very experienced in Perl, so I can't comment on actual implementation. If it works for you, then probably this is fine.

@Ma27
Copy link
Member Author

Ma27 commented Jun 4, 2019

Ideally all NixOS container options should be allowed to specify on command line. This is small step towards that, hence 👍

Full ack, in fact I'm planning to do this in the long term (and fix the container backend of nixops accordingly).

I'm not sure how far I'll go then, but this was (at least for me) the biggest issue I had and I was motivated to fix it to get a bit more familiar the the code :)

@globin globin merged commit 2cca718 into NixOS:master Jun 6, 2019
@Ma27 Ma27 deleted the configure-ipv4-for-imperative-container branch June 6, 2019 07:41
Ma27 added a commit to Ma27/nixops that referenced this pull request Jun 13, 2019
With that patch it's possible to deploy containers and assign them to
different IPs than the next available one from `10.233.0.0/16`. This
patch depends on #60029[1] and can only be used with NixOS 19.09
(a.k.a nixos-unstable at the time of writing).

The behavior can be tested using a simple deployment like this:

``` nix
{
  container = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
      deployment.container.localAddress = "10.235.1.2";
      deployment.container.hostAddress = "10.235.1.1";
    };

  # the feature is optional, by default the next free addresses
  # from 10.233.0.0/16 will be used here.
  container2 = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
    };

  horst = { ... }:
    { deployment.targetEnv = "libvirtd";
    };
}
```

[1] NixOS/nixpkgs#60029
Ma27 added a commit to Ma27/nixops that referenced this pull request Jun 13, 2019
With that patch it's possible to deploy containers and assign containers to
different IPs rather than the next available /24 subnet from `10.233.0.0/16`.
This patch depends on #60029[1] and can only be used with NixOS 19.09
(a.k.a nixos-unstable at the time of writing).

The behavior can be tested using a simple deployment like this:

``` nix
{
  container = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
      deployment.container.localAddress = "10.235.1.2";
      deployment.container.hostAddress = "10.235.1.1";
    };

  # the feature is optional, by default the next free addresses
  # from 10.233.0.0/16 will be used here.
  container2 = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
    };

  horst = { ... }:
    { deployment.targetEnv = "libvirtd";
    };
}
```

[1] NixOS/nixpkgs#60029
Ma27 added a commit to Ma27/nixops that referenced this pull request Jun 13, 2019
With that patch it's possible to deploy containers and assign those to
different IPs rather than the next available /24 subnet from `10.233.0.0/16`.
This patch depends on #60029[1] and can only be used with NixOS 19.09
(a.k.a nixos-unstable at the time of writing).

The behavior can be tested using a simple deployment like this:

``` nix
{
  container = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
      deployment.container.localAddress = "10.235.1.2";
      deployment.container.hostAddress = "10.235.1.1";
    };

  # the feature is optional, by default the next free addresses
  # from 10.233.0.0/16 will be used here.
  container2 = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
    };

  horst = { ... }:
    { deployment.targetEnv = "libvirtd";
    };
}
```

[1] NixOS/nixpkgs#60029
Ma27 added a commit to Ma27/nixops that referenced this pull request Jun 14, 2019
With that patch it's possible to deploy containers and assign those to
different IPs rather than the next available /24 subnet from `10.233.0.0/16`.
This patch depends on #60029[1] and can only be used with NixOS 19.09
(a.k.a nixos-unstable at the time of writing).

The behavior can be tested using a simple deployment like this:

``` nix
{
  container = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
      deployment.container.localAddress = "10.235.1.2";
      deployment.container.hostAddress = "10.235.1.1";
    };

  # the feature is optional, by default the next free addresses
  # from 10.233.0.0/16 will be used here.
  container2 = { resources, ... }:
    { deployment.targetEnv = "container";
      deployment.container.host = resources.machines.horst;
    };

  horst = { ... }:
    { deployment.targetEnv = "libvirtd";
    };
}
```

[1] NixOS/nixpkgs#60029
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants