NixOS: Run Docker containers as declarative systemd services #55179

benley · 2019-02-04T08:41:47Z

Motivation for this change

https://discourse.nixos.org/t/deploying-docker-containers-declaratively/693

Related to #26075 but the implementation is less general; this one just offers an interface to run containers as systemd units from pre-existing Docker images.

cc @copumpkin

TODO:

nixos-discourse · 2019-02-04T08:45:34Z

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/deploying-docker-containers-declaratively/693/9

nlewo

It would be nice to add a test for this module also.

nixos/modules/virtualisation/docker-containers.nix

benley · 2019-02-04T18:59:56Z

It would be nice to add a test for this module also.

I absolutely will add tests for this stuff.

costrouc · 2019-02-04T20:15:31Z

Is it too complicated for this PR to address mapping ports as well? Would this require systemd.sockets? I'm thinking with port mapping this PR would cover 90% of all use cases.

benley · 2019-02-04T20:54:41Z

Is it too complicated for this PR to address mapping ports as well? Would this require systemd.sockets? I'm thinking with port mapping this PR would cover 90% of all use cases.

It wouldn't be too hard to handle mapping ports here, as long as we don't try to cover all possible networking setups. I'll have a go at adding it.

benley · 2019-02-04T20:55:20Z

(and no, I don't believe this will require interacting with systemd.sockets)

benley · 2019-02-04T22:24:52Z

I've added support for publishing ports, setting environment variables, and overriding user and workdir.

nixos/modules/virtualisation/docker-containers.nix

expipiplus1 · 2019-02-08T13:06:43Z

Looking forward to using this!

I think it would be worthwhile to take a look at this repository: https://github.com/ibuildthecloud/systemd-docker as well as the linked discussions moby/moby#6791 and https://groups.google.com/forum/#!topic/coreos-dev/wf7G6rA7Bf4/discussion

infinisil

The code looks good to me now (I haven't tested anything though)

tomberek · 2019-03-24T19:45:10Z

Tested. Seems to work well and respond as expected to configuration.nix changes. There are some nuances to deal with like obtaining images (preloader seems difficult to setup), using hostnames or /etc/hosts in containers to avoid hard-coding IPs. Overall, good work! This can serve as a band-aid to deploy applications that are only feasible to run as upstream-provided docker images (or are difficult to port, eg: i got onlyoffice to work with nextcloud using docker-containers).

danbst · 2019-03-24T23:00:33Z

thanks all!

benley · 2019-03-25T20:21:03Z

Woot!

expipiplus1 · 2019-04-11T07:38:59Z

Just checking, but did anyone (@benley, @danbst) take a look at the link I posted: https://github.com/ibuildthecloud/systemd-docker as well as the linked discussions moby/moby#6791 and https://groups.google.com/forum/#!topic/coreos-dev/wf7G6rA7Bf4/discussion

It's not really my area, but I couldn't see any handling of this edge case in this PR

The short of it is that systemd does not actually supervise the Docker container but instead the Docker client. This makes systemd incapable of reliably managing Docker containers without hitting a bunch of really odd situations.

nyanloutre · 2019-04-11T09:39:21Z

Le 11 avril 2019 09:39:06 GMT+02:00, Joe Hermaszewski <notifications@github.com> a écrit :

Just checking, but did anyone ***@***.***, @danbst) take a look at the link I posted: https://github.com/ibuildthecloud/systemd-docker as well as the linked discussions moby/moby#6791 and https://groups.google.com/forum/#!topic/coreos-dev/wf7G6rA7Bf4/discussion It's not really my area, but I couldn't see any handling of this edge case in this PR >The short of it is that systemd does not actually supervise the Docker container but instead the Docker client. This makes systemd incapable of reliably managing Docker containers without hitting a bunch of really odd situations. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #55179 (comment)

If you want to avoid ugly unit files you can use a daemon-less container runtime like rkt or podman. I would be more in favor of podman as it should be compatible with all docker CLI flags and is more active than rkt. Daemon-less mean the CLI is forking as the container so if you kill it for example you kill the container. -- Paul TREHIOU

benley · 2019-04-11T21:38:00Z

Just checking, but did anyone (@benley, @danbst) take a look at the link I posted: https://github.com/ibuildthecloud/systemd-docker as well as the linked discussions moby/moby#6791 and https://groups.google.com/forum/#!topic/coreos-dev/wf7G6rA7Bf4/discussion

It's not really my area, but I couldn't see any handling of this edge case in this PR

The short of it is that systemd does not actually supervise the Docker container but instead the Docker client. This makes systemd incapable of reliably managing Docker containers without hitting a bunch of really odd situations.

Yes, I studied these cases extensively as I was putting this together. The way this is implemented in NixOS, systemd does not attempt to own the container or the processes within it. Instead, it just asks docker to run a thing, keeps it in the foreground, and if the container exits then systemd will know because the docker run command terminates. When systemd wants to stop or restart a container, it also just asks docker to do it with docker stop.

I would love to swap out the implementation at some point to avoid using the Docker daemon, using runc or podman or something like those, and I did try to build this in a way that would make that possible.

expipiplus1 · 2019-04-14T11:32:33Z

@benley great, thanks!

…5179) * WIP: Run Docker containers as declarative systemd services * PR feedback round 1 * docker-containers: add environment, ports, user, workdir options * docker-containers: log-driver, string->str, line wrapping * ExecStart instead of script wrapper, %n for container name * PR feedback: better description and example formatting * Fix docbook formatting (oops) * Use a list of strings for ports, expand documentation * docker-continers: add a simple nixos test * waitUntilSucceeds to avoid potential weird async issues * Don't enable docker daemon unless we actually need it * PR feedback: leave ExecReload undefined (cherry picked from commit c940053)

chrissound · 2020-02-29T15:23:59Z

Is there any documentation for this?

Figured it out:

Here is an example:

{ config, pkgs, ... }:

{

  config.docker-containers = {
    hackagecompare = {
      image = "chrissound/hackagecomparestats-webserver:latest";
      ports = ["127.0.0.1:3010:3010"];
      volumes = [
        "/root/hackagecompare/packageStatistics.json:/root/hackagecompare/packageStatistics.json"
      ];
      cmd = [
        "--base-url"
        "\"/hackagecompare\""
      ];
    };
  };
}

And then the usual systemd interface:

journalctl -u docker-hackagecompare.service
systemctl status docker-hackagecompare.service

expipiplus1 · 2020-03-09T05:15:49Z

@chrissound You should consider putting that in the nixos manual.

fungalcofe · 2021-01-17T10:42:50Z

docker-containers module looks to have been renamed oci-containers and there is now documentation attached.

https://search.nixos.org/options?channel=20.09&from=0&size=30&sort=relevance&query=oci-containers

WIP: Run Docker containers as declarative systemd services

2be46ee

benley added 6.topic: nixos 2.status: work-in-progress 8.has: module (new) 11.by: nixpkgs-member labels Feb 4, 2019

benley requested a review from infinisil as a code owner February 4, 2019 08:41

benley mentioned this pull request Feb 4, 2019

Declaratively running a docker image #37553

Closed

GrahamcOfBorg added 8.has: documentation 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Feb 4, 2019

nlewo requested changes Feb 4, 2019

View reviewed changes

nixos/modules/virtualisation/docker-containers.nix Outdated Show resolved Hide resolved

nixos/modules/virtualisation/docker-containers.nix Outdated Show resolved Hide resolved

PR feedback round 1

d43974b

docker-containers: add environment, ports, user, workdir options

d2f21ed

benley added 2 commits February 4, 2019 17:44

docker-containers: log-driver, string->str, line wrapping

f4f0bb4

ExecStart instead of script wrapper, %n for container name

33c1e31

infinisil requested changes Feb 5, 2019

View reviewed changes

PR feedback: better description and example formatting

12cda86

This comment has been minimized.

Sign in to view

infinisil reviewed Feb 5, 2019

View reviewed changes

nixos/modules/virtualisation/docker-containers.nix Outdated Show resolved Hide resolved

benley added 2 commits February 4, 2019 19:52

Fix docbook formatting (oops)

2bdaa5e

Use a list of strings for ports, expand documentation

ce835a7

infinisil approved these changes Feb 8, 2019

View reviewed changes

benley added the 1 - Ready label Mar 18, 2019

benley requested a review from nlewo March 18, 2019 18:22

danbst merged commit c940053 into NixOS:master Mar 24, 2019

benley deleted the benley/docker-containers branch March 25, 2019 20:22

aszlig mentioned this pull request Mar 26, 2019

Feature Request: Show changes in NixOS option definitions NixOS/ofborg#344

Open

duckfullstop mentioned this pull request Jun 16, 2019

docker-containers: More container options, and tweaks to launch logic #63241

Draft

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NixOS: Run Docker containers as declarative systemd services #55179

NixOS: Run Docker containers as declarative systemd services #55179

benley commented Feb 4, 2019 •

edited

nixos-discourse commented Feb 4, 2019

nlewo left a comment

benley commented Feb 4, 2019

costrouc commented Feb 4, 2019

benley commented Feb 4, 2019

benley commented Feb 4, 2019

benley commented Feb 4, 2019

This comment has been minimized.

expipiplus1 commented Feb 8, 2019

infinisil left a comment

tomberek commented Mar 24, 2019

danbst commented Mar 24, 2019

benley commented Mar 25, 2019

expipiplus1 commented Apr 11, 2019

nyanloutre commented Apr 11, 2019 via email

benley commented Apr 11, 2019

expipiplus1 commented Apr 14, 2019

chrissound commented Feb 29, 2020 •

edited

expipiplus1 commented Mar 9, 2020

fungalcofe commented Jan 17, 2021

NixOS: Run Docker containers as declarative systemd services #55179

NixOS: Run Docker containers as declarative systemd services #55179

Conversation

benley commented Feb 4, 2019 • edited

Motivation for this change

nixos-discourse commented Feb 4, 2019

nlewo left a comment

Choose a reason for hiding this comment

benley commented Feb 4, 2019

costrouc commented Feb 4, 2019

benley commented Feb 4, 2019

benley commented Feb 4, 2019

benley commented Feb 4, 2019

This comment has been minimized.

expipiplus1 commented Feb 8, 2019

infinisil left a comment

Choose a reason for hiding this comment

tomberek commented Mar 24, 2019

danbst commented Mar 24, 2019

benley commented Mar 25, 2019

expipiplus1 commented Apr 11, 2019

nyanloutre commented Apr 11, 2019 via email

benley commented Apr 11, 2019

expipiplus1 commented Apr 14, 2019

chrissound commented Feb 29, 2020 • edited

expipiplus1 commented Mar 9, 2020

fungalcofe commented Jan 17, 2021

benley commented Feb 4, 2019 •

edited

chrissound commented Feb 29, 2020 •

edited