Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker-containers: add missing kubelet dependency for container systemd services #78313

Conversation

DavHau
Copy link
Member

@DavHau DavHau commented Jan 22, 2020

Motivation for this change

Currently docker container systemd services generated by the option config.docker-containers can crash on startup when kubelet is not seeding docker images fast enough.
I found zero documentation on how to preload docker images via nixos config. But from reading #49379 I guess the right way to do it in production is by using services.kubernetes.kubelet.seedDockerImages .
Therefore the systemd services of config.docker-containers should probably depend on kubelet and wait for it to start.

Things done

Extend systemd service in virtualisation/docker-containers.nix to depend also on kubelet.service in case kubelet is activated via config.services.kubernetes.kubelet.enable

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@DavHau DavHau changed the title add missing kubelet dependency for docker container systemd services docker-containers: add missing kubelet dependency for container systemd services Jan 23, 2020
Copy link
Member

@nlewo nlewo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand well, you would like to load Docker image in order to use them with the docker-containers module. There is currently nothing to do this and it seems to me all images used by docker-containers have to be fetched from an Docker registry.

In this context, I'm not in favor of adding a dependency to the kubelet service, just to load Docker images.

Instead, I think you could add a seedDockerImage option in the docker-container module and add some code to load this Docker image in the ExecStartPre script of the container systemd service (something similar to the kubelet module).

If you need to load Docker images for testing purpose, you could use the dockerPreloader module to speed up your tests since it avoids the Docker load at runtime.

@DavHau
Copy link
Member Author

DavHau commented Jan 23, 2020

Good points. I was not aware that loading docker images is so trivial that it can be done in a one liner and has nothing to do with kubelet at all.

Since docker-containers.nix creates an individual systemd service for each container and we might want to load multiple images in one go, i would not do the preloading inside these services but in a separated docker-images-preloader.service and make the other services depend on that one.

So after this we would have:

  • options.docker-containers to autorun containers
  • options.docker-images to preload docker images.

With having this, we could also remove services.kubernetes.kubelet.seedDockerImages and instead make the kubelet service depend on the new docker-images-preloader service. It does the exact same thing anyways and the options would be unnecessarily redundant

What do you think?

@nlewo
Copy link
Member

nlewo commented Jan 24, 2020

Why would you like to load all images in one go?
I think it would be nicer to let each container associated systemd service manage its own image. Suppose you are running several images and you want to update only one image: all containers would be restarted since they all depend on the same systemd service (the one in charge of loading images). However, if each container systemd service is in charge of loading the image it uses, when you update an image, only this container will be impacted.

Also, I think it would be easier to add a execPreStart in this module than creating a new module;)

@nlewo
Copy link
Member

nlewo commented Jan 24, 2020

Take a look at #78366 ;)

@mkaito
Copy link
Contributor

mkaito commented Jan 24, 2020

You can always mess with systemd.services.docker-foo.preStart, which is types.lines and will just append your stuff. However, for this use case, have a look at #78366.

I'm neutral on the changes proposed here, but they seem unnecessary.

@DavHau
Copy link
Member Author

DavHau commented Jan 24, 2020

Amazing! #78366 is exactly what i was looking for. This can be closed. Thanks!

@nlewo nlewo closed this Jan 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants