dockerTools.pullImage: Skopeo pulls images by digest #38371

nlewo · 2018-04-03T09:05:08Z

[EDITED]

Skopeo is used to pull images from a Docker registry (instead of a
Docker daemon inside a VM).

An image reference is specified with its name, its digest. The digest is an immutable image identifier (unlike image name
and tag).

Skopeo can be used to get the Digest of an image, for instance:
skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'

I still have to

update nixpkgs manual

but I would prefer to have some feedback first.

Note: six months ago, I already tryied to use Skopeo to pull images but these patches have been reverted because the Skopeo output format was not compliant with the Docker legacy format (it broke several script/tests). The latest Skoepo version is now compliant with it.
The onTopOfPulledImage docker test has been added to validate this change.

cc @Profpatsch @kuznero @gilligan

Motivation for this change

Remove VM needs and pull with digest. This also fixes #29271.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Profpatsch · 2018-04-03T09:16:43Z

pkgs/build-support/docker/default.nix

+      # If the image digest is specified, the tag is only used at
+      # Docker image loading (to set a tag), ie. it is not used to
+      # pull the image (both digest and tag are not supported by
+      # Skopeo yet).


Does this mean we can’t fetch specific tags with this new implementation? Or does it mean you have to fetch the digest first and use that? What happens if the tag is set but the digest isn’t?

We can still use image tags to pull images.
If the tag is set but the digest is not, the tag is used to pull the image.
But if you specify a digest and a tag, Skopeo ignores the tag to pull the image. In this case, the tag is just used at image creation.

Hm, then it should really be ref (either a tag or a digest) and finalImageName ? ref (what the image is named). That way you don’t have to ignore user input silently.

The tag is only ignored to pull the image if a specified image digest doesn't match to the specified image tag. In this case, we could ensure the digest matches the expected tag by using skopeo inspect before pulling the image.

I would like to avoid to have to encode the attribute type (digest or tag) in the reference string. For the user, it is more convenient to just have to set an attribute named digest or tag instead of having to put a : or a @ in the image reference string.
Moreover, the tag is always required (even if digest is specified) to create the image.

Another way would be to create a pullImage that only use digest and a pullImageByTag function that could take different type of arguments.
I'm not sure if we have to let - forever - the user pull images with a tag since they are not immutable.

nlewo · 2018-04-06T06:49:16Z

@Profpatsch I've tryied to handle both tag and digest ID but I think pulling images by both tag and digest is too confusing. Moreover, it seems more Nix compliant to pull by digest which are immutable.
So, I pushed a new patch set where we can only pull by digest. What do you think about this?

Profpatsch · 2018-04-06T11:23:26Z

I think that’s the way to go, yes.
Now all that’s missing is updating the relevant parts of the dockerTools manual section as far as I can see.

gilligan · 2018-04-06T11:27:01Z

Nice, this PR is an improvement for sure. Thanks for the effort. Only documentation left :-)

Profpatsch · 2018-04-09T16:40:44Z

Docs diff looks fine. LGTM?

nlewo · 2018-04-11T13:14:48Z

@GrahamcOfBorg test docker-tools

GrahamcOfBorg · 2018-04-11T13:15:30Z

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

GrahamcOfBorg · 2018-04-11T13:54:13Z

Failure on x86_64-linux (full log)

Attempted: tests.docker-tools

Partial log (click to expand)

Cooking the image...
Finished.
building path(s) ‘/nix/store/lmvi5jy5m17ziy8hn38wgy9mgbwda5as-closure-info’
/nix/store/aj9rz4k3xvg8q9fzqfnb0q123707zp0n-builder: line 1: .attrs.sh: No such file or directory
builder for ‘/nix/store/9akbgyhsi7mzmq8lhgq0n3qjzlnspqlf-closure-info.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/7m2418fiibkyh1xl5wxla54rpgh29jdy-run-nixos-vm.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/18ak37v1wijghbwlradm8yx7fz73ycc4-nixos-vm.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/lwqxzlhgjai7cwps2l9vmq6r9f644kpi-nixos-test-driver-docker-tools.drv’: 3 dependencies couldn't be built
cannot build derivation ‘/nix/store/ppk5jlmj0gf8n9r6lclk78xv0ihc0w6v-vm-test-run-docker-tools.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/ppk5jlmj0gf8n9r6lclk78xv0ihc0w6v-vm-test-run-docker-tools.drv’ failed

nlewo · 2018-04-13T15:14:48Z

@GrahamcOfBorg test docker-tools

GrahamcOfBorg · 2018-04-13T15:15:19Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.docker-tools

Partial log (click to expand)

while evaluating ‘hydraJob’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:162:14, called from /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/nixos/release.nix:23:16:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:179:13:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:146:13:
while evaluating the attribute ‘buildCommand’ of the derivation ‘vm-test-run-docker-tools’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-test-driver-docker-tools’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-vm’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘text’ of the derivation ‘run-nixos-vm’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/build-support/closure-info.nix:9:1, called from /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/nixos/modules/virtualisation/qemu-vm.nix:105:13:
assertion failed at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/build-support/closure-info.nix:11:1

GrahamcOfBorg · 2018-04-13T15:18:56Z

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

nlewo · 2018-04-14T07:11:25Z

@grahamc it seems like the ofborg builder is using Nix1 (assert builtins.langVersion >= 5; is failing). I'm surprised since I'm pretty sure I already used ofborg to run Nixos 18.03 tests. Do you know what is happening?

nlewo · 2018-04-14T07:15:45Z

Test failure is not related to this PR. I locally ran docker-tools tests which succeeded.

nlewo · 2018-04-20T13:52:11Z

I would have liked more feedback on this... since my last attempt has been reverted!
@Mic92 @puffnfresh @globin WDYT?

puffnfresh

There's also https://github.com/awakesecurity/hocker - how does that compare to Skopeo?

Mic92 · 2018-04-22T22:54:41Z

pkgs/build-support/docker/examples.nix

-    # this hash will need change if the tag is updated at docker hub
-    sha256 = "0nncn9pn5miygan51w34c2p9qssi96jgsaqv44dxxdprc8pg0g83";
+    imageDigest = "sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b";
+    sha256 = "0mqjy3zq2v6rrhizgb9nvhczl87lcfphq9601wcprdika2jz7qh8";


I think we should have a release note for if hashes changes.

see nixpkgs/nixos/doc/manual/release-notes/rl-1809.xml

Mic92 · 2018-04-22T22:56:39Z

nixos/doc/manual/release-notes/rl-1809.xml

@@ -77,6 +77,8 @@ following incompatible changes:</para>
 <itemizedlist>
  <listitem>
    <para>
+      <literal>dockerTools.pullImage</literal> relies on image digest
+      instead of image tag to download the image.


Ok this was already done. It should also mention that hashes will change.

Profpatsch · 2018-04-23T03:09:20Z

@puffnfresh See https://nixos.wiki/wiki/Workgroup:Container for a short synopsis on many relevant tools.

puffnfresh · 2018-04-23T03:18:34Z

@Profpatsch really cool, thank you!

nlewo · 2018-04-23T07:54:33Z

@puffnfresh Skopeo is not related to Nix. It is used by many others project to manipulate container images (Docker, OCI, ...) across container image registries. One advantages of Hocker is that each layer is stored in the Nix store. This means if several images shares the same layer, it only exists one time. This is not currently the case with our implementation of pullImage, but I would like to also provide this with Skopeo (it can pull layer one per one).

Skopeo is used to pull images from a Docker registry (instead of a Docker deamon in a VM). An image reference is specified with its name and its digest which is an immutable image identifier (unlike image name and tag). Skopeo can be used to get the digest of an image, for instance: $ skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'

nlewo · 2018-05-02T19:21:36Z

@Mic92 @Profpatsch Does this PR look good to you?

GrahamcOfBorg added 8.has: documentation 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Apr 3, 2018

Profpatsch reviewed Apr 3, 2018

View reviewed changes

nlewo force-pushed the pr/pull-digest-id branch from f771dc3 to 0c620f5 Compare April 6, 2018 06:43

nlewo changed the title ~~[wip] dockerTools.pullImage: Skopeo pulls images with digest or tag~~ [wip] dockerTools.pullImage: Skopeo pulls images with digest Apr 6, 2018

nlewo changed the title ~~[wip] dockerTools.pullImage: Skopeo pulls images with digest~~ [wip] dockerTools.pullImage: Skopeo pulls images by digest Apr 6, 2018

nlewo force-pushed the pr/pull-digest-id branch from 0c620f5 to bc8ec99 Compare April 6, 2018 06:50

GrahamcOfBorg added 6.topic: nixos 8.has: changelog labels Apr 9, 2018

nlewo changed the title ~~[wip] dockerTools.pullImage: Skopeo pulls images by digest~~ dockerTools.pullImage: Skopeo pulls images by digest Apr 9, 2018

nlewo force-pushed the pr/pull-digest-id branch from 4b25bf8 to 1a69af0 Compare April 13, 2018 15:13

nlewo force-pushed the pr/pull-digest-id branch from 1a69af0 to bf1a205 Compare April 20, 2018 08:40

puffnfresh approved these changes Apr 22, 2018

View reviewed changes

Mic92 reviewed Apr 22, 2018

View reviewed changes

nlewo force-pushed the pr/pull-digest-id branch from bf1a205 to 3a58380 Compare April 23, 2018 08:05

nlewo added 2 commits May 2, 2018 18:05

dockerTools.pullImage: documentation and release note

66b4812

nlewo force-pushed the pr/pull-digest-id branch from 3a58380 to 66b4812 Compare May 2, 2018 16:58

globin merged commit d35dcb1 into NixOS:master May 2, 2018

kliu128 mentioned this pull request May 20, 2018

kubernetes: update pullImage syntax #40813

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dockerTools.pullImage: Skopeo pulls images by digest #38371

dockerTools.pullImage: Skopeo pulls images by digest #38371

nlewo commented Apr 3, 2018 •

edited

Profpatsch Apr 3, 2018

nlewo Apr 3, 2018

Profpatsch Apr 3, 2018

nlewo Apr 3, 2018

nlewo commented Apr 6, 2018

Profpatsch commented Apr 6, 2018

gilligan commented Apr 6, 2018

Profpatsch commented Apr 9, 2018

nlewo commented Apr 11, 2018

GrahamcOfBorg commented Apr 11, 2018

GrahamcOfBorg commented Apr 11, 2018

nlewo commented Apr 13, 2018

GrahamcOfBorg commented Apr 13, 2018

GrahamcOfBorg commented Apr 13, 2018

nlewo commented Apr 14, 2018

nlewo commented Apr 14, 2018

nlewo commented Apr 20, 2018

puffnfresh left a comment

Mic92 Apr 22, 2018

Mic92 Apr 22, 2018

Mic92 Apr 22, 2018

nlewo Apr 23, 2018

Profpatsch commented Apr 23, 2018

puffnfresh commented Apr 23, 2018

nlewo commented Apr 23, 2018

nlewo commented May 2, 2018

dockerTools.pullImage: Skopeo pulls images by digest #38371

dockerTools.pullImage: Skopeo pulls images by digest #38371

Conversation

nlewo commented Apr 3, 2018 • edited

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nlewo commented Apr 6, 2018

Profpatsch commented Apr 6, 2018

gilligan commented Apr 6, 2018

Profpatsch commented Apr 9, 2018

nlewo commented Apr 11, 2018

GrahamcOfBorg commented Apr 11, 2018

GrahamcOfBorg commented Apr 11, 2018

nlewo commented Apr 13, 2018

GrahamcOfBorg commented Apr 13, 2018

GrahamcOfBorg commented Apr 13, 2018

nlewo commented Apr 14, 2018

nlewo commented Apr 14, 2018

nlewo commented Apr 20, 2018

puffnfresh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Profpatsch commented Apr 23, 2018

puffnfresh commented Apr 23, 2018

nlewo commented Apr 23, 2018

nlewo commented May 2, 2018

nlewo commented Apr 3, 2018 •

edited