Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dockertools buildImage: support new-style image specs #51528

Merged
merged 1 commit into from Dec 7, 2018
Merged

dockertools buildImage: support new-style image specs #51528

merged 1 commit into from Dec 7, 2018

Conversation

grahamc
Copy link
Member

@grahamc grahamc commented Dec 4, 2018

Docker images used to be, essentially, a linked list of layers. Each
layer would have a tarball and a json document pointing to its parent,
and the image pointed to the top layer:

imageA  ----> layerA
                |
                v
              layerB
                |
                v
              layerC

The current image spec changed this format to where the Image defined
the order and set of layers:

imageA  ---> layerA
        |--> layerB
        `--> layerC

For backwards compatibility, docker produces images which follow both
specs: layers point to parents, and images also point to the entire
list:

imageA  ---> layerA
        |      |
        |      v
        |--> layerB
        |      |
        |      v
        `--> layerC

This is nice for tooling which supported the older version and never
updated to support the newer format.

Our buildImage code only supported the old version, so in order for
buildImage to properly generate an image based on another image
with fromImage, the parent image's layers must fully support the old
mechanism.

This is not a problem in general, but is a problem with
buildLayeredImage.

buildLayeredImage creates images with newer image spec, because
individual store paths don't have a guaranteed parent layer. Including
a specific parent ID in the layer's json makes the output less likely
to cache hit when published or pulled.

This means until now, buildLayeredImage could not be the input to
buildImage.

The changes in this PR change buildImage to only use the layer's
manifest when locating parent IDs. This does break buildImage on
extremely old Docker images, though I do wonder how many of these
exist.

This work has been sponsored by Target.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@grahamc grahamc requested review from globin and nlewo December 4, 2018 21:41
@grahamc
Copy link
Member Author

grahamc commented Dec 4, 2018

This is a "RFC" stage PR, I think there is a lot of opportunity for improvement here.

nlewo
nlewo approved these changes Dec 5, 2018
View reviewed changes
Copy link
Member

@nlewo nlewo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

FYI, even for "layered" images, I think (not entirely sure) you could generate the old style format. I think in your description, your are forgotten an indirection which is the layer directory: a identical layer could exists in different "layer directory", each of these "layer directories" having its own local configuration. This layer configuration would be created at image creation time, when layers order is known.
But this is not really important since this format is deprecated:/

@nlewo
Copy link
Member

nlewo commented Dec 5, 2018

@GrahamcOfBorg test docker-tools

@grahamc
Copy link
Member Author

grahamc commented Dec 5, 2018 •
edited

@GrahamcOfBorg build nixosTests.docker-tools

@grahamc
Copy link
Member Author

grahamc commented Dec 5, 2018 •
edited

I wonder what sort of fundamental tooling we'd need for these scripts. Maybe we have a few functions here, hiding behind ugly bash:

  • given a directory + a bit of metadat, produce a layer
  • take a list of layers to produce an image
  • iterate over an image's layers

I took a look at umoci (https://umo.ci) and it seems interesting, though can't produce an individual layer. I'm afraid our unique approach to packages makes us require unique tooling ... I also looked at buildah https://github.com/containers/buildah but it seems to require capabilities which aren't available in a build sandbox.

@grahamc
dockerTools.buildImage: support using a layered image in fromImage
c88337c 
Docker images used to be, essentially, a linked list of layers. Each
layer would have a tarball and a json document pointing to its parent,
and the image pointed to the top layer:

    imageA  ----> layerA
                    |
                    v
                  layerB
                    |
                    v
                  layerC

The current image spec changed this format to where the Image defined
the order and set of layers:

    imageA  ---> layerA
            |--> layerB
            `--> layerC

For backwards compatibility, docker produces images which follow both
specs: layers point to parents, and images also point to the entire
list:

    imageA  ---> layerA
            |      |
            |      v
            |--> layerB
            |      |
            |      v
            `--> layerC

This is nice for tooling which supported the older version and never
updated to support the newer format.

Our `buildImage` code only supported the old version, so in order for
`buildImage` to properly generate an image based on another image
with `fromImage`, the parent image's layers must fully support the old
mechanism.

This is not a problem in general, but is a problem with
`buildLayeredImage`.

`buildLayeredImage` creates images with newer image spec, because
individual store paths don't have a guaranteed parent layer. Including
a specific parent ID in the layer's json makes the output less likely
to cache hit when published or pulled.

This means until now, `buildLayeredImage` could not be the input to
`buildImage`.

The changes in this PR change `buildImage` to only use the layer's
manifest when locating parent IDs. This does break buildImage on
extremely old Docker images, though I do wonder how many of these
exist.

This work has been sponsored by Target.
@grahamc
Copy link
Member Author

grahamc commented Dec 5, 2018

My force-push is identical to the version reviewed by @nlewo already, just squashed and with a nicer commit message.

@grahamc grahamc mentioned this pull request Dec 5, 2018
Closed
10 tasks
@nlewo
Copy link
Member

nlewo commented Dec 7, 2018

@grahamc hmm, yes, maybe we need our own tooling :(

We would also need a function that pushes an image on top of a parent image. In our current tooling, this part is quite complicated (to ignore already existing files). I wonder how this is required thanks to buildLayeredImage: all files are in the store and only links are created, so by construction, we avoid (at least reduce) the duplicated files problem.

Regarding the tooling, I wonder how a combination of umoci + skopeo could do the job:

  • createLayer :: Dir -> Layer
    umoci creates a single layered image and skopeo extracts its layer (skopeo copy dir://).
  • makeImage :: [Layer] -> Image
    we create a directory of layers and use skopeo copy dir://$dir docker-archive://$out to pack the image

But maybe it would be more complicated than creating our own tools :)

Another point is regarding the way we pack the image. With recursiveNix, I think we would like to dynamically create the image in order to not have to duplicate layers in the Nix store. In this case, the image format in the Nix store could be the skopeo dir where each layer would be a link to the layer in the Nix store.

@nlewo
Copy link
Member

nlewo commented Dec 7, 2018

Thx!

@nlewo nlewo merged commit f7e67be into NixOS:master Dec 7, 2018
@grahamc grahamc deleted the buildImage-on-layered-image branch December 7, 2018 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@puffnfresh puffnfresh puffnfresh approved these changes

@nlewo nlewo nlewo approved these changes

@globin globin Awaiting requested review from globin

Assignees
No one assigned
Labels
6.topic: nixos 8.has: documentation 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@grahamc @nlewo @puffnfresh @GrahamcOfBorg