Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dockerTools.streamLayeredImage: Store the customisation layer as a tarball #95409

Merged
merged 1 commit into from Sep 14, 2020
Merged

dockerTools.streamLayeredImage: Store the customisation layer as a tarball #95409

merged 1 commit into from Sep 14, 2020

Conversation

utdemir
Copy link
Member

@utdemir utdemir commented Aug 14, 2020

Motivation for this change

This fixes as issue described here, where permissions set by 'extraCommands' were ignored. We fix it by storing the customisation layer as a tarball rather than a regular directory. Another minor added advantage is that we can precalculate the checksum of the customization layer when building the store path.

The existing NixOS tests pass (nix-build -A nixosTests.docker-tools), with one modification where we were asserting that a derivation in 'contents' do not end up in /nix/store of the image if it doesn't have any dependencies; however I do find that behaviour counter-intuitive; since that derivation is declared in 'contents', I'd definitely expect it to end up in the images /nix/store, whether it has any dependencies or not. It looks like that test is introduced here; @roberth can you tell me if I am understanding it correctly?

cc @thatsmydoing @purcell

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@roberth
Copy link
Member

roberth commented Aug 14, 2020

I've added the test for arion to make sure its base image is built correctly, which seems to be still the case with this PR.
Whether the existing behavior is counter-intuitive, I don't know. contents in buildImage (not buildLayeredImage, I know) has the following doc:

contents is a derivation that will be copied in the new layer of the resulting image. This can be similarly seen as ADD contents/ / in a Dockerfile. By default it's null.

It would be nice to be able to combine non-symlink files by listing derivations in contents. Those paths would have to be in the store in case of self-referencing outputs, but otherwise it's a waste of space.

For completeness I'll note here that you only need to use contents for adding to the root filesystem. To add to the store, you can just reference them config.Cmd.

roberth
roberth requested changes Aug 14, 2020
View reviewed changes
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These names seem to be inaccurate.

pkgs/build-support/docker/default.nix Outdated Show resolved Hide resolved
@adrian-gierakowski adrian-gierakowski mentioned this pull request Aug 15, 2020
Closed
10 tasks
roberth
roberth reviewed Aug 26, 2020
View reviewed changes
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for the no-store-paths test case, the most useful property still holds. The others are micro-optimizations. It seems like the test case now has a useless assertions about custom-true though.
Otherwise looks good afaict.

@utdemir
dockerTools.streamLayeredImage: Store the customisation layer as a ta…
ae82f81 
…rball

This fixes as issue described here[1], where permissions set by 'extraCommands'
were ignored by Nix.

[1] NixOS#91084 (comment)
@utdemir
Copy link
Member Author

utdemir commented Sep 4, 2020

Good catch @roberth . I removed those assertions. Sorry about the late reply.

@GrahamcOfBorg test docker-tools

@ofborg ofborg bot added the 6.topic: nixos label Sep 4, 2020
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/223

@AndersonTorres AndersonTorres merged commit a5931fa into NixOS:master Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@roberth roberth roberth approved these changes

Assignees
No one assigned
Labels
6.topic: nixos 8.has: documentation 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@utdemir @roberth @nixos-discourse @AndersonTorres