Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockerTools.buildImage: add option to use nix output hash as tag #42781

Merged
merged 1 commit into from Jul 6, 2018

Conversation

lo1tuma
Copy link
Member

@lo1tuma lo1tuma commented Jun 29, 2018

Motivation for this change

I want the tag of my docker image to only change when its content gets changed. Nix already tracks all the dependencies of the image and represents that in the nix-store path. I would like to re-use that value for the docker image tag.

When setting useOutputHashAsTag to true the given tag will be ignored and the extracted hash of $out will be used instead.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@nlewo
Copy link
Member

nlewo commented Jun 30, 2018

I think this could become the default value of tag. We could set tag ? null, if tag is null then the tag is the output hash otherwise, the user can explicitly set a tag value, such as latest. This also reduces the number of arguments and simplifies documentation.

@lo1tuma
Copy link
Member Author

lo1tuma commented Jun 30, 2018

@nlewo I was thinking about the same, but this would be a breaking change for everyone who currently relies on the default value "latest".

@lo1tuma
Copy link
Member Author

lo1tuma commented Jul 4, 2018

If a breaking API change is not a problem then I’m happy to implement the suggestions from @nlewo.

@nlewo
Copy link
Member

nlewo commented Jul 5, 2018

I don't think a API breaking change is a problem if it is an improvement and if we mention it in the release changelog.

@Profpatsch @zimbatm WDYT about the default tag value?

Copy link
Member

@zimbatm zimbatm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 for the idea.

I don't mind if we break backward-compatibility so it could happen if tag is null. That also makes it clearer what happens if tag is specified, then it will override the default.

@@ -477,6 +479,13 @@ rec {
passthru.buildArgs = args;
passthru.layer = layer;
} ''
${lib.optionalString useOutputHashAsTag ''
outName="$(basename "$out")"
outHash=$(echo "$outName" | cut -c-32)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think cut -d - -f 1 would be more resistant to change

@Profpatsch
Copy link
Member

Referring to the output hash by string splitting in bash seems kind of wrong. I could have sworn there was a builtin or library function that already does that, but I couldn’t find anything.

I don't mind if we break backward-compatibility so it could happen if tag is null

I’d prefer that, combined with good function docs.

@zimbatm
Copy link
Member

zimbatm commented Jul 5, 2018

I could have sworn there was a builtin or library function that already does that, but I couldn’t find anything.

I built a pure nix function to do that in the past but it wouldn't work in that case as it depends on the derivation output to calculate the hash.

@lo1tuma
Copy link
Member Author

lo1tuma commented Jul 6, 2018

Thanks for all the feedback.
I’ve implemented the proposed changed and adapted the documentation and release notes.

A nix builtin function would be nice indeed. I’ve experimented a little bit with placeholder "out", but that doesn’t work because I can’t manipulate its return value by passing it to other nix functions because placeholder only returns a magic string which will be substituted with the real value later at build-time.

@Profpatsch
Copy link
Member

I think it’s the best we can do at the moment, and it shouldn’t be a problem.

@Profpatsch Profpatsch merged commit 39e678e into NixOS:master Jul 6, 2018
@Profpatsch
Copy link
Member

Thanks, merged.

@nlewo
Copy link
Member

nlewo commented Jul 6, 2018

hmm, maybe tests are failing because they are expecting the tag latest

@GrahamcOfBorg test docker-tools

@GrahamcOfBorg
Copy link

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

1 similar comment
@GrahamcOfBorg
Copy link

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: tests.docker-tools

Partial log (click to expand)

docker: exit status 125
docker: output:
error: command `docker run --rm bash bash --version' did not succeed (exit code 125)
command `docker run --rm bash bash --version' did not succeed (exit code 125)
cleaning up
killing docker (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/8bbqsj6w1wsznx8a59h4vyhca1qdizal-vm-test-run-docker-tools.drv' failed with exit code 255
error: build of '/nix/store/8bbqsj6w1wsznx8a59h4vyhca1qdizal-vm-test-run-docker-tools.drv' failed

@lo1tuma lo1tuma deleted the dockertools-tag branch July 6, 2018 14:01
@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: tests.docker-tools

Partial log (click to expand)

docker: exit status 125
docker: output:
error: command `docker run --rm bash bash --version' did not succeed (exit code 125)
command `docker run --rm bash bash --version' did not succeed (exit code 125)
cleaning up
killing docker (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/053v805qmah6p6027awrc4yg0n489dzl-vm-test-run-docker-tools.drv' failed with exit code 255
error: build of '/nix/store/053v805qmah6p6027awrc4yg0n489dzl-vm-test-run-docker-tools.drv' failed

@nlewo
Copy link
Member

nlewo commented Jul 6, 2018

Fixed by #43119

@Profpatsch
Copy link
Member

Ah, I missed querying ofborg. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants