Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockerTools.buildImage: support impure dates #47005

Merged

Conversation

graham-at-target
Copy link
Contributor

@graham-at-target graham-at-target commented Sep 20, 2018

Because dates are an impurity, by default buildImage will use a static
date of one second past the UNIX Epoch. This can be a bit frustrating
when listing docker images in the CLI:

$ docker image list
REPOSITORY   TAG      IMAGE ID       CREATED        SIZE
hello        latest   08c791c7846e   48 years ago   25.2MB

If you want to trade the purity for a better user experience, you can
set created to now.

pkgs.dockerTools.buildImage {
  name = "hello";
  tag = "latest";
  created = "now";
  contents = pkgs.hello;

  config.Cmd = [ "/bin/hello" ];
}

and now the Docker CLI will display a reasonable date and sort the
images as expected:

$ docker image list
REPOSITORY   TAG      IMAGE ID       CREATED              SIZE
hello        latest   de2bf4786de6   About a minute ago   25.2MB
Motivation for this change

the date issue is a real issue. I often am asking the question of "is this the image I just built? what's the timestamp on it?"

-- https://twitter.com/StewOConnor/status/1042671588447936512

cc @stew

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

Because dates are an impurity, by default buildImage will use a static
date of one second past the UNIX Epoch. This can be a bit frustrating
when listing docker images in the CLI:

    $ docker image list
    REPOSITORY   TAG      IMAGE ID       CREATED        SIZE
    hello        latest   08c791c7846e   48 years ago   25.2MB

If you want to trade the purity for a better user experience, you can
set created to now.

    pkgs.dockerTools.buildImage {
      name = "hello";
      tag = "latest";
      created = "now";
      contents = pkgs.hello;

      config.Cmd = [ "/bin/hello" ];
    }

and now the Docker CLI will display a reasonable date and sort the
images as expected:

    $ docker image list
    REPOSITORY   TAG      IMAGE ID       CREATED              SIZE
    hello        latest   de2bf4786de6   About a minute ago   25.2MB
@stew
Copy link

stew commented Sep 20, 2018

amazing nerdsnipe! Thank you so much, @graham-at-target !!

@grahamc grahamc requested review from nlewo and aszlig and removed request for nlewo September 20, 2018 16:00
''
jq ".created = \"$(TZ=utc date --iso-8601="seconds")\"" ${pure} > $out
'';
in if created == "now" then impure else pure;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was some question about why not just use builtins.currentTime.

builtins.currentTime will never be a cache hit, whereas this method will cache while still achieving the intended effect.

@lheckemann
Copy link
Member

I like it!

@globin globin merged commit a32d7e0 into NixOS:master Sep 20, 2018
@graham-at-target graham-at-target deleted the dockertools-image-impure-dates branch September 20, 2018 16:48
@edolstra
Copy link
Member

This seems like a bad idea because it breaks binary reproducibility.

@grahamc
Copy link
Member

grahamc commented Sep 20, 2018

@edolstra do you think it is bad to have the ability to break binary reproducibility? This change keeps the reproducibility by default, and but does allow users to make a trade-off.

@edolstra
Copy link
Member

It doesn't just provide the ability, but advertises it in the manual as "providing better user experience" (so nudges the user in that direction). BTW, apart from the minor cosmetic aspect of improving the "CREATED" column, how does this improve user experience?

@grahamc
Copy link
Member

grahamc commented Sep 20, 2018

Well it does provide a better user experience if you're looking at the output of docker image list. The lines are ordered by when they're created, and having your newly loaded image appear by the oldest make them hard to find. I could replace the vague "improved user experience" nudge with "ordered docker image list output"

@grahamc
Copy link
Member

grahamc commented Sep 20, 2018

what if the docs said:

You can break binary reproducibility but have a sorted, meaningful CREATED column by setting created to now.

edit: changed and to but

@LnL7
Copy link
Member

LnL7 commented Sep 20, 2018

This isn't enabled by default right? I don't see a big problem with providing an option that has a small (cosmetic) impurity.

@grahamc
Copy link
Member

grahamc commented Sep 20, 2018

My impression is the biggest concern is in the too-positive nudge for users to turn it on without understanding the cost.

@edolstra
Copy link
Member

@grahamc Yeah something like that sounds good.

@grahamc
Copy link
Member

grahamc commented Sep 21, 2018 via email

@Profpatsch
Copy link
Member

Sounds good! As long as we keep binary reproducibility by default and set up big warning signs, I don’t see why that shouldn’t be an option.

@nlewo
Copy link
Member

nlewo commented Sep 21, 2018

That's nice.
Also note the Digest (Docker content-addressable identifier) of the image is impacted (such as binary reproducibility).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet