Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dockerTools.buildImageWithNixDb: simplifications and switch to closure info #49414

Merged
merged 1 commit into from Nov 12, 2018
Merged

dockerTools.buildImageWithNixDb: simplifications and switch to closure info #49414

merged 1 commit into from Nov 12, 2018

Conversation

nlewo
Copy link
Member

@nlewo nlewo commented Oct 29, 2018

Since Nix 2 is now the stable Nix version, we can use closureInfo
which simplifies the Nix database initialisation (size and hash are
included in the "dump").

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

@nlewo
Copy link
Member Author

nlewo commented Oct 29, 2018

@GrahamcOfBorg test docker-tools

@GrahamcOfBorg
Copy link

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: tests.docker-tools

Partial log (click to expand)

syncing
docker: running command: sync
docker# [   90.731468] dhcpcd[764]: veth628c80a: removing interface
docker: exit status 0
test script finished in 92.21s
cleaning up
killing docker (pid 597)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/8pwfayhwizmspxiiqiq55r3ww37a72ry-vm-test-run-docker-tools

@nlewo nlewo mentioned this pull request Oct 29, 2018
Closed
9 tasks
@FRidh
Copy link
Member

FRidh commented Oct 29, 2018

If I am correct we already rely on Nix 2 features so I don't see any reason why not.

@FRidh FRidh requested a review from grahamc October 29, 2018 18:48
dingxiangfei2009
dingxiangfei2009 reviewed Oct 30, 2018
View reviewed changes
pkgs/build-support/docker/default.nix Outdated
echo "Copying everything to /nix/store (will take a while)..."
cp -prd $storePaths nix/store/
${nix}/bin/nix-store --verify --check-contents
${nix}/bin/nix-store --load-db < ${closureInfo {rootPaths = contents;}}/registration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

contents can be a list of derivations. It can also be one single derivation that defines the docker image layer.
See this:

nixpkgs/pkgs/build-support/docker/default.nix

Line 268 in 2f9fc89

let contentsList = if builtins.isList contents then contents else [ contents ];

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would have been nice to have typed Nix here :)
I fixed this. Thanks!

dingxiangfei2009
dingxiangfei2009 reviewed Oct 30, 2018
View reviewed changes
pkgs/build-support/docker/default.nix
# https://github.com/NixOS/nix/commit/c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a
storePaths=$(cat ${nixRegistration contents}/storePaths)
echo "Copying everything to /nix/store (will take a while)..."
cp -prd $storePaths nix/store/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the closure?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closure is copied by the buildImage function. This copy was needed to do the --check-contents at the layer level. This is no longer required (with Nix 2.0) to initialize correctly the database.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see.

@nlewo
Copy link
Member Author

nlewo commented Oct 30, 2018

@FRidh yes, it seems now we can. In fact, I already submitted this PR 6 months back but it was too early to rely on Nix 2 features.

@joachifm joachifm mentioned this pull request Nov 11, 2018
Merged
9 tasks
@nlewo
dockerTools.buildImageWithNixDb: simplifications and switch to closur…
c12f756 
…eInfo

Since Nix 2 is now the stable Nix version, we can use closureInfo
which simplifies the Nix database initialisation (size and hash are
included in the "dump").
@nlewo nlewo merged commit 9e4456c into NixOS:master Nov 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dingxiangfei2009 dingxiangfei2009 dingxiangfei2009 left review comments

@grahamc grahamc Awaiting requested review from grahamc

Assignees
No one assigned
Labels
8.has: documentation 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@nlewo @GrahamcOfBorg @FRidh @dingxiangfei2009