dockerTools.buildImage: add /nix/store with correct permissions #38837

eonpatapon · 2018-04-12T11:09:44Z

This adds /nix/store to the layer files with correct permissions so that non-root users can use the store
inside the container.

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Fixes #38835.

puffnfresh · 2018-04-12T11:40:22Z

pkgs/build-support/docker/default.nix

+        # image /nix/store has read permissions for non-root users.
+        mkdir -p nix/store
+        chmod -R 555 nix
+        tar -rpf temp/layer.tar --mtime="@$SOURCE_DATE_EPOCH" --owner=0 --group=0 ./nix


This introduces code about the Nix store into buildImage, I think it's been agnostic about the content until this. I don't think everything which uses buildImage will necessarily use /nix. I'd prefer to keep references outside of this function.

I think it was the case before this patch: ce838e5

However I understand your point but looking at the code I'm not sure where to put this elsewhere.

Maybe we could check if some file in newFiles is in /nix/store/. If yes we add /nix/store to the layer.

Yeah I like that better. Not great, but I think it's better.

I think you can test if layerClosure is non empty to create /nix and /nix/store. Moreover, we should also test if /nix and /nix/store are not in baseFiles (not already existing in parent layers).

LnL7 · 2018-04-12T22:29:05Z

I think that I ran into this same problem with LnL7/nix-docker#14 (comment), but didn't really look into it afterwards. I'll do some testing since I've still seen problems on some systems after the workaround.

eonpatapon · 2018-04-13T08:48:59Z

With the latest patch I'm checking that there is more than one item in $layerClosure. There is always one item which is the layer definition. More if there is some /nix/store paths to be added.

nix/store path is added to $layerFiles so that the code checks later that nix/store is not already present in a parent layer.

nlewo · 2018-04-14T07:12:18Z

@GrahamcOfBorg test docker-tools

GrahamcOfBorg · 2018-04-14T07:12:39Z

Success on aarch64-linux

Attempted: tests.docker-tools

No partial log is available.

GrahamcOfBorg · 2018-04-14T07:12:53Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.docker-tools

Partial log (click to expand)

while evaluating ‘hydraJob’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:162:14, called from /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/nixos/release.nix:23:16:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:179:13:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/lib/customisation.nix:146:13:
while evaluating the attribute ‘buildCommand’ of the derivation ‘vm-test-run-docker-tools’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-test-driver-docker-tools’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-vm’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘text’ of the derivation ‘run-nixos-vm’ at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/build-support/closure-info.nix:9:1, called from /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/nixos/modules/virtualisation/qemu-vm.nix:105:13:
assertion failed at /home/ofborg/ofborg2/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c2/pkgs/build-support/closure-info.nix:11:1

nlewo · 2018-04-14T07:14:25Z

Tests failure is not related to this PR.

LnL7 · 2018-04-14T14:18:58Z

@GrahamcOfBorg test docker-tools docker-tools-overlay

GrahamcOfBorg · 2018-04-14T14:19:16Z

Success on aarch64-linux

Attempted: tests.docker-tools, tests.docker-tools-overlay

No partial log is available.

GrahamcOfBorg · 2018-04-14T14:19:32Z

No attempt on x86_64-linux (full log)

The following builds were skipped because they don't evaluate on x86_64-linux: tests.docker-tools, tests.docker-tools-overlay

Partial log (click to expand)

while evaluating ‘hydraJob’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/lib/customisation.nix:162:14, called from /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/nixos/release.nix:23:16:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/lib/customisation.nix:179:13:
while evaluating the attribute ‘drvPath’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/lib/customisation.nix:146:13:
while evaluating the attribute ‘buildCommand’ of the derivation ‘vm-test-run-docker-tools-overlay’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-test-driver-docker-tools-overlay’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-vm’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘text’ of the derivation ‘run-nixos-vm’ at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/build-support/closure-info.nix:9:1, called from /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/nixos/modules/virtualisation/qemu-vm.nix:105:13:
assertion failed at /home/ofborg/ofborg1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/builder-7c6f434c1/pkgs/build-support/closure-info.nix:11:1

LnL7

One of the tests broke (buildImageWithNixDb).

$ nix-build nixos/release.nix -A tests.docker-tools
docker: must succeed: docker load --input='/nix/store/i8m0p37qm8k492n40z9b3fhr7mwxsphs-docker-image-nix.tar.gz'
docker# Error processing tar file(duplicates of file paths not supported):
docker: exit status 1

Profpatsch · 2018-04-16T01:33:45Z

I ran into this as well:

> docker run sangha-api:deployment
/bin/sangha: error while loading shared libraries: libpthread.so.0: cannot open shared object file: Permission denied

Applying this patch seems to fix the problem. We should make sure the tests pass again and add a regression test, though.

Profpatsch · 2018-04-16T02:55:02Z

Ah, it did fix it for one image, the other has the duplicate error:

43155a6a7d67: Loading layer [==================================================>]  93.69MB/93.69MB
35312af7c514: Loading layer [========================================>          ]  32.77kB/40.96kB
ApplyLayer duplicates of file paths not supported stdout: {"layerSize":280}
 stderr:

LnL7 · 2018-04-16T06:14:04Z

@Profpatsch I already added a test in #38933. I looks like overlay2 is basicallly the only storage driver that doesn't suffer from this issue.

eonpatapon · 2018-04-16T08:14:16Z

@GrahamcOfBorg test docker-tools docker-tools-overlay

Fixes NixOS#38835.

nlewo · 2018-04-16T08:30:34Z

@GrahamcOfBorg test docker-tools docker-tools-overlay
@eonpatapon you need to be whitelisted to use ofBorg.

GrahamcOfBorg · 2018-04-16T08:30:50Z

Success on aarch64-linux

Attempted: tests.docker-tools, tests.docker-tools-overlay

No partial log is available.

GrahamcOfBorg · 2018-04-16T08:32:19Z

Success on aarch64-linux

Attempted: tests.docker-tools, tests.docker-tools-overlay

No partial log is available.

nlewo · 2018-04-16T13:51:59Z

@Profpatsch @LnL7 I successfully executed nix-build nixos/release.nix -A tests.docker-tools -A tests.docker-tools-overlay. Is it good for you?

LnL7 · 2018-04-16T18:31:19Z

Backported to 18.03 in f3353ff.

GrahamcOfBorg · 2018-04-17T02:30:25Z

Success on x86_64-linux (full log)

Attempted: tests.docker-tools, tests.docker-tools-overlay

Partial log (click to expand)

syncing
docker: running command: sync
docker: exit status 0
test script finished in 193.24s
cleaning up
killing docker (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/4cn0rvkmw0jp4k180xbpbd06x793w5y9-vm-test-run-docker-tools
/nix/store/fq45vnlznbdfagw41ba61vmnfry8rks1-vm-test-run-docker-tools-overlay

GrahamcOfBorg · 2018-04-17T02:33:44Z

Success on x86_64-linux (full log)

Attempted: tests.docker-tools, tests.docker-tools-overlay

Partial log (click to expand)

syncing
docker: running command: sync
docker: exit status 0
test script finished in 360.90s
cleaning up
killing docker (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/4cn0rvkmw0jp4k180xbpbd06x793w5y9-vm-test-run-docker-tools
/nix/store/fq45vnlznbdfagw41ba61vmnfry8rks1-vm-test-run-docker-tools-overlay

GrahamcOfBorg added 8.has: documentation 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Apr 12, 2018

puffnfresh suggested changes Apr 12, 2018

View reviewed changes

eonpatapon force-pushed the fix-38835 branch 2 times, most recently from 95bb159 to 590caca Compare April 13, 2018 08:34

nlewo mentioned this pull request Apr 14, 2018

dockerTools.buildImage: do not add /nix/store in the tar stream #34939

Merged

8 tasks

nlewo approved these changes Apr 14, 2018

View reviewed changes

LnL7 suggested changes Apr 14, 2018

View reviewed changes

eonpatapon force-pushed the fix-38835 branch from 590caca to be3005f Compare April 16, 2018 08:13

dockerTools.buildImage: add /nix/store with correct permissions

9751771

Fixes NixOS#38835.

eonpatapon force-pushed the fix-38835 branch from be3005f to 9751771 Compare April 16, 2018 08:19

LnL7 approved these changes Apr 16, 2018

View reviewed changes

LnL7 merged commit c4dea09 into NixOS:master Apr 16, 2018

LnL7 added the 8.has: port to stable A PR already has a backport to the stable release. label Apr 16, 2018

eonpatapon deleted the fix-38835 branch April 17, 2018 07:50

nlewo mentioned this pull request Mar 18, 2022

images are created with 0600 /nix and /nix/store nlewo/nix2container#12

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dockerTools.buildImage: add /nix/store with correct permissions #38837

dockerTools.buildImage: add /nix/store with correct permissions #38837

eonpatapon commented Apr 12, 2018

puffnfresh Apr 12, 2018

eonpatapon Apr 12, 2018

puffnfresh Apr 13, 2018

nlewo Apr 13, 2018

LnL7 commented Apr 12, 2018

eonpatapon commented Apr 13, 2018

nlewo commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

nlewo commented Apr 14, 2018

LnL7 commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

LnL7 left a comment

Profpatsch commented Apr 16, 2018

Profpatsch commented Apr 16, 2018

LnL7 commented Apr 16, 2018

eonpatapon commented Apr 16, 2018

nlewo commented Apr 16, 2018 •

edited

GrahamcOfBorg commented Apr 16, 2018

GrahamcOfBorg commented Apr 16, 2018

nlewo commented Apr 16, 2018

LnL7 commented Apr 16, 2018

GrahamcOfBorg commented Apr 17, 2018

GrahamcOfBorg commented Apr 17, 2018

dockerTools.buildImage: add /nix/store with correct permissions #38837

dockerTools.buildImage: add /nix/store with correct permissions #38837

Conversation

eonpatapon commented Apr 12, 2018

puffnfresh Apr 12, 2018

Choose a reason for hiding this comment

eonpatapon Apr 12, 2018

Choose a reason for hiding this comment

puffnfresh Apr 13, 2018

Choose a reason for hiding this comment

nlewo Apr 13, 2018

Choose a reason for hiding this comment

LnL7 commented Apr 12, 2018

eonpatapon commented Apr 13, 2018

nlewo commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

nlewo commented Apr 14, 2018

LnL7 commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

GrahamcOfBorg commented Apr 14, 2018

LnL7 left a comment

Choose a reason for hiding this comment

Profpatsch commented Apr 16, 2018

Profpatsch commented Apr 16, 2018

LnL7 commented Apr 16, 2018

eonpatapon commented Apr 16, 2018

nlewo commented Apr 16, 2018 • edited

GrahamcOfBorg commented Apr 16, 2018

GrahamcOfBorg commented Apr 16, 2018

nlewo commented Apr 16, 2018

LnL7 commented Apr 16, 2018

GrahamcOfBorg commented Apr 17, 2018

GrahamcOfBorg commented Apr 17, 2018

nlewo commented Apr 16, 2018 •

edited