Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockerTools: use skopeo on the right platform #63865

Merged
merged 1 commit into from Jul 31, 2019

Conversation

dingxiangfei2009
Copy link
Contributor

Motivation for this change

When building a Docker image using any one of the dockerTools in pkgsCross, a skopeo on the host platform is chosen and usually this leads to unrecognised Exec format. This PR fixes this issue by assigning the right build packages on the build platform.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@arianvp
Copy link
Member

arianvp commented Jun 28, 2019

Lets just get rid of the pkgs argument altogether. To make sure we're not making more mistakes like this. Because I assume the ./examples stuff doesn't work either with cross compilation now; does it?

@dingxiangfei2009
Copy link
Contributor Author

@arianvp Good point. I am going to try out, and maybe also fix, the examples, too.

@dingxiangfei2009
Copy link
Contributor Author

@arianvp examples are still solid.

@arianvp
Copy link
Member

arianvp commented Jun 30, 2019

Great. LGTM!

@arianvp
Copy link
Member

arianvp commented Jun 30, 2019

@GrahamcOfBorg test docker-tools

@arianvp
Copy link
Member

arianvp commented Jun 30, 2019

@GrahamcOfBorg test docker-tools

There seemed to have been some kind of transient failure

@arianvp
Copy link
Member

arianvp commented Jun 30, 2019

Alas, the error doesn't seem to be transient on x86_64 https://logs.nix.ci/?key=nixos/nixpkgs.63865&attempt_id=91cb4421-6100-4582-9c07-25d92ee2f597 Not sure what is going on here.

@dingxiangfei2009
Copy link
Contributor Author

@arianvp I notice that on master (aka. f3282c8#diff-f5fad34a5e280fe300df690ac61dc673), dockerTools.examples.nginx cannot build already. I am doing a bisect.

@dingxiangfei2009
Copy link
Contributor Author

@arianvp This is the reason for that error: 4a4561c#diff-f5fad34a5e280fe300df690ac61dc673R122

It is deleting the VM drive before qemu exits, which makes it panic.

@dingxiangfei2009 dingxiangfei2009 force-pushed the dockertools-cross-compilation branch 2 times, most recently from 74594a6 to fbd040e Compare July 2, 2019 06:50
@arianvp
Copy link
Member

arianvp commented Jul 2, 2019

@GrahamcOfBorg test docker-tools

@arianvp
Copy link
Member

arianvp commented Jul 2, 2019

I think I just spawned a test seconds before you force-pushed. So it might fail again haha

@arianvp
Copy link
Member

arianvp commented Jul 2, 2019

@GrahamcOfBorg test docker-tools

@arianvp
Copy link
Member

arianvp commented Jul 2, 2019

Hmm, the test seems to time out on x86_64 whilst docker load'ing the container. that doesn't sound good :/

@dingxiangfei2009
Copy link
Contributor Author

@arianvp I tried the docker load command locally with the x86_64 image, and it works fine.

I want to try running these tests locally. How can I run this specific test?

@arianvp
Copy link
Member

arianvp commented Jul 3, 2019

nix-build ./nixos/tests/docker-tools.nix

@dingxiangfei2009
Copy link
Contributor Author

@arianvp Work fine here. I cannot reproduce the timeout.

See the gist:
https://gist.github.com/dingxiangfei2009/91b7d763853f4c6823e310792adf420f

@arianvp
Copy link
Member

arianvp commented Jul 3, 2019

@dingxiangfei2009 the reason is that not all build machines support nested kvm, so they emulate x86 and are slow. so this is why it's failing. Not because there is something wrong with the test.

Lets get this merged

@srhb
Copy link
Contributor

srhb commented Jul 31, 2019

This looks correct to me and I don't think there should be a reason to suspect that it should be slower on Hydra than the current release-blocker docker test. Let's keep an eye on it regardless. :)

@srhb srhb merged commit 19abdb7 into NixOS:master Jul 31, 2019
@dingxiangfei2009 dingxiangfei2009 deleted the dockertools-cross-compilation branch August 2, 2019 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants