Docker-related cross-compilation fixes #68900

nspin · 2019-09-16T13:49:59Z

Motivation for this change

Fixes cross-compilation for:

cni
tini
containerd
docker

Things done

Notify maintainers

cc @

nspin · 2019-09-16T14:08:35Z

pkgs/applications/virtualization/docker/default.nix

@@ -183,7 +196,7 @@ rec {
    '';

    preFixup = ''
-      find $out -type f -exec remove-references-to -t ${go} -t ${stdenv.cc.cc} '{}' +
+      find $out -type f -exec remove-references-to -t ${go.nativeDrv or go} -t ${stdenv.cc.cc} '{}' +


In the case of cross-compilation, go refers to the host derivation, but we need the build derivation. .nativeDrv is only present when we're cross-compiling (this is the result of an optimization in splice.nix).

There are many similar cases in nixpkgs where a native dependency is meant to be interpolated into a string. For example, grep for remove-references-to -t ${go}. As in the case above, this does not work when cross-compiling.

There are no explicit references to .nativeDrv like this in nixpkgs. Such explicit references don't feel like the right solution here, especially considering the fact that the .nativeDrv attribute is unstable, and will someday be renamed to pkgBuildHost. Should we add library functions to wrap explicit access?

Related: #49526 (comment) and #68895

I think buildPackages.go would be better here? /cc @Ericson2314

Especially for removing references, the go interpolated into that string must match the one used as a nativeBuildInput. So, buildPackages.go would have to replace each instance of go in that file. The drawback of this approach is that go becomes harder to override for this package.

Yes that is a the drawback, but splicing is evil in using things like builtins.tryEval and still falling over itself when the packages are too different. It's better to use buildInputs.go, I'm afraid. If you really want, you could do goForBuild = buildPackages.go in the override list or something and use that.

I agree that this is the best solution in sight right now.

nspin · 2019-09-17T22:56:46Z

pkgs/applications/virtualization/docker/default.nix

    # Optimizations break compilation of libseccomp c bindings
    hardeningDisable = [ "fortify" ];

-    nativeBuildInputs = [ pkgconfig ];
+    inherit (go.nativeDrv or go) GOOS GOARCH;


Same issue with .nativeDrv here.

stale · 2020-06-01T09:23:39Z

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

Search for maintainers and people that previously touched the
related code and @ mention them in a comment.
Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
irc.freenode.net.

nspin · 2020-06-01T09:34:19Z

(In response to the automatic marking of this PR as stale)

I'm still interested in how cross compilations issues like this one, brought about as a side effect of splicing, are best addressed.

Mic92 · 2020-06-08T13:37:25Z

Sorry for letting this getting stalled. If you could rebase this. I will have a look.

betaboon · 2020-06-08T16:24:43Z

i ran into this issue today and got it building with the help of @Mic92.
he advised to convert the package to use buildGoPackage.

this is the diff i ended up with: https://gist.github.com/betaboon/b0f7d8311297fe6ae6e751783c876272

Mic92 · 2020-06-08T16:31:12Z

@betaboon can you make this a PR?

betaboon · 2020-06-09T09:26:44Z

@nspin do you want to pick this back up and get it merged or should i take over?

i think #50925 is still blocking docker from successfully cross-compiling. i ended up overriding it to btrfs-progs = null; for docker, which deactivates the btrfs-support thus the cross-compile succeeds.

cc @Mic92

nspin · 2020-06-12T01:10:26Z

@betaboon If you've got a patch ready, go for it! I'm also happy to finish up if you'd prefer that.

However, I noticed this added line in your gist:

find $out/libexec -type f -exec remove-references-to -t ${buildPackages.go} '{}' +

This line faces the same issue that I tried to address in this PR. If buildGoPackages is overridden (to, for example, override the Go version), buildPackages.go no longer refers to the intended derivation.

This isn't a big deal in itself, but while cross-compilation Nixpkgs I found a number of similar instances. In this case, this problem doesn't get in the way of the derivation's functionality, it just may result in a bloated closure. #68900 (comment) suggests a good immediate solution for cases where this sort of issue is more of a problem.

betaboon · 2020-06-24T08:34:31Z

@nspin I'm just preparing a PR and building for armv7l here to see if it works fine.

on the remove references-to-issue: @Mic92 suggested adding find $out/libexec -type f -exec ${removeExpr removeReferences} '{}' + || true here

nspin · 2020-07-03T06:36:20Z

Great! It appears that the go argument of go-packages/generic.nix is always passed explicitly with a value of the form buildPackages.go_*, so that should indeed work.

betaboon · 2020-07-03T07:24:46Z

@nspin i just pushed the changes to a new pr #68900

but this is still blocked by #50925 (on armv7l)

nspin added 4 commits September 16, 2019 13:42

cni: add aarch64-linux as platform

e0527ce

tini: fix cross-compilation

6176cf7

containerd: fix cross-compilation

12c5361

docker: fix cross-compilation

c59e9a0

ofborg bot added the 8.has: documentation label Sep 16, 2019

nspin commented Sep 16, 2019

View reviewed changes

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Sep 16, 2019

nspin commented Sep 17, 2019

View reviewed changes

bjornfor added the 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on label Oct 6, 2019

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020

nspin mentioned this pull request Jun 1, 2020

openvswitch: fix cross-compilation #68895

Closed

10 tasks

betaboon mentioned this pull request Jul 3, 2020

Docker cross compile fixes v2 #92140

Merged

10 tasks

nspin closed this Jul 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker-related cross-compilation fixes #68900

Docker-related cross-compilation fixes #68900

nspin commented Sep 16, 2019 •

edited

nspin Sep 16, 2019 •

edited

nspin Sep 16, 2019

matthewbauer Sep 17, 2019

nspin Sep 17, 2019

Ericson2314 Jun 8, 2020

nspin Jun 12, 2020

nspin Sep 17, 2019

stale bot commented Jun 1, 2020

nspin commented Jun 1, 2020

Mic92 commented Jun 8, 2020

betaboon commented Jun 8, 2020

Mic92 commented Jun 8, 2020

betaboon commented Jun 9, 2020

nspin commented Jun 12, 2020 •

edited

betaboon commented Jun 24, 2020

nspin commented Jul 3, 2020

betaboon commented Jul 3, 2020

Docker-related cross-compilation fixes #68900

Docker-related cross-compilation fixes #68900

Conversation

nspin commented Sep 16, 2019 • edited

Motivation for this change

Things done

Notify maintainers

nspin Sep 16, 2019 • edited

Choose a reason for hiding this comment

nspin Sep 16, 2019

Choose a reason for hiding this comment

matthewbauer Sep 17, 2019

Choose a reason for hiding this comment

nspin Sep 17, 2019

Choose a reason for hiding this comment

Ericson2314 Jun 8, 2020

Choose a reason for hiding this comment

nspin Jun 12, 2020

Choose a reason for hiding this comment

nspin Sep 17, 2019

Choose a reason for hiding this comment

stale bot commented Jun 1, 2020

nspin commented Jun 1, 2020

Mic92 commented Jun 8, 2020

betaboon commented Jun 8, 2020

Mic92 commented Jun 8, 2020

betaboon commented Jun 9, 2020

nspin commented Jun 12, 2020 • edited

betaboon commented Jun 24, 2020

nspin commented Jul 3, 2020

betaboon commented Jul 3, 2020

nspin commented Sep 16, 2019 •

edited

nspin Sep 16, 2019 •

edited

nspin commented Jun 12, 2020 •

edited