Make cross-compiling with the crossSystem argument work (v2) #21388

DavidEGrayson · 2016-12-24T05:35:56Z

This is version 2 of my previous pull request (#18386). Compared to my previous pull request, this has a bit of a smaller scope: it does not add a test directory and it does not try to fix the deprecated --with-headers option used to compile GCC cross-compilers. Also, it does not do as much fixing of the top-level logic, thanks to the merged pull request #19940 by @Ericson2314 which cleaned up a lot of the problems in that area.

This pull request fixes several things about cross-compiling with the crossSystem argument:

It fixes the expressions for several derivations (gcc5, boehmgc, libffi, libxml2, readline, libtool, libiconv) so that their native derivations do not change when the top-level crossSystem argument is provided.
It avoids using bootstrap tools to compile the cross-compilers; it uses the normal stdenv.

To accomplish these goals, some new things were added:

stdenv.mkDerivationWithCrossArg
lib.overrideNativeDrv
lib.overrideNativeDrvs

Motivation for this change

For motivation information, see my comment on the old pull request: #18386 (comment)

Things done to test

This pull request was tested using the expression here:

https://gist.github.com/DavidEGrayson/8f962b3ec6edf06365ecb0c70a88029f

The tests passed, all the derivations were successfully built, and the rpiHello derivation was successfully run on a Raspberry Pi. This pull request should not cause a mass rebuild of native derivations.

Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Future work

Here are some similar things that could be done in the future:

Search for more improper uses of stdenv.cross, possibly remove stdenv.cross to prevent future errors.
Stop using the deprecated --with-headers option to configure GCC cross-compilers.
Apply these GCC fixes to the GCC 6 expression.
Get cross-compiling working on Darwin too; this pull request is mainly about getting it to work on Linux.
Add automated tests to make sure the problems fixed by this PR do not reoccur.

- Fix the expressions for several derivations (gcc5, boehmgc, libffi, libxml2, readline, libtool, libiconv) so that their native derivations do not change when the top-level crossSystem argument is provided. - Don't use bootstrap tools to compile the cross-compilers, use the normal stdenv. To accomplish these goals, some new things were added: - stdenv.mkDerivationWithCrossArg - lib.overrideNativeDrv - lib.overrideNativeDrvs This commit was tested using the expression here: https://gist.github.com/DavidEGrayson/8f962b3ec6edf06365ecb0c70a88029f The tests passed, all the derivations were successfully built, and the rpiHello derivation was successfully run on a Raspberry Pi. This commit should not cause a mass rebuild of native derivations. Some future work that would be nice: - It would be nice to stop using the deprecated --with-headers option to configure GCC cross-compilers. - It would be nice to apply these GCC fixes to the GCC 6 expression. - Get cross-compiling working on Darwin too; this commit is mainly about getting it to work on Linux.

mention-bot · 2016-12-24T05:35:58Z

@DavidEGrayson, thanks for your PR! By analyzing the history of the files in this pull request, we identified @FRidh, @Ericson2314 and @edolstra to be potential reviewers.

DavidEGrayson · 2016-12-24T05:42:05Z

CC @shlevy and @bjornfor because they both asked for status updates on the previous pull request. Also @dezgeg because he helped make the previous pull request work. Also, @viric and @rasendubi.

Ericson2314 · 2016-12-24T20:08:42Z

@DavidEGrayson you pinged me mentioning conflicts with my #21268 . Well besides the files changed there is a difference in approach. Currently in nixpkgs, as we both know, there is a only a very sloppy attempt to make nativeDrvs and crossDrvs. You clean that up with overrideNativeDrv, whereas I throw it out altogether. So were I to rebase mine on yours, I'd need to revert using those two lib functions (there's not bad, just no longer have any use), and changes like pythonSupportReally, whereas if you revert on mine, your diff should just shrink dramatically with no need to revert any of my changes.

Ericson2314 · 2016-12-24T20:34:58Z

On a diffferent note, it would be amazing to get your tests into https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/release-cross.nix so they (I think) will be tested by hydra.

DavidEGrayson · 2017-01-12T19:52:12Z

I posted this pull request one day before Christmas Eve. Now that the holidays are over, maybe a nixpkgs member would be interested in looking at it and merging it? There was a lot of interest on my previous attempt (#18386) and I thought people were gung-ho to get it merged in.

A lot of this pull request is just fixing logic errors in the recipes for individual packages that are due to people misunderstanding how stdenv.cross works.

Mic92 · 2017-01-13T00:18:38Z

I currently have not a good understanding on how stdenv works.
However I want nixos on my ARM boards too, so I do my best and give it a shoot.
When rebasing your pull request against current master,
I get the following evaluation error after, when trying to rebuild my system:

building Nix...
building the system configuration...
error: value is null while a Boolean was expected, at /etc/nixos/nixpkgs/pkgs/development/libraries/libxslt/default.nix:7:8
(use ‘--show-trace’ to show detailed location information)

This is the failing file: nixpkgs/pkgs/development/libraries/libxslt/default.nix

assert pythonSupport -> python2 != null;
assert pythonSupport -> libxml2.pythonSupport; # <--- this is failing line

you changed in libxml2/default.nix

-, pythonSupport ? (! stdenv ? cross) }:
+, pythonSupport ? null }:

As I said don't know enough about stdenv and cross compilation in this context to know what the right fix in this case would be...

DavidEGrayson · 2017-01-13T00:52:50Z

@Mic92 Thanks, that would have to be fixed before this PR is merged. If you're looking for a quick solution, maybe just remove the assertion that is failing. For the longer term, I'd have to look into populating libxml2.pythonSupport with a proper boolean variable that says whether it supports Python or not.

Mic92 · 2017-01-13T01:09:27Z

Unfortunately this is a mass-rebuild on NixOS. I currently do not have the resources locally to test this here. Probably someone with access to hydra could take a look at it. Because staging should be only used for changes, which are already expected to work, what is the appropriate action in this case @vcunat ?

DavidEGrayson · 2017-01-13T02:14:35Z

Hmmm, I have tests and code to try to make sure that this PR would not be a mass rebuild. What is the first derivation that is getting rebuilt for you when you try it out?

Ericson2314

@DavidEGrayson Sorry for being slow. While I prefer the more general approach of my PR long term, that one is somewhat blocked on review. Also it would be nice to get cross into a better stage for my own comparing hashes in that one.

I don't want to penalize you for mine being blocked, so even though my PRs would mean undoing some of the work here, I'm now willing to merge this as soon as you

make these changes
make as many tests as possible part of release-cross.nix (recall my Improve release-cross tests #21414). For example, or rpiCrossSystem is slightly different right? Maybe we want both variants in there? Maybe put the rest in maintainers/scripts (e.g. the HEAD, HEAD^ comparing ones)?

It would be great to get the green light from Travis too, but I understand that's often broken for silly reasons.

Ericson2314 · 2017-01-13T03:21:21Z

pkgs/top-level/stage.nix

-  # crossStdenv adapter.
+  # stdenvOverrides is used to avoid having multiple of versions
+  # of certain dependencies that were used in bootstrapping the
+  # standard environment.
  stdenvOverrides = self: super:


#21415 will imminently do half of this. See pkgs/stdenv/cross/default.nix for dealing with the rest.

Ericson2314 · 2017-01-13T03:38:56Z

pkgs/top-level/all-packages.nix

      cross = crossSystem;
      crossStageStatic = false;
+      noSysDirs = true;
+      isl = isl_0_14;


Ok for now, but we should figure out why this can't be the default so this and the Linux stdenv don't need to special case.

Ericson2314 · 2017-01-13T03:39:39Z

pkgs/top-level/all-packages.nix

        cross = crossSystem;
        crossStageStatic = true;
-        langCC = false;
+        noSysDirs = true;
+        isl = isl_0_14;


Ok for now, see isl comment below

Ericson2314 · 2017-01-13T03:45:03Z

pkgs/top-level/all-packages.nix

+  # standalone libiconv, just in case you want it.
+  libiconv =
+    let
+      nativeDrv = if stdenv.isGlibc then stdenv.cc.libc


Please use glibcIconv to strip unneeded

Ericson2314 · 2017-01-13T03:45:51Z

pkgs/top-level/all-packages.nix

+        else if stdenv.isDarwin then darwin.libiconv
+        else libiconvReal;
+      crossDrv =
+        if crossSystem.libc == "glibc" then libcCross


Similarly, please use glibcIconv

Ericson2314 · 2017-01-13T05:08:30Z

pkgs/development/compilers/gcc/5/default.nix

-stdenv.mkDerivation ({
+stdenv.mkDerivationWithCrossArg ( hostCrossSystem:
+  let
+    targetCrossSystem = if cross != null then cross else hostCrossSystem;


Please comment when targetCrossSystem is non-null, I presume this is is a function of build and host and target being equal or not equal in some combination.

Ericson2314 · 2017-01-13T05:09:57Z

pkgs/development/libraries/libxml2/default.nix

+
+stdenv.mkDerivationWithCrossArg(cross:
+  let
+    pythonSupportReally = if pythonSupport != null then pythonSupport


Please do something like pythonSupportReally, matching the others, so the body of this stays the same.

Mic92 · 2017-01-14T09:01:42Z

@DavidEGrayson this would be the rebuild log, when doing nixos-rebuild dry-build:
https://gist.github.com/a92f9e47bf87f801034221aa48fa3647

DavidEGrayson · 2017-01-26T18:29:32Z

I am closing this. @Ericson2314 is working hard to clean up cross-compiling in nixpkgs, so this pull request would probably just get in his way or be redundant.

Instead of working on this stuff, I am pursuing a project called nixcrpkgs where I will make my own Nix expressions for cross-compiling and just use nixpkgs as the base platform for my build system. Since I don't need to worry about supporting a million different use cases, my expressions can be much cleaner and simpler than what has accumulated in nixpkgs over the years, especially when it comes to building a GCC cross-compiler.

Ericson2314 · 2017-01-26T18:33:20Z

@DavidEGrayson Thanks, definitely watching that repo. I had opened #21471 which I hope your repo + my nixpkgs PRs will greatly accelerate.

DavidEGrayson mentioned this pull request Dec 24, 2016

Make cross-compiling with the crossSystem argument work #18386

Closed

7 tasks

DavidEGrayson mentioned this pull request Dec 24, 2016

libiconv: Correct cross-building #15021

Closed

5 tasks

Ericson2314 self-requested a review December 24, 2016 20:26

Ericson2314 added this to the 17.03 milestone Dec 24, 2016

This was referenced Dec 25, 2016

Improve release-cross tests #21414

Merged

Sane cross-compiling through bootstrapping #21268

Closed

rasendubi added the 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on label Dec 27, 2016

Mic92 added the 1.severity: mass-rebuild label Jan 13, 2017

Ericson2314 requested changes Jan 13, 2017

View reviewed changes

globin added the 2.status: merge conflict label Jan 25, 2017

Ericson2314 closed this Feb 6, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make cross-compiling with the crossSystem argument work (v2) #21388

Make cross-compiling with the crossSystem argument work (v2) #21388

DavidEGrayson commented Dec 24, 2016 •

edited

mention-bot commented Dec 24, 2016

DavidEGrayson commented Dec 24, 2016 •

edited

Ericson2314 commented Dec 24, 2016

Ericson2314 commented Dec 24, 2016

DavidEGrayson commented Jan 12, 2017 •

edited

Mic92 commented Jan 13, 2017 •

edited

DavidEGrayson commented Jan 13, 2017 •

edited

Mic92 commented Jan 13, 2017 •

edited

DavidEGrayson commented Jan 13, 2017

Ericson2314 left a comment •

edited

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Ericson2314 Jan 13, 2017

Mic92 commented Jan 14, 2017

DavidEGrayson commented Jan 26, 2017 •

edited

Ericson2314 commented Jan 26, 2017

Make cross-compiling with the crossSystem argument work (v2) #21388

Make cross-compiling with the crossSystem argument work (v2) #21388

Conversation

DavidEGrayson commented Dec 24, 2016 • edited

Motivation for this change

Things done to test

Future work

mention-bot commented Dec 24, 2016

DavidEGrayson commented Dec 24, 2016 • edited

Ericson2314 commented Dec 24, 2016

Ericson2314 commented Dec 24, 2016

DavidEGrayson commented Jan 12, 2017 • edited

Mic92 commented Jan 13, 2017 • edited

DavidEGrayson commented Jan 13, 2017 • edited

Mic92 commented Jan 13, 2017 • edited

DavidEGrayson commented Jan 13, 2017

Ericson2314 left a comment • edited

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Ericson2314 Jan 13, 2017

Choose a reason for hiding this comment

Mic92 commented Jan 14, 2017

DavidEGrayson commented Jan 26, 2017 • edited

Ericson2314 commented Jan 26, 2017

DavidEGrayson commented Dec 24, 2016 •

edited

DavidEGrayson commented Dec 24, 2016 •

edited

DavidEGrayson commented Jan 12, 2017 •

edited

Mic92 commented Jan 13, 2017 •

edited

DavidEGrayson commented Jan 13, 2017 •

edited

Mic92 commented Jan 13, 2017 •

edited

Ericson2314 left a comment •

edited

DavidEGrayson commented Jan 26, 2017 •

edited