Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove legacy cross, add aliases #42060

Closed
wants to merge 4 commits into from

Conversation

matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented Jun 15, 2018

Motivation for this change

This removes the legacy cross stuff & replace with equivalent aliases.

In addition, pkgsCross attribute is introduced which maps each of the example platforms to a cross compilation package set.

We also need to add eglibc attribute (which has apparently never been packaged!) The AVR alias is also apparently broken.

Related: #39087

/cc @Ericson2314

@matthewbauer matthewbauer changed the title Remove legacy cross, add aliases [wip] Remove legacy cross, add aliases Jun 15, 2018
};

avr-atmel = {
config = "avr-atmel-linux-avrabi";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely it must be none not linux?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, thanks.

avr-atmel = {
config = "avr-atmel-linux-avrabi";
platform = {
kernelArch = "avr";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AVR is a 8-bit microcontroller, it doesn't run Linux.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok hopefully it will not be needed with "none" above.

@@ -95,6 +95,8 @@ rec {

wasm32 = { bits = 32; significantByte = littleEndian; family = "wasm"; };
wasm64 = { bits = 64; significantByte = littleEndian; family = "wasm"; };

avr = { bits = 32; significantByte = littleEndian; family = "avr"; };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The family is optional, it can be skipped.

arm-frc-linux-gnueabi-binutils = pkgsCross.arm-frc-linux-gnueabi.buildPackages.binutils;
arm-frc-linux-gnueabi-gcc = pkgsCross.arm-frc-linux-gnueabi.buildPackages.gcc;

gcc-arm-embedded-4_8 = pkgsCross.armv7l-hf-multiplatform.buildPackages.gcc48;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't look at all the same. gcc-arm-embedded has things like soft-float libraries, some embedded libc, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I'll look into it. Is there another example platform that fits better? I can add a new one, my thought was that gcc-arm-embedded was just kind of a generic "arm" toolchain - I probably misunderstood the use case though.


avrgcclibc = throw "avrgcclibs are now separate packages, install avrbinutils, avrgcc and avrlibc";
avrbinutils = pkgsCross.avr-atmel.buildPackages.binutils;
avrgcc = pkgsCross.avr-atmel.buildPackages.gcc;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to contain wrapped versions of the toolchains? I doubt that's going to work for the 8-bit micros since all the hardening stuff etc. is irrelevant and probably harmful.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I wonder how to handle that. There is a gcc-unwrapped correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, maybe we can disable hardening for certain target platforms?

@dezgeg
Copy link
Contributor

dezgeg commented Jun 15, 2018

How has this been tested?

# Useful attribute of systems to cross compile to.
pkgsCross = lib.mapAttrs (n: crossSystem:
nixpkgsFun { inherit crossSystem; })
lib.systems.examples;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do people like doing this? We do it in reflex-platform & it works out nicely for convenience.

/cc @Ericson2314

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main thing is that packages inside nixpkgs shouldn't depend on this. Maybe we can make it some sort of default overlay that the tarball job can disable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Maybe this should be split off from the rest of PR? 😇

Seems review-able and useful independently :).

@dtzWill
Copy link
Member

dtzWill commented Jun 23, 2018

Omg pkgsCross would be amazing!!

Maybe next we could add a muslPkgs or something for musl-native builds.

Regardless, this would help users quite a bit I think-- even veteran nixpkgs users often aren't sure how to invoke things for cross (and even for those that do this is a lot nicer than some complicated argument!).

Will hopefully look through this more closely soon :).

@dezgeg
Copy link
Contributor

dezgeg commented Jun 23, 2018

Yes, there are definitely use cases for pkgsCross. For instance odroid-xu3-bootloader (which packages a install script) could already benefit. Ditto for tegra-uboot-flasher-scripts if I ever manage to package that monster.

@matthewbauer
Copy link
Member Author

matthewbauer commented Jun 23, 2018

Maybe next we could add a muslPkgs or something for musl-native builds.

Yeah, I would definitely like a "versatile" solution so that we don't need to do one for each possible permutation. Would pkgsLocal.musl64 be okay? We can also set up convenience aliases for muslPkgs if that's still desired.

@dtzWill
Copy link
Member

dtzWill commented Jun 25, 2018

Yeah, I would definitely like a "versatile" solution so that we don't need to do one for each possible permutation. Would pkgsLocal.musl64 be okay? We can also set up convenience aliases for muslPkgs if that's still desired.

Either of these sound great to me! 😁

@matthewbauer
Copy link
Member Author

Okay giving up on gcc-arm-embedded for now because it definitely require more information than I originally thought. We now just remove:

  • mentor
  • avr

and the arm-frc-linux-gnueabi stuff is already removed so we provide aliases for it too.

@matthewbauer
Copy link
Member Author

@GrahamcOfBorg build pkgsCross.avr-atmel.stdenv.cc.cc arm-frc-linux-gnueabi-gcc

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Partial log (click to expand)

Merge failed

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Partial log (click to expand)

Merge failed

inherit pkgs;


# Override system. This is useful to build i686 packages on x86_64-linux.
forceSystem = system: kernel: nixpkgsFun {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer I think all nixpkgsFun uses from all-packages.nix can be removed with this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome - there was one left for xbursttools that we can replace with nixpkgsCross.

# Used by wine, firefox with debugging version of Flash, ...
pkgsi686Linux = forceSystem "i686-linux" "i386";

callPackage_i686 = if stdenv.system == "i686-linux" || stdenv.system == "x86_64-linux"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if stdenv.hostPlatform.isx86 I think we can simplify this now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay to just remove the check? It should be done in another place.

@matthewbauer
Copy link
Member Author

@GrahamcOfBorg build pkgsCross.avr-atmel.stdenv.cc.cc arm-frc-linux-gnueabi-gcc xbursttools

@matthewbauer matthewbauer changed the title [wip] Remove legacy cross, add aliases Remove legacy cross, add aliases Jun 25, 2018
};
in
pkgsCross.buildPackages.gccCrossStageStatic;
gccCross = pkgsCross.ben-nanonote.buildPackages.gcc;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be gcc or gccCrossStageStatic? @Ericson2314

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gccCrossStageStatic. No libc is needed.

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Merge failed

@matthewbauer
Copy link
Member Author

@GrahamcOfBorg build pkgsCross.avr-atmel.stdenv.cc.cc arm-frc-linux-gnueabi-gcc xbursttools

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: pkgsCross.avr-atmel.stdenv.cc.cc, arm-frc-linux-gnueabi-gcc, xbursttools

Partial log (click to expand)

 # include <sgidefs.h>
           ^~~~~~~~~~~
compilation terminated.
make: *** [libpthread/nptl/sysdeps/unix/sysv/linux/Makefile.commonarch:137: libpthread/nptl/sysdeps/unix/sysv/linux/lowlevelrwlock.h] Error 1
builder for '/nix/store/7wydjyh3nbizkbmsx1vdsch7w4sbfvrl-uclibc-ng-1.0.30-mipsel-unknown-linux-uclibc.drv' failed with exit code 2
cannot build derivation '/nix/store/5p52ipkj5l5mf73qi3qv30n8q7zaq8ac-mipsel-unknown-linux-uclibc-binutils-wrapper-2.30.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3h5mv58c8j1ri058ishyba64m65mksgr-mipsel-unknown-linux-uclibc-stage-final-gcc-debug-7.3.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/r7x4qy2645g8xlgc2f872w42ggg0ha7r-mipsel-unknown-linux-uclibc-stage-final-gcc-debug-wrapper-7.3.0.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/w5gxkf3y973v3cq73s602fykpvlnaqly-xburst-tools-2011-12-26.drv': 1 dependencies couldn't be built
error: build of '/nix/store/18qgdpyvmps8n0wlpfw0vls8lbxldy07-avr-unknown-none-stage-final-gcc-debug-7.3.0.drv', '/nix/store/w5gxkf3y973v3cq73s602fykpvlnaqly-xburst-tools-2011-12-26.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

these paths will be fetched (64.80 MiB download, 455.23 MiB unpacked):
  /nix/store/537ksl4zn2q1ybn1bdidq9kaxzf95lnh-ncurses-6.1-abi5-compat
  /nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update
copying path '/nix/store/537ksl4zn2q1ybn1bdidq9kaxzf95lnh-ncurses-6.1-abi5-compat' from 'https://cache.nixos.org'...
copying path '/nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update' from 'https://cache.nixos.org'...
/nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

these paths will be fetched (64.41 MiB download, 452.11 MiB unpacked):
  /nix/store/hp2cjb64xl9n7zgzd3972q9bwbz9gssc-gcc-arm-embedded-6-2017-q2-update
copying path '/nix/store/hp2cjb64xl9n7zgzd3972q9bwbz9gssc-gcc-arm-embedded-6-2017-q2-update' from 'https://cache.nixos.org'...
/nix/store/hp2cjb64xl9n7zgzd3972q9bwbz9gssc-gcc-arm-embedded-6-2017-q2-update

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: pkgsCross.avr-atmel.stdenv.cc.cc, arm-frc-linux-gnueabi-gcc, xbursttools

Partial log (click to expand)

checking for avr-unknown-none-ar... avr-unknown-none-ar
configure: error: Wrong C compiler found; check the PATH!
builder for '/nix/store/ri859k40yp3igwnlmxpr9f5fiij12vib-avr-libc-2.0.0-avr-unknown-none.drv' failed with exit code 1
cannot build derivation '/nix/store/zj9cc0lq6mmp3ydz4mq31psad8rbsjf0-avr-unknown-none-binutils-wrapper-2.30.drv': 1 dependencies couldn't be built
building '/nix/store/zcmj4r5r945f3qq7wxmbjjs7r585n5l3-stdenv-darwin.drv'...
cannot build derivation '/nix/store/apkyvqcypa2x9wkci1azd9pahxyic1jm-avr-unknown-none-stage-final-gcc-debug-7.3.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/q8i8bg3sxilfky010vdanfmwm1w22d0a-glibc-2.27-arm-frc-linux-gnueabi.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/38xy7fd3v30n7mqw4hqkiq9bxjlmf462-arm-frc-linux-gnueabi-binutils-wrapper-2.30.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/97pjzp3d5mkigl40l034ys7h8cjxmcz0-arm-frc-linux-gnueabi-stage-final-gcc-debug-7.3.0.drv': 2 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/97pjzp3d5mkigl40l034ys7h8cjxmcz0-arm-frc-linux-gnueabi-stage-final-gcc-debug-7.3.0.drv', '/nix/store/apkyvqcypa2x9wkci1azd9pahxyic1jm-avr-unknown-none-stage-final-gcc-debug-7.3.0.drv', '/nix/store/w0qpj0v9n4gfl41imqsfmn5lwrrqb2hv-xburst-tools-2011-12-26.drv' failed

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Attempted: pkgsCross.avr-atmel.stdenv.cc.cc, arm-frc-linux-gnueabi-gcc, xbursttools

Partial log (click to expand)

./scripts/gcc-version.sh: line 27: gcc: command not found
make[1]: *** Documentation/Kbuild: Is a directory.  Stop.
make: *** [Makefile:1303: _clean_Documentation] Error 2
builder for '/nix/store/4yvlqmc3jwlc4n6rzpd6yhnrdccrinpd-linux-headers-4.15-mipsel-unknown-linux-uclibc.drv' failed with exit code 2
cannot build derivation '/nix/store/3jh74l4v7ks186cbyqml86khqr7ilpbz-uclibc-ng-1.0.30-mipsel-unknown-linux-uclibc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/93aafx897h811874drfs48dnmil8dzrd-mipsel-unknown-linux-uclibc-binutils-wrapper-2.30.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9ppsjz7zab5a53rj0nwrl6qng4b85m4f-mipsel-unknown-linux-uclibc-stage-final-gcc-debug-7.3.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/yk6a70prln2bxxarad06jcs207n27k4i-mipsel-unknown-linux-uclibc-stage-final-gcc-debug-wrapper-7.3.0.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/w0qpj0v9n4gfl41imqsfmn5lwrrqb2hv-xburst-tools-2011-12-26.drv': 1 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/97pjzp3d5mkigl40l034ys7h8cjxmcz0-arm-frc-linux-gnueabi-stage-final-gcc-debug-7.3.0.drv', '/nix/store/apkyvqcypa2x9wkci1azd9pahxyic1jm-avr-unknown-none-stage-final-gcc-debug-7.3.0.drv', '/nix/store/w0qpj0v9n4gfl41imqsfmn5lwrrqb2hv-xburst-tools-2011-12-26.drv' failed

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

these paths will be fetched (67.24 MiB download, 478.17 MiB unpacked):
  /nix/store/fx35r63vd21lfrs8r2zvvi39y4rrf738-gcc-arm-embedded-6-2017-q2-update
copying path '/nix/store/fx35r63vd21lfrs8r2zvvi39y4rrf738-gcc-arm-embedded-6-2017-q2-update' from 'https://cache.nixos.org'...
/nix/store/fx35r63vd21lfrs8r2zvvi39y4rrf738-gcc-arm-embedded-6-2017-q2-update

@matthewbauer matthewbauer deleted the better-cross branch June 27, 2018 21:00
@matthewbauer matthewbauer restored the better-cross branch June 27, 2018 21:00
@matthewbauer matthewbauer reopened this Jun 27, 2018
@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

/nix/store/hp2cjb64xl9n7zgzd3972q9bwbz9gssc-gcc-arm-embedded-6-2017-q2-update

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

/nix/store/fx35r63vd21lfrs8r2zvvi39y4rrf738-gcc-arm-embedded-6-2017-q2-update

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: gcc-arm-embedded

Partial log (click to expand)

these paths will be fetched (64.80 MiB download, 455.23 MiB unpacked):
  /nix/store/537ksl4zn2q1ybn1bdidq9kaxzf95lnh-ncurses-6.1-abi5-compat
  /nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update
copying path '/nix/store/537ksl4zn2q1ybn1bdidq9kaxzf95lnh-ncurses-6.1-abi5-compat' from 'https://cache.nixos.org'...
copying path '/nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update' from 'https://cache.nixos.org'...
/nix/store/bl2rcqq49gjzk2im4zhjz7j9m9whnvp5-gcc-arm-embedded-6-2017-q2-update

@dtzWill
Copy link
Member

dtzWill commented Jun 28, 2018

pkgsLocal <3<3<3

Brief pkgsCross testing of musl things looks good too (seems to work same, and have identical hashes, as the much more cumbersome --arg crossSystem ... previously needed (although often I ended up stashing that in a file)). Anyway, woohoo!

@nonsequitur
Copy link
Contributor

Evaluating avrlibc fails with

error: assertion failed at /tmp/nixpkgs/pkgs/top-level/all-packages.nix:6253:25

@matthewbauer
Copy link
Member Author

I think that's okay. The same thing will happen when trying any other cross libc. Eventually we want to make libc just an ordinary library but there is still too much dependence on the compiler.

You should be able to do:

nix-build -A pkgsCross.avr-atmel.avrlibc

(I think it will fail at some point though).

@dtzWill
Copy link
Member

dtzWill commented Jul 1, 2018

(apparently there are conflicts to resolve, FWIW!)

Is there a TODO of things left to address?

@matthewbauer
Copy link
Member Author

@GrahamcOfBorg build arm-frc-linux-gnueabi-gcc

I think the main thing is to make sure the new toolchains are close enough to what they are replacing. This could take a while & my main goal is to just have this stuff ready by August when master branches off.

If anyone can look at the avrlibc stuff that would be helpful - have no idea why it fails.

@GrahamcOfBorg
Copy link

Failure on x86_64-darwin (full log)

Partial log (click to expand)

Merge failed

@GrahamcOfBorg
Copy link

Failure on aarch64-linux (full log)

Partial log (click to expand)

Merge failed

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Partial log (click to expand)

Merge failed

More discussion in NixOS@ea67004

This readds some attributes that previously referred to chips used in the First
Robotic Competition. arm-frc-linux-gnueabi is the relevant quadruple.
This can now use new cross compilation infrastructure. Aliases are
provided.
These toolchains are no longer supported anymore according to https://www.arm-blog.com/arm-toolchain-from-mentor-is-dead/.
@matthewbauer
Copy link
Member Author

Closing for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants