uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support #36232

lopsided98 · 2018-03-02T23:32:00Z

Primarily, this PR updates U-Boot to the latest version, and fixes cross compilation. It also adds support for building U-Boot for the Rock64.

I had to rebase one of the patches from @dezgeg's repository to work with 2018.01, so I included it in this PR. I also added a patch to increase the maximum length of filenames read from an extlinux.conf file, which makes it possible to boot using a cross compiled kernel (which is normally too long because of the target suffix). Personally, I like keeping patches in the nixpkgs tree, but they could be moved to @dezgeg's repo.

The Rock64 is not yet supported in mainline U-Boot, but a community member has provided a patched version of 2018.01. To make a bootable U-Boot image for the Rock64, an ARM Trusted Firmware image is required. I created a new package for this firmware, which is structured similarly to the U-Boot package. Currently it only has a package for the RK3328 (the SOC in the Rock64). I am close to getting to the Rock64 to boot without any binary blobs, but U-Boot's TPL (which initializes the DRAM) causes the kernel to hang/crash soon after booting. Therefore, I am currently using a binary TPL provided by Rockchip.

Lastly, I switched to using mainline U-Boot for the Sheevaplug and Guruplug. These were added in 2010 and have received no maintenance since. Today, mainline U-Boot has a config for both of these boards, which build successfully. I have no way to test them, but they didn't even build before so this is better than it was.

I have tested that all the packages can be cross-compiled, and I tested native builds for the ODROID-XU3/4, Raspberry Pi 2 and 3, and Rock64. I have only tested booting with the Rock64, but I have an ODROID-XU4, Raspberry Pi 2 and Raspberry Pi 3 that I will be able to test in a few weeks (I don't have physical access to them right now, and I don't want to risk a boot failure).

Please let me know if you have any suggestions or things you want me to change.

cc @dezgeg

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

dezgeg · 2018-03-03T01:14:37Z

Nice! I will take a closer look tomorrow. The U-Boot v2018.03 release is due on 12.03.2018 so I think we can skip straight to it when that's released.

dezgeg · 2018-03-03T21:11:08Z

pkgs/misc/uboot/default.nix

-        url = https://github.com/dezgeg/u-boot/commit/pythonpath-2017-11.patch;
-        sha256 = "162b2lglp307pzxsf9m7nnmzwxqd7xkwp5j85bm6bg1a38ngpl9v";
-      })
+      ./pythonpath-2018-01.patch


I uploaded these to:

https://github.com/dezgeg/u-boot/commit/pythonpath-2018-03.patch
https://github.com/dezgeg/u-boot/commit/extlinux-path-length-2018-03.patch

so we can keep using fetchpatch.

lopsided98 · 2018-03-04T02:24:00Z

@dezgeg I rebased it to use your patches.

dezgeg · 2018-03-05T22:21:40Z

pkgs/misc/uboot/default.nix

-    makeFlags = [ "DTC=dtc" ] ++ extraMakeFlags;
+    makeFlags = [
+      "DTC=dtc"
+      "ARCH=${if hostPlatform.platform.kernelArch == "arm64" then "arm"


Actually, setting $ARCH shouldn't be required with U-Boot? It's auto-set by CONFIG_SYS_ARCH from the defconfig.

Yeah, it seems to build fine without it.

lopsided98 · 2018-03-05T22:58:36Z

I decided to removeAttrs all the defined arguments to buildUBoot. I had already done that for extraMeta, because it caused the build to fail since it is an attrset. targetPlatforms will have the same problem once #34444 is merged (see here), so I felt I might as well exclude all the unneeded attributes. Does this seem good to you?

dezgeg · 2018-03-07T12:57:16Z

Hmm, not sure that's worth it, I except the list will quickly get out of date as people forget to add to the list of variables to remove. Maybe we can remove targetPlatforms and instead use extraMeta.platforms directly, so only removing extraMeta suffices?

lopsided98 · 2018-03-08T05:42:48Z

I implemented what you suggested, but I added a little hack that basically transforms platforms to hydraPlatforms. Hopefully #34444 will be merged before this PR, but if not, this will prevent Hydra from attempting to build packages for the wrong platform.

dezgeg · 2018-03-08T15:13:07Z

If I remember correctly, it works the other way around; i.e. without that PR Hydra would typically attempt to build less packages (it would complain about e.g. error: Package ‘uboot-rpi_defconfig-2017.11’ in pkgs/misc/uboot/default.nix:77 is not supported on ‘x86_64-linux’, refusing to evaluate. even when cross-compilation is set up properly), so it shouldn't be needed. Things might have changed in the meantime though.

lopsided98 · 2018-03-08T18:30:17Z

Yes, that was the case before, because platforms would be compared against the build platform. This would prevent Hydra from building the packages, but it would also prevent me from cross compiling them. My current hack does not set platforms, which allows me to cross compile the packages locally, but prevents Hydra from trying to build them by setting hydraPlatforms.

On second thought, there's really no need for this hack because, until this issue is fixed, I can just set NIXPKGS_ALLOW_BROKEN if I want to cross-compile, so I decided to remove the hack to make things cleaner.

dezgeg · 2018-03-13T12:59:48Z

On second thought, there's really no need for this hack because, until this issue is fixed, I can just set NIXPKGS_ALLOW_BROKEN if I want to cross-compile, so I decided to remove the hack to make things cleaner.

Great, thanks!

v2018.03 is now out, you can squash in https://github.com/dezgeg/nixpkgs/commit/0c0cb5730d987f16b5fefd31b0d472446c1d0acb.patch to the first commit. It drops a patch that I sent in upstream before the release (in a different file though, so it still looks like it would apply). Also @bendlas pointed out some additional changes in pythonpath-2018-03.patch so the hash has changed. Also https://github.com/NixOS/nixpkgs/commit/fe3acfa41abeb81fac56e5bbfc6d9e0fd20cd65c.patch seems to be now needed for the tools to build, you can squash or pick it up as a separate change, whatever you prefer.

On my side I will try to test this on my boards this friday.

The bootloader is currently non-functional because it lacks the ARM Trusted Firmware image (bl31.elf).

ARM trusted firmware is required as part of the boot process on some ARMv8-A boards. Currently, only the RK3328 is supported in nixpkgs. This makes the Rock64 u-boot image bootable.

This fixes problems with cross compiled kernels failing to boot.

This stops the kernel from crashing immediately after boot.

These derivations have not seen any updates since they were created in 2010, and some of their sources have disappeared. There are upstream configs for these boards, so these are now used, and they build correctly. I have no way of testing them, and I don't if anyone even uses either board with Nix anymore.

lopsided98 · 2018-03-13T17:46:59Z

I can't get the tools to cross compile using sandbox_defconfig (native builds work). It is unable to find an openssl header:

In file included from tools/default_image.c:20:0:
include/image.h:1058:12: fatal error: openssl/evp.h: No such file or directory

I looked into the problem with allnoconfig. It seems like the build system is not setting values for certain hex and int config options without default values (or with conditional default values that are not enabled). This is not new behavior (it happens in 2018.01 with allyesconfig), but recently CONFIG_SYS_TEXT_BASE was changed (see u-boot/u-boot@278b90c) so that it triggers this issue with allnoconfig. I have worked around this by manually giving this option an arbitrary value. This makes native and cross builds of ubootTools work. We could also look into trying to fix this upstream.

lopsided98 · 2018-03-13T18:54:26Z

I tested 2018.03 on my Raspberry Pi 2 and ODROID-XU4 and haven't seen any problems.

dezgeg · 2018-03-14T12:56:30Z

In file included from tools/default_image.c:20:0:
include/image.h:1058:12: fatal error: openssl/evp.h: No such file or directory

Ah, so the sandbox has CONFIG_FIT_SIGNATURE enabled, so it now attempts to build the fit_info and fit_check_sign tools which does need the OpenSSL headers but they aren't found. Mysterious.

I looked into the problem with allnoconfig. It seems like the build system is not setting values for certain hex and int config options without default values (or with conditional default values that are not enabled). This is not new behavior (it happens in 2018.01 with allyesconfig), but recently CONFIG_SYS_TEXT_BASE was changed (see u-boot/u-boot@278b90c) so that it triggers this issue with allnoconfig. I have worked around this by manually giving this option an arbitrary value. This makes native and cross builds of ubootTools work.

Ok, sounds fine.

dezgeg · 2018-03-16T11:50:39Z

I have tested so far on RPi 1, RPi 3 (in both 32 & 64-bit modes), Wandboard Quad, Beaglebone Black, pcDuino 3 Nano and QEMU for both ARM & AArch64. I will test a couple of more boards soon but all is fine here as well.

dezgeg · 2018-03-16T11:55:09Z

BTW @lopsided98 if you happen to have a spare SD card and cycles, would you mind testing if https://www.cs.helsinki.fi/u/tmtynkky/sd-image-armv7l-with-linux-rpi.img boots for you on RPi 2? It's an image built with linux_rpi aka the Raspbian kernel fork. I haven't had any success on RPi3 while it should work in theory.

dezgeg · 2018-03-18T17:13:48Z

Applied to master.

lopsided98 · 2018-03-19T00:17:04Z

I tested your SD card image, and it hangs after u-boot starts the kernel. Here is the boot log:

U-Boot 2017.11 (Nov 14 2017 - 01:08:06 +0000)

DRAM:  948 MiB
RPI 2 Model B (0xa21041)
MMC:   sdhci@7e300000: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   No ethernet found.
starting USB...
USB0:   Core Release: 2.80a
scanning bus 0 for devices... 5 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
reading /extlinux/extlinux.conf
851 bytes read in 20 ms (41 KiB/s)
------------------------------------------------------------
1:      NixOS - Default
Enter choice: 1:        NixOS - Default
Retrieving file: /extlinux/../nixos/h1dgpfgw25ybghnycq2b8vmc6zhpfr80-initrd-initrd
reading /extlinux/../nixos/h1dgpfgw25ybghnycq2b8vmc6zhpfr80-initrd-initrd
7183744 bytes read in 512 ms (13.4 MiB/s)
Retrieving file: /extlinux/../nixos/wjxr12srr7prdvc6hlkyy814x2kby1yl-linux-4.9.59-1.20171029-zImage
reading /extlinux/../nixos/wjxr12srr7prdvc6hlkyy814x2kby1yl-linux-4.9.59-1.20171029-zImage
3395792 bytes read in 265 ms (12.2 MiB/s)
append: systemConfig=/nix/store/v22c57rphxybzp9aha8l5zyc207s3543-nixos-system-nixos-18.09pre-git init=/nix/store/v22c57rphxybzp9aha8l5zyc207s3543-nixos-system-nixos-18.09pre-git/init loglevel=7 console=ttyS0,115200n8 console=ttymxc0,115200n8 console=ttyAMA0,115200n8 console=ttyO0,115200n8 console=ttySAC2,115200n8 console=tty0
Retrieving file: /extlinux/../nixos/wjxr12srr7prdvc6hlkyy814x2kby1yl-linux-4.9.59-1.20171029-dtbs/bcm2836-rpi-2-b.dtb
reading /extlinux/../nixos/wjxr12srr7prdvc6hlkyy814x2kby1yl-linux-4.9.59-1.20171029-dtbs/bcm2836-rpi-2-b.dtb
16693 bytes read in 47 ms (346.7 KiB/s)
Kernel image @ 0x3000000 [ 0x000000 - 0x33d0d0 ]
## Flattened Device Tree blob at 00000100
   Booting using the fdt blob at 0x000100
   reserving fdt memory region: addr=0 size=1000
   Using Device Tree in place at 00000100, end 00007234

Starting kernel ...

GrahamcOfBorg added 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 0 10.rebuild-linux: 11-100 labels Mar 2, 2018

dezgeg reviewed Mar 3, 2018

View reviewed changes

lopsided98 force-pushed the uboot-fixes branch 3 times, most recently from 34d6566 to 242405e Compare March 4, 2018 02:19

Mic92 mentioned this pull request Mar 5, 2018

u-boot: 2017.11 -> v2018.03-rc3 #36340

Closed

8 tasks

lopsided98 force-pushed the uboot-fixes branch from 242405e to 46fb1c0 Compare March 5, 2018 22:19

lopsided98 requested review from edolstra and nbp as code owners March 5, 2018 22:19

dezgeg reviewed Mar 5, 2018

View reviewed changes

lopsided98 force-pushed the uboot-fixes branch from 46fb1c0 to 679e30f Compare March 5, 2018 22:38

lopsided98 force-pushed the uboot-fixes branch 3 times, most recently from 1a52271 to 7b50251 Compare March 7, 2018 00:22

lopsided98 mentioned this pull request Mar 7, 2018

lib: Fix #30902 #34444

Merged

8 tasks

lopsided98 force-pushed the uboot-fixes branch from 7b50251 to e130537 Compare March 8, 2018 05:33

lopsided98 force-pushed the uboot-fixes branch from e130537 to c9557aa Compare March 8, 2018 18:27

GrahamcOfBorg added 10.rebuild-linux: 1-10 and removed 10.rebuild-linux: 11-100 labels Mar 8, 2018

lopsided98 changed the title ~~uboot: 2017.11 -> 2018.01, cross fixes, Rock64 support~~ uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support Mar 13, 2018

lopsided98 force-pushed the uboot-fixes branch from c9557aa to d79b459 Compare March 13, 2018 16:23

lopsided98 added 8 commits March 13, 2018 13:40

uboot: 2017.11 -> 2018.03, cross fixes

f184747

uboot: add Rock64 support

875388c

The bootloader is currently non-functional because it lacks the ARM Trusted Firmware image (bl31.elf).

maintainers: add lopsided98

1779cda

arm-trusted-firmware: init

31a1f6c

ARM trusted firmware is required as part of the boot process on some ARMv8-A boards. Currently, only the RK3328 is supported in nixpkgs. This makes the Rock64 u-boot image bootable.

uboot: add patch to increase maximum path length in extlinux.conf

22ccace

This fixes problems with cross compiled kernels failing to boot.

uboot: rock64: use binary blob for TPL

2892b88

This stops the kernel from crashing immediately after boot.

uboot: fix cross-build of ubootTools

b2d77fe

lopsided98 force-pushed the uboot-fixes branch from d79b459 to b2d77fe Compare March 13, 2018 17:46

dezgeg closed this Mar 18, 2018

Ericson2314 mentioned this pull request Mar 18, 2018

lib: Fix #30902 on 18.03 #37337

Merged

8 tasks

lopsided98 deleted the uboot-fixes branch April 7, 2018 01:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support #36232

uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support #36232

lopsided98 commented Mar 2, 2018

dezgeg commented Mar 3, 2018

dezgeg Mar 3, 2018

lopsided98 commented Mar 4, 2018

dezgeg Mar 5, 2018 •

edited

lopsided98 Mar 5, 2018

lopsided98 commented Mar 5, 2018

dezgeg commented Mar 7, 2018

lopsided98 commented Mar 8, 2018

dezgeg commented Mar 8, 2018

lopsided98 commented Mar 8, 2018

dezgeg commented Mar 13, 2018

lopsided98 commented Mar 13, 2018

lopsided98 commented Mar 13, 2018

dezgeg commented Mar 14, 2018

dezgeg commented Mar 16, 2018

dezgeg commented Mar 16, 2018

dezgeg commented Mar 18, 2018

lopsided98 commented Mar 19, 2018 •

edited

uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support #36232

uboot: 2017.11 -> 2018.03, cross fixes, Rock64 support #36232

Conversation

lopsided98 commented Mar 2, 2018

dezgeg commented Mar 3, 2018

dezgeg Mar 3, 2018

Choose a reason for hiding this comment

lopsided98 commented Mar 4, 2018

dezgeg Mar 5, 2018 • edited

Choose a reason for hiding this comment

lopsided98 Mar 5, 2018

Choose a reason for hiding this comment

lopsided98 commented Mar 5, 2018

dezgeg commented Mar 7, 2018

lopsided98 commented Mar 8, 2018

dezgeg commented Mar 8, 2018

lopsided98 commented Mar 8, 2018

dezgeg commented Mar 13, 2018

lopsided98 commented Mar 13, 2018

lopsided98 commented Mar 13, 2018

dezgeg commented Mar 14, 2018

dezgeg commented Mar 16, 2018

dezgeg commented Mar 16, 2018

dezgeg commented Mar 18, 2018

lopsided98 commented Mar 19, 2018 • edited

dezgeg Mar 5, 2018 •

edited

lopsided98 commented Mar 19, 2018 •

edited