raspberrypi: update firmware, linux, tools to latest (~2020-12-01) #107637

drewrisinger · 2020-12-26T16:05:27Z

Motivation for this change

Various packages in the Raspberry Pi ecosystem were out of date.

Some commits might need squashed, wanted to leave them separate for now to clarify why/what I did.

Potentially controversial: noticed duplicate raspberrypi-tools && libraspberrypi. These have the same source & practically the same derivation. I chose to keep libraspberrypi because it had a slightly cleaner derivation, but I'll leave it up to the maintainers to decide.

Things done

drewrisinger · 2020-12-26T16:08:32Z

pkgs/development/misc/rpiboot/unstable.nix

-    platforms = stdenv.lib.platforms.unix;
+    license = licenses.asl20;
+    maintainers = with maintainers; [ cartr ];
+    platforms = [ "aarch64-linux" "armv7l-linux" "armv6l-linux" ];


This technically builds on regular Linux, but it's really targeted at ARM devices/the Pi, so I think this is reasonable.
https://github.com/raspberrypi/usbboot/blob/d3760e119385a179765f43a50a8e051a44127c25/Readme.md#building

Isn't this a tool I could run on my workstation to "fastload" a raspberry Pi? If so, building it on x86_64-linux too would be appreciated ;-)

prusnak

untested, changes look fine

pkgs/development/libraries/libraspberrypi/default.nix

flokli · 2020-12-27T14:39:21Z

pkgs/development/misc/rpiboot/unstable.nix

-    platforms = stdenv.lib.platforms.unix;
+    license = licenses.asl20;
+    maintainers = with maintainers; [ cartr ];
+    platforms = [ "aarch64-linux" "armv7l-linux" "armv6l-linux" ];


Isn't this a tool I could run on my workstation to "fastload" a raspberry Pi? If so, building it on x86_64-linux too would be appreciated ;-)

pkgs/os-specific/linux/kernel/linux-rpi.nix

Update to the new default branch for the Raspberry Pi vendor kernel.

This only copies unfree redistributables, don't need a compiler for that.

These packages were duplicated, and often weren't being updated in sync. The only difference between them was the lack of pkg-config in libraspberrypi, which is easily fixable.

This tool can run perfectly fine on a x86_64-linux workstation - it's primarily meant to boot Raspberry Pi CM/CM3/Zero devices, not to run on them.

flokli · 2020-12-27T16:06:34Z

I rebased this on latest master, incorporated the fixes from https://github.com/NixOS/nixpkgs/pull/107689/files and fixed the duplicate attribute in the kernel expression.

libraspberrypi provided an empty directory. The during NixOS#107637, this was refactored to use cmakeFlags. cmakeFlags can't use `$out` directly. `$out` is a bash variable, so to use it there, `${placeholder "out"}` is needed, otherwise it'll pick `"$out"` literally.

ghost · 2020-12-30T14:13:09Z

My i2s sound card does not work with the 5.4 rpi4 or rpi3 kernel. The device tree overlays fail to build:

Error at 'compatible': FDT_ERR_NOTFOUND
builder for '/nix/store/bk30gbx07bv0303ji08dzhblwrsc8jd8-device-tree-overlays.drv' failed with exit code 1

This can be reproduced with the following config lines:

  boot.kernelPackages = pkgs.linuxPackages_rpi4;
  hardware.deviceTree.enable = true;
  hardware.deviceTree.overlays = [ "${pkgs.linux_rpi4}/dtbs/overlays/hifiberry-amp.dtbo" ];

flokli · 2020-12-30T23:24:05Z

@petabyteboy yes, the raspberrypi device tree files unfortunately seem to use some custom format/extension which doesn't seem to be understood by the mainline dtc compiler, and its error messages are terrible too.

raspberrypi/linux#3198 (comment) claims it should work, but apparently it doesn't (or broke again, who knows).

We might want to open a new issue if it clearly doesn't work with the upstream dtc compiler.

The raspberrypi-specific dtmerge tool seems to be able to work with these overlays - I stumbled over kwohlfahrt@7ba456e, which allows using that for hardware.deviceTree instead of dtc - we might want to upstream to this nixpkgs.

I did some ground-work in fixing raspberrypi-tools to build on x86_64-linux (so I can still cross-compile raspberrypi closures from x86_64 (PR merged in #107689, moved to libraspberrypi in this PR), but moved over to a Pine64 for the time being.

Happy to review a PR that allows compiling devicetress with dtmerge :-)

ghost · 2020-12-31T00:03:22Z

I could make my Raspberry Pi 4 work with the HifiBerry Amp2 (which is using the hifiberry-dacplus overlay) by using dtmerge:

{ config, pkgs, lib, ... }:

let
  dtbsWithSymbols = pkgs.stdenv.mkDerivation {
    name = "dtbs-with-symbols";
    inherit (config.boot.kernelPackages.kernel) src nativeBuildInputs depsBuildBuild;
    patches = map (patch: patch.patch) config.boot.kernelPackages.kernel.kernelPatches;
    buildPhase = ''
      patchShebangs scripts/*
      substituteInPlace scripts/Makefile.lib \
        --replace 'DTC_FLAGS += $(DTC_FLAGS_$(basetarget))' 'DTC_FLAGS += $(DTC_FLAGS_$(basetarget)) -@'
      make ${pkgs.stdenv.hostPlatform.platform.kernelBaseConfig} ARCH="${pkgs.stdenv.hostPlatform.platform.kernelArch}"
      make dtbs ARCH="${pkgs.stdenv.hostPlatform.platform.kernelArch}"
    '';
    installPhase = ''
      make dtbs_install INSTALL_DTBS_PATH=$out/dtbs  ARCH="${pkgs.stdenv.hostPlatform.platform.kernelArch}"
    '';
  };

in {
  boot.kernelPackages = pkgs.linuxPackages_rpi4;
  
  hardware.deviceTree.enable = true;
  hardware.deviceTree.filter = "bcm2711-rpi-4-b.dtb";
  hardware.deviceTree.package = lib.mkForce (
    pkgs.runCommand "device-tree-overlays" {
      buildInputs = with pkgs; [ findutils libraspberrypi ];
    } ''
      cd ${dtbsWithSymbols}/dtbs
      for dtb in $(find . -type f -name "${config.hardware.deviceTree.filter}")
      do
        install -D $dtb $out/$dtb

        ${lib.concatMapStrings (param: ''
          dtmerge -d $out/$dtb{,-merged} - ${param}
          mv $out/$dtb{-merged,}
        '') [ "spi=on" ]}

        ${lib.concatMapStrings (overlay: ''
          dtmerge -d $out/$dtb{,-merged} ${overlay}
          mv $out/$dtb{-merged,}
        '') [ "${config.boot.kernelPackages.kernel}/dtbs/overlays/hifiberry-dacplus.dtbo" ]}
      done
    ''
  );
  
  [...]
}

However my Raspberry Pi 3 is out of luck. Changing boot.kernelPackages to pkgs.linuxPackages_rpi3 and hardware.deviceTree.filter to bcm2837-rpi-3-b-plus.dtb makes it fail even using dtmerge:

DTOVERLAY[debug]: using platform 'bcm2835'
DTOVERLAY[debug]: overlay map loaded
root module: vc4
DTOVERLAY[debug]: /__symbols__:i2c=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_arm=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_vc=i2c1
DTOVERLAY[error]: can't find symbol 'vdd_3v3_reg'

Or when applying the hifiberry-amp overlay:

DTOVERLAY[debug]: using platform 'bcm2835'
DTOVERLAY[debug]: overlay map loaded
DTOVERLAY[debug]: /__symbols__:i2c=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_arm=i2c0
DTOVERLAY[debug]: /__symbols__:i2c_vc=i2c1
DTOVERLAY[error]: can't find symbol 'sound'

These are the same errors I get when trying to apply these overlays to a recent upstream linux kernel by the way.

ghost · 2020-12-31T02:36:35Z

I should first say a setup that was quite common with NixOS aarch64 users was actually never supported by upstream: The RPi3 is supposed to be used in 32-bit arm and work exactly like an RPi2, while we use it in aarch64 mode, where it has its own defconfig and so on.

From 4.19 to 5.4 this change broke raspberry pi specific device tree overlays for the RPi3 aarch64 config:

raspberrypi/linux@9a6d1cb#diff-0c179e16b6e7236b33310a13729679c1c136d2b7a216339ff652d5c4db3a1e21L33

Both the RPi4 aarch64 config and the RPi3 and lower 32-bit arm configs import bcm270x-rpi.dtsi, which specifies some things for raspberry-pi-specific overlays, like vdd_3v3_reg, that are not upstreamed:

result/arch/arm/boot/dts/bcm270x-rpi.dtsi: /* Downstream modifications to bcm2835-rpi.dtsi */
result/arch/arm/boot/dts/bcm270x-rpi.dtsi:      vdd_3v3_reg: fixedregulator_3v3 {
result/arch/arm/boot/dts/bcm2708-rpi.dtsi:#include "bcm270x-rpi.dtsi"
result/arch/arm/boot/dts/bcm2711-rpi.dtsi:#include "bcm270x-rpi.dtsi"

But bcm2835-rpi.dtsi does not include bcm270x-rpi.dtsi, so the RPi3 aarch64 config can not use raspberry pi specific overlays.

I will try to send a patch to the raspberrypi/linux to add back support for this use case.

ghost · 2020-12-31T05:20:00Z

Nevermind, this was already the case with the last 4.19 rpi3 aarch64 kernel, so it's not a problem that was introduced with this PR.

flokli · 2021-01-01T18:57:51Z

@petabyteboy I linked to here from https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi_4, cause others seem to be running into this issue. If you went into contact with upstream, can you please link from here?

drewrisinger commented Dec 26, 2020

View reviewed changes

prusnak approved these changes Dec 26, 2020

View reviewed changes

drewrisinger mentioned this pull request Dec 26, 2020

Raspberry Pi 4 support #63720

Closed

lovesegfault requested a review from samueldr December 26, 2020 21:36

flokli reviewed Dec 27, 2020

View reviewed changes

matthewbauer and others added 13 commits December 27, 2020 16:48

linux_rpi: 4.19.118 -> 5.4.51

44dae8d

Update to the new default branch for the Raspberry Pi vendor kernel.

linux_rpi: 5.4.51 -> 5.4.79

6f74787

raspberrypiWirelessFirmware: 2019-08-16 -> 2020-12-01

a128c9c

raspberrypifw: 20201022 -> 20201201

860cb18

raspberrypifw: use stdenvNoCC

924d461

This only copies unfree redistributables, don't need a compiler for that.

raspberrypi-tools: remove in favor of libraspberrypi

979a1e1

These packages were duplicated, and often weren't being updated in sync. The only difference between them was the lack of pkg-config in libraspberrypi, which is easily fixable.

libraspberrypi: 2020-05-28 -> 2020-11-30

65172da

libraspberrypi: merge with raspberrypi-tools

3f64a4e

libraspberrypi: fix cross-compile conventions

e89369f

rpiboot: unstable-2020-05-11 -> unstable-2020-10-20

8f0ace2

rpi-eeprom: raspberrypi-tools -> libraspberrypi

2e83f19

rpi-eeprom: unstable-2020-10-05 -> 2020-12-11

4fe8a8b

rpiboot-unstable: enable build on x86_64-linux

5deb8af

This tool can run perfectly fine on a x86_64-linux workstation - it's primarily meant to boot Raspberry Pi CM/CM3/Zero devices, not to run on them.

flokli force-pushed the dr-pr-update-raspberrypi branch from 15e1538 to 5deb8af Compare December 27, 2020 16:04

flokli approved these changes Dec 27, 2020

View reviewed changes

prusnak approved these changes Dec 27, 2020

View reviewed changes

ofborg bot added 8.has: clean-up 8.has: package (new) labels Dec 27, 2020

ofborg bot requested review from tavyc, cartr, lopsided98, dasJ, dezgeg and tkerber December 27, 2020 16:32

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Dec 27, 2020

flokli merged commit dcbe3b4 into NixOS:master Dec 27, 2020

flokli mentioned this pull request Dec 28, 2020

libraspberrypi: fix output #107807

Merged

10 tasks

flokli mentioned this pull request Jan 10, 2021

Improve device-tree overlay support #79370

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

raspberrypi: update firmware, linux, tools to latest (~2020-12-01) #107637

raspberrypi: update firmware, linux, tools to latest (~2020-12-01) #107637

drewrisinger commented Dec 26, 2020

drewrisinger Dec 26, 2020

flokli Dec 27, 2020

prusnak left a comment

flokli Dec 27, 2020

flokli commented Dec 27, 2020

ghost commented Dec 30, 2020

flokli commented Dec 30, 2020

ghost commented Dec 31, 2020 •

edited by ghost

ghost commented Dec 31, 2020

ghost commented Dec 31, 2020

flokli commented Jan 1, 2021

raspberrypi: update firmware, linux, tools to latest (~2020-12-01) #107637

raspberrypi: update firmware, linux, tools to latest (~2020-12-01) #107637

Conversation

drewrisinger commented Dec 26, 2020

Motivation for this change

Things done

drewrisinger Dec 26, 2020

Choose a reason for hiding this comment

flokli Dec 27, 2020

Choose a reason for hiding this comment

prusnak left a comment

Choose a reason for hiding this comment

flokli Dec 27, 2020

Choose a reason for hiding this comment

flokli commented Dec 27, 2020

ghost commented Dec 30, 2020

flokli commented Dec 30, 2020

ghost commented Dec 31, 2020 • edited by ghost

ghost commented Dec 31, 2020

ghost commented Dec 31, 2020

flokli commented Jan 1, 2021

ghost commented Dec 31, 2020 •

edited by ghost