nixos/hardware.deviceTree: new module #60422

kwohlfahrt · 2019-04-29T13:36:59Z

Motivation for this change

Add support for custom device-tree files, as well as configuring overlays applied to device-trees. These are needed in embedded devices to expose non-discoverable hardware to the kernel, and are common on Raspberry Pi and similar.

This does not include support for Raspberry Pi style device-tree parameters (dtparam), as there does not seem to be a command-line tool available to apply these parameters to an overlay (if there is one in nixpkgs, I'm happy to extend this PR to include support).

@samueldr was particularly interested on IRC.

An example configuration might be:

hardware.deviceTree = {
  enable = true;
  base = pkgs.deviceTree.raspberryPiDtbs;
  overlays = [ "${pkgs.deviceTree.raspberryPiOverlays}/w1-gpio.dtbo" ];
};

Things done

This can be used to build a sd-image, that contains appropriately modified device-tree files. I'll test the result over the next few days, but would appreciate some comments on the API in the meantime.

I'm particularly not sure about how things should be split between nixos and nixpkgs, and whether I've understood the options interface correctly.

artemist · 2019-04-29T15:11:21Z

pkgs/os-specific/linux/device-tree.nix

+    '';
+  });
+
+  raspberryPiDtbs = stdenvNoCC.mkDerivation {


Are you sure you want to put the raspberry pi overlays here? This package could grow a bit large if you start adding overlays for other boards.

Nope, I'm not at all sure. Where would you suggest they go?

I would probably put them in separate nix files in pkgs/os-specific/linux/overlays (or something like that), but I'd like a second opinion before I suggest you do that.

I've moved them - does it look better now?

samueldr · 2019-04-29T23:12:50Z

pkgs/os-specific/linux/device-tree.nix

@@ -0,0 +1,28 @@
+{ stdenvNoCC, dtc, findutils, raspberrypifw }:


Somewhat related to @artemist's comment:

Nothing in pkgs/os-specific/linux/ is directly at the root, everything ends up in sub-folder. The generic tooling implementation (applyOverlays) would be fine in default.nix in that sub-folder.

Additionally, the kernel folder can be used as an inspiration, where linux-rpi.nix provides the kernel for the raspberry pi family. The raspberry pi foundation's device tree things could be found under pkgs/os-specific/linux/device-tree/raspberrypi.nix for example.

I've moved these files as well.

samueldr · 2019-04-29T23:18:38Z

pkgs/top-level/all-packages.nix

@@ -159,6 +159,8 @@ in

  demoit = callPackage ../servers/demoit { };

+  deviceTree = callPackage ../os-specific/linux/device-tree.nix {};


I don't know what's the better implementation for the "API" in all-packages.nix for providing device trees.

I'm thinking that yes, helpers like applyOverlays is fine under deviceTree, but thinking that the raspberry pi blobs (and possibly others in the future) shouldn't be found under that name. Just like we don't have linux.v4_4 but linux_4_4. raspberryPiDtbs and raspberryPiOverlays might be better names, but suggestions welcome.

I've moved these packages, does the new organization look better?

nixos/modules/system/boot/loader/generic-extlinux-compatible/extlinux-conf-builder.sh

samueldr · 2019-04-29T23:26:06Z

👋 Hi! Amazing contribution.

Some minor changes asked, and some questions left unanswered, but at a glance this looks good.

I guess I need to set myself up to test soon :).

kwohlfahrt · 2019-04-30T10:09:15Z

Spoiler alert: the example suggested doesn't work as-is.

The initial problem is that the .dtb file in raspberrypifw is not named according to the uboot convention. I will fix this for the raspberryPiDtbs derivation (see pkgs/os-specific/linux/kernel/linux-rpi.nix for similar). EDIT: In fact, this should probably be fixed in uboot-builder.nix, not linux-rpi.nix, since it's a uboot specific problem.

Secondly, once the w1-gpio overlay is enabled, the boot process hangs after loading the kernel. This could be a hardware issue though, I haven't confirmed that my 1-wire device works on stock raspbian.

samueldr · 2019-04-30T13:26:40Z

Mixing raspberry pi foundation device trees with mainline kernels may not work. I am not 100% sure though. From what I understand, while they describe the same hardware, the way they are described in the targeted Linux kernel versions is different.

I know that, at least with a dtbo, there were changes required for it to begin kind of working (it failed due to what I assume were other issues to check).

kwohlfahrt · 2019-05-01T16:39:39Z

Interesting, I thought I'd compared the .dts files in both kernels and found them to be identical. And then compiled them to .dtb and found them to also be identical to raspberrypifw with the exception of the __symbols__ section. But I'll check that again, and also try with a RPi kerne.

rehno-lindeque · 2019-05-02T21:12:56Z

nixos/modules/hardware/device-tree.nix

+
+  config = mkIf (cfg.enable) {
+    hardware.deviceTree.package = if (cfg.overlays != [])
+      then pkgs.deviceTree.applyOverlays cfg.base cfg.overlays else cfg.base;


This prevents you from setting package manually. I believe that if if package needs to be settable as an option there probably needs to be some kind of logic like

mkIf (cfg.enable) (mkIf (cfg.overlays != [] || cfg.base != null) { hardware.deviceTree.package = ...; })

I.e. With just package set I'm getting

The unique option `hardware.deviceTree.package' is defined multiple times, in `...' and `/nix/store/.../nixos/modules/hardware/device-tree.nix'

I don't think package needs to bet settable - if overlays is empty, then base just becomes package. Is making internal = true; the correct way of making it not settable?

Is making internal = true; the correct way of making it not settable?

I'm not sure, possibly? Should it be exposed via mkOption at all? I imagined that it would have just been a simple let binding if it wasn't intended to be exposed as a settable option, but I could be wrong about that.

let package = if (cfg.overlays != []) then pkgs.deviceTree.applyOverlays cfg.base cfg.overlays else cfg.base; in ...

kwohlfahrt · 2019-05-09T08:49:20Z

I have moved the packages containing device-tree files. I'll next test exactly what the compatibility is between linux kernels, rpi kernels and raspberrypifw. If there are significant issues, we may want to add a deviceTrees to kernelPackages_xyz, so we can have easy access to .dtbs matching a particular kernel?

edolstra · 2019-05-09T11:18:28Z

pkgs/top-level/all-packages.nix

@@ -159,6 +159,10 @@ in

  demoit = callPackage ../servers/demoit { };

+  deviceTree = callPackage ../os-specific/linux/device-tree {};


No camelCase in package attributes please.

deviceTree is not a package, it currently only contains the applyOverlays attribute, which is for applying overlays to derivations containing device-tree files. Should this still be renamed?

deviceTree_rpi is a package, I will rename this to device-tree_rpi (to parallel linux_rpi) unless somebody suggests a better name.

kwohlfahrt · 2019-05-15T22:19:47Z

I've tested that the following configurations work on my Raspberry Pi (with the u-boot loader):

With linuxPackages_4_19:

defaults
deviceTree.enable = true;
linuxPackages_4_19 does not work with deviceTree.base = pkgs.device-tree_rpi. Should there be any checks for compatibility, or should we just document that setting deviceTree.base may give an unbootable system?

With linuxPackages_rpi:

defaults
deviceTree.enable = true
deviceTree.base = pkgs.device-tree_rpi seems to be OK.
the w1-gpio1 overlay works, and my device is available (yay! that was the point of this for me)

Edit: The first time I tried deviceTree.base it I couldn't get SSH access, but when it restarted plugged into my monitor & keyboard it came up fine. Moved back to it's usual headless location it has worked since, without any changes, so I'm putting that one down to a network issue.

kwohlfahrt · 2019-05-18T16:13:53Z

@samueldr could you have another look at this? I think I have addressed all the comments and it is ready for testing/review now that it is working for me.

I would maybe like to move the renaming bcm*.dtb from linux-rpi.nix and device-tree/raspberrypi.nix into a shared place, maybe in the device-tree folder?

(re-reviewing)

Add support for custom device-tree files, and applying overlays to them. This is useful for supporting non-discoverable hardware, such as sensors attached to GPIO pins on a Raspberry Pi.

In response to comments, create a sub-folder for deviceTree packages (starting with rpi), and a top-level package for helpers.

This is just as messy as it is for the kernel files. Maybe it should be done in the uboot boot-loader?

kwohlfahrt · 2019-08-16T11:36:04Z

@infinisil and @samueldr, is anything preventing this from being merged? I'd like to have it in for 19.09...

I've been running this on my rpi3 with a temperature sensor for several months with no issues.

samueldr · 2019-08-16T17:26:12Z

Nothing, except being unexpectedly busy lately. Thanks for the contribution!

nixos-discourse · 2020-09-23T23:13:50Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/enabling-spi-on-the-raspberry-pi-3/9122/3

kwohlfahrt requested a review from infinisil as a code owner April 29, 2019 13:36

kwohlfahrt force-pushed the device-tree branch from 122779c to a8a1414 Compare April 29, 2019 13:37

ofborg bot added 6.topic: nixos 8.has: module (update) labels Apr 29, 2019

kwohlfahrt force-pushed the device-tree branch from a8a1414 to 589efe4 Compare April 29, 2019 14:02

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Apr 29, 2019

kwohlfahrt force-pushed the device-tree branch from 589efe4 to a79561d Compare April 29, 2019 15:05

artemist reviewed Apr 29, 2019

View reviewed changes

samueldr previously requested changes Apr 29, 2019

View reviewed changes

rehno-lindeque reviewed May 2, 2019

View reviewed changes

kwohlfahrt force-pushed the device-tree branch from 264637e to 494bb18 Compare May 9, 2019 08:50

ofborg bot added 8.has: package (new) 10.rebuild-linux: 1-10 and removed 10.rebuild-linux: 0 labels May 9, 2019

edolstra reviewed May 9, 2019

View reviewed changes

kwohlfahrt force-pushed the device-tree branch from 494bb18 to da06351 Compare May 15, 2019 21:21

kwohlfahrt force-pushed the device-tree branch 2 times, most recently from 61affde to d57af83 Compare May 18, 2019 16:21

kwohlfahrt force-pushed the device-tree branch from d57af83 to 62c6552 Compare June 4, 2019 21:46

kwohlfahrt force-pushed the device-tree branch 2 times, most recently from d813aec to a2b8050 Compare June 19, 2019 18:30

kwohlfahrt force-pushed the device-tree branch from a2b8050 to d0834d9 Compare June 24, 2019 22:17

kwohlfahrt force-pushed the device-tree branch from d0834d9 to 32595da Compare July 5, 2019 08:55

samueldr approved these changes Jul 23, 2019

View reviewed changes

kwohlfahrt force-pushed the device-tree branch from 32595da to 1bf5644 Compare August 1, 2019 15:39

kwohlfahrt and others added 3 commits August 7, 2019 13:51

nixos/hardware.deviceTree: new module

dd0a951

Add support for custom device-tree files, and applying overlays to them. This is useful for supporting non-discoverable hardware, such as sensors attached to GPIO pins on a Raspberry Pi.

nixos/hardware.deviceTree: Move things around

28cf80a

In response to comments, create a sub-folder for deviceTree packages (starting with rpi), and a top-level package for helpers.

device-tree_rpi: fixup dtb names

766e095

This is just as messy as it is for the kernel files. Maybe it should be done in the uboot boot-loader?

kwohlfahrt force-pushed the device-tree branch from 1bf5644 to 766e095 Compare August 7, 2019 12:51

samueldr merged commit b750ebf into NixOS:master Aug 16, 2019

kwohlfahrt deleted the device-tree branch August 19, 2019 19:31

samueldr mentioned this pull request Sep 24, 2019

Raspberry Pi 3 Support #22014

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/hardware.deviceTree: new module #60422

nixos/hardware.deviceTree: new module #60422

kwohlfahrt commented Apr 29, 2019 •

edited

Loading

artemist Apr 29, 2019

kwohlfahrt Apr 29, 2019

artemist Apr 29, 2019

kwohlfahrt May 9, 2019

samueldr Apr 29, 2019

kwohlfahrt May 9, 2019

samueldr Apr 29, 2019

kwohlfahrt May 9, 2019

samueldr commented Apr 29, 2019 •

edited

Loading

kwohlfahrt commented Apr 30, 2019 •

edited

Loading

samueldr commented Apr 30, 2019

kwohlfahrt commented May 1, 2019

rehno-lindeque May 2, 2019 •

edited

Loading

kwohlfahrt May 4, 2019

rehno-lindeque May 5, 2019

kwohlfahrt commented May 9, 2019

edolstra May 9, 2019

kwohlfahrt May 15, 2019

kwohlfahrt commented May 15, 2019 •

edited

Loading

kwohlfahrt commented May 18, 2019

kwohlfahrt commented Aug 16, 2019

samueldr commented Aug 16, 2019

nixos-discourse commented Sep 23, 2020

		@@ -0,0 +1,28 @@
		{ stdenvNoCC, dtc, findutils, raspberrypifw }:

		@@ -159,6 +159,8 @@ in

		demoit = callPackage ../servers/demoit { };

		deviceTree = callPackage ../os-specific/linux/device-tree.nix {};

		@@ -159,6 +159,10 @@ in

		demoit = callPackage ../servers/demoit { };

		deviceTree = callPackage ../os-specific/linux/device-tree {};

nixos/hardware.deviceTree: new module #60422

nixos/hardware.deviceTree: new module #60422

Conversation

kwohlfahrt commented Apr 29, 2019 • edited Loading

Motivation for this change

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samueldr commented Apr 29, 2019 • edited Loading

kwohlfahrt commented Apr 30, 2019 • edited Loading

samueldr commented Apr 30, 2019

kwohlfahrt commented May 1, 2019

rehno-lindeque May 2, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kwohlfahrt commented May 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kwohlfahrt commented May 15, 2019 • edited Loading

kwohlfahrt commented May 18, 2019

kwohlfahrt commented Aug 16, 2019

samueldr commented Aug 16, 2019

nixos-discourse commented Sep 23, 2020

kwohlfahrt commented Apr 29, 2019 •

edited

Loading

samueldr commented Apr 29, 2019 •

edited

Loading

kwohlfahrt commented Apr 30, 2019 •

edited

Loading

rehno-lindeque May 2, 2019 •

edited

Loading

kwohlfahrt commented May 15, 2019 •

edited

Loading