installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) #51397

samueldr · 2018-12-02T16:26:33Z

Motivation for this change

Eventually make NixOS work with SBBR and EBBR compliant platforms.

A good first step for distros is to provide a UEFI-bootable image for ARM.

This PR makes the build process for the installer image work for AArch64.

Testing on real-world hardware

You will need an AArch64 machine which can UEFI boot. This is not as onerous as one could think. Mainline u-boot has the ability to boot UEFI-based images (this is an exercise left to the reader).

One of the most popuplar SBC, the Raspberry Pi, in its third major revision, the Raspberry Pi 3B, can boot Tiano Core, which is a reference UEFI implementation, using @andreiw's 64-bit Tiano Core UEFI for the Raspberry Pi 3. This is one of the way I have verified the work.

First, pick an SD card you wan to use as your UEFI firmware, it can be really small, it only needs to fit the Tiano Core build. Format one FAT32 partition on it, name it as you want, I chose $TIANOCORE$ to make sure I don't accidentally use it. Copy the contents of one of the pre-built Tiano Core builds (found under Binary/prebuilt/*/RELEASE) to the root of the FAT32 partition. Eject your SD card and put it in the Raspberry Pi. Ensure it boots to Tiano Core (it should show up a gigantic raspberry with a countdown timer, it will eventually boot the UEFI shell).

Then, burn the AArch64 iso as you would the normal x86_64 iso, (a simple dd). Plug that USB drive to the Raspberry Pi, boot it, it should automatically continue to GRUB. For the graphical console to work on the Raspberry Pi, you will need to add cma=32M to the command line (press e to edit an entry, CTRL+X to boot it.) This PR does not touch the kernel parameters as they are a board-specific quirk.

NOTE: it's possible that the graphical console may fail late in stage-2, before being given control of the system. This is related to the mainline kernel's vc4 modesetting driver not liking it when the Raspberry Pi's own firmware writes to the buffer (probably wrongly oversimplified explanation). If you get some weirdness there (mainly, being shown older boot messages), it's not this PR's fault. Things to try: (1) have a powerful enough, and properly rated power supply (2) add avoid_warnings=1 to the Raspberry Pi's config.txt. It is possible, most times, to recover by switching back and forth between VTs, so CTRL+ALT+F2 followed by CTRL+ALT+F1 may recover it.

Related testing

A bad rough draft of this implementation was tested by @nbp on a SoftIron Overdrive 1000. I would like it if @nbp could test this cleaned-up build. (It should be 99% equivalent.)

Things done

✔️ Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)
- ✔️ NixOS
- ⬜ macOS
- ⬜ other Linux distributions
⬜ Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
⬜ Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
✔️ Tested execution of all binary files (usually in ./result/bin/)
🆖 Determined the impact on package closure size (by running nix path-info -S before and after)
✔️ Assured whether relevant documentation is up to date
✔️ Fits CONTRIBUTING.md.

cc @dezgeg (arm and aarch64 knowledgeable)

nixos/modules/installer/cd-dvd/iso-image.nix

samueldr · 2018-12-03T14:13:50Z

On an AArch64 machine, my Raspberry Pi 3, I tested the iso image booting under qemu-kvm.

# Get a release-18.09 commit with a good cache coverage
git checkout 6ea49f6f8630eab8824daf3542d11fafcbcd525c
nix-build -A OVMF.fd -A qemu
./result/bin/qemu-kvm -nographic \
    --enable-kvm -bios ./result-fd/FV/QEMU_EFI.fd \
    -device e1000,netdev=net0 -netdev user,id=net0 \
    -cpu cortex-a53 -M virt \
    -m 512M \
    -cdrom ~/installer.iso

After a small while qemu will show the grub menu on the serial output.

~~It is possible -cpu needs to be changed to your host architecture (lscpu | grep '^Model name') as AFAIK qemu will neither auto-detect nor start with another architecture.~~ You cna use -cpu host to copy the host's CPU configuration for kvm.

nixos/modules/installer/cd-dvd/iso-image.nix

dezgeg · 2018-12-12T23:14:15Z

nixos/modules/installer/cd-dvd/iso-image.nix

+      throw "Unsupported architecture";
+
+  # Syslinux (and isolinux) only supports x86-based architectures.
+  isx86Compatible = pkgs.stdenv.isi686 || pkgs.stdenv.isx86_64;


Nit: Maybe this could be something like biosBootable or x86BiosBootable? It should be possible to build EFI-only x86 images after all.

Hmm, while you're right for the documented purpose, it's also not right for one feature, whether or not to add the efi_uga module for GRUB. I may as well change how it is added by instead checking for the existence of said module; it isn't entirely right either to say it's an x86 thing.

Does attempting to load efi_uga cause any problems in practice? IIRC it doesn't since install-grub.pl will put it into the final grub.cfg anyway on nixos-rebuild switch and it has worked with the community box.

The issue here is that efi_uga isn't built. Removing the newly added if.

Adding additional modules: - efi_uga /nix/store/d07hfl9whmwjhj4pnajk1yz0fnr51j91-grub-2.02/bin/grub-mkimage: error: cannot open `/nix/store/d07hfl9whmwjhj4pnajk1yz0fnr51j91-grub-2.02/lib/grub/arm64-efi/efi_uga.mod': No such file or directory. builder for '/nix/store/vhwh044bcn4fp1f7hs5hpiixw6m0vhdy-efi-directory.drv' failed with exit code 1 cannot build derivation '/nix/store/17vd0jkmdh7aq3fbdvkcarqc5nm63whh-nixos-19.03.git.ad27b06-aarch64-linux.iso.drv': 1 dependencies couldn't be built error: build of '/nix/store/17vd0jkmdh7aq3fbdvkcarqc5nm63whh-nixos-19.03.git.ad27b06-aarch64-linux.iso.drv' failed

So it's not an issue of attempting to load efi_uga, but more of an issue where since it doesn't exist, grub-mkimage gets mad. I am not putting the other modules in similar conditionals to ensure they exist. If they don't then something is up with the grub build; either grub was updated and a module was removed (which may need manual intervention) or the build was broken.

Do note that the iso is not at all using the plumbing used when using the nixos configuration. So anything using nixos-rebuild switch is not relevant here :/.

samueldr · 2018-12-13T00:30:08Z

Commenting here on a change I made: Yes, I'm using a for loop on on element, but I'm (uselessly?) pre-empting a change where other modules could be conditionally added. I don't like it too much, but otherwise it feels also weird to add only one if to the mix :/.

samueldr requested a review from infinisil as a code owner December 2, 2018 16:26

GrahamcOfBorg added 6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Dec 2, 2018

grahamc reviewed Dec 2, 2018

View reviewed changes

nixos/modules/installer/cd-dvd/iso-image.nix Outdated Show resolved Hide resolved

samueldr changed the title ~~installer: Adds AArch64 UEFI installer support.~~ installer: Adds AArch64 UEFI installer support. (work towards SBBR and EBBR support) Dec 2, 2018

samueldr changed the title ~~installer: Adds AArch64 UEFI installer support. (work towards SBBR and EBBR support)~~ installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) Dec 2, 2018

samueldr force-pushed the feature/aarch64-uefi branch from c1f4dd8 to ad518d3 Compare December 2, 2018 18:56

samueldr force-pushed the feature/aarch64-uefi branch from ad518d3 to fb41dcf Compare December 5, 2018 02:22

dezgeg reviewed Dec 12, 2018

View reviewed changes

nixos/modules/installer/cd-dvd/iso-image.nix Show resolved Hide resolved

dezgeg reviewed Dec 12, 2018

View reviewed changes

nixos/modules/installer/cd-dvd/iso-image.nix Outdated Show resolved Hide resolved

dezgeg reviewed Dec 12, 2018

View reviewed changes

installer: Adds AArch64 UEFI installer support.

ad27b06

samueldr force-pushed the feature/aarch64-uefi branch from fb41dcf to ad27b06 Compare December 13, 2018 00:26

samueldr merged commit 321d48d into NixOS:master Dec 17, 2018

samueldr mentioned this pull request Dec 19, 2018

nixos/release-combined: adds aarch64-linux as supported #52534

Merged

samueldr deleted the feature/aarch64-uefi branch February 12, 2019 02:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) #51397

installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) #51397

samueldr commented Dec 2, 2018 •

edited

samueldr commented Dec 3, 2018 •

edited

dezgeg Dec 12, 2018

samueldr Dec 12, 2018

dezgeg Dec 13, 2018

samueldr Dec 13, 2018 •

edited

samueldr commented Dec 13, 2018

installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) #51397

installer: Adds AArch64 UEFI installer support. (Work towards SBBR and EBBR support) #51397

Conversation

samueldr commented Dec 2, 2018 • edited

Motivation for this change

Testing on real-world hardware

Related testing

Things done

samueldr commented Dec 3, 2018 • edited

dezgeg Dec 12, 2018

Choose a reason for hiding this comment

samueldr Dec 12, 2018

Choose a reason for hiding this comment

dezgeg Dec 13, 2018

Choose a reason for hiding this comment

samueldr Dec 13, 2018 • edited

Choose a reason for hiding this comment

samueldr commented Dec 13, 2018

samueldr commented Dec 2, 2018 •

edited

samueldr commented Dec 3, 2018 •

edited

samueldr Dec 13, 2018 •

edited