Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/install-grub: include child configs in grub menu #45345

Merged
merged 4 commits into from Jul 13, 2019

Conversation

vmandela
Copy link
Contributor

@vmandela vmandela commented Aug 19, 2018

Add configs listed under the fine-tune subdirectory to the grub menu.
Use specified configuration name for the entry if available.

Motivation for this change

I need to switch between slightly different NixOS configurations at Work and Home. Nixos child configuration is perfect for this except I cannot choose the configurations from the boot menu. Child configurations showing up in the grub menu seems to have been disabled in the commit f07f221. I have ported the changes for handling child configurations from bash to perl.

This PR also includes a test nixos/tests/installer-child.nix to verify the functionality.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Fits CONTRIBUTING.md.

Related PR's
  1. How to show configs from nested.clone in the grub menu How to show configs from nested.clone in the grub menu #44495
  2. roaming laptop: network proxy configuration roaming laptop: network proxy configuration #27535

@samueldr
Copy link
Member

The changes are not tested yet. I am trying to create a test based on installer.nix

This is why I have labeled this WIP. Do tell if it is wrong.

@vmandela vmandela force-pushed the grub-child-config branch 2 times, most recently from de60a93 to ba6f22f Compare August 21, 2018 15:19
@vmandela vmandela changed the title [RFC] nixos/install-grub: include child configs in grub menu nixos/install-grub: include child configs in grub menu Aug 24, 2018
@vmandela
Copy link
Contributor Author

@samueldr Please remove the WIP label from the PR. I have added a test and verified it. This PR is ready for review.

@vmandela
Copy link
Contributor Author

@samueldr @srhb Could you please review the PR and/or suggest reviewers ?

  • Thank you

@samueldr
Copy link
Member

samueldr commented Aug 25, 2018

Hi, thanks for the contribution!

First, a friendly note, you did nothing wrong in asking! Though it's not unusual to have a PR aging, like a fine wine, for a couple of days once it is opened. More than a few days without activity then can become a worry. Some of the contributors (all?) work in spurts. I personally had this PR in my sights since the moment I saw it (which is why I labelled it); fixing a long-term regression is great!


Now, to the actual review! I'll be leaving other comments along the way, but I have a first task, an easy one, that needs your collaboration: The whole nixpkgs project is expected to be indented using spaces, two-per-indent-level for .nix files. nixos/tests/installer-child.nix has mixed indents, with a section using tabs.

Expect more comments as now I'm actually reviewing the thing 😄.

@samueldr
Copy link
Member

samueldr commented Aug 25, 2018

There may be an issue with the configuration names.

Given this option in the root configuration: boot.loader.grub.configurationName = "HI THERE ♥";, the default configuration is named Default (instead of the expected HI THERE ♥); the child configuration are named as expected. Though, now that I am re-reading, this may come from the fact that the default configuration (already) was always named "NixOS - Default" and this is not really relevant; though the behaviour is surprising.

configuration.nix
{ config, pkgs, lib, ... }:

{
  imports = [ ./hardware-configuration.nix ];

  # Use the systemd-boot EFI boot loader.
  boot.loader.systemd-boot.enable = false;
  boot.loader.grub = {
    efiSupport = true;
    efiInstallAsRemovable = true;
  };

  users.users.root.initialHashedPassword = "";
  users.users.alice =
  { isNormalUser = true;
    home = "/home/alice";
    description = "Alice Foobar";
    extraGroups = [ "wheel" "networkmanager" ];
    initialHashedPassword = "";
  };
  services.mingetty.autologinUser = "root";
  system.stateVersion = "18.09";

  boot.loader.grub.configurationName = "HI THERE ♥";
  nesting.clone = [
    {
      boot.loader.grub.configurationName = lib.mkForce "Other";

      environment.etc = {
        "gitconfig".text = "
        [core]
        gitproxy = none for work.com
        ";
      };
    }
    {
      #boot.loader.grub.configurationName = lib.mkForce "Otter";
    }
    {
      #boot.loader.grub.configurationName = lib.mkForce "Utter nonsense";
    }

    # Verifying ordering
    {
      #boot.loader.grub.configurationName = lib.mkForce "1";
    }
    {
      #boot.loader.grub.configurationName = lib.mkForce "3";
    }
    {
      #boot.loader.grub.configurationName = lib.mkForce "2";
    }
  ];

  virtualisation.useEFIBoot = true;
}

image

This is somewhat confusing since the configuration in your example is using the name "Default", making this seem like an issue. Let's make a note that this needs to be fixed in another PR, in another time, otherwise it's going to feel weird for the end-users of child configurations. Also keeping this now irrelevant comment up to document this for other testers :).

@vmandela
Copy link
Contributor Author

@samueldr Thanks for the review. I apologize if I was pushy. My only intent was to get some comments over the weekend so that I could address them. Once the work week starts, it becomes difficult to take time out.

I have fixed the indenting in installer-child.nix. Please let me know if there are other changes to be done.

@samueldr
Copy link
Member

Don't worry, I understand why, this was mainly to assuage fears of your PR being forgotten.

I was stopped in the middle of reviewing yesterday, I'm finishing up my points soon.

@samueldr
Copy link
Member

As with the previous implementation, and the last existing implementation, this will not create entries for previous generations; only the entries for the current generations will be made available in the menu,

It is also noted in this 2009 comment.

@samueldr
Copy link
Member

samueldr commented Aug 26, 2018

Hi, I think the nixos/tests/installer-child.nix file was inadvertently replaced with an older revision without the actual tests verifying the child config.

Here's the diff from the previous revision in case you lost it.

Here's the diff from current state to previous state for that file.

I will continue reviewing the installer child test as it was; when it was actually testing the feature.

Copy link
Member

@samueldr samueldr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The installer-child.nix file is mostly a copy of the installer.nix file. This seems a bit wasteful, and will not be able to get value from updates to the installer.nix tests.

It should be possible to integrate the child configurations tests directly into installer.nix; using the appropriate hooks to add the additional test commands. Though, I must think you already explored the possibility: were there anything stopping you from integrating? What needs to be done to integrate them better?

Furthermore, I would like to see an additional test for the child configurations using EFI grub; in theory there shouldn't be any differences, but building both tests ensures it will also work on EFI.


As for install-grub.pl, the changes are minimal, and seems to replicate the previous behaviour, which is assumed to be correct for this review.

This will create a bigger feature-support gap between grub and the other bootloaders, but it shouldn't really concern you an neither concern this PR, though it maybe will need to be somehow documented.

@vmandela vmandela force-pushed the grub-child-config branch 2 times, most recently from 627805b to bbe8bf2 Compare August 27, 2018 08:54
@vmandela
Copy link
Contributor Author

The installer-child.nix file is mostly a copy of the installer.nix file. This seems a bit wasteful, and will not be able to get value from updates to the installer.nix tests.

It should be possible to integrate the child configurations tests directly into installer.nix; using the appropriate hooks to add the additional test commands. Though, I must think you already explored the possibility: were there anything stopping you from integrating? What needs to be done to integrate them better?

I agree that including these tests in installer.nix would be a better approach.

On my system, test for one configuration takes roughly 45 mins. Most the time in the test is spent in the nixos-install step. Due to the turn around time, I opted for a minimal test case rather than integrating the test into installer.nix. I am running nixos in a virtualbox guest on a Windows 10 host. I dont know if this configuration is contributing to the large test runtime.

Furthermore, I would like to see an additional test for the child configurations using EFI grub; in theory there shouldn't be any differences, but building both tests ensures it will also work on EFI.

I can add this.

@samueldr
Copy link
Member

On my system, test for one configuration takes roughly 45 mins. Most the time in the test is spent in the nixos-install step. Due to the turn around time, I opted for a minimal test case rather than integrating the test into installer.nix. I am running nixos in a virtualbox guest on a Windows 10 host. I dont know if this configuration is contributing to the large test runtime.

Yes it is! VirtualBox does not support nested virtualization, so you're getting pure emulation for the tests! (You're a patient individual!)


If the issue is about running the other tests, when running the tests for installer.nix you are not forced to run them all. Using nix-build on the test file, you can give an attribute name using -A to select one test to run.

.../nixpkgs $ nix-build nixos/tests/installer.nix -A testNameHere

@infinisil
Copy link
Member

Looks good to me

@samueldr What about you?

@vmandela
Copy link
Contributor Author

rebased.

@vmandela
Copy link
Contributor Author

@samueldr pinging for review

@vmandela
Copy link
Contributor Author

vmandela commented Jul 8, 2019

@samueldr pinging for review.

samueldr
samueldr previously approved these changes Jul 9, 2019
@samueldr samueldr dismissed their stale review July 9, 2019 22:26

approved too quickly... looking at one thing still

@samueldr
Copy link
Member

samueldr commented Jul 9, 2019

@GrahamcOfBorg test installer.simpleUefiGrub

Here it fails with:

ub.drv'...
starting VDE switch for network 1
running the VM test script
error: syntax error at (eval 21) line 115, near "() "
syntax error at (eval 21) line 148, near ";
}"
(0.00 seconds)
syntax error at (eval 21) line 115, near "() "
syntax error at (eval 21) line 148, near ";
}"
vde_switch: EOF on stdin, cleaning up and exiting
cleaning up
(0.00 seconds)
builder for '/nix/store/qb2j9m2gikch879ipfrgbgixlgyvr5l5-vm-test-run-installer-simpleUefiGrub.drv' failed with exit code 2

The syntax errors happening in the newly added branch in the tests. The clone tests do work, though.

Copy link
Member

@samueldr samueldr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(See previous comment)

@samueldr
Copy link
Member

samueldr commented Jul 9, 2019

Making the whole block of code conditional in nix, instead of in perl, fixes the issue. (Though, reformat the added block accordingly too.)

diff --git a/nixos/tests/installer.nix b/nixos/tests/installer.nix
index 3218475c38a..93479134712 100644
--- a/nixos/tests/installer.nix
+++ b/nixos/tests/installer.nix
@@ -188,8 +188,8 @@ let
       $machine->waitForUnit("network.target");
       $machine->shutdown;

+      ${optionalString testCloneConfig ''
       # Tests for validating clone configuration entries in grub menu
-      if (${toString testCloneConfig}) {
         # Reboot Machine
         $machine = createMachine({ ${hdFlags} qemuFlags => "${qemuFlags}", name => "clone-default-config" });
         ${preBootCommands}
@@ -222,7 +222,7 @@ let
         $machine->succeed("test -e /etc/gitconfig");

         $machine->shutdown;
-      }
+      ''}

     '';

(And to test, just in case, nix-build nixos/release.nix -A tests.installer.simpleUefiGrubClone.x86_64-linux and nix-build nixos/release.nix -A tests.installer.simpleUefiGrub.x86_64-linux)

@vmandela
Copy link
Contributor Author

@samueldr Thanks. I will make the changes.

Add configs listed under the fine-tune subdirectory to the grub menu.
Use specified configuration name for the entry if available.
- Create a child configuration named "Work" with an extra config file.
- Name the default configuration as "Home" :-)
- Once the VM is setup, reboot and verify that it has booted into
default configuration.
- Reboot into the "Work" configuration via grub.
- Verify that we have booted into the "Work" configuration and that
the extra config file is present.

This test works for the simple grub configuration and simple UEFI
Grub configuration. UEFI Systemd is not included in the test.
@vmandela
Copy link
Contributor Author

vmandela commented Jul 11, 2019

@GrahamcOfBorg test simpleUefiGrub
@GrahamcOfBorg test simpleUefiGrubClone
@GrahamcOfBorg test simple
@GrahamcOfBorg test simpleClone

@worldofpeace
Copy link
Contributor

@GrahamcOfBorg test simpleUefiGrub
@GrahamcOfBorg test simpleUefiGrubClone
@GrahamcOfBorg test simple
@GrahamcOfBorg test simpleClone

@vmandela need to have builder privileges to use this

@vmandela
Copy link
Contributor Author

@worldofpeace Thanks for launching the build. I did not know of the builder privileges.

@JohnAZoidberg
Copy link
Member

@vmandela You can get privileges by adding yourself to here https://github.com/NixOS/ofborg/blob/released/config.extra-known-users.json and creating a PR.

@samueldr
Copy link
Member

@GrahamcOfBorg test installer.simpleUefiGrub
@GrahamcOfBorg test installer.simpleUefiGrubClone
@GrahamcOfBorg test installer.simple
@GrahamcOfBorg test installer.simpleClone

@infinisil infinisil merged commit 2d7bce2 into NixOS:master Jul 13, 2019
@samueldr
Copy link
Member

Thank you for the contribution, and you patience, @vmandela.

@vmandela
Copy link
Contributor Author

@samueldr @infinisil Thanks for the reviews. Glad to have this merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants