Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

modules/virtualisation/qemu-vm: always enable serial console #40004

Merged
merged 5 commits into from May 7, 2018
Merged

modules/virtualisation/qemu-vm: always enable serial console #40004

merged 5 commits into from May 7, 2018

Conversation

flokli
Copy link
Contributor

@flokli flokli commented May 5, 2018

Motivation for this change

Don't make serial console activation dependent on graphical.
Even in qemu graphical mode, you can switch to the serial console via
Ctrl+Alt+3.

With that being done, you also don't need to specify
systemd.services."serial-getty@ttyS0".enable = true; either as described in
https://nixos.wiki/wiki/Cheatsheet#Building_a_service_as_a_VM_.28for_testing.29,
as systemd automatically spawns a getty on consoles passwd via cmdline.

This also means, vms built by nixos-rebuild build-vm can simply be run
properly in nographic mode by appending -nographic to result/bin/run-*-vm,
without the need to explicitly add platform-specific QEMU_KERNEL_PARAMS.

/cc @dezgeg

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@flokli
Copy link
Contributor Author

flokli commented May 5, 2018

@GrahamcOfBorg test ipv6

@GrahamcOfBorg
Copy link

Failure on x86_64-linux (full log)

Attempted: tests.ipv6

Partial log (click to expand)

error: the VM quit before connecting
the VM quit before connecting
cleaning up
killing server (pid 660)
killing router (pid 593)
killing client (pid 672)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/g83pjmqqybvsxiiwcdzz1mnwl4vs5acx-vm-test-run-ipv6.drv' failed with exit code 255
error: build of '/nix/store/g83pjmqqybvsxiiwcdzz1mnwl4vs5acx-vm-test-run-ipv6.drv' failed

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: tests.ipv6

Partial log (click to expand)

server: exit status 0
3 out of 3 tests succeeded
test script finished in 48.67s
cleaning up
killing router (pid 627)
killing client (pid 646)
killing server (pid 659)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/fjnlv11dk71zplp36j5mg455wlca6dz9-vm-test-run-ipv6

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: qemu

Partial log (click to expand)

/nix/store/88qv0sp9id8cwdn9nswjakf7qqzwdj0y-qemu-2.12.0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: qemu

Partial log (click to expand)

these paths will be fetched (46.50 MiB download, 380.56 MiB unpacked):
  /nix/store/578aifl9n3k0bp0m7lbapwhawpv0599g-qemu-2.12.0
copying path '/nix/store/578aifl9n3k0bp0m7lbapwhawpv0599g-qemu-2.12.0' from 'https://cache.nixos.org'...
/nix/store/578aifl9n3k0bp0m7lbapwhawpv0599g-qemu-2.12.0

@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: qemu

Partial log (click to expand)

  /nix/store/nzp6p9gnyr0764l9w6hp38hvjsndm150-libpcap-1.8.1
  /nix/store/piv0dr0d8k3i04k7rgvx65cmd51r22s1-qemu-2.12.0
  /nix/store/qwg3hxrmqk1dcb58l7pnzvr73s9zy7xd-snappy-1.1.7
  /nix/store/v3qkzl4wckyqrzv9wgfijwhj7f4qqjkd-vde2-2.3.2
copying path '/nix/store/2lgzwx884ndzppqj8n5y9wcic8mh1xvh-lzo-2.10' from 'https://cache.nixos.org'...
copying path '/nix/store/qwg3hxrmqk1dcb58l7pnzvr73s9zy7xd-snappy-1.1.7' from 'https://cache.nixos.org'...
copying path '/nix/store/nzp6p9gnyr0764l9w6hp38hvjsndm150-libpcap-1.8.1' from 'https://cache.nixos.org'...
copying path '/nix/store/v3qkzl4wckyqrzv9wgfijwhj7f4qqjkd-vde2-2.3.2' from 'https://cache.nixos.org'...
copying path '/nix/store/piv0dr0d8k3i04k7rgvx65cmd51r22s1-qemu-2.12.0' from 'https://cache.nixos.org'...
/nix/store/piv0dr0d8k3i04k7rgvx65cmd51r22s1-qemu-2.12.0

@dezgeg
Copy link
Contributor

dezgeg commented May 5, 2018

Sounds like a good idea. Does this affect the graphical usage though? I have some recollection that for some reason or another, I needed to specify console=ttyS0,115200n8 console=tty0 and in this specific order for things to work in both cases.

@flokli
Copy link
Contributor Author

flokli commented May 5, 2018 •
edited

Right, order is important, and I missed that :-)

Setting multiple console= will cause those consoles to be enabled, but kernel messages will normally only be sent to the preferred console (the last one passed), which will be the one /dev/console is pointing to (that will be the one normally used for initrd and systemd output).

So I fear this is also platform-specific, or what is tty0 pointing to on qemu aarch32/aarch64 'virt' board?

Regarding /dev/console: We should probably should just use /dev/console in modules/system/boot/stage-1-init.sh instead of what we currently do (tty1 for stage1, and parsing cmdline for stage2) /cc @edolstra

@dezgeg
Copy link
Contributor

dezgeg commented May 5, 2018

tty0 should always be the first virtual console regardless of platform... which in the case of QEMU for ARM/AArch64 will point to nothingness, until you plug in a emulated graphics card.

Kernel messages will be sent to all passed consoles, but only the last one will be the one /dev/console is pointing to (that will be the one normally used for initrd and systemd output).

Right, that was the problem; it's actually impossible to have e.g. the initrd ask for a passphrase and be able to respond to it from either the virtual console or the serial console.

Fortunately, most of the time the user doesn't need to interact with stage-1, just see its output. In fact stage-1 already tries to capture the output so it ends up in journalctl: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh#L89. Maybe that code could be extended so that it shows up on the other consoles as well. (In fact, I don't understand why that isn't happening with the current code, given it's writing to /dev/kmsg).

@flokli
Copy link
Contributor Author

flokli commented May 5, 2018

I changed the PR. Now consoles on tty0 and $qemuSerialDevice will be enabled, with tty0 being the preferred one - as long as cfg.graphics is true (the default). Otherwise, the console order will be reversed.

flokli added 2 commits May 5, 2018 19:43
@flokli
modules/virtualisation/qemu-vm: always enable serial console
4ab293a 
Always enable both tty and serial console, but set preferred console
depending on cfg.graphical.
Even in qemu graphical mode, you can switch to the serial console via
Ctrl+Alt+3.

With that being done, you also don't need to specify
`systemd.services."serial-getty@ttyS0".enable = true;` either as described in
https://nixos.wiki/wiki/Cheatsheet#Building_a_service_as_a_VM_.28for_testing.29,
as systemd automatically spawns a getty on consoles passwd via cmdline.

This also means, vms built by 'nixos-rebuild build-vm' can simply be run
properly in nographic mode by appending `-nographic` to `result/bin/run-*-vm`,
without the need to explicitly add platform-specific QEMU_KERNEL_PARAMS.
@flokli
modules/virtualisation/qemu-vm: set console baudrate to 115200
d19c12d
@GrahamcOfBorg
Copy link

Success on x86_64-darwin (full log)

Attempted: qemu

Partial log (click to expand)

/nix/store/piv0dr0d8k3i04k7rgvx65cmd51r22s1-qemu-2.12.0

@GrahamcOfBorg
Copy link

Success on aarch64-linux (full log)

Attempted: qemu

Partial log (click to expand)

/nix/store/578aifl9n3k0bp0m7lbapwhawpv0599g-qemu-2.12.0

@GrahamcOfBorg
Copy link

Success on x86_64-linux (full log)

Attempted: qemu

Partial log (click to expand)

  /nix/store/16fdpxdyx3rws7n3zw6fpmcch4hj1qaj-spice-0.13.3
  /nix/store/88qv0sp9id8cwdn9nswjakf7qqzwdj0y-qemu-2.12.0
  /nix/store/8hcwnmgmmw6a3hsw411xa9j3swa05idl-vte-0.52.1
  /nix/store/c8i4flxb5qc31v6zi1jpmd7512amyms8-gtk+3-3.22.29
copying path '/nix/store/c8i4flxb5qc31v6zi1jpmd7512amyms8-gtk+3-3.22.29' from 'https://cache.nixos.org'...
copying path '/nix/store/01rlpkwczfnrda2nk2bjm60jcdals8v4-cyrus-sasl-2.1.26' from 'https://cache.nixos.org'...
copying path '/nix/store/8hcwnmgmmw6a3hsw411xa9j3swa05idl-vte-0.52.1' from 'https://cache.nixos.org'...
copying path '/nix/store/16fdpxdyx3rws7n3zw6fpmcch4hj1qaj-spice-0.13.3' from 'https://cache.nixos.org'...
copying path '/nix/store/88qv0sp9id8cwdn9nswjakf7qqzwdj0y-qemu-2.12.0' from 'https://cache.nixos.org'...
/nix/store/88qv0sp9id8cwdn9nswjakf7qqzwdj0y-qemu-2.12.0

@dezgeg
Copy link
Contributor

dezgeg commented May 5, 2018

This one sounds good to me.

BTW, regarding /dev/console, most other initrd implementations don't use it either, due to e.g. this (in Debian initramfs-tools):

commit 619f2caded3910e5be4338c7b01d183ed4484eb6
Author: Ben Hutchings <ben@decadent.org.uk>
Date:   Wed Dec 9 18:54:59 2015 +0000

    scripts/functions: Connect panic shell to normal tty device
    
    Currently the panic shell's controlling tty is /dev/console which is
    not fully functional - the shell can't provide job control and more
    can't work out the screen size for paging.
    
    Fix this by reading /proc/consoles to find out the underlying tty
    device and then connecting the shell to it directly with the aid of
    setsid.
    
    Closes: #512679
    Signed-off-by: Ben Hutchings <ben@decadent.org.uk>

diff --git a/scripts/functions b/scripts/functions
index 531d1bd..67ce319 100644
--- a/scripts/functions
+++ b/scripts/functions
@@ -33,6 +33,8 @@ log_end_msg()
 
 panic()
 {
+       local console rest
+
        if command -v chvt >/dev/null 2>&1; then
                chvt 1
        fi
@@ -48,7 +50,20 @@ panic()
 
        run_scripts /scripts/panic
 
-       REASON="$@" PS1='(initramfs) ' /bin/sh -i </dev/console >/dev/console 2>&1
+       # Try to use setsid, which will enable job control in the shell
+       # and paging in more
+       if command -v setsid >/dev/null 2>&1; then
+               read console rest </proc/consoles
+               if [ "${console}" = "tty0" ]; then
+                       # Need to choose a specific VT
+                       console="tty1"
+               fi
+               # We don't have 'setsid -c' so we need to setsid, open
+               # the tty, and finally exec an interactive shell
+               REASON="$@" PS1='(initramfs) ' setsid sh -c "exec sh -i <>/dev/${console} 1>&0 2>&1"
+       else
+               REASON="$@" PS1='(initramfs) ' sh -i </dev/console >/dev/console 2>&1
+       fi
 }
 
 maybe_break()

@flokli
Copy link
Contributor Author

flokli commented May 5, 2018

This explains it. Thanks!

Additionally, I misread stage-1-init.sh - code using /dev/${console} lives inside a fail() function, and in almost all cases, the console variable already mirrors the preferred console when being called - with the exception of a boot.debug1 or debug1 passed via cmdline before the console= parameters.

@dezgeg dezgeg merged commit 7499e4a into NixOS:master May 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
6.topic: nixos 8.has: module (update) 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@flokli @GrahamcOfBorg @dezgeg