nixos/tests: follow-up to the closure reduction PR #101598

andir · 2020-10-24T20:40:11Z

Motivation for this change

This is a follow-up on #49403 where I made a big oversight regarding the QEMU executable that is being described within the NixOS module system. That package actually drives the VMs that are being started. I previously only changed the QEMU package that was used within the test driver script.

I've tried to detail the changes in the commits. Please let me know if there are any open questions and/or remarks.

While working on this I noticed that in 5500dc8 we introduced the --keep-vm-state flag and defaulted to that flag not being set. This lead to the runInMachine tests not longer
working and that going unnoticed for quite some time now.

cc @Ma27 @rnhmjoj #101473

nixos/lib/testing-python.nix

rnhmjoj · 2020-10-24T21:46:33Z

Can you change nixos-build-vms to use the driverInterative? It should work, now.

andir · 2020-10-24T23:19:21Z

@rnhmjoj that is actually already the case. Due to the changes that @Ma27 merges the nixos-build-vms output defaults to the fullfeatured QEMU even thought the driver Attribute is used.

xaverdh · 2020-10-25T07:07:04Z

so does / should this resolve the eval error currently present on unstable-small ?

rnhmjoj · 2020-10-25T08:42:10Z

@andir yes, but do you think that driverInteractive is more explicit that virtualisation.qemu.package = pkgs.qemu;? From this line alone it's not clear what's going on.

andir · 2020-10-25T11:17:48Z

Can you linke me that error or quote it? Not sure what that error is.

…

On Sun, 25 Oct 2020, 08:07 xaverdh, ***@***.***> wrote: so does / should this resolve the eval error currently present on unstable-small ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#101598 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE365D3PZGXLMFJTPJ2GZLSMPFCHANCNFSM4S52PVRQ> .

xaverdh · 2020-10-25T11:41:11Z

error: "error: --- ThrownError --- hydra-eval-jobs\nThe option virtualisation.qemu' does not exist. Definition values:\n- In /nix/store/zp8k91zbq2rpj5ky8r629zw2z85s3rw4-source/nixos/modules/testing/test-instrumentation.nix':\n {\n consoles = [ ];\n package = {\n _type = "override";\n content = <derivation /nix/store/q72h2cdcb9zjgiay5gdgzwddjkbjr7xq-qemu-host-cpu-only-for-vm-tests-5.1.0.drv>;\n ..."

(from https://hydra.nixos.org/jobset/nixos/unstable-small#tabs-errors)

xaverdh · 2020-10-25T11:43:08Z

Not sure if that is indeed what is preventing eval though. There is also the mysterious

error: [json.exception.type_error.302] type must be string, but is null

at the tail of that log..

andir · 2020-10-25T11:51:49Z

I belive the fromer could be fixed (will verify) the latter is (hopefully) unrelated as master has been stuck for a while now.

Funnily enough my hydra instance was able to eval the release jobset of this PR just fine. I'll also let it build the small release set to see if that also succeeds.

andir · 2020-10-25T12:00:14Z

@andir yes, but do you think that driverInteractive is more explicit that virtualisation.qemu.package = pkgs.qemu;? From this line alone it's not clear what's going on.

You might have a good point there. So would it be clearer if we switched the nixos-build-vms expression to driverInteractive and just remove the explicit override in the nix-build-vms module?

andir · 2020-10-25T12:10:20Z

Oh yeah indeed. The small jobset is failing with the same eval error here:

hydra-eval-jobs returned exit code 1:
error: "error: --- ThrownError --- hydra-eval-jobs\nThe option `virtualisation.qemu' does not exist. Definition values:\n- In `/nix/store/c9mv62dd7r442yiigamw4q1xwx73pfi0-source/nixos/modules/testing/test-instrumentation.nix':\n    {\n      consoles = [ ];\n      package = {\n        _type = \"override\";\n        content = <derivation /nix/store/q72h2cdcb9zjgiay5gdgzwddjkbjr7xq-qemu-host-cpu-only-for-vm-tests-5.1.0.drv>;\n    ..."
error: [json.exception.type_error.302] type must be string, but is null

I managed to reproduce it locally with this command: nix-instantiate nixos/release-small.nix -A nixos.tests.boot.biosCdrom.x86_64-linux.

I'll have a look why this test is complaining while others are just fine.

xaverdh · 2020-10-25T12:16:58Z

I think it is due to #101246 actually..

andir · 2020-10-25T12:18:41Z

I think it is due to #101246 actually..

Yeah, I just started fixing that. The problem is because the virtualisation.qemu options do not always exist and thus we must (continue) to check if they exist before we set them.

andir · 2020-10-25T12:44:33Z

@xaverdh I opened a PR for that fix as it can go in on it's own: #101645

andir · 2020-10-25T12:44:50Z

Once that lands I'll rebase this PR ontop of it and give it another spin.

Ma27

LGTM.
As @rnhmjoj stated, nixos-build-vms should use driverInteractive now rather than my workaround btw.

Previously you would be able to override only the QEMU package to be used in the test runner. Frankly that doesn't help a lot if you are trying to get a graphical session. The graphical session requires the option in the NixOS module system to bet set to the correct QEMU package. In this commit I moved most of the test node configuration and transformations into the `mkDriver` function (previously called `driver`). The motivation was to be able to create a `driver` instance with a given QEMU package that will be used consistently througout the test expression.

In a previous commit I broke this as there is no longer one testDriver but only a function to generate one based on some QEMU inputs.

In 5500dc8 we introduced the --keep-vm-state flag and defaulted to that flag not being set. This lead to the `runInMachine` tests not longer working and that going unnoticed for quite some time now.

This reverts commit aab534b & uses the driverInteractive attribute for the test driver instead. This has the same effect but removes the extra module in the nixos-build-vms code.

rnhmjoj

Looks good to me. Thank you for change I mentioned.

ofborg bot added 6.topic: nixos 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Oct 24, 2020

picnoir reviewed Oct 24, 2020

View reviewed changes

nixos/lib/testing-python.nix Show resolved Hide resolved

andir force-pushed the nixos-build-vms-qemu branch from 1a4350f to adc0964 Compare October 24, 2020 23:18

andir mentioned this pull request Oct 25, 2020

nixos/tests: only apply qemu parameters if the options are defined #101645

Merged

30 tasks

KamilaBorowska mentioned this pull request Oct 25, 2020

test-driver.py: remove bufsize=1 from Popen calls #101538

Merged

10 tasks

Ma27 approved these changes Oct 25, 2020

View reviewed changes

andir added 5 commits October 25, 2020 20:09

nixos/tests: format the testing-python.nix file more consistenly

61b09f5

nixos/tests: fix testDriver reference in runInMachine function

fa25d84

In a previous commit I broke this as there is no longer one testDriver but only a function to generate one based on some QEMU inputs.

nixos/tests: fix runInMachine

73635b8

In 5500dc8 we introduced the --keep-vm-state flag and defaulted to that flag not being set. This lead to the `runInMachine` tests not longer working and that going unnoticed for quite some time now.

nixos-build-vms: use the driverInteractive attribute instead

d4fb7da

This reverts commit aab534b & uses the driverInteractive attribute for the test driver instead. This has the same effect but removes the extra module in the nixos-build-vms code.

andir force-pushed the nixos-build-vms-qemu branch from adc0964 to d4fb7da Compare October 25, 2020 19:15

ofborg bot added the 8.has: module (update) label Oct 25, 2020

rnhmjoj approved these changes Oct 25, 2020

View reviewed changes

andir merged commit 1088f05 into NixOS:master Oct 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/tests: follow-up to the closure reduction PR #101598

nixos/tests: follow-up to the closure reduction PR #101598

andir commented Oct 24, 2020

rnhmjoj commented Oct 24, 2020

andir commented Oct 24, 2020

xaverdh commented Oct 25, 2020

rnhmjoj commented Oct 25, 2020 •

edited

andir commented Oct 25, 2020 via email

xaverdh commented Oct 25, 2020

xaverdh commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

xaverdh commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

Ma27 left a comment

rnhmjoj left a comment

nixos/tests: follow-up to the closure reduction PR #101598

nixos/tests: follow-up to the closure reduction PR #101598

Conversation

andir commented Oct 24, 2020

Motivation for this change

rnhmjoj commented Oct 24, 2020

andir commented Oct 24, 2020

xaverdh commented Oct 25, 2020

rnhmjoj commented Oct 25, 2020 • edited

andir commented Oct 25, 2020 via email

xaverdh commented Oct 25, 2020

xaverdh commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

xaverdh commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

andir commented Oct 25, 2020

Ma27 left a comment

Choose a reason for hiding this comment

rnhmjoj left a comment

Choose a reason for hiding this comment

rnhmjoj commented Oct 25, 2020 •

edited