nixosTests.installer: Port installer and ZFS test to python #78670

tfc · 2020-01-28T07:56:03Z

Motivation for this change

#72828

Things done

cc @flokli @worldofpeace

tfc · 2020-01-28T07:56:13Z

@GrahamcOfBorg test installer zfs

worldofpeace · 2020-01-31T00:32:05Z

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

tfc · 2020-01-31T08:15:47Z

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

worldofpeace · 2020-01-31T22:01:54Z

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

tfc · 2020-02-01T08:31:18Z

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

Sorry, my fault. Please have a look again @worldofpeace

flokli · 2020-02-03T09:46:57Z

nixos/lib/test-driver/test-driver.py

@@ -566,6 +567,42 @@ def screenshot(self, filename: str) -> None:
            if ret.returncode != 0:
                raise Exception("Cannot convert screenshot")

+    def copy_from_host_via_shell(self, source: str, target: str) -> None:


Is there a reason we have this?

Looking at 510ada7, these could probably also just use copy_from_host…

At first i am using copy_from_host, but with the machine configs that follow later in the script, it doesn't work any longer as the VMs do not seem to support this. For that reason i added the shell variant as an addition.

I assume this is because /tmp/shared from nixos/modules/virtualisation/qemu-vm.nix isn't setup in the installer.

Can we just invoke the mount command manually before running copy_from_host?

We could, but i really don't like the implications of this: Currently, tests are agnostic of how this is all implemented and this allows for transparently changing the virt backend etc (apart from the create_machine parameters which are pretty much qemu-specific, but these are also used by only a minority of tests).

As soon as tests begin doing such things, they would become testdriver-implementation specific and i am pretty sure we don't want that. I'd rather have two different functions that communicate which preconditions they assume and let the user choose from them, before users hard-code testdriver-details into their tests.

We had an offline discussion on that. Agreed on not leaking implementation-specific details about how things are shared into the actual test code. Ideally, this should be encapsulated behind the corresponding methods.

flokli · 2020-02-03T10:08:21Z

nixos/lib/test-driver/test-driver.py

+    def copy_from_host_via_shell(self, source: str, target: str) -> None:
+        """Copy a file from the host into the guest by piping it over the
+        shell into the destination file. Works without host-guest shared folder.
+        Prefer copy_from_host for large files.


Suggested change

Prefer copy_from_host for large files.

Prefer copy_from_host whenever possible.

This implies it's okay to use it for small files.

In fact, we should use copy_from_host whenever possible, and only use the base64 and shell dance if we have to IMHO.

flokli · 2020-02-03T17:31:20Z

@tfc The zfs tests are still missing from all-tests.nix. Can you add them there?

tfc · 2020-02-03T18:16:00Z

@GrahamcOfBorg test zfs

flokli · 2020-02-04T11:09:51Z

@GrahamcOfBorg build nixosTests.installer.simpleUefiSystemdBoot

flokli · 2020-02-04T13:50:16Z

@tfc the above test seems to fail: https://logs.nix.ci/?attempt_id=0cfe37cd-ae01-49d3-9997-aa34ffe60ae9&key=nixos%2Fnixpkgs.78670

worldofpeace · 2020-02-04T14:34:11Z

the above test case also fails on master btw https://hydra.nixos.org/build/111477951, but I can confirm it currently fails for an related reasons to that one.

flokli · 2020-02-04T21:40:08Z

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

flokli · 2020-02-04T21:40:30Z

Also not sure if the installer tests block channel releases, so we should be extra careful here.

tfc · 2020-02-05T06:21:18Z

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

Don't worry, i am on it. It's just that i had no extra time yesterday.

tfc · 2020-02-05T17:10:22Z

@GrahamcOfBorg test installer.simpleUefiSystemdBoot

tfc · 2020-02-05T21:14:43Z

So the current state is that the simpleUefiSystemdBoot test sometimes succeeds, sometimes doesn't (on both x86_64 and aarch64). When it fails, it fails like this:

nixos-install calls systemd bootctl and that produces:

machine # copying channel...
machine # installing the boot loader...
machine # setting up /etc...
machine # Initializing machine ID from random generator.
machine # Failed to open file system "/dev/block/253:1": No such file or directory
machine # Traceback (most recent call last):
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 240, in <module>
machine #     main()
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 199, in main
machine #     subprocess.check_call(["/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl", "--path=/boot", "--no-variables", "install"])
machine #   File "/nix/store/4c4c5ig32lqhsgmcncnb72c3a4bwj66n-python3-3.7.6/lib/python3.7/subprocess.py", line 363, in check_call
machine #     raise CalledProcessError(retcode, cmd)
machine # subprocess.CalledProcessError: Command '['/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl', '--path=/boot', '--no-variables', 'install']' returned non-zero exit status 1.
machine: output:
error: command `nixos-install < /dev/null >&2` failed (exit code 1)
cleaning up
killing machine (pid 9)
(0.00 seconds)
builder for '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed with exit code 1
error: build of '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed

tfc · 2020-02-05T22:48:00Z

@GrahamcOfBorg test installer.btrfsSimple installer.luksroot-format1 installer.simpleClone installer.swraid installer.btrfsSubvolDefault installer.luksroot-format2 installer.simpleLabels installer.zfsroot installer.btrfsSubvols installer.lvm installer.simpleProvided installer.encryptedFSWithKeyfile installer.separateBoot installer.simpleUefiGrub installer.grub1 installer.separateBootFat installer.simpleUefiGrubClone installer.luksroot installer.simple installer.simpleUefiSystemdBoot

tfc · 2020-02-06T07:23:45Z

@GrahamcOfBorg test installer-btrfsSimple installer-luksroot-format1 installer-simpleClone installer-swraid installer-btrfsSubvolDefault installer-luksroot-format2 installer-simpleLabels installer-zfsroot installer-btrfsSubvols installer-lvm installer-simpleProvided installer-encryptedFSWithKeyfile installer-separateBoot installer-simpleUefiGrub installer-grub1 installer-separateBootFat installer-simpleUefiGrubClone installer-luksroot installer-simple installer-simpleUefiSystemdBoot

test driver

tfc · 2020-02-07T12:24:47Z

@GrahamcOfBorg test installer

flokli · 2020-02-08T14:36:23Z

Judging from hydra, simpleUefiSystemdBoot seems to have been flaky in the perl test driver as well, so merging this in.

worldofpeace · 2020-02-08T14:51:50Z

Good for me as well.

nixosTests.installer: Port installer and ZFS test to python (cherry picked from commit dd5f92f)

ofborg bot added 6.topic: nixos 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Jan 28, 2020

tfc mentioned this pull request Jan 28, 2020

nixos/tests: Leftover tasks towards unified Python integration tests #72828

Closed

tfc requested review from flokli, blitz, adisbladis and jonringer January 28, 2020 08:55

worldofpeace added this to the 20.03 milestone Jan 30, 2020

tfc force-pushed the port-installer-test branch from 7cc4a1d to ab649e2 Compare January 31, 2020 08:15

tfc force-pushed the port-installer-test branch from ab649e2 to 380134d Compare February 1, 2020 08:30

flokli reviewed Feb 3, 2020

View reviewed changes

tfc force-pushed the port-installer-test branch 3 times, most recently from 7a47b7c to 11de1ae Compare February 3, 2020 17:23

tfc force-pushed the port-installer-test branch from 11de1ae to c2864b8 Compare February 3, 2020 17:33

tfc force-pushed the port-installer-test branch from c2864b8 to 0495eae Compare February 5, 2020 17:05

disassembler modified the milestones: 20.03, 20.09 Feb 5, 2020

tfc force-pushed the port-installer-test branch from 0495eae to 81172e1 Compare February 5, 2020 22:20

tfc force-pushed the port-installer-test branch from 81172e1 to 30d8d1d Compare February 6, 2020 07:22

Jacek Galowicz added 4 commits February 7, 2020 13:24

nixos/test: Add copy_from_host and copy_from_host_via_shell to python

798fcaa

test driver

nixosTests.installer: Port to python

059845b

nixosTests.zfs: Refactor

3bb2941

nixosTests.zfs: Add to all-tests.nix

942f0fd

tfc force-pushed the port-installer-test branch from 30d8d1d to 942f0fd Compare February 7, 2020 12:24

flokli merged commit dd5f92f into NixOS:master Feb 8, 2020

dtzWill pushed a commit to dtzWill/nixpkgs that referenced this pull request Feb 8, 2020

Merge pull request NixOS#78670 from tfc/port-installer-test

c43a45f

nixosTests.installer: Port installer and ZFS test to python (cherry picked from commit dd5f92f)

tfc deleted the port-installer-test branch February 9, 2020 22:18

This was referenced Feb 18, 2020

nixos/tests/installer: port to python #73237

Closed

[WIP] nixos/tests/installer: support useEFIBoot #73768

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixosTests.installer: Port installer and ZFS test to python #78670

nixosTests.installer: Port installer and ZFS test to python #78670

tfc commented Jan 28, 2020

tfc commented Jan 28, 2020

worldofpeace commented Jan 31, 2020

tfc commented Jan 31, 2020

worldofpeace commented Jan 31, 2020

tfc commented Feb 1, 2020

flokli Feb 3, 2020

tfc Feb 3, 2020

flokli Feb 3, 2020

tfc Feb 3, 2020 •

edited

flokli Feb 3, 2020

flokli Feb 3, 2020

flokli commented Feb 3, 2020

tfc commented Feb 3, 2020

flokli commented Feb 4, 2020

flokli commented Feb 4, 2020

worldofpeace commented Feb 4, 2020

flokli commented Feb 4, 2020

flokli commented Feb 4, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 6, 2020

tfc commented Feb 7, 2020

flokli commented Feb 8, 2020

worldofpeace commented Feb 8, 2020

	Prefer copy_from_host for large files.
	Prefer copy_from_host whenever possible.

nixosTests.installer: Port installer and ZFS test to python #78670

nixosTests.installer: Port installer and ZFS test to python #78670

Conversation

tfc commented Jan 28, 2020

Motivation for this change

Things done

tfc commented Jan 28, 2020

worldofpeace commented Jan 31, 2020

tfc commented Jan 31, 2020

worldofpeace commented Jan 31, 2020

tfc commented Feb 1, 2020

flokli Feb 3, 2020

Choose a reason for hiding this comment

tfc Feb 3, 2020

Choose a reason for hiding this comment

flokli Feb 3, 2020

Choose a reason for hiding this comment

tfc Feb 3, 2020 • edited

Choose a reason for hiding this comment

flokli Feb 3, 2020

Choose a reason for hiding this comment

flokli Feb 3, 2020

Choose a reason for hiding this comment

flokli commented Feb 3, 2020

tfc commented Feb 3, 2020

flokli commented Feb 4, 2020

flokli commented Feb 4, 2020

worldofpeace commented Feb 4, 2020

flokli commented Feb 4, 2020

flokli commented Feb 4, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 5, 2020

tfc commented Feb 6, 2020

tfc commented Feb 7, 2020

flokli commented Feb 8, 2020

worldofpeace commented Feb 8, 2020

tfc Feb 3, 2020 •

edited