@GrahamcOfBorg test keymap.colemak

Success on x86_64-linux (full log)

Attempted: tests.keymap.colemak

syncing
machine: running command: sync
machine: exit status 0
2 out of 2 tests succeeded
test script finished in 18.36s
cleaning up
killing machine (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/a3j8c8clkqdvklalzjzmk9d5bkxw2dhi-vm-test-run-keymap-colemak

Works great here even under quite extreme load (load average 18) on my laptop using a https://ark.intel.com/products/95451.

Great work! 😃

Looks excellent! Thanks!

Failure on aarch64-linux (full log)

Attempted: tests.keymap.colemak

cannot build derivation '/nix/store/6fs5pcs38yk83zshbwpd19h067w4llrx-etc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/4r9i25dqkzfyjflqy9xh7p569jnvi61f-stage-1-init.sh.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/z1hpnas8mh2jhz6z8qy7715hpv971l6i-initrd.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/7d870isrnxysjiryzdfx3z63xh7b7xp0-nixos-system-machine-18.09.git.dd4d1a4.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/srdj9y73bdxf0m12gcirwa0jjsa9vd6f-closure-info.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/kqam08i23b5cjblaynj6apxvg344dhiq-run-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/2nsagj1fqjkn0nb2cp7ywcdav1ymsb0r-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/cl4kasd95x05d38rzl2vrqsk7ap2mp52-nixos-test-driver-keymap-colemak.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/m26z62v6hi9qvvmq25sqy7q29fg7rsnd-vm-test-run-keymap-colemak.drv': 1 dependencies couldn't be built
�[31;1merror:�[0m build of '/nix/store/m26z62v6hi9qvvmq25sqy7q29fg7rsnd-vm-test-run-keymap-colemak.drv' failed

People of the future, this is the first Hydra evaluation of nixos:release-18.03 with this commit:

https://hydra.nixos.org/eval/1451135

https://hydra.nixos.org/build/73238407/nixlog/13

Unfortunately, no dice. Shall I revert?

Oops. Please revert on 18.03 while I check it out. Looks like an issue with xdotool and the window manager, will probably have to modify a little for 18.03.

Thanks @srhb for catching this.

Strangely, the same test succeeded in the same eval for i686: https://hydra.nixos.org/build/73238415
Plus I cannot reproduce the above failure (which definitely looks like a deterministic error) on the same git revision.
The test different derivation hashes on the same commit differ between hydra and my local machine (this seems to be the case for other tests as well, whereas normal packages have identical hashes).
Do we have an impurity in the test framework? 😕
Appreciate any ideas how to analyze this...

The tests are quite often impure. And uh, did someone rerun the tests or did I completely misread that there was an error? My old look shows it's fine now. Am I dreaming?

And I'm not sure it looked to me like a deterministic error. Could it be that the property in question does not exist if we ask too fast?

You're not dreaming, I saw the error too ... But I doubt someone restarted that job. No idea what happened.

And I'm not sure it looked to me like a deterministic error. Could it be that the property in question does not exist if we ask too fast?

Maybe. The only thing I found in xdotool doc is that windowactivate may not work with all window managers.

@xeji OK, good! And yes, I saw that too, and I think what's going on is that it doesn't work yet during whatever startup is happening. I can't imagine that the VM would be different between our systems and Hydras on that point.

Since I do waitForWindow before xdotool, the window is already there. But maybe the WM hasn't decorated and set all of the window's properties yet. I could wrap the xdotool call in a waitUntilSucceeds() loop. Or use windowfocus alternative to windowactivate.

If windowfocus works regardless, that seems like the way to go!

(But if we can find a step to actually waitUntilSucceeds that makes this work, I think that's more than acceptable!)

Well the manpage says about windowfocus:

Uses XSetInputFocus which may be ignored by some window managers or programs.

That should be testable in the by spawning another window with focus and switching, right?

Good idea, I'll do that.

I haven't seen any case where the old test failed because of losing focus after the second xterm window was removed. Now there's only one fullscreen window. So I wonder if the xdotool is just an extra failsafe that creates more trouble than it prevents...

Oh, good point. I forgot about that. In that case, feel free to nuke it! :-) I thought it would help us find a safe spot to start the test, but that may indeed not be the case at all.

Let's wait and see how the tests work run on master. I'll do some more experiments and then make an updated version.

Thanks again for your help!