Thanks! I noticed there was some Hydra failure but didn't understand why. Actually, I still don't understand why. The batch command is run in the VM / guest, which has a separate "load" metric from the host.

The test also runs "at", waits 1.5 minutes and expects "atd" to have performed the job by then. That's not deterministic either. How about polling every 30s or so, for batch and atd to perform their job, with timeout after 15 minutes? (If that's not enough, I think Hydra is run under way too heavy load.)

I've never seen the at test fail. Relative timing looks reliable: at now + 1min is done before sleep 1.5min, no matter the system load. This test makes sense to me, no reason to change it.
As for batch, waiting longer as you suggested may make the test pass in most cases - but then what is its significance? You cannot really test the desired "execute when system load is low" behavior unless you can reliably simulate high or low system load. No way to do that on Hydra.

Relative timing looks reliable: at now + 1min is done before sleep 1.5min, no matter the system load.

I bet if the VM spawned a few CPU hogging processes with high enough priority, we could make "at" fail too. So I think both tests are somewhat non-deterministic, and that batch could be made reliable "enough" with some more sleep + poll.

But to be pragmatic, I'm ok with removing the test.