@stephenmcgruer here's a draft PR that shows what would happen if we unpin. Just rebased to retrigger CI.

So more (in fact, most) tests pass now, except a couple of testdriver actions tests.

On Taskcluster I see many unexpected timeouts, notably including the test_driver.bless() test. Seems pretty broken still.

To make sure we're looking at the same thing, https://tools.taskcluster.net/groups/c9RfDywcSFemXC-SfZVqdg is the link I get from the Taskcluster failure above. Only infrastructure/tests is failing, and from the raw logs I see:

Got 36 unexpected results

Most are UNEXPECTED-TIMEOUT, 5 are UNEXPECTED-NOTRUN. Is this aligned with what you're seeing here @foolip ? (Since you and @Hexcles may actually be seeing the same thing and just interpreting it differently, so I thought exact expectations would help :))

Running locally with:

./wpt manifest --rebuild -p ~/Desktop/MANIFEST.json
./wpt run --log-tbpl - --yes --manifest ~/Desktop/MANIFEST.json --metadata infrastructure/metadata/ --install-fonts --binary=/usr/bin/google-chrome-unstable --channel dev chrome infrastructure/ 2>&1 | tee /tmp/log.txt

I get "Got 0 unexpected results" - all tests seem to run ok. Confirmed that my /usr/bin/google-chrome-unstable is Chrome 79.0.3928.4, and this was using ChromeDriver 79.0.3921.0 (15bace96a14b174529914de372e04fd14c78eac7-refs/heads/master@{#698801}) which I got from https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/698801/chromedriver_linux64.zip

Logs - wpt-run.txt

@stephenmcgruer yep the logs in https://taskcluster-artifacts.net/Uw5Tt56rQQS51pOrTz5iEA/0/public/logs/live_backing.log is what I was looking at, with lots of UNEXPECTED-TIMEOUT. UNEXPECTED-NOTRUN is actually a side effect of timeouts, in the status of a subtest that wasn't run because a preceding test timed out.

Closing this, but #19367 is assigned to @stephenmcgruer so it won't be forgotten.