await here to avoid .then

Wait, you can grab the results of the resolve too? :O

That's super cool, I wonder if we should update https://web-platform-tests.org/writing-tests/testharness-api.html?highlight=promise_test#promise-tests (yet again) to suggest await-ing rather than then-ing for this trick. (see the 'Note that in the promise chain constructed ...' section)

I would definitely suggest updating those docs. I get the impression they were written before async/await were widespread enough to be depended upon.

Was this test broken, or why does it need to be changed to run these two bits in parallel? The overall test would read better if just using await.

I'm not sure how you could do this with await. I'd love your help to figure that out. I had real trouble picking the expected flow of this test apart originally, and a Promise.all was the only way I could get it to work.

I'd ask Paul, but looking at the original it looks to me like the first bit is setting up the stuff that matters to the test, and the test_driver call is just there to generate the reports.

So there is an order dependency, but one has to create the observer first.

I'd first create a reportsPromise, and await that after the disconnect.

Promise.all works and there is no raciness or anything to worry about, but it is harder to tell what the test does with it, I think.

Right, I see. Ok, I made the ordering explicit, but in doing so I actually discovered something I didn't expect - if the assert fails then I get a harness level error.

I need to dig into why, but for now I've uploaded a variant with a deliberately failing error to show this off.

Ah, got it. This is a nasty case.

The ReportingObserver's callback is actually called during the call to test_driver.generate_test_report, not afterwards when we are await-ing on observerPromise. This places it in a context where testharness.js can't really 'see' it (I'm not very sure why, I find testharness.js very hard to reason about), so the error from the assert_false propagates to the unhandledrejection handler and the harness is set to ERROR.

Wrapping the then in a t.step_func 'works' but is very unintuitive for the reader (imo):

diff --git a/reporting/disconnect.html b/reporting/disconnect.html
index ffbfe5a8e9..3907740181 100644
--- a/reporting/disconnect.html
+++ b/reporting/disconnect.html
@@ -12,11 +12,11 @@ promise_test(async test => {
     let observerPromise = new Promise(resolve => {
       observer = new ReportingObserver(resolve);
       observer.observe();
-    }).then(reports => {
+    }).then(test.step_func(reports => {
       assert_false(true, 'Testing what happens if this assert fails')
       assert_equals(reports.length, 1);
       assert_equals(reports[0].body.message, "Test message.");
-    });
+    }));
 
     // The observer should still receive this report even though disconnect()
     // is called immediately afterwards.

It's roughly equivalent to the following I think.

    let firstResolve;
    let promiseA = new Promise(resolve => {
      firstResolve = resolve;
    }).then(reports => {
      assert_false(true, 'Testing what happens if this assert fails')
    });

    await new Promise(resolve => {
      setTimeout(() => {
        firstResolve();
        setTimeout(() => {
          resolve();
        }, 100);
      }, 100);
    });

    await promiseA;

I don't have a great solution to this other than to use t.step_func and document it clearly. Any thoughts @foolip ?

What you ran into in 52af27e was almost certainly an unhandled promise rejection, because nothing was awaiting the result of that promise, the test function was still awaiting the test_driver call.

If you make yourself a reportsPromise which can never reject but only resolves with the reports, and put the asserts after const reports = await reportsPromise, this problem should go away.

But I admit this is a case where using Promise.all makes it obvious that promise rejections at any time are handled.

Ah, right, that makes sense.

I've switched to your suggestion; I think it is slightly clearer in terms of the expected ordering (though of course its technically more complex, observerPromise will actually be resolved as part of test_driver.generate_test_report but we still need to await it to get the results).

Wait, you can grab the results of the resolve too? :O

That's super cool, I wonder if we should update https://web-platform-tests.org/writing-tests/testharness-api.html?highlight=promise_test#promise-tests (yet again) to suggest await-ing rather than then-ing for this trick. (see the 'Note that in the promise chain constructed ...' section)

I'm not sure how you could do this with await. I'd love your help to figure that out. I had real trouble picking the expected flow of this test apart originally, and a Promise.all was the only way I could get it to work.