@zouhir - are you still interested in pursuing using Saucelabs here?

yes we are -- we'd really love to delete all these custom server & webdriver we have here: https://github.com/flackr/scroll-timeline/tree/master/test

Periodic ping - are there still plans to work on this in the next few months?

Unfortunately at this point I doubt we will be able to action this issue from the Ecosystem-Infra side. If the interactions team (cc @flackr ) are interested in this we could probably re-prioritize in either Q1 or Q2 2021, but we've not heard from them on this at all.

We're also quite ham-stringed by not having access to saucelabs - we'd probably need to arrange at least a temporary account for testing if we wanted to get this working.

Of course, PRs are always welcome :)

Hi, I am interested in putting some time into a solution that works more widely, but I have had success within the app-history space running the tests individually through playwright against an independent app-history implementation

I am still testing this further to verify these results, however currently this app-history implementation shows tests passing for the current CI run of them.

The current code for this functionality can be seen here

Implementation specific notes

To inject the appHistory variable itself, I needed to set it on globalThis before the test script loaded, or to define it directly in scope. Going with the direct shift and replace worked the easiest for this first pass, it allowed me to mark the test script too as type="module" and utilise const { AppHistory } = await import("/esnext/index.js");

This implementation is targeted towards more than just the web, supporting deno, & node, so it does not have any direct integration yet with location, a bunch of the tests available however relied directly. In the wrapping code for each test, there is injected code to mock location, window, and i/iframe. This renders all these tests more in the realm of unit tests for me, as I am no longer directly integrating with the browser (outside of using it to resolve the given scripts), this however is still within the scope of what I am after here.

To detect pass/fail I had mocked the usage of the async_test + test functions (+ others) utilised by the tests, this then calls the function exposed to the window through playwright. (I am looking to drop these in favour of using testharness.js, but I was looking for the most direct approach to achieve testing stability quickly)

These tests take about 3+ minutes to run at the moment, to install the git repo as a dependency takes a moment, so by default it is not included as a dependency outside of the CI job, and instead is installed each time CI runs

From this implementation's point of view I can step through each failing test one by one, and align to the expectation of the reference implementation