New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Latest Servo run on wpt.fyi is from March 6 #24257
Comments
Seems like everything is running, but the wdspec jobs time out, and we therefore don't ingest the results. I'm not sure what the fix is here. We could increase the number of chunks for wdspec, or change the configuration so servo doesn't run them, or disable tests we don't expect servo to support in that directory. |
Report exceptions for async script executions to webdriver Improving the accuracy of the async script execution for our webdriver implementation allows us to run many more webdriver tests without timeouts, which should help with web-platform-tests/wpt#24257. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #27036 and #27035 and #27031. - [x] There are tests for these changes
Report exceptions for async script executions to webdriver Improving the accuracy of the async script execution for our webdriver implementation allows us to run many more webdriver tests without timeouts, which should help with web-platform-tests/wpt#24257. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #27036 and #27035 and #27031. - [x] There are tests for these changes
Report exceptions for async script executions to webdriver Improving the accuracy of the async script execution for our webdriver implementation allows us to run many more webdriver tests without timeouts, which should help with web-platform-tests/wpt#24257. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #27036 and #27035 and #27031. - [x] There are tests for these changes
Report exceptions for async script executions to webdriver Improving the accuracy of the async script execution for our webdriver implementation allows us to run many more webdriver tests without timeouts, which should help with web-platform-tests/wpt#24257. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #27036 and #27035 and #27031. - [x] There are tests for these changes
Now we have: "ERROR Unsupported test type print-reftest for product servo" |
Ah, we worked around that locally by only running with a fixed set of supported test-types. |
That being said, it's not clear to me why Safari and Edge are not affected by that problem, because I don't see any support in their executors for the new print-reftest test type. |
Looks like webkitgtk is actually hitting the same issue:
|
Ok, so there are only explicit print-reftest jobs for 1) servo, 2) firefox, 3) chrome, 4) webkitgtk. |
Safari and Edge run on Azure Pipelines (so we can access Windows and MacOS). They use a totally different way of choosing their runs and uploading the results, see In short they run:
Then they manually upload their results to wpt.fyi, via I suspect for them, the print-reftests will fail to run but that won't stop runs being uploaded. |
We should probably change the taskcluster yml to only run print reftests for browsers that support them. Unfortunately that's quite a lot of duplication right now. We could add more logic to make it easier or just live with a big copy&paste job |
Is anyone owning the next steps on this issue? I'm downgrading from urgent to roadmap as it has no clear owner, has had no activity for multiple weeks, and we've survived since March without Servo runs. (Which is not to say that's ok, but to acknowledge that its difficult to consider this 'urgent' if 4 months can pass with the issue existing). |
* print-reftests are currently not supported for this two browsers. So we need to avoid triggering those runs on taskcluster for them to avoid failing the whole run. This fixes: web-platform-tests#24360 and web-platform-tests#24257. * This patch removes the print-reftest suite from the general block and adds a new block for it and for the browsers that support this tests on taskcluster (chrome and firefox).
…#24576) * print-reftests are currently not supported for this two browsers. So we need to avoid triggering those runs on taskcluster for them to avoid failing the whole run. This fixes: web-platform-tests#24360 and web-platform-tests#24257. * This patch removes the print-reftest suite from the general block and adds a new block for it and for the browsers that support this tests on taskcluster (Chrome and Firefox).
* print-reftests are currently not supported for this two browsers. So we need to avoid triggering those runs on taskcluster for them to avoid failing the whole run. This fixes: #24360 and #24257. * This patch removes the print-reftest suite from the general block and adds a new block for it and for the browsers that support this tests on taskcluster (Chrome and Firefox).
I don't really know where the code that triggers those runs lives, or where logs are stored from those runs, so I'm filing this here.
The text was updated successfully, but these errors were encountered: