New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add RFC for crash
test type
#33
Conversation
@emilio I think you were interested in this. If you have feedback it would be great, particularly if you have evidence that sharing load-type tests (e.g. gecko crashtests) between engines is effective at finding issues. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wpt already has a bunch of crash tests, that might use testharness.js or reftest just to be able to run. e.g.
css/css-overflow/shrink-to-fit-auto-overflow-relayout-crash.html
css/css-pseudo/first-letter-of-html-root-crash.html
This was my concern while reading the proposal, so I'm glad you included it! Personally, I'm pessimistic that tests like these do provide much value between implementations. I'm no implementer, though, so I'm very curious about what @emilio has to say on the subject. |
And this was a very simple test runner, there's tons of test-cases that I haven't ran or what not. |
I'd rather call them crash tests, given that's what they're called everywhere else. |
The reason for not calingthem crashtests was that crashing is only one possible failure mode; they could also fail with an assert or sanitizer failure or similar in vendor infrastructure. But I'm sympathetic to the idea that "load" can be confusing and there is precedent here. |
I think that's an overly pedantic viewpoint to take when there's prior art in all vendors here already, even if the name is slightly confusing. (Just make all assertions and saniziter failures crashes, pff!) |
web-platform-tests/wpt#20017 is a draft PR for this RFC. It's not done yet (hence draft), but what's there seems to work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the only substantive decision left here is whether to use -crash
or -crashtest
in the filename. The former requires some additional work to rename existing files, but is also shorter and closer to existing practice for WebKit. I'd like to make a decision there and then close out this issue so we can land the changes.
My preference is for |
OK changed to |
Doing some archeology here, I wonder when we changed the wait class from |
Add a test type that just ensures that a page can be loaded without error.