Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement assert_precondition (RFC) #19844

Closed
foolip opened this issue Oct 23, 2019 · 11 comments · Fixed by #19993
Closed

Implement assert_precondition (RFC) #19844

foolip opened this issue Oct 23, 2019 · 11 comments · Fixed by #19993
Assignees
@foolip
Labels
infra priority:roadmap

Comments

@foolip
Copy link
Member

foolip commented Oct 23, 2019

This issue is to implement RFC #16: 'PRECONDITION_FAILED' subtest result (status)

There are at least two parts to this:

Then there will be many tests that could be updated a few of them included in #16689.
@foolip foolip added the infra label Oct 23, 2019
@foolip foolip self-assigned this Oct 23, 2019
This was referenced Oct 23, 2019
Closed
Merged
@foolip
Copy link
Member Author

foolip commented Oct 23, 2019

The wpt side of this change should also update the documentation added in #19628.

@foolip
Copy link
Member Author

foolip commented Oct 23, 2019

The mozlog changes are up for review in https://phabricator.services.mozilla.com/D50175

@foolip
Copy link
Member Author

foolip commented Oct 24, 2019

https://bugzilla.mozilla.org/show_bug.cgi?id=1589056 is now done, and the next step is for @jgraham to release mozlog 5.0.

@foolip
Copy link
Member Author

foolip commented Oct 25, 2019

https://pypi.org/project/mozlog/5.0/ is now released, thanks @jgraham!

@foolip
Copy link
Member Author

foolip commented Oct 25, 2019

Oops, we're going to get these errors:

ERROR: mozrunner 7.6.0 has requirement mozlog~=4.2.0, but you'll have mozlog 5.0 which is incompatible.
ERROR: mozprofile 2.3.0 has requirement mozlog~=4.2.0, but you'll have mozlog 5.0 which is incompatible.
ERROR: mozversion 2.1.0 has requirement mozlog~=4.0, but you'll have mozlog 5.0 which is incompatible.

@jgraham how can we resolve this? Do those also need new releases to fix this? Shall I send a mozilla-central patch that bumps the minor version of each?

@foolip
Copy link
Member Author

foolip commented Oct 25, 2019

Discussed on IRC. I've sent https://phabricator.services.mozilla.com/D50595.

@foolip
Copy link
Member Author

foolip commented Oct 28, 2019

Those three packages now have updated releases. Thanks @jgraham!

@foolip foolip mentioned this issue Oct 28, 2019
Closed
@foolip
Copy link
Member Author

foolip commented Oct 28, 2019

mozlog itself updated in #19884

foolip added a commit that referenced this issue Oct 30, 2019
@foolip
[testharness.js] introduce assert_precondition
129bfc4 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
@foolip foolip mentioned this issue Oct 30, 2019
Merged
foolip added a commit that referenced this issue Oct 30, 2019
@foolip
[testharness.js] introduce assert_precondition
c871a67 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
foolip added a commit that referenced this issue Oct 30, 2019
@foolip
[testharness.js] introduce assert_precondition
4fdb848 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
@foolip
Copy link
Member Author

foolip commented Nov 1, 2019

This came up at our improving interop with WPT talk at https://webkit.org/meeting/ today.

When this is resolved, if there are tests that pass spuriously in WebKit, or that fail when they shouldn't have been run, then assert_precondition could be used to allow filtering out those cases in wpt.fyi.

@cdumez @othermaciej @smfr @rniwa do you have specific cases for me to look at? I could try fixing a few of those as part of proving that this all works end-to-end.

foolip added a commit that referenced this issue Nov 2, 2019
@foolip
[testharness.js] introduce assert_precondition
3e9e171 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
@smfr
Copy link
Contributor

smfr commented Nov 2, 2019

Any test in ambient-light, battery-status, bluetooth, vibration on Safari.

@foolip
Copy link
Member Author

foolip commented Nov 4, 2019

Thanks @smfr, I've included one spurious vibration pass in #19993 to show how it works, and that it works.

foolip added a commit that referenced this issue Nov 6, 2019
@foolip
[testharness.js] introduce assert_precondition
ec21367 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
foolip added a commit that referenced this issue Nov 6, 2019
@foolip
[testharness.js] introduce assert_precondition
46efb2d 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
foolip added a commit that referenced this issue Nov 8, 2019
@foolip
[testharness.js] introduce assert_precondition
962c338 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
foolip added a commit that referenced this issue Nov 8, 2019
@foolip
[testharness.js] introduce assert_precondition
17543bc 
This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of #16689.

Fixes #19844.
moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this issue Nov 29, 2019
@foolip @moz-wptsync-bot
Bug 1592592 [wpt PR 19993] - [testharness.js] introduce `assert_preco…
393d9c3 
…ndition`, a=testonly

Automatic update from web-platform-tests
[testharness.js] introduce `assert_precondition`

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of web-platform-tests/wpt#16689.

Fixes web-platform-tests/wpt#19844.

--
[vibration] use `assert_precondition` to avoid spurious pass

This demonstrates the use of `assert_precondition` for subtests.

--

wpt-commits: 17543bc30bdf65838e9ec8e3b3b6fd210dced66e, a90e89fb031b0b357305ac9d874968d8451b1181
wpt-pr: 19993
xeonchen pushed a commit to xeonchen/gecko that referenced this issue Nov 29, 2019
@foolip @moz-wptsync-bot
Bug 1592592 [wpt PR 19993] - [testharness.js] introduce `assert_preco…
63c21bf 
…ndition`, a=testonly

Automatic update from web-platform-tests
[testharness.js] introduce `assert_precondition`

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of web-platform-tests/wpt#16689.

Fixes web-platform-tests/wpt#19844.

--
[vibration] use `assert_precondition` to avoid spurious pass

This demonstrates the use of `assert_precondition` for subtests.

--

wpt-commits: 17543bc30bdf65838e9ec8e3b3b6fd210dced66e, a90e89fb031b0b357305ac9d874968d8451b1181
wpt-pr: 19993
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this issue Nov 30, 2019
@marco-c
Bug 1592592 [wpt PR 19993] - [testharness.js] introduce `assert_preco…
2c28c2f 
…ndition`, a=testonly

Automatic update from web-platform-tests
[testharness.js] introduce `assert_precondition`

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of web-platform-tests/wpt#16689.

Fixes web-platform-tests/wpt#19844.

--
[vibration] use `assert_precondition` to avoid spurious pass

This demonstrates the use of `assert_precondition` for subtests.

--

wpt-commits: 17543bc30bdf65838e9ec8e3b3b6fd210dced66e, a90e89fb031b0b357305ac9d874968d8451b1181
wpt-pr: 19993

UltraBlame original commit: 9b458e91114235eb43711dbac7a3acfe30c83fce
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this issue Nov 30, 2019
@marco-c
Bug 1592592 [wpt PR 19993] - [testharness.js] introduce `assert_preco…
348bece 
…ndition`, a=testonly

Automatic update from web-platform-tests
[testharness.js] introduce `assert_precondition`

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of web-platform-tests/wpt#16689.

Fixes web-platform-tests/wpt#19844.

--
[vibration] use `assert_precondition` to avoid spurious pass

This demonstrates the use of `assert_precondition` for subtests.

--

wpt-commits: 17543bc30bdf65838e9ec8e3b3b6fd210dced66e, a90e89fb031b0b357305ac9d874968d8451b1181
wpt-pr: 19993

UltraBlame original commit: 9b458e91114235eb43711dbac7a3acfe30c83fce
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this issue Nov 30, 2019
@marco-c
Bug 1592592 [wpt PR 19993] - [testharness.js] introduce `assert_preco…
45ef81a 
…ndition`, a=testonly

Automatic update from web-platform-tests
[testharness.js] introduce `assert_precondition`

This depends on mozlog 5.0 for the new PRECONDITION_FAILED status:
https://bugzilla.mozilla.org/show_bug.cgi?id=1589056

Implements web-platform-tests/rfcs#16.

Includes parts of web-platform-tests/wpt#16689.

Fixes web-platform-tests/wpt#19844.

--
[vibration] use `assert_precondition` to avoid spurious pass

This demonstrates the use of `assert_precondition` for subtests.

--

wpt-commits: 17543bc30bdf65838e9ec8e3b3b6fd210dced66e, a90e89fb031b0b357305ac9d874968d8451b1181
wpt-pr: 19993

UltraBlame original commit: 9b458e91114235eb43711dbac7a3acfe30c83fce
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@foolip foolip

Labels
infra priority:roadmap
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[testharness.js] introduce assert_precondition web-platform-tests/wpt
3 participants
@smfr @stephenmcgruer @foolip