Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use assert_approx_equals in MediaSource tests #17266

Merged
merged 1 commit into from Jun 12, 2019

Conversation

foolip
Copy link
Member

@foolip foolip commented Jun 11, 2019

The threeDecimalPlaces helper using number.toFixed(3) allowed for
a maximum difference of 0.001 for a pair like 0.0005 / 0.00149999, but
had no lower limit on the difference, a pair like 0.0004999 / 0.0005
would fail.

This may be the cause of a Safari failure !EQ(6.548, 6.549):
https://wpt.fyi/results/media-source/mediasource-endofstream.html?run_id=255090001

The `threeDecimalPlaces` helper using `number.toFixed(3)` allowed for
a maximum difference of 0.001 for a pair like 0.0005 / 0.00149999, but
had no lower limit on the difference, a pair like 0.0004999 / 0.0005
would fail.

This may be the cause of a Safari failure `!EQ(6.548, 6.549)`:
https://wpt.fyi/results/media-source/mediasource-endofstream.html?run_id=255090001
@foolip
Copy link
Member Author

foolip commented Jun 11, 2019

One of the tests I touched is flaky in Chrome:

Test Subtest Results Messages
/media-source/mediasource-endofstream.html OK: 3/5, TIMEOUT: 2/5
/media-source/mediasource-endofstream.html MediaSource.endOfStream(): media element notified that it now has all of the media data PASS: 3/5, TIMEOUT: 2/5 Test timed out

But that isn't plausibly caused by the changes made here.

From the wpt_report.json.gz I see that when the test passed it took about 500 ms, so it's not that it's too slow. Perhaps there's a race condition in the test where it runs faster the 2nd and following times, making it more prone to time out then.

@foolip
Copy link
Member Author

foolip commented Jun 11, 2019

Reloading it over and over locally I can't get it to time out. ./wpt run --verify --channel dev --binary which google-chrome-unstable chrome /media-source/mediasource-endofstream.html also passes. I'll call that sufficiently investigated.

@foolip
Copy link
Member Author

foolip commented Jun 11, 2019

The wpt.fyi checks show that this fixes a failing subtest in both Chrome and Safari which previously failed as "assert_equals: SegmentInfo duration should still roughly match mediaSource duration expected 6.548 but got 6.549":
https://wpt.fyi/results/media-source/mediasource-endofstream.html?diff&filter=ADC&run_id=255040003&run_id=246670005
https://wpt.fyi/results/media-source/mediasource-endofstream.html?diff&filter=ADC&run_id=250980007&run_id=229000004

Apart from that results are unaffected.

@wolenetz would you mind reviewing this now?

0.001,
"sourceBuffer.buffered range begins where expected before EOS");
assert_less_than_equal(sourceBuffer.buffered.end(0),
expectedBufferedRangeMaxEndTimeBeforeEOS + 0.001,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the + 0.001 to keep the same behavior as before. This is the only change that isn't a straight replacement with assert_approx_equals.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for fixing this.

@foolip foolip merged commit f73128f into master Jun 12, 2019
@foolip foolip deleted the foolip/media-source-approx branch June 12, 2019 07:03
marcoscaceres pushed a commit that referenced this pull request Jul 23, 2019
The `threeDecimalPlaces` helper using `number.toFixed(3)` allowed for
a maximum difference of 0.001 for a pair like 0.0005 / 0.00149999, but
had no lower limit on the difference, a pair like 0.0004999 / 0.0005
would fail.

This may be the cause of a Safari failure `!EQ(6.548, 6.549)`:
https://wpt.fyi/results/media-source/mediasource-endofstream.html?run_id=255090001
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants