Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reland "[webdriver] Close old windows at the end of each test as well as beginning" #25567

Merged
merged 5 commits into from Sep 18, 2020

Conversation

stephenmcgruer
Copy link
Contributor

@stephenmcgruer stephenmcgruer commented Sep 16, 2020

This is a reland of #24879 (which was reverted in #25544). It updates the code
to only close the windows if the user hasn't asked us to pause after a test has run.

From the original PR:


Previously, we closed old windows at the start of each test. This was nice in terms
of defensiveness (don't assume the last run left the world in a good state), but made
it hard to find problematic tests that left dialogs open (since they wouldn't throw until
the next test).

Instead, this patch does both - close both at the start and end of a test. This should
improve the blaming situation, whilst still being defensive.

There are potential performance implications to this patch, however test runs are
inconclusive. Full runs of Chrome and Safari show +- 2%, which is possibly within
margin of error. Running locally, some directories showed a ~2% slowdown, whilst
others had little or no difference.

@wpt-pr-bot wpt-pr-bot added infra wptrunner The automated test runner, commonly called through ./wpt run labels Sep 16, 2020
@stephenmcgruer
Copy link
Contributor Author

@jgraham @gsnedders - PTAL. I'm not confident in re-using close_after_done here - it is only used today in marionette (despite being set all over the place...) and I don't quite understand what it is meant to be for. (It sounds related...)

I'm also not confident in my methodology for deciding when we can and can't close windows, there's quite likely something I've missed :D

@stephenmcgruer
Copy link
Contributor Author

Capturing IRC comments so I don't lose them.

smcgruer_[EST]: So close_after_done originally was supposed to be about running each test in a clean browsing context (i.e. new tab) vs navigating a single tab
10:04 AM It wasn't really supposed to be about cleaning up other windows
10:04 AM <smcgruer_[EST]> Stephen McGruer Ah, ok. I'm happy to use a new flag then
10:05 AM "may_pause_after_test"
10:05 AM Or if we want to be commanding: "close_windows_after_test"

jgraham also suggested just getting rid of close_after_done since it was never really utilized, but I'd like to do that cleanup separately from this change.

@stephenmcgruer
Copy link
Contributor Author

@jgraham @gsnedders - I've switched to may_pause_after_test, PTAL. (Still interested in thoughts on where, if anywhere, to move the added logic in tools/wptrunner/wptrunner/wptrunner.py to)

@wpt-pr-bot wpt-pr-bot temporarily deployed to wpt-preview-25567 September 17, 2020 13:31 Inactive
tools/wptrunner/wptrunner/wptrunner.py Outdated Show resolved Hide resolved
tools/wptrunner/wptrunner/executors/executorwebdriver.py Outdated Show resolved Hide resolved
@wpt-pr-bot wpt-pr-bot temporarily deployed to wpt-preview-25567 September 17, 2020 17:38 Inactive
@stephenmcgruer stephenmcgruer merged commit ed3c2cd into master Sep 18, 2020
@stephenmcgruer stephenmcgruer deleted the smcgruer/reland-window-closing branch September 18, 2020 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra wptrunner The automated test runner, commonly called through ./wpt run
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants