Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pass server config to WebDriver via a file instead of an env variable. #28834

Merged
merged 2 commits into from May 10, 2021
Merged

Pass server config to WebDriver via a file instead of an env variable. #28834

merged 2 commits into from May 10, 2021

Conversation

letitz
Copy link
Contributor

@letitz letitz commented May 5, 2021 •
edited

Instead of serializing the whole config in an environment variable, I amended the code to dump the config as JSON in a temporary file (the tests are conveniently executed against a temporary directory) and pass the file path in the environment variable instead.

This should avoid failures due to the 16-bit length limit being exceeded for good. In other words, no more issues like #21705 (comment) and #12461 :) This also removes the need to special-case origin-policy subdomains.

This change was extracted from #28768 per @jgraham's suggestion.

@wpt-pr-bot wpt-pr-bot added infra serve webdriver wg-testing_browser wptrunner The automated test runner, commonly called through ./wpt run wptserve labels May 5, 2021
jgraham
jgraham approved these changes May 5, 2021
View reviewed changes
Copy link
Contributor

@jgraham jgraham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@jgraham
Copy link
Contributor

jgraham commented May 5, 2021

Test failure is ./serve/test_serve.py:1:1: F401 'json' imported but unused

@foolip
Copy link
Member

foolip commented May 5, 2021

Hmm, "Azure Pipelines (tools/wpt/ tests: macOS + Python 3.6)" failed in a novel way:

Error: Download failed on Cask 'firefox-nightly' with message: bad argument (expected URI object or URI string)

https://raw.githubusercontent.com/Homebrew/homebrew-cask-versions/master/Casks/firefox-nightly.rb exists, but perhaps there was a network error. I'll close and reopen to see if it was a fluke.

We could probably install Firefox using ./wpt install instead here...

@foolip foolip closed this May 5, 2021
@foolip foolip reopened this May 5, 2021
@letitz
Copy link
Contributor Author

letitz commented May 6, 2021

Different failure this time in "Azure Pipelines (tools/wpt/ tests: macOS + Python 3.6)": a flaky manifest test, it seems.

manifest/tests/test_manifest.py:122:

<snip>

message = "Hypothesis test_manifest_to_json(s=[<Mock id='4408810848'>,\n <Mock id='4412119456'>,\n <Mock id='4412281184'>,\n <Mo...>,\n <Mock id='4418202064'>]) produces unreliable results: Falsified on the first call but did not on a subsequent one"

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis test_manifest_to_json(s=[<Mock id='4408810848'>,
E            <Mock id='4412119456'>,
E            <Mock id='4412281184'>,
E            <Mock id='4418804272'>,
E            <Mock id='4417926144'>,
E            <Mock id='4418202064'>]) produces unreliable results: Falsified on the first call but did not on a subsequent one

Trying to close and re-open once more.

@letitz letitz closed this May 6, 2021
@letitz letitz reopened this May 6, 2021
@foolip
Copy link
Member

foolip commented May 6, 2021

@letitz also try rebasing to be sure you're testing the latest CI configuration.

@letitz
Copy link
Contributor Author

letitz commented May 6, 2021

The wpt-chrome-dev-stability and wpt-firefox-nightly-stability tasks both timed out, but they were running mostly fine up to there. What do we do now?

@LukeZielinski LukeZielinski assigned foolip and unassigned LukeZielinski May 6, 2021
@letitz
Copy link
Contributor Author

letitz commented May 6, 2021

Tried rebasing per @foolip's suggestion.

@foolip
Copy link
Member

foolip commented May 6, 2021

@letitz it looks like flake8 is still failing, did you accidental revert that fix?

@letitz
Copy link
Contributor Author

letitz commented May 10, 2021

Oh, I think I totally did. Sorry! Not very familiar with the PR workflow yet, especially around how to handle commits added to PRs by others. Fixing...

foolip
foolip approved these changes May 10, 2021
View reviewed changes
Copy link
Member

@foolip foolip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is still running, but changes look good!

@foolip
Copy link
Member

foolip commented May 10, 2021

OK, stability jobs have timed out due to #7660 and wpt.fyi checks point to changes in test results due to #28925. I'll admin merge.

@foolip foolip merged commit cd82725 into web-platform-tests:master May 10, 2021
DamieFC pushed a commit to DamieFC/chromium that referenced this pull request May 12, 2021
Pass server config via a file.
1f1ca91 
This change unblocks importer, where webdriver tests are failing due to changes in web-platform-tests/wpt#28834

Change-Id: Ia07ed877aa808da41a49260900a390c87e9c5521
No-Export: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2890259
Reviewed-by: Jason Chase <chasej@chromium.org>
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Kyle Ju <kyleju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#882198}
T3-M4 pushed a commit to bayandin/chromedriver that referenced this pull request May 13, 2021
Pass server config via a file.
cceae6e 
This change unblocks importer, where webdriver tests are failing due to changes in web-platform-tests/wpt#28834

Change-Id: Ia07ed877aa808da41a49260900a390c87e9c5521
No-Export: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2890259
Reviewed-by: Jason Chase <chasej@chromium.org>
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Kyle Ju <kyleju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#882198}
@letitz letitz deleted the wd-server-config branch May 26, 2021 08:55
@whimboo
Copy link
Contributor

whimboo commented Dec 7, 2021

This caused a regression for several tests across browsers. We are currently investigating in https://bugzilla.mozilla.org/show_bug.cgi?id=1744249.

mjfroman pushed a commit to mjfroman/moz-libwebrtc-third-party that referenced this pull request Oct 14, 2022
Pass server config via a file.
8206cdc 
This change unblocks importer, where webdriver tests are failing due to changes in web-platform-tests/wpt#28834

Change-Id: Ia07ed877aa808da41a49260900a390c87e9c5521
No-Export: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2890259
Reviewed-by: Jason Chase <chasej@chromium.org>
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Kyle Ju <kyleju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#882198}
NOKEYCHECK=True
GitOrigin-RevId: 1f1ca91d06821cb5dd2fe057d4efa4fc572ee158
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jgraham jgraham jgraham approved these changes

@foolip foolip foolip approved these changes

@andreastt andreastt Awaiting requested review from andreastt

@AutomatedTester AutomatedTester Awaiting requested review from AutomatedTester

@LukeZielinski LukeZielinski Awaiting requested review from LukeZielinski

@mjzffr mjzffr Awaiting requested review from mjzffr

@shs96c shs96c Awaiting requested review from shs96c

@whimboo whimboo Awaiting requested review from whimboo

Assignees

@foolip foolip

Labels
infra serve webdriver wg-testing_browser wptrunner The automated test runner, commonly called through ./wpt run wptserve
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@letitz @jgraham @foolip @whimboo @wpt-pr-bot @LukeZielinski