Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebKitGTK: rework how the minibrowser is downloaded on the CI (TaskCluster) and add weekly runs for beta channel #25263

Merged
merged 1 commit into from Sep 30, 2020

Conversation

clopez
Copy link
Contributor

@clopez clopez commented Aug 27, 2020

Until now, when running the WebKitGTK tests on the taskcluster CI a pre-running hook was triggerred that installed some custom Debian packages for the stable channel and unpacked a tarball for the nightly channel. The tarball from the nightly channel required to install a lot of extra (uneeeded) dependencies (it was not optimized) slowing down the CI tests.

We recently re-worked how the pre-built products are generated, now we provide built products for stable, beta and nightly using the same bundle type: a zip file that contains just webkitgtk with the minimum libraries needed for it and is built for the target operating system where it will run. Currently we support Ubuntu LTS and LTS-1. TC CI runs in Ubuntu LTS.

This patch add supports for the command wpt --install-browser for product webkitgtk_minibrowser and uses this command to install the browser on the CI. The built product downloaded not longer requires using a specific path and is unpacked in the local directory.

This also means end-users can also install webkitgtk via this way and use it. But currently it will only work for them if they run Ubuntu LTS. We may add support for more distros in the future.

This patch also adds weekly runs for the beta channel.

The built products are available at https://webkitgtk.org/built-products/x86_64/release and automatic bots generate them daily.
The bots generating the built products for the nightly channel are at https://build.webkit.org and the ones for the stable and beta channels at https://build-webkit.igalia.com

@clopez clopez changed the title WebKitGTK: rework how the minibrowser is downloaded on the CI (TaksCluster) and add weekly runs for beta channel WebKitGTK: rework how the minibrowser is downloaded on the CI (TaskCluster) and add weekly runs for beta channel Aug 27, 2020
@clopez
Copy link
Contributor Author

clopez commented Sep 1, 2020

ping reviewers? //cc @jgraham

tools/wpt/utils.py Outdated Show resolved Hide resolved
tools/wpt/utils.py Outdated Show resolved Hide resolved
tools/wpt/browser.py Show resolved Hide resolved
tools/wpt/browser.py Show resolved Hide resolved
@clopez
Copy link
Contributor Author

clopez commented Sep 2, 2020

Pushed a new commit changing:

  • use six.moves in tools/wpt/utils.py
  • Properly update the function signature for find_webdriver and all the related classes

Please let me know if more changes are needed. Thanks!

@clopez
Copy link
Contributor Author

clopez commented Sep 4, 2020

BTW.. I see 3 checks on the CI failed after the last fix-up commit (just a change on the pattern of the sha256checksum files) to accommodate to a recent upstream webkit change. I don't think this 3 failures are valid.. after checking the log that seems to me like some random failure on that CI machines.

clopez added a commit to clopez/wpt that referenced this pull request Sep 28, 2020
…atform-tests#25263)

 * Until now, when running the WebKitGTK tests on the taskcluster CI
a pre-running hook was triggerred that installed some custom Debian packages
for the stable channel and unpacked a tarball for the nightly channel.
The tarball from the nightly channel required to install a lot of extra
(uneeeded) dependencies (it was not optimized) slowing down the CI tests.

* We recently re-worked how the pre-built products are generated, now we
provide built products for stable, beta and nightly using the same bundle
type: a zip file that contains just webkitgtk with the minimum libraries
needed for it and is built for the target operating system where it will
run. Currently we support Ubuntu LTS and LTS-1. TC CI runs in Ubuntu LTS.

* This patch add supports for the command "wpt --install-browser" for product
webkitgtk_minibrowser and uses this command to install the browser on the CI.
The built product downloaded not longer requires using a specific path and
is unpacked in the local directory.

* This also means end-users can also install webkitgtk via this way and use it.
But currently it will only work for them if they run Ubuntu LTS.
We may add support for more distros in the future.

* Finally this commit also schedules weekly webkitgtk_minibrowser runs for the
beta channel on the CI.
@clopez
Copy link
Contributor Author

clopez commented Sep 28, 2020

Sorry for the late replies, I was on holidays this past weeks.
I have checked the PR not longer applies on top of master (there were some conflicts due to 28bab3e which added a new function variant of find_webdriver() but that was easy to resolve).
Due to this i'm merging all the previous changes into one and force-pushing the rebased commit.
Sorry if this makes it a bit more difficult to continue the review. Regarding the previous version the main change is just:

  • rename download_url_to_descriptor() to get_download_to_descriptor(), improve the function docstrings to make clearer its use-case and update all the callers of it to use the new name.

@clopez
Copy link
Contributor Author

clopez commented Sep 29, 2020

The failure on the CI seems unrelated to this commit. I tracked it down to be caused by #25774 (comment)

…atform-tests#25263)

 * Until now, when running the WebKitGTK tests on the taskcluster CI
a pre-running hook was triggerred that installed some custom Debian packages
for the stable channel and unpacked a tarball for the nightly channel.
The tarball from the nightly channel required to install a lot of extra
(uneeeded) dependencies (it was not optimized) slowing down the CI tests.

* We recently re-worked how the pre-built products are generated, now we
provide built products for stable, beta and nightly using the same bundle
type: a zip file that contains just webkitgtk with the minimum libraries
needed for it and is built for the target operating system where it will
run. Currently we support Ubuntu LTS and LTS-1. TC CI runs in Ubuntu LTS.

* This patch add supports for the command "wpt --install-browser" for product
webkitgtk_minibrowser and uses this command to install the browser on the CI.
The built product downloaded not longer requires using a specific path and
is unpacked in the local directory.

* This also means end-users can also install webkitgtk via this way and use it.
But currently it will only work for them if they run Ubuntu LTS.
We may add support for more distros in the future.

* Finally this commit also schedules weekly webkitgtk_minibrowser runs for the
beta channel on the CI.
@clopez
Copy link
Contributor Author

clopez commented Sep 29, 2020

Rebased again on top of master to avoid the unrelated failure on the CI

@clopez
Copy link
Contributor Author

clopez commented Sep 29, 2020

Everything green on the CI now

@clopez clopez requested a review from jgraham September 30, 2020 13:09
tools/wpt/browser.py Show resolved Hide resolved
@clopez
Copy link
Contributor Author

clopez commented Sep 30, 2020

Thanks!

@clopez clopez merged commit 7de20a9 into web-platform-tests:master Sep 30, 2020
ziransun pushed a commit to ziransun/wpt that referenced this pull request Oct 6, 2020
…atform-tests#25263)

 * Until now, when running the WebKitGTK tests on the taskcluster CI
a pre-running hook was triggerred that installed some custom Debian packages
for the stable channel and unpacked a tarball for the nightly channel.
The tarball from the nightly channel required to install a lot of extra
(uneeeded) dependencies (it was not optimized) slowing down the CI tests.

* We recently re-worked how the pre-built products are generated, now we
provide built products for stable, beta and nightly using the same bundle
type: a zip file that contains just webkitgtk with the minimum libraries
needed for it and is built for the target operating system where it will
run. Currently we support Ubuntu LTS and LTS-1. TC CI runs in Ubuntu LTS.

* This patch add supports for the command "wpt --install-browser" for product
webkitgtk_minibrowser and uses this command to install the browser on the CI.
The built product downloaded not longer requires using a specific path and
is unpacked in the local directory.

* This also means end-users can also install webkitgtk via this way and use it.
But currently it will only work for them if they run Ubuntu LTS.
We may add support for more distros in the future.

* Finally this commit also schedules weekly webkitgtk_minibrowser runs for the
beta channel on the CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants