Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include date of Firefox Nightly in wpt_report.json #13052

Closed
foolip opened this issue Sep 18, 2018 · 11 comments
Closed

Include date of Firefox Nightly in wpt_report.json #13052

foolip opened this issue Sep 18, 2018 · 11 comments
Labels
infra priority:roadmap wptrunner The automated test runner, commonly called through ./wpt run

Comments

@foolip
Copy link
Member

foolip commented Sep 18, 2018

The Firefox Nightly version will be something like "64.0a1" but that doesn't change for each build. In the UI the date is included.

@jgraham, is there any way we could get this information?

@jgraham
Copy link
Contributor

jgraham commented Sep 19, 2018

Yeah, mozversion can get it from the application.ini file, assuming we know the path to the actual firefox binary (which we do in our setup, but don't in general e.g. if firefox is actually a shell script)

@foolip
Copy link
Member Author

foolip commented Sep 19, 2018

@Hexcles, I realize there might be an existing and somewhat serious problem here. If the results receiver uses the version to reject cases where the different chunks got different versions of the browsers, it's possible we have already accepted runs that are for a mix of browser version. WDYT, what's the priority of fixing this?

@Hexcles
Copy link
Member

Hexcles commented Sep 19, 2018

If the results receiver uses the version to reject cases where the different chunks got different versions of the browsers, it's possible we have already accepted runs that are for a mix of browser version.

Sorry I don't quite get what you meant. All (and only) chunks belonging to the same task arrive in a single request to results receiver.

@jgraham
Copy link
Contributor

jgraham commented Sep 19, 2018

There is a race condition though; since all the tasks download the latest Firefox nightly, if there happens to be a new release between the first and last tasks starting, they won't all get the same version.

In the long term we should fix this with a taskgraph that picks a nightly build and uses that version for all child tasks.

@foolip
Copy link
Member Author

foolip commented Sep 19, 2018

Yep, that's the problem I meant.

@gsnedders gsnedders added infra wptrunner The automated test runner, commonly called through ./wpt run labels Sep 30, 2018
@foolip foolip changed the title Include date of Firefox Nightly in wptreport.json Include date of Firefox Nightly in wpt_report.json Oct 6, 2018
@mdittmer
Copy link
Contributor

Ping from your friendly neighbourhood ecosystem infra rotation

If this is priority:roadmap should it have an assignee?

@foolip
Copy link
Member Author

foolip commented Dec 12, 2018

@jgraham do you reckon this is an easy fix?

@jgraham
Copy link
Contributor

jgraham commented Dec 20, 2018

I think "get the date into the file" is an easy fix and "get the tasks to all use the same version" is harder.

@foolip
Copy link
Member Author

foolip commented Dec 20, 2018

This issue is just about "get the date into the file", then we can start least know when we're getting a mix of browser versions.

@lukebjerring
Copy link
Contributor

IIRC, web-platform-tests/wpt.fyi#1068 covered being more lenient with inconsistent version (build).

@foolip
Copy link
Member Author

foolip commented Mar 15, 2019

With browser_build_id and browser_changeset now included in the reports for Firefox I consider this fixed. We could detect runs with mixed versions in wpt.fyi using this.

@foolip foolip closed this as completed Mar 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra priority:roadmap wptrunner The automated test runner, commonly called through ./wpt run
Projects
None yet
Development

No branches or pull requests

6 participants