Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

./wpt check-stability fails with ManifestVersionMismatch if older MANIFEST.json is present #12635

Closed
foolip opened this issue Aug 22, 2018 · 16 comments
Closed

./wpt check-stability fails with ManifestVersionMismatch if older MANIFEST.json is present #12635

foolip opened this issue Aug 22, 2018 · 16 comments
Labels
infra manifest priority:backlog

Comments

@foolip
Copy link
Member

foolip commented Aug 22, 2018

After #12563 some PRs have been failing like this when running check-stability:

+./wpt check-stability firefox:nightly --output-bytes 3145728 --metadata /home/travis/meta/ --install-fonts
browser_setup
INFO:/home/travis/build/web-platform-tests/wpt/tools/ci:# Firefox (nightly) #
INFO:/home/travis/build/web-platform-tests/wpt/tools/ci:# Firefox (nightly) #
INFO:/home/travis/build/web-platform-tests/wpt/tools/ci:Testing web-platform-tests at revision b5f28dc229d27f1599abe541d53890693249c6aa
INFO:/home/travis/build/web-platform-tests/wpt/tools/ci:Testing web-platform-tests at revision b5f28dc229d27f1599abe541d53890693249c6aa
Traceback (most recent call last):
  File "./wpt", line 5, in <module>
    wpt.main()
  File "/home/travis/build/web-platform-tests/wpt/tools/wpt/wpt.py", line 132, in main
    rv = script(*args, **kwargs)
  File "/home/travis/build/web-platform-tests/wpt/tools/ci/check_stability.py", line 252, in run
    ignore_changes, skip_tests)
  File "/home/travis/build/web-platform-tests/wpt/tools/ci/check_stability.py", line 192, in get_changed_files
    manifest_path=manifest_path)
  File "/home/travis/build/web-platform-tests/wpt/tools/wpt/testfiles.py", line 212, in affected_testfiles
    wpt_manifest = load_manifest(manifest_path)
  File "/home/travis/build/web-platform-tests/wpt/tools/wpt/testfiles.py", line 193, in load
    wpt_manifest = manifest.load(wpt_root, manifest_path)
  File "/home/travis/build/web-platform-tests/wpt/tools/manifest/manifest.py", line 239, in load
    rv = Manifest.from_json(tests_root, json.load(f), types=types, meta_filters=meta_filters)
  File "/home/travis/build/web-platform-tests/wpt/tools/manifest/manifest.py", line 183, in from_json
    raise ManifestVersionMismatch
tools.manifest.manifest.ManifestVersionMismatch

(From https://travis-ci.org/web-platform-tests/wpt/jobs/419337736#L600)

The PRs are running code where CURRENT_VERSION is 5, so this must mean there's a MANIFEST.json on disk with version 4. Exactly how it got there I don't know, but it's a situation that should be handled. Steps to reproduce:

curl --location https://github.com/web-platform-tests/wpt/releases/download/merge_pr_12493/MANIFEST-7abda4e7227b65fba2452980d106ed5f11d51b2b.json.gz | gunzip > MANIFEST.json
./wpt check-stability firefox:nightly

(Works even if not on a branch with changes.)

#12589 is a similar issue, comment #12589 (comment) is relevant.

As discussed there, the fix ought to be making manifest.py handle this, so that ./wpt check-stability has the same behavior as ./wpt run.
@foolip
Copy link
Member Author

foolip commented Aug 22, 2018

@gsnedders I'll be OOO for the rest of the week, might you have time to look at this?

@foolip
Copy link
Member Author

foolip commented Aug 22, 2018

Known affected PRs:
#12530
#12542
#12555
#12556

I will try to get those merged without blocking on a fix for this.

@Hexcles
Copy link
Member

Hexcles commented Aug 22, 2018

Earlier on IRC, @gsnedders suggested rebasing and retrying. And I did try in #12556, but the stability checks are still failing.

Shall we just purge the cache?

@foolip foolip mentioned this issue Aug 22, 2018
Merged
@TimothyGu
Copy link
Member

As I noted on IRC, purging the cache worked for the two PRs I tried it on.

@Hexcles
Copy link
Member

Hexcles commented Aug 22, 2018

I'm not sure if it's safe to purge all cache.

@foolip
Copy link
Member Author

foolip commented Aug 22, 2018

Can someone paste how to purge the cache, for others that need it? (This issue has repro steps so don't need to experiment on Travis.)

@Hexcles
Copy link
Member

Hexcles commented Aug 22, 2018 •
edited

Admins can purge cache for an individual build, or drop all cache: https://travis-ci.org/web-platform-tests/wpt/caches

@TimothyGu
Copy link
Member

I'm not sure if it's safe to purge all cache.

What possible harm could it do other than longer build time?

@TimothyGu
Copy link
Member

The Travis CI cache page doesn't display all PR branches however. If you need to get it working now, here's the equivalent curl command:

# the number of the pull request
PRID=12016
# you can find the token through any request to api.travis-ci.org in DevTools
YOUR_TOKEN=yourtoken

curl "https://api.travis-ci.org/repo/644577/caches?branch=PR.$PRID" \
  -X DELETE \
  -H 'Pragma: no-cache' \
  -H 'Origin: https://travis-ci.org' \
  -H 'Travis-API-Version: 3' \
  -H 'Cache-Control: no-cache' \
  -H "Authorization: token $YOUR_TOKEN"

I'm still pretty convinced that purging all cache is the right thing to do here.

@gsnedders
Copy link
Member

Did someone purge all caches? The master cache is much smaller than it was a few days ago when we first started discussing this.

@Hexcles
Copy link
Member

Hexcles commented Aug 24, 2018

Yes. We did yesterday.

@gsnedders
Copy link
Member

Is this therefore fixed and can be closed?

@TimothyGu
Copy link
Member

Probably. Can always reopen if there are remaining issues.

@foolip
Copy link
Member Author

foolip commented Aug 24, 2018

The same will happen next time we bump the manifest version, so fixing this is still important, but not urgent.

@foolip foolip reopened this Aug 24, 2018
@jkillian jkillian mentioned this issue Aug 25, 2018
Merged
@foolip foolip mentioned this issue Aug 27, 2018
Closed
@foolip
Copy link
Member Author

foolip commented Oct 30, 2018

@gsnedders, should this be assigned to you, do you plan to work on it? I'm going to assume not, so I'll downgrade priority, leaving assigned to you just in case.

@gsnedders gsnedders removed their assignment Nov 1, 2018
@foolip
Copy link
Member Author

foolip commented May 6, 2021

./wpt check-stability is not even a command any longer, so this is stale. (Maybe the problem still exists, but I filed the issue and am not curious enough to find out.)

@foolip foolip closed this as completed May 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
infra manifest priority:backlog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@gsnedders @foolip @Hexcles @TimothyGu