Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge runs stuck since ~July 31st #24846

Closed
stephenmcgruer opened this issue Aug 2, 2020 · 4 comments
Closed

Edge runs stuck since ~July 31st #24846

stephenmcgruer opened this issue Aug 2, 2020 · 4 comments

Comments

@stephenmcgruer
Copy link
Contributor

Noticed yesterday when I tried to issue a trigger run on both Edge Stable and Edge Dev. All Edge runs appear to be stuck:

https://dev.azure.com/web-platform-tests/wpt/_build?definitionId=1&_a=summary&view=branches

In all cases where the pipeline is meant to be running Edge, it just says "Queued" for the Edge runners.

Example runs:

image

@stephenmcgruer
Copy link
Contributor Author

This appears to no longer be stuck, so dropping to roadmap. Would love to know why it happened, but if we can't find that out then we should just close this.

@mustjab
Copy link
Contributor

mustjab commented Aug 3, 2020

We schedule a weekly VM re-creation on a Sunday, which usually takes Edge runners down for 30 minutes or so. I've double checked our pipeline and it looks like that happened without errors.

@stephenmcgruer
Copy link
Contributor Author

I guess we won't be able to see what happened then - it looks like the VM re-creation went back and caught up the old pending jobs too. Note that these were definitely pending before the VM re-creation, it was not the re-creation itself that caused the pending state

@mustjab
Copy link
Contributor

mustjab commented Aug 3, 2020

The re-creation of VMs happens on Sunday at 9PM PST. I can disable it, if we don't think it's needed anymore. We should be cleaning up VM state after every run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants