Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests are flakey and unstable #382

Open
runspired opened this issue Aug 4, 2022 · 1 comment
Open

Tests are flakey and unstable #382

runspired opened this issue Aug 4, 2022 · 1 comment

Comments

@runspired
Copy link
Contributor

tests will fail in unpredictable patterns, and this is somewhat hidden by CI timing out. Obervable in #381 where a readme change leads to failures.
@mixonic
Copy link
Contributor

mixonic commented Aug 4, 2022

There are two issues:

  • First, there are unpredictable failures in the test suite. These are papered over by a retry script which re-runs the whole suite on CI if there is a failure, and only gives up after several attempts.
  • Second, there is something in the test suite or runner which causes the github action to hang. This second item is what the timeout circumvents. Without a timeout, it simply hangs for hours.

mixonic added a commit that referenced this issue Aug 4, 2022
@mixonic
Lower a CI timeout back to 9 minutes
c0414ec 
There is something in the vertical collection test suite which
periodically hangs the action runner. Extending the timeout to 15
minutes doesn't make it more likely we get a green build, instead it
increases the delay you need to wait before retrying.

Relates to #382
@mixonic mixonic mentioned this issue Aug 4, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mixonic @runspired