Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reveal flaky test [WIP] #21793

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

lacostej
Copy link
Collaborator

See #21792

Does it also fail on CI?

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.
  • I've added or updated relevant unit tests.

Motivation and Context

Description

Testing Steps

@lacostej
Copy link
Collaborator Author

lacostej commented Jan 14, 2024

Here's one example of the issue I am trying to uncover being triggered.

https://app.circleci.com/pipelines/github/fastlane/fastlane?branch=pull%2F21799
https://app.circleci.com/pipelines/github/fastlane/fastlane/6110/workflows/6f6fa7ff-6f2e-4fc1-8664-bb17b55b0a15/jobs/95892/tests#failed-test-0

The current PR is just trying to show the issue but hits a ulimit issue. I'll first try to reproduce it here.

update: the PR now reproduces the issue.

@lacostej lacostej marked this pull request as draft February 12, 2024 16:38
@lacostej lacostej force-pushed the fix/command_executor_flaky_test branch 2 times, most recently from da34e30 to 7c88f14 Compare February 28, 2024 06:15
limit amount of tests to not reach CircleCI limits

increase the amount of times the tests are run

Rubocop

Reduce amount of tests to please the CI gods
…ail with 'Can't get Master/Slave device' error
@lacostej lacostej force-pushed the fix/command_executor_flaky_test branch from 7c88f14 to e9d6667 Compare February 28, 2024 06:24
…astlane#21792)

We don't have a simple way to disable the buffering in the terminal that is cross platform but quoting the parameter to echo does the job.
@lacostej lacostej force-pushed the fix/command_executor_flaky_test branch from e9d6667 to 349c676 Compare February 29, 2024 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant