Exiting when no tests are reported? #2832
-
|
We recently switched to using the solution-level mutation testing for our GitHub repos (where we use Stryker to gate PRs) and I noticed that some of our repos will fail Stryker because they do not have any tests. It seems to me that, at least for our use, it makes sense for Stryker to report a score of 0 (basically saying no test coverage=no mutant coverage) rather than exiting with the message "No test has been detected." Previously, using the project-level testing, it would just return a score of 0 with this message:
So I guess I'm wondering what the reasoning is behind having Stryker exit in this scenario instead of returning a score of 0? Maybe with a message telling you to add some tests? And if there is value in making this change or having the option for this behavior? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
It probably is an oversight in the implementation of solution mode instead of a conscious decision however it does make sense. In project mode we know what your test project is so we can more or less safely determine that all our processing is correct and that you simply don't have tests. However solution mode tries to match your test and sut projects automatically for you. I feel much less confident that there really are no unit tests in a whole solution, rather we did something wrong so we must error. Why are you running Stryker on solutions without unit tests? |
Beta Was this translation helpful? Give feedback.
-
|
Okay, makes sense. We have stryker running on all of our repos as part of our CI/CD setup, so because some of our repos do not have any unit tests (for whatever reason, some of the code may be complicated to test) stryker is still running and failing on them. So our options here would be to either create tests (possibly dummy ones) or to run at the project level (not favorable since many of our repos contain more than one project). |
Beta Was this translation helpful? Give feedback.
-
|
Would it not make more sense to disable Stryker for those repos that do not have tests? Make it a step you can disable with a variable. It seems like a waste of time to start Stryker on repos where you already know you won't get a useful result. Your ci feedback loop is imo too valuable to waste in that way. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the tip, I'll see about implementing it into our workflow. |
Beta Was this translation helpful? Give feedback.
It probably is an oversight in the implementation of solution mode instead of a conscious decision however it does make sense. In project mode we know what your test project is so we can more or less safely determine that all our processing is correct and that you simply don't have tests. However solution mode tries to match your test and sut projects automatically for you. I feel much less confident that there really are no unit tests in a whole solution, rather we did something wrong so we must error.
Why are you running Stryker on solutions without unit tests?