MSTest analyzers

[Evangelink]

Summary

This ticket will host rules suggestions so we can discuss and order them to implement most requested/critical ones first.

Rules

Title	Category	Description	Default severity	Priority (for implementation)
Avoid test method prefixes	Naming	Test method name should not start with Test, Verify or Ensure (case-sensitive)	Disabled	Low
Test method names should follow given pattern	Naming	Test method name should follow a given regex pattern. The pattern can be defined in .editorconfig file.	Disabled	Low
Test methods should be defined only on test classes	Usage	Test methods should only appear on test classes	Warning	High
Refactor test method to not use [DataRow]	Design	If a method accepts only a single [DataRow] it should be refactored to not require arguments	Info	Medium
Test method with parameters should have a data attribute	Design	Parameterized test methods should be marked with some data attribute or should be refactored to not have parameters.	Warning	High
Remove TestContext property if it is not used	Design	...	Info	Low

• Add rule for each attribute that should be set only on test methods
• Add rules for usage of assertions
  • Do not do
    • Assert.IsTrue(true)
    • Assert.IsFalse(false)
    • Assert.IsTrue(a == b)
    • Assert.IsFalse(a == b)

Tasks

Beta Give feedback

We're having trouble rendering your tasklist, it's most likely has one of the following formatting errors:

• indented tasks (i.e. nested tasklist; support for this is coming!)
• an empty task (i.e. - [ ] on a line by itself)
• any empty new lines (before/after the tasklist or in between tasks)
• duplicate issue/pr links
• a draft task exceeds 512 characters

Please check the tasklists documentation for more information. 🔗 Feedback Discussion

Thank you for participating in the Private Beta ❤️

No tasks being tracked yet.

[gao-artur]

Another suggestion: ensure Test/Class/Assembly Initialize/Cleanup methods have a correct signature.

[gao-artur]

Suggestion: Ensure the TestContext parameter received in the method annotated with the Class/Assembly Initialize attribute is not stored in a static class field (this is an actual mistake I see a lot in tests. The static field is then used in the test with the assumption it holds the current test context)

[Evangelink]

@gao-artur Adding your suggestions to the table above.

[Evangelink]

Few extra cases or analyzers, not sure how it's best to handle:

• Test/Class/Assembly init/cleanup should be specified only once per type
• Test/Class/Assembly init/cleanup should be on a class marked wiht TestClass (or inheriting)

[Evangelink]

New rule: When a generic class is marked as [TestClass] it should be made abstract to clarify intent which is that the test methods, test init/cleanup and class init/cleanup (if marked as inherited) will be run only through subclasses. If the abstract is not specified, it could imply that the user expects that the tests/init/cleanups would be called which isn't the case.