Make it clear that you cannot filter the list of tests that get displayed

[rfrancioso-gpsw]

• In my pipeline I was trying to output a clean list of all the tests that would get run, before actually running them.
• When I do run tests, I use a filter, so I was also trying to use the same filter when listing tests.
• I spent over an hour trying to understand why the filter I was specifying was not working.
• In the end it ended up being because a specified filter has no effect when listing tests; only when running tests.

I've updated the documentation to make this clear. It would be nice if VSTest was also updated at some point in the future to actually support filtering when listing.

[prmerger-automator]

@rfrancioso-gpsw : Thanks for your contribution! The author(s) have been notified to review your proposed change.

[learn-build-service-prod]

Learn Build status updates of commit 39686b0:

✅ Validation status: passed

File	Status	Preview URL	Details
docs/test/vstest-console-options.md	✅Succeeded

For more details, please refer to the build report.

For any questions, please:

• Try searching the learn.microsoft.com contributor guides
• Post your question in the Learn support channel

[prmerger-automator]

@rfrancioso-gpsw : Thanks for your contribution! The author(s) have been notified to review your proposed change.

[Jak-MS]

@Mikejo5000

• Can you review this PR?
• IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

[Mikejo5000]

@Evangelink Does anyone on the team need to review this PR? It looks OK to me.

[Evangelink]

I would rather update the /TestCaseFilter entry to mention it's not working with list tests but that's a personal feeling so I am not blocking here.

[Mikejo5000]

#sign-off OK, thanks. It can be updated again later.

[rfrancioso-gpsw]

@Evangelink Related to test case filtering...

Would anyone on your team be able to answer this by any chance? I'm really hoping there's a way to accomplish this, but it doesn't seem like there is. If there isn't it would be nice if this functionality could be added in a future release.

https://stackoverflow.com/questions/78144390/how-to-run-all-unit-tests-that-dont-have-a-trait

[Evangelink]

Sure thing, it will depends on the test framework you are using so I'll let you have a look at this doc page https://learn.microsoft.com/dotnet/core/testing/selective-unit-tests

[rfrancioso-gpsw]

Thanks for the link @Evangelink. I took a look, but all the operators that are called out there don't seem to handle the following scenario. Imagine you have two categories of tests:

• Regular Unit Tests
  • These don't have an assigned category (because there's nothing special about them).
• Slow Tests
  • These do have an assigned category: ([TestCategory("Slow")])

For a PR build, you now want to run all Unit Tests that don't have an assigned category. These don't work:
/TestCaseFilter:"TestCategory="
/TestCaseFilter:"TestCategory=''"

I know you can do this instead: /TestCaseFilter:"TestCategory!=Slow"

But imagine that in addition to there being "Slow", there's also 10 other categories that need to be excluded as well. And when new categories get added in the future, you don't want to have to keep updating your filter.

Is there no way to simply select all tests that haven't been assigned a category?

[Evangelink]

Sadly it's not possible and it's a limitation of VSTest, see microsoft/vstest#2818