Add option to not use SHA1 and don't use it for default test case Id #15260
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is a draft of change that we would like to make in vstest. For a long time we've been using sha1 to hash the test ids, which we want to remove for these reasons:
We would like to replace with xxhash128, and a versioned guid (v8 guid, with 4 bits allocated for our version, so we can tell the guid is new and what hashing algorithm was used - not implemented in the code yet)
Impact
MSTest
MSTest already uses xxhash128 in v4. v3 relies on object model 17.14 which does not have the change.
NUnit
Nunit relies on eqthash, or keeps the ID to the default, depending on the setting UseNUnitIdforTestCaseId which is said to maybe cause problems, so it defaults to testcaseId]
https://github.com/nunit/nunit3-vs-adapter/blob/master/src/NUnitTestAdapter/TestConverter.cs#L194-L197
https://grep.app/search?f.repo=nunit%2Fnunit3-vs-adapter&q=eqthash.guidfromstring
XUnit
Our code is not used, but SHA1 is used. Might be a good idea to move to xxhash128 as well.
https://github.com/xunit/visualstudio.xunit/blob/28c6028b64ffb700922e1a0c8d99ea041bfd03fa/src/xunit.runner.visualstudio/Sinks/VsDiscoverySink.cs#L121
BenchmarkDotNet
https://github.com/dotnet/BenchmarkDotNet/blob/e2d30d37433444b0f43088727a14885d76c56bfc/src/BenchmarkDotNet.TestAdapter/BenchmarkCaseExtensions.cs#L85-L93
Should move to the new testIdProvider.
Others
No other usages for eqtHash and testIdProvider were found on grep.app, so many people probably depend on the default behavior implemented in TestCase.
Opt-out
To be defined.
Environment variable and runsetting seem like the natural choice to set this per run, but that also seems quite annoying.
Assembly level attribute could be set by a test framework via build targets to opt-out.
We could also publish a way to get the default sha1 based test ID, so the adapters can set it totp TestCase.Id, and avoid the built in way to calculate the id.
https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs#L173
In any case the goal is to remove the SHA1 based api soon.