Using TUnit in azure pipelines (test results and coverage)

[gentledepp]

Hi!

I am trying to use TUnit in azure devops pipeline.

Using the out-of-the box VSTest tasks (1,2,3) does not work.
It does find the dll, but no "platform host" => so I install the Microsoft.NET.Test.Sdk nuget but then it does not find any tests - only the DLL

- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*Tests.dll
      **\*Test.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'
    codeCoverageEnabled: False
    vsTestVersion: 'latest'

what does work is running the tests using

  - task: PowerShell@2
    displayName: "dotnet test"
    inputs:
      targetType: 'inline'
      script: 'dotnet test $(solution) -c Debug -- --report-trx --coverage'

this runs the tests, but does not show the test results in the test results page.
This is a shame because if a test fails I have no idea which one :-D

Also, the code coverege page does not get any data either:

Has anyone got that to work already?

[thomhurst]

TUnit isn't built on top of VSTest as explained in the readme. Microsoft.NET.Test.Sdk should NOT be used with TUnit as it breaks it.

The second way using dotnet test would be the way to go for now.

As for not seeing results, you'll want to raise an issue with Microsoft and ask them to add support for Microsoft Testing Platform within Azure DevOps.

[gentledepp]

wow... so fast... 😅
You are awesome... nich was right :-D

In the meantime, I managed to get it to work!

Here is my default devops pipeline:

# note: add a trigger that suits your needs
trigger:
  - develop 

# note: a windows machine is not required, but we use it anyways!
pool:
   vmImage: 'windows-latest'

variables:
  buildConfiguration: 'Debug'
  solution: 'MyAwesomeSolution.sln'
  buildPlatform: 'AnyCPU'
  
steps:
  - task: NuGetToolInstaller@1
  
  # simplest way to ensure everything you need is installed
  - task: PowerShell@2
    displayName: "dotnet workload restore"
    inputs:
      targetType: 'inline'
      script: 'dotnet workload restore src\$(solution)'

 # Note: We are used a private nuget feed in devops -> so we need to use the vstsfeed option - you may not need this!
  - task: NuGetCommand@2
    displayName: 'restore nuget packages'
    inputs:
      command: 'restore'
      restoreSolution: '**/*$(solution)'
      feedsToUse: 'select'
      vstsFeed: '<yourfeed>'
  
  # NOTE: This is where the magic happens. We specify an explicit results-directory for the subsequent tasks!
  - task: PowerShell@2
    displayName: "dotnet test"
    inputs:
      targetType: 'inline'
      script: 'dotnet test src\$(solution) -c Debug -- --report-trx --coverage  --results-directory $(Agent.TempDirectory)/TestResults'
 
  # this task publishes the test results
  - task: PublishTestResults@2
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '*.trx'
      mergeTestResults: true
      failTaskOnFailureToPublishResults: true
      failTaskOnMissingResultsFile: true
      searchFolder:  $(Agent.TempDirectory)/TestResults
  
  # and this the coverage results
  - task: PublishCodeCoverageResults@2
    inputs:
      summaryFileLocation: '$(Agent.TempDirectory)/TestResults/*.coverage'
      failIfCoverageEmpty: true
  

This works like a charm...

...except that the coverage report is "for pro users only"?

What the heck 😅
Can anybody let me know what kind of joke this is? :-D

aah... i see... the PublishCodeCoverageResults task uses the ReportGenerator which requires a paid license... ugh...

[R-Vulcan]

The DotNetCoreCLI task with command 'test' automatically publishes test results, but to make it work with TUnit i had to set the results directory again after the --.

The task does this by default but this gets ignored because its not set as an application argument (after --).

- task: DotNetCoreCLI@2
  displayName: 'Run Unit Tests'
  inputs:
    command: test
    testRunTitle: 'Unit Tests'
    projects: 'tests/**/*Tests.csproj'
    arguments: '-c Release -- --coverage --report-trx --results-directory $(Agent.TempDirectory)'

Generated CLI command in pipeline:

/opt/hostedtoolcache/dotnet/dotnet test /home/vsts/work/1/s/.../SomeTests.csproj --logger trx --results-directory /home/vsts/work/_temp -c Release -- --coverage --report-trx --results-directory /home/vsts/work/_temp

Note how the --results-directory is set a second time after the --

[frankhaugen]

@thomhurst could we have this challenge and solution in the actual docs? I have been googling quite a bit to get this going correctly and it was in the act of going to ask a question here I stumbled over this discussion. A small "CI" section with DevOps examples would be nice.

Any objections or should I make a PR?

[thomhurst]

I didn't realise it was a challenge as CI is mostly just running the CLI so dotnet run or dotnet test. You can make a PR, sure. Under the docs examples bit I reckon

[frankhaugen]

I didn't realise it was a challenge as CI is mostly just running the CLI so dotnet run or dotnet test. You can make a PR, sure. Under the docs examples bit I reckon

Its basically the small flag -- that's not blatantly stated in MS's docs about the TrxReports -lib even. So a working example just makes it easier: PR #2041

[agray]

I think you will find ReportGenerator doesn't NEED a licence. You CAN get a pro licence if you REALLY need it.