Report interesting state in progress bar

[nohwnd]

Summary

Tests that do long running work, or have interesting state to share should be able to do so via the current progress bar.

Background and Motivation

It is nice to see that test counts are increasing, but when tests don't increase user is left in the dark about what is happening.

There can be different reasons for this, such as:

• test simply takes a lot of time
• there is setup needed before all tests - e.g. build of tested projects, download of tested artifact
• test is stuck

Proposed Feature

Add a status bar indicator that is per assembly, and a new type of message that can be sent similar to TestExecutionTimingInfo that will update the status (per-assembly).

The runner is given information about the progress update being available, frequency in ms, and available output width.

The message will contain:

• the status as a string. e.g. Running test: abc
• optional UID of a resource that the message is about: this id is opaque, but can be used to dismiss a message when test completes / etc. The goal here is to avoid showing message when the test updated and we got a complete result. And avoid having to send a specific property for the dismissal of a message that might not be on the screen anymore.
• optional bool to make the message stick on screen until next update: this is to avoid having to keep track of long-running tests just to update the UI every 3 seconds.

The control of what the content of the message will be is left to the executor, as well as the frequency on which it will be sent. But the sender should consider that:

• Not every message is guaranteed to be shown

• Longer messages are trimmed, only 1 line is available

• Message stays on screen until:

  • the next message,
  • or 5 seconds if not sticky
  • or until being dismissed because the resource with the same uid finished.

• The progress bar is rendered every 500ms/3s (currently), more frequent messages only put pressure to the communication channel.

Examples

{
   "status": "Building solution TestAssets.sln"
}

{
   "status": "Restoring solution TestAssets.sln"
}

{
   "status": "Test <abc> is taking more than 10 seconds"
   "sticky": true
   "uid": "abctestuid"
}

// test node update
{
   "uid": "abctestuid"
    // test completed passed
}

You'd see:

[+0/-0/?0] Abc.dll (net8.0|x64)                         (0s)

[+0/-0/?0] Abc.dll (net8.0|x64)                         (1s)
  Building solution TestAssets.sln

[+0/-0/?0] Abc.dll (net8.0|x64)                         (2s)
  Restoring solution TestAssets.sln  // message was 
  // rewritten because it is newer

[+0/-0/?0] Abc.dll (net8.0|x64)                         (7s)
  // message went away after 5s it was not sticky

[+0/-0/?0] Abc.dll (net8.0|x64)                         (45s)
   Test <abc> is taking more than 10 seconds 
   // stays on screen it is sticky and 
   // we don't send any other updates

[+1/-0/?0] Abc.dll (net8.0|x64)                         (1m 40s)
   // message went away because we received test 
   // completed, and the uid, and we called 
  // "DismissMessageForUid"