Terminal logger integration

[nohwnd]

.NET 9 preview1 (since 9.0.100-alpha.1.24073.1) is shipping with MSBuild TerminalLogger enabled by default for most dotnet workloads, including dotnet test. The experience is not perfect, so I am looking at ways to improve it, together with the terminal logger team.

The change for the vstest logger to integrate better with MSBuild was done here #2702 , it traces errors via msbuild output so it is visible in binary logger viewer etc.

This experience, paired with disabling other workarounds such as not reusing test nodes is enabled by default in net9 and can be opt-out by -p:VSTestUseMSBuildOutput=false build property or by MSBUILDENSURESTDOUTFORTASKPROCESSES=1 (after #4853). The change is implemented here: dotnet/sdk#38098

My wishlist:

Temporary output (appears when it happens and is rolled out when there is too much):

• getting passed tests as temporary output on the screen
• tests can be colorized (a "bug" doubles the time, most likely this is not accounting for the ansi codes to go missing once rendered? )
  

Static output:

• identification of the "task", so we can distinguish build success from test success
• success of test assembly, similar how it is reported for successfully built project
• single summary of test run with test counts
• summary of attachments

[nohwnd]

Current bugs:

• Regression with console logger in .NET 9 #4852
  Fixed by: Fix terminal logger encoding & error #4853
• [NETSDKE2E]Garbage issue happens in new terminal logger output in CLI on Loc OS. dotnet/msbuild#9674
  Fixed by: Fix terminal logger encoding & error #4853
• Allow disabling via Env variable
  Fixed by: Suppress vstest terminal logger via environment variable dotnet/sdk#38286
  and Fix terminal logger encoding & error #4853
• Setting VSTestUseMSBuildOutput in project does not disable the output Can't opt out of VSTestUseMSBuildOutput in project file dotnet/sdk#39127

[nohwnd]

Progress so far:

What also cannot be seen here is that binlog is getting full summary of the failure, and full summary passed tests. As well as messages from outputs. E.g.

There is also one summary for the whole run, and not summary per project/tfm.

[nohwnd]

msbuild part: dotnet/msbuild@main...exp/terminal-logger-for-vstest
vstest part: nohwnd/vstest@main...terminal-logger-for-vstest

discussion points:

• colorization of "targets" (described above)

• how to evolve this (msbuild does not ship directly with fixed version of vstest, so we will have hard time changing the messages?)

• better integration pattern ? All the update messages go to binlog where it is messy
  

• allowing the new Test target to use this as well (-p:UseMSBuildTestInfrastructure=true)

• allowing MSTest runner to use this as well - we use generic target and generic names for the messages. should be good enough for now.

• detecting and disabling TerminalLogger mode, so user does not get spammed with all passed tests

• optimizations (I am for sure committing a lot of sins against string parsing best practice)

[nohwnd]

• scaling on multi tfm, when I have 4 projects, each with 3 targets (and 12 cores) I see only 4 test projects run at the same time - will be fixed by Run tests in different tfms in parallel dotnet/sdk#38569
  

Dispatch to inner needs to somehow parallelize

Solution proposed here, works great. Just needs the Property rename, because disabling it won't fully disable test parallelization. E.g. -p:TestTfmInParallel=false

[nohwnd]

Next up:

• Skipped! when all tests are skipped, rather than passed.
• support for multiline error output, which is soon coming to msbuild
• attachments in summary
• fix how trx logger outputs the file path, to be picked up be attachments
• ROLL BACK: roll back the passed and skipped tests, while we wait for proper progress api in msbuild

[nohwnd]

We ended up writing most of the output in net9-preview6, making it as useful as before.