Test diffing is broken

[333fred]

Type: Bug

There's some new test output auto-diffing feature that is rendering the test output window entirely useless for Roslyn. The actual test output from the log is:

[xUnit.net 00:00:00.43]   Starting:    Microsoft.CodeAnalysis.CSharp.Emit3.UnitTests
[xUnit.net 00:00:01.10]     Microsoft.CodeAnalysis.CSharp.Test.OverloadResolutionPriorityTests.ExtensionsOnlyFilteredByApplicability [FAIL]
[xUnit.net 00:00:01.10]       
[xUnit.net 00:00:01.10]       Expected:
[xUnit.net 00:00:01.10]       Actual:
[xUnit.net 00:00:01.10]                       // (9,11): error CS0121: The call is ambiguous between the following methods or properties: 'E1.R(int)' and 'E2.R(int)'
[xUnit.net 00:00:01.11]                       //         x.R(); // E1.R(int)
[xUnit.net 00:00:01.11]                       Diagnostic(ErrorCode.ERR_AmbigCall, "R").WithArguments("E1.R(int)", "E2.R(int)").WithLocation(9, 11)
[xUnit.net 00:00:01.11]       Diff:
[xUnit.net 00:00:01.11]       ++>                 Diagnostic(ErrorCode.ERR_AmbigCall, "R").WithArguments("E1.R(int)", "E2.R(int)").WithLocation(9, 11)
[xUnit.net 00:00:01.11]       Stack Trace:
[xUnit.net 00:00:01.11]         /home/fred/git/roslyn/src/Compilers/Test/Core/Diagnostics/DiagnosticExtensions.cs(99,0): at Microsoft.CodeAnalysis.DiagnosticExtensions.Verify(IEnumerable`1 actual, DiagnosticDescription[] expected, Boolean errorCodeOnly)
[xUnit.net 00:00:01.11]         /home/fred/git/roslyn/src/Compilers/Test/Core/Diagnostics/DiagnosticExtensions.cs(48,0): at Microsoft.CodeAnalysis.DiagnosticExtensions.Verify(IEnumerable`1 actual, DiagnosticDescription[] expected)
[xUnit.net 00:00:01.11]         /home/fred/git/roslyn/src/Compilers/Test/Core/Diagnostics/DiagnosticExtensions.cs(63,0): at Microsoft.CodeAnalysis.DiagnosticExtensions.Verify(ImmutableArray`1 actual, DiagnosticDescription[] expected)
[xUnit.net 00:00:01.11]         /home/fred/git/roslyn/src/Compilers/Test/Core/Diagnostics/DiagnosticExtensions.cs(109,0): at Microsoft.CodeAnalysis.DiagnosticExtensions.VerifyDiagnostics[TCompilation](TCompilation c, DiagnosticDescription[] expected)
[xUnit.net 00:00:01.11]         /home/fred/git/roslyn/src/Compilers/CSharp/Test/Emit3/OverloadResolutionPriorityTests.cs(2635,0): at Microsoft.CodeAnalysis.CSharp.Test.OverloadResolutionPriorityTests.ExtensionsOnlyFilteredByApplicability()
[xUnit.net 00:00:01.11]            at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[xUnit.net 00:00:01.11]            at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
[xUnit.net 00:00:01.11]   Finished:    Microsoft.CodeAnalysis.CSharp.Emit3.UnitTests

However, the test results window doesn't the full output anymore. It just shows the first line of the comment; the error text here tells me exactly what I need to copy into the test file, and it's very important that it shows exactly as it was produced.

This is what I see, which is just useless to me. I end up needing to go into the raw test output to retrieve the real message.

Extension version: 1.13.6
VS Code version: Code 1.95.1 (65edc4939843c90c34d61f4ce11704f09d3e5cb6, 2024-10-31T05:14:54.222Z)
OS version: Linux x64 5.15.0-124-generic
Modes:

System Info

Item	Value
CPUs	AMD Ryzen 9 5950X 16-Core Processor (32 x 2667)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off webnn: disabled_off
Load (avg)	1, 3, 4
Memory (System)	62.70GB (30.99GB free)
Process Argv	. --crash-reporter-id c9f05a69-d019-464b-b1f2-b43e5fb75046
Screen Reader	no
VM	0%
DESKTOP_SESSION	regolith
XDG_CURRENT_DESKTOP	Regolith:GNOME-Flashback:GNOME
XDG_SESSION_DESKTOP	regolith
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805cf:30301675
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
2e7ec940:31000449
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
0ee40948:31013168
dvdeprecation:31068756
dwnewjupyter:31046869
impr_priority:31102340
nativerepl1:31139838
pythonrstrctxt:31112756
nativeloc1:31134641
cf971741:31144450
iacca1:31171482
notype1cf:31157160
5fd0e150:31155592
dwcopilot:31170013
j44ff735:31179530

[peterwald]

cc @ocallesp

[AbhitejJohn]

@ocallesp : Is there a way of turning off the new diff view for results? There's got to be a VSCode setting for this.

[ocallesp]

WIP. Fix in code review

[ocallesp]

Fix was merged today. It should be available in the next release

[AbhitejJohn]

@ocallesp : Can you also share a little more context on the fix? Possibly a screenshot for the scenario Fred shared above after the fix.