Dogfood --report-junit and OpenTelemetry processing in CI by Evangelink · Pull Request #8965 · microsoft/testfx

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

Amaury Levé (Evangelink) · 2026-06-09T12:58:53Z

Follow up to the AzDO/TRX dogfooding pattern (#0028908d): now also exercise the newest report extension (JUnitReport, #8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs.

JUnit Report (`--report-junit`)

test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path.
azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation).
Each test project's Program.cs explicitly calls AddJUnitReportProvider(). TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReference consumers.

OpenTelemetry processing

test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference.
Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders.
No exporter is wired. The SDK still creates real Activity/Counter instances because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is then dropped at the export stage. That's sufficient to exercise the handler (which has been actively optimised recently, e.g. [efficiency-improver] perf: single-pass PropertyBag walk in OpenTelemetryResultHandler.HandleTestResult #8938) without polluting CI logs. We can wire an actual exporter later if we want to publish the data anywhere.
VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match the existing AOT-aware Program.cs pattern there (the OTel SDK uses reflection).

Verification

Built all 10 affected test projects locally (Debug).
Smoke-ran Microsoft.Testing.Extensions.UnitTests end-to-end with both extensions registered: 431 tests pass and a ~230 KB JUnit XML report is emitted (<testsuites name="Microsoft.Testing.Extensions.UnitTests" tests="431" failures="0" ...>).

Copilot

Pull request overview

This PR extends the existing CI dogfooding pattern for MTP extensions by enabling JUnit XML reporting (--report-junit) and exercising the OpenTelemetry result handling pipeline during the repo’s own unit test runs.

Changes:

Updated multiple test-host Program.cs entrypoints to register AddJUnitReportProvider() and to wire AddOpenTelemetryProvider(...AddTestingPlatformInstrumentation...).
Updated test/Directory.Build.targets to add ProjectReferences for Microsoft.Testing.Extensions.JUnitReport and Microsoft.Testing.Extensions.OpenTelemetry, and to inject --report-junit / --report-junit-filename into TestingPlatformCommandLineArguments.
Updated Azure Pipelines scripts to pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" when running dotnet test --test-modules (which bypasses MSBuild evaluation).

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SourceGeneration.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.SelfRealExamples.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/MSTest.Analyzers.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/UnitTests/Microsoft.Testing.Extensions.VSTestBridge.UnitTests/Program.cs	Registers JUnit report and conditionally wires OpenTelemetry when not `NATIVE_AOT`.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Program.cs	Registers JUnit report + OpenTelemetry instrumentation in this unit test host.
test/Directory.Build.targets	Adds extension `ProjectReference`s and injects JUnit CLI options into MTP runs.
eng/pipelines/steps/test-non-windows.yml	Adds JUnit CLI options to the Release `--test-modules` path.
azure-pipelines.yml	Adds JUnit CLI options to the Release `--test-modules` path on Windows.
.github/workflows/shared/grade-tests-shared.md	Documentation update: switch guidance example table to raw HTML for fixed column widths.

Copilot's findings

Files reviewed: 14/14 changed files
Comments generated: 1

Follow up to the AzDO/TRX dogfooding pattern: now also exercise the newest report extension (JUnitReport, microsoft#8850) and the OpenTelemetry processing pipeline end-to-end as part of our own CI test runs. JUnit: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.JUnitReport as a ProjectReference for every test project that opts into MTP and wires --report-junit / --report-junit-filename into the MSBuild path. - azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml also pass --report-junit --report-junit-filename "{asm}_{tfm}.xml" on the Release dotnet test --test-modules path (which bypasses MSBuild evaluation). - Each test project's Program.cs explicitly calls AddJUnitReportProvider() (TestingPlatformBuilderHook auto-registration only fires for NuGet consumers, not source ProjectReferences). OpenTelemetry: - test/Directory.Build.targets adds Microsoft.Testing.Extensions.OpenTelemetry as a ProjectReference. - Each test project's Program.cs registers AddOpenTelemetryProvider with AddTestingPlatformInstrumentation on both the tracer and meter builders. No exporter is wired: the SDK still creates real Activities and Counters because the sources/meters now have listeners, so data flows through the full OpenTelemetryResultHandler pipeline and is dropped at the export stage. Sufficient to exercise the handler in CI without polluting logs. - VSTestBridge.UnitTests gates OTel registration on !NATIVE_AOT to match its existing AOT-aware Program.cs pattern (the OTel SDK uses reflection). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… programs Copilot reviewer caught (and Linux/Windows Debug CI confirmed): the 5 integration/acceptance test projects under test/IntegrationTests/ also opt into the Microsoft.Testing.Platform via EnableMSTestRunner / UseInternalTestFramework, so they pick up the --report-junit / --report-junit-filename arguments injected from test/Directory.Build.targets, but they never registered the JUnitReport extension in their explicit Program.cs files. The Debug runs failed at startup with `Unknown option 'report-junit'`. Add AddJUnitReportProvider() alongside the existing AddTrxReportProvider() call and an AddOpenTelemetryProvider(...) block (mirroring the 10 unit-test Program.cs files) to: - PlatformServices.Desktop.IntegrationTests - MSTest.Acceptance.IntegrationTests - Microsoft.Testing.Platform.Acceptance.IntegrationTests - MSTest.IntegrationTests - MSTest.VstestConsoleWrapper.IntegrationTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-09T14:33:31Z

Rebased onto latest main to drop the two stale commits from PR #8961 (f53ecc246 + 1bad60019) that were already squash-merged as 2615f8bd7. gh pr diff now lists only the 18 files this PR actually owns .github/workflows/shared/grade-tests-shared.md is gone.

For the Windows Release failure, two things were going on in the previous run:

Failed to create a unique log file after 3 seconds (8 unit-test EXEs log_yyMMddHHmmssfff.diag). FileLogger.CreateFileStream only has millisecond precision and reuses the same DateTimeOffset on retry, so concurrent test hosts under heavy --diagnostic traffic spin on the same filename until the 3s timeout. This is a pre-existing infra bug on main (filed mentally as a follow-up); the extra extension load this PR adds (JUnitReport + OpenTelemetry) likely just pushed startup contention over the edge.
Watchdog_EmitsAfterQuietWindow_WhenStillTracking / Watchdog_EmitsForLateTracking_WhenQuietWindowElapsedWithNoWork (ShutdownProgressReporterTests) timing-sensitive assertions Expected value to be greater than or equal to the lower bound; took 7.9s and 15.8s respectively, consistent with the agent being oversubscribed from Initial commit! 🎉 #1 above.

Re-running CI on the rebased commit will follow up if either reproduces.

CopilotAI review requested due to automatic review settings June 9, 2026 12:58

Copilotstarted reviewing on behalf of Amaury Levé (Evangelink)June 9, 2026 12:59 View session

CopilotAI reviewed Jun 9, 2026
View reviewed changes

Comment threadtest/Directory.Build.targets

Amaury Levé (Evangelink)and others added 2 commits June 9, 2026 16:30

Amaury Levé (Evangelink)force-pushed the dev/amauryleve/dogfood-junit-otel branch from 889d67a to d6edbcaCompare June 9, 2026 14:32

Amaury Levé (Evangelink) enabled auto-merge (squash) June 10, 2026 08:37

YuliiaKovalova approved these changes Jun 10, 2026
View reviewed changes

Amaury Levé (Evangelink) disabled auto-merge June 10, 2026 09:21

Amaury Levé (Evangelink) merged commit 662eede into microsoft:mainJun 10, 2026
35 of 37 checks passed

Amaury Levé (Evangelink) deleted the dev/amauryleve/dogfood-junit-otel branch June 10, 2026 09:22