Add Microsoft.Testing.Extensions.JUnitReport extension (#4268)

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4

[Amaury Levé (Evangelink)]

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.JUnitReport that emits JUnit-format XML reports compatible with Jenkins, GitLab, Azure DevOps and other CI parsers.

Closes#4268

CLI

• --report-junit — enable the report
• --report-junit-filename <name> — override the default filename

Design

Full RFC: docs/RFCs/016-JUnit-Report.md

Highlights:

• Flat <testsuites>/<testsuite> for broad parser compatibility. Nested <testsuite> was rejected because Jenkins/Surefire handle depth-2+ inconsistently.
• MTP tree-of-tests preserved via a per-testcase <property name="testpath" value="A/B/C"/>. This is the v1 representation; we can evolve later if/when parsers gain richer hierarchy support.
• Strict <testcase> child ordering (properties → skipped/error/failure → system-out/-err) — required by Surefire/Jenkins.
• Atomic write: temp file + rename to avoid partial reports.
• Defensive caps: MaxTestPathLength = 64K, MaxParentChainDepth = 1024 with cycle detection; bounded stream/message/identity/trait sizes.
• Outcome mapping: passed → empty body, skipped → <skipped>, failed → <failure>, errored/timedOut → <error>. Discovered/InProgress nodes are not emitted as testcases but participate in the parent-chain capture.
• XML safety: XmlWriterSettings.CheckCharacters = false + XmlSafeText helper replacing invalid chars with U+FFFD.

Implementation

• 17 files under src/Platform/Microsoft.Testing.Extensions.JUnitReport/ (net8.0, net9.0, netstandard2.0) modelled on the existing HtmlReport extension.
• MSBuild auto-registration GUID 9F3CD35A-4E62-43EF-B7F1-426D30E850ED wired into Microsoft.Testing.Platform.csproj and TestFx.slnx.
• Localized resources mirrored from HtmlReport; XLF regenerated via UpdateXlf (never hand-edited).

Tests

All green locally:

• 5 new acceptance tests × 3 TFMs = 15/15 passing (JUnitReportTests):
  • JUnit_WhenReportJUnitIsNotSpecified_JUnitReportIsNotGenerated
  • JUnit_WhenReportJUnitIsSpecified_JUnitReportIsGeneratedInDefaultLocation
  • JUnit_WhenReportJUnitFilenameIsSpecified_JUnitReportIsGeneratedWithThatName
  • JUnit_WhenReportJUnitFilenameContainsPath_JUnitReportIsGeneratedInThatPath
  • JUnit_WhenTestsFailOrSkip_JUnitReportContainsExpectedOutcomes
• HelpInfoAllExtensionsTests and MSBuild.KnownExtensionRegistration updated with the new option/registration assertions (43/43 passing).

Verification commands

.\build.cmd -pack -c Debug
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~JUnitReportTests"
dotnet run --project test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj --no-build -c Debug -- --filter "FullyQualifiedName~HelpInfoAllExtensionsTests|FullyQualifiedName~KnownExtensionRegistration|FullyQualifiedName~HelpInfoTests"

Compatibility note

The extension option --report-junit collides with xUnit pre-4.0's identically-named CLI option. This is fatal-by-design and documented in the RFC: enabling both at once will fail fast.

[Copilot]

Pull request overview

Adds a new Microsoft.Testing.Platform (MTP) extension, Microsoft.Testing.Extensions.JUnitReport, to generate JUnit-style XML reports (CLI + MSBuild known-extension registration), along with acceptance-test and help/info updates.

Changes:

• Introduces the new Microsoft.Testing.Extensions.JUnitReport extension (capture + engine + CLI options + MSBuild builder hook + packaging assets).
• Updates acceptance/integration tests to cover registration and basic report generation + updates help/info expected output for “all extensions”.
• Adds an RFC documenting schema/design choices and wires the new project into the solution/platform internals visibility.

Show a summary per file

File	Description
TestFx.slnx	Adds the new JUnitReport project to the solution.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj	Grants InternalsVisibleTo for the new extension assembly.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs	Extends known-extension registration assertions to include JUnitReport.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs	Adds acceptance tests for JUnit report generation and filename validation.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates --help/--info expected output to include JUnitReport options/provider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Microsoft.Testing.Extensions.JUnitReport.csproj	New extension project definition (TFMs, packaging layout, linked helpers).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportEngine.cs	Core JUnit XML generation logic (suites/testcases, XML writing, atomic write).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGenerator.cs	Session handler + data consumer that captures test results and emits the artifact.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportGeneratorCommandLine.cs	Defines/validates --report-junit and --report-junit-filename.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/JUnitReportExtensions.cs	Public builder extension AddJUnitReportProvider.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestingPlatformBuilderHook.cs	MSBuild hook entrypoint for known-extension registration.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/TestResultCapture.cs	Projects TestNodeUpdateMessage into capped DTOs and tracks parent chain entries.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/CapturedTestResult.cs	DTO representing a captured terminal test result.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/BannedSymbols.txt	Adds banned API usage rules for the new project.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PACKAGE.md	NuGet package readme for the new extension.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/PublicAPI/PublicAPI.Unshipped.txt	Declares new public APIs for API review/baseline.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/build/Microsoft.Testing.Extensions.JUnitReport.props	Build assets for MSBuild consumption.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildTransitive/Microsoft.Testing.Extensions.JUnitReport.props	Transitive build assets import.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/buildMultiTargeting/Microsoft.Testing.Extensions.JUnitReport.props	MSBuild known-extension registration item + GUID.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/ExtensionResources.resx	Adds localized strings for CLI/help/errors/artifact metadata.
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.cs.xlf	Localized resource XLF (cs).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.de.xlf	Localized resource XLF (de).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.es.xlf	Localized resource XLF (es).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.fr.xlf	Localized resource XLF (fr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.it.xlf	Localized resource XLF (it).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ja.xlf	Localized resource XLF (ja).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ko.xlf	Localized resource XLF (ko).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pl.xlf	Localized resource XLF (pl).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.pt-BR.xlf	Localized resource XLF (pt-BR).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.ru.xlf	Localized resource XLF (ru).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.tr.xlf	Localized resource XLF (tr).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hans.xlf	Localized resource XLF (zh-Hans).
src/Platform/Microsoft.Testing.Extensions.JUnitReport/Resources/xlf/ExtensionResources.zh-Hant.xlf	Localized resource XLF (zh-Hant).
docs/RFCs/016-JUnit-Report.md	Adds the RFC describing schema/design/testing strategy.

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 8

[Youssef Fahmy (Youssef1313)]

Is there a specification for JUnit that this implementation followed?

IIRC, there were multiple versions of JUnit results file (possibly with breaking changes?). Have you researched that? If so, can we explicitly document what version this is?

[Amaury Levé (Evangelink)]

Youssef Fahmy (@Youssef1313) great question — short answer: there is no single canonical ""JUnit XML"" specification. The format originated as the XML output of the Ant <junit> task and has since been re-implemented (with slight variations) by Jenkins, Maven Surefire, pytest, NUnit, xUnit, and many others.

For this extension I targeted the Jenkins / Maven Surefire flavor, which is the closest thing to a de-facto industry standard and the one consumed by the widest set of CI systems (Jenkins, Azure DevOps, GitLab, CircleCI, etc.). This is documented in RFC 016 (docs/RFCs/016-JUnit-Report.md) under the ""Schema choice"" section.

Sources I consulted while designing the schema and writing the emitter:

• Jenkins junit.xsd
• Apache Maven Surefire XSD
• testmoapp/junitxml — community write-up of the de-facto fields
• xUnit MTP extension --report-junit output (cross-referenced for naming / attribute conventions)
• pytest --junitxml output
• NUnit nunit3-junit.xslt transform

If you'd like, I'm happy to add the explicit links above as a Normative references section in the RFC so the schema lineage is preserved alongside the design.

[Copilot]

Copilot's findings

• Files reviewed: 35/35 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Amaury Levé (@Evangelink) follow-ups addressed in bfc4b46:

1. Flavors comparison table — Added a new table in RFC 016 listing the differences between Ant, Maven Surefire, Jenkins, IntelliJ, and Gradle JUnit flavors (root element, attributes, retry handling, <system-out> placement). See the "JUnit XML flavors" section.

2. Retry handling — Expanded the RFC's retry section to cover both modes:

• MSTest [Retry(N)]: only the final outcome is reported (UnitTestRunner.cs calls retryResult.TryGetLast()). One <testcase> per test, matching what users see in the test explorer.
• MTP --retry-failed-tests: each attempt re-runs the entire test-host process and produces its own JUnit XML file in a per-attempt subdirectory. No in-file aggregation.
• When duplicate names occur within a single report (e.g. parameterized tests, theory data), the file suffixes with [attempt N] starting at 1.

3. Full-XML acceptance assertions — Refactored Microsoft.Testing.Platform.Acceptance.IntegrationTests/JUnitReportTests.cs to parse the report with XDocument and added AssertWellFormedJUnitReport + AssertTestcaseChildOrdering helpers. Volatile fields (time, timestamp, hostname, id) are simply not asserted on; deterministic fields use exact matches.

4. MSTest-side acceptance tests — New file MSTest.Acceptance.IntegrationTests/JUnitReportTests.cs with two tests:

• JUnitReport_WithMSTestRetryAttribute_EmitsFinalOutcomePerTest — uses [Retry(3)], asserts exactly one <testcase> per test with the final outcome.
• JUnitReport_WithMTPRetryFailedTestsExtension_EmitsOneReportPerAttempt — uses --retry-failed-tests 3, asserts two XML files (one per attempt) in separate subdirectories.

About the spec Youssef asked about: I didn't follow a single official spec — JUnit XML has no canonical schema. The RFC documents which flavors were consulted (Ant, Maven Surefire, Jenkins, IntelliJ, Gradle) and which conventions were adopted. The "JUnit XML flavors" section in RFC 016 now spells out the differences explicitly. The closest thing to a normative reference is the Apache Ant <junit> task schema, which we follow for the core element/attribute names.

Build is green, all acceptance tests pass locally (21 MTP + 2 MSTest).

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 1

[Amaury Levé (Evangelink)]

Follow-ups:

• ✅ Experimental marker: the public surface is already gated on TPEXP:

  • [Experimental(""TPEXP"", UrlFormat = ""https://aka.ms/testingplatform/diagnostics#{0}"")] on JUnitReportExtensions
  • [TPEXP] prefix in PublicAPI/PublicAPI.Unshipped.txt
  • > **⚠️ Experimental:** callout in PACKAGE.md

  This mirrors Microsoft.Testing.Extensions.HtmlReport exactly. The TestingPlatformBuilderHook class is intentionally not marked (same as HtmlReport).

• ✅ The last open review thread (BuildTestPath truncation marker) was addressed in 968befa and is now resolved.

[Amaury Levé (Evangelink)]

Also switched to the experimental <PreReleaseVersionLabel>alpha</PreReleaseVersionLabel> versioning scheme (with <SuppressFinalPackageVersion>true</SuppressFinalPackageVersion>), so the package now ships as 1.0.0-alpha.* like the other experimental MTP extensions (Logging, OpenTelemetry, AI) rather than the repo-default -preview.* label. Commit: 9ae7335

[Copilot]

Copilot's findings

• Files reviewed: 38/38 changed files
• Comments generated: 2

[Amaury Levé (Evangelink)]

Follow-up commits on this branch:

• 9ae7335fc — Apply the experimental versioning scheme (VersionPrefix=1.0.0, PreReleaseVersionLabel=alpha, SuppressFinalPackageVersion=true) to Microsoft.Testing.Extensions.JUnitReport.csproj.
• f5ddd3465 — Convert the JUnit acceptance test from XDocument-based structural checks to verbatim full-content snapshots covering both the single-passing-test default and the mixed-outcomes (pass/fail/skip/error across two suites) scenario. Runtime-variable XML attributes (time, timestamp, hostname) are folded into stable tokens via attribute-scoped regexes; everything else (attribute order, suite ids, counts, exit-code, <properties> contents, self-closing <failure>/<skipped>/<error> elements) must match byte-for-byte.
• 203b803f8 — Plumb a dedicated MicrosoftTestingExtensionsJUnitReportVersion property/placeholder through AcceptanceTestBase so the asset csproj can reference the (now-experimental) JUnitReport package at the version actually produced by the local pack.

All 21 JUnitReportTests pass locally against the new snapshots.

[Amaury Levé (Evangelink)]

Follow-up commit on this branch:

• 14257d1 — Bound RawUid/ParentRawUid/_parentChain keys to MaxIdentityFieldLength (addresses the two outstanding identity-bounding review comments) and switch HelpInfoAllExtensionsTests to the dedicated \\$ placeholder so the test no longer requests the platform version (fixes the NU1102 CI failure).

[Copilot]

Copilot's findings

• Files reviewed: 39/39 changed files
• Comments generated: 4