Show failure details in GitHub Actions step-summary collapsible sections

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Azat Mukhametshin (azat-msft)]

Fixes#10591

What

The GitHub Actions step summary listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception:System.InvalidOperationException

Location:src/Calc.cs:42

Expected: 42
Actual: 41
at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list. Existing GitHub error/warning annotations are unchanged.

Bounding the output

GitHub caps a job summary at 1 MiB and drops it entirely when exceeded — it does not truncate. Every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, [... truncated] appended
Message rows	30 lines	clipped, [... truncated] appended
Stack trace length	4,000 chars	clipped, [... truncated] appended
Stack trace rows	30 frames	clipped, [... truncated] appended
Failure list	20 per project	Showing the first 20 of N failed tests
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

The budget is shared, not per-section: the cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone. The final size check is made under the writer lock, in bytes, so two concurrent projects cannot both conclude they fit.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

• Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
• The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Testing

• 21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic, and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
• 2 acceptance tests driving a real MTP session with an exception-carrying failure.
• HelpInfoAllExtensionsTests--help / --info expectations updated.
• End-to-end runs in CI across green, small-detail, oversized-detail, 5,000-failure and 30-project shapes: azat-msft/gh-report-validation.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

[Azat Mukhametshin (azat-msft)]

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

1. Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
2. Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
3. Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

• A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

 GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

• This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

• remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

 if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

• The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

• The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

 string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

• This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

 int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
int perModuleBudget = detailsBudget / moduleCount;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

• The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

 GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

• remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

 if (detailsBuilder.Length > remainingBudget)

[Copilot]

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

• The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

 int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

• When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

 if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

• An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

 return GitHubActionsFailureDetails.MaxTotalDetailsLength;

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 124.7 AIC · ⌖ 1.29 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:324

• The shared budget starts only after the aggregate-level coverage table has already been appended. CiCoverageSummary.Aggregate preserves every module's coverage thresholds, so a large multi-module run can exceed the 1 MiB cap in this preamble before any SummaryStage can degrade it; the condensed fallback renders the same preamble and is refused too. Please bring aggregate coverage under the byte budget (or explicitly omit/truncate it) before rendering modules.

 var budget = SummaryBudget.ForAggregate(alreadyWrittenBytes + Encoding.UTF8.GetByteCount(builder.ToString()), moduleCount);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:494

• This selects the condensed form solely from bytes already in the file. If the newly rendered full section itself crosses the cap (for example, through an unbounded coverage table), the writer returns false and the caller drops the project entirely; it never retries BuildMinimalMarkdown. Please retry the minimal verdict on a size refusal so the documented full → compact → condensed degradation also handles an oversized current project.

 var budget = SummaryBudget.ForProject(currentLength);
bool condense = budget.Stage is SummaryStage.Condensed or SummaryStage.Unlisted;
string markdown = condense
? BuildMinimalMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode)
: BuildMarkdown(snapshot, assemblyName, _targetFrameworkMoniker.Value, exitCode, coverage, _sections, _includeFailureDetails, budget);

[Azat Mukhametshin (azat-msft)]

Both B-grade findings from the expert test review are now addressed.

• EffectiveStepSummaryLimit_IsSlightlyBelowTheDocumentedLimit was split in abd61b6 into that test plus DegradationThresholds_AreOrderedWithHeadroom, matching the suggested shape.
• BuildMarkdown_WithFailureDetails_RendersCollapsibleSection now asserts the whole fenced block in one go (7d46698) rather than the message and stack trace separately, so it would catch them rendering outside the code block — where an assertion diff's leading spaces and angle brackets get eaten as markdown and stack frames fold onto the line above.

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 196.5 AIC · ⌖ 0.999 AIC · ⊞ 16.9K · ◷

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:315

• This final length check still leaves a TOCTOU window: GetSummaryLength() closes its handle before ReplaceFile, so a framework or other writer that does not use this lock can append between these calls and have its new bytes silently overwritten by the staged snapshot. This is especially likely while test frameworks append their own summaries concurrently. Keep a destination handle that denies writes (while permitting delete/replace) through the swap, or avoid replacing the shared file.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:336

• The length recheck does not make this replacement safe against other summary producers. The summary handle has already been released, and foreign writers do not acquire this extension's lock file, so an append can land after this check and before ReplaceFile; the replacement then silently deletes that content. Avoid replacing the shared file to hoist the notice (for example, append the notice instead), or use a protocol that can atomically coordinate with every writer—the current check leaves a TOCTOU window.

 if (GetSummaryLength() is long lengthBeforeSwap && lengthBeforeSwap != lengthAtCapture)

[Copilot]

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:439

• The condensed aggregate fallback loses module identity: modules with the same assembly/TFM (for example x64 and arm64 runs, or retry attempts) render identical labels even though the full path disambiguates them with architecture/attempt/session. Preserve architecture and include attempt/session when the identity is duplicated so readers can map each verdict to its module.

 private static void AppendCondensedModuleLine(StringBuilder builder, CiRunSummaryModule module)
=> builder.Append(BuildCondensedLine(
module.AssemblyName,
module.TargetFramework,
module.TotalTests,
module.PassedTests,
module.FailedTests,
module.SkippedTests,
module.FailedTests > 0 || GitHubActionsExitCode.IndicatesFailure(module.ExitCode)));

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

• This modified C# file is currently UTF-8 without a BOM, but .editorconfig:66-67 requires utf-8-bom for all C# files. Re-save it with the BOM so the change follows the repository's encoding convention.

[github-actions]

🧵 Parallel-safety audit — PR #10633

Parallelization — assemblies audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Extensions.UnitTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)
Microsoft.Testing.Platform.Acceptance.IntegrationTests	MethodLevel	CPU count (Workers = 0)	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Both assemblies opt in via [assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in their Program.cs — every test method, including the ones this PR adds, is a live concurrent chunk. No .runsettings/testconfig.json override or DisableParallelization was found in either project.

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no Critical/High/Warning/Info findings.

The PR's changed test surface is:

• GitHubActionsSummaryReporterTests.cs (+1945/-99) — dozens of new [TestMethod]s exercising StepSummaryWriter/GitHubActionsSummaryReporter rendering, budget, and truncation logic.
• GitHubActionsReportTests.cs (+72) — two new acceptance tests plus a new "failex" mode branch in the shared test-asset harness.
• GitHubActionsCommandLineProviderTests.cs (+34) — new [DataRow]s and validation tests for the new GitHubActionsFailureDetails CLI option.
• CiRunSummaryAggregationTests.cs (+2/-1) — adds a Mock<ILoggerFactory> constructor argument to match a production signature change.

Reviewed every added/modified test body for the category A–D taxonomy:

• No process-global mutation — no Environment.SetEnvironmentVariable, Directory.SetCurrentDirectory, Console.Set*, culture mutation, or new mutable static field. The new private static helpers (CountOccurrences, AssertSingleNotice, NewWriter, BudgetOf) are pure, stateless functions.
• No shared filesystem path collisions — every test that touches disk uses Path.GetTempFileName() (unique per call) or a GUID-suffixed temp directory ("mtp-fragment-" + Guid.NewGuid().ToString("N")), and each wraps its I/O in try/finally with File.Delete/Directory.Delete. No hardcoded shared literal paths.
• No [ResourceLock] / [DoNotParallelize] changes — none of the four changed test files declare, add, or remove either attribute, and no sibling test in the same projects uses them either (checked via project-wide grep), so there is no near-miss/key-mismatch or coverage-gap surface to reconcile.
• No over-serialization — nothing here defers or serializes tests unnecessarily.

Nothing to flag for parallel-safety.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 88.5 AIC · ⌖ 1.5 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10633

This PR adds --report-gh-failure-details (collapsible per-failure sections with exception/location/stack trace) to the GitHub Actions report extension, plus substantial new coverage for step-summary budget degradation, foreign-writer race safety, and truncation-notice handling. The new/modified tests are uniformly strong: focused scenarios, one clear behavior per test, meaningful equality/contains/exception assertions, and unusually good "why" comments explaining the race or regression each test guards against (e.g. UTF‐8 byte budgeting for non‐ASCII failures, staged-file swap semantics, foreign-writer interference). No high-confidence actionable findings were identified, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new GitHubActionsReportTests. WhenTestFailsWithException_ SummaryExpandsTheFailureIntoACollapsibleSection	4/4 killed	End-to-end run asserts details, exception type and message all land in the collapsible section.	—
A (90–100)	new GitHubActionsReportTests. WhenFailureDetailsAreDisabled_ SummaryKeepsTheCompactFailureList	3/3 killed	Confirms the off-switch suppresses details while keeping the compact list.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsInvalid_ WhenFailureDetailsValueIsNotOnOrOffAsync	2/2 killed	Pins both the invalid verdict and the exact on/off error message for the new option.	—
A (90–100)	new GitHubActionsCommandLineProviderTests. ValidateOptionArgumentsAsync_ ReturnsValid_ WhenFailureDetailsValueIsOffAsync	1/1 killed	Simple, focused acceptance-path check for the new option's valid value.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetails_ RendersCollapsibleSection	5/5 killed	Asserts the whole fenced diagnostics block as one string, avoiding false positives from partial matches.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ WithFailureDetailsDisabled_ KeepsCompactFailureList	3/3 killed	Verifies both presence of the compact line and absence of detail markers.	—
A (90–100)	new GitHubActionsSummaryReporterTests. BuildMarkdown_ FailureWithoutDetails_ FallsBackToCompactLine	2/2 killed	Covers the no-diagnostics fallback branch cleanly.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendFailuresSection_ ChargesTheBudgetInBytes_ NotCharacters	2/2 killed	Uses real multi-byte UTF-8 text to distinguish byte- vs char-charged budgets; a genuinely valuable regression guard.	—
A (90–100)	new GitHubActionsSummaryReporterTests. DegradationThresholds_ ShedDiagnosticsBeforeWholeSections	3/3 killed	Checks both the constant ordering and the actual rendering behavior at the threshold.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ LeavesTheSummaryIntact_ WhenTheRewriteFailsPartway	3/3 killed	Injects a throwing stream via a mocked file system to prove the staged-write-then-swap design; strong isolation via a temp dir.	—
A (90–100)	new GitHubActionsSummaryReporterTests. AppendStepSummaryWithLeadingNoticeAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Uses a real interfering file system to prove the race window is closed; asserts every foreign append survived.	—
A (90–100)	new GitHubActionsSummaryReporterTests. UpsertStepSummaryWithRetryAsync_ DoesNotOverwriteAForeignAppendWhenAttemptsRunOut	3/3 killed	Same race-safety guard as the notice variant, applied to the upsert path.	—
A (90–100)	new GitHubActionsSummaryReporterTests. GetSummaryLengthExcludingSection_ ReportsTheRawLength_ WithoutReadingAnOversizedFile	2/2 killed	Custom throwing Stream proves the size-guard-before-read ordering; regression fails loudly.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 158.9 AIC · ⌖ 1.04 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/StepSummaryWriter.cs:765

• A re-upsert can leave a stale truncation warning at the top of the file. If this aggregation previously required a notice but a later rendering fits completely (for example after its inputs, option, or available budget changes), leadingNoticeFactory is null and this block never removes the old notice, even though the aggregation section is replaced. The summary then incorrectly claims that details or whole project sections are missing. Please associate notices with their aggregation/writer and remove or recompute this aggregation's notice during replacement while preserving notices required by other sections.

 string? leadingNotice = leadingNoticeFactory?.Invoke(otherProjectSections);
if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
&& GetLeadingNoticeStrength(existing) < GetLeadingNoticeStrength(leadingNotice!))
{
existing = leadingNotice + StripLeadingTruncationNotice(existing);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx:245

• This warning labels {1} as GitHub's enforced limit, but callers pass EffectiveStepSummaryLimit (1,048,574), while the documented/enforced limit represented by GitHubStepSummaryLimit is 1,048,576; the two-byte reduction is this reporter's safety margin. A refusal at the margin therefore tells users GitHub would reject content that may still be under GitHub's actual cap. Please describe {1} as the reporter's safety limit (and regenerate the XLF files), or report the documented limit separately.

 <data name="StepSummaryLimitExceededWarning" xml:space="preserve">
<value>The GitHub job summary file is {0} bytes and appending this report section would exceed the {1}-byte limit GitHub enforces per step. GitHub discards an oversized job summary in full rather than truncating it, so appending would have lost every section, including those other test projects already wrote. This section was skipped to keep the rest of the summary intact. This usually means many test projects, or other tools, are writing to the same job summary; see the workflow log or the test report for these results.</value>
<comment>{0} is the current size of the job summary file in bytes, {1} is the limit in bytes.</comment>

• Files reviewed: 36/37 changed files
• Comments generated: 0 new
• Review effort level: Balanced