Skip to content

Update --show-stdout / --show-stderr help text to reflect LLM-aware default - #8797

Merged
Amaury Levé (Evangelink) merged 1 commit into
mainfrom
dev/amauryleve/fix-8772
Jun 3, 2026
Merged

Update --show-stdout / --show-stderr help text to reflect LLM-aware default#8797
Amaury Levé (Evangelink) merged 1 commit into
mainfrom
dev/amauryleve/fix-8772

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Follow-up from #8771 / fixes#8772.

When an LLM/AI agent environment is detected, the defaults for --show-stdout and --show-stderr flip from All to Failed (see TerminalOutputDevice.GetShowOutputMode). The option descriptions in PlatformResources.resx still hard-coded Default is 'All'. though, so agents invoking --help saw misleading guidance about the very default that was changed for their benefit.

Changes

  • Updated TerminalShowStdoutOptionDescription and TerminalShowStderrOptionDescription in PlatformResources.resx to:

    Valid values are 'All', 'Failed', 'None'. Default is 'All' (or 'Failed' when an LLM/AI agent environment is detected).

  • Regenerated the 13 XLF localization files via dotnet msbuild .../Microsoft.Testing.Platform.csproj /t:UpdateXlf.

  • Updated the wildcard / regex expectations in:

    • test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.cs
    • test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
    • test/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.cs

Validation

Locally green on net8.0 / net10.0 / net462:

  • HelpInfoTests (21 tests) — passed
  • HelpInfoAllExtensionsTests (9 tests) — passed
  • MSTest HelpInfoTests (6 tests) — passed

…efault
Mention that the default flips from 'All' to 'Failed' when an LLM/AI agent environment is detected (added in #8771).
Fixes#8772
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings June 3, 2026 11:36

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Microsoft.Testing.Platform’s CLI help text for --show-stdout / --show-stderr so it accurately describes the LLM/AI-aware default behavior introduced in #8771, and keeps the help/info acceptance tests and localized resources in sync.

Changes:

  • Updated the two terminal option description resource strings to document the conditional default (All, or Failed in detected LLM/AI agent environments).
  • Regenerated all 13 localized XLF files for the updated resources.
  • Updated help/info acceptance test expectations (including regex escaping where needed for the --info output).
Show a summary per file
FileDescription
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/HelpInfoTests.csUpdates expected --help output text for --show-stdout / --show-stderr.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoTests.csUpdates expected --help and --info outputs (regex-escaped) for the revised option descriptions.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.csUpdates expected --help and --info outputs (with all extensions) for the revised option descriptions.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resxUpdates the canonical English resource strings for the two option descriptions.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlfRegenerated localization file reflecting updated source strings (marked for review).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlfRegenerated localization file reflecting updated source strings (marked for review).

Copilot's findings

  • Files reviewed: 17/17 changed files
  • Comments generated: 0

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expert Review — PR #8797: Update --show-stdout / --show-stderr help text to reflect LLM-aware default

21/21 dimensions clean — no findings.


Dimension Assessment

#DimensionVerdict
1Algorithmic Correctness✅ N/A — no logic changed
2Threading & Concurrency✅ N/A — no shared state touched
3Security & IPC Contract Safety✅ N/A
4Public API & Binary Compatibility✅ N/A — no API surface changed
5Performance & Allocations✅ N/A
6Cross-TFM Compatibility✅ N/A
7Resource & IDisposable Management✅ N/A
8Defensive Coding at Boundaries✅ N/A
9Localization & Resources✅ LGTM — .resx updated; all 13 XLF files regenerated via dotnet msbuild /t:UpdateXlf, not manually edited; <source> blocks updated and stale <target> state correctly demoted to needs-review-translation in each locale (cs, de, es, fr, it, ja, ko, pl, pt-BR, ru, tr, zh-Hans, zh-Hant).
10Test Isolation✅ N/A
11Assertion Quality✅ LGTM — test literals match the new string exactly; the one location that uses regex matching (HelpInfoTests.cs ~line 287) correctly escapes the parentheses as \( and \).
12Flakiness Patterns✅ N/A
13Test Completeness & Coverage✅ LGTM — all three help-info test files updated: HelpInfoTests.cs (MTP), HelpInfoAllExtensionsTests.cs (MTP), HelpInfoTests.cs (MSTest). Both the short-form help block and the verbose diagnostic block (--list-tests style) are covered.
14Data-Driven Test Coverage✅ N/A
15Code Structure & Simplification✅ N/A
16Naming & Conventions✅ N/A
17Documentation Accuracy✅ LGTM — new text accurately reflects TerminalOutputDevice.GetShowOutputMode behavior: default is All, flips to Failed when LLM/AI agent environment detected. Wording is concise and correct.
18Analyzer & Code Fix Quality✅ N/A
19IPC Wire Compatibility✅ N/A
20Build Infrastructure & Dependencies✅ N/A
21Scope & PR Discipline✅ LGTM — single concern: string update + XLF regeneration + test expectations. No mixed refactoring. References #8771 and fixes #8772.

Notes

  • The needs-review-translation state in the XLF files is the correct XLIFF lifecycle state for "source changed, translation pending re-review." No action needed beyond what the localization pipeline will handle automatically.
  • The parentheses-escaping difference between test files is intentional: files that use Contains or verbatim string matching use literal (); files that use regex matching escape to \(\). Both are correct for their respective assertion styles.

Generated by Expert Code Review (on open) for issue #8797 · sonnet46 1.8M

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) June 3, 2026 13:44
@Evangelink
Amaury Levé (Evangelink) merged commit b8a83c6 into mainJun 3, 2026
36 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/fix-8772 branch June 3, 2026 13:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update --show-stdout / --show-stderr help text to reflect LLM-aware default

3 participants

@Evangelink@YuliiaKovalova