Let explicit --minimum-expected-tests govern the zero-tests verdict (#7457)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)

[Amaury Levé (Evangelink)]

Summary

testfx side of #7457. Makes an explicit --minimum-expected-tests N govern the "zero tests ran" (exit code 8) verdict so that a dotnet test --test-modules / global-filter orchestrator can control the zero-tests behavior for the whole run rather than failing once per module.

Scope note: an earlier iteration also allowed the literal value --minimum-expected-tests 0 (from the now-collapsed #9724). That has been reverted — see "Why --minimum-expected-tests 0 was dropped" below — so this PR is limited to the verdict-governance change plus a defensive parse guard.

Problem

In TestApplicationResult.GetProcessExitCode() the ZeroTests (8) verdict was computed before and independently of--minimum-expected-tests, so --minimum-expected-tests N with zero tests returned 8, not 9. An orchestrator that normalizes a per-module 8 (empty module) could not distinguish "empty because the global filter matched nothing here" from "empty but the user asked for a stricter local minimum".

Change

An explicit --minimum-expected-tests N now governs the count-based verdict and supersedes ZeroTests(8): a run of fewer than N tests yields MinimumExpectedTestsPolicyViolation (exit code 9), even when zero tests ran. When the option is not set, the existing ZeroTests / --zero-tests-policy behavior is unchanged. (The value parse is guarded with a single-argument pattern match + int.TryParse.)

This is exactly what the dotnet test orchestrator (dotnet/sdk#55167) relies on to normalize a per-module 8 to success without swallowing a stricter per-module minimum forwarded via -- --minimum-expected-tests N, which now surfaces as exit code 9. Both dotnet test --minimum-expected-tests 10 -- --minimum-expected-tests 2 verdicts remain independently observable.

Why --minimum-expected-tests 0 was dropped

Allowing an explicit 0 created a UX divergence (originally raised in review): the exit code was 0 (success) but the terminal reporter still rendered a failed "Zero tests ran" verdict, because GetMinimumExpectedTests() returns 0 for both "unset" and "explicitly 0" and the shared verdict helpers couldn't tell them apart. Fixing that cleanly needs a change to TerminalTestReporterOptions.MinimumExpectedTests, which is vendored to dotnet/sdk (a cross-repo contract change).

Since allowing 0 is not required by the orchestrator design (that relies only on the "explicit N → exit 9" behavior above) and #7457 item #2 ("--min 0 == --ignore-exit-code 8") is already served by the existing --ignore-exit-code 8, we reverted the 0 support. Keeping 0 invalid preserves the unambiguous meaning of GetMinimumExpectedTests() == 0 ("unset") and avoids the divergence entirely. Tracking issue #9744 is closed as obsolete.

Tests

• TestApplicationResultTests: --minimum-expected-tests N + zero tests → exit code 9.
• ExecutionTests (acceptance): --filter-uid matching nothing + --minimum-expected-tests 3 → exit code 9.

Unit tests pass locally (TestApplicationResultTests + PlatformCommandLineProviderTests: 103/103). Acceptance tests require -pack and run in CI.

Related

• SDK orchestrator change (whole-run zero-tests verdict): Apply dotnet test zero-tests verdict at the whole-run level (microsoft/testfx#7457) dotnet/sdk#55167
• Collapsed & partially-reverted follow-up: Allow --minimum-expected-tests 0 (#7457) #9724

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR (testfx side of #7457) makes --minimum-expected-tests take precedence over the "zero tests ran" (exit code 8) verdict in TestApplicationResult.GetProcessExitCode(). Previously the ZeroTests verdict was computed independently and always won for empty runs, and --minimum-expected-tests 0 was rejected by validation. Now an explicitly-provided --minimum-expected-tests governs the count-based verdict: 0 accepts an empty run (like --ignore-exit-code 8), while N with zero tests reports a minimum-expected violation (exit code 9) instead of 8. This is the enabling precondition for a dotnet test --test-modules orchestrator (SDK-side follow-up) to control zero-tests behavior for a whole run rather than per module.

Changes:

• Allow --minimum-expected-tests 0 (validation now rejects only negative / non-integer values) and update the validation message wording.
• When --minimum-expected-tests is set, use it to decide the count-based verdict and skip the ZeroTests/--zero-tests-policy branch; otherwise the existing zero-tests behavior is unchanged.
• Regenerate localization (resx + 13 xlf) and add unit + acceptance coverage.

Show a summary per file

File	Description
src/Platform/Microsoft.Testing.Platform/Services/TestApplicationResult.cs	Core change: explicit --minimum-expected-tests governs the verdict via an if/else, superseding the ZeroTests branch.
src/Platform/Microsoft.Testing.Platform/CommandLine/PlatformCommandLineProvider.cs	Validation relaxed from value <= 0 to value < 0 so 0 is accepted.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx	Updated error message to "non-negative integer".
src/Platform/.../Resources/xlf/PlatformResources.*.xlf (13 files)	Regenerated via UpdateXlf; source updated, targets marked needs-review-translation.
test/UnitTests/.../Services/TestApplicationResultTests.cs	Added tests for min 0 + zero tests → Success and min N + zero tests → violation.
test/UnitTests/.../CommandLine/PlatformCommandLineProviderTests.cs	0 now valid; negatives/non-integers invalid; renamed/added test cases.
test/IntegrationTests/.../ExecutionTests.cs	Acceptance coverage for zero-test scenarios via --filter-uid 2; updated validation-message expectation.
docs/Changelog-Platform.md	Changelog entry for the behavior change.

I verified the branching logic, argument-parsing safety (arity + Length==1 validation guards argumentList[0]), the test asset (uids 0/1, so --filter-uid 2 yields zero tests), and that no stale references to the old validation message remain. I also confirmed the terminal-reporter verdict divergence for --minimum-expected-tests 0 matches the intended --ignore-exit-code 8 equivalence. No blocking issues were found, but the change alters a shipped CLI option's exit-code semantics (a public behavioral contract) and coordinates with a cross-repo dotnet/sdk change, which warrants human sign-off.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 3
• Review effort level: Low

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Medium

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

#	Dimension	Verdict
1	Algorithmic Correctness	🟡 1 MODERATE
21	Scope & PR Discipline	⚪ 1 NIT

✅ 20/22 dimensions clean.

• Algorithmic Correctness — Terminal reporter verdict ("Zero tests ran" / IsRunFailed=true) diverges from exit code (Success) when --minimum-expected-tests 0 is used with an empty run. GetMinimumExpectedTests() returns 0 for both "not set" and "explicitly set to 0", so the terminal reporter can't distinguish the two and always shows the failure-like verdict. Low-priority follow-up.
• Scope — Changelog entry for PR #9724 is included in this PR (#9709), coupling merge ordering.

Overall: The core logic change is correct and well-tested. The if/else refactoring cleanly separates the --minimum-expected-tests path from the --zero-tests-policy path, and the boundary condition change (<= 0 → < 0) correctly enables the zero-minimum scenario. Tests cover the key edge cases (zero with no tests → success, N>0 with no tests → violation). The --zero-tests-policy is silently superseded when --minimum-expected-tests is explicitly set, which is the documented intent.

No blocking issues.

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Medium

[github-actions]

🧪 Test quality grade — PR #9709

Grade	Test	Notes
B (80–89)	new ExecutionTests. Exec_ WhenMinimumExpectedTestsIsSpecifiedAndNoTestsRun_ ResultIsMinimumExpectedTestsPolicyViolation	Single exit-code assertion correctly pins the behavioral contract; adding an AssertOutputContains check would give extra end-to-end confidence that the right diagnostic message is emitted.
B (80–89)	new TestApplicationResultTests. GetProcessExitCodeAsync_ If_ MinimumExpectedTests_ Set_ And_ No_ Tests_ Ran_ Returns_ MinimumExpectedTestsPolicyViolation	Correct equality assertion on exit code 9; method name contains "Async" but return type is void — low-priority naming inconsistency with the async siblings in the same class, no behavior impact.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 62.3 AIC · ⌖ 7.62 AIC · ⊞ 9.5K · [◷]( · ◷)