Add MSTest.Windows.UIAutomation for desktop UI tests

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[Amaury Levé (Evangelink)]

Summary

Revives and redesigns the unmerged work from #7810 as a narrowly scoped preview integration.

• add the MSTest.Windows.UIAutomation package for MSTest-managed application lifecycle and UIA2 window access
• expose it through MSTest.Sdk with <EnableWindowsUIAutomation>true</EnableWindowsUIAutomation> for MTP and VSTest
• provide abstract ApplicationTest and WindowTest bases with overridable process launch, window discovery, and shutdown hooks
• support non-Windows cross-targeted builds while preserving explicit EnableWindowsTargeting values
• add a Character Map sample plus acceptance coverage for both runners, SDK validation, process cleanup, cancellation, custom discovery, launcher-child handoff, and custom shutdown

Scope

This preview targets unpackaged, full-trust Win32, WinForms, and WPF applications running in an interactive Windows session. It does not provide packaged MSIX/UWP/WinUI activation, elevated-process automation, a headless desktop, locators, automatic waits, screenshots, or a multi-window object model.

Validation

• build.cmd -pack
• WindowsUIAutomationSdkTests: 26 passed
• non-Windows solution-filter pack with OS=Unix

Supersedes #7810.

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Demo sample test; checks non-empty title but not a specific expected value.	If a known app title is deterministic, assert the exact expected string.
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Demo sample illustrating WindowTest usage; asserts control type only.	Could also assert MainWindow.Current.IsEnabled/IsOffscreen for a stronger visibility check.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Follows established acceptance-test pattern (mirrors PlaywrightSdkTests); asserts exit code and exact pass/fail/skip counts, correctly gated by OSCondition for Windows-only desktop UI automation.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies both VSTest engine banner and exact passed/failed/skipped summary line for the VSTest path.	—

Note: Mutation analysis reported N/A for all four tests — the acceptance tests spawn out-of-process test hosts against a Windows-only physical UI (Character Map), and the sample tests are documentation demos, so the production entry points (native app automation) could not be resolved/exercised for pseudo-mutation in this review. All four tests are gated correctly ([OSCondition(OperatingSystems.Windows, ...)] on the acceptance tests) and follow the repo's existing SDK-acceptance-test conventions (TestAssetFixture, DynamicData over TargetFrameworks, AssertOutputContainsSummary/AssertOutputContains) closely, matching sibling files like PlaywrightSdkTests.cs. No high-confidence actionable findings warranted an inline suggestion.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 47.6 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary — MSTest.Windows.AppTesting

Clean PR that adds a new MSTest.Windows.AppTesting package with application lifecycle and Windows UI Automation base classes, wired through MSTest.Sdk. Two inline comments posted; neither is blocking.

#	Dimension	Verdict
1	Algorithmic Correctness	✅ OK — startup loop handles edge cases (process exit, timeout) correctly
2	Threading & Concurrency	⚠️ Minor — AutomationTest uses [TestClass] (MTA) while subclasses use [STATestClass]; see inline comment
3	Security & IPC	✅ N/A
4	Public API & Binary Compat	✅ OK — all public API in PublicAPI.Unshipped.txt, no init accessors, types unsealed by design (base-class hierarchy), XML docs present
5	Performance & Allocations	✅ OK — not a hot path
6	Cross-TFM Compatibility	✅ OK — targets net8.0-windows / net9.0-windows only, validated in .targets
7	Resource & IDisposable	✅ OK — Process disposed in finally; CloseMainWindow → Kill cascade is correct
8	Defensive Coding	✅ OK — boundary checks for process exit, null handle, and timeout present
9	Naming & Code Style	✅ OK
10	Logging & Observability	✅ N/A
11	Error Messages & UX	✅ OK — clear exception messages with variable names
12	Tests & Test Quality	✅ OK — acceptance tests cover MTP and VSTest, OS-gated
13	Documentation	✅ OK — PACKAGE.md present, XML docs thorough
14	Localization	✅ N/A — no user-facing resx strings
15	Backward Compatibility	✅ N/A — new package
16	Build & Pack Integration	✅ OK — verify-nupkgs.ps1 updated, MSBuild integration mirrors Aspire/Playwright patterns
17	Serialization	✅ N/A
18	Tracing & Telemetry	✅ N/A
19	Nullability	✅ OK — null! used for lifecycle-initialized properties, consistent with rest of codebase
20	TODOs & Leftovers	✅ None found
21	Scope Discipline	✅ Single concern
22	Script & Tooling Hygiene	✅ OK — PowerShell hash table update is correct

Overall: Looks good. The two inline comments are non-blocking suggestions for future hardening.

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 2 · 2

New issues introduced by this change (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because…
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by…
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a…
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that…

What changed in this PR

Adds Windows desktop UI automation support to MSTest through a new package, SDK integration, sample, and acceptance coverage.

Changes:

• Adds application lifecycle and UI Automation base classes.
• Integrates the package into MSTest.Sdk for MTP and VSTest.
• Adds Character Map samples, acceptance tests, and package verification.

File	Description
TestFx.slnx	Includes the new package project.
WindowsAppTestingSdkTests.cs	Tests MTP and VSTest integration.
WindowTest.cs	Exposes the main UI Automation window.
TestFramework.Windows.AppTesting.csproj	Defines the Windows package.
PublicAPI.Unshipped.txt	Tracks new public APIs.
PublicAPI.Shipped.txt	Initializes the shipped baseline.
PACKAGE.md	Documents package usage.
GlobalUsings.cs	Imports MSTest APIs globally.
AutomationTest.cs	Adds the hierarchy root.
ApplicationTest.cs	Manages application lifecycle.
VSTest.targets	Enables the feature under VSTest.
Sdk.props.template	Defines feature and version properties.
NativeAOT.targets	Rejects unsupported NativeAOT usage.
ClassicEngine.targets	Enables the feature under MTP.
WindowsAppTesting.targets	Adds validation, package reference, and using.
MSTest.Sdk.csproj	Packs and versions the feature target.
ProjectUsingWindowsAppTesting.csproj	Adds the SDK sample configuration.
CharacterMapTests.cs	Demonstrates desktop UI tests.
verify-nupkgs.ps1	Verifies package contents.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🧪 Expert test review — PR #10862

Reviewed 4 new test methods introduced by this PR across the new TestFramework.Windows.AppTesting sample and its acceptance test. All four are focused, single-behavior tests with meaningful equality/non-empty assertions on the exercised contract (ControlType, window Name, process exit code, and MSTest/VSTest summary counts), correctly awaited async work, and no anti-pattern findings. No high-confidence actionable issues were found, so no inline suggestions were posted.

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	0/0 (no meaningful points)	Single meaningful equality assertion on ControlType; behavior is thin OS-automation plumbing with no internal branch logic to mutate.	—
A (90–100)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	0/0 (no meaningful points)	Clear non-empty-title assertion with a descriptive failure message.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	End-to-end acceptance test asserting exit code and exact pass/fail/skip summary against a generated real project.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Verifies exit code and VSTest-specific summary output; async call correctly awaited.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 66.8 AIC · ⌖ 1.12 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 3 · 3

New issues introduced by this change (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain…
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.…

Pre-existing issues (4)

Severity	Finding
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View comment
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View comment

Suppressed comments (3)

samples/public/DemoMSTestSdk/ProjectUsingWindowsAppTesting/ProjectUsingWindowsAppTesting.csproj:1

• This public sample is not included in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. eng/build-samples.ps1:60-64 builds discovered solutions rather than standalone projects, so the new sample is currently omitted from sample CI and can silently break. Add the project and its Debug/Release configurations to that solution.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Features/WindowsAppTesting.targets:11

• The added acceptance asset always targets *-windows, and no other test exercises this validation message, so removing or breaking this target would leave all current tests green. Add a non-Windows-target build case that enables the feature and asserts this diagnostic.

 <Target Name="_ValidateWindowsAppTestingPlatform" BeforeTargets="Restore;Build"
Condition=" '$(TargetFramework)' != '' and '$(_IsWindowsTfm)' != 'true' ">
<Error Text="MSTest.Windows.AppTesting requires a Windows target framework (e.g. net8.0-windows). Current TargetFramework: '$(TargetFramework)'." />

src/TestFramework/TestFramework.Windows.AppTesting/ApplicationTest.cs:124

• The Character Map acceptance path closes normally and does not assert that cleanup terminated the process, so this forced-kill branch is untested; a regression here can leave desktop processes running after the suite. Add a test application that ignores CloseMainWindow and verify that cleanup terminates its process tree.

 _ = appProcess.CloseMainWindow();
if (!appProcess.WaitForExit(5000))
{
appProcess.Kill(entireProcessTree: true);
_ = appProcess.WaitForExit(5000);

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers…

Pre-existing issues (2)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View comment
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View comment

Issues resolved since last review (4)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.AppTesting/​ApplicationTest.cs — The acceptance tests exercise only a responsive Character Map process. Add coverage for an app that… View resolved comment
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — No test exercises this validation path: every generated acceptance TFM appends -windows. Add a… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — The non-SDK example cannot restore as written: MSTestWindowsAppTestingVersion is only defined by… View resolved comment
	samples/​public/​DemoMSTestSdk/​ProjectUsingWindowsAppTesting/​ProjectUsingWindowsAppTesting.csproj — This project is not listed in samples/public/DemoMSTestSdk/DemoMSTestSdk.sln. Because… View resolved comment

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.AppTesting/PACKAGE.md:12

• This standalone getting-started snippet does not pin MSTest.Sdk, and the README does not provide a global.json pin, so users copying it from the NuGet package page cannot resolve the project SDK. Include an SDK version here, consistent with src/Package/MSTest.Sdk/PACKAGE.md:9-16.

<Project Sdk="MSTest.Sdk">

src/Package/MSTest.Sdk/Sdk/Sdk.props.template:45

• The repository's complete MSTest.Sdk property reference in docs/glossary.md:312-343 still lists only Aspire and Playwright and omits both new controls. Add an EnableWindowsAppTesting compatibility row (including its NativeAOT error and VSTest support) and document MSTestWindowsAppTestingVersion, otherwise the advertised complete reference is stale as soon as this feature ships.

 <EnableWindowsAppTesting Condition=" '$(EnableWindowsAppTesting)' == '' ">false</EnableWindowsAppTesting>
<EnablePlaywright Condition=" '$(EnablePlaywright)' == '' ">false</EnablePlaywright>
<AspireHostingTestingVersion Condition=" '$(AspireHostingTestingVersion)' == '' ">${AspireHostingTestingVersion}</AspireHostingTestingVersion>
<MicrosoftNETTestSdkVersion Condition=" '$(MicrosoftNETTestSdkVersion)' == '' ">${MicrosoftNETTestSdkVersion}</MicrosoftNETTestSdkVersion>
<MSTestWindowsAppTestingVersion Condition=" '$(MSTestWindowsAppTestingVersion)' == '' ">${MSTestWindowsAppTestingVersion}</MSTestWindowsAppTestingVersion>

[github-actions]

🧪 Expert test review — PR #10862

The pre-step's extract data (test-file/line-range TSV) was unavailable in this run, so this review was derived directly from the PR's file diffs. Two new test files were added: a sample demo (CharacterMapTests.cs) and an acceptance-test suite (WindowsAppTestingSdkTests.cs) for the new EnableWindowsAppTesting MSTest.Sdk feature.

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ IsVisible	N/A	Sample/demo asset; single ControlType assertion, adequate for illustrating the API.	—
B (80–89)	new CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	N/A	Sample/demo asset; only checks the title is non-empty, not a specific expected value.	—
B (80–89)	new WindowsAppTestingSdkTests. ApplicationTearDown_ WhenApplicationNeverCreatesWindow_ TerminatesProcess	N/A	Solid scenario, but the hardcoded 1s `WaitForExit` and silent `catch (ArgumentException)` could mask a slow-terminate flake under CI load.	Add an explicit fail message covering the not-yet-exited case and consider a longer/backoff wait.
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingMSTestRunner_ RunsDesktopTests	N/A	Exercises the real MSTest runner path end-to-end and asserts exit code plus exact pass/fail/skip summary.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenUsingVSTest_ RunsDesktopTests	N/A	Covers the VSTest bridge path with deterministic exit-code and output assertions.	—
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	N/A	Verifies the build-time guard with the exact expected error message.	—
A (90–100)	new WindowsAppTestingSdkTests. ApplicationSetup_ WhenApplicationExitsBeforeCreatingWindow_ ReportsClearFailure	N/A	Asserts the specific failure exit code and diagnostic message for the early-exit scenario.	—

Note on mutation column: production code for the new TestFramework.Windows.AppTesting classes (WindowTest, ApplicationTest, AutomationTest) is Windows-only and process/UI-Automation driven, so meaningful pseudo-mutation execution could not be resolved in this sandbox; marked N/A throughout rather than guessed.

No inline suggestions were posted — findings above are minor reliability/coverage notes rather than concrete, safely-anchorable mechanical fixes.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 54.2 AIC · ⌖ 1 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1

Pre-existing issues (1)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Features/​WindowsAppTesting.targets — This check is case-sensitive, although the SDK accepts target framework monikers… View comment

Issues resolved since last review (2)

Severity	Finding
	src/​Package/​MSTest.Sdk/​Sdk/​Sdk.props.template — These new user-facing SDK properties are missing from the SDK documentation tables.… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsAppTestingSdkTests.cs — This VSTest leg omits the roll-forward setting used by the MTP leg above. The repo-local toolchain… View resolved comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsAppTestingSdkTests.cs:150

• The generated asset imports Microsoft.MSTest.Windows.AppTesting explicitly, so these acceptance tests do not exercise the implicit Using added by WindowsAppTesting.targets. Since the public sample relies on that implicit import and is intentionally excluded from sample CI, removing this using is needed to make a regression in the SDK feature fail automated coverage.

using Microsoft.MSTest.Windows.AppTesting;

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
A (90–100)	new WindowsAppTestingSdkTests. EnableWindowsAppTesting_ WhenTargetFrameworkUsesUppercase_ IsAccepted	1/1 killed	Directly exercises the case-insensitive TargetPlatformIdentifier fix; reverting it fails the build.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 48.8 AIC · ⌖ 0.905 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: 1 · 2

New issues introduced by this change (3)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication…
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing…

Issues resolved since last review (1)

Severity	Finding
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​PACKAGE.md — The getting-started example does not compile when copied into the shown SDK project because… View resolved comment

[github-actions]

🧪 Expert test review — PR #10862

Grade	Test	Mutation	Notes	How to improve
C (70–79)	new CharacterMapTests. CharacterMap_ MainWindow_ HasExpectedTitle	1/2 killed	Only checks title is non-empty; a wrong/garbled window title still passes.	Assert the title contains the expected app name, e.g. "Character Map".
C (70–79)	new sample CharacterMapTests. CharacterMap_ MainWindow_ HasTitle	1/2 killed	Only checks title is non-empty; a wrong window title still passes (same gap as the acceptance-test twin).	Assert the title contains "Character Map" or a similar expected substring.
B (80–89)	CharacterMapTests. CharacterMap_ MainWindow_ IsVisible (both copies)	2/2 killed	Clear, focused assertion on ControlType protects the primary discovery contract.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/4 killed	Validates exit code and PID file end-to-end; relies on shared fixture/env var protocol without asserting it explicitly.	Assert stdout mentions the passed test names to catch silent early exits.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	3/4 killed	Mirrors the MSTest-runner test for VSTest path; same shallow output verification.	Assert the VSTest trx/output confirms the expected test count ran.
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	3/3 killed	Asserts non-zero exit and a specific, clear error message for the misconfiguration case.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Covers a real case-sensitivity boundary in TFM/property matching with a clear success assertion.	—
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/3 killed	Verifies the build succeeds but doesn't check the Windows TFM's automation feature is actually enabled in output.	Assert the build log/output confirms UI automation support for the windows TFM specifically.
B (80–89)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	2/2 killed	Protects the "explicit false is preserved" contract with a targeted assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts both failure exit code and a diagnostic message naming the early-exit cause.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Uses AssertProcessExitedAsync to actively confirm process termination, not just a passing exit code.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Confirms the custom FindWindow override path actually executed, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	2/2 killed	Confirms the custom shutdown override ran, protecting the extensibility contract.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Verifies process termination via AssertProcessExitedAsync after cancellation, a strong isolation-relevant assertion.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Confirms guaranteed disposal even when derived cleanup throws — the core regression this PR fixes.	—

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · [◷]( · ◷)

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 189.7 AIC · ⌖ 2.72 AIC · ⊞ 16.9K · ◷

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Issues resolved since last review (3)

Severity	Finding
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — This cleanup-failure case has the same PID race: the empty test proceeds directly to failing… View resolved comment
	test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​WindowsUIAutomationSdkTests.cs — The child writes its PID asynchronously, but this empty test can finish and enter StopApplication… View resolved comment
	src/​TestFramework/​TestFramework.Windows.UIAutomation/​WindowTest.cs — FindWindow is documented as supporting launcher processes, but this unconditional check aborts… View resolved comment

Suppressed comments (2)

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:479

• The cancellation timer can fire and cleanup can kill PowerShell before its asynchronous command creates the PID file. The outer test unconditionally reads that file, so slow scheduling turns this cleanup check into a FileNotFoundException. Write applicationProcess.Id synchronously before scheduling cancellation instead of relying on the child command.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:417

• This PID is written asynchronously by the child, but cleanup starts after a fixed three-second discovery timeout and can terminate PowerShell before it ever executes Set-Content on a heavily loaded worker. AssertProcessExitedAsync then fails with a missing file instead of checking cleanup. Record applicationProcess.Id synchronously from FindWindow before polling, as the other cleanup cases do.

 "-NoProfile -NonInteractive -Command \"$PID | Set-Content -LiteralPath $env:MSTEST_UI_AUTOMATION_PID_FILE; Start-Sleep -Seconds 30\"");

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:50

• ApplicationShutdownTimeout is first validated during disposal, after the process has been launched. If an override returns a negative value (or one above Int32.MaxValue milliseconds), cleanup throws before CloseMainWindow/Kill; Dispose then clears and disposes the Process object while leaving the application running. Validate the timeout before Process.Start so an invalid configuration cannot leak a process.

 ProcessStartInfo startInfo = CreateProcessStartInfo();

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:88

• AutomationElement.FromHandle also throws ArgumentException when the HWND is no longer valid. A window can be destroyed or replaced between reading MainWindowHandle and this call (especially during splash-screen transitions), so that transient race currently fails initialization instead of continuing discovery. Treat ArgumentException like ElementNotAvailableException and retry.

 : AutomationElement.FromHandle(mainWindowHandle);

src/TestFramework/TestFramework.Windows.UIAutomation/ApplicationTest.cs:96

• CloseMainWindow() returns false when no close message was sent (for example, a windowless process or a disabled main window), but the code still waits the full shutdown timeout before killing. This adds the default five-second delay to cleanup paths where graceful shutdown was never requested. Use the return value to proceed directly to Kill when it is false.

 _ = applicationProcess.CloseMainWindow();
if (!applicationProcess.WaitForExit(shutdownTimeoutMilliseconds))

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/TestFramework/TestFramework.Windows.UIAutomation/WindowTest.cs:112

• An override returning Timeout.InfiniteTimeSpan is treated as already expired here, and any other negative value produces a misleading application timeout instead of rejecting invalid configuration. Since this is a public customization point, validate the value once and either support infinite polling explicitly or reject it with ArgumentOutOfRangeException, consistent with ApplicationShutdownTimeout.

 MainWindow = window;
return;

src/TestFramework/TestFramework.Windows.UIAutomation/TestFramework.Windows.UIAutomation.csproj:5

• This assignment overwrites an EnableWindowsTargeting value supplied earlier (for example from Directory.Build.props) on every non-Windows evaluation. That breaks the PR's value-preservation contract and differs from the conditional fallback in the SDK target and sample; only default it when the property is unset.

 <EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None

[github-actions]

🧵 Parallel-safety audit — PR #10862

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
MSTest.Acceptance.IntegrationTests	MethodLevel (assembly-level [assembly: Parallelize(Scope = MethodLevel, Workers = 0)] in Program.cs, pre-existing, unchanged by this PR)	CPU count	coverable once MSTEST0074–0077 ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 1 — by severity: Critical 0 · High 0 · Warning 0 · Info 1.

This PR's only changed test file is a brand-new class,
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs
(714 added lines), decorated [TestClass] [DoNotParallelize]. All other files this PR touches
(Program.cs, AssemblyCleanupTests.cs, etc. carrying [assembly: Parallelize]) are pre-existing
and unmodified here, so scope resolution is context, not a live config change from this PR.

Top actions:

1. Nothing blocking — the class-level [DoNotParallelize] is an appropriate, deliberate choice for tests that launch real Windows desktop processes and drive UI automation; no changes recommended.

---

Info

• [D · High confidence]test/IntegrationTests/MSTest.Acceptance.IntegrationTests/WindowsUIAutomationSdkTests.cs:11 — new class-level [DoNotParallelize] defers all ~17 test methods in this class to run sequentially after the assembly's MethodLevel parallel set drains — a genuine category-D throughput cost, since this assembly otherwise runs with Workers = 0 (CPU count). Not recommended to narrow to [ResourceLock]: the tests spawn real child processes that open OS-level desktop windows and assert on window discovery/PID termination (CharacterMapTests, EarlyExitTests, StartupTimeoutTests, ...) — the actual point of contention is the OS window manager/desktop session, not a single named in-process resource a lock could scope. Sequential execution here is the safer, deliberate tradeoff; no fix needed.
• [B · High confidence, no finding] All per-test artifact paths (pidFile, shutdownMarker, childPidFile, launchMarker) are built with Path.Combine(AssetFixture.ProjectPath, $"{Guid.NewGuid():N}.<ext>") — unique per invocation, so no cross-test collision even though the base AssetFixture.ProjectPath directory is shared across all methods in the class. Confirmed safe; not reported as a finding, noted for completeness since the class-level [DoNotParallelize] means it wouldn't matter today regardless.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 72.7 AIC · ⌖ 3.97 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10862

Reviewed the 15 new acceptance-test methods in WindowsUIAutomationSdkTests.cs covering the new EnableWindowsUIAutomation MSBuild feature and the ApplicationTest/WindowTest base classes (src/TestFramework/TestFramework.Windows.UIAutomation). Test classes embedded inside the fixture's SourceCode string literal (e.g. CharacterMapTests, StartupTimeoutTests, CustomWindowDiscoveryTests, ...) are test assets generated on disk, not compiled methods in this file, so they were not graded separately — their behavior is exercised indirectly through the outer acceptance-test assertions (exit code, summary, process-exit checks).

Grade	Test	Mutation	Notes	How to improve
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingMSTestRunner_ RunsDesktopTests	3/3 killed	Verifies exit code, exact pass/fail summary counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenUsingVSTest_ RunsDesktopTests	4/4 killed	Cross-checks VSTest output text, exact pass counts, and process termination.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkIsNotWindows_ FailsWithClearError	2/2 killed	Asserts both build failure and the exact diagnostic message text.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenTargetFrameworkUsesUppercase_ IsAccepted	2/2 killed	Confirms success exit code and absence of the case-sensitivity error message.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenCrossTargetingFromNonWindows_ BuildsSuccessfully	2/2 killed	Checks resolved MSBuild property value separately from build success.	—
A (90–100)	WindowsUIAutomationSdkTests. EnableWindowsUIAutomation_ WhenWindowsTargetingIsExplicitlyDisabled_ PreservesValue	1/1 killed	Narrow, focused check on the opt-out property value.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenApplicationExitsBeforeDiscovery_ ReportsClearFailure	2/2 killed	Asserts the exact failure exit-code path and diagnostic wording for an early-exit process.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTearDown_ WhenApplicationNeverExposesWindow_ TerminatesProcess	3/3 killed	Verifies the timeout message, failure exit code, and that the stuck process is still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenWindowDiscoveryIsCustomized_ UsesOverride	2/2 killed	Success summary depends on the fixture's own invocation-count assertion, giving indirect but real coverage of the override hook.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenShutdownIsCustomized_ UsesOverride	3/3 killed	Confirms the custom shutdown hook ran (marker file) and the process was still terminated.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenTestIsCanceled_ StopsDiscoveryAndTerminatesProcess	3/3 killed	Covers cooperative cancellation during discovery plus process cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationTest_ WhenDerivedCleanupFails_ StillTerminatesProcess	3/3 killed	Targets the specific guarantee that Dispose still runs after a derived [TestCleanup] throws.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowTest_ WhenLauncherExitsBeforeChildWindow_ DiscoversAndStopsChild	3/3 killed	Exercises the launcher/child-process discovery override end to end, including child cleanup.	—
A (90–100)	WindowsUIAutomationSdkTests. ApplicationSetup_ WhenShutdownTimeoutIsInvalid_ DoesNotLaunchProcess	3/3 killed	Verifies the process is never launched, catching any reordering of the timeout-validation step.	—
A (90–100)	WindowsUIAutomationSdkTests. WindowSetup_ WhenDiscoveryTimeoutIsInvalid_ FailsAndTerminatesProcess	3/3 killed	Confirms the exact validation message and that the already-launched process is still cleaned up.	—

No high-confidence actionable inline findings were identified; assertions consistently match the exact wording and control flow of the production code in ApplicationTest.cs / WindowTest.cs, and process/resource cleanup is verified for every scenario that launches a process. The class is correctly marked [DoNotParallelize] given its shared use of environment-variable-based marker files across process-spawning tests.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 105.5 AIC · ⌖ 2.67 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Copilot review overview

Review tier: Balanced
Findings: None