Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Add client-declared `IsStateful` capability to MTP by Evangelink · Pull Request #9789 · microsoft/testfx · GitHub
Skip to content

Add client-declared IsStateful capability to MTP - #9789

Merged
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw
Jul 9, 2026
Merged

Add client-declared IsStateful capability to MTP#9789
Amaury Levé (Evangelink) merged 4 commits into
mainfrom
dev/amauryleve/supreme-chainsaw

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Relates to #6494.

Introduces a client-declared statefulness capability so a test framework can distinguish:

  • a stateful client — one that persists an addressable set of test nodes for the whole session and keeps each node in its last-known state until explicitly updated (e.g. an IDE test explorer); from
  • a stateless client — one that consumes updates as a stream and does not retain node state after the run (e.g. dotnet test).

This is the "stateless vs stateful" axis discussed in #6494: rather than having a framework infer the environment (parsing --server, sniffing client name/DesignMode, etc.), the client declares how it will consume results, and the framework branches on that. It cleanly answers scenarios like "is pre-enumerating theory rows worth it?" without coupling to transport or environment.

Design

  • New experimental IClientCapabilities { bool IsStateful } surfaced via a new IClientInfo.Capabilities getter, so the C# shape mirrors the wire protocol's clientInfo / capabilities split 1:1.
  • The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful, right next to the existing debuggerProvider.
  • Backward compatible / opt-in: absent or false ⇒ stateless (current behavior). Console host defaults to stateless; server host builds the value from the client's declared capabilities.
  • A framework reads it during a run via serviceProvider.GetClientInfo().Capabilities.IsStateful.

Notes / follow-ups

  • The server only reads this; it becomes fully useful once real clients (VS Test Explorer, dotnet test) start sending it — a coordination point with the VS/SDK teams, exactly like debuggerProvider.
  • Public API additions are all [TPEXP] (experimental), so no shipped-API break.
  • Deliberately scoped to the stateless/stateful axis; the "not run" / explicit-tests discussion (How do I report a test that's not run - add ExplicitTestNodeStateProperty #2538) is intentionally out of scope.

Testing

  • Microsoft.Testing.Platform builds clean across all TFMs (net8.0/net9.0/netstandard2.0, i.e. both the STJ and Jsonite serializer paths).
  • Microsoft.Testing.Extensions.VSTestBridge.UnitTests and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean.

Expose a client-declared statefulness capability so test frameworks can
distinguish a stateful client (persists an addressable set of test nodes
and their last-known state, e.g. an IDE test explorer) from a stateless
client (streams updates, e.g. dotnet test).
- Add experimental IClientCapabilities { IsStateful } surfaced via
IClientInfo.Capabilities, mirroring the wire protocol's
clientInfo/capabilities split.
- Wire isStateful through the server-mode initialize handshake under
capabilities.testing, backward-compatible (absent => stateless).
- Default stateless in console host; build from client capabilities in
server host.
- Document the isStateful client capability in the protocol intro.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:26

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a client-declared statefulness capability to Microsoft.Testing.Platform (MTP), relating to issue #6494. It lets a client declare (rather than the framework infer) whether it retains test-node state for the whole session (stateful, e.g. an IDE Test Explorer) or consumes updates as a stream (stateless, e.g. dotnet test). The flag travels through the server-mode initialize handshake under capabilities.testing.isStateful alongside the existing debuggerProvider, is exposed via a new experimental IClientCapabilities surfaced from IClientInfo.Capabilities, and is fully backward compatible (absent/false ⇒ stateless).

Changes:

  • New experimental IClientCapabilities { bool IsStateful } ([TPEXP]) and a new IClientInfo.Capabilities getter, wired through ClientInfoService/ClientCapabilitiesService.
  • Both server-mode deserializer paths (STJ and Jsonite) read the optional isStateful flag, defaulting to false; console host is stateless, server host reads the client's declared value.
  • Protocol docs and acceptance/unit test call sites updated for the new field.
Show a summary per file
FileDescription
src/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.csNew experimental interface exposing IsStateful (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/ClientCapabilitiesService.csNew internal record implementing the interface (missing UTF-8 BOM).
src/Platform/Microsoft.Testing.Platform/Services/IClientInfo.csAdds Capabilities getter to the client-info contract.
src/Platform/Microsoft.Testing.Platform/Services/ClientInfoService.csThreads IClientCapabilities through the record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/RpcMessages.csAdds IsStateful to the ClientCapabilities wire record.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/JsonRpcMethods.csAdds the "isStateful" JSON key constant.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/SerializerUtilities.Deserializers.csJsonite path reads optional isStateful.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Json.Deserializers.csSTJ path reads optional isStateful; lacks a unit test.
src/Platform/Microsoft.Testing.Platform/Hosts/ServerTestHost.RequestExecution.csBuilds capabilities from the client's declared value.
src/Platform/Microsoft.Testing.Platform/Hosts/ConsoleTestHost.csConsole host declares stateless.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txtDeclares the three new [TPEXP] API members.
docs/mstest-runner-protocol/001-protocol-intro.mdDocuments isStateful under capabilities.testing.
test/.../ObjectModel/RunSettingsPatcherTests.csUpdates constructor calls (BOM stripped).
test/.../ObjectModel/ObjectModelConvertersTests.csUpdates constructor calls (BOM stripped).
test/.../ServerMode/v1.0.0/ClientCapabilities.csAdds IsStateful (defaulted) to the test wire record.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

CopilotAI review requested due to automatic review settings July 9, 2026 16:35

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

Clean, well-scoped PR that adds the plumbing for a client-declared isStateful capability through the MTP initialize handshake. The design mirrors the existing debuggerProvider pattern and is properly experimental.

Verdict Table

#DimensionVerdictNotes
1Algorithmic Correctness✅ LGTMBoth deserializers correctly handle the optional field with safe defaults
2Threading & Concurrency✅ N/ANo shared mutable state introduced; ConsoleTestHost field is static readonly, ServerTestHost field is set during single-threaded init
3Public API & Binary Compat⚠️ MODERATEAdding Capabilities to shipped IClientInfo is source-breaking for external implementors — mitigated by [Experimental("TPEXP")] (see inline)
4IPC Wire Compat✅ LGTMOptional field, absent → false, backward-compatible with older clients
5Security & IPC Safety✅ N/ANo new trust boundary or input validation concerns
6Resource & Lifetime✅ N/ANo disposable or long-lived resources introduced
7Performance✅ LGTMNegligible overhead — one extra boolean in deserialization and one extra record field
8Error Handling✅ LGTMBoth deserializer paths handle missing/malformed isStateful gracefully
9Cross-TFM✅ LGTMNo TFM-conditional code; System.Text.Json path and Jsonite path both updated
10Localization✅ N/ANo user-facing strings
11Test Completeness⚠️ NITNo test exercises isStateful: true end-to-end (see inline); acceptable since no behavioral branching exists yet
12Naming & Conventions✅ LGTMIClientCapabilities, IsStateful, ClientCapabilitiesService follow repo conventions
13Documentation✅ LGTMProtocol doc and XML doc comments are accurate and consistent
14Code Structure✅ LGTMMinimal surface area; follows interface → internal record pattern
15PublicAPI.Unshipped.txt✅ LGTMAll 3 new API entries correctly declared with [TPEXP] prefix
16init accessor ban✅ LGTMNo init accessors
17Scope Discipline✅ LGTMSingle concern; follow-ups (consumer usage, client-side sending) tracked in #6494
18–22Remaining dimensions✅ N/ANot applicable to this change

Key Observations

  1. Wire compatibility ✅ — Both the STJ (Json.Deserializers.cs) and Jsonite (SerializerUtilities.Deserializers.cs) paths treat isStateful as optional, defaulting to false. Older clients that don't send the field will continue to work unchanged.

  2. API evolution note — The new Capabilities property on IClientInfo is technically a source break for external implementors, but IClientInfo is [Experimental("TPEXP")] so this is expected. If IClientInfo is ever graduated, a default interface method or separate interface will be needed (see inline comment).

  3. Pure plumbing — No code currently reads IsStateful to change behavior. The value flows from the wire → internal ClientCapabilities record → ClientCapabilitiesServiceIClientInfo.Capabilities and stops there. This is intentional per the PR description; actual behavioral branching comes in a follow-up once clients start sending the flag.

Overall: well-designed, minimal, and backward-compatible. No blocking issues found.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 15/15 changed files
  • Comments generated: 5
  • Review effort level: Medium

Comment threadsrc/Platform/Microsoft.Testing.Platform/Services/IClientCapabilities.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The build fails with 21 RS0051 errors because new internal API symbols introduced in this PR are not declared in InternalAPI/InternalAPI.Unshipped.txt. The public API entries in PublicAPI/PublicAPI.Unshipped.txt were correctly updated for the IClientCapabilities interface, but the corresponding internal implementation types were missed.

Root cause: Missing internal API declarations in InternalAPI.Unshipped.txt

This project tracks both public and internal API surfaces. The PR introduces three groups of internal API changes that all need entries in src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

Group 1 — New ClientCapabilitiesService record (12 errors)

The entirely new internal sealed record ClientCapabilitiesService(bool IsStateful) : IClientCapabilities generates 12 undeclared symbols (constructor, properties, operators, Equals, GetHashCode, ToString, Clone, Deconstruct).

Group 2 — Modified ClientInfoService record (4 errors)

Adding the IClientCapabilities Capabilities parameter to the existing record changes the constructor signature and Deconstruct method, and adds the new Capabilities property with getter/init.

Group 3 — Modified ServerMode.ClientCapabilities record + JsonRpcStrings constant (5 errors)

Adding bool IsStateful to ClientCapabilities(bool DebuggerProvider, bool IsStateful) generates a new constructor, Deconstruct, and the IsStateful property. The new JsonRpcStrings.IsStateful constant also needs a declaration.


Proposed fix

Add the following lines to src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt:

const Microsoft.Testing.Platform.ServerMode.JsonRpcStrings.IsStateful = "isStateful" -> string!
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider, bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider, out bool IsStateful) -> void
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.get -> bool
Microsoft.Testing.Platform.ServerMode.ClientCapabilities.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.<Clone>$() -> Microsoft.Testing.Platform.Services.ClientCapabilitiesService!
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ClientCapabilitiesService(bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Deconstruct(out bool IsStateful) -> void
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? other) -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.get -> bool
Microsoft.Testing.Platform.Services.ClientCapabilitiesService.IsStateful.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.get -> Microsoft.Testing.Platform.Services.IClientCapabilities!
Microsoft.Testing.Platform.Services.ClientInfoService.Capabilities.init -> void
Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version, Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version, out Microsoft.Testing.Platform.Services.IClientCapabilities! Capabilities) -> void
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.Equals(object? obj) -> bool
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.GetHashCode() -> int
override Microsoft.Testing.Platform.Services.ClientCapabilitiesService.ToString() -> string!
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator !=(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool
static Microsoft.Testing.Platform.Services.ClientCapabilitiesService.operator ==(Microsoft.Testing.Platform.Services.ClientCapabilitiesService? left, Microsoft.Testing.Platform.Services.ClientCapabilitiesService? right) -> bool

Additionally, mark the old record signatures (now replaced) as removed:

*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.ClientCapabilities(bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.ServerMode.ClientCapabilities.Deconstruct(out bool DebuggerProvider) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.ClientInfoService(string! Id, string! Version) -> void
*REMOVED*Microsoft.Testing.Platform.Services.ClientInfoService.Deconstruct(out string! Id, out string! Version) -> void

Note: The *REMOVED* entries are not causing RS0051 errors today (RS0017 may be a warning in this project), but they are necessary for accurate API tracking since the old constructor/Deconstruct signatures no longer exist.


Build overview
PropertyValue
Result❌ FAILED
Duration208.8s
MSBuild18.8.0-preview-26302-115+f7b4c5716
Projects49
Errors22 (21 RS0051 + 1 meta Build failed)
Warnings0
Failed projectsMicrosoft.Testing.Platform.csprojBuild.projNonWindowsTests.slnf (cascade)
All MSBuild errors (21)
#CodeFileSymbol
1RS0051JsonRpcMethods.cs:43const JsonRpcStrings.IsStateful
2RS0051RpcMessages.cs:93ClientCapabilities.ClientCapabilities(bool, bool)
3RS0051RpcMessages.cs:93ClientCapabilities.Deconstruct(out bool, out bool)
4RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.get
5RS0051RpcMessages.cs:93ClientCapabilities.IsStateful.init
6RS0051ClientCapabilitiesService.cs:6ClientCapabilitiesService (type + 11 auto-generated members)
7–16RS0051ClientCapabilitiesService.cs:6.ctor, .IsStateful, .Equals, .GetHashCode, .ToString, ==, !=, .<Clone>$, .Deconstruct
17RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.get
18RS0051ClientInfoService.cs:6ClientInfoService.Capabilities.init
19RS0051ClientInfoService.cs:6ClientInfoService.ClientInfoService(string!, string!, IClientCapabilities!)
20RS0051ClientInfoService.cs:6ClientInfoService.Deconstruct(out string!, out string!, out IClientCapabilities!)
21RS0051ClientInfoService.cs:6ClientInfoService(string!, string!, IClientCapabilities!)

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · commit f66b623

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K · [◷]( · )

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · 141.2 AIC · ⌖ 9.08 AIC · ⊞ 7.3K ·

…ful tests
- Declare new internal symbols (JsonRpcStrings.IsStateful, ClientCapabilities/
ClientCapabilitiesService/ClientInfoService members) in InternalAPI.Unshipped.txt
to satisfy the newly-added InternalAPI tracking analyzer (RS0051) that broke the
Linux CI build after merging main.
- Restore the required UTF-8 BOM (charset=utf-8-bom) on the new and edited .cs files.
- Add unit tests covering both deserializer paths (System.Text.Json and Jsonite)
for isStateful: true and the absent (stateless) default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 16:54

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

@Evangelink
Amaury Levé (Evangelink) enabled auto-merge (squash) July 9, 2026 17:04
@EvangelinkAmaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Jul 9, 2026
@github-actions

This comment has been minimized.

The ClientCapabilities and ClientInfoService record constructors (and their
Deconstruct methods) changed signature, removing the old ones that are declared
in InternalAPI.Shipped.txt. Declare them as *REMOVED* in InternalAPI.Unshipped.txt
so the Public/Internal API analyzer (RS0017) does not fail the Arcade -warnaserror build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings July 9, 2026 17:06

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9789

9 test methods graded across 2 files (4 new, 5 modified). All tests pass the basics; the four new deserialization tests each make a single targeted boolean assertion — clean and focused, but could be strengthened by setting debuggerProvider: false to rule out field-aliasing false-passes.

GradeTestNotes
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithIsStatefulTrue_
JsonitePath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
ClientCapabilities_
WithoutIsStateful_
JsonitePath_
DefaultsToStateless
Single Assert.IsFalse covers the default-stateless path cleanly; no further issues.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithIsStatefulTrue_
StjPath_
SurfacesStatefulClient
Single Assert.IsTrue; setting debuggerProvider: false would guard against field-aliasing false-passes.
B (80–89)new JsonTests.
Deserialize_
InitializeRequest_
WithoutIsStateful_
StjPath_
DefaultsToStateless
Single Assert.IsFalse verifies default-stateless behavior; symmetric with the true-path test, no issues.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenNoRunSettingsProvided_
CreateRunSettingsWithResultsDirectoryElement
Single equality assertion on ResultsDirectory; mechanical constructor-arg update, test body is clean.
B (80–89)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsExists_
MergesParameters
Six thorough merge assertions; body is ~32 lines — consider splitting into focused preserve/update/add tests.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WhenRunSettingsDoesNotExist_
AddParameters
Four assertions fully verify both added parameters (name + value each); focused and clean.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsContainingResultsDirectory_
EntryIsNotOverridden
Two assertions verify non-override behavior and existing element preservation; clear and complete.
A (90–100)mod RunSettingsPatcherTests.
Patch_
WithRunSettingsProvidedButMissingResultsDirectory_
AddsElement
Two complementary assertions check added ResultsDirectory and preserved Canary element.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 96.6 AIC · ⌖ 9.1 AIC · ⊞ 9.5K · [◷]( · )

@Evangelink
Amaury Levé (Evangelink) merged commit 4842e15 into mainJul 9, 2026
160 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/supreme-chainsaw branch July 9, 2026 18:59
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-reviewAwaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Evangelink@0101