Add black-box baseline tests for dotnet-test pipe protocol (#51615 prep) by Evangelink · Pull Request #9153 · microsoft/testfx

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Amaury Levé (Evangelink) · 2026-06-15T13:38:13Z

Prep PR for dotnet/sdk#51615 ("Improve stdout/stderr handling of MTP test apps under dotnet test").

This PR is intentionally product-code-free. It introduces a black-box
acceptance harness and three baseline tests that pin today's wire-format
behavior of Microsoft.Testing.Platform running under
--server dotnettestcli, so that the upcoming behavior changes (live
stdout/stderr forwarding, IOutputDevice routing through the pipe,
protocol bump to 1.0.1) can be reviewed against an explicit before/after.

Why a black-box harness?

The production IPC types under src/Platform/Microsoft.Testing.Platform/IPC/*
(NamedPipeServer, IRequest, IResponse, NamedPipeBase, all base
serializers) carry [Microsoft.CodeAnalysis.Embedded], which means they
are emitted as private copies per assembly and cannot be referenced from
the acceptance test project even with InternalsVisibleTo.

Rather than copy-link ~30 files into the test project (and silently
diverge from product over time), the harness re-implements the wire
format on top of System.IO.Pipes.NamedPipeServerStream. As a side
benefit, it now also doubles as a contract test for the wire format.

What's in the PR

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/
├── DotnetTestPipeProtocol.cs ← framing + handshake/event encode/decode (BCL only)
├── FakeDotnetTestSdk.cs ← `FakeDotnetTestSdk.RunAsync(testHost, ...)` harness
├── FakeDotnetTestSdkResult.cs ← DTO returned by harness
└── DotnetTestPipeBaselineTests.cs ← 3 baseline `[TestMethod]`s with inline asset

Baseline tests

DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today
pins that the test app advertises 1.0.0 and that a fake SDK
advertising 1.0.0 negotiates 1.0.0.
DotnetTestPipe_EmitsTestSessionStartAndEnd pins that a single
no-op session produces exactly one TestSessionEvent with
SessionType=TestSessionStart and one with TestSessionEnd.
DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline
pins that today the test app emits nothing to stdout/stderr under
pipe mode for a passing no-op run (the banner is already silenced
via the existing _isServerMode check in TerminalOutputDevice).

Test results

total: 3
failed: 0
succeeded: 3
duration: 3s 736ms

No product code changes. No public API changes.

Follow-ups (separate PRs)

#51615 Phase 1 — testfx: route warnings/errors/text through pipe
under --server dotnettestcli instead of writing to local IConsole
(closesConsider: Disable MTP's own TerminalTestReporter when running under server mode (pipe specifically maybe?) #7161 too). Adds new LogMessage message.
#51615 Phase 2 — protocol bump to 1.0.1 (both sides advertise
1.0.1;1.0.0) plus new StandardStreamChunk message for live
stdout/stderr forwarding (opt-in via handshake flag).
#51615 Phase 3 — SDK side: consume new messages and add CLI
surface (--live-output or equivalent).

Once Phase 1 lands, these baseline tests will need to be adjusted to
match the new contracts; today they intentionally describe the current
behavior so the diff is reviewable.

…otnet/sdk#51615) This PR introduces a hand-rolled `FakeDotnetTestSdk` harness plus three baseline acceptance tests that pin today's wire-format behavior of `Microsoft.Testing.Platform` running under `--server dotnettestcli`. The harness is intentionally black-box: it uses only `System.IO.Pipes.NamedPipeServerStream` from the BCL and re-implements the dotnet-test wire format (`int32 size | int32 serializerId | body`, with leading `ushort fieldCount` in handshake/event bodies) directly. This was necessary because the production IPC types under `src/Platform/Microsoft.Testing.Platform/IPC/*` are decorated with `[Microsoft.CodeAnalysis.Embedded]` and so cannot be referenced across assembly boundaries even with `InternalsVisibleTo`. As a side benefit the harness now also acts as a contract test for the wire format. The three tests: 1. `DotnetTestPipe_TestAppAdvertisesAndNegotiatesProtocolV100_Today` pins protocol negotiation: the test app advertises `1.0.0`, and a fake SDK that advertises `1.0.0` negotiates `1.0.0`. 2. `DotnetTestPipe_EmitsTestSessionStartAndEnd` pins that a single no-op session produces exactly one `TestSessionEvent` with `SessionType=TestSessionStart` and one with `TestSessionEnd`. 3. `DotnetTestPipe_ChildEmitsNoStdoutOrStderrForPassingRun_Baseline` pins that today the test app emits nothing to stdout/stderr under pipe mode for a passing no-op run (banner is already silenced via the existing `_isServerMode` check in `TerminalOutputDevice`). These baselines lock down the current behavior so subsequent work on dotnet/sdk#51615 (live stdout/stderr forwarding, `IOutputDevice` routing through the pipe, protocol bump to 1.0.1) can be reviewed against an explicit before/after. No product code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds a black-box acceptance test harness and baseline tests that exercise the current --server dotnettestcli --dotnet-test-pipe wire protocol behavior of Microsoft.Testing.Platform when run under dotnet test, to enable clear before/after review for the upcoming dotnet/sdk#51615 behavior changes.

Changes:

Introduces a minimal pipe protocol reader/writer (DotnetTestPipeProtocol) implemented using only BCL types.
Adds a fake “dotnet test SDK” harness (FakeDotnetTestSdk) that hosts the pipe server, performs handshake negotiation, and captures frames.
Adds three baseline acceptance tests that pin current protocol negotiation, session start/end events, and “no stdout/stderr” behavior under pipe mode.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdkResult.cs	DTO capturing handshake/messages/process output observed during a pipe-mode run.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs	Fake SDK pipe server harness + handshake/version negotiation logic.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs	Black-box framing + handshake and session-event decoding/encoding helpers.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs	Baseline acceptance tests + inline test asset used to drive pipe-mode behavior.

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 3

Amaury Levé (Evangelink) · 2026-06-15T13:46:09Z

🧪 Test quality grade — PR #9153

3 new integration tests graded across 1 file. All three earn A — clear Arrange/Act/Assert structure, multiple meaningful assertions with rich diagnostic failure messages, and no anti-patterns. The baseline-contract pattern (documenting current behavior so regressions are visible as intentional diffs) is a particularly valuable technique for protocol tests.

Δ	Test	Grade	Band	Notes
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ ChildEmitsNoStdoutOrStderrForPassingRun_ Baseline`	A	90–100	Three complementary negative assertions cover stdout banner, full stdout, and stderr; each embeds captured output in its failure message.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ EmitsTestSessionStartAndEnd`	A	90–100	Verifies both event count and ordering; diagnostic message surfaces the actual sequence on failure.
new	`DotnetTestPipeBaselineTests. DotnetTestPipe_ TestAppAdvertisesAndNegotiatesProtocolV100_ Today`	A	90–100	Null preconditions + two equality assertions pin the version-negotiation contract; messages explain the expected baseline evolution path.

_{This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Re-run with /grade-tests.}

Amaury Levé (Evangelink)

- Trim SDK supported-version tokens and project app versions via Select - Fix misleading CurrentUserOnly comment (harness targets $(NetCurrent)) - Advance stream past extra SessionType bytes to stay aligned - Remove unnecessary Task.Run wrapper around already-async ExecuteAsync - Make SelectHighestMutuallySupportedVersion private static - Add assertion message for SentHandshakeReply guard - Use Regex-typed DoesNotMatchRegex overload to keep Compiled flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI review requested due to automatic review settings June 15, 2026 13:38

Copilotstarted reviewing on behalf of Amaury Levé (Evangelink)June 15, 2026 13:38 View session

CopilotAI reviewed Jun 15, 2026
View reviewed changes

github-code-qualityBot found potential problems Jun 15, 2026
View reviewed changes

Comment thread...s/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/FakeDotnetTestSdk.cs Fixed

Amaury Levé (Evangelink) commented Jun 15, 2026
View reviewed changes

Amaury Levé (Evangelink) enabled auto-merge (squash) June 16, 2026 07:03

Amaury Levé (Evangelink) disabled auto-merge June 16, 2026 08:05

Amaury Levé (Evangelink) merged commit f5704c4 into mainJun 16, 2026
88 checks passed

Amaury Levé (Evangelink) deleted the dev/amauryleve/mtp-pipe-stdout-#51615 branch June 16, 2026 08:05

Amaury Levé (Evangelink) mentioned this pull request Jun 16, 2026
Stream discovered tests to the SDK for --list-tests json under --server (#8661) #9192
Merged