Uh oh!
There was an error while loading. Please reload this page.
Add silence-driven progress heartbeat for SimpleAnsi/NoAnsi modes (#9139) - #9147
Merged
Merged
Conversation
…nal reporter Fixes#9138 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implements the silence-driven heartbeat renderer (#9139) so CI / piped / file-redirected runs get a progress signal between banner and summary, without the rejected fixed-cadence per-DLL summary (#6753). - Extract IProgressRenderer strategy; keep existing in-place redraw as CursorProgressRenderer (ANSI cursor modes, unchanged behavior). - Add SilenceDrivenHeartbeatRenderer for SimpleAnsi/NoAnsi: - E1 silence heartbeat (one line after N seconds of no completion), - E2 slow-test surfacing with exponential backoff, - E3 failures continue to print inline. - Collapse the SimpleAnsi/NoAnsi progress gating in TerminalOutputDevice. - Knobs (env vars only): MTP_PROGRESS_SILENCE_SECONDS (default 30), MTP_PROGRESS_SLOW_TEST_SECONDS (default 60); 0 disables. - Localized resources + regenerated xlf; unit tests for all rules. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Youssef Fahmy (Youssef1313)
commented
Jun 15, 2026
Member
Amaury Levé (@Evangelink) Can you please add some before/after screenshots showing the behavior difference? |
…etailed The CI test steps passed --no-progress (and --output detailed), which force-disabled progress so the new SilenceDrivenHeartbeatRenderer added in this PR was never exercised on CI (SimpleAnsi mode). Dropping these flags makes the testfx CI fall into the heartbeat path and dogfood the feature. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Jan Krivanek (JanKrivanek)
approved these changes
Jun 15, 2026
Amaury Levé (Evangelink)
commented
Jun 15, 2026
MemberAuthor
🧪 Test quality grade — PR #914710 tests graded (8 new, 2 modified) across 2 files. All tests score A. The new
This advisory comment was generated automatically. Grades are heuristic
|
…eat-9139 # Conflicts: # test/IntegrationTests/MSTest.Acceptance.IntegrationTests/ShowOutputOptionTests.cs
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Microsoft.Testing.Platform’s terminal progress reporting to work in SimpleAnsi/NoAnsi scenarios (CI, redirected output) by introducing a renderer strategy abstraction and adding a new silence-driven “heartbeat” renderer that emits durable progress lines only when needed. It also wires in environment-variable knobs for heartbeat thresholds, adds localized resource strings, and updates unit tests and CI scripts accordingly.
Changes:
- Introduces
IProgressRendererwithCursorProgressRenderer(existing in-place redraw) andSilenceDrivenHeartbeatRenderer(new durable-line heartbeat/slow-test output). - Enables progress in non-cursor terminal modes by selecting an appropriate renderer rather than force-disabling progress.
- Adds new localized resource strings + unit tests, and updates CI pipeline invocations.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/TerminalTestReporterTests.cs | Updates tests to pass the new renderer dependency into TestProgressStateAwareTerminal. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/SilenceDrivenHeartbeatRendererTests.cs | Adds deterministic unit tests covering silence heartbeat, slow-test backoff, and write-through behavior. |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf | Adds new trans-units for heartbeat/slow-test strings (state=new). |
| src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx | Adds 3 new localized strings used by the heartbeat renderer. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/TerminalOutputDevice.cs | Removes forced progress-disable for SimpleAnsi/NoAnsi and adds env-var threshold parsing into options. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TestProgressStateAwareTerminal.cs | Routes tick/write behavior through IProgressRenderer and adds NotifyTestCompleted() hook. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporterOptions.cs | Adds options for heartbeat silence threshold and slow-test threshold. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.TestCompletion.cs | Notifies the progress renderer on each test completion to reset silence timing. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.cs | Selects cursor vs heartbeat renderer based on terminal capability and passes it into the progress-aware terminal wrapper. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/SilenceDrivenHeartbeatRenderer.cs | Implements silence heartbeat + slow-test durable line emission logic. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/IProgressRenderer.cs | Defines the rendering strategy interface and thread-safety expectations. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/CursorProgressRenderer.cs | Extracts the existing in-place redraw behavior behind IProgressRenderer. |
| eng/pipelines/steps/test-non-windows.yml | Removes --no-progress/--output detailed from CI dotnet test invocations. |
| azure-pipelines.yml | Removes --no-progress/--output detailed from Windows CI dotnet test invocations. |
Copilot's findings
- Files reviewed: 26/26 changed files
- Comments generated: 8
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Debug Test step routes through Microsoft.Testing.Platform.MSBuild's InvokeTestingPlatform task, whose default TestingPlatformCaptureOutput=true buffers each test exe's stdout into a per-module .log file and never logs it to the AzDO console (except on failure). That hid the silence-driven heartbeat introduced in #9147: the renderer was firing but the output was invisible to anyone watching the live pipeline. Add -p:TestingPlatformCaptureOutput=false to the Debug 'dotnet test' invocations in azure-pipelines.yml and eng/pipelines/steps/test-non-windows.yml so stdout is forwarded to Log.LogMessage(MessageImportance.High, ...). The Release step uses --test-modules, which bypasses MSBuild entirely, so it already streams per-module output and needs no change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Publish _clock via Volatile.Write/Read to avoid reading a stale null across threads. - Report a slow test's actual elapsed time instead of the scheduled threshold, so a delayed tick does not under-report the runtime. - Let each IProgressRenderer choose its tick cadence; the silence-driven heartbeat now ticks once per second instead of every 500ms. - Add regression test for the delayed-tick elapsed-time reporting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…eat-9139 # Conflicts: # azure-pipelines.yml # eng/pipelines/steps/test-non-windows.yml
Uh oh!
There was an error while loading. Please reload this page.
Amaury Levé (Evangelink) added a commit
that referenced
this pull request
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the silence-driven progress heartbeat renderer for the
SimpleAnsiandNoAnsiterminal modes. Fixes#9139.Problem
Today
TerminalOutputDeviceforce-disables progress forSimpleAnsi/NoAnsi, so CI / piped / file-redirected runs get zero progress signal between the banner and the summary. The old fix (a per-DLL summary every 3s) was rejected as spam (#6753). This design is silence-driven, not time-driven.What changed
Renderer abstraction (
OutputDevice/Terminal/)IProgressRenderer— strategy for per-tick render, write-wrapping, and completion notification.CursorProgressRenderer— the existing in-place redraw, extracted verbatim (no behavior change for ANSI cursor modes).SilenceDrivenHeartbeatRenderer— new, emits single durable lines only when needed:running... N completed, M failed | active: Xafter N seconds of no completion; repeats once per interval during prolonged silence. A healthy fast suite emits nothing.[slow] still running after …: <test> (<asm>|<tfm>|<arch>)per running test over the threshold, with exponential backoff (60s → 2m → 4m …). Durable scrollback line.Gating (
TerminalOutputDevice.cs)noProgress || ansiMode is NoAnsi or SimpleAnsito justnoProgress; renderer is chosen by resolved terminal capability. Test-host controller /--list-tests/ server mode still suppress.Knobs (env vars only, no new CLI flags)
MTP_PROGRESS_SILENCE_SECONDS(default30;0disables E1)MTP_PROGRESS_SLOW_TEST_SECONDS(default60;0disables E2)Localization — 3 new strings in
PlatformResources.resxwith all 13.xlffiles regenerated via/t:UpdateXlf.Before / After
The feature currently doesn't work with
dotnet testand requires a change there but when running the exe itself (or viadotnetordotnet run) we can see the change.Before: no info during the 20s of the run.
After (the screenshot belows sets progress silence as 2s and slow tests as 2s too):
For a more real use case, on this pipeline with progress disabled and default output level, we have a gap of ~16.5min dead-air window. With this feature (assuming dotnet test handling is done), we would have about:
Tests
SilenceDrivenHeartbeatRendererTests.cs(silence fire / no-fire, repeat-per-interval, completion reset, disabled, slow-test backoff, below-threshold, disabled, write-through).TestProgressStateAwareTerminalctor signature.Microsoft.Testing.Platform.UnitTestsgreen. No new CLI flag, so--help/--infoacceptance expectations are unaffected.Out of scope (deliberate)
IProgressEnricherextension hook — name is TBD and per-CI extensions are out of scope (Silence-driven progress heartbeat renderer for SimpleAnsi/NoAnsi terminal modes #9139); also avoids new public API surface. The renderer is structured so it can be added later without churn.