Skip to content

Complete Azure DevOps live-published test results - #11142

Merged
Amaury Levé (Evangelink) merged 3 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/fix-azure-devops-pending-results
Sep 9, 2026
Merged

Amaury Levé (Evangelink) merged 3 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/fix-azure-devops-pending-results

Conversation

@Evangelink

@Evangelink Amaury Levé (Evangelink) commented Sep 9, 2026

Copy link
Copy Markdown
Member

Azure DevOps keeps live-published test results in Pending when the payload contains a terminal outcome but omits the result-level state. This change publishes state: "Completed" for every terminal result so completed runs report accurate passed, failed, and incomplete counts.

The regression coverage verifies:

  • MTP-to-Azure DevOps mapping for passed, failed, skipped, timed out, and cancelled results.
  • The actual initial POST payload through the browser-WASM fake Azure DevOps REST endpoint.
  • The initial POST and rerun PATCH payload contracts, including the separate PATCH serializer configuration.

The adjacent-path audit confirmed retry attempts use the same parent result model. Azure DevOps TestSubResult has no result-state field, so no equivalent sub-result change is required.

Fixes: #11122

Publish terminal result state alongside outcome so Azure DevOps does not leave individual results pending after the run completes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assert that both initial POST and rerun PATCH requests preserve the completed result state.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The fix and regression coverage are complete, with no unresolved issues.

Review tier: Balanced
Findings: None

What changed in this PR

Fixes Azure DevOps live-published terminal test results remaining Pending by explicitly setting their state to Completed.

Changes:

  • Adds state: "Completed" to terminal results.
  • Adds unit and browser-WASM regression coverage.
  • Updates the internal API baseline.
File Description
test/​UnitTests/​Microsoft.Testing.Extensions.UnitTests/​AzureDevOpsLivePublishingTests.cs Verifies terminal-state mapping.
test/​IntegrationTests/​Microsoft.Testing.Platform.Acceptance.IntegrationTests/​BrowserWasmExecutionTests.cs Verifies serialized REST payload states.
src/​Platform/​Microsoft.Testing.Extensions.AzureDevOpsReport/​InternalAPI/​InternalAPI.Unshipped.txt Tracks the new internal getter.
src/​Platform/​Microsoft.Testing.Extensions.AzureDevOpsReport/​AzureDevOpsLivePublishingModels.cs Adds the completed result state.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI review requested due to automatic review settings September 9, 2026 07:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The completed state is consistently implemented and covered by unit and integration tests.

Review tier: Balanced
Findings: None

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Pipeline Test Triage — Final Resolution

Build: [1588618]((dev.azure.com/redacted) (microsoft.testfx, full/completed analysis)
PR: #11142 — head 3cf82ae0 (matches expected head/merge SHAs)

Result: No actionable test failures found.

  • All 90,841 collected test records across CTRF/TRX/JUnit reports for this build are passed or skipped (400 skipped, all with expected reasons such as OS-restriction or environment gating). failureOrRetryCount = 0, diagnosticCount = 0, slowRegressionCount = 0 (37 candidates flagged but none met the 10-sample/60s-floor/3x-p95 slowness threshold). Evidence is complete (evidenceIncomplete: false).
  • The pipeline's overall failed result stems from the MacOS Release job being cancelled due to a worker timeout (Job cancelled due to worker timeout, Publish Test Results folders step also cancelled), not from any test assertion failure. The Test task itself completed successfully before the cancellation, and the pipeline timeline explicitly notes: "macOS is non-blocking for PRs; this failure does not block the merge."
  • This is an infrastructure/agent-timeout issue rather than a test-product defect, and is out of scope for this test-triage agent (ordinary build/agent failures are handled separately). No recurring or historical pattern for a specific test was identified since there is no failing test to correlate.

Conclusion: No issue created — no durable test defect, flaky test, crash, or slowness regression threshold was met. The macOS job cancellation is a non-blocking infrastructure timeout; if it recurs across multiple builds, it would warrant separate build-infrastructure investigation rather than a test-triage issue.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · auto · 36.3 AIC · ⌖ 1.78 AIC · ⊞ 9.3K · [◷]( · )

Retrigger the required build after Azure DevOps canceled the macOS job due to an infrastructure worker timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Amaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Sep 9, 2026
@Evangelink
Amaury Levé (Evangelink) merged commit 594cb69 into microsoft:main Sep 9, 2026
33 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/fix-azure-devops-pending-results branch September 9, 2026 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-review Awaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Azure DevOps live-published results remain Pending after the test run completes

3 participants