Skip to content

fix(ci): create tracesDir before stacks write + add MCP test retries - #40735

Closed
Sebastien Tardif (SebTardif) wants to merge 2 commits into
microsoft:mainfrom
SebTardif:fix/mcp-test-ci-retries
Closed

fix(ci): create tracesDir before stacks write + add MCP test retries#40735
Sebastien Tardif (SebTardif) wants to merge 2 commits into
microsoft:mainfrom
SebTardif:fix/mcp-test-ci-retries

Conversation

@SebTardif

@SebTardifSebastien Tardif (SebTardif) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix ENOENT when writing .stacks file during live tracing (the directory may not exist yet)
  • Add retries: process.env.CI ? 3 : 0 to the MCP test config, matching every other test suite

Bug: ENOENT on .stacks file (tracing.spec.ts)

tracingStarted() in localUtils.ts computes the stacks file path inside tracesDir, but when tracesDir is provided (not a temp dir), the directory is never created. The zip() function does mkdir(path.dirname(...), { recursive: true }) before writing the trace zip, but addStackToTracingNoReply() assumes the directory exists and calls writeFile() directly.

On Windows CI, this causes an intermittent ENOENT because the directory creation timing between the browser launch (which creates tracesDir for its own trace files) and the first client-side stacks write is less predictable:

Error: ENOENT: no such file or directory, open
'D:\...\test-results\tracing-...-firefox\.playwright-mcp\traces\trace-1778265446373.stacks'
at async open (node:internal/fs/promises:639:25)
at async Object.writeFile (node:internal/fs/promises:1219:14)

Fix: call mkdir({ recursive: true }) in tracingStarted() when tracesDir is provided, so the directory is guaranteed to exist before any stacks are written. This matches the existing pattern in zip().

Example CI failure: PR #40722, windows-latest - firefox at tracing.spec.ts:54.

Retries: match other test configs

The MCP test suite is the only test config in the project without CI retries:

ConfigRetries
tests/library/playwright.config.tsprocess.env.CI ? 3 : 0
tests/installation/playwright.config.tsprocess.env.CI ? 3 : 0
tests/electron/playwright.config.tsprocess.env.CI ? 3 : 0
tests/extension/playwright.config.tsprocess.env.CI ? 2 : 0
tests/android/playwright.config.tsprocess.env.CI ? 1 : 0
tests/mcp/playwright.config.tsnone

The ENOENT bug is a concrete fix, but the MCP tests also have non-deterministic failures that only retries can address:

  • McpError: Connection closed during teardown races (e.g., config.ini.spec.ts, video.spec.ts on webkit Windows)
  • Flaky toHaveCount assertions on timing-sensitive UI elements (e.g., annotate.spec.ts)
  • HTTP transport lifecycle races (e.g., http.spec.ts on webkit Windows)

These are the same class of non-deterministic failures that retries already handle in every other test suite.

When tracesDir is provided (not a temp dir), the directory may not
exist yet when the first live stack is written. The zip() function
already does mkdir before writing, but addStackToTracingNoReply
assumes the directory exists and fails with ENOENT on Windows CI
where directory creation timing is less predictable.
@SebTardifSebastien Tardif (SebTardif) changed the title fix(ci): add retries for MCP tests to match other test configsfix(ci): create tracesDir before stacks write + add MCP test retriesMay 8, 2026
@SebTardif

Copy link
Copy Markdown
ContributorAuthor

Closing: the .stacks mkdir fix duplicates #40730, and the retries alone don't address the root cause (Connection closed in annotate tests on Windows). The team is already actively working on these issues.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@SebTardif