diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index af91e690f5..48354bbd57 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -171,13 +171,19 @@ Records before 2026-07-28 were written by hand and had drifted: 146 lines carrie | 2026-07-30 | claude/outstanding-issues-triage-24c8ow | 8d2710fd6cbdc84e8c50a6c9bc0a1e1a0cd612c8 | open PR changed-scope review | APPROVE: completed items 095, 096, 104, 109, and 115 move to archive with no deletion, duplicate ID, or stale next-id. | check:outstanding-issues PASS; check:branch-review-ledger PASS; diff review; no unresolved threads | | 2026-07-30 | claude/latency-findings-impl-s8g01v | e7ff5e933ba1f34d5adbd46dd77c38aced11ed44 | open PR changed-scope review | APPROVE: ordering-risk documentation is accurate and the near-bottom refusal guard now proves its geometry is non-vacuous before asserting no hide. | diff check PASS; focused test review; no unresolved threads; exact-head Production UI required | | 2026-07-30 | claude/ci-testing-review-2l8klp | 2e2160bc8b9d2d824209c217c67cb9cac1be3a8d | open PR changed-scope review | APPROVE: three-way UI sharding, critical-first gating, measured drag travel, and gate-manifest updates preserve required-check aggregation and deterministic Playwright settings. | check:github-actions PASS; check:ci-scope PASS; check:gate-manifest PASS; ledger guards PASS; exact-head sharded Production UI required | +| 2026-07-30 | PR #1432 | a2b53c815b3c060dec2619af2855a63f9f496858 | Playwright browser preflight review and repair | fixed; focused tests pending coordinator | Prettier PASS; issues guard PASS; focused Vitest blocked by active Playwright lease | +| 2026-07-30 | PR #1432 | f85995ade3a19513a531713724813adc742c360d | Playwright browser preflight verification | focused tests pass; typecheck lease-blocked | 16 focused tests PASS; Prettier PASS; typecheck admission blocked | | 2026-07-30 | PR #1430 | a9ae22ac4915e86d51ee05787059382a39bd8ba8 | phone chrome diagnostics and merge repair | fixed and ready for CI | issues guard; ledger guard; 37 focused tests; phone-chrome dry-run | +| 2026-07-30 | PR-1432 | 7c7b63cf40d59652954e539ce1b3027005916bf1 | PR #1432 Playwright browser preflight final exact-head review | fixed existing project-isolation contract after preflight refactor; no remaining findings | preflight and isolation Vitest 9/9; typecheck pass; Prettier and diff checks pass | | 2026-07-30 | PR-1440 | f7260cc6a0da87cb4df1ac95ef962967e667c3f0 | PR #1440 issue #102 ordering correction | accurately restores the two canary-gated retrieval ordering constraints; no findings | outstanding-issues and ledger guards pass; documentation-only diff | -| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | 11e7e8653c4742108feee4e9164ba066a92fc18a | ci-testing-review-capture | Captures #141, the one finding from the PR #1427 session that was never recorded: ui-phone-scroll.spec.ts:973 Services result anchor jumped on viewport shrink (run 30534158395 shard 1, :1133, 120 passed). Distinct from #127 and not fixed by #1427 - that head already carried #1427's runway poll and travel assertion, the journey is the Services result canvas not the document route, and the assertion is anchor stability across a resize not a chrome hide. Narrowing recorded: the sibling documentScrollTop assertion at :1137 did NOT fail, so the scroll position held while the element moved, ruling out the scroll-restoration class. Branch restarted from origin/main because #1427 merged as 040ce97 (verified by content not ancestry: shard matrix, dragScrollUntilHidden and the gate-count guard all present on main). NOTE for #127: its updated row shows the trace-based refutation of the short-drag hypothesis was itself misread (maxOffset 2753 read at a different trace moment than the failing drag; pre-runway value was 1153), so the root cause withdrawn during the #1427 session was in fact correct - a third-party refutation needs the same verification as a third-party fix claim. | verify:cheap PASS (Test Files 435 passed; Tests 4569 passed, 4 skipped, exit 0); check:outstanding-issues PASS (139 rows, unique ids, next-id=142 above highest); prettier --check . PASS. Docs-only: ui_changed should be false and the Chromium shards should skip. | -| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | e6c704c6c8cd989825adf30ecbffeb57344bfa56 | ci-testing-review-capture | SUPERSEDES the 11e7e865 capture row: the Services viewport-anchor flake is now #142, not #141. Main's #141 (design-sync --clinical-accent-strong token gap from PR #1443) landed while this branch was open, colliding on the same id. Merge conflict in docs/outstanding-issues.md resolved by rebuilding from origin/main and re-applying the Services capture as #142 with next-id=143. Content of the capture is unchanged. | check:outstanding-issues PASS (140 rows, 68 open, 72 archived, next-id=143); merge-tree was CONFLICTING on outstanding-issues only; conflict markers removed | -| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | 8be4f703d5729b4aa10e73ee8fbc77e03f400b8b | ci-testing-review-capture | Withdraws an invalid inference from the earlier records for this PR, on a correct Codex finding. Those rows argued that because the sibling documentScrollTop assertion did not fail, the scroll position held and scroll-restoration causes were ruled out. Playwright aborts a test at the first failing expect, so once anchorTop threw, documentScrollTop NEVER EXECUTED - its absence from the output shows nothing. The #142 row now says so and the class is not ruled out. The capture itself stands: the Services viewport-anchor failure is real, intermittent on byte-identical code (pass/pass/fail/pass-on-rerun), and distinct from #127. Separately CodeRabbit flagged :973 vs :1133 as inconsistent and then withdrew it: :973 is the test declaration and :1133 the thrown assertion, both reported by Playwright, and declaration lines drift (898 / 973 / 1041 across three tree states) which is why the exact title is the durable identity. | check:outstanding-issues PASS (140 rows, unique ids, next-id=143). Lesson: reasoning from an assertion that never ran is the same verified-vs-assumed error this session already hit twice in the other direction. | +| 2026-07-30 | PR-1432 | a5d234302b57be6f7ce5d1957c9ec00bc7f191f0 | PR #1432 Playwright preflight and phone-scroll reliability | cross-platform preflight fails closed and production focus-restore race is removed from the phone-scroll proof; no remaining findings | preflight tests 9 passed; focused Chromium journey 2 passed; formatting and ledger guards pass | +| 2026-07-30 | PR-1432 | 330086eff76f704ce6b9cf5405aeecfdd375027c | PR #1432 visual-config preflight follow-up | visual runs now preflight chromium-artifacts instead of the unrelated main browser matrix; unknown configs fail closed | config-selection tests added; formatting passes; exact-head CI pending | | 2026-07-30 | PR-1445 | 07933e08cff6c7d81345e02c03727032ccf522b6 | PR #1445 close duplicate issue | correctly archives duplicate #140 while preserving #133 as the surviving open conflict-frequency record; no findings | outstanding-issues and docs-link guards pass; docs-only diff | | 2026-07-30 | claude/organize-local-worktree-d22bc3 | 2f26a53b5aeb3df451cf7b1d04f80b07edf0d6fe | docs organisation: dated-record filing, docs index gaps, orientation maps | PR #1436 opened — 5 dated docs filed into docs/audit and docs/archive, root codex-cloud-review moved under docs/prompts, 17 docs README index gaps closed, root data/ documented in CLAUDE.md + codebase-index; no product code, schema or RAG surface touched | docs:check-links 1368 refs pass; docs:check-scripts 378 pass; docs:check-index OK; format:check whole-tree clean; verify:cheap 26 static gates + lint + typecheck pass, unit 4562 pass / 1 pre-existing Windows path-separator failure in tests/repo-hygiene.test.ts | | 2026-07-30 | PR-1436 | 9d8e081f3e7003d4f2210b00a7b7e54bf7ca2f0b | PR #1436 documentation organization and link repair | fixed stale no-driver wording and renumbered three union-collided issue records; no remaining findings | docs index, links, scripts, outstanding-issues, and ledger guards pass | | 2026-07-30 | cursor/ci-hygiene-gates-1bf5 | b660dbc5a10d7ca3da03541028017f0abc6b5bd3 | ci-hygiene-gates-merge-readiness | NOT READY: cancel-to-green behavior still allowed required PR CI to pass incorrectly; fixed at subsequent head 8f3283d00da274dee507a1b8e9b611321d1f35be | check:ci-scope; check:gitleaks-pinned; scope-classify PR files ui_changed=false; cancelled-as-neutral simulation exposed #095 | +| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | 11e7e8653c4742108feee4e9164ba066a92fc18a | ci-testing-review-capture | Captures #141, the one finding from the PR #1427 session that was never recorded: ui-phone-scroll.spec.ts:973 Services result anchor jumped on viewport shrink (run 30534158395 shard 1, :1133, 120 passed). Distinct from #127 and not fixed by #1427 - that head already carried #1427's runway poll and travel assertion, the journey is the Services result canvas not the document route, and the assertion is anchor stability across a resize not a chrome hide. Narrowing recorded: the sibling documentScrollTop assertion at :1137 did NOT fail, so the scroll position held while the element moved, ruling out the scroll-restoration class. Branch restarted from origin/main because #1427 merged as 040ce97 (verified by content not ancestry: shard matrix, dragScrollUntilHidden and the gate-count guard all present on main). NOTE for #127: its updated row shows the trace-based refutation of the short-drag hypothesis was itself misread (maxOffset 2753 read at a different trace moment than the failing drag; pre-runway value was 1153), so the root cause withdrawn during the #1427 session was in fact correct - a third-party refutation needs the same verification as a third-party fix claim. | verify:cheap PASS (Test Files 435 passed; Tests 4569 passed, 4 skipped, exit 0); check:outstanding-issues PASS (139 rows, unique ids, next-id=142 above highest); prettier --check . PASS. Docs-only: ui_changed should be false and the Chromium shards should skip. | +| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | e6c704c6c8cd989825adf30ecbffeb57344bfa56 | ci-testing-review-capture | SUPERSEDES the 11e7e865 capture row: the Services viewport-anchor flake is now #142, not #141. Main's #141 (design-sync --clinical-accent-strong token gap from PR #1443) landed while this branch was open, colliding on the same id. Merge conflict in docs/outstanding-issues.md resolved by rebuilding from origin/main and re-applying the Services capture as #142 with next-id=143. Content of the capture is unchanged. | check:outstanding-issues PASS (140 rows, 68 open, 72 archived, next-id=143); merge-tree was CONFLICTING on outstanding-issues only; conflict markers removed | +| 2026-07-30 | PR #1446 / claude/ci-testing-review-2l8klp | 8be4f703d5729b4aa10e73ee8fbc77e03f400b8b | ci-testing-review-capture | Withdraws an invalid inference from the earlier records for this PR, on a correct Codex finding. Those rows argued that because the sibling documentScrollTop assertion did not fail, the scroll position held and scroll-restoration causes were ruled out. Playwright aborts a test at the first failing expect, so once anchorTop threw, documentScrollTop NEVER EXECUTED - its absence from the output shows nothing. The #142 row now says so and the class is not ruled out. The capture itself stands: the Services viewport-anchor failure is real, intermittent on byte-identical code (pass/pass/fail/pass-on-rerun), and distinct from #127. Separately CodeRabbit flagged :973 vs :1133 as inconsistent and then withdrew it: :973 is the test declaration and :1133 the thrown assertion, both reported by Playwright, and declaration lines drift (898 / 973 / 1041 across three tree states) which is why the exact title is the durable identity. | check:outstanding-issues PASS (140 rows, unique ids, next-id=143). Lesson: reasoning from an assertion that never ran is the same verified-vs-assumed error this session already hit twice in the other direction. | | 2026-07-30 | claude/x3-rag-coverage-gate-qx9j7d (PR #1454, squashed as 102bb1f) | 102bb1f5edf09e666d1be5934ff5dfb2aa5abcf0 | X3/#086 evidence coverage gate extraction from rag.ts into rag-coverage-gate.ts | clean and landed — byte-identical move verified against pre-merge main, rag.ts 5030->4780, budget ratcheted to 4780, no back-edge, public re-export preserved; squash captured 100% of branch content, nothing orphaned | workflow:rag-lab, focused vitest 81/81, check:maintainability-budgets 4780/4780, check:rag:fixtures 36 golden, eval:rag:offline 567/567, typecheck, lint, check:knip, format:check, verify:cheap 4569 passed, npm test 4569 passed, verify:pr-local build+bundle-scan, post-merge npm test on main 4574 passed | +| 2026-07-30 | PR #1432 | 74adc5aa3f8a4dad659c7a40490288ef8efcb82e | Playwright browser preflight and phone-sheet focus repair | APPROVE after current-main sync: browser-project resolution fails closed, phone-sheet focus is stable, and no stale issue-ledger state remains. | 3 focused files 45 passed; phone-chrome dry-run; installed-lock parity; docs and ledger guards; formatting | diff --git a/docs/scripts-index.md b/docs/scripts-index.md index e2c848a67e..c1a0e5c815 100644 --- a/docs/scripts-index.md +++ b/docs/scripts-index.md @@ -16,22 +16,22 @@ migration has shipped (see `docs/maturity-backlog-workorders.md` L1). ## Runner & guard infrastructure [infra] -| Script | Role | -| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -| `run-heavy.mjs` | Acquires shared/exclusive cross-worktree leases (`test-run-lock.mjs`) so focused checks can overlap safely | -| `run-tsx.mjs`, `run-vitest.mjs`, `run-playwright.mjs`, `run-eval-safe.mjs` | Typed/test/e2e/eval entrypoint wrappers | -| `dev-free-port.mjs`, `ensure-local-server.mjs` | Project-stable localhost port selection + background server ensure | -| `check-node-engine.cjs`, `install-git-hooks.mjs`, `guard-push.mjs`, `guard-next-build.mjs` | Install/preflight guards | -| `ci-change-scope.mjs`, `ci-triage.mjs`, `pr-policy.mjs`, `pr-mergeability.mjs` | CI change classification + PR policy + conflict signal (self-tested via `check:ci-scope`/`check:ci-triage`/`check:pr-policy`/`check:pr-mergeability`) | -| `check-outstanding-issues.mjs`, `check-pr-mergeability-workflow.mjs` | Outstanding-issues ID/marker/no-driver guard + PR mergeability workflow contract | -| `check-installed-lock-parity.mjs`, `phone-chrome-plan.mjs`, `verify-phone-chrome.mjs` | Lock-trust preflight plus change-scoped phone contracts, ownership journeys, and smart full-UI escalation | -| `final-merge-audit.mjs` | Fail-closed local merge-tree audit; explicit provider mode adds PR/check/thread/tree/deployment proof | -| `child-process-result.mjs`, `cli-utils.ts`, `productivity-core.mjs` | Shared helpers | -| `test-focused.mjs`, `test-run-selection.mjs`, `test-cache-path.mjs`, `test-environment.mjs` | Backs `npm run test:focused` — change-scoped selection, cache pathing, env setup; fails closed for deleted files and test infrastructure | -| `primary-checkout-lease.mjs`, `test-run-lock.mjs`, `clean-worktree.mjs` | Cross-worktree lease arbitration for the primary checkout, plus worktree cleanup | -| `resolve-tsx-cli.mjs`, `register-server-only.mjs`, `enable-server-only-stub.mjs` | tsx CLI resolution and `server-only` import shims | -| `check-format-changed.mjs`, `check-base-freshness.mjs`, `check-local-presence.mjs` | Push-time helpers behind `guard-push.mjs`: changed-file formatting, stale-base and local-presence checks | -| `yaml-contract.mjs`, `sensitive-text.mjs`, `design-system-contract-utils.mjs` | Shared parsing/redaction/contract helpers used by the gates | +| Script | Role | +| ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| `run-heavy.mjs` | Acquires shared/exclusive cross-worktree leases (`test-run-lock.mjs`) so focused checks can overlap safely | +| `run-tsx.mjs`, `run-vitest.mjs`, `run-playwright.mjs`, `run-eval-safe.mjs` | Typed/test/e2e/eval entrypoint wrappers | +| `dev-free-port.mjs`, `ensure-local-server.mjs` | Project-stable localhost port selection + background server ensure | +| `check-node-engine.cjs`, `install-git-hooks.mjs`, `guard-push.mjs`, `guard-next-build.mjs` | Install/preflight guards | +| `ci-change-scope.mjs`, `ci-triage.mjs`, `pr-policy.mjs`, `pr-mergeability.mjs` | CI change classification + PR policy + conflict signal (self-tested via `check:ci-scope`/`check:ci-triage`/`check:pr-policy`/`check:pr-mergeability`) | +| `check-outstanding-issues.mjs`, `check-pr-mergeability-workflow.mjs` | Outstanding-issues ID/marker/no-driver guard + PR mergeability workflow contract | +| `check-installed-lock-parity.mjs`, `phone-chrome-plan.mjs`, `verify-phone-chrome.mjs`, `playwright-browser-preflight.mjs` | Lock-trust preflight, change-scoped phone contracts, and Playwright browser-binary preflight before build | +| `final-merge-audit.mjs` | Fail-closed local merge-tree audit; explicit provider mode adds PR/check/thread/tree/deployment proof | +| `child-process-result.mjs`, `cli-utils.ts`, `productivity-core.mjs` | Shared helpers | +| `test-focused.mjs`, `test-run-selection.mjs`, `test-cache-path.mjs`, `test-environment.mjs` | Backs `npm run test:focused` — change-scoped selection, cache pathing, env setup; fails closed for deleted files and test infrastructure | +| `primary-checkout-lease.mjs`, `test-run-lock.mjs`, `clean-worktree.mjs` | Cross-worktree lease arbitration for the primary checkout, plus worktree cleanup | +| `resolve-tsx-cli.mjs`, `register-server-only.mjs`, `enable-server-only-stub.mjs` | tsx CLI resolution and `server-only` import shims | +| `check-format-changed.mjs`, `check-base-freshness.mjs`, `check-local-presence.mjs` | Push-time helpers behind `guard-push.mjs`: changed-file formatting, stale-base and local-presence checks | +| `yaml-contract.mjs`, `sensitive-text.mjs`, `design-system-contract-utils.mjs` | Shared parsing/redaction/contract helpers used by the gates | ## Verification gates [live] diff --git a/docs/testing.md b/docs/testing.md index 9c2592c240..e8cecbb28f 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -53,7 +53,9 @@ Reference examples: `tests/icon-button.dom.test.tsx` (accessible-name contract), ## Playwright ownership -The repository runner exclusively builds and serves each Playwright production app. It selects a safe port, verifies `/api/local-project-id`, uses an isolated `.next-playwright/` build directory, replaces provider configuration with inert loopback values, and removes its server and output on success, failure, or signal. Playwright configuration never starts a server. The production boot guard permits this demo profile only when the output is isolated, provider mode is offline, credentials are absent, and the Supabase URL is the inert `127.0.0.1:1` target. +The repository runner exclusively builds and serves each Playwright production app. It selects a safe port, verifies `/api/local-project-id`, uses an isolated `.next-playwright/` build directory, replaces provider configuration with inert loopback values, and removes its server and output on success, failure, or signal. Playwright configuration never starts a server. The production boot guard permits this demo profile only when the output is isolated, provider mode is offline, credentials are absent, and the Supabase URL is the inert `127.0.0.1:1` target. Before acquiring the heavy lock or building, the runner preflights the Chromium (or requested Firefox/WebKit) executable — including the default `chrome-headless-shell` binary and any `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` override — and exits non-zero immediately when it is missing, so a launch-infra failure cannot be mistaken for product-test failures after a multi-minute build. + +When capturing Playwright or `verify:phone-chrome` output through a shell pipe (`cmd 2>&1 | tee …`), enable `set -o pipefail` (or avoid the pipe). Without it, bash reports the pipeline exit from `tee` (`0`) while the log still ends in `N failed` — a measurement artifact that previously looked like a green-when-broken gate (outstanding-issues #120). The Node runners themselves already propagate Playwright’s exit status. Blocking tests run with zero retries. CI publishes list, JUnit, and JSON reports. Failed-test classification parses JUnit test cases and uses exact spec/title matches; a job name is never enough to classify a failure as a known flake. diff --git a/scripts/phone-chrome-plan.mjs b/scripts/phone-chrome-plan.mjs index b5f15c0a75..5776e69298 100644 --- a/scripts/phone-chrome-plan.mjs +++ b/scripts/phone-chrome-plan.mjs @@ -19,8 +19,8 @@ const patterns = { docs: [/^docs\//, /^AGENTS\.md$/], infrastructure: [ /^package(?:-lock)?\.json$/, - /^scripts\/(?:check-installed-lock-parity|phone-chrome-plan|verify-phone-chrome)\.mjs$/, - /^tests\/(?:installed-lock-parity|verify-phone-chrome)\.test\.ts$/, + /^scripts\/(?:check-installed-lock-parity|phone-chrome-plan|playwright-browser-preflight|run-playwright|verify-phone-chrome)\.mjs$/, + /^tests\/(?:installed-lock-parity|playwright-browser-preflight|verify-phone-chrome)\.test\.ts$/, ], playwrightHelper: [/^tests\/playwright-(?:scroll|settlement)\.ts$/], dashboard: [ diff --git a/scripts/playwright-browser-preflight.mjs b/scripts/playwright-browser-preflight.mjs new file mode 100644 index 0000000000..63c829d29b --- /dev/null +++ b/scripts/playwright-browser-preflight.mjs @@ -0,0 +1,157 @@ +#!/usr/bin/env node +import { existsSync } from "node:fs"; +import path from "node:path"; +import { chromium, firefox, webkit } from "playwright"; + +const BROWSER_TYPES = { + chromium, + firefox, + webkit, +}; + +export const playwrightProjectNames = Object.freeze({ + chromium: "chromium", + chromiumMockups: "chromium-mockups", + chromiumArtifacts: "chromium-artifacts", + firefox: "firefox", + webkit: "webkit", +}); + +const DEFAULT_CONFIG_PROJECTS = Object.freeze({ + "playwright.config.ts": [ + playwrightProjectNames.chromium, + playwrightProjectNames.chromiumMockups, + playwrightProjectNames.firefox, + playwrightProjectNames.webkit, + ], + "playwright.visual.config.ts": [playwrightProjectNames.chromiumArtifacts], +}); + +const PROJECT_BROWSER_FAMILIES = Object.freeze({ + [playwrightProjectNames.chromium]: "chromium", + [playwrightProjectNames.chromiumMockups]: "chromium", + [playwrightProjectNames.chromiumArtifacts]: "chromium", + [playwrightProjectNames.firefox]: "firefox", + [playwrightProjectNames.webkit]: "webkit", +}); + +// Mirrors Playwright's chromium-headless-shell executable table for the +// platform directory exposed by chromium.executablePath(). Keep this explicit: +// Linux arm64 intentionally uses chrome-linux/headless_shell, unlike x64. +const CHROMIUM_HEADLESS_SHELL_LAYOUTS = Object.freeze({ + "chrome-linux64": ["chrome-headless-shell-linux64", "chrome-headless-shell"], + "chrome-linux": ["chrome-linux", "headless_shell"], + "chrome-mac-x64": ["chrome-headless-shell-mac-x64", "chrome-headless-shell"], + "chrome-mac-arm64": ["chrome-headless-shell-mac-arm64", "chrome-headless-shell"], + "chrome-win64": ["chrome-headless-shell-win64", "chrome-headless-shell.exe"], +}); + +/** + * Derive the default headless-shell binary Playwright launches for Chromium + * tests. `chromium.executablePath()` points at full Chrome for Testing; the + * default headless project uses chrome-headless-shell instead. + */ +export function defaultChromiumHeadlessShellPath(chromeExecutablePath = chromium.executablePath()) { + const normalized = chromeExecutablePath.replaceAll("\\", "/"); + const segments = normalized.split("/"); + const chromiumIndex = segments.findLastIndex((segment) => /^chromium-\d+$/.test(segment)); + const chromiumDirectory = segments[chromiumIndex]; + const platformDirectory = segments[chromiumIndex + 1]; + const shellLayout = CHROMIUM_HEADLESS_SHELL_LAYOUTS[platformDirectory]; + if (chromiumIndex < 0 || !chromiumDirectory || !shellLayout) return null; + const revision = chromiumDirectory.slice("chromium-".length); + const browsersRoot = segments.slice(0, chromiumIndex).join("/"); + return path.join(browsersRoot, `chromium_headless_shell-${revision}`, ...shellLayout); +} + +export function requestedPlaywrightBrowserProjects(args = []) { + const projects = new Set(); + let configPath = "playwright.config.ts"; + for (let index = 0; index < args.length; index += 1) { + const token = args[index]; + if (token === "--config" && args[index + 1]) { + configPath = args[index + 1]; + index += 1; + continue; + } + if (token.startsWith("--config=")) { + configPath = token.slice("--config=".length); + continue; + } + if (token === "--project" && args[index + 1]) { + projects.add(args[index + 1]); + index += 1; + continue; + } + if (token.startsWith("--project=")) projects.add(token.slice("--project=".length)); + } + if (projects.size === 0) { + const configName = path.basename(configPath); + return DEFAULT_CONFIG_PROJECTS[configName] ?? [`config:${configPath}`]; + } + return [...projects]; +} + +function browserFamilyForProject(project) { + return PROJECT_BROWSER_FAMILIES[project] ?? null; +} + +export function resolvePlaywrightBrowserExecutable(family, env = process.env) { + if (family === "chromium") { + const override = env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH?.trim(); + if (override) { + return { + family, + path: override, + source: "PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH", + }; + } + return { + family, + path: defaultChromiumHeadlessShellPath(), + source: "playwright chromium-headless-shell", + }; + } + return { + family, + path: BROWSER_TYPES[family].executablePath(), + source: `playwright ${family}`, + }; +} + +export function playwrightBrowserPreflight(args = [], env = process.env) { + const projects = requestedPlaywrightBrowserProjects(args); + const unsupportedProjects = projects.filter((project) => !browserFamilyForProject(project)); + const families = [...new Set(projects.map(browserFamilyForProject).filter(Boolean))]; + const missing = unsupportedProjects.map((project) => ({ + family: "unknown", + path: "", + source: `unmapped Playwright project ${project}`, + })); + for (const family of families) { + const resolved = resolvePlaywrightBrowserExecutable(family, env); + if (!resolved.path || !existsSync(resolved.path)) missing.push(resolved); + } + if (missing.length === 0) { + return { ok: true, projects, checked: families.map((family) => resolvePlaywrightBrowserExecutable(family, env)) }; + } + const details = missing.map((entry) => `- ${entry.family} (${entry.source}): ${entry.path}`).join("\n"); + return { + ok: false, + projects, + missing, + message: + `Playwright browser preflight failed before the production build.\n` + + `Missing executable(s):\n${details}\n` + + `Install matching browsers with \`npx playwright install\`, or set ` + + `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH to a launchable Chromium binary. ` + + `Do not treat a later "N failed" summary as a product regression when the browser binary is absent.`, + }; +} + +export function assertPlaywrightBrowsersReady(args = [], env = process.env, { stderr = console.error } = {}) { + const result = playwrightBrowserPreflight(args, env); + if (result.ok) return result; + stderr(result.message); + process.exit(1); +} diff --git a/scripts/run-playwright.mjs b/scripts/run-playwright.mjs index a8efd0ac8f..784ed5b32a 100644 --- a/scripts/run-playwright.mjs +++ b/scripts/run-playwright.mjs @@ -6,6 +6,7 @@ import net from "node:net"; import path from "node:path"; import { fileURLToPath } from "node:url"; import { childProcessExitCode, childProcessFailureSummary } from "./child-process-result.mjs"; +import { assertPlaywrightBrowsersReady } from "./playwright-browser-preflight.mjs"; import { offlineTestEnvironment } from "./test-environment.mjs"; import { acquireHeavyRunLock } from "./test-run-lock.mjs"; import { @@ -45,6 +46,12 @@ const mockupProjectRequested = argument === "--project=chromium-mockups" || (argument === "--project" && playwrightArgs[index + 1] === "chromium-mockups"), ); + +// Fail loud on missing browser binaries before the heavy lock or production build. +// Otherwise launch failures surface as "N failed" product tests and are easy to misread +// when a caller pipes output without `pipefail` (outstanding-issues #120). +assertPlaywrightBrowsersReady(playwrightArgs); + const runId = `${process.pid}-${Date.now()}`; const relativeRunRoot = `.next-playwright/${runId}`; const absoluteRunRoot = path.join(projectRoot, relativeRunRoot); diff --git a/scripts/verify-phone-chrome.mjs b/scripts/verify-phone-chrome.mjs index 9d5e59bfe3..251b80c270 100644 --- a/scripts/verify-phone-chrome.mjs +++ b/scripts/verify-phone-chrome.mjs @@ -1,5 +1,6 @@ #!/usr/bin/env node import { execFileSync, spawnSync } from "node:child_process"; +import { pathToFileURL } from "node:url"; import { childProcessExitCode } from "./child-process-result.mjs"; import { phoneChromePlan, renderPhoneChromeCommand } from "./phone-chrome-plan.mjs"; @@ -46,29 +47,45 @@ function changedFiles(explicitFiles) { return JSON.parse(result).files; } -function run(command) { +export function runPhoneChromeCommand(command, { spawn = spawnSync } = {}) { const executable = process.platform === "win32" && command.executable === "npm" ? "cmd.exe" : command.executable; const args = executable === "cmd.exe" ? ["/d", "/s", "/c", [command.executable, ...command.args].join(" ")] : command.args; - const result = spawnSync(executable, args, { stdio: "inherit" }); + const result = spawn(executable, args, { stdio: "inherit" }); return childProcessExitCode(result); } -const options = parseArgs(process.argv.slice(2)); -const plan = phoneChromePlan(changedFiles(options.files), { fullMode: options.fullMode }); -console.log(`Phone chrome inputs: ${plan.files.length ? plan.files.join(", ") : "(none detected)"}`); -console.log(`Full UI policy: ${plan.fullMode} (${plan.fullSelected ? "selected" : "not selected"})`); -for (const note of plan.notes) console.log(`Note: ${note}`); - -if (options.dryRun) { - console.log("\nPhone chrome verification plan (dry run):"); - for (const stage of plan.stages) console.log(`- [${stage.id}] ${renderPhoneChromeCommand(stage.command)}`); - process.exit(0); +export function runPhoneChromeStages( + stages, + { runCommand = runPhoneChromeCommand, exit = process.exit, log = console.log } = {}, +) { + for (const stage of stages) { + log(`\n[phone-chrome:${stage.id}] ${stage.label}`); + log(`> ${renderPhoneChromeCommand(stage.command)}`); + const exitCode = runCommand(stage.command); + if (exitCode !== 0) { + // Announce on stderr so a piped caller without pipefail still has an + // unambiguous failure line in the captured log (outstanding-issues #120). + console.error(`[phone-chrome] stage "${stage.id}" failed; exiting with code ${exitCode}`); + exit(exitCode); + return exitCode; + } + } + return 0; } -for (const stage of plan.stages) { - console.log(`\n[phone-chrome:${stage.id}] ${stage.label}`); - console.log(`> ${renderPhoneChromeCommand(stage.command)}`); - const exitCode = run(stage.command); - if (exitCode !== 0) process.exit(exitCode); +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + const options = parseArgs(process.argv.slice(2)); + const plan = phoneChromePlan(changedFiles(options.files), { fullMode: options.fullMode }); + console.log(`Phone chrome inputs: ${plan.files.length ? plan.files.join(", ") : "(none detected)"}`); + console.log(`Full UI policy: ${plan.fullMode} (${plan.fullSelected ? "selected" : "not selected"})`); + for (const note of plan.notes) console.log(`Note: ${note}`); + + if (options.dryRun) { + console.log("\nPhone chrome verification plan (dry run):"); + for (const stage of plan.stages) console.log(`- [${stage.id}] ${renderPhoneChromeCommand(stage.command)}`); + process.exit(0); + } + + runPhoneChromeStages(plan.stages); } diff --git a/tests/playwright-browser-preflight.test.ts b/tests/playwright-browser-preflight.test.ts new file mode 100644 index 0000000000..92758022a7 --- /dev/null +++ b/tests/playwright-browser-preflight.test.ts @@ -0,0 +1,121 @@ +import { describe, expect, it } from "vitest"; +import { + defaultChromiumHeadlessShellPath, + playwrightBrowserPreflight, + playwrightProjectNames, + requestedPlaywrightBrowserProjects, + resolvePlaywrightBrowserExecutable, +} from "../scripts/playwright-browser-preflight.mjs"; + +describe("playwright browser preflight", () => { + it("defaults to every configured project when no project is requested", () => { + const configuredProjects = [ + playwrightProjectNames.chromium, + playwrightProjectNames.chromiumMockups, + playwrightProjectNames.firefox, + playwrightProjectNames.webkit, + ]; + expect(requestedPlaywrightBrowserProjects([])).toEqual(configuredProjects); + expect(requestedPlaywrightBrowserProjects(["tests/ui-smoke.spec.ts"])).toEqual(configuredProjects); + }); + + it("uses the selected config's projects when no project flag is present", () => { + expect(requestedPlaywrightBrowserProjects(["--config=playwright.visual.config.ts"])).toEqual([ + playwrightProjectNames.chromiumArtifacts, + ]); + expect(requestedPlaywrightBrowserProjects(["--config", "playwright.visual.config.ts"])).toEqual([ + playwrightProjectNames.chromiumArtifacts, + ]); + }); + + it("fails closed for an unmapped config without explicit projects", () => { + const result = playwrightBrowserPreflight(["--config=future.config.ts"]); + expect(result.ok).toBe(false); + expect(result.missing?.[0]?.source).toContain("unmapped Playwright project config:future.config.ts"); + }); + + it("collects explicit --project flags", () => { + expect(requestedPlaywrightBrowserProjects(["--project=firefox", "--project", "webkit"])).toEqual([ + "firefox", + "webkit", + ]); + }); + + it("derives the headless-shell binary Playwright launches by default", () => { + expect( + defaultChromiumHeadlessShellPath( + "/home/ubuntu/.cache/ms-playwright/chromium-1234/chrome-linux64/chrome", + )?.replaceAll("\\", "/"), + ).toBe( + "/home/ubuntu/.cache/ms-playwright/chromium_headless_shell-1234/chrome-headless-shell-linux64/chrome-headless-shell", + ); + expect( + defaultChromiumHeadlessShellPath( + "/home/ubuntu/.cache/ms-playwright/chromium-1234/chrome-linux/chrome", + )?.replaceAll("\\", "/"), + ).toBe("/home/ubuntu/.cache/ms-playwright/chromium_headless_shell-1234/chrome-linux/headless_shell"); + expect( + defaultChromiumHeadlessShellPath( + "/Users/test/Library/Caches/ms-playwright/chromium-1234/chrome-mac-arm64/Chromium.app/Contents/MacOS/Chromium", + )?.replaceAll("\\", "/"), + ).toBe( + "/Users/test/Library/Caches/ms-playwright/chromium_headless_shell-1234/chrome-headless-shell-mac-arm64/chrome-headless-shell", + ); + expect( + defaultChromiumHeadlessShellPath( + "C:/Users/test/AppData/Local/ms-playwright/chromium-1234/chrome-win64/chrome.exe", + )?.replaceAll("\\", "/"), + ).toBe( + "C:/Users/test/AppData/Local/ms-playwright/chromium_headless_shell-1234/chrome-headless-shell-win64/chrome-headless-shell.exe", + ); + expect(defaultChromiumHeadlessShellPath("/opt/custom/chrome")).toBeNull(); + }); + + it("honours PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH for Chromium projects", () => { + const resolved = resolvePlaywrightBrowserExecutable("chromium", { + NODE_ENV: "test", + PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH: "/tmp/custom-chrome", + }); + expect(resolved).toEqual({ + family: "chromium", + path: "/tmp/custom-chrome", + source: "PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH", + }); + }); + + it("fails closed when the required Chromium binary is missing", () => { + const result = playwrightBrowserPreflight(["--project=chromium"], { + NODE_ENV: "test", + PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH: `/tmp/missing-chrome-${Date.now()}`, + }); + expect(result.ok).toBe(false); + expect(result.message).toContain("Playwright browser preflight failed before the production build"); + expect(result.message).toContain("Missing executable"); + expect(result.missing?.[0]?.family).toBe("chromium"); + }); + + it("fails closed for a project missing from the shared project map", () => { + const result = playwrightBrowserPreflight(["--project=future-browser"]); + expect(result.ok).toBe(false); + expect(result.missing?.[0]?.source).toContain("unmapped Playwright project future-browser"); + }); + + it("passes when an override path exists", async () => { + const { mkdtempSync, writeFileSync, rmSync } = await import("node:fs"); + const { tmpdir } = await import("node:os"); + const { join } = await import("node:path"); + const directory = mkdtempSync(join(tmpdir(), "pw-preflight-")); + const binary = join(directory, "chrome"); + try { + writeFileSync(binary, ""); + const result = playwrightBrowserPreflight(["--project=chromium-mockups"], { + NODE_ENV: "test", + PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH: binary, + }); + expect(result.ok).toBe(true); + expect(result.checked?.[0]?.path).toBe(binary); + } finally { + rmSync(directory, { recursive: true, force: true }); + } + }); +}); diff --git a/tests/test-runner-safety.test.ts b/tests/test-runner-safety.test.ts index aa02246e4e..740eb832bc 100644 --- a/tests/test-runner-safety.test.ts +++ b/tests/test-runner-safety.test.ts @@ -605,6 +605,7 @@ describe("provider-safe test environment", () => { it("builds and starts an isolated production server for Playwright", () => { const runner = readFileSync(new URL("../scripts/run-playwright.mjs", import.meta.url), "utf8"); + const preflight = readFileSync(new URL("../scripts/playwright-browser-preflight.mjs", import.meta.url), "utf8"); const baseUrl = readFileSync(new URL("../scripts/playwright-base-url.ts", import.meta.url), "utf8"); const ragRunner = readFileSync(new URL("../scripts/eval-rag-offline.mjs", import.meta.url), "utf8"); const playwrightConfig = readFileSync(new URL("../playwright.config.ts", import.meta.url), "utf8"); @@ -622,6 +623,16 @@ describe("provider-safe test environment", () => { expect(runner).toContain("body === null || body.includes(missingErrorComponentsNeedle)"); expect(runner).not.toContain("if (!body || body.includes(missingErrorComponentsNeedle))"); expect(runner).not.toContain("supabase.co"); + // Missing browser binaries must fail before the heavy lock / production build (#120). + expect(runner).toContain("assertPlaywrightBrowsersReady(playwrightArgs);"); + expect(runner.indexOf("assertPlaywrightBrowsersReady(playwrightArgs);")).toBeLessThan( + runner.indexOf("lock = acquireHeavyRunLock("), + ); + expect(runner.indexOf("assertPlaywrightBrowsersReady(playwrightArgs);")).toBeLessThan( + runner.indexOf("console.log(`Building isolated production Playwright app"), + ); + expect(preflight).toContain("chromium_headless_shell"); + expect(preflight).toContain("PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH"); expect(packageJson.scripts["test:e2e:pr"]).toContain('--grep-invert "@quarantine|@mockup"'); expect(packageJson.scripts["test:e2e:regression"]).toContain('--grep-invert "@critical|@quarantine|@mockup"'); expect(baseUrl.indexOf("if (!allowEnsure)")).toBeLessThan(baseUrl.indexOf("findExistingLocalProjectUrl();")); diff --git a/tests/ui-phone-scroll.spec.ts b/tests/ui-phone-scroll.spec.ts index c34488c5f9..8818494cb1 100644 --- a/tests/ui-phone-scroll.spec.ts +++ b/tests/ui-phone-scroll.spec.ts @@ -718,7 +718,16 @@ for (const phoneOwner of ["browser document", "standalone PWA main"] as const) { await expect(page.getByTestId("document-section-sheet")).toHaveCount(0); await expect(page.locator("body")).not.toHaveCSS("overflow", "hidden"); await expect(sectionTrigger).toBeFocused(); - await sectionTrigger.evaluate((element) => element.blur()); + // Move focus into the reader instead of blurring to . Sheet focus + // restoration intentionally retries after 50 ms when focus fell through + // to ; a bare blur races that retry in production builds and can + // re-pin the header after this assertion has already continued. + await content.evaluate((element) => { + element.tabIndex = -1; + element.focus({ preventScroll: true }); + }); + await expect(content).toBeFocused(); + await expect(sectionTrigger).not.toBeFocused(); // Reduced motion removes the transition but retains the out-of-flow // geometry. Prove another hide/reveal cycle cannot displace the reader. diff --git a/tests/verify-phone-chrome.test.ts b/tests/verify-phone-chrome.test.ts index 5e6cf3a9f2..3f89fe5854 100644 --- a/tests/verify-phone-chrome.test.ts +++ b/tests/verify-phone-chrome.test.ts @@ -1,5 +1,6 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { phoneChromePlan } from "../scripts/phone-chrome-plan.mjs"; +import { runPhoneChromeStages } from "../scripts/verify-phone-chrome.mjs"; const ids = (files: string[], fullMode: "auto" | "always" | "never" = "auto") => phoneChromePlan(files, { fullMode }).stages.map((stage) => stage.id); @@ -60,3 +61,25 @@ describe("phoneChromePlan", () => { }, ); }); + +describe("runPhoneChromeStages", () => { + it("aborts on the first non-zero stage exit and never continues", () => { + const exit = vi.fn() as unknown as typeof process.exit; + const runCommand = vi.fn().mockReturnValueOnce(0).mockReturnValueOnce(1).mockReturnValueOnce(0); + const stages = [ + { id: "lock-parity", label: "ok", command: { executable: "npm", args: ["run", "a"] } }, + { + id: "focused-browser", + label: "browser", + command: { executable: "node", args: ["scripts/run-playwright.mjs"] }, + }, + { id: "full-ui", label: "full", command: { executable: "npm", args: ["run", "verify:ui"] } }, + ]; + + const code = runPhoneChromeStages(stages, { runCommand, exit, log: () => undefined }); + + expect(code).toBe(1); + expect(exit).toHaveBeenCalledWith(1); + expect(runCommand).toHaveBeenCalledTimes(2); + }); +});