docs(issues): close #290 as not-reproducible over three full ui-smoke runs, capture the browser-preflight traps (#312) - #1884
Conversation
…flight traps #290 archived. Could not reproduce on current main (3dc1f37): full ui-smoke --project=chromium run three times gave 97 passed / 0 failed / 0 flaky each (291 test executions, zero failures), and the named reliable reproducer run in isolation with --repeat-each=5 gave 5 passed at 0.98-1.2s against the row's documented ~11.5s failure signature. Recorded as evidence, not proof, and the archive says so: the row measured 2/5 and 3/5 full-run failure rates, so one clean run proves little and three puts it at roughly 6-22% under the old behaviour. Resolved by drift — main has moved several hundred commits past the row's base 9ab3b73 and nothing here touched src/. Reopen rather than re-file if it recurs. #312 (new) captures two things that cost two failed runs first: - check:playwright-browser-revision reporting "OK (managed-or-unconstrained)" means no browser root is FORCED, not that browsers exist. The container had chromium-1194 against Playwright 1.62.1's required 1234. - Installing the matching revision is a viable first option that archived #255 does not mention: npx playwright install chromium took ~1 minute, and --project=chromium skips the unused firefox/webkit requirement because the preflight honours project scoping. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017paT42ZVMf8jaLtkjFxdy5
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in:46 minutes Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BigSimmo
commented
Aug 12, 2026
Independent verification from a second session (worked #290 in parallel before finding this PR), with one correction to the archive's causal story. Measured in a fresh cloud container (Chromium headless-shell 1234 installed per this PR's #312 note):
Correction: the Summary attributes the recovery to drift ("Resolved by drift… main has advanced several hundred commits"). But the bad SHA itself — the exact commit where #290 measured 2/5 and 3/5 failures — passes 30/30 executions in this container, including the full-suite shape and under induced load. So drift is not what resolved it: the failure was specific to the original machine/environment (its load profile or toolchain), not to the commit window. The archive row's "reopen on recurrence" instruction still stands; suggest a future reader treat the Not-reproducible remains the right disposition, and archiving #290 is the right call — no second PR needed from my side. One residual mechanism note for whoever meets a recurrence: Generated by Claude Code |
Uh oh!
There was an error while loading. Please reload this page.
Summary
#290archived — theui-smokephone scroll-hide flake could not be reproduced on currentmain(3dc1f37). Fulltests/ui-smoke.spec.ts --project=chromiumrun three consecutive times: 97 passed / 0 failed / 0 flaky each (3.0m, 2.8m, 2.8m) — 291 test executions, zero failures. All four named members passed every time. Separately, the row's named "reliable reproducer" was run in isolation with--repeat-each=5: 5 passed, each in 0.98–1.2s, against the row's documented failure signature of an ~11.5s timeout versus ~1.4s when passing. They sit in the passing regime, not marginally inside it.Recorded as evidence, not proof, and the archive entry says so. The row measured failure rates of 2/5 and 3/5 on full-suite runs, so a single clean run had roughly even odds even under the old behaviour; three consecutive clean runs put that at roughly 6–22%. Meaningful, not conclusive, for an intermittent failure. The archive tells a future reader to reopen rather than re-file, and keeps the reproducer name and the ~11.5s-vs-~1.4s timing signature as the fastest way to recognise a recurrence.
Resolved by drift, not by anything in this PR.
mainhas advanced several hundred commits past the row's base commit9ab3b73a(9 August) and nothing in this session touchedsrc/.#312(new) captures two browser-proof traps that cost two failed runs before the measurement could start:check:playwright-browser-revisionreportingOK (managed-or-unconstrained)means no browser root is forced — it does not assert any browser exists. I read it as a green light forverify:ui; the container actually hadchromium-1194while Playwright 1.62.1 requireschromium_headless_shell-1234, withfirefox-1538andwebkit-2336absent entirely.#255does not mention:npx playwright install chromiumfetched 114.7 MiB in about a minute and made local Chromium proof possible. Also recorded:PLAYWRIGHT_SKIP_BROWSER_DOWNLOADwas empty in this container despite the environment note implying otherwise, and--project=chromiumskips the unused Firefox/WebKit requirement becausescripts/playwright-browser-preflight.mjs:127-152honours project scoping — so two ~100MB downloads are avoidable.Verification
npm run verify:pr-localBrowser evidence, captured to files and read from the real process exit code rather than through a pipe:
Docs-only diff, so the router skipped lint, typecheck, the unit suite and the build. No
src/change, so no UI verification is owed beyond theui-smokeevidence above — which is itself the point of the PR.Risk and rollout
#290is archived while the flake is merely dormant — mitigated by recording the probability arithmetic explicitly, keeping the reproducer name and timing signature in the archive, and instructing a reopen rather than a re-file.git revertthis single commit;#290returns to the open table with its original text intact.Notes
The
--project=chromiumscoping detail is worth keeping in mind for any future local browser proof in a cloud session: the preflight demands every project's browser by default, so an unscoped run insists on Firefox and WebKit even for a Chromium-only spec.Generated by Claude Code