docs(issues): record the intermittent mobile-root CLS that failed Lighthouse budget - #2232
Conversation
…hthouse budget Orphaned by the #2204 merge: this record was committed while the Lighthouse re-run was still in flight and deliberately held back so the push would not cancel it. The re-run answered the question and the PR merged in the same window, so the record never reached main. Re-applied here on the merged base. The same Lighthouse job ran three times on identical code across two heads of #2204 whose only difference was deleting one JSON file under docs/outstanding-issues-inbox/ -- no source, asset, route or style change: head c8b7bcd PASS head 09ff450 FAIL mobile-root cls +0.207 vs baseline (max +0.02) head 09ff450 PASS (re-run of the same commit) 0.223 against a 0.016 baseline. Every other cell was comfortably inside tolerance on the failing run, including the mobile-documents-search LCP and TBT cells an earlier, withdrawn record had wrongly accused from an ungradeable local measurement. Two problems in one row because they share a cause. A required check inside pr-required flips pass/fail on a diff that cannot influence it, so it can block any PR at random -- and the grader already re-confirms an out-of-budget cell twice and takes the majority, so the instability is wider than one spike. And 0.223 is a 10x overshoot of the budget, not a wobble: something on the mobile home route sometimes shifts layout badly during load. A phone user would feel it when it happens. The record says explicitly not to widen the tolerance or re-baseline to absorb 0.223, and points at the retained artifact (9447841108) whose per-run reports name the shifting elements. Verification: check:ledger-write-discipline and check:outstanding-issues pass against the merged base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKbNfTZM1vzUTRsuAS4Mxv
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in:7 minutes Limit details: You’ve used the included review currently available. Your 87 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
Comment |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:6971771d4e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "summary": "Lighthouse budget mobile-root CLS is intermittent: 0.223 vs 0.016 baseline on one run, ~0.000 on the next, same code", | ||
| "detail": "MEASURED ON CI, NOT INFERRED. PR #2204 ran the same Lighthouse budget job on two consecutive heads whose only difference was deleting one JSON file under docs/outstanding-issues-inbox/ -- no source, asset, route or style change, nothing that can affect layout. Head c8b7bcdd PASSED. Head 09ff450c FAILED with exactly one metric out of tolerance: 'mobile-root cls +0.207 vs baseline (max +0.02)', measuring 0.223 against a 0.016 baseline (job 96780922258, run 32485431566). Every other cell was comfortably inside tolerance on the failing run: mobile-root LCP 2261 vs 2253, TBT 312 vs 296; mobile-documents-search LCP 2274 vs 2272, TBT 394 vs 348; desktop-documents-search LCP 804 vs 866; desktop-root LCP 815 vs 822. TWO SEPARATE PROBLEMS. (1) GATE RELIABILITY: a required check in pr-required flips pass/fail on a diff that cannot influence it, so it can block any PR at random. Note the grader already re-confirms an out-of-budget cell twice and takes the majority, so this survived that mechanism -- the instability is wider than a single spike. (2) PROBABLE REAL DEFECT ON THE MOBILE HOME ROUTE: 0.223 is not a marginal overshoot of a 0.02 tolerance, it is a large layout shift that sometimes occurs during load on '/' at mobile viewport and sometimes does not. Likely candidates are a webfont swap, an image or media element without reserved dimensions, or late-hydrating chrome (the phone composer/header reserve is a known-sensitive area per docs/search-chrome-behaviour.md). A user on a phone would feel this when it happens. Worth reproducing directly rather than only through the budget gate. DO NOT respond by widening the CLS tolerance or refreshing the baseline to absorb 0.223 -- that would encode an intermittent user-visible shift as acceptable. Diagnose which element shifts first. The Lighthouse artifact for the failing run is retained (artifact 9447841108) and contains the per-run reports, which identify the shifting elements. RELATED CAUTION FOR TRIAGE: main's CI runs do not exercise this job (path-scoped, perf scope only), so a green main run is not evidence the gate passes there -- the ci-triage bot correctly classified it 'not baselined'.", |
There was a problem hiding this comment.
When the inbox is reconciled, this add is applied alongside the pre-existing docs/outstanding-issues-inbox/5f472de7-797f-49ba-8ba5-b5b6d4a7ad7a.json, which already tracks the same bistable mobile-root CLS values (0.016/0.223), CI-only reproduction, random Lighthouse failures, artifact investigation, and prohibition on widening the budget. Because the requests have distinct ULIDs and add/add conflicts are not deduplicated, applyRequestBatch accepts both and creates two canonical issues, splitting status and risking duplicate remediation. Cancel this request or merge its additional evidence into the earlier one; a two-request reconciliation simulation is the smallest check that exposes the duplicate rows.
AGENTS.md reference: AGENTS.md:L1060-L1066
Useful? React with 👍 / 👎.
Summary
One inbox record, orphaned by the #2204 merge. It was committed while the
Lighthouse budgetre-run was still in flight and deliberately held back so pushing would not cancel that run. The re-run answered the question and #2204 merged in the same window, so the record never reachedmain. Re-applied here on the merged base.Why it is worth keeping. The same Lighthouse job ran three times on identical code, across two heads of #2204 whose only difference was deleting one JSON file under
docs/outstanding-issues-inbox/— no source, asset, route or style change:c8b7bcdd09ff450cmobile-root cls +0.207 vs baseline (max +0.02)09ff450c(re-run, same commit)0.223 measured against a 0.016 baseline. Every other cell was comfortably inside tolerance on the failing run:
mobile-rootLCP 2261 vs 2253 and TBT 312 vs 296,mobile-documents-searchLCP 2274 vs 2272,desktop-rootLCP 815 vs 822.That last detail matters. An earlier version of this record accused
mobile-documents-searchLCP and TBT, inferred from a local run the grader had explicitly refused to score for browser drift (Chromium 141 against a baseline captured on 151). CI showed those cells were fine, so that record was withdrawn during #2204. This one replaces it with the measurement CI actually produced.Two problems, recorded as one row because they share a cause:
pr-requiredflips pass/fail on a diff that cannot influence it, so it can block any PR at random. The grader already re-confirms an out-of-budget cell twice and takes the majority, so the instability is wider than a single spike.docs/search-chrome-behaviour.md. A phone user would feel it.The record says explicitly not to widen the CLS tolerance or re-baseline to absorb 0.223, since that would encode a user-visible shift as acceptable, and points at the retained artifact (
9447841108) whose per-run reports name the shifting elements. It also captures the triage caution thatmainnever exercises this job at all (path-scoped to perf changes), so a greenmainrun is not evidence either way.Verification
npm run check:ledger-write-discipline—passed for 5db9d1817675..HEADnpm run check:outstanding-issues—411 rows (61 open, 350 archived) … no ids deleted from base 5db9d1817675Single added JSON file under
docs/outstanding-issues-inbox/. No source, test, config, schema or asset change, so no behavioural gate applies. Runnpm run issues:reconcilefrom a dedicated ledger branch after this lands.Risk and rollout
git revertof the single commit.Generated by Claude Code