Skip to content

chore(perf): refresh the Lighthouse baseline from a pinned-browser CI run - #2134

Merged
BigSimmo merged 6 commits into
mainfrom
claude/lighthouse-baseline-refresh
Aug 18, 2026
Merged

chore(perf): refresh the Lighthouse baseline from a pinned-browser CI run#2134
BigSimmo merged 6 commits into
mainfrom
claude/lighthouse-baseline-refresh

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

  • The committed baseline carried updatedAt: 2026-08-13T12:02, so every PR this week was graded against five-day-old measurements. That is what blocked fix(home): give the shared mode home the full hero design on phones #2112 all afternoon: desktop /therapy-compass measured ~956 ms on PR runners against a 793 ms baseline, tripping the +20%/+100 ms tolerance by roughly four milliseconds — passing twice and failing three times on identical code.
  • Measured on main at 533989f (the fix(home): give the shared mode home the full hero design on phones #2112 merge commit) through the dispatch-only refresh job, on the pinned Chromium 151 the gate itself uses. Nothing here is a local measurement: the baseline is a browser- and runner-specific artefact and the workflow refuses to produce it anywhere else.
  • The drift is two-directional, which is the argument for refreshing rather than raising a ceiling:
CellOldNewΔ
desktop-documents-search783866+83
desktop-therapy-compass793849+56
desktop-dsm846849+3
desktop-root868822−46
desktop-forms876756−120
mobile-* (all five)−98 to −113

Most routes got faster since 13 August. A stale baseline is not merely lenient — it had stopped describing the application in either direction, and the two desktop regressions were invisible underneath that.

  • What this deliberately does not answer: ~56–83 ms of genuine growth on two desktop routes, neither of which changed its own code in that window. That is shared-chunk growth across the intervening merges, and it deserves its own investigation rather than being absorbed silently here. Refreshing the baseline makes it measurable from a known point; it does not excuse it.

Verification

  • npx vitest run tests/check-lighthouse-budget.test.tsTests 64 passed (64)
  • npx prettier --check lighthouse-budget.jsonAll matched files use Prettier code style!
  • Diff audited line by line: only metric values and updatedAt change.routes, strategies, tolerance, enforce: true and the pinned lighthouseVersion: 12.8.2 are untouched — confirmed by filtering the diff for any non-metric line.
  • Single chromeVersion across all ten cells (Chrome/151.0.0.0), which the refresh job asserts and which keeps the baseline internally consistent.

Not run: browser, build, or provider-backed gates. This diff is one data file.

Risk and rollout

  • Risk: low, but it is a gate-sensitivity change and worth reading as such. Raising desktop-documents-search and desktop-therapy-compass means a future regression on those two routes must exceed a higher bar before the budget notices. That is the correct trade against a baseline that no longer described reality — but it is the reason the two increases are called out individually above rather than buried in an aggregate.
  • The five mobile cells and desktop-forms move down, tightening those budgets.
  • Rollback: revert the commit; the previous baseline returns exactly.
  • Provider or production effects: None. No application code, migration, or deployment behaviour is touched.

Notes

  • Two measurements were taken, an hour apart, on different main commits: desktop-therapy-compass read 864 ms on 5ef065c and 849 ms on 533989f. The 15 ms spread between them is runner scatter, and it is roughly the margin by which fix(home): give the shared mode home the full hero design on phones #2112 kept failing — useful context for how tight that cell sits.
  • This also corrects a figure I had been quoting: the drift on that route is ~56–71 ms, not the ~163 ms implied by comparing PR-runner readings against the old baseline. The rest was scatter on the PR runners.
  • Related: fix(ci): stop cancelling main's CI so the base branch is actually verified #2124 stopped main's CI from cancelling itself. It does not make Lighthouse budget run on main — that job is path-scoped and skips on most pushes, which is the mechanism that let this baseline go five days without anyone noticing. Worth a follow-up: either schedule the refresh, or alert when the baseline exceeds an age.

Generated by Claude Code

… run
The committed baseline carried `updatedAt: 2026-08-13T12:02`, so every PR this
week was graded against five-day-old measurements. That is what blocked #2112
all afternoon: `desktop /therapy-compass` measured ~956ms on PR runners against
a 793ms baseline, tripping the +20%/+100ms tolerance by roughly four
milliseconds, passing twice and failing three times on identical code.
Measured on main at 533989f (the #2112 merge commit) via the dispatch-only
refresh job, on the pinned Chromium 151 the gate itself uses. Nothing here is a
local measurement: the baseline is a browser- and runner-specific artefact and
the workflow refuses to produce it anywhere else.
The drift is real but two-directional, which is the argument for refreshing
rather than raising a ceiling:
desktop-documents-search 783 -> 866 (+83)
desktop-therapy-compass 793 -> 849 (+56)
desktop-dsm 846 -> 849 (+3)
desktop-root 868 -> 822 (-46)
desktop-forms 876 -> 756 (-120)
mobile-* (all five) (-98 to -113)
Most routes got FASTER since 13 August. A stale baseline is not merely lenient;
it had stopped describing the application in either direction, and the two
desktop regressions were invisible underneath that.
The remaining question this does not answer: ~56-83ms of genuine growth on two
desktop routes, neither of which changed its own code in that window. That is
shared-chunk growth across the intervening merges and deserves its own
investigation rather than being absorbed silently here.
Only metric values and `updatedAt` change; routes, strategies, tolerances,
enforcement and the pinned Lighthouse version are untouched.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in:57 minutes

Limit details: You’ve used the included review currently available. Your 102 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e011c0b0-7bf6-4ed9-991b-67a2db7f46a0

📥 Commits

Reviewing files that changed from the base of the PR and between 32f45b2 and 6bffd19.

📒 Files selected for processing (2)
  • docs/branch-review-records/e56e0fe8053c2af58ddd7cfa2149ac3cae54fcf65a803a76b3efe20eb2328933.record.md
  • lighthouse-budget.json

Comment @coderabbitai help to get the list of available commands.

@supabase

supabaseBot commented Aug 18, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@BigSimmo
BigSimmo merged commit baec777 into mainAug 18, 2026
26 checks passed
@BigSimmo
BigSimmo deleted the claude/lighthouse-baseline-refresh branch August 18, 2026 18:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@BigSimmo@claude