Skip to content

docs: preserve the review-coverage decision and the auto-fixer collision finding - #2259

Merged
BigSimmo merged 9 commits into
mainfrom
claude/preserve-review-coverage-docs
Aug 21, 2026
Merged

docs: preserve the review-coverage decision and the auto-fixer collision finding#2259
BigSimmo merged 9 commits into
mainfrom
claude/preserve-review-coverage-docs

Conversation

@BigSimmo

@BigSimmoBigSimmo commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • docs/decisions/ccz4hb-review-coverage.md — the decision document for ledger #CCZ4HB (PR churn has exhausted the review-bot budget, so pull requests are now landing with no automated review at all). Measured from local git history only, with an explicit method-and-limits table: 285 of the last 1,190 merged PRs (23.9%) changed only documentation, and 190 (16.0%) changed nothing but this repo's own ledger and issues records, while CodeRabbit refills at 24 reviews/day against a measured mean of 25.4 merged PRs/day over 31 days — demand exceeds supply structurally, counting only merged PRs and only one review each. Recommends, in order: exclude documentation-only PRs via .coderabbit.yamlpath_filters; make the existing bookkeeping-bundling rule mechanical with a PreToolUse hook modelled on pr-handoff-stop.sh; then size vendor spend against the residual. It rejects the "same scope" branch gate on evidence — 87.4% of merged PRs share at least one file with another merged within a day, so such a gate would refuse roughly seven in ten legitimate branches. Three open questions need the user and are listed in section 6.
  • docs/outstanding-issues-inbox/7c0f035c-2b63-47c2-b5a6-6df0118fdb9f.json — the inbox request for the auto-fixer collision (P2, rec): two independent auto-fixers race on the same PR review comment, and only one is documented. On fix(hooks): recognise a relative core.hooksPath so the push guard self-disables #2249 a single Codex review comment had more than one eligible responder — this repo's codex-autofix-review-comments.yml, and an app-level "Autofix pull requests" watcher that also instructed a live session to do the identical work. Two complete fixes were built independently; one was discarded. docs/agents-guide.md line 44 already assigns "Primary PR code-review + automatic resolve" to Codex, whose workflow carries trusted-bot login gating, a per-PR dedup marker, one repair pass per PR lifetime, and a skip-codex-review opt-out; the app-level watcher has none of those and the string appears in neither the tooling map nor the repository. Sibling of the ledger item recording the same failure between two human-driven sessions.

Both artefacts existed only in volatile locations — the decision document had been written solely into an agent worktree under .claude/worktrees/, which routine cleanup wipes — and would have been lost. They are bundled because they are one investigation: why automated review is not happening, and why two systems answered one comment.

The canonical docs/outstanding-issues.md table is deliberately not touched. Inbox requests travel with ordinary branches; only issues:reconcile edits the table.

Corrections made during review

Two Codex review findings were confirmed against the repository and fixed; both threads are resolved.

  • The inbox request originally carried the wrong payload (0dd8fe98…json). It queued the five Windows unit-test failures at cdfcbaccd, which origin/main already recorded in docs/outstanding-issues-inbox/67bf71cf-…json with newer Linux evidence showing they do not reproduce and the suite is green — since reconciled by docs(issues): reconcile 19 queued ledger requests into the canonical ledger #2260/docs(issues): reconcile 19 queued ledger requests and close #316 #2261 into applied/. Reconciling both would have created a second, stale row asserting live breakage, and the collision finding named in this PR's title was absent from the diff entirely. Replaced via npm run issues:add in e8442c1e. The replacement request is grounded in local git history rather than narrative: 687b166d and c9d990aa edit the same hook on the same branch under two different author identities, b9414fa0 and 94b6c291 are two separate merges of main into that one branch minutes apart, and 828106a5 merges the branch into itself to reconcile the divergence.
  • The decision document equated share-of-merged-PRs with share-of-review-credits. Its own §2.1 already conceded that CodeRabbit bills per push and that push counts are unrecoverable from a squash-merged history, and PRs merged after exhaustion consumed no credit at all. Six passages nonetheless asserted a credit figure derived from the PR share. All six now report merge volume, which is what was measured, and mark the credit conversion as an unverified estimate to be sized against the observed refill (91da3920). The recommendation and its order are unchanged; only the precision of the sizing depended on the conversion.

Verification

  • npm run verify:pr-local- failed: (none); completed check:runtime, check:installed-lock-parity, format:changed, sitemap:check, docs:check-index, docs:check-inventory, docs:check-scripts, docs:check-links, check:branch-review-ledger, check:outstanding-issues, check:ledger-write-discipline.
  • npm run check:ledger-write-disciplineLedger write discipline passed for 5ab74333cc77..HEAD.
  • npm run check:outstanding-issuesLedger inbox check passed: 2 pending request(s), 504 applied. and Outstanding-issues guard passed: 431 rows (72 open, 359 archived), unique display and durable ids, collision-free allocation enabled … no ids deleted from base 5ab74333cc77.
  • npm run docs:check-linksdocs link check passed: 2159 repo path references resolve.
  • npm run docs:check-scriptsdocs script-ref check passed: 558 npm-run reference(s) resolve to real scripts.
  • npm run docs:check-inventoryDocs inventory current: 255 script files, 258 npm scripts.
  • npx prettier --check on both added files — All matched files use Prettier code style!

This is a documentation-only diff adding two new files and modifying none. scripts/pr-policy.mjs classifies it clinicalRisk: false, operationalRisk: false, ragRanking: false, ui: false. No executable code, configuration, workflow, or generated artefact is touched, so Build, Unit coverage, Production UI and the domain jobs are correctly skipped by CI scope.

UI verification not run: no UI, routing, styling, reduced-motion, or forced-colors behaviour is touched.

Risk and rollout

  • Risk: None to runtime. Two new documentation files; nothing existing is modified, and no code path reads either of them. The inbox request is inert until a deliberately serialized issues:reconcile run applies it.
  • Rollback: revert the commits; both files are additions.
  • Provider or production effects: None.
  • RAG impact: none.

Notes

  • The recommendations inside the decision document are proposals, not changes — nothing in .coderabbit.yaml or any hook is altered here. Adopting them needs the three answers in its section 6.
  • #CCZ4HB's framing was re-checked against origin/main before writing: no gate addressing it has been added since it was filed, so the premise still holds.
  • The display id #V1ENZC, used in the original version of this description, came from the lost original request. Display ids are allocated at reconcile time, so the restored request will receive a fresh one.

…ion finding
Both artefacts existed only in volatile locations and would have been lost to an
ordinary cleanup. Bundled because they are the same investigation: why automated
review is not happening, and why two systems answered one review comment.
- docs/decisions/ccz4hb-review-coverage.md — the decision document for ledger
#CCZ4HB (PR churn has exhausted the review-bot budget, so PRs are landing with
no automated review). Measured from local git history only, with an explicit
method-and-limits table: 285 of the last 1,190 merged PRs (23.9%) changed only
documentation and 190 (16.0%) changed nothing but this repo's own ledger and
issues records, while CodeRabbit refills at 24 reviews/day against a measured
mean of 25.4 merged PRs/day over 31 days. Recommends, in order: exclude
documentation-only PRs via .coderabbit.yaml path_filters, make the existing
bookkeeping-bundling rule mechanical, then size vendor spend against the
residual. Rejects the "same scope" branch gate on evidence — 87.4% of merged
PRs share a file with another merged within a day, so it would refuse roughly
seven in ten legitimate branches. Three open questions need the user and are
listed in section 6. It had been written only into an agent worktree under
.claude/worktrees/, which is wiped by routine cleanup.
- docs/outstanding-issues-inbox/0dd8fe98… — the inbox request for #V1ENZC (P2
rec): two independent auto-fixers race on the same PR review comment, and only
one is documented. On #2249 a single Codex comment had three eligible
responders — the repo's codex-autofix-review-comments.yml, the app-level
"Autofix pull requests" watcher (which fixed and resolved it), and a live
session the same watcher instructed to do the identical work. Two complete
fixes were built; one was discarded. docs/agents-guide.md already assigns
automatic resolve to Codex, whose workflow carries trusted-bot gating, a per-PR
dedup marker, one repair pass per PR lifetime and a skip label; the app-level
watcher has none of these and is in neither the map nor the repo. Sibling of
#292, which is the same failure between human sessions.
The canonical docs/outstanding-issues.md table is deliberately NOT touched: inbox
requests travel with ordinary branches and only issues:reconcile edits the table.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@supabase

supabaseBot commented Aug 21, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in:21 minutes

Limit details: You’ve used the included review currently available. Your 89 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: dd2ae48b-6220-4f57-a205-6d0fe32f6ff3

📥 Commits

Reviewing files that changed from the base of the PR and between 5ab7433 and c3b1fc9.

📒 Files selected for processing (2)
  • docs/decisions/ccz4hb-review-coverage.md
  • docs/outstanding-issues-inbox/0dd8fe98-c2c5-4180-abd6-eab0622171f7.json

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:a611c7af7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment threaddocs/outstanding-issues-inbox/0dd8fe98-c2c5-4180-abd6-eab0622171f7.json Outdated
Comment threaddocs/decisions/ccz4hb-review-coverage.md Outdated
@BigSimmo
BigSimmo enabled auto-merge (squash) August 21, 2026 20:44
BigSimmoand others added 7 commits August 22, 2026 04:49
The inbox request added by a611c7a carried the wrong payload. It queued the
five Windows unit-test failures at cdfcbac, which origin/main already records
in docs/outstanding-issues-inbox/67bf71cf-25cb-40d9-a8a2-bbf993bf6b29.json with
newer Linux evidence showing they do not reproduce and the suite is green.
Reconciling both would have created a second, stale row asserting live breakage.
Worse, the auto-fixer collision finding named in this PR's title and body was
absent from the diff entirely, so the artefact this PR exists to preserve would
have been lost anyway.
Replace the payload with the intended request, grounded in local git history
rather than narrative: 687b166 and c9d990a edit the same hook on the same
branch from two different identities, b9414fa and 94b6c29 are two separate
merges of main into that one branch minutes apart, and 828106a merges the
branch into itself to reconcile the divergence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApB8NBygQn9cosxCQ8omk9
Codex review on #2259 objected that share-of-merged-PRs cannot be converted
into share-of-review-credits, and the objection holds. Section 2.1 already
concedes that CodeRabbit bills per push and that push counts are unrecoverable
from a squash-merged history, yet the summary, the 2.3 heading, the 2.3 body,
the Step 1 recommendation, the expected-effect paragraph and the .coderabbit.yaml
sketch all still asserted a credit figure derived from the PR share. PRs that
merged after exhaustion consumed no credit at all.
Restate each as what was actually measured - merge volume - and mark the credit
conversion as an unverified estimate to be sized against the observed refill.
The direction of the recommendation is unchanged: the underlying reasoning does
not depend on the conversion, only the precision of its sizing did.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApB8NBygQn9cosxCQ8omk9
@BigSimmo
BigSimmo merged commit e73d99b into mainAug 21, 2026
40 checks passed
@BigSimmo
BigSimmo deleted the claude/preserve-review-coverage-docs branch August 21, 2026 21:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@BigSimmo@claude