- Notifications
You must be signed in to change notification settings - Fork 0
docs(issues): record the review-bot budget exhaustion and two local-environment findings#2263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Uh oh!
There was an error while loading. Please reload this page.
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
3af177d
docs(issues): record the review-bot budget exhaustion and two local-e…
BigSimmo 4e47ff5
Merge branch 'main' into claude/ledger-env-findings
BigSimmo 5ca1553
fix(ledger): reissue stale environment updates
BigSimmo 15e0f7f
Merge branch 'main' into claude/ledger-env-findings
BigSimmo 7e0a885
Merge branch 'main' into claude/ledger-env-findings
BigSimmo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
10 changes: 10 additions & 0 deletions
10 docs/outstanding-issues-inbox/0c9671c5-34ed-4c53-b896-3a61108816ba.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "0c9671c5-34ed-4c53-b896-3a61108816ba", | ||
| "createdOn": "2026-08-22", | ||
| "action": "cancel", | ||
| "payload": { | ||
| "requestId": "416afc07-6d2a-479d-a10d-c323bf037767", | ||
| "reason": "Stale optimistic-concurrency fingerprint. Superseded by cab0fdc2-b88c-45bb-98ac-3e4acffcddd2, which rereads #6GW95D from current main, retains the fleet-wide safety evidence, and re-scopes its priority and summary." | ||
| } | ||
| } |
12 changes: 12 additions & 0 deletions
12 docs/outstanding-issues-inbox/416afc07-6d2a-479d-a10d-c323bf037767.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "416afc07-6d2a-479d-a10d-c323bf037767", | ||
| "createdOn": "2026-08-21", | ||
| "action": "update", | ||
| "payload": { | ||
| "id": "#6GW95D", | ||
| "detail": "PREMISE CORRECTED 2026-08-22: the disk pressure this row was written against is gone. It records a '51%-full Dev Drive'; D: has since been expanded and now measures 80 GB total, 22 GB used, 59 GB available -- 27% used. So worktree removal is no longer urgent on space grounds, and this row should not be actioned as a capacity emergency. WHAT IS STILL TRUE, AND IS A DIFFERENT PROBLEM: .claude/worktrees/ has accumulated directories that are no longer git worktrees at all. Counted 2026-08-22: 30 directories, of which 12 are registered in `git worktree list` and belong to live sessions, and 18 are ORPHANS that git no longer knows about. Sixteen of the eighteen are completely empty and two hold a single entry. They cost almost no disk -- which is exactly why nothing reclaims them -- but they make `ls .claude/worktrees` useless for telling live work from debris, and that ambiguity is what preceded two separate incidents in which a cleanup sweep deleted an in-use worktree mid-session. ONE OF THEM CANNOT BE REMOVED YET: phase-4-index-restoration-b0f4ea is empty and unregistered, but both `rmdir` and PowerShell `Remove-Item` fail with 'being used by another process' -- a live session still holds it as its working directory. It will clear when that session exits; no process was killed to force it. NEXT: separate the two concerns. (a) Reclaim the 18 orphan directories, which is safe to script because 'not in `git worktree list`' plus 'no files' is a strong test -- but the script must skip a directory it cannot remove rather than escalating. (b) Re-scope or close the capacity half of this row, since 59 GB free removes its original justification. STOP: never delete a directory that IS registered in `git worktree list`, even if it looks idle -- that is precisely the mistake that destroyed live work twice.", | ||
| "source": "measured 2026-08-22 on the Windows workstation (df -h /d; git worktree list vs .claude/worktrees listing); session Phase 5 close-out", | ||
| "baseRowFingerprint": "2edd7af71e7e182cf74874a1de2ebb134df9043b945ad47319f7aacbc0fdca2c" | ||
BigSimmo marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| } | ||
| } | ||
12 changes: 12 additions & 0 deletions
12 docs/outstanding-issues-inbox/61ed61bb-e3f1-406d-9842-5068bf30dfcc.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "61ed61bb-e3f1-406d-9842-5068bf30dfcc", | ||
| "createdOn": "2026-08-21", | ||
| "action": "update", | ||
| "payload": { | ||
| "id": "#CCZ4HB", | ||
| "detail": "CONFIRMED IN THE WILD 2026-08-22, with numbers. This row predicted that PR churn would exhaust the review-bot budget and leave PRs landing unreviewed. That happened today, observed directly on PR #2258: CodeRabbit posted a Fair-Usage notice reading 'You have reached a temporary PR review limit', 'Next review available in: 46 minutes', '89 included PR review attempts over the past 7 days set your current allowance at 1 review per hour', and -- the part that makes this a billing decision rather than a pacing one -- 'Your organization has reached its usage spending cap.' CONSEQUENCE OBSERVED, NOT INFERRED: PR #2261, the serialized reconcile that closes #316, was opened inside that window and is expected to land with no automated review. It was verified locally against check:outstanding-issues and check:ledger-write-discipline, so this was judged acceptable for a ledger-only diff -- but the general case is not acceptable, because merging to main now deploys database migrations to the live clinical project automatically (D4 ON, AGENTS.md). CONTRIBUTING CAUSE, OWNED: the 2026-08-22 Phase 5 session alone opened three PRs (#2250, #2258, #2261). Two of the three were unavoidable by contract -- a serialized reconcile must be its own fresh-base branch, and an immutable review record cannot be edited in place -- which is worth noting because it means the AGENTS.md 'PR bundling' rule cannot fully solve this on its own. TWO OPTIONS, NOT ONE: (a) raise or remove the CodeRabbit spending cap in the billing tab, which is the only option that restores review coverage on a repo that auto-deploys to production; or (b) accept reduced automated review and lean on verify:pr-local plus the domain gates, in which case that acceptance should be written down here rather than left implicit. RECOMMENDATION: (a). STOP: do not change billing or spending-cap settings without the owner deciding; this row is a decision request.", | ||
| "source": "CodeRabbit Fair-Usage notice on PR #2258, 2026-08-22; PRs #2250/#2258/#2261; AGENTS.md 'PR bundling' and 'Supabase project safety'", | ||
| "baseRowFingerprint": "bfc39e062f42860bc22db9ef0a32c955317e8d8dd73f75d35a3f8de95933eb0e" | ||
| } | ||
| } |
12 changes: 12 additions & 0 deletions
12 docs/outstanding-issues-inbox/a01b19c4-f58a-4d57-915a-f9e62849ca6c.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "a01b19c4-f58a-4d57-915a-f9e62849ca6c", | ||
| "createdOn": "2026-08-22", | ||
| "action": "update", | ||
| "payload": { | ||
| "id": "#CCZ4HB", | ||
| "detail": "UPDATE 2026-08-21 (second session, three PRs later): STILL HAPPENING, and now measured on consecutive PRs rather than one. CodeRabbit skipped review on PR #2252, #2255 and #2256 — every PR opened in that session — reporting: 'Your 88 included PR review attempts over the past 7 days set your current allowance at 1 review per hour' and 'Your organization has reached its usage spending cap.' Countdown observed shrinking across the session (48 minutes -> 32 -> 27), i.e. the hourly allowance was being consumed as fast as it refilled. Two refinements to the row below. (1) The 2026-08-18 reading was 101 reviews/7 days; this one is 88 attempts/7 days, so the trailing window is falling but the spending cap is the binding constraint now, not the rate limit — raising the cap is therefore the lever, and it is an account/billing decision the repo cannot gate its way out of. (2) Draft PRs are skipped outright ('Review skipped. Draft detected'), so the repo's own default of opening PRs as drafts guarantees no bot review until someone undrafts — and undrafting mid-CI also cancels the in-flight run and reds the required aggregate (observed on #2255 and #2256). That interaction is worth stating in AGENTS.md: the draft default, the review-bot budget, and the cancel-in-progress churn are the same problem seen from three sides. Next unchanged: decide between a churn gate, raising the cap, or accepting unreviewed merges deliberately. PRIOR RECORD: CodeRabbit on PR #2113: '101 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour. Your organization has reached its usage spending cap.' The Codex connector reported its own usage limit on the same PR. Net effect: #2113 received ZERO automated review, and so will subsequent PRs until the cap resets or credits are added. AGENTS.md 'PR bundling' already measured the CI half of this cost on 2026-07-30 (437 PR-triggered runs over ~3 days, ~40% cancelled mid-run, ~12 Production-UI-hours burned on runs that never completed). This is the second bill for the same behaviour and the more dangerous one, because CI waste is money while missing review is undetected defects — and the PRs most likely to need review are the ones landing during a churn spike. The bundling rule exists as prose in AGENTS.md and is evidently not binding; the newtask skill also asks the question in prose. Decide whether it gets a gate. Note the repo has already learned this lesson once in a different area: .claude/hooks/pr-handoff-stop.sh states in its own header that 'prose rules in AGENTS.md have not held, a denied tool call does.' Next: decide between (a) a push/PR-creation gate that refuses a new branch when an open PR of the same scope exists, (b) raising the bot spending cap, or (c) accepting unreviewed merges deliberately rather than by accident. Stop rule: do not weaken any required check to compensate for missing bot review. UPDATE 2026-08-22: the problem remains confirmed on PR #2263: CodeRabbit reported a temporary review limit, a one-review-per-hour allowance, and an exhausted organisation spending cap. This is an owner billing/process decision, not a check failure that can be bypassed. The current decision remains: raise the cap to restore automated review coverage, or explicitly document an alternative review policy. Do not weaken required checks to compensate.", | ||
| "source": "CodeRabbit comments on PRs #2252, #2255, #2256 (2026-08-21); prior evidence PR #2113 (2026-08-18); CodeRabbit review-limit notice on PR #2263 (2026-08-22)", | ||
| "baseRowFingerprint": "623b041eaa1803db51012e06d60423af9b2d1886339141fc1cc58cb678268498" | ||
| } | ||
| } |
14 changes: 14 additions & 0 deletions
14 docs/outstanding-issues-inbox/cab0fdc2-b88c-45bb-98ac-3e4acffcddd2.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "cab0fdc2-b88c-45bb-98ac-3e4acffcddd2", | ||
| "createdOn": "2026-08-22", | ||
| "action": "update", | ||
| "payload": { | ||
| "id": "#6GW95D", | ||
| "pri": "P3", | ||
| "summary": "Fleet-wide worktree inventory and safe orphan cleanup remain, but the Dev Drive capacity emergency is resolved", | ||
| "detail": "UPDATE 2026-08-21 (later the same day): the scope is wider than .claude/worktrees, and the previous update's 69 is itself a floor. An accidental `npx vitest run` issued from C:/Users/joshs instead of the repo root enumerated 253 copies of just TWO test files across the home directory, i.e. 253 separate checkouts of this repository, spread over at least six distinct roots: .codex/worktrees, .gemini/antigravity/worktrees, .copilot/repos/copilot-worktrees, .local/share/opencode/worktree, Documents/Codex/<date>/<chat>/work, and an AppData/Local/pnpm store projects directory (which also contained its own .worktrees). Temp copies under AppData/Local/Temp/guard-push-format-* were still present too. So every AI tool in the fleet keeps its own worktree root and none of them is covered by a sweep scoped to .claude/worktrees. Most had no node_modules (the run failed them with 'Cannot find package'), so raw disk is likely below 253 x 0.89 GB, but the count is the point: any recursive tool run from the home directory now walks all of them. Re-scope this row from 'nine landed worktrees' to a fleet-wide inventory across all six roots before pruning, and keep the #XCAX01 safety preconditions — the 2026-08-21 data-loss incident is why this is not a one-line delete. PRIOR RECORD: UPDATE 2026-08-21 (measured on this machine): the figure recorded here is badly understated and moving the wrong way. git worktree list reported 69 registered worktrees under .claude/worktrees, not nine. At the ~0.89 GB / 51,735-file node_modules per worktree measured in #6SMMB4, even partial installs put this well beyond the 4.5 GB recorded, on a Dev Drive noted as 51% full. SAFETY, learned the hard way on 2026-08-21: a sweep removed an in-use worktree during this very session and destroyed its uncommitted work - see the separate row filed for that. Re-measure actual on-disk size, then prune, but confirm each worktree is landed, holds no uncommitted or unpushed work, AND is not currently checked out by a live agent session before removing it. UPDATE 2026-08-22: the capacity premise is now false. D: has been expanded to 80 GB total, with 22 GB used and 59 GB available (27% used), so this is no longer a space emergency. The worktree safety/inventory evidence remains relevant: 253 checkouts across at least six roots, and the #XCAX01 preconditions remain mandatory. A same-day .claude/worktrees check found 30 directories: 12 registered to live sessions and 18 unregistered orphans. Sixteen orphans were empty and two held one entry; one empty unregistered directory could not be removed because a live process held it. NEXT: keep this as a low-priority fleet-inventory and safe-orphan-cleanup task. Any future cleanup must cover every root, default to report-only, skip a directory when liveness or cleanliness cannot be proved, and never delete a registered worktree.", | ||
| "source": "Accidental home-directory vitest run, 2026-08-21: 253 test-file copies across six worktree roots; Dev Drive / worktree inventory re-measured 2026-08-22", | ||
| "baseRowFingerprint": "7874ab76abbd500d29125d16fa21bd6a2ebc422c47871609152d4fe786c418fb" | ||
| } | ||
| } |
14 changes: 14 additions & 0 deletions
14 docs/outstanding-issues-inbox/d83ca697-3bb1-431d-b740-9691b5a20b39.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "d83ca697-3bb1-431d-b740-9691b5a20b39", | ||
| "createdOn": "2026-08-21", | ||
| "action": "add", | ||
| "payload": { | ||
| "pri": "P3", | ||
| "type": "rec", | ||
| "summary": "Local process-spawn starvation silently makes every git push take minutes; the one-command diagnostic is not written down anywhere", | ||
| "detail": "MEASURED TWICE, THREE DAYS APART. On 2026-08-20 a session recorded `gh --version` at 1m37s and concluded gh itself was broken, which made every git push take 5-10 minutes because .githooks/pre-push -> guard-push.mjs invokes gh several times per push. On 2026-08-22 the same binary measured 0.201s, 0.262s and 0.278s on three consecutive runs. IT WAS NEVER GH. The decisive comparison is a binary with nothing to do with gh: `node --version` was 17s on 2026-08-20 and 0.080s on 2026-08-22, a ~200x change. `git --version` measures 0.069s. gh is unchanged between the two readings -- same 2.97.0, same C:\\Program Files\\GitHub CLI\\gh.exe on local disk (not a network path, not a shim), same 41,775,416 bytes, and ZERO extensions installed, so the usual suspects (startup update check, extension probing, slow path) are all excluded. Windows Defender real-time protection was ON in BOTH readings, so Defender alone is not sufficient to cause it either; exclusions could not be read without administrator rights. WHAT ACTUALLY CHANGED: the machine had been rebooted about an hour before the fast reading. The condition is machine-wide process-spawn starvation that accumulates with long uptime and many concurrent agent sessions, and a reboot clears it. The precise mechanism cannot be pinned down retroactively once the state is gone, and that is stated rather than guessed at. IT IS ALREADY REBUILDING: one hour after the reboot the machine showed 49 node processes, 31 claude processes, 548 processes total, and 6.9 GB free of 32 GB. So this recurs rather than being a one-off, and the next session to hit it will again mistake it for a broken tool. NEXT (cheap, docs-only): write the diagnostic down where a session will find it -- `time node --version` is the whole test. Sub-second means the machine is healthy and a slow push is something else; multiple seconds means spawn starvation, and the fix is to close sessions or reboot, NOT to debug gh, git, or the pre-push guard. Natural home is the traps section of AGENTS.md or docs/testing.md. STOP: do not change Windows security settings or add a Defender exclusion on this evidence -- the two readings show Defender is not the discriminating variable, and a security-settings change needs the owner regardless.", | ||
| "source": "measured 2026-08-20 and 2026-08-22 on the Windows workstation; session 2026-08-22 Phase 5 close-out", | ||
| "issueUlid": "01M0K1TH6KVV83VA74C3B7DMQR" | ||
| } | ||
| } |
10 changes: 10 additions & 0 deletions
10 docs/outstanding-issues-inbox/f048c9c7-fc23-4361-8cb8-38bee74e3405.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| { | ||
| "version": 2, | ||
| "id": "f048c9c7-fc23-4361-8cb8-38bee74e3405", | ||
| "createdOn": "2026-08-22", | ||
| "action": "cancel", | ||
| "payload": { | ||
| "requestId": "61ed61bb-e3f1-406d-9842-5068bf30dfcc", | ||
| "reason": "Stale optimistic-concurrency fingerprint. Superseded by a01b19c4-f58a-4d57-915a-f9e62849ca6c, reread from current main so reconciliation does not stop at the stale request." | ||
| } | ||
| } |
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.