test(codex-cloud): budget the shell-policy case for Windows Git Bash spawns - #2255
Conversation
…spawns "writes managed shell policy behaviorally and preserves unrelated Codex config" failed in a full `npm run test` on a Windows workstation while passing in isolation there, and passing everywhere on Linux. It is not a state failure. The test spawns `bash scripts/setup-codex-cloud.sh` twice, which is cheap on Linux and expensive on Windows, where every spawn goes through Git Bash. Measured on that workstation running this file alone: 24.96s against the 30s default in vitest.config.mts — 83% of the budget with nothing else competing. A full run puts four workers on the machine at once, and it tips over. The sibling case immediately below already carries an explicit 120s budget for exactly this reason (it measures 112.9s on the same machine). This one was missed. Give it the same budget and record why, so the next person reading it does not have to re-derive the Windows cost. No assertion changes: the test does the same work and checks the same things. Only the time it is allowed to take moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JEzkroXdtL9kYo8rW5eZh2
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in:27 minutes Limit details: You’ve used the included review currently available. Your 88 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
Comment |
Uh oh!
There was an error while loading. Please reload this page.
… was wrong Four merge-safe inbox requests; none touches the canonical ledger. Corrects my own earlier record. The request filed in #2252 reasoned that failures 1 and 2 were deterministic state assertions and therefore real Windows-only defects rather than load noise. That was wrong for failure 1: the assertions are fixed, but the runtime is not, because the test spawns Git Bash twice. It is cancelled and replaced by one record carrying the resolved status of all five — two fixed (#2255, #2256), two unreproducible with a legibility fix (#2252), and hosted-migration-role-guard still genuinely unexplained, with the untracked-file hypothesis and the diagnostic to capture if it recurs. Updates #CCZ4HB with the second measurement of the review-bot budget: every PR opened this session was skipped, the countdown shrank across it, and the binding constraint has moved from the rate limit to the organisation spending cap — a billing decision no repo gate can substitute for. It also names the interaction worth writing down: the draft-by-default rule guarantees no bot review until someone undrafts, and undrafting mid-CI cancels the run and reds the required aggregate. Draft default, review budget and cancel-in-progress churn are one problem seen from three sides. Updates #6GW95D with a wider scope than it records. A stray `npx vitest` from the home directory enumerated 253 checkouts of this repository across six distinct roots — .codex, .gemini, .copilot, opencode, Documents/Codex, and a pnpm store with its own .worktrees. A sweep scoped to .claude/worktrees covers none of them. The #XCAX01 safety preconditions still apply before any pruning. Both updates restate the prior text verbatim rather than replacing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JEzkroXdtL9kYo8rW5eZh2
Summary
codex-cloud-setup› "writes managed shell policy behaviorally and preserves unrelated Codex config" an explicit120_000ms budget, matching the sibling case directly below it, and record why in a comment.This closes out the first of the five failures reported from a full
npm run teston a Windows workstation. It is not the state failure it looked like.What the Windows evidence showed
Re-running the two deterministic cases on that machine in isolation, both passed — but the timings are the finding:
The failing case spawns
bash scripts/setup-codex-cloud.shtwice. That is cheap on Linux and expensive on Windows, where every spawn goes through Git Bash. 24.96 s against the 30 stestTimeoutinvitest.config.mtsis 83% of the budget with nothing else running. A fullnpm run testputs four workers (maxWorkers: 4) on the machine at once, and it tips over — which is exactly why it failed there in the full suite, passed there in isolation, and passes everywhere on Linux.The sibling case immediately below already carries
}, 120_000)for the same reason, and measures 112.9 s on that machine. This one was simply missed when that budget was added.Correcting the prior analysis
PR #2252 recorded these two cases as deterministic state assertions, and reasoned that a Windows-only failure in either would therefore be a real Windows-only defect rather than load noise. That reasoning was wrong for this case: the assertions are deterministic, but the runtime is not, because of the subprocess cost. The conclusion it led to — "prioritise item 2, it suggests real repo drift" — was also wrong.
validateRepository()is happy on both platforms.hosted-migration-role-guard› "accepts the current repository state" also passes now at 1.55 s, nowhere near its budget. Its original failure is therefore still unexplained: 1.55 s would have to degrade roughly twentyfold under load to time out, which is a stretch, and nothing in this PR addresses it. It may have been a transient condition on that machine — a stray untracked file under a guarded path would do it, sincerepositoryEntries()grades untracked files — that has since cleared. Left open rather than assumed fixed.Verification
npx vitest run tests/codex-cloud-setup.test.ts—Tests 34 passed (34)npm run lint— passed (4371 input files)npm run typecheck— passed (4371 input files)npm run format— clean;tests/codex-cloud-setup.test.ts (unchanged)Verification not run: npm run test(full suite) — the full offline suite was run green on this content's parent in PR #2252 (696 passed (696)files,7770 passed | 4 skipped); this diff changes one test's time budget and no assertion or source behaviour, so it cannot alter any other file's result.UI verification not run:no production UI, routing, styling, or browser behaviour in the diff.Risk and rollout
it()call plus a comment. No assertion changes, no source changes, no behaviour change — the test does the same work and checks the same things; only the time it is allowed to take moves. The risk of raising a timeout is that a genuine hang takes longer to surface; bounded here by the same 120 s the neighbouring case already uses.git revertthe single commit.Notes
Four of the five originally reported failures remain open:
hosted-migration-role-guard› "accepts the current repository state" — passes now at 1.55 s; original cause unexplained, see above.http-readiness› "honours the overall deadline when every response remains open" — untested on Windows so far; a deadline shape, so the same load sensitivity is plausible.document-viewer-page-virtualizationcases — PR test(document-viewer): name the cause when render-ahead is never scheduled, and record the five unreproducible Windows failures #2252 made these fail legibly if they recur, which is what a re-run on that machine would now reveal.Worth running on the Windows workstation when convenient, since the timings are the diagnostic:
Generated by Claude Code