Uh oh!
There was an error while loading. Please reload this page.
test(cli): bind-probe the e2e serve port and name a lost race - #12523
Conversation
The three draw sites in packages/cli/test drew a port blind with Math.random() under a docblock claiming a run "never contends with another agent's dev server on this host". Several agents share one container here and a measured full-suite run went 1 failed / 2101 passed on `Port 49402 is already in use`, clean on an isolated re-run. Unify on one draw in test/helpers/serve-process.ts, make it a real bind probe (listen 0.0.0.0:0, read the assigned port, close, spawn), and say in the docblock what that does and does not guarantee -- it stops drawing ports already HELD for a neighbour's lifetime and narrows the window to the close-to-spawn gap; it is still TOCTOU. Make the residual loss legible: portContentionError() turns `serve exited 1 before "Server is ready"` into a failure that names port contention and the port, at all three spawn sites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjujZN219uFzBhSYfMykCd
A transcript of the measured stderr cannot notice that the source it was copied from has been reworded, so the pin reads the live template out of src/commands/serve.ts and asserts the detector fires on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjujZN219uFzBhSYfMykCd
📓 Docs Drift Check8 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 23 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 06ea8bcf997e0d5a2c221644be1cb6895e6b62e0 && git checkout 06ea8bcf997e0d5a2c221644be1cb6895e6b62e0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 14b114553c3e7c61672e3d25a1d15036f7561e27 8f8c4654fee0ee136d550358e739fc4dc48b294f && git checkout -B drift-repro 14b114553c3e7c61672e3d25a1d15036f7561e27 && git merge --no-ff 8f8c4654fee0ee136d550358e739fc4dc48b294f
node scripts/docs-audit/affected-docs.mjs --json 14b114553c3e7c61672e3d25a1d15036f7561e27 |
Uh oh!
There was an error while loading. Please reload this page.
Fixes#12441
Three
packages/cli/testsites drew theos serveport blind withMath.random(), under a docblock claiming a run "never contends with another agent's dev server on this host". Several agents share one container in this fleet, and a measured full-suite run went 1 failed / 2101 passed on✗ Port 49402 is already in use, clean on an isolated re-run. Test-only change.What landed
① One draw, and it is a real bind probe.
reservePort()intest/helpers/serve-process.tslistens on0.0.0.0:0, reads the port the kernel assigned, closes, and hands it over. The two overlapping inline ranges are gone —serve-node-env-production-default(41000-60000) andserve-app-anchored-optional-import(40000-60000) could collide with each other under--maxWorkers > 1, so three independent draws was itself half the defect.randomPort()survives asString(reservePort())for the 8 other files in this directory that pass a port straight into an argv list; renaming it would be pure churn across files this card does not own.② The docblock states the residual race instead of denying it. It says what the mechanism buys — ports already held for a neighbour's whole lifetime are off the table, and the window narrows from "the whole run" to "one close-to-spawn gap" — and that it is still TOCTOU. It also states the one property that is worse than the old range (the kernel draws from the ephemeral range, 32768-60999 here, shared with outbound source ports) rather than hiding it.
③ ⭐ A lost race now fails legibly.
portContentionError()turnsserve exited 1 before "Server is ready"into a failure naming port contention and the port, wired into all three spawn sites. The port is read out of the child's own diagnostic, not out of what the harness thought it reserved. This is the half that pays: the measured cost was never the lost run, it was an unreproducible red suite on a file the reader has no reason to connect to a port.④ Unchanged, deliberately.
os serve's production no-auto-select rule (#11113) is correct and untouched; no port is pinned.The probe is an instrument — proved, both directions
test/serve-port-bind-probe.test.ts(new) is the pin, 9 tests, 1.29s. It spawns nothing but the probe's ownnode -echild.portIsFree(held)must answer false; after release, true. A probe that reports "free" for a port you are holding is not an instrument, and every claim the docblock makes rests on this.probeBindtos.listen(0, …)— i.e. a probe that ignores the port it was asked about — was confirmed on disk (anchor occurrences 1 → 0, injected 1, blob35e0e2aa→d97a2123) under a restore trap, and the run went 1 failed / 7 passed with exactlyAssertionError: the bind probe called held port 35307 FREE — it is not an instrument. Restore verified by blob equality to HEAD and an emptygit diff HEAD.serve.tswas reworded. One test reads the livePort ${requestedPort} is already in usetemplate out ofsrc/commands/serve.tsand asserts the detector fires on it, so the day that wording changes this reds instead of going quietly illegible.Verification
Union re-run at the final commit
8f8c4654f, all exit 0:check:nul-bytes·check:cli-test-child-env·check:cross-package-test-inputs·check:test-source-alias·check:type-check-coverage·check:published-files·check:slot-lookup·check:page-declaration-shape·check:type-source-resolution·check:objectql-double-limit·check:engine-double-contract·check:where-matcher·check:query-options-erasure·check-ci-filter-parity·check-comment-mask-adoption·check-plugin-teardown-shape·docs-audit/check-affected-docsGate list derived with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstackfrom the real change set (4 paths), not from a hand-written diff.check:cli-test-child-envcaught a real finding mid-work: the probe's first draft passed{ ...process.env, NODE_OPTIONS: '' }toexecFileSync, a bulk copy into a spawn (0 → 1 over this file's ceiling). Repaired at the source — the probe child now getsenv: {}, since a barenetbind reads no variable at all. Measured side effect: ~74.7 ms/draw → ~38 ms/draw.pnpm lintnarrowed, and the narrowing is declared with its three pieces of evidence. ① Population read from ESLint's own config (not from a guess about which files count): all 4 changed paths came back linted, none ignored. ② File count read from--format json: 4 files, 0 errors, 0 warnings. ③ Invariance for untouched files: this repo runs oneeslint.config.mjswhich never enables type-aware linting for any file (noparserOptions.project, no typed@typescript-eslintrules — stated and positive-controlled in the config itself at line 328), so no edit in this diff can move any untouched file's verdict. The repo-wide scan is CI's run.pnpm --filter @objectstack/cli typecheckis green and says NOTHING about this PR.packages/cli/tsconfig.jsonisinclude: ["src"], sotest/**is in no tsc program — measured, 0 of the 4 edited files appear intsc --listFiles. That layer is ledgered (TEST_DEBT['@objectstack/cli'] = 146) and re-measured bycheck:type-check-debt --re-measure, which needs the whole workspace closure built. Measured the delta directly instead, same compilerOptions, HEAD vs merge basee57744536: 0 errors on these 4 paths in both trees. The first attempt at that measurement was vacuous — tsc bailed with TS5112 and compiled nothing while reporting a clean-looking 0; caught by a positive control (plantingconst x: number = portIsFree(1)produced no finding). Re-run with--ignoreConfig, the control producesTS2322as it must, and the 0 above is from that instrument.Not run locally, named rather than implied:
check:i18n/check:i18n-coverage(needpackages/cli/dist, prerequisite not met),check:type-check-debt --re-measure(needs the full built closure), and the two heavy e2e files this touches —serve-node-env-production-defaultandserve-app-anchored-optional-importare among the 20 real-CLI spawners that are 56.1% of this package's file wall. The shared verify lock was held continuously by two sibling cards for ~37 minutes across three attempts (exit 99, NOT MEASURED, holder named each time); the pin itself was narrowed out of the lock on measurement — 906 ms of test body, single worker, no build — and that narrowing is declared here rather than left silent.For the seat, not folded in
packages/cli/vitest.config.tsis claimed concurrently bypackages/cli/vitest.config.ts's suite-cost section is a 2026-08-20 measurement whose population changed under it — and it carries two inconsistent file/test vintages in adjacent sections #12499 and is untouched. Checked whether this change makes anything in its cost header stale: it does not. Its measurements are dated (2026-08-20) rather than live claims, and ~14 draws × ~38 ms is inside the noise of a 495.8 s wall.serve-mcp-stdio-answers.e2e.test.ts,serve-mcp-capability-collision.e2e.test.ts,serve-stdio-stdout-purity.e2e.test.ts. They get the bind probe (they callrandomPort()), but they spawnbin/run.jsdirectly withNODE_ENVunset — production posture, hard exit 1 on a taken port — without routing throughrunServe(), soportContentionError()never sees their output.runServe()path and is not fixed here:bin/run-dev.jspinsNODE_ENV=development, soserve.tstakesportAutoShiftAllowedandgetAvailablePort()drifts the child to a different port with no diagnostic at all.runServe()never learns the child's real port, so a test that then talks to the port it asked for reaches whatever else holds it.Generated by Claude Code