Skip to content

fix(devx): bound vitest's inner worker pool from the invocation layer - #12182

Merged
yinlianghui merged 3 commits into
mainfrom
claude/issue-11958-vitest-inner-pool-bound
Aug 25, 2026
Merged

fix(devx): bound vitest's inner worker pool from the invocation layer#12182
yinlianghui merged 3 commits into
mainfrom
claude/issue-11958-vitest-inner-pool-bound

Conversation

@yinlianghui

Copy link
Copy Markdown
Collaborator

Fixes#11958

turbo's outer --concurrency=50% (#11954, for #11938) bounds how many package
test tasks run at once. It does not bound vitest's own pool inside each of
them, and nothing else does either: 40 of the 41 vitest.config.ts files say
nothing about pool sizing, and the single mention — in packages/cli — is a
comment recording a rejected lever, not live config (verified). So every
package takes vitest's default of max(availableParallelism() - 1, 1), which
scales with the host, not with the shard it was given.

Measured first, on a 4-CPU / 15GB container

7-package fleet (core, platform-objects, metadata, service-settings,
service-storage, plugin-sharing, service-datasource — each ≥27 test
files, so each saturates its inner pool). Peak RSS is of the vitest processes
only, sampled at 0.4s and scoped to the run's own process tree. outer is
turbo's --concurrency, inner is vitest's pool per task.

outer × innerpeak workersworker RSStotal RSSwall
2 × default(3)62202 MB3332 / 3896 MB76s / 93s
2 × 241409 MB3323 / 3228 MB84s / 97s
2 × 482475 MB4320 MB95s
2 × 8165700 MB7363 MB93s
4 × default(3)122869 / 3482 MB5485 / 6794 MB74s / 74s
4 × 282717 MB5817 MB75s
4 × 141322 MB4430 MB77s

Peak concurrent workers is exactly outer × inner in all seven
combinations — the compounding is measured, not reasoned. Both terms grow with
core count, so the product grows quadratically: a 32-core box would run
16 × 31 = 496 workers where this one runs 6.

Two honest notes on the table. Total RSS is noisy — two same-config repeats
differed by 1.3GB and by 17s of wall — because the per-task parent process is
~1GB and its peak does not coincide with the workers'. Worker RSS is the
low-noise signal
and it is the one that tracks worker count. And capping from
3 to 2 does nearly nothing (3332→3323 MB): at a pool that small, each worker
simply accumulates more modules, offsetting the count. The cap only pays once
the pool is large — 8→4 workers cuts worker RSS 5700MB → 2475MB (−57%).

The fix, and the two traps it is shaped around

At the invocation layer, in the root test script — no vitest.config.ts is
touched
, honouring #10149's recorded reasoning (quoted in
packages/cli/vitest.config.ts) that worker allocation is a property of the
shard, decided by whatever invokes turbo, not of any one package's config.

⚠️VITEST_MAX_WORKERS alone does nothing. Turbo filters task
environments. Measured: through turbo the variable set to 1 spawned 3
workers — the unbounded default — while the same variable on a direct
vitest run spawned 1. turbo.json's globalPassThroughEnv entry is what
makes the lever real; it is declared globally because per-task configs
(@objectstack/spec#test and friends) would each need their own copy, and a
spelling that reaches 68 of 72 packages is precisely the silent no-op this is
supposed to prevent. Verified against two packages that carry task-specific
turbo overrides.

⚠️vitest's maxWorkers is a PIN, not a ceilingresolveMaxWorkers()
returns the configured value outright rather than min()-ing it with the
default. Measured: a flat 4 produced 8 workers at outer=2, where the
default produces 6. A flat number therefore raises the count on small boxes.
So the cap is computed against the host's own cores in
scripts/vitest-worker-cap.mjs and only ever lowers.

Every claim above was checked by observing the worker count, never by a
flag being accepted without error.

Cost

None measured. The ceiling is 4, and vitest's default is cores - 1, so
this is a no-op on any host with ≤ 5 cores — this container and today's CI
runners included (verified end-to-end through pnpm test: 3 workers before,
3 after). In the regime where it binds, wall-clock was 93s → 95s, inside this
box's run-to-run noise. It bounds growth without taxing anyone now.

Scope note

The card's "could OOM at a higher core count" is a hypothesis about larger
boxes
, and it stays one — nothing here reproduces an OOM, and the card itself
says "hypothetical". What is measured is the outer × inner law and the memory
that follows from it.

CI's four turbo run test sites are included because they bypass the root
test script entirely; each gets the same computed cap, which is a no-op on
4-core runners. A failed script yields an empty value, which is vitest's own
"use the default" signal.

Verification

pnpm lint (full repo, exit 0) and the 22 gate families derived by
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, all
green at final commit a3dc841400. check:entry-guard caught a real defect in
the first draft — the script exports bindings and so must not run on import;
fixed in the second commit.

No changeset: root package.json is private: true and no published package's
source changes — skip-changeset.


Generated by Claude Code

turbo's outer `--concurrency=50%` (#11954) bounds how many package `test`
tasks run at once, but not vitest's own pool inside each of them. 40 of the
41 `vitest.config.ts` files say nothing about pool sizing — the single
mention, in `packages/cli`, is a comment recording a REJECTED lever — so
every package takes vitest's default of `max(cores - 1, 1)`, which scales
with the host rather than with the shard it was given. Peak workers is the
product of the two, and both terms grow with core count.
Measured on a 4-CPU/15GB container, the product law holds exactly:
2x3=6, 4x3=12, 4x2=8, 4x1=4 concurrent workers observed.
The bound goes at the invocation layer, per #10149's recorded reasoning that
worker allocation is a property of the shard rather than of any one package's
config. No `vitest.config.ts` is touched.
Two traps this shape exists to avoid, both measured rather than assumed:
- turbo filters task environments, so `VITEST_MAX_WORKERS` alone does
NOTHING. Through turbo it spawned 3 workers (the unbounded default)
while the same variable on a direct `vitest run` spawned 1. The
`globalPassThroughEnv` entry is what makes the lever real.
- vitest's `maxWorkers` is a PIN, not a ceiling — `resolveMaxWorkers()`
returns the configured value outright. A flat `4` produced 8 workers at
outer=2 where the default produces 6, i.e. a flat number RAISES the
count on small boxes. So the cap is computed against the host's own
cores and only ever lowers.
A no-op on any host with <= 5 cores, today's CI runners included. In the
regime where it binds (outer=2, inner 8 -> 4 on the 7-package fleet) it cut
worker RSS 5700MB -> 2475MB for 93s -> 95s of wall, inside this box's noise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
The file exports `workerCap`/`WORKER_CEILING` for tests and callers, so its
top-level dispatch must not run inside an importer — `check:entry-guard`
measured 8 of 39 unguarded exporters ending the importer mid-import, five of
them exit 0. Value resolution moves into an exported `resolveValue()` and the
write sits behind `isEntrypoint`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
@github-actionsgithub-actionsBot added size/m ci/cd dependencies Pull requests that update a dependency file labels Aug 25, 2026
@yinlianghuiyinlianghui added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 25, 2026 — with Claude
@yinlianghuiClaude

Copy link
Copy Markdown
CollaboratorAuthor

Type Check · consumer gates red — base-branch failure, not this PR's. No fix will be pushed here.

Diagnosed from the job log rather than inferred. The failing step is packages/client's check:exported-any-returns, and its verdict names the cause exactly:

❌ 1 stale exported-any-returns.json entr(y/ies) — the gap is closed, delete the entry:
• ObjectStackClient.packages.update — no longer resolves to `any`
ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL @objectstack/client@17.2.0 check:exported-any-returns

Byte-for-byte the same failure already observed on the merge-queue branches for #12146, #12154 and #12164.

Cause, verified on origin/main: PR #12062 (22c42c9b21, for #11925) bound packages.update so it no longer resolves to any, but its diff is three files — .changeset/…, packages/client/src/index.ts, packages/client/src/return-type-precision.test.ts — and the ledger is not among them. packages/client/exported-any-returns.json:17 still carries the row. The ledger is shrink-only and judged exactly, so a stale entry fails. This PR is based on 22c42c9b21 and inherits it.

Why it cannot be this diff. This PR touches turbo.json, the root test script, CI's turbo run test sites, and adds scripts/vitest-worker-cap.mjs. It contains no TypeScript source, nothing under packages/client, and no mechanism by which a test-runner worker cap could change what an SDK method's return type resolves to.

Already filed by two other seats — #12180 (priority:p0) and #12184 — both dispatched.packages/client is domain:cli's surface. I filed no third card and dispatched no dev; I flagged on #12184 that those two are duplicates of each other so they converge to one fix rather than colliding.

Next step: this PR waits for main to go green, then queues unchanged. If a rebase is wanted at that point I will merge main in rather than rebase.

The measurement table and the two traps in this PR's body are unaffected — they were produced and verified before this base breakage existed, and none of them runs in the failing job.


Generated by Claude Code

@yinlianghui
yinlianghui added this pull request to the merge queueAug 25, 2026
Merged via the queue into main with commit d18bc32Aug 25, 2026
33 checks passed
@yinlianghui
yinlianghui deleted the claude/issue-11958-vitest-inner-pool-bound branch August 25, 2026 14:46
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vitest's own worker pool scales with host cores, uncapped, in 40/41 packages — compounds with turbo's outer --concurrency

2 participants

@yinlianghui@claude