Uh oh!
There was an error while loading. Please reload this page.
fix(devx): bound vitest's inner worker pool from the invocation layer - #12182
Conversation
turbo's outer `--concurrency=50%` (#11954) bounds how many package `test` tasks run at once, but not vitest's own pool inside each of them. 40 of the 41 `vitest.config.ts` files say nothing about pool sizing — the single mention, in `packages/cli`, is a comment recording a REJECTED lever — so every package takes vitest's default of `max(cores - 1, 1)`, which scales with the host rather than with the shard it was given. Peak workers is the product of the two, and both terms grow with core count. Measured on a 4-CPU/15GB container, the product law holds exactly: 2x3=6, 4x3=12, 4x2=8, 4x1=4 concurrent workers observed. The bound goes at the invocation layer, per #10149's recorded reasoning that worker allocation is a property of the shard rather than of any one package's config. No `vitest.config.ts` is touched. Two traps this shape exists to avoid, both measured rather than assumed: - turbo filters task environments, so `VITEST_MAX_WORKERS` alone does NOTHING. Through turbo it spawned 3 workers (the unbounded default) while the same variable on a direct `vitest run` spawned 1. The `globalPassThroughEnv` entry is what makes the lever real. - vitest's `maxWorkers` is a PIN, not a ceiling — `resolveMaxWorkers()` returns the configured value outright. A flat `4` produced 8 workers at outer=2 where the default produces 6, i.e. a flat number RAISES the count on small boxes. So the cap is computed against the host's own cores and only ever lowers. A no-op on any host with <= 5 cores, today's CI runners included. In the regime where it binds (outer=2, inner 8 -> 4 on the 7-package fleet) it cut worker RSS 5700MB -> 2475MB for 93s -> 95s of wall, inside this box's noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
The file exports `workerCap`/`WORKER_CEILING` for tests and callers, so its top-level dispatch must not run inside an importer — `check:entry-guard` measured 8 of 39 unguarded exporters ending the importer mid-import, five of them exit 0. Value resolution moves into an exported `resolveValue()` and the write sits behind `isEntrypoint`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
yinlianghui
commented
Aug 25, 2026
|
Fixes#11958
turbo's outer
--concurrency=50%(#11954, for #11938) bounds how many packagetesttasks run at once. It does not bound vitest's own pool inside each ofthem, and nothing else does either: 40 of the 41
vitest.config.tsfiles saynothing about pool sizing, and the single mention — in
packages/cli— is acomment recording a rejected lever, not live config (verified). So every
package takes vitest's default of
max(availableParallelism() - 1, 1), whichscales with the host, not with the shard it was given.
Measured first, on a 4-CPU / 15GB container
7-package fleet (
core,platform-objects,metadata,service-settings,service-storage,plugin-sharing,service-datasource— each ≥27 testfiles, so each saturates its inner pool). Peak RSS is of the vitest processes
only, sampled at 0.4s and scoped to the run's own process tree.
outeristurbo's
--concurrency,inneris vitest's pool per task.Peak concurrent workers is exactly
outer × innerin all sevencombinations — the compounding is measured, not reasoned. Both terms grow with
core count, so the product grows quadratically: a 32-core box would run
16 × 31 = 496workers where this one runs 6.Two honest notes on the table. Total RSS is noisy — two same-config repeats
differed by 1.3GB and by 17s of wall — because the per-task parent process is
~1GB and its peak does not coincide with the workers'. Worker RSS is the
low-noise signal and it is the one that tracks worker count. And capping from
3 to 2 does nearly nothing (3332→3323 MB): at a pool that small, each worker
simply accumulates more modules, offsetting the count. The cap only pays once
the pool is large — 8→4 workers cuts worker RSS 5700MB → 2475MB (−57%).
The fix, and the two traps it is shaped around
At the invocation layer, in the root
testscript — novitest.config.tsistouched, honouring #10149's recorded reasoning (quoted in
packages/cli/vitest.config.ts) that worker allocation is a property of theshard, decided by whatever invokes turbo, not of any one package's config.
VITEST_MAX_WORKERSalone does nothing. Turbo filters taskenvironments. Measured: through turbo the variable set to
1spawned 3workers — the unbounded default — while the same variable on a direct
vitest runspawned 1.turbo.json'sglobalPassThroughEnventry is whatmakes the lever real; it is declared globally because per-task configs
(
@objectstack/spec#testand friends) would each need their own copy, and aspelling that reaches 68 of 72 packages is precisely the silent no-op this is
supposed to prevent. Verified against two packages that carry task-specific
turbo overrides.
maxWorkersis a PIN, not a ceiling —resolveMaxWorkers()returns the configured value outright rather than
min()-ing it with thedefault. Measured: a flat
4produced 8 workers at outer=2, where thedefault produces 6. A flat number therefore raises the count on small boxes.
So the cap is computed against the host's own cores in
scripts/vitest-worker-cap.mjsand only ever lowers.Every claim above was checked by observing the worker count, never by a
flag being accepted without error.
Cost
None measured. The ceiling is 4, and vitest's default is
cores - 1, sothis is a no-op on any host with ≤ 5 cores — this container and today's CI
runners included (verified end-to-end through
pnpm test: 3 workers before,3 after). In the regime where it binds, wall-clock was 93s → 95s, inside this
box's run-to-run noise. It bounds growth without taxing anyone now.
Scope note
The card's "could OOM at a higher core count" is a hypothesis about larger
boxes, and it stays one — nothing here reproduces an OOM, and the card itself
says "hypothetical". What is measured is the
outer × innerlaw and the memorythat follows from it.
CI's four
turbo run testsites are included because they bypass the roottestscript entirely; each gets the same computed cap, which is a no-op on4-core runners. A failed script yields an empty value, which is vitest's own
"use the default" signal.
Verification
pnpm lint(full repo, exit 0) and the 22 gate families derived bynode scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, allgreen at final commit
a3dc841400.check:entry-guardcaught a real defect inthe first draft — the script exports bindings and so must not run on import;
fixed in the second commit.
No changeset: root
package.jsonisprivate: trueand no published package'ssource changes —
skip-changeset.Generated by Claude Code