Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .changeset/spec-dts-heap-ceiling-fits-container.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
---
"@objectstack/spec": patch
---

build(spec): bound the DTS pass's heap ceiling to something the build container actually has (#12677)

The DTS pass ran under `NODE_OPTIONS="--max-old-space-size=12288"`. A heap
ceiling is a promise to V8 that the memory exists: below it, V8 defers major GCs
and lets the resident set grow. A ceiling **above** the container's memory does
not permit a bigger build — it converts a recoverable JS heap error into a
kernel SIGKILL, because the process meets the container limit long before V8
considers the ceiling reached, and `exit 137` carries no diagnostic.

That is what took the docs site down: every `objectstack.ai` production deploy
since 2026-08-25 died with `@objectstack/spec:build` exit 137. Vercel builds the
docs app with `turbo run build --filter=@objectstack/docs` inside one
fixed-memory container, and this package is that app's only workspace
dependency — so this pass met the container limit alone.

The ceiling is now `6144`. Measured on this package's DTS pass inside a cgroup
capped at 8192 MB, as peak anonymous RSS of the whole process tree:

| ceiling | result | peak RSS | wall |
|---|---|---|---|
| 12288 (before) | ok, ~0.9 GB spare | 7290 MB | 132s |
| 6144 (after) | ok, ~2.3 GB spare | 5794 MB | 134s |
| 5120 | ok | 5328 MB | 148s |
| 4096 | `ERR_WORKER_OUT_OF_MEMORY` | — | 113s |

No output change: every completing ceiling emitted a byte-identical declaration
tree (122 files, compared as one sha256 over all of them). The number buys
headroom and costs GC time, not build time.

Should the live type graph ever outgrow `6144`, the pass now fails **loud**
with `ERR_WORKER_OUT_OF_MEMORY` instead of being killed silently. Raising the
number past what the build container has would trade that diagnostic back for
`exit 137`.
2 changes: 1 addition & 1 deletion packages/spec/package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -243,7 +243,7 @@
"spec-changes.json"
],
"scripts": {
"build": "pnpm gen:schema && pnpm gen:openapi && tsup && if [ -z \"$OS_SKIP_DTS\" ]; then NODE_OPTIONS=\"--max-old-space-size=12288\" BUILD_DTS=true tsup; fi && node ../../scripts/check-dts-emitted.mjs && node ../../scripts/check-dev-prereqs.mjs --stamp",
"build": "pnpm gen:schema && pnpm gen:openapi && tsup && if [ -z \"$OS_SKIP_DTS\" ]; then NODE_OPTIONS=\"--max-old-space-size=6144\" BUILD_DTS=true tsup; fi && node ../../scripts/check-dts-emitted.mjs && node ../../scripts/check-dev-prereqs.mjs --stamp",
"dev": "tsc --watch",
"clean": "rm -rf dist",
"gen:schema": "OS_EAGER_SCHEMAS=1 tsx scripts/build-schemas.ts",
Expand Down
43 changes: 42 additions & 1 deletion packages/spec/tsup.config.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -138,7 +138,48 @@ const swapServerOnlyGrammarArm: Plugin = {
},
};

// Generate DTS separately to avoid memory issues
/**
* Generate DTS separately to avoid memory issues — this pass is by far the
* heaviest thing this package's build does, and `package.json` runs it under an
* explicit `--max-old-space-size` for a reason worth stating where the next
* author will look.
*
* ⚠️ THE HEAP CEILING MUST FIT THE SMALLEST CONTAINER THIS BUILD RUNS IN.
* `--max-old-space-size` is a promise to V8 that the memory is there: below it,
* V8 defers major GCs and lets the resident set grow. A ceiling ABOVE the
* container's memory therefore does not "allow a big build", it converts a
* recoverable JS heap error into a kernel SIGKILL — the process is killed at
* the container limit long before V8 ever considers the ceiling reached, and
* exit 137 carries no diagnostic at all.
*
* That is not hypothetical: at a 12288 ceiling this pass was killed on every
* Vercel docs deploy for two days (`@objectstack/spec:build` exit 137), because
* the docs site is built by `turbo run build --filter=@objectstack/docs` inside
* one fixed-memory build container and this package is its only workspace
* dependency — so this pass meets the container's limit ALONE, with no
* parallelism to cap.
*
* Measured on this package's DTS pass, inside a cgroup capped at 8192 MB
* (peak anonymous RSS of the whole process tree):
*
* ceiling result peak RSS wall
* 12288 ok, but only ~0.9 GB spare 7290 MB 132s
* 6144 ok 5794 MB 134s
* 5120 ok 5328 MB 148s
* 4096 ERR_WORKER_OUT_OF_MEMORY — 113s
*
* 6144 is chosen as the largest ceiling whose WORST case still fits: V8 cannot
* exceed it, and the pass's non-heap overhead measured ~250 MB, so the bound is
* ~6.4 GB inside an 8 GB container. Every completing ceiling emitted a
* byte-identical declaration tree (122 files, one sha256 over all of them), so
* this number buys headroom and costs nothing but GC time.
*
* If this pass starts failing with `ERR_WORKER_OUT_OF_MEMORY`, the live type
* graph has outgrown 6144 — that is a loud, actionable failure and the point of
* the ceiling. ⛔ Do not "fix" it by raising the number past what the build
* container has; that trades this error back for the silent exit 137. Shrink
* the graph, or split the pass across entries.
*/
const isDts = process.env.BUILD_DTS === 'true';

const mainConfig: Options = {
Expand Down
Loading