Uh oh!
There was an error while loading. Please reload this page.
build(docs): stop emitting the 348 MB of server source maps the OOM-killed build was paying for - #12712
Merged
Merged
Conversation
…illed build was paying for Every objectstack.ai production deploy on 2026-08-27 died with exit 137 and Vercel's `errorCode: "out_of_memory"` on a 4-core/8192 MB build machine. The build now declares `experimental.turbopackSourceMaps: false`, suppressing 260 map files totalling 348 MB that no production serverless function reads. The knob matters because of where the kill lands: every failing log places it between `Creating an optimized production build ...` and `Compiled successfully` with no output in between, i.e. inside the Turbopack compile phase. Neither knob already present reaches there -- `experimental.cpus` bounds static-generation workers that have not spawned yet, and `--max-old-space-size` bounds V8 while Turbopack allocates from Rust outside it. Measured, local cold build of all 403 pages, peak single-process RSS: 5191 MB -> 4757 MB (-434 MB, -8.4%); compile 22.6s -> 19.5s. Three other candidates measured as noise and are retired: turbopackScopeHoisting (4806 MB), --max-old-space-size=2048 (4734 MB), turbopackFileSystemCacheForBuild (4779 MB). Not paired with `turbopackMinify: false`, which takes a further 972 MB off the peak but inflates client JS from 5.8 MB to 16 MB (+176%). Confining minification to the server side is not available: `experimental.serverMinification` is read only by the webpack config, never on the Turbopack path. Buys margin; does not prove the ceiling is cleared. The measurement is macOS/arm64 against a Linux/x86_64 container where the same phase measured 7.6 GB (#12683), leaving roughly 15% headroom. #12683's option A (larger build machine) is unaffected and remains the answer if the next build still dies. Refs #12711, #12683, #12698 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Production docs deploys have been OOM-failing all day (
exit 137, VercelerrorCode: "out_of_memory", 4-core/8192 MB build machine). This suppresses 348 MB of server source maps that nothing in production reads.Where the build actually dies
Every failing log places the kill between
Creating an optimized production build ...andCompiled successfully, with zero output in between:So the peak is the Turbopack compile phase. The build never reaches
Collecting page data/Generating static pages— which retires the prerender-based attribution in #12683, and with it the 403 build-time OG PNGs and 403 markdown routes as suspects. They aregenerateStaticParamswork in a phase the process dies before.It also explains why neither knob already in the tree reaches this:
experimental.cpus: 2bounds static-generation workers that have not spawned yet, and--max-old-space-sizebounds V8 while Turbopack allocates from Rust outside it. Both confirmed by measurement below — which is #12683's own finding, now with the mechanism attached.Measured (local cold build, all 403 pages, peak single-process RSS)
turbopackSourceMaps: falseturbopackScopeHoisting: false--max-old-space-size=2048turbopackFileSystemCacheForBuild: trueturbopackMinify: falseincludeProcessedMarkdown: falsegetLLMText)The negative rows are recorded so they are not re-tried.
Why
turbopackMinify: falseis not in hereIt takes another 972 MB off the peak, but client JS goes 5.8 MB -> 16 MB (+176%) — a cost every reader pays on every visit, to save memory in a machine they never touch. Confining it to the server side (where the memory is: 589 MB of server chunks vs 5.8 MB client) is not on offer:
experimental.serverMinificationis read only bydist/build/webpack-config.jsand never on the Turbopack path.What this does not claim
It buys margin; it does not prove the ceiling is cleared. The measurement is macOS/arm64 against a Linux/x86_64 container where the same phase measured 7.6 GB (#12683) vs 4.7 GB locally — the ratio is the transferable part, and −8.4% onto 7592 MB leaves ~15% headroom, which is thin. #12683's option A (larger build machine) is untouched by this PR and remains the answer if the next production build still dies. Merging this first costs one build cycle to find out.
Secondary signal worth watching: Vercel spends 11 minutes compiling what takes 20s locally. A 4-vs-10 core gap does not explain 30x — the rest is thrashing near the ceiling, so a successful build should also come back much faster.
Verification
Merge, then:
Any id other than
dpl_2nfWjGjSwZjakVEmUG1kBWD6697rmeans the outage is over.Also corrects #12698
#12698's action 2 ("fix the Ignored Build Step") rests on a misreading. The project's rule is
if [ "$VERCEL_ENV" == "production" ]; then exit 1; else exit 0; fi—exit 1= build,exit 0= skip — so production always builds and Preview always skips. TheCanceled by Ignored Build Stepon PR #12684's head was that PR's Preview being skipped, as intended. Pushes tomaindo trigger production builds; they were failing, not skipped. No dashboard change is needed there.Refs #12711, #12683, #12698
🤖 Generated with Claude Code