Uh oh!
There was an error while loading. Please reload this page.
feat(cli+spec): compile a project of N packages into one packages[] artifact, with the assembled package body declared (ADR-0130 D4 producer, #14242 B) - #14513
Conversation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
#14430 landed) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
📓 Docs Drift CheckThis PR changes 4 package(s): 32 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 136 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin ffedbf020099904723635623f5f06196b38586ab && git checkout ffedbf020099904723635623f5f06196b38586ab
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7286dd58e806ed321cbdfc23a1c455db8f80b1ad 23c209f918682f73abbcb1fe23b8b3bb1a4cc6c8 && git checkout -B drift-repro 7286dd58e806ed321cbdfc23a1c455db8f80b1ad && git merge --no-ff 23c209f918682f73abbcb1fe23b8b3bb1a4cc6c8
node scripts/docs-audit/affected-docs.mjs --json 7286dd58e806ed321cbdfc23a1c455db8f80b1ad
|
…ose — the per-key mapped alias emitted the collections shape a second time and OOM'd the type-check-debt re-measure (#14439) `AssembledPackageBody` / `AssembledPackageBodyParsed` referenced `(typeof STACK_DEFINITION_COLLECTIONS_SHAPE)[K]` per collection key. Because those aliases are exported, the declaration emit wrote the shape const into `stack.zod-*.d.ts` a second time (21,443 lines beside the 42,449 the stack schema already inlines), and every consumer program re-inferred all ~35 collection input/output types once more. `Type Check · debt ledger` on the PR went red: the `qa/http-conformance` TEST_DEBT re-measure exceeded the 4096 MB ceiling the gate pins as CI's, while the merge-queue run on the base passed the same step. Reproduced locally (exit 3) on b4b9732. The aliases now keep the DERIVED key set (`AssembledPackageBodyKey`, still read off `COMPOSE_KEY_DISPOSITIONS`) and type each collection as `unknown`. Nothing exported references the shape const any more, so the second copy is not emitted and the per-key inference does not run. `assembledPackageBodyShape()` keeps its `Pick<typeof STACK_DEFINITION_COLLECTIONS_SHAPE, K>` return type (internal, not emitted), so a disposition key missing from the collections shape is still a compile error, and the RUNTIME schema still carries every collection's full declaration — refusals are unchanged. One spec pin narrows its element access at the point of use, as every reader of an assembled body already does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
…lti-package-build # Conflicts: # scripts/check-stack-collection-maps.mjs
hotlong
commented
Sep 2, 2026
CI red: |
… so the stack schema's printed declaration carries no named alias (#14439) A named type alias inside `ObjectStackDefinitionSchema`'s printed type (`manifest: z.ZodType<AssembledPackageBodyParsed, AssembledPackageBody>` on the `packages` element) can only be IMPORTED by the declaration bundler, never inlined. `system/environment-artifact.zod.ts` embeds the stack type, so the bundler turned `stack.zod` into a shared chunk and gave the `environment-artifact` chunk an import edge into it: every consumer of `@objectstack/spec/system` started loading the entire stack schema declaration it never loaded before. Measured on the `qa/http-conformance` TEST_DEBT re-measure program, same shape as the gate builds it, 8 GB cap so the peak is measured rather than hit: base `53d36892` 691,580 lines of definitions / 4,473,321 K heap; PR head `3878a582` 734,202 (+42,622 — the size of the stack schema's declaration) / 4,875,249 K — over the 4096 MB ceiling `scripts/check-type-check-coverage.mjs` pins as CI's, which is the red `Type Check · debt ledger` lane on both earlier pushes. `AssembledPackageBodySchema` is now annotated `z.ZodType<Record<string, unknown>, Record<string, unknown>>` and the two ADR-0122 aliases are derived FROM the schema (`z.input` / `z.infer`), so nothing named can re-enter the stack schema's printed type. The runtime schema is unchanged: manifest fields plus every collection, key set still derived from `COMPOSE_KEY_DISPOSITIONS`. What consumers lose is static field typing inside an assembled body, which the PR's readers (`compile.ts`, `artifact-packages.ts`) never relied on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
hotlong
commented
Sep 2, 2026
CI red → root cause measured, second fix pushed as |
base 53d36892 | PR head 3878a582 | PR head 23c209f9 | |
|---|---|---|---|
| Files | 906 | 912 | 906 |
| Lines of Definitions | 691,580 | 734,202 (+42,622) | 691,218 |
| Types | 1,801,110 | 1,973,165 | 1,798,148 |
| Instantiations | 7,342,314 | 8,072,109 | 7,327,734 |
| Memory used (tsc's own count) | 4,473,321 K | 4,875,249 K | 4,434,453 K |
Two things follow from that table. First, the base already needs more than 4 GB of heap by tsc's own count and clears the 4096 MB ceiling only through GC pressure — that program has almost no headroom on main, independent of this PR (filed as #14569 for the maintainer). Second, the PR had added +42,622 lines of definitions to a program that imports @objectstack/spec/system, not @objectstack/spec — and 42,622 is the size of ObjectStackDefinitionSchema's declaration.
Mechanism.system/environment-artifact.zod.ts embeds the stack type. On base the declaration bundler inlines that expansion into the environment-artifact chunk, so consumers of @objectstack/spec/system never load the stack schema's declaration. This PR had put a named type alias inside the stack schema's printed type — packages: z.ZodArray<z.ZodObject<{ manifest: z.ZodType<AssembledPackageBodyParsed, AssembledPackageBody, …> }>> — and a named alias can only be imported from the chunk that declares it. So stack.zod became a shared chunk, the environment-artifact chunk grew an import { AssembledPackageBodyParsed, AssembledPackageBody } from './stack.zod-*.js' edge, and every consumer of @objectstack/spec/system started loading the entire 2 MB / 42k-line stack declaration. (e167d438 had removed a second, additive cost — the alias's (typeof STACK_DEFINITION_COLLECTIONS_SHAPE)[K] mapping emitted the collections shape a second time, 21,443 lines — but left the named-alias edge in place, which is why that push stayed red.)
Fix (23c209f9).AssembledPackageBodySchema is annotated with a structural type, z.ZodType<Record<string, unknown>, Record<string, unknown>>, and the two ADR-0122 aliases are derived from the schema (z.input / z.infer) rather than the schema from them. Nothing named appears inside the stack schema's printed type any more; the built dist is back to the base topology (63 declaration files, no stack.zod chunk, the environment-artifact chunk's imports identical to base). The runtime schema is untouched — manifest fields plus every collection, key set still derived from COMPOSE_KEY_DISPOSITIONS — so refusals are unchanged. What consumers lose is static field typing inside an assembled body, which this PR's readers (compile.ts, artifact-packages.ts) never relied on; two pins narrow at their point of use.
Verification on 23c209f9 (exit codes captured before any pipe, dependency closure rebuilt): @objectstack/spec typecheck incl. check:scripts-typecheck and check:test-typecheck 0; @objectstack/cli typecheck 0; pins — spec assembled-package-body 11 + stack-artifact-packages 17, cli build-multi-package-artifact e2e 6, objectql artifact-load-path 14; check:api-surface 0; check:export-origins 0; check:spec-parsed-alias 0; check-stack-collection-maps 0 (self-test 18 assertions). CI's Type Check · debt ledger lane on this head is the authoritative re-measure.
Generated by Claude Code
hotlong
commented
Sep 2, 2026
Contract review — PASS (Clause-② YES, accept-set narrows in one direction)Disclosure: reviewer and dispatcher are the same session ( What was checked
Non-blocking observations (recorded, not asked)
Removing Generated by Claude Code |
Fixes#14439
Fixes#14242
Part of #14122
ADR-0130 D4's producer side — the last platform piece before a product can be split into modules. A project is now N ordinary
defineStackpackages plus one project-levelcomposeStacks([...], { manifest: 'preserve' }), compiled into one artifact carryingpackages[], and the load gate parses each entry against a declaration that can actually describe it.The problem, as three parse seams
An assembled
packages[]body has to survive three parses, and every one of them isObjectStackDefinitionSchema:defineStackpackages.0.manifest.objects.0: Expected string but received objectcompile.tsObjectStackDefinitionSchema.safeParse(lowering.lowered)metadata/src/plugin.ts_parseAndRegisterArtifactManifestSchemawere silently STRIPPEDArtifactPackageEntrySchemawraps its body asmanifest: ManifestSchema, whoseobjectsisz.array(z.string())— glob patterns, the authoring-time shape. What the ADR-0130 load path registers is an assembled body whoseobjectsare definitions. One schema was describing two lifecycle stages of one noun, so #14240 could only gate the wrapper.#14242 recorded three roads; the maintainer took B on 2026-09-02. Road C (widen
ManifestSchema.objectsinto a union of both spellings) was rejected by name: a union that accepts both stages makes neither stage checkable.What changed
@objectstack/spec— the assembled stage is declared.AssembledPackageBodySchema= the manifest's fields plus every metadata collection the stack schema declares. The key set is DERIVED fromCOMPOSE_KEY_DISPOSITIONS(the total table that already refuses to compile when a top-level key has no composition rule) and typed as aPickoverSTACK_DEFINITION_COLLECTIONS_SHAPEkeyed byAssembledPackageBodyKey— so a collection declared in that table but missing from the collections shape is a compile error, not a key that quietly goes missing from every package body.objects,datasources,permissions), the collection wins. That precedence is not chosen here: it isAppPlugin's flatten order ({ ...manifest, ...bundle }) stated as a declaration instead of re-derived at three seams.ArtifactPackageSchemais the artifact-layer entry ({ manifest: ASSEMBLED_BODY }, same reserved wrapper position as D4's{ ref, integrity }future), andObjectStackDefinitionSchema.packagesrefers to it — which is what makes seams 2 and 3 accept the shape.ArtifactPackageEntrySchemastays exactly as it was: the authoring entry, manifest-only. It needed no widening, because onepackageskey can serve both stages without a union — the authoring form is an instance of the assembled form (a package identity carrying no collections), not a second branch of it. The one place they genuinely disagree is the glob spelling, and there the assembled meaning wins.STACK_DEFINITION_COLLECTIONS_SHAPEand is spread back intoObjectStackDefinitionSchema. Two surfaces need that exact key set and must not be able to disagree; it is also what breaks the declaration cycle (packages' element schema is built FROM the shape, so it cannot live in it). Key order is preserved, andAssembledPackageBodySchemacarries an explicit type annotation because inferring it emitted the manifest plus ~35 collections a second time inside the stack schema's own.d.tsandtscrefused to serialize it (TS7056).composeStacks(..., { manifest: 'preserve' })now folds each input stack's own metadata onto its manifest. Composition is the last moment per-package attribution exists — the composed stack flattens every collection to the top level, and a flattened array cannot say which package each item came from. Reconstructing the split downstream is not a harder version of this; it is impossible.@objectstack/cli—os build/os compilereadpackages[]. When the loaded definition carries one:functionvalue thatJSON.stringifydrops without a word, and apackages-carrying artifact is registered THROUGH that list, so the hook would simply not exist at boot. Callables are de-duplicated by function IDENTITY, so the artifact's two copies of one handler name the same ref.runAuthoringRules('build', …),@objectstack/lint's one registry) runs once per package, de-duplicated against the union run. Composition flattens, so the union is strictly more permissive than the packages it was built from; the artifact registers per package, so the per-package answer is the one the runtime lives with.packages[i]are assembled bodies.os devboots the same shape from source.@objectstack/objectql— the load gate is a full parse.resolveArtifactPackageOrderappliesArtifactPackageSchemato the whole entry instead of filtering the verdict down to wrapper-level issues. The body handed toregisterAppis still the caller's original — the parse is a gate, and a parsed clone would carryManifestSchema's defaults and drop undeclared keys, which is what would make the two D4 branches disagree (D7). The module header's record of the mismatch is rewritten.Accept-set change, in one direction (Clause-② = YES). A
packages[]entry whose body carries authoring globs where the assembled stage carries definitions is now refused — atdefineStack, atos build, and at load. Nothing in the field produces that shape:packages[]had no producer at all before this PR.needs:contract-reviewattached; PR left draft for the PM.Fixture
examples/app-multi-package— two packages, one namespace, one artifact:com.example.multi.core(type: 'app', ownscrm_accountand the app) andcom.example.multi.orders(type: 'module', noscopekey, ownscrm_orderwhoseaccountfield looks upcrm_account). The module is listed FIRST in the composition on purpose: it declaresdependencieson the core package and the load path sorts throughresolvePluginOrder, so array order is not what decides — and the'last'manifest pick then gives the artifact its App's identity.pnpm --filter @objectstack/example-multi-package buildemitsdist/objectstack.jsonwithpackages[](7.2 KB, artifact manifestcom.example.multi.core, packages[orders, core], per-package objects[crm_order]/[crm_account]).GET /api/v1/packageson the booted fixture, the artifact's own two rows (manifest bodies trimmed to their identity fields here; the full rows carry the assembledobjects/apps):[ { "id": "com.example.multi.core", "type": "app", "namespace": "crm", "scope": "project", "version": "1.0.0", "objects": ["crm_account"], "apps": ["multi_crm"], "dependencies": null, "writable": false }, { "id": "com.example.multi.orders", "type": "module", "namespace": "crm", "scope": "project", "version": "1.0.0", "objects": ["crm_order"], "apps": [], "dependencies": { "com.example.multi.core": "^1.0.0" }, "writable": false } ]writable: falseon both rows is asserted directly — PR #14430 merged intomainat 07:59 UTC while this branch was in flight, so no TODO was left. One honest note:scopereadsprojectrather than being absent, becauseManifestSchema.scopecarries.default('project')and bothdefineStackand the compile parse materialise it. The fixture authors noscopekey; the pipeline fills it. That is exactly why #14430's verdict has to come fromengine.manifestsrather than from the row.Verification
Exit codes captured before any pipe, on
060414178and re-run after the last commit.@objectstack/spec452 files / 12,185 tests,@objectstack/objectql257 / 4,459,@objectstack/cli229 / 2,623 — all passing.pnpm --filter @objectstack/spec --filter @objectstack/cli --filter @objectstack/objectql run typecheck— green, bothcheck:test-typecheckledgers included.@objectstack/example-multi-packageand@objectstack/dogfoodtypecheck green too.packages/spec/src/assembled-package-body.test.ts(11 — the derived key set, both stages refusing the other's spelling, and one test per parse seam),packages/cli/test/build-multi-package-artifact.e2e.test.ts(6, realos buildin a temp project),packages/qa/dogfood/test/multi-package-artifact.dogfood.test.ts(5, real boot + real HTTP door), plus 3 added topackages/objectql/src/artifact-load-path.test.ts(14 total).examples/app-showcasecompiled through the identical command on this branch (9508c9a4) and on its merge base (1d8ad0ff, a separate worktree, its own install and closure build): normalized sha25644edc51248fa15ea204e383be66ebcd939528fe7433d5244ddebaa91be6d6fa8on both. Two worktree-derived variations are normalized and named rather than hidden — the absolute paths the compile bakes in from its cwd, and the esbuild bundle hash, which is content-addressed over those same paths; the raw sizes differ by exactly the 20 bytes of path-length difference. The artifact carries nopackageskey on either side. The compile door's own negative half is pinned in the e2e file (nopackageskey minted, ref names unchanged, the per-package leg never announced).Ablations — predicted red set stated before running, both legs, restores by bytes
(a) delete the CLI
packages[]branch (the lowering's package walk + the per-package rule leg, bothif (false && …)). Predicted: the two build assertions red, the four schema/shape ones green, the dogfood boot unaffected. Observed: 2 failed / 4 passed — exactlythe fixture reaches the code path … per-package leg ranandLOWERS the callables inside a package body — under the SAME ref as the top level. The CLI e2e runs the CLI fromsrcthroughbin/run-dev.js, so there is nodistleg for the subject; asserted in the script rather than assumed. Restore leg: both blobs equal to HEAD bygit hash-object,git diff HEADempty, 0 markers left, suite 6/6 green.The dogfood boot staying green under (a) is a real finding, not a gap:
os dev/bootStackregister from source where the callables are still live functions, so the dropped-handler defect is artifact-only.(b) delete the assembled-form gate —
ArtifactPackageSchema's body member replaced with an ungated record, which removes the judgment at BOTH the compile parse and the load gate at once. Every consumer resolves spec throughdist, so both legs rebuilt@objectstack/specand ranscripts/ablation-dist-preflight.mjs. Predicted: spec 1 of 11, objectql 2 of 14, cli 1 of 6. Observed exactly that:the ASSEMBLED entry refuses authoring GLOBS where definitions belong,refuses a body carrying authoring GLOBS (#14242 B)+refuses a body whose collection is malformed at the item level, andrefuses a package body carrying authoring GLOBS, naming the path. Mutation confirmed on disk by anchor counts (before 1 / after 0 / injected 1) and indistby the preflight (marker in 12 built files). Restore leg: source blob equal to HEAD, whole-treegit status --porcelainclean, rebuilt, preflight--absentgreen over 221 built files, all three suites green again (11/11, 14/14, 6/6).Gates
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, re-derived against the actual diff (90 commands after the last commits, 80 at first derivation). Findings found and fixed:check:api-surface— 6 added exports; regenerated withgen:api-surface.check:export-origins— regenerated.check:stack-collection-maps— a real break my restructure caused: the gate extracts the stack-collection set from the schema's inline shape literal, and the collections now arrive through a spread. Taught it to follow a line-anchored...IDENTspread to aconst IDENT = {in the same source, and to returnnull(its loud refusal) rather than an empty set when a spread cannot be resolved — an empty set reconciles perfectly against every site, which is the failure the refusal exists for. 3 self-test assertions added for both directions, and the pass line's hand-written assertion count (already one below the truth) is now computed. Re-run: 8 enumerations reconciled against 31 declared collections, 0 failures — the same set as before the move, which is what proves the restructure changed no key.check:i18n-coverage— the new example was unbaselined;--updateadded it at 0 untranslated strings.check:type-source-resolution— the new example resolved spec's types throughdist; fixed withpathsat spec's source (plus thelib/types/noEmitconsequencespackages/qa/downstream-contractdocuments for the same reason), not with a registry entry.packages/objectql/src/registry-invalidate.test.tsimportedtype ServiceObjectfrom the package root, where it has never been exported. It was masked as aTS2459ledger entry; the annotation shrink pushed it toTS2305andcheck:test-typecheckrefused. Fixed at the import (@objectstack/spec/data, the spelling its 20-odd siblings use) and the ledger re-recorded.NOT MEASURED (prerequisite unmet, none related to this diff; CI measures all of them):
check-test-completeness(needs a savedturbo run testlog),pm/check-half-states(no GitHub route from this seat),check:type-check-debt(OOMs here — needs a full-repo build),check:pm-dispatch-gates(green earlier in this run; a later re-run hit the container's 10-minute foreground cap).check-engine-split-ratiorefused on the shallow clone untilgit fetch --shallow-since, then measured green.check-system-context-censusgreen, no repair needed. Everything else in the derived set is green.Boundaries held
⛔ No second authoring spelling. ⛔
validateSingleAppuntouched — each package is still a single-app stack; the project is iterated bypackages[]. ⛔ManifestSchema.objectsnot widened. ⛔ No ADR edited. ⛔ Marketplace / install surfaces untouched (ADR-0019 D2/D3). ⛔ Cross-artifact co-ownership (D8) not attempted.One cost, stated rather than hidden: a multi-package artifact carries its definitions twice — flattened at the top level (which the metadata service's artifact door iterates) and again inside
packages[i](whichObjectQL.registerAppiterates). Dropping the flattened half would leave a booted instance with no views, flows or permission sets, since that door reads only the top level. De-duplicating it means teaching the metadata door to readpackages[], which is its own decision and its own card — filed as #14512 (three roads, with the measurement) rather than smuggled in here.🤖 Generated with Claude Code
https://claude.ai/code/session_01UHvF5hyiZjnCyExFnfQB8m
Generated by Claude Code