feat(spec): gate the shipped llms.txt against the real package surface

[os-warren]

Fixes#11344

Option B of the card ("checked hand-keeping"), per the triage fence: a freshness gate over
packages/spec/llms.txt, no new generation pipeline.

Note: two angle-bracket path placeholders in the first version of this body were eaten by the
GitHub body sanitizer (a known behaviour — AGENTS.md, "GitHub mutates body BYTES"). They are
spelled without angle brackets below.

Why

packages/spec/llms.txt ships in the published tarball (files array, registered in
check-published-files.mjs as "Protocol summary for LLM consumers") and has no generator, so
nothing ever re-derived what it asserts. It is read by the audience least able to notice staleness
and most likely to turn a claim into code: an agent that reads - `FormSchema`: layout under
import * as UI from '@objectstack/spec/ui' writes an import that does not resolve.

Re-measuring on origin/main this stroke — the card's premise held, and was wider than it
recorded. #10833 fixed one phantom row (IGraphQLService) and the header provenance line; it did
not reach the rest:

Drift	Measured
symbols advertised that no entry point exports	11
packages advertised that do not exist	2 (@objectstack/nextjs, @objectstack/nestjs)
schema-inventory rows disagreeing with the tree	12 of 13, plus qa missing entirely
schema heading vs. the sum of its own table	171 vs 170 (and vs 207 real)
package heading vs. the workspace	19 vs 68

The IUIService row is the one that argues for a gate rather than another hand-audit: that
contract was removed in v11 (this package's own CHANGELOG.md: "Remove the deprecated
IUIService contract (use IMetadataService) — 11.0"), and the row outlived it by two majors.
Worse — #10833's changeset cited the IUIService row beside it as a live precedent for how to
treat a superseded contract, never noticing it was itself dead. A hand-audit standing directly next
to the defect did not see it.

The gate — check:llms-txt

packages/spec/scripts/check-llms-txt.ts, wired as check:llms-txt beside its siblings, with a
--self-test first leg per house pattern. Five invariants: SUBPATH, NAMED, COUNTED,
LISTED, STRUCTURE (the script header is the authority; each is named there with the defect
it catches).

Authorities, all checked-in — the gate needs no build:

• symbols → api-surface/ shards, already kept equal to the built dist/*.d.ts by
  check:api-surface, and shipped in the same tarball so a consumer can verify any claim without
  leaving the package. (src/ would be the wrong authority twice over: a name can be declared
  there and never re-exported, and a name can be exported from a barrel with no matching
  export const line anywhere — DriverInterfaceSchema is exactly that, absent to a src grep
  and perfectly importable.)
• subpaths → the manifest exports keys.
• per-domain counts → the *.zod.ts population under each domain directory of packages/spec/src,
  counted recursively. Not a semantics I picked: two of the file's thirteen rows (kernel 32,
  ui 18) still match this measure exactly, which recovers the derivation the original author used.
• package count → the non-private @objectstack/* workspace set, enumerated the way
  check-published-files.mjs enumerates it, with the same loud refusal on a workspace glob richer
  than a bare directory or a single directory level.

Symbol claims resolve at the strictness their position earns, which is the one piece of design
worth reviewing:

1. Layer bullets (§1) → the union of all entry points. That section groups concepts into
   architectural layers, and a concept legitimately lives in a neighbouring entry point
   (IDataDriver is described under the data layer and exported from ./contracts, exactly as
   this file's own §5 example imports it). Demanding entry-point agreement there would manufacture
   failures for true claims. Self-test case 6 pins this deliberately, so tightening it later reads
   as a decision rather than a cleanup.
2. Namespace bullets (§4) → that entry point only. The heading is literally
   import * as N from '@objectstack/spec/x', so the entry point is part of the claim.
3. Contract table rows (§6) → ./contracts only. This is the position IGraphQLService and
   IUIService occupied.

Fenced import ... from '@objectstack/spec/x' statements are read at strictness 2 — the most
directly copy-pasteable claim in the file.

Deliberately out of population, stated in the script header: prose; code-fence bodies (only
import specifiers are read — deciding which other identifiers in a snippet are protocol symbols is
a judgement the gate would have to invent); and the N+ lower-bound figures (46+ types,
7,095+, 1,470+) — see Open question.

No gen:llms-txt, and the classification is a safety property rather than bookkeeping, the
same one check:objectui-pin-citations carries. The numbers are not the claim, the prose beside
them is: restamping | integration | 7 | Connector (Database, File Storage, GitHub, MQ, SaaS, Vercel) | to 1 would leave a freshly-dated row listing six connectors that do not exist —
converting a loud staleness into a silent lie. Registered NO_GENERATOR in the check:generated
ledger (without which this PR fails its own --reconcile-only step).

Repairs — every one evidence-pinned, none invented

The gate is born red without these. Repair rule applied, stated so each is reviewable:
(1) exists under another entry point → keep (§1) or re-attribute (§4); (2) phantom whose
own bullet text denotes a real same-domain export → rename to it; (3) phantom with no
counterpart → delete the claim (#10833's landed precedent on this file: a deprecation note for
something with no replacement invents a migration path that does not exist).

Was	Now	Rule	Evidence
AnalyticsSchema	CubeSchema	2	bullet says "OLAP cubes, measures, dimensions"; ./data exports Cube/CubeSchema/DimensionSchema/defineCube
IdentitySchema	OrganizationSchema	2	./identity exports OrganizationSchema, MemberSchema, SCIMUserSchema; no Identity* anywhere
EndpointSchema	ApiEndpointSchema	2	./api
ThemeSchema	deleted (x2)	3	a Theme substring scan matches nothing across all 17 shards
RAGPipelineSchema	KnowledgeSourceSchema	2	a RAG/Rag prefix scan matches nothing; ./ai exports the Knowledge* family
MCPSchema	MCPServerRefSchema	2	./ai
FilterSchema	FilterArraySchema	2	bullet says "Data retrieval AST"; ./data
IDataDriver under /data	moved to the ./contracts note	1	exported from ./contracts
FormSchema	FormViewSchema	2	./ui
PolicySchema	KernelSecurityPolicySchema	2	bullet says "Security rules"; ./kernel
System block on /system	Kernel block on /kernel	1	all three of its symbols are ./kernel exports
ContractSchema	ResponseEnvelopeConfigSchema, ApiErrorSchema	2	bullet says "Request/Response envelopes"; ./api
IUIService row	deleted	3	removed in v11 per this package's CHANGELOG; exists nowhere
@objectstack/nextjs, @objectstack/nestjs	replaced with driver-sql, types	3	packages/adapters/ holds only hono

§3's table is rewritten against the tree (14 domains, 207) and §7's heading corrected to 68, with a
line making the curation explicit rather than implicit — the check:quick-reference-counts lesson
that a curated table must say it is curated.

Verification

All readings on the final commit 4ebac563a; every exit code captured before any pipe.

• pnpm --filter @objectstack/spec check:llms-txt → exit 0:
  check-llms-txt self-test: 20 cases pass. and
  packages/spec/llms.txt: 97 claim(s) re-derived — every advertised symbol resolves against api-surface/ (17 entry points), every subpath is a real exports key, the inventory matches src/ (14 domains, 207 schemas) and the package table matches the workspace (68 published).
• Born-red reading (the gate against the file as origin/main had it): exit 1, 35 findings
  — 18 named, 13 count, 4 listed — reproducing every drift measured by hand above.
• Ablation (from the committed state, restore trap armed): re-injected the IUIService row.
  On disk: injected-token count 0 → 1, git diff --stat1 insertion(+). Mutated run exit 1,
  naming llms.txt:173 [named] service-contract table advertises IUIService, which @objectstack/spec/contracts does not export (it exists nowhere). Restore leg: injected count back
  to 0, git status --porcelain clean, gate exit 0. No rebuild leg is stated because the gate
  reads no dist — its inputs are llms.txt, api-surface/*.json, package.json, src/ and
  pnpm-workspace.yaml, all source or checked-in.
• pnpm --filter @objectstack/spec typecheck → exit 0 (tsc --noEmit +
  check:scripts-typecheck + check:test-typecheck).
• pnpm --filter @objectstack/spec check:generated --reconcile-only → exit 0 (the new script is
  classified).
• Gate families from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack on the
  real changeset (6 paths, derived from the merge base — no hand-written path list), all exit 0:
  check:published-files, check:nul-bytes, check:merge-driver, check:slot-lookup,
  check:test-source-alias, check:type-source-resolution, check:cross-package-test-inputs,
  check:override-consistency, check:release-page-status, check:changeset-gate-self-tests,
  check:objectui-changeset, check:node-version, check:required-contexts,
  check:shard-attestation, check:workflow-status-functions, check:pnpm-filter-targets,
  check:type-check-coverage, spec check:liveness / check:empty-state /
  check:strictness-ledger / check:variant-docs, and the check-adr-0087-registration,
  check-empty-changeset, check-changeset-no-major, check-changeset-fixed,
  check-osv-exemptions, check-aggregator-roster, check-step-collectors,
  check-plugin-teardown-shape scripts.
• pnpm --filter @objectstack/spec exec vitest run --maxWorkers=2 → exit 0:
  Test Files 420 passed (420) · Tests 11218 passed (11218).

Declared narrowing

pnpm check:type-check-debtrefuses in an unbuilt worktree — its --re-measure leg needs 56
workspace dependencies' dist/*.d.ts on disk and says so explicitly rather than measuring a
different world. Not run here, and it cannot be moved by this diff: @objectstack/spec carries no
DEBT/TEST_DEBT ledger entry, and the only TypeScript added sits in packages/spec/scripts/, whose
program is tsconfig.scripts.json — covered by check:scripts-typecheck (green above) and holding
zero ledger entries by design. Its coverage half, pnpm check:type-check-coverage, ran green. CI
builds the closure before this step.

Repo-wide pnpm lint was not run locally; CI owns that scan.

Open question for the maintainer

The three N+ figures — 46+ field types, 7,095+.describe() annotations, 1,470+ JSON
Schemas (really 1,586 today) — are out of the gate's population on purpose. They are checkable
only in the weak sense "real is at least claimed", which goes red only if the protocol shrinks,
so such a check would sit green while the figure drifts arbitrarily far — the shape
check-quick-reference-counts names when it refuses a comparison that can never fail. Making them
honest means choosing between an exact number that churns on most spec PRs and no number at all;
that is a content decision, not something this gate should settle by picking an operator. Happy to
follow up either way.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 1 changed package(s)), so this run has no opinion about the docs.

What this run could not see

• 1 changed file(s) yielded no anchor (packages/spec/llms.txt) — pages documenting those are invisible to this run

Coarse fallback — 126 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 88b9d749a1eb76b53966b0081006d2c5bccb341d → packageMentionDocs.