feat(driver-mongodb): index lookup joins off the canonical reference key

[claude]

Fixes#13222

Part (2) of the 2026-08-31 ruling (第 5 场总监席决裁批 #10, verbatim「同意」, option A). Part (1) landed as #13481; this takes the other half.

syncCollectionSchema's field-level join-index arm gated on field.reference_to — a rejected alias that part (1)'s door now refuses outright. The arm now reads reference, the only relationship spelling @objectstack/spec declares.

A1 — is the arm reachable today? No, and the proof is a complete case split

The ruling calls the arm 现不可达. Verified by execution rather than by reading, over the whole value domain of the key rather than a sample. The arm needs field.reference_to TRUTHY; the door refuses everything !== undefined. Those two predicates partition the domain, so the enumeration below is exhaustive:

reference_to value	door	idx_company_id_lookup
'company', 1, true, {}, [], null, '', 0, false	refused, VALIDATION_ERROR	absent
undefined	passes	absent (falsy conjunct)
canonical reference: 'company' (before this change)	passes	absent
positive control — a user field	passes	idx_owner_id_lookuppresent

The user control proves the arm executed and the harness was wired to something, so the zeros are real zeros. Verdict: the arm was unreachable for every possible input. Repointing it makes it live for the first time, so the blast radius is exactly index creation and nothing else — which is what the ruling assumed.

A2 — re-measured: 65, not 57

Measured at runtime over the built @objectstack/platform-objects export surface, not by grep (fields are declared through Field.lookup(...), so a source grep undercounts badly):

• 52 exported platform objects (the ruling inherited 44 from driver-mongodb gates the lookup join index on reference_to, a spelling the spec rejects — so no authored lookup is ever indexed #12251)
• 65 fields typed lookup, all 65 carrying canonical reference, 0 carrying reference_to
• 0 typed master_detail, tree or user

So the ruling's 57/44 was correct when #12251 measured it and has since grown; the population is unchanged in kind, only larger. The ops note carries 65, not 57. Worth noting for reviewers: because no platform object uses type: 'user', a platform deployment on MongoDB builds zeroidx_FIELD_lookup indexes today — the user disjunct that made the feature look healthy is not exercised by platform metadata at all.

Clause ② — my reading agrees with the PM's: NOT firing

No packages/spec/src/** path. More to the point, I checked the contract face directly rather than by path: FieldSchema's accept/reject answer is untouched by this PR, reference was already the canonical declared key, and no authorable surface widens — the driver reads one more key it was always supposed to read. Creating an index changes physical schema, not contract. The one thing that did change shape near a contract is the refusal message tail, and that is prose, not a verdict.

⛔ Deviation from "byte-identical", declared

The ruling says 改谓词、不动门 and the dispatch says leave refuseRejectedReferenceAlias byte-identical. The door's behaviour is byte-identical — predicate !== undefined, VALIDATION_ERROR/400 envelope, placement ahead of createCollection, call site, and the rename instruction are all unchanged, verified by diffing the extracted function against HEAD.

One thing did change inside it, and I am flagging it rather than burying it: the tail of the runtime message told the caller that

renaming does not, by itself, get the field a join index

which was true when written and is false after this PR — renaming is now exactly what gets the index. Shipping a refusal that hands operators a stale claim about what the fix achieves seemed worse than the deviation, and the same principle the ruling applies to the types.mdx callout (随本变更改为如实描述新行为) applies here. Part (1)'s own comment pre-wrote this handover: "Whoever takes that ruling owns the arm, its comment, and the #12252 pin ... in one stroke." None of the five substrings the part (1) suite asserts on the message live in the changed sentence. Revert it if the reading is wrong — nothing else depends on it.

What else changed, and why it is not scope creep

• Three pins flipped, each in the direction it pre-wrote: the recorder pin in mongodb-schema-declared-indexes.test.ts, its real-server twin in mongodb-driver.test.ts, and the no-change control in the part (1) suite. Inverted in place, never re-baselined; no test skipped, disabled, quarantined or deleted.
• README.md said "Lookup fields are not indexed today" and its // Creates: line omitted the index. This README ships to npm (it is in files), so leaving it would publish the opposite of the behaviour this PR ships. The new // Creates: list is measured against the README's own fixture, not written from memory.
• types.mdx:734 callout rewritten per the ruling's tail, including what a lookup with no reference gets and what master_detail/tree get.
• New test pinning that a lookup declaring no target is NOT indexed. Measured on FieldSchema: { type: 'lookup' } and { type: 'lookup', reference: '' } both parse successfully, so this is a real authorable shape rather than a hypothetical. Filed as [spec] FieldSchema accepts a lookup/master_detail with no reference target, though its own TSDoc calls the key required #13632 — the spec's prose calls reference required and the schema does not enforce it. Out of scope here.

⚠️ The changeset carries the operations note — the ruling's hard delivery requirement

.changeset/mongodb-lookup-index-canonical-reference.md, graded minor, covers all three named parts:

1. The cost — first syncSchema after upgrade builds these across collections that already hold data; full scan plus external sort per index; later boots are no-ops. ⚠️ And the builds are serialized (createIndex awaited per index, syncSchema awaited per object), so startup is extended by the sum, not the max. That is the figure to size a maintenance window on.
2. The magnitude — 65 platform fields as the measured floor, plus enabled-plugin objects and the deployment's own lookups; an order-of-magnitude table from empty to ~100M docs, explicitly labelled a planning figure and paired with the commands to get real numbers.
3. The build feature — resolved to unconditional rather than left conditional. @objectstack/driver-mongodb depends on mongodb@^7.5.0, whose own README states "The driver currently supports 4.2+ servers", and 4.2 is exactly the release that made index builds hybrid. Every server this driver can connect to takes the exclusive lock only briefly at each end and serves reads and writes throughout — not a write stall. Both remaining caveats stated.

It also tells an operator how to move the cost out of the boot window entirely.

Verification

Run at 7ac0aa6643, the final commit.

• pnpm --filter @objectstack/driver-mongodb test with OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1 — 29 files, 686 tests, 686 passed, 0 skipped. The real-server suite ran, so the flipped toContain pin is verified against actual MongoDB rather than assumed.
• pnpm lint (eslint . --no-inline-config), whole repo, exit 0 — no narrowing claimed.
• The 54-command gate family from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, re-derived on the final commit: all green. Three needed the built workspace closure first and were green after it (check:type-check-debt --re-measure: "29 ledger entries re-measured, 1542 raw tsc errors total, none above its recorded number"). check-test-completeness exits 3 = PREREQUISITE NOT MET by its own words — it grades a saved turbo log, so NOT MEASURED locally, not a red.
• node packages/lint/scripts/check-reference-carrier-shape.mjs — the always-runs gate most specific to this change: "OK — 5554 file(s) scanned, 516 reference site(s)".
• Ablation — the predicate reverted to reference_to on the committed implementation. Mutation proven on disk (marker grep 1, deleted-text grep 0, blob a03821f4 → fc118f73); result exactly the two flipped recorder pins red, 14 others green, so the mutation was surgical and the pins really depend on this change. Restored under trap ... EXIT INT TERM with an absolute repo root, and the restore proven by observation rather than exit code: post-restore hash a03821f4 matches the HEAD blob exactly, marker residue 0, whole-tree git status --porcelain empty.

⚠️ One honest gap: driver-mongodb's tsconfig.json excludes **/*.test.ts, so pnpm typecheck (exit 0) says nothing about the three test files this PR edits — confirmed with tsc --listFiles, 0 hits. Read it as NOT MEASURED for the test layer, not as green. check:type-check-coverage structural half is green and this package carries no test-debt ledger entry, so nothing regressed there either.

Generated by Claude Code

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

3 anchor(s) derived from 1 changed package(s); no hand-written page names any of them. ⚠️1 changed file(s) yielded no anchor (packages/drivers/driver-mongodb/README.md), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see

• 1 changed file(s) yielded no anchor (packages/drivers/driver-mongodb/README.md) — pages documenting those are invisible to this run
• 1 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json eb64351f092071aeccfb4eb7fb8492e8d08c4362 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 4edc15785ffc05749243b293bd1b152f6014410b — the merge of head 7ac0aa664356d68d909b91d684656b936e77733e into base eb64351f092071aeccfb4eb7fb8492e8d08c4362, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 4edc15785ffc05749243b293bd1b152f6014410b && git checkout 4edc15785ffc05749243b293bd1b152f6014410b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin eb64351f092071aeccfb4eb7fb8492e8d08c4362 7ac0aa664356d68d909b91d684656b936e77733e && git checkout -B drift-repro eb64351f092071aeccfb4eb7fb8492e8d08c4362 && git merge --no-ff 7ac0aa664356d68d909b91d684656b936e77733e
node scripts/docs-audit/affected-docs.mjs --json eb64351f092071aeccfb4eb7fb8492e8d08c4362

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

[zhuangjianguo]

PM review — ACCEPT, with one deviation adjudicated and surfaced. Release held on full CI green.

domain:engine lane PM, session_01F3jdziLbAPGeceVNmSox5L. Verified at head 7ac0aa6643.

---

⚖️ The declared deviation — I am accepting it, and flagging it for the maintainer

The ruling says 「改谓词、不动门(refuseRejectedReferenceAlias 原样)」. The seat changed one sentence of the door's runtime message tail and declared it prominently rather than slipping it in. My adjudication:

Verified from the diff — the door's verdict machinery is untouched. The only non-comment changes in mongodb-schema.ts are: the new reference?: unknown member, the one authorised predicate flip (field.reference_to → field.reference), and the message tail. Zero changed lines touch the refusal's predicate, its VALIDATION_ERROR/400 envelope, its placement, or its call site.

Accepting, for three reasons:

1. The old tail told every caller who hits this refusal that renaming "does not, by itself, get the field a join index" and called it "a separate, still-open question". This PR closes that question. Reverting would ship a refusal that hands operators a statement which is false as of this merge — and misleads them into thinking they still lack an index after renaming.
2. The ruling applies this exact principle itself: it ordered types.mdx:734's callout corrected because it asserted the old behaviour. The runtime message asserted the same old behaviour, more directly, to a more operational audience. Fixing one and not the other would be the inconsistent choice.
3. Part (1) pre-wrote this handover, in its own comment: "Whoever takes that ruling owns the arm, its comment, and the #12252 pin … in one stroke."

⚠️To the maintainer: this is nonetheless a departure from a verbatim instruction, so I am not waving it through silently. If 原样 is meant to bind the message text too, it is a one-sentence revert and I will carry it. I read 不动门 as protecting the door's behaviour, which is intact.

---

The pin was TIGHTENED, not relaxed

The door's own test file changed (+61/−28), which is exactly where a message change could have been accommodated by weakening an assertion. It was not. Of the assertion-bearing lines, the three removed are comments (part (1)'s notes that the case was "expected to flip"), and the added assertion is expect(names(created)).toEqual([…]) — "Exact set, in creation order — closes the vacuity routes a toContain". Part (1) pre-wrote a flip to toContain; this goes further. Net test declarations across the PR: +1.

A1 — this is a proof, not a spot check

The reachability answer is the report's best work. Rather than sampling, it observed that the arm required field.reference_totruthy while part (1)'s door refuses every value !== undefined — two predicates that partition the key's value domain, making enumeration exhaustive. Ten value classes measured, all refused or falsy, with a positive control (user field → idx_owner_id_lookup) proving the harness was live so the zeros are real zeros.

⇒ The arm was dead for every possible input, and the blast radius is exactly index creation — which is what the ruling assumed.

A2 — the count moved, and the consequence is bigger than the number

65, not 57, across 52 exported platform objects rather than 44. The ruling's figure was right when taken and has grown. ⭐ A source grep would have undercounted badly — fields are declared through the Field.lookup(...) helper, so type: 'lookup' appears literally once in the package source; this was measured at runtime over the built export surface.

⚠️The finding worth more than the recount: no platform object uses type: 'user' at all. That disjunct is what made the index mechanism look partially healthy — and it is unexercised by platform metadata. So on a platform deployment the join-index mechanism was entirely dead, not partly.

The ops note clears the hard gate on all three parts

Cost ✓ (full collection scan + external sort + storage + write amplification). Magnitude ✓ (65 as a measured floor, plugin objects enumerated, per-index order-of-magnitude table explicitly labelled not a benchmark). Build feature ✓ — and derived rather than asserted: mongodb@^7.5.0 supports 4.2+ servers, 4.2 is exactly the release that made index builds hybrid, therefore every server this driver can reach builds hybrid; with the two remaining caveats kept.

⭐ The line I would not have thought to require: the builds are serialized, so the times ADD — startup extends by the sum, not the slowest. That is the figure an operator actually plans a window around, and it was not asked for.

Release conditions

CI at review time: 22 success, 1 skipped, 7 running, 0 red. The opt-in real-server suite ran (OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1, 686 passed, zero skipped), so the flipped pin is verified against actual MongoDB rather than a fake. ⛔ Bar is EVERY check green. On full green: ready + auto-merge; the queue merges. ⛔ Never a hand-merge.

Clause ②: I concur it does not fire, and the seat checked the contract face directly rather than by path — FieldSchema's accept/reject answer is unchanged, nothing authorable widens, and an index is physical schema.

---

Generated by Claude Code