Skip to content

perf(security): batch the derived half of bootstrapSystemCapabilities, unnarrowed - #12521

Merged
os-support-ai merged 1 commit into
mainfrom
claude/issue-11520-derived-half-batched-read
Aug 26, 2026
Merged

perf(security): batch the derived half of bootstrapSystemCapabilities, unnarrowed#12521
os-support-ai merged 1 commit into
mainfrom
claude/issue-11520-derived-half-batched-read

Conversation

@os-support-ai

@os-support-aios-support-ai commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Fixes#11520

Batches the DERIVED half of bootstrapSystemCapabilities, unnarrowed — option 1 of the three the card enumerated. ⛔ Option 2 (bucket-scoped read) is not taken, and this PR adds a pin that stops it being taken by accident.

The measurement that decided the card, done first

The card's premise is that the derived half needs X = the lowest-id row for the name, unscoped and installation-wide, while the shared index in seed-name-lookup.ts was built for the curated half, which carries the #8470 predicate inside its query. So: can buildExistingByName / readNamePage answer the unscoped question as they stand?

Yes, with no change to the shared file. Four properties, each read off the merged source rather than inferred:

PropertyWhereConsequence
No predicate ⇒ where is { name: { $in } } and nothing elsereadNamePage: { name: { $in: names }, ...(equals ?? {}) } — the spread contributes nothing when omittedThe batched read is the set-widening of the per-item { name }, with no key added
seedCtx(undefined) is { isSystem: true }per-organization-catalog.tsByte-identical to this file's SYSTEM_CTX; the read stays unscoped and cross-organization
Unscoped, resolveOwnOrganizationRow returns list[0] with no bucket filterper-organization-catalog.tsThe index answers X, not B — this is the single fact that decides the card
Pages arrive id-ordered and every row per name is kept in arrival orderreadNamePage passes limit; buildExistingByName accumulatesindex.get(name)[0] is the same row tryFind(…, 1)[0] returned under #4363's ORDER BY id ASC

The sibling seeder on this very table already records the same equivalence for its own unscoped adoption — bootstrap-declared-capabilities.ts: "an unscoped lookup is EXACTLY the question the per-item read asked — resolveOwnOrganizationRow returns the first row when no organization is given, which is what tryFind(…, 1)[0] returned."

Why the card's own objection expired

The card recommended option 1 but noted it "costs new machinery in a file four callers now share". That cost was paid by PR #11962: readNamePage now asks for one row more than its page budget and reports the overflow as truncated = "could not answer", degrading — loudly — to the per-item read. So "batching it unnarrowed needs an unbounded read" stopped being true. The worst case is now the old per-item cost plus a warning naming the budget.

What did NOT change, and is now pinned

⛔ The derived read is not narrowed to organization_id: null. Narrowing asks for B, the bucket occupant, which differs from X whenever an organization's row sorts lower — and the two divergences land on ruled ground in both directions: it would stop #8751's platformStampedInOrg signal firing in exactly the case its doc says it is counted for, and it would turn #8552's deliberate decline-to-seed into an insert. Neither has a maintainer ruling.

New test: ⭐ the DERIVED read is UNNARROWED — name only, no bucket predicate (#8552/#8751), asserting the emitted where and its key set separately (a leaked organization_id: undefined would pass toEqual while changing the question the driver is asked).

One behaviour change, named rather than buried

A derived name whose existence read cannot answer is now DECLINED (counted in unreadable) instead of being read as absent.

Previously the derived half read through tryFind, which catches and returns [] — so an unreadable database read as "absent" and routed every derived name to its INSERT branch. Where the read failed but the write did not (a transient read timeout, a lagging replica) that is a duplicate placeholder, refused only where the unique index happens to exist, and silent either way because the blockedCurated diagnostic is curated-only. Declining is what the shared oracle's module header already requires of every other caller, and is the direction #10946 chose deliberately for the curated half.

Consequently unreadable and its summary warning now cover both halves; the warning reports the whole definition set as its total rather than the curated count it would otherwise exceed.

Round trips

bootstrap-seed-round-trips.test.ts stated the residue as 1 + derived and its own doc said "a later card that batches it is expected to move these numbers deliberately." Moved, with the reason recorded in the suite doc:

  • before: finds = [1, 6, 21] at derived sizes [0, 5, 20]
  • after: finds = [1, 2, 2] — one curated read, one derived read, and at size 0 the derived read is not issued at all because buildExistingByName returns before reading when no name survives its filter.

The anti-vacuity half is kept explicit: a finds of 2 reached by skipping the derived half would satisfy that line, so unchanged is pinned alongside it (CURATED + d at every size) to prove every derived name was looked up, judged ours, and found already correct.

Dissolution verification of both new negative pins

Each mutation was proven on disk (git hash-object before/after, anchor matched exactly once), run, then restored with git checkout HEAD -- against the absolute repo-root path under an EXIT INT TERM trap, with git diff HEAD and git status --porcelain both empty afterwards.

1 — narrow the derived read to the platform bucket (option 2's exact shape): 12 tests RED, and they are the right twelve:

× does NOT backfill the platform bucket — the empty bucket is the ruled posture [#8552]
× counts the misplaced stamp, and reports the bucket it leaves unseeded [#8751]
× the organization copy survives (adverse — the org row sorts FIRST …) [#8751]
× ⭐ the DERIVED read is UNNARROWED — `name` only, no bucket predicate
Tests 12 failed | 81 passed (93)

That is empirical confirmation that option 2 really does reverse both rulings, that the new pin catches it, and that what ships here is not it.

2 — restore the old swallow (unknown && !isDerived): exactly 1 test RED, the new tri-state pin, AssertionError: expected 8 to be 10 (8 curated names vs 8 + 2 derived).

Verification

Gate union re-derived on the final tree with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (no hand-built path list): 3 paths vs merge base 7c368e85c, three-dot. All 19 matched families plus the convention-triggered set ran green on head 1c752670.

  • pnpm --filter @objectstack/plugin-security run test83 files, 1541 tests passed
  • pnpm --filter @objectstack/plugin-security run typecheck — exit 0
  • OK ObjectQL double limit conformance holds … none new · check-i18n-bundles: OK (9 packages — all bundles in sync)
  • ⚠️check:i18n first returned PREREQUISITE NOT MET — the workspace CLI is not built, which the gate itself states checks nothing. Read as NOT MEASURED, not as a pass: the CLI was built and the gate re-run to get the real verdict above.
  • ⚠️ The package typecheckexcludes its test files (its tsconfig.json says so) and the package carries no test-typecheck ledger, so the types of the edited test file are NOT MEASURED here — it is exercised at runtime by vitest. Verified with tsc --listFiles: the source file is in the program, the test file is not.
  • Lint narrowed to the two changed files (--no-inline-config --format json): 2 files, 0 errors, 0 warnings. The narrowing is sound because eslint.config.mjs states in its own text that this repo "never enables type-aware linting (no parserOptions.project, no typed @typescript-eslint rules) for ANY file, test or not" — there is no cross-file program, so this diff cannot move the verdict on any untouched file.

bootstrap-platform-admin.ts was not touched (read-coupled to the pin in #12512, which has since merged — this branch is based on 7c368e85, that merge). Zero packages/spec.


Generated by Claude Code

…, unnarrowed
The curated half was batched into one predicated `$in` read; the derived half —
the union of every `systemPermissions` string nothing declares — still read one
row at a time, so a rebuild cost `1 + derived` round trips.
Two objections had kept it per-item. Narrowing the read to the platform bucket
answers a different question and reverses ruled ground; batching it unnarrowed
needed an unbounded read. The second expired when `readNamePage` started asking
for one row more than its page budget and reporting the overflow as
"could not answer", degrading loudly to the per-item read.
The derived half now consults its own unpredicated `buildExistingByName` index.
Measured equivalence to the read it replaces: no predicate, so the emitted
where is `{ name: { $in } }` and nothing else; `seedCtx()` is the same
`{ isSystem: true }` context; and unscoped `resolveOwnOrganizationRow` returns
the first row with no bucket filter, so the index resolves to the same
lowest-id row installation-wide the per-item read returned. Steady-state
rebuild: 2 reads at every derived size.
The first objection stands and is now pinned, not merely documented — a new
test asserts the derived read's key set is `name` alone. Narrowing it would
silence the platform-stamped-in-org anomaly signal in exactly the case its doc
says it is counted for, and would seed the platform bucket in the case ruled
must be left alone.
One behaviour change: a derived name whose read cannot answer is now declined
and counted in `unreadable`, not read as absent. The old path swallowed a
failed read into `[]` and attempted an insert — a duplicate placeholder
wherever the read failed but the write did not.
Co-authored-by: Claude
@github-actionsgithub-actionsBot added size/m documentation Improvements or additions to documentation tests tooling labels Aug 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/plugin-security, touching 3 documentable anchor(s).

3 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/permissions/authorization.mdx(via sys_capability (literal))
  • content/docs/permissions/capabilities.mdx(via sys_capability (literal))
  • content/docs/permissions/permission-sets.mdx(via sys_capability (literal))

2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v15.mdx(via sys_capability (literal))
  • content/docs/releases/v17.mdx(via sys_capability (literal))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 14 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 14b114553c3e7c61672e3d25a1d15036f7561e27packageMentionDocs.

Which tree this was computed on

This run read content/docs from 50dbe4f03aa79a3660004135edc6c28eac9a257f — the merge of head 1c752670517f0af4ae965f3a6094406ff59f0053 into base 14b114553c3e7c61672e3d25a1d15036f7561e27, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 50dbe4f03aa79a3660004135edc6c28eac9a257f && git checkout 50dbe4f03aa79a3660004135edc6c28eac9a257f
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 14b114553c3e7c61672e3d25a1d15036f7561e27 1c752670517f0af4ae965f3a6094406ff59f0053 && git checkout -B drift-repro 14b114553c3e7c61672e3d25a1d15036f7561e27 && git merge --no-ff 1c752670517f0af4ae965f3a6094406ff59f0053
node scripts/docs-audit/affected-docs.mjs --json 14b114553c3e7c61672e3d25a1d15036f7561e27

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 14b114553c3e7c61672e3d25a1d15036f7561e27 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

2 participants

@os-support-ai@claude