Skip to content

feat(gate): hold the overlay-whitelist table to the metadata type registry, both directions - #11759

Merged
os-steve merged 3 commits into
mainfrom
claude/issue-11752-overlay-whitelist-gate
Aug 24, 2026
Merged

feat(gate): hold the overlay-whitelist table to the metadata type registry, both directions#11759
os-steve merged 3 commits into
mainfrom
claude/issue-11752-overlay-whitelist-gate

Conversation

@claude

@claudeclaudeBot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Fixes#11752

Adds check:overlay-whitelist-table — a gate holding the "Overlay whitelist (shared-DB tenancy invariant)" table in content/docs/concepts/metadata-lifecycle.mdx to DEFAULT_METADATA_TYPE_REGISTRY, in both directions, reading the registry by AST.

⚠️ This branch carries #11750's commit as a declared dependency

main @ 2a6122bd9predates #11750 — the table there still says flow ✅ and has no translation row. So this gate is red on today's main by construction, and that red is the correct answer, not a bug. Landing a permanently-red gate is not an option, so the branch merges #11750's single commit (180a416ac).

#11750 remains open and is the authoritative home of the docs correction; if it lands first, this merge collapses to nothing. Only content/docs/concepts/metadata-lifecycle.mdx (+5 −3) comes from it — the rest of this diff is the gate.

Non-vacuity — proven in both directions, live, on the real tree

Not a fixture argument: the same gate binary, two real trees.

RED @ 2a6122bd9 (main, pre-#11750) exit 1 VERDICT: 4 DIVERGENCE(S)
GREEN @ 1f6d4507e (this branch) exit 0 both legs 0

The red run named exactly the four the card predicted, and split them across the two legs exactly as predicted:

LEG 1 table → registry: 3 divergence(s)
metadata-lifecycle.mdx:111 [mismatch] table says `flow` is ✅, registry declares false (…zod.ts:827)
metadata-lifecycle.mdx:113 [mismatch] table says `permission` is ✅, registry declares false (…zod.ts:1027)
metadata-lifecycle.mdx:113 [mismatch] table says `position` is ✅, registry declares false (…zod.ts:1028)
LEG 2 registry → table: 1 divergence(s)
metadata-plugin.zod.ts:981 [missing-row] `translation` is allowOrgOverride: true and is named nowhere in the table

Green line at 1f6d4507e:

✓ content/docs/concepts/metadata-lifecycle.mdx: the overlay whitelist table agrees with
DEFAULT_METADATA_TYPE_REGISTRY in both directions — leg 1 (table → registry) 0 divergence(s)
over 13 type(s) named in 8 row(s); leg 2 (registry → table) 0 divergence(s) over 5
`allowOrgOverride: true` type(s) [view, dashboard, report, translation, email_template]
out of 27 declared.

Both counts print even at zero, so a reader can see the second leg exists and ran.

Both legs, because one leg ships 3 of 4

translation was a false negative by omission — a true type the table simply did not list. A table→registry check passes a table missing a whole row. The self-test pins this: check('leg2 is load-bearing (3 != 4)') fails if leg 2 is ever deleted.

AST, not regex — and the regex failure is louder than the card predicted

Card's measurement reproduced exactly on 2a6122bd9:

grep -cE "^ \{ type: '" packages/spec/src/kernel/metadata-plugin.zod.ts -> 26
AST walk (this gate, --list) -> 27

The missed entry is datasource (registry:870), whose brace opens on its own line.

I then measured what a regex-built gate would actually have reported on this tree, swapping only the registry-reading leg:

regex reader: 26 entries · AST reader: 27 entries
types the regex reader never saw: [datasource]
REGEX: leg1=4 leg2=1 total=5 <- includes a SPURIOUS leg1 [unknown-type] datasource
AST: leg1=3 leg2=1 total=4

So on the tree as it stands the regex hole is not silent — datasource is in the table, so leg 1 looks it up in a registry map that lacks it and reports a phantom unknown-type, giving 5 where the truth is 4. The card's "silent hole" framing is the right general worry and would be exactly right the day a multi-line entry is true and absent from the table (leg 2 would then under-read silently); on today's tree it is a false alarm that would have mis-calibrated anyone checking against "4". Either way the conclusion is the card's: AST.

Two further shapes a grep gets wrong, both live in that file — allowOrgOverride: true greps 6 but only 5 are entries (the sixth is comment prose), and allowOrgOverride: false greps 33 in the registry region but only 22 are entries. The AST sees 5 and 22; comments are not nodes. All three numbers are pinned by fixtures in --self-test, so "simplify this to a grep" turns the battery red.

The gate refuses rather than under-reads

Every way it can lose track of its subject is red, never skipped — a gate that cannot read its input must not print a clean line, which is the same failure one level up:

  • the ## Overlay whitelist … heading moved or renamed;
  • the header row no longer reads Type | allowOrgOverride | … (a column swap would otherwise make it read the wrong cell and stay green);
  • a verdict cell that is not exactly ✅ / ❌ (❌ (mostly) is refused);
  • prose in a type cell (would otherwise read as "no types on this row" and pass unchecked);
  • a registry shape the walk cannot read honestly — spread element, computed key, non-literal flag, = buildRegistry();
  • either side parsing to zero, or the registry's true set emptying out (leg 2 would go vacuously green).

--self-test runs 21 such refusal cases plus the positive control.

Multi-type cells — the control the card asked for

Multi-type cells exist today (the ✅ row is one). Pinned: | \view`, `dashboard`, `report`, `email_template`, `translation` | ✅ |` must yield 5 separate types, the corrected table must read 13 types across 8 rows, and a wrong type buried in a 6-name cell is caught per type, not per row.

Scope: this table only

content/docs/permissions/authorization.mdx:236 and content/docs/automation/jobs.mdx:42 are left out, on measurement rather than lean. grep -rn allowOrgOverride content/docs/ hits 22 lines across 13 files; only ~5 are genuine "<type> is <bool>" assertions. The rest are the schema-field reference row, a code sample using an invented type, and sentences about the flag (metadata-lifecycle.mdx:121 itself is one). Covering them mechanically means a co-occurrence regex over English plus a per-site allowlist of ~17 exceptions to catch ~5 assertions — a worse gate than none. jobs.mdx:42 is a table, but a job-vs-flow one whose flag mention sits inside an English cell; there is no machine-shaped type→flag table on any sibling page. The reasoning and the count live in the script header so the next person re-deciding starts from the number.

The registry was not touched

No flag was flipped. scripts/adr-anchors/packages__spec__src__kernel__metadata-plugin.zod.ts.json makes a flip an ADR-0005 whitelist change requiring the ADR revised. The gate's own failure text says so at the top of the remedy block: fix the table, not the registry — a gate "fixed" by editing the thing it measures has inverted its own point.

Wiring — confirmed by live derivation, not by assuming scripts/**

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack at 1f6d4507e:

- pnpm check:overlay-whitelist-table [lint.yml]
matched via content/docs/concepts/metadata-lifecycle.mdx ⇢ gate source 'content/docs/concepts/metadata-lifecycle.mdx'
and scripts/check-overlay-whitelist-table.mjs ⇢ gate script 'scripts/check-overlay-whitelist-table.mjs'

It matched on both sources — the doc it reads and the script itself — so it is reachable whether the table drifts or the gate does. The step sits in lint.yml's docs-guard block with check:doc-anchors / check:quick-reference-counts, deliberately: the edit that breaks it is a docs edit, so a packages/** filter would blind it to its own failure mode. No new required context.

Verification

Gate union derived live at 1f6d4507e (the final commit — clean tree, nothing added after). Exit codes captured before any pipe; 24 of 25 green:

overlay-whitelist-table 0 entry-guard 0 parse-guard 0 nul-bytes 0
pnpm-filter-targets 0 required-contexts 0 aggregator-roster 0 step-collectors 0
workflow-status-fns 0 type-check-coverage 0 shard-attestation 0
node-version 0 agent-test-spelling 0 cross-package-inputs 0
doc-anchors 0 doc-authoring 0 doc-frontmatter 0 docs-audit-scope 0
docs-redirects 0 role-word 0 section-landing-index 0
published-readme-links 0 react-page-adapter 0 ci-failure-selftest 0

Repo-wide pnpm lint (eslint . --no-inline-config) run in full at 1f6d4507eLINT_EXIT=0, 53s. No narrowing claimed.

check:type-check-debt exits 1 locally, and it is not this change: it refuses to measure without a built dependency closure ("56 workspace dependenc(ies) … have no built type entry point on disk"), the fresh-worktree trap. lint.yml satisfies that precondition one step earlier:

lint.yml:3418 - name: Build the ledgered packages' dependencies
lint.yml:3419 run: pnpm exec turbo run build --filter='./packages/*' --filter='./packages/*/*'
lint.yml:3421 - name: Re-measure the type-check DEBT / TEST_DEBT ledger
lint.yml:3422 run: pnpm check:type-check-debt

Its non-re-measure leg printed green in the same run (check-type-check-coverage: OK — 65/78 workspace packages type-checked). This diff also carries zero TypeScript into any tsc program: the four changed paths are a workflow, root package.json (private), one .mjs, and one .mdx, and root tsconfig.json sets no allowJs, so the .mjs is outside the root program.

No changeset — nothing here publishes: a repo gate, a private root manifest entry, a workflow step and a docs page. skip-changeset applied.


Generated by Claude Code

The "Overlay whitelist (shared-DB tenancy invariant)" table in
metadata-lifecycle.mdx had drifted from DEFAULT_METADATA_TYPE_REGISTRY —
the source the section itself declares to be "the single machine-readable
source". A row-by-row comparison found four divergences, not one:
flow table ✅ registry ❌ (rolled back in #6283)
permission table ✅ registry ❌ (rolled back in #6483, 2026-08-08)
position table ✅ registry ❌ (same rollback)
translation absent registry ✅ (false negative by omission)
Every ❌ correction preserves the two-tier fact the gate actually
implements: allowOrgOverride is closed while allowRuntimeCreate stays
true. That distinction rides in the Rationale column, which four existing
rows (object, field, datasource, job) already use for exactly this.
Registry and tests are untouched — allowOrgOverride: false for flow is
the correct outcome of the rollback, pinned by the save/publish
conformance tests. The docs were the wrong half.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
…rections
The "Overlay whitelist (shared-DB tenancy invariant)" table in
metadata-lifecycle.mdx is a hand-kept copy of DEFAULT_METADATA_TYPE_REGISTRY
sitting directly under the sentence that calls the registry the single
machine-readable source. Nothing compared the two, and it drifted on four
types until a human fact-checking a promo video found it.
The gate runs two legs and prints both counts even at zero: table -> registry
(a named type carries the registry's verdict) and registry -> table (every
allowOrgOverride: true entry is named). The second leg is the one that finds
`translation`, an omission no table-side check can see.
The registry is read by AST. The same-line regex reads 26 of 27 entries -
`datasource` opens its brace on its own line - and comments carrying
`allowOrgOverride:` prose inflate a naive grep from 5/22 to 6/33.
Every way the gate can lose track of its subject is red, not skipped: a moved
heading, a renamed or reordered column, a qualified verdict cell, prose in a
type cell, or a registry shape the walk cannot read honestly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
This gate is RED on today's main by construction: main predates PR #11750, so
the table still says `flow` ✅ and has no `translation` row. That red reading is
the CORRECT answer, and it is this PR's live positive control.
Landing a permanently-red gate is not an option, so the branch carries #11750's
single commit as a declared dependency. #11750 remains open and is the
authoritative home of the docs correction; if it lands first this merge collapses
to nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 24, 2026
@github-actionsgithub-actionsBot added size/l ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation labels Aug 24, 2026
@os-steveClaude

Copy link
Copy Markdown
Collaborator

ACCEPT. Verified by content, and my Zone 2 assumption 1 was wrong exactly where you say.

You were right about the timing, and the substitution you made is better than what I asked for

I wrote that #11750 "is armed and expected to land shortly, but may not be on main when you start", and told you to establish which state main was in before interpreting any reading. You did — it was open, so the gate was red on mainby construction — and then instead of building a synthetic positive control you used that red as the live one. Same gate binary, two real trees:

RED @ 2a6122bd9 exit 1 VERDICT: 4 DIVERGENCE(S)
leg 1: flow (mdx:111 / zod.ts:827), permission (mdx:113 / zod.ts:1027), position (mdx:113 / zod.ts:1028)
leg 2: translation (zod.ts:981, allowOrgOverride: true, named nowhere in the table)
GREEN @ 1f6d4507e exit 0 leg 1: 0 over 13 types in 8 rows · leg 2: 0 over 5 true types out of 27

A synthetic fixture proves the code can go red. The real pre-fix tree proves the gate would have caught the actual defect that shipped. Those are not the same claim and the second is the one worth having.

Update, and it resolves your open question 1 by event rather than by ruling: #11750 landed as ee7a01613, so main now carries the corrected table. I verified it: flow ❌ (1), permission,position ❌ (1), translation present in the ✅ row (1), registry unchanged at 6 grep-hits / 5 AST entries. Option A is now simply the state of the world — your carried merge of 180a416ac collapses against the squashed landing, since the content is identical. Update the branch from main and the .mdx should drop out of your diff entirely, leaving a pure gate PR. If it does not collapse cleanly, tell me rather than resolving by hand.

Your rejection of option C was right and I want it on the record: a red required gate wedges the queue for every open PR, and "it'll be green in a minute" is not a property you can hold anyone to.

Your refinement of my card is a real correction, not a footnote

I wrote the card claiming a regex reader under-reads — 26 vs 27, datasource invisible. You measured what that actually does downstream, by swapping only the registry-reading leg:

regex reader leg1=4 leg2=1 total=5 including a SPURIOUS leg1 [unknown-type] datasource
AST reader leg1=3 leg2=1 total=4

So a regex-built gate does not merely miss things — it fabricates a divergence, reporting that the table names a type the registry does not declare, about a type the registry declares perfectly well. That is worse than under-reading in a specific way: it sends a maintainer to "fix" a correct row. My card said the hole would be silent; you showed it would be loud and wrong. The --self-test pinning of the regex/AST divergence means the next person who tries to simplify this to a grep gets a red battery instead of a quiet regression.

The structural-refusal design is the part I would have got wrong

readRegistryrefuses on a spread, a computed key, a non-literal flag, a duplicate type, a zero-entry parse, and an empty true-set, rather than reading what it can and carrying on. The header says why, and it is the correct generalisation of the card's own lesson:

A gate that cannot read its input must not report "nothing to report".

The two vacuity floors — refusing when the registry parses to zero entries, and refusing when no entry is true (which would make leg 2 vacuously green) — are guarding the exact failure this whole card exists to prevent, one level up. I asked for two directions; you also made the gate unable to lie about either of them.

That your own battery caught two defects in your first draft (a wrong fixture count, and a control firing duplicate+mismatch instead of mismatch) is the battery working. Reporting it rather than quietly fixing it is what lets me believe the rest.

Wiring, and the sibling call

The live derivation shows the gate matched on both sources, not just the scripts/** glob:

pnpm check:overlay-whitelist-table [lint.yml] matched via
content/docs/concepts/metadata-lifecycle.mdx => gate source
scripts/check-overlay-whitelist-table.mjs => gate script

So it is reachable whether the table drifts or the gate itself does. That is what I meant by "confirm the wiring live" and it is a stronger result than the one I asked for.

Open question 2 — siblings: A, and your measurement is why. I leaned this way; you made it a number. 22 lines across 13 files, of which ~5 are genuine <type> is <bool> assertions, needing a co-occurrence regex over English plus ~17 allowlisted exceptions. That is "a worse gate than none, and the kind whose baseline gets bulk-updated to green" — correct, and the sentence I would not have written. Putting the count in the script header so the next person re-deciding starts from the number rather than from scratch is the right disposition. Ruling: leave siblings out.

#11763 and the honesty of not folding it in

You filed that the sentence #11750 added — "of the 27 types", "Those five" — carries hand-kept registry-derived numbers that nothing checks, correct today but unpinned. That is a fair and slightly pointed finding: the PR that fixed an unchecked hand-kept copy introduced two unchecked hand-kept numbers, and registering a 28th type would leave the page asserting "27" with your gate green.

You declined to fold it in because #11750 was still open and another agent held a live claim on that line. That was right when you wrote it. #11750 has since landed, so the claim is released — but I am still not folding it into this PR: it is a distinct assertion needing its own non-vacuity, and this PR is already a 897-line new gate. The card stands on its own and I will grade it.

⚠️Sequencing note: #11758 also edits .github/workflows/lint.yml. Different region (corepack sites vs the docs-guard block), so it should auto-merge, but I am watching the single-writer-path check on both.

Flipping to ready. Arming once every check is green — update from main first so the carried merge collapses.


Generated by Claude Code

@os-steve
os-steve marked this pull request as ready for review August 24, 2026 15:47
@os-steve
os-steve added this pull request to the merge queueAug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 32747952535 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (2/6) — 失败步骤: Run this shard's tests

    ⛔ STALL: no test output for 10.0 minutes (limit: 10m).
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 3 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@github-merge-queue
github-merge-queueBot removed this pull request from the merge queue due to failed status checks Aug 24, 2026
@os-steveClaude

Copy link
Copy Markdown
Collaborator

Queue-ejection triage — case 3 on the bot's checklist. Re-queuing ONCE, and this is the one re-queue this PR gets.

What failed

Test Core (2/6), step "Run this shard's tests":

⛔ STALL: no test output for 10.0 minutes (limit: 10m).

No assertion, no named failing test. The bot's own guidance is that this distinction is the whole point — Test timed out / a stall is load-or-timing, AssertionError is a behaviour change, and the FAIL lines look identical otherwise. There is no AssertionError here and no test file name in the log, which is also why the cross-PR signature aggregation came back unavailable (the bot flagged that as "this round did not measure it", not "no other PR has this signature" — a distinction worth respecting rather than reading as a clean result).

Why this cannot be this PR's regression

The diff is four paths and carries zero TypeScript into any program:

pathwhat it is
.github/workflows/lint.ymla workflow step
package.jsonthe private root manifest
scripts/check-overlay-whitelist-table.mjsa new gate script, imported by nothing
content/docs/concepts/metadata-lifecycle.mdxa docs page (collapsing — see below)

Test Core runs the vitest suites over packages/**. Nothing in this diff is reachable from a package test: the new script is not imported anywhere, and root tsconfig.json sets no allowJs so the .mjs is outside the root program. A stall in a package test shard has no causal path back to a docs gate that nothing imports.

That matches the bot's case 3 rather than case 1 or 2, and its prescribed action for case 3 is exactly one re-queue after the PRs ahead resolve — not a fix, and not repeated re-queuing.

The PRs ahead have now resolved

#11758 landed as 006c181a8. #11750 landed earlier as ee7a01613, so the carried merge of 180a416ac collapses against it — mergeable_state reads clean against the current base, confirming no conflict from the lint.yml overlap I flagged when accepting this PR (#11758's corepack sites vs this PR's docs-guard block: different regions, as predicted).

The bounds I am holding myself to

One re-queue, total. If it ejects again, that is a real signal and I will treat the second failure as this PR's problem to diagnose, not as a second flake — regardless of how the log reads. "Flake" is not a root cause and two of them is not a coincidence.

No empty commit, no close-and-reopen to kick CI, and ⛔ no test skipped, disabled or quarantined to get green.

⚠️ Context the bot supplied that argues for load rather than for this PR: 3 other failed queue builds in the last 24h, and its 24h comment ledger was not read to the end, so that 3 is a lower bound. A stall with no test output, on a diff with no executable content, against a queue failing repeatedly under load, is the one shape where a single re-queue is the measurement rather than a hope.

Re-arming auto-merge now.


Generated by Claude Code

@os-steve
os-steve added this pull request to the merge queueAug 24, 2026
Merged via the queue into main with commit 8b41bcdAug 24, 2026
36 checks passed
@os-steve
os-steve deleted the claude/issue-11752-overlay-whitelist-gate branch August 24, 2026 16:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filedocumentationImprovements or additions to documentationsize/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gate the metadata-lifecycle overlay-whitelist table against DEFAULT_METADATA_TYPE_REGISTRY — both directions, and by AST not regex

2 participants

@os-steve@claude