From 180a416ac5864e69e03f3e668e3c08da65c29e5b Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 14:58:25 +0000 Subject: [PATCH 1/2] docs: correct the overlay-whitelist table against the registry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The "Overlay whitelist (shared-DB tenancy invariant)" table in metadata-lifecycle.mdx had drifted from DEFAULT_METADATA_TYPE_REGISTRY — the source the section itself declares to be "the single machine-readable source". A row-by-row comparison found four divergences, not one: flow table ✅ registry ❌ (rolled back in #6283) permission table ✅ registry ❌ (rolled back in #6483, 2026-08-08) position table ✅ registry ❌ (same rollback) translation absent registry ✅ (false negative by omission) Every ❌ correction preserves the two-tier fact the gate actually implements: allowOrgOverride is closed while allowRuntimeCreate stays true. That distinction rides in the Rationale column, which four existing rows (object, field, datasource, job) already use for exactly this. Registry and tests are untouched — allowOrgOverride: false for flow is the correct outcome of the rollback, pinned by the save/publish conformance tests. The docs were the wrong half. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx --- content/docs/concepts/metadata-lifecycle.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index 32853d9871..4cf009faa4 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -107,15 +107,17 @@ In shared-database multi-tenancy, **most metadata types must not be per-org cust | Type | `allowOrgOverride` | Rationale | | :--- | :---: | :--- | -| `view`, `dashboard`, `report`, `email_template` | ✅ | Pure rendering. Per-org customization is safe. | -| `flow` | ✅ | Per-org overlays are allowed for automation definitions. | +| `view`, `dashboard`, `report`, `email_template`, `translation` | ✅ | Pure rendering / render-time content. Per-org customization is safe. | +| `flow` | ❌ | Flows carry execution side-effects (events, jobs, audit), and the `flow` entry declares `supportsOverlay: false` — the loader cannot merge a per-org flow overlay, so the write permission granted a write nothing could read back: an org-scoped flow overlay wrote successfully and **lost its binding on the next cold start**. Rolled back (#6283) so the silent phantom is a loud `403 not_overridable` at the moment of the write. **`allowRuntimeCreate` stays `true`** — a tenant may still author a *brand-new* flow through the runtime API (ADR-0070 package-first authoring); what is closed is overlaying a *packaged* flow per org. | | `agent` | ❌ | Agents are platform-owned and closed to third parties (ADR-0063 §2) — no per-org agent fork. | -| `permission`, `position` | ✅ | Per-org overlays are allowed; tenant-level controls layer on top. | +| `permission`, `position` | ❌ | Authorization correctness — a per-org overlay of a packaged permission set *is* silent privilege drift (ADR-0005's security row). Rolled back from an unratified `true` by the 2026-08-08 maintainer ruling (#6483); `position` takes the amendment's `false` default for new types. **`allowRuntimeCreate` stays `true`**: runtime-created sets (including package-bound rows materialized through the metadata door) keep working, while an admin-door edit of a *code-declared* set refuses with `403 not_overridable` — edit the package and re-publish (ADR-0086 two-doors). | | `object` | ❌ | Defines the table schema. Overriding a packaged object would break existing data — but `allowRuntimeCreate: true`, so tenants *can* author brand-new objects, **and this is also how a new field is added** (write the object with the field in `fields`). | | `field` | ❌ | **Also `allowRuntimeCreate: false` since protocol 17** (#7893). A field is not a standalone artifact — fields are authored inside their object (`ObjectSchema.fields`), so a `field` write minted a *separate* `sys_metadata` row keyed `('field','.')` that nothing ever composed into the parent: `PUT /meta/field/showcase_task.zz_probe` answered 200 `state=active`, the row read back `_diagnostics.valid: true`, and `GET /meta/object/showcase_task` never listed the field. The door is closed rather than bridged: add a field by writing its **object** (`PUT /meta/object/:name`, or `**/*.object.ts` and redeploy), which both persists and composes. Existing rows are untouched — they were inert before the change too — and remain deletable. | | `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. (`allowRuntimeCreate: true` — the datasource wizard persists `origin: 'runtime'` rows.) | | `job` | ❌ | **Also `allowRuntimeCreate: false` since protocol 17** (#4509). `JobSchema.handler` names a function in the compiled bundle's function table, which a runtime writer has no way to reach — so a job created in Studio or through `PUT /meta` parsed, saved, reported success and was never scheduled. The door is closed rather than bridged: `job` stays first-class through `*.job.ts` / `defineStack({ jobs, functions })`, where every schedule shape, `retryPolicy` and `timeout` does reach the scheduler. Existing rows are untouched — they were never scheduled — and `migrateStoredMetadata` reports them `skipped`. | +Those five are the **complete** `allowOrgOverride: true` set: of the 27 types in `DEFAULT_METADATA_TYPE_REGISTRY`, every other one is `false`. The ❌ rows above are the `false` types whose *second* tier (`allowRuntimeCreate`) is worth calling out; any type not listed is `allowOrgOverride: false`. + There is no `workflow` metadata type (per [ADR-0020](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0020-state-machine-converge-and-enforce.md), record state machines are a `state_machine` validation). Nor is there a standalone `validation` type any more — it was retired in protocol 17 under [ADR-0088](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0088-metadata-kind-admission-and-retirement.md) because `ValidationRuleSchema` carries no object-binding key, so a rule authored through that door could never say what it protected; author rules in the object's own `validations[]` instead. The runtime gate is implemented in `OVERLAY_ALLOWED_TYPES` (derived from the registry) and enforced by `SysMetadataRepository.put()`. The gate is **two-tier** — `allowOrgOverride: false` is not the same as "no runtime writes": From d6f8bdd1e614e7977160e85c66cd9ec22643f49f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 24 Aug 2026 15:27:03 +0000 Subject: [PATCH 2/2] feat(gate): hold the overlay-whitelist table to the registry, both directions The "Overlay whitelist (shared-DB tenancy invariant)" table in metadata-lifecycle.mdx is a hand-kept copy of DEFAULT_METADATA_TYPE_REGISTRY sitting directly under the sentence that calls the registry the single machine-readable source. Nothing compared the two, and it drifted on four types until a human fact-checking a promo video found it. The gate runs two legs and prints both counts even at zero: table -> registry (a named type carries the registry's verdict) and registry -> table (every allowOrgOverride: true entry is named). The second leg is the one that finds `translation`, an omission no table-side check can see. The registry is read by AST. The same-line regex reads 26 of 27 entries - `datasource` opens its brace on its own line - and comments carrying `allowOrgOverride:` prose inflate a naive grep from 5/22 to 6/33. Every way the gate can lose track of its subject is red, not skipped: a moved heading, a renamed or reordered column, a qualified verdict cell, prose in a type cell, or a registry shape the walk cannot read honestly. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx --- .github/workflows/lint.yml | 17 + package.json | 1 + scripts/check-overlay-whitelist-table.mjs | 897 ++++++++++++++++++++++ 3 files changed, 915 insertions(+) create mode 100644 scripts/check-overlay-whitelist-table.mjs diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 1daf4e2f80..1dc9214e4b 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -1101,6 +1101,23 @@ jobs: - name: Quick-reference section counts match their tables run: pnpm check:quick-reference-counts + # #11752: the "Overlay whitelist (shared-DB tenancy invariant)" table in + # metadata-lifecycle.mdx is a hand-kept copy of DEFAULT_METADATA_TYPE_REGISTRY + # sitting directly under the sentence that calls the registry the single + # machine-readable source. It drifted on FOUR types (flow/permission/position + # said ✅ against a registry that says ❌ after the #6283/#6483 rollbacks; + # `translation` is ✅ in the registry and had no row at all) and was caught by + # a human fact-checking a promo video (#11664), not by CI. The gate compares + # both directions — a table→registry check alone passes a table missing a + # whole row, which is exactly how `translation` survived — and reads the + # registry by AST: the same-line regex sees 26 of the 27 entries, because + # `datasource` opens its brace on its own line. + # It lives in this job with the other docs guards: the edit that breaks it is + # a docs edit, so a packages/** paths filter would blind it to its own + # failure mode. + - name: Overlay whitelist table matches the metadata type registry + run: pnpm check:overlay-whitelist-table + # #9604: the runtime-services chapter's pages are enumerated in three # hand-written places (meta.json "pages", the chapter list in # runtime-services/index.mdx, the services.* table in kernel/index.mdx) and diff --git a/package.json b/package.json index 938ab1e442..7ac1174e79 100644 --- a/package.json +++ b/package.json @@ -47,6 +47,7 @@ "check:template-version-sync": "node scripts/sync-template-versions.mjs --self-test", "check:role-word": "node scripts/check-role-word.mjs --self-test && node scripts/check-role-word.mjs", "check:quick-reference-counts": "node scripts/check-quick-reference-counts.mjs --self-test && node scripts/check-quick-reference-counts.mjs", + "check:overlay-whitelist-table": "node scripts/check-overlay-whitelist-table.mjs --self-test && node scripts/check-overlay-whitelist-table.mjs", "check:runtime-services-index": "node scripts/check-runtime-services-index.mjs --self-test && node scripts/check-runtime-services-index.mjs", "check:skill-frame-sync": "node scripts/check-skill-frame-sync.mjs --self-test && node scripts/check-skill-frame-sync.mjs", "check:skill-frame-freshness": "node scripts/check-skill-frame-freshness.mjs --self-test && node scripts/check-skill-frame-freshness.mjs", diff --git a/scripts/check-overlay-whitelist-table.mjs b/scripts/check-overlay-whitelist-table.mjs new file mode 100644 index 0000000000..41a815c8e3 --- /dev/null +++ b/scripts/check-overlay-whitelist-table.mjs @@ -0,0 +1,897 @@ +#!/usr/bin/env node +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * check-overlay-whitelist-table (#11752) -- the "Overlay whitelist (shared-DB + * tenancy invariant)" table in `content/docs/concepts/metadata-lifecycle.mdx` + * must agree with `DEFAULT_METADATA_TYPE_REGISTRY`, in BOTH directions. + * + * node scripts/check-overlay-whitelist-table.mjs # judge the checked-in tree + * node scripts/check-overlay-whitelist-table.mjs --list # every type and where it is asserted + * node scripts/check-overlay-whitelist-table.mjs --self-test # prove the battery can go red + * + * ## The gap this closes + * + * The section states its own contract: "The whitelist lives in **one** place: + * `MetadataTypeRegistryEntry.allowOrgOverride`". The table under that sentence + * is hand-maintained -- no generator writes the page, there is no `DO NOT EDIT` + * marker -- so the single source of truth had a hand-kept copy sitting directly + * beneath it with nothing holding the two together. + * + * It drifted on FOUR types and stayed drifted long enough to be caught by a + * human fact-checking a promo video (#11664), not by CI: + * + * type table said registry says + * ----------- ---------- ------------- + * flow ✅ ❌ (rolled back #6283) + * permission ✅ ❌ (rolled back #6483) + * position ✅ ❌ (same rollback) + * translation absent ✅ + * + * #11750 corrected the table. This gate is what stops the next drift. + * + * ## Why BOTH directions (constraint 1, established by measurement on #11750) + * + * `translation` was a false negative BY OMISSION -- a type the registry marks + * `allowOrgOverride: true` that the table simply did not list. A one-directional + * table→registry gate ("every row I can see agrees with the registry") passes + * on a table that is missing a whole row. Three of the four defects were + * findable from the table side; the fourth was findable ONLY from the registry + * side. A one-legged gate would have shipped 3/4 and reported clean. + * + * So the comparison runs as two legs, and BOTH counts are printed even when + * both are zero -- a reader has to be able to see that the second leg exists + * and ran: + * + * LEG 1 table → registry: every type named in the table carries the + * registry's verdict for it (and is a type the registry declares). + * LEG 2 registry → table: every registry entry with `allowOrgOverride: true` + * is named somewhere in the table. + * + * Leg 2 is deliberately asymmetric: it demands rows only for the `true` set. + * The table is a WHITELIST -- it enumerates what may be overridden, plus the + * `false` types whose second tier (`allowRuntimeCreate`) is worth calling out. + * Requiring a row for all 27 types would demand a table nobody wants and would + * be the kind of gate that gets deleted rather than obeyed. + * + * ## Why AST, not regex (constraint 2, and the whole reason this file is 400 + * lines instead of a grep) + * + * This is not a style preference. A same-line regex silently UNDER-READS this + * exact registry. Measured on `origin/main` @ `2a6122bd9`: + * + * grep -cE "^ \{ type: '" packages/spec/src/kernel/metadata-plugin.zod.ts -> 26 + * this file's AST walk -> 27 + * + * The missed entry is `datasource`, whose object literal opens `{` on its own + * line, so `type:` and `allowOrgOverride:` land on separate lines. A + * regex-built gate would have been born with one type invisible to it -- and + * invisible in the direction that matters, because an entry the gate never + * parsed can never be reported as an omission from the table. It would have + * been a gate with a permanent, silent hole, printing a green line. + * + * Two more shapes in the same file that a regex reads wrong, both present today: + * + * grep -c 'allowOrgOverride: true' -> 6, but only 5 are ENTRIES; the sixth + * is prose inside a code comment. + * grep -c 'allowOrgOverride: false' -> 33 inside the registry region, but + * only 22 are entries; the other 11 are + * comments discussing the flags. + * + * The AST sees 5 and 22. Comments are not nodes. `--self-test` pins all three + * of these numbers against fixtures, including a fixture whose regex/AST counts + * differ, so the next person to "simplify this to a grep" gets a red battery + * rather than a quiet under-read. + * + * ## Structural findings are RED, never skipped + * + * Every way this gate can lose track of its subject is a finding: + * + * - the `## Overlay whitelist ...` heading is gone or renamed; + * - the table's header row no longer reads `Type | allowOrgOverride | ...` + * (a column swap would otherwise make the gate read the WRONG column and + * keep printing green); + * - a verdict cell holds anything but a bare ✅ / ❌; + * - a type cell holds anything but backticked type names; + * - the registry is not an array literal of object literals with literal + * `type` / `allowOrgOverride` values (a spread, a computed key, an + * identifier flag, a helper call). + * + * That last one is the route-around clause. If the registry ever grows a shape + * this walk cannot read honestly, the gate REFUSES rather than skipping the + * entry -- because skipping is exactly how the regex version would have failed, + * one level up. A gate that cannot read its input must not report "nothing to + * report". + * + * ## Scope: this table only, deliberately + * + * Sibling pages state the same facts correctly in PROSE, not in a machine-shaped + * type→flag table: `content/docs/permissions/authorization.mdx:236` + * ("`permission` declares `allowOrgOverride: false`"), + * `content/docs/automation/jobs.mdx:42` (a job-vs-flow comparison table whose + * flag mention sits inside an English cell), `content/docs/ai/agents.mdx`, + * `content/docs/permissions/capabilities.mdx`, + * `content/docs/references/system/email-template.mdx`. + * + * Measured before deciding: `grep -rn allowOrgOverride content/docs/` hits 22 + * lines across 13 files. Of those, ~5 are genuine " is " assertions; + * the rest are the schema-field reference row, a code sample using an invented + * type, and sentences ABOUT the flag rather than about any type + * (`metadata-lifecycle.mdx:121` itself is one). Covering them mechanically + * means a co-occurrence regex over English plus a per-site allowlist of ~17 + * exceptions to catch ~5 assertions -- a worse gate than none, and the kind + * whose baseline gets bulk-updated to green. They are left out on purpose, and + * this paragraph exists so the next person re-deciding it starts from the count + * rather than from scratch. + */ + +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import ts from 'typescript'; +import { parseSourceFile } from './ts-parse.mjs'; +import { isEntrypoint } from './invoked-as.mjs'; + +const ROOT = new URL('..', import.meta.url).pathname.replace(/\/$/, ''); + +const REGISTRY_FILE = 'packages/spec/src/kernel/metadata-plugin.zod.ts'; +const REGISTRY_CONST = 'DEFAULT_METADATA_TYPE_REGISTRY'; +const DOC_FILE = 'content/docs/concepts/metadata-lifecycle.mdx'; + +/** The section that owns the table. Pinned: if it moves, the gate goes red. */ +const HEADING = '## Overlay whitelist (shared-DB tenancy invariant)'; +/** Any `##` heading -- what ends the section. */ +const ANY_H2 = /^##\s+/; +/** `|:---|:---:|:---|` -- the alignment row that proves the line above is a header. */ +const TABLE_DIVIDER = /^\|[\s:|-]+\|\s*$/; +const TABLE_LINE = /^\s*\|/; + +/** The two columns this gate reads, by name. A rename or reorder is a finding. */ +const COL_TYPE = 'Type'; +const COL_FLAG = 'allowOrgOverride'; + +const YES = '✅'; +const NO = '❌'; + +/** A metadata type name as the registry spells it. */ +const TYPE_NAME = /^[a-z][a-z0-9_]*$/; +/** A backticked token inside a table cell. */ +const BACKTICKED = /`([^`]*)`/g; + +// --------------------------------------------------------------------------- +// Registry side -- AST +// --------------------------------------------------------------------------- + +/** + * Read `DEFAULT_METADATA_TYPE_REGISTRY` by AST. + * + * @returns {{ entries: Array<{type: string, allowOrgOverride: boolean, line: number}>, + * findings: Array<{kind: string, message: string, line: number}> }} + * `findings` non-empty means the registry could not be read honestly. It is + * never "read what we could and carry on" -- see the header. + */ +export function readRegistry(text, fileLabel = REGISTRY_FILE) { + const source = parseSourceFile(fileLabel, text, ts.ScriptKind.TS); + const findings = []; + const entries = []; + const lineOf = (node) => source.getLineAndCharacterOfPosition(node.getStart(source)).line + 1; + + let array = null; + const findDecl = (node) => { + if ( + ts.isVariableDeclaration(node) && + ts.isIdentifier(node.name) && + node.name.text === REGISTRY_CONST + ) { + array = node.initializer ?? null; + return; + } + ts.forEachChild(node, findDecl); + }; + findDecl(source); + + if (array === null) { + findings.push({ + kind: 'structure', + line: 1, + message: `no \`${REGISTRY_CONST}\` declaration with an initializer in this file`, + }); + return { entries, findings }; + } + if (!ts.isArrayLiteralExpression(array)) { + findings.push({ + kind: 'structure', + line: lineOf(array), + message: + `\`${REGISTRY_CONST}\` is no longer a plain array literal (found ` + + `${ts.SyntaxKind[array.kind]}). This gate reads entries positionally; ` + + `it refuses rather than guessing.`, + }); + return { entries, findings }; + } + + for (const element of array.elements) { + const line = lineOf(element); + if (!ts.isObjectLiteralExpression(element)) { + findings.push({ + kind: 'structure', + line, + message: + `registry element is ${ts.SyntaxKind[element.kind]}, not an object literal — ` + + `a spread or computed element cannot be read as a type/flag pair, and ` + + `skipping it is how a type goes invisible to this gate.`, + }); + continue; + } + + let type = null; + let flag = null; + let bad = false; + + for (const prop of element.properties) { + if (!ts.isPropertyAssignment(prop)) { + // A shorthand or spread inside the entry. Only fatal if it could carry + // one of the two keys this gate reads — which, unread, it could. + findings.push({ + kind: 'structure', + line: lineOf(prop), + message: + `registry entry contains a ${ts.SyntaxKind[prop.kind]} member; ` + + `\`type\` / \`${COL_FLAG}\` may be hidden inside it, so the entry cannot be read honestly.`, + }); + bad = true; + continue; + } + const key = ts.isIdentifier(prop.name) + ? prop.name.text + : ts.isStringLiteral(prop.name) + ? prop.name.text + : null; + if (key === null) { + findings.push({ + kind: 'structure', + line: lineOf(prop), + message: 'registry entry has a computed property key; the entry cannot be read honestly.', + }); + bad = true; + continue; + } + if (key === 'type') { + if (!ts.isStringLiteral(prop.initializer)) { + findings.push({ + kind: 'structure', + line: lineOf(prop), + message: '`type` is not a string literal; the entry cannot be read honestly.', + }); + bad = true; + continue; + } + type = prop.initializer.text; + } else if (key === COL_FLAG) { + const kind = prop.initializer.kind; + if (kind !== ts.SyntaxKind.TrueKeyword && kind !== ts.SyntaxKind.FalseKeyword) { + findings.push({ + kind: 'structure', + line: lineOf(prop), + message: + `\`${COL_FLAG}\` is not a \`true\`/\`false\` literal (found ` + + `${ts.SyntaxKind[kind]}); a non-literal flag cannot be compared to a ✅/❌ cell.`, + }); + bad = true; + continue; + } + flag = kind === ts.SyntaxKind.TrueKeyword; + } + } + + if (bad) continue; + if (type === null) { + findings.push({ + kind: 'structure', + line, + message: 'registry entry declares no `type`; the entry cannot be joined to a table row.', + }); + continue; + } + // `allowOrgOverride` is optional in the schema and defaults to false + // (see the reference row for MetadataTypeRegistryEntry). An entry that + // omits it is a legitimate `false`. + entries.push({ type, allowOrgOverride: flag === true, line }); + } + + const seen = new Map(); + for (const e of entries) { + if (seen.has(e.type)) { + findings.push({ + kind: 'structure', + line: e.line, + message: `registry declares \`${e.type}\` twice (also at :${seen.get(e.type)}).`, + }); + } else seen.set(e.type, e.line); + } + + // Vacuity floor. A parse that finds nothing must not read as "nothing to report". + if (findings.length === 0 && entries.length === 0) { + findings.push({ + kind: 'structure', + line: lineOf(array), + message: `\`${REGISTRY_CONST}\` parsed to ZERO entries — the gate has no input and must not report clean.`, + }); + } + if (findings.length === 0 && !entries.some((e) => e.allowOrgOverride)) { + findings.push({ + kind: 'structure', + line: lineOf(array), + message: + `no entry in \`${REGISTRY_CONST}\` has \`${COL_FLAG}: true\` — leg 2 would be ` + + `vacuously green. If the whitelist really did empty out, this table (and ADR-0005) ` + + `need rewriting, not this gate.`, + }); + } + + return { entries, findings }; +} + +// --------------------------------------------------------------------------- +// Doc side -- table rows +// --------------------------------------------------------------------------- + +/** Split a markdown table line into cells (an escaped `\|` is not a separator). */ +export function splitCells(line) { + const parts = line.trim().split(/(? c.trim()); +} + +/** Strip markdown emphasis/backticks to compare a header cell by name. */ +function headerName(cell) { + return cell.replace(/[`*_]/g, '').trim(); +} + +/** + * Read the whitelist table. + * + * @returns {{ rows: Array<{types: string[], allow: boolean, line: number}>, + * findings: Array<{kind: string, message: string, line: number}> }} + */ +export function readTable(text) { + const lines = text.split('\n'); + const findings = []; + const rows = []; + + const headingIdx = lines.findIndex((l) => l.trim() === HEADING); + if (headingIdx === -1) { + findings.push({ + kind: 'structure', + line: 1, + message: + `no \`${HEADING}\` heading in this page. The gate reads the table under that ` + + `heading; if the section was renamed or moved, re-point this gate — a heading ` + + `it cannot find must not read as a table with nothing wrong.`, + }); + return { rows, findings }; + } + + let end = lines.length; + for (let i = headingIdx + 1; i < lines.length; i++) { + if (ANY_H2.test(lines[i])) { + end = i; + break; + } + } + + // The first header+divider pair inside the section. + let headerIdx = -1; + for (let i = headingIdx + 1; i < end - 1; i++) { + if (TABLE_LINE.test(lines[i]) && TABLE_DIVIDER.test(lines[i + 1])) { + headerIdx = i; + break; + } + } + if (headerIdx === -1) { + findings.push({ + kind: 'structure', + line: headingIdx + 1, + message: `the \`${HEADING}\` section contains no markdown table.`, + }); + return { rows, findings }; + } + + const header = splitCells(lines[headerIdx]).map(headerName); + if (header[0] !== COL_TYPE || header[1] !== COL_FLAG) { + findings.push({ + kind: 'structure', + line: headerIdx + 1, + message: + `table header reads [${header.map((h) => `"${h}"`).join(', ')}]; this gate reads ` + + `column 0 as "${COL_TYPE}" and column 1 as "${COL_FLAG}". A reordered or renamed ` + + `column would otherwise be read from the wrong cell, silently.`, + }); + return { rows, findings }; + } + + for (let i = headerIdx + 2; i < end; i++) { + const raw = lines[i]; + if (!TABLE_LINE.test(raw)) { + if (raw.trim() === '') continue; + break; // the table ended; prose follows + } + const cells = splitCells(raw); + const line = i + 1; + if (cells.length < 2) { + findings.push({ kind: 'structure', line, message: 'table row has fewer than two cells.' }); + continue; + } + + // --- type cell: backticked names ONLY, one or more, comma-separated. + const cell = cells[0]; + const names = []; + let m; + BACKTICKED.lastIndex = 0; + while ((m = BACKTICKED.exec(cell)) !== null) names.push(m[1].trim()); + const residue = cell.replace(BACKTICKED, '').replace(/[,\s]/g, ''); + if (names.length === 0 || residue !== '') { + findings.push({ + kind: 'structure', + line, + message: + `type cell ${JSON.stringify(cell)} is not a plain list of backticked type names` + + (residue === '' ? '.' : ` (leftover text ${JSON.stringify(residue)}).`) + + ` Prose here would be read as "no types on this row" and the row would pass unchecked.`, + }); + continue; + } + const badName = names.find((n) => !TYPE_NAME.test(n)); + if (badName !== undefined) { + findings.push({ + kind: 'structure', + line, + message: `type cell names \`${badName}\`, which is not a metadata type name.`, + }); + continue; + } + + // --- verdict cell: exactly ✅ or ❌. + const verdict = cells[1]; + if (verdict !== YES && verdict !== NO) { + findings.push({ + kind: 'structure', + line, + message: + `\`${COL_FLAG}\` cell is ${JSON.stringify(verdict)}; it must be exactly ` + + `"${YES}" or "${NO}". A qualified verdict ("${YES} (mostly)") is a fact this ` + + `gate cannot compare to a boolean.`, + }); + continue; + } + + rows.push({ types: names, allow: verdict === YES, line }); + } + + if (findings.length === 0 && rows.length === 0) { + findings.push({ + kind: 'structure', + line: headerIdx + 1, + message: 'the whitelist table parsed to ZERO rows — the gate has no input and must not report clean.', + }); + } + + return { rows, findings }; +} + +// --------------------------------------------------------------------------- +// The two legs +// --------------------------------------------------------------------------- + +/** + * @returns {{ leg1: Array, leg2: Array, tableTypes: Map }} + */ +export function compare(entries, rows) { + const registry = new Map(entries.map((e) => [e.type, e])); + const tableTypes = new Map(); + const leg1 = []; + const leg2 = []; + + for (const row of rows) { + for (const type of row.types) { + if (tableTypes.has(type)) { + leg1.push({ + kind: 'duplicate', + type, + line: row.line, + message: + `\`${type}\` is listed twice in the table (also at :${tableTypes.get(type).line}) — ` + + `two rows can disagree, and a reader has no way to know which is authoritative.`, + }); + continue; + } + tableTypes.set(type, row); + + const entry = registry.get(type); + if (entry === undefined) { + leg1.push({ + kind: 'unknown-type', + type, + line: row.line, + message: + `the table lists \`${type}\`, which \`${REGISTRY_CONST}\` does not declare — ` + + `either the type was retired and the row outlived it, or the name is misspelt.`, + }); + continue; + } + if (entry.allowOrgOverride !== row.allow) { + leg1.push({ + kind: 'mismatch', + type, + line: row.line, + message: + `the table says \`${type}\` is ${row.allow ? YES : NO}, the registry declares ` + + `\`${COL_FLAG}: ${entry.allowOrgOverride}\` (${REGISTRY_FILE}:${entry.line}).`, + }); + } + } + } + + for (const entry of entries) { + if (entry.allowOrgOverride && !tableTypes.has(entry.type)) { + leg2.push({ + kind: 'missing-row', + type: entry.type, + line: entry.line, + message: + `\`${entry.type}\` is \`${COL_FLAG}: true\` (${REGISTRY_FILE}:${entry.line}) and is ` + + `named nowhere in the table — an overridable type the whitelist page does not ` + + `disclose. This is the direction a table→registry gate cannot see.`, + }); + } + } + + return { leg1, leg2, tableTypes }; +} + +// --------------------------------------------------------------------------- +// Self-test -- the non-vacuity battery +// --------------------------------------------------------------------------- + +/** + * A registry fixture that reproduces every shape the real file has, INCLUDING + * the multi-line entry that defeats a same-line regex and the comment lines + * that inflate a naive `grep -c`. + */ +const FIXTURE_REGISTRY = ` +export const ${REGISTRY_CONST}: MetadataTypeRegistryEntryParsed[] = [ + // Pure rendering. ${COL_FLAG}: true is safe here — this comment is PROSE and + // must not be counted as an entry (${COL_FLAG}: true appears twice in it). + { type: 'view', label: 'View', supportsOverlay: true, ${COL_FLAG}: true, loadOrder: 50 }, + { type: 'dashboard', label: 'Dashboard', supportsOverlay: true, ${COL_FLAG}: true, loadOrder: 60 }, + { type: 'report', label: 'Report', supportsOverlay: true, ${COL_FLAG}: true, loadOrder: 60 }, + { type: 'translation', label: 'Translation', supportsOverlay: true, ${COL_FLAG}: true, loadOrder: 90 }, + { type: 'email_template', label: 'Email Template', supportsOverlay: true, ${COL_FLAG}: true, loadOrder: 85 }, + // Rolled back (#6283): ${COL_FLAG}: false, and this line is a comment too. + { type: 'flow', label: 'Flow', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 80 }, + { type: 'permission', label: 'Permission Set', supportsOverlay: true, ${COL_FLAG}: false, loadOrder: 15 }, + { type: 'position', label: 'Position', supportsOverlay: true, ${COL_FLAG}: false, loadOrder: 15 }, + { type: 'agent', label: 'AI Agent', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 90 }, + { type: 'object', label: 'Object', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 10 }, + { type: 'field', label: 'Field', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 20 }, + // Declared by the registry, deliberately NOT named in the table fixture: the + // table is a whitelist, so leg 2 must not demand a row for a false type. + { type: 'hook', label: 'Hook', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 30 }, + { type: 'job', label: 'Background Job', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 80 }, + // THE SHAPE A REGEX MISSES: opens \`{\` on its own line, so \`type:\` and + // \`${COL_FLAG}:\` land on separate lines from the brace. + { + type: 'datasource', + label: 'Datasource', + supportsOverlay: false, + ${COL_FLAG}: false, + loadOrder: 5, + }, +]; +`; + +/** The table as it stood BEFORE #11750 — the ready-made positive control. */ +const FIXTURE_TABLE_BEFORE = ` +${HEADING} + +Prose above the table. + +| Type | \`${COL_FLAG}\` | Rationale | +| :--- | :---: | :--- | +| \`view\`, \`dashboard\`, \`report\`, \`email_template\` | ${YES} | Pure rendering. | +| \`flow\` | ${YES} | Per-org overlays are allowed for automation definitions. | +| \`agent\` | ${NO} | Platform-owned (ADR-0063 §2). | +| \`permission\`, \`position\` | ${YES} | Tenant-level controls layer on top. | +| \`object\` | ${NO} | Defines the table schema. | +| \`field\` | ${NO} | Authored inside the object. | +| \`datasource\` | ${NO} | Connection strings. | +| \`job\` | ${NO} | Handler lives in the compiled bundle. | + +Prose after the table. + +## Next section +`; + +/** The table as #11750 left it. */ +const FIXTURE_TABLE_AFTER = FIXTURE_TABLE_BEFORE.replace( + `| \`view\`, \`dashboard\`, \`report\`, \`email_template\` | ${YES} |`, + `| \`view\`, \`dashboard\`, \`report\`, \`email_template\`, \`translation\` | ${YES} |`, +) + .replace(`| \`flow\` | ${YES} |`, `| \`flow\` | ${NO} |`) + .replace(`| \`permission\`, \`position\` | ${YES} |`, `| \`permission\`, \`position\` | ${NO} |`); + +function run(registryText, tableText) { + const reg = readRegistry(registryText, 'fixture.ts'); + const tab = readTable(tableText); + if (reg.findings.length || tab.findings.length) { + return { structure: [...reg.findings, ...tab.findings], leg1: [], leg2: [], entries: reg.entries }; + } + const { leg1, leg2 } = compare(reg.entries, tab.rows); + return { structure: [], leg1, leg2, entries: reg.entries, rows: tab.rows }; +} + +export function selfTest() { + const failures = []; + const check = (name, ok, detail = '') => { + if (!ok) failures.push(`${name}${detail ? ` — ${detail}` : ''}`); + }; + + // ---- 1. THE POSITIVE CONTROL: the pre-#11750 table must produce exactly 4. + const before = run(FIXTURE_REGISTRY, FIXTURE_TABLE_BEFORE); + check('before/structure-clean', before.structure.length === 0, JSON.stringify(before.structure)); + const beforeTotal = before.leg1.length + before.leg2.length; + check('before/total=4', beforeTotal === 4, `got ${beforeTotal}`); + check('before/leg1=3', before.leg1.length === 3, `got ${before.leg1.length}`); + check('before/leg2=1', before.leg2.length === 1, `got ${before.leg2.length}`); + const beforeL1 = before.leg1.map((f) => `${f.kind}:${f.type}`).sort(); + check( + 'before/leg1 names flow,permission,position as mismatches', + JSON.stringify(beforeL1) === + JSON.stringify(['mismatch:flow', 'mismatch:permission', 'mismatch:position']), + JSON.stringify(beforeL1), + ); + check( + 'before/leg2 names translation as the omission', + before.leg2.length === 1 && + before.leg2[0].type === 'translation' && + before.leg2[0].kind === 'missing-row', + JSON.stringify(before.leg2.map((f) => `${f.kind}:${f.type}`)), + ); + + // ---- 2. The corrected table must be green on BOTH legs. + const after = run(FIXTURE_REGISTRY, FIXTURE_TABLE_AFTER); + check('after/structure-clean', after.structure.length === 0, JSON.stringify(after.structure)); + check('after/leg1=0', after.leg1.length === 0, JSON.stringify(after.leg1)); + check('after/leg2=0', after.leg2.length === 0, JSON.stringify(after.leg2)); + + // ---- 3. A ONE-LEGGED gate would have shipped 3 of the 4. Pinned so that + // deleting leg 2 cannot pass this battery. + check('leg2 is load-bearing (3 != 4)', before.leg1.length !== beforeTotal); + + // ---- 4. AST vs REGEX on the fixture: the numbers must differ, in the + // direction the header documents. + const regexEntryCount = (FIXTURE_REGISTRY.match(/^ {2}\{ type: '/gm) || []).length; + const astEntryCount = before.entries.length; + check('fixture/ast=14', astEntryCount === 14, `got ${astEntryCount}`); + check( + 'fixture/regex under-reads by exactly the multi-line entry', + regexEntryCount === astEntryCount - 1, + `regex ${regexEntryCount}, ast ${astEntryCount}`, + ); + check( + 'fixture/AST reads the multi-line `datasource` entry', + before.entries.some((e) => e.type === 'datasource' && e.allowOrgOverride === false), + ); + + // ---- 5. Comments are not entries. `grep -c` over the fixture inflates both + // flags; the AST count must be the smaller, correct one. + const grepTrue = (FIXTURE_REGISTRY.match(new RegExp(`${COL_FLAG}: true`, 'g')) || []).length; + const astTrue = before.entries.filter((e) => e.allowOrgOverride).length; + check('fixture/ast true-set = 5', astTrue === 5, `got ${astTrue}`); + check( + 'fixture/grep over-counts true because of comment prose', + grepTrue > astTrue, + `grep ${grepTrue}, ast ${astTrue}`, + ); + + // ---- 6. MULTI-TYPE CELL control: five names in one cell become five types. + const multi = readTable(FIXTURE_TABLE_AFTER); + const rowWith5 = multi.rows.find((r) => r.types.length === 5); + check( + 'multi-type cell yields 5 separate types', + rowWith5 !== undefined && + JSON.stringify(rowWith5.types) === + JSON.stringify(['view', 'dashboard', 'report', 'email_template', 'translation']), + JSON.stringify(rowWith5 && rowWith5.types), + ); + const flatCount = multi.rows.reduce((n, r) => n + r.types.length, 0); + check('after/table names 13 types across 8 rows', flatCount === 13 && multi.rows.length === 8, + `${flatCount} types / ${multi.rows.length} rows`); + + // ---- 7. A mismatch inside a MULTI-TYPE cell is caught per type, not per row. + const oneWrongInCell = FIXTURE_TABLE_AFTER.replace( + `| \`view\`, \`dashboard\`, \`report\`, \`email_template\`, \`translation\` | ${YES} |`, + `| \`view\`, \`dashboard\`, \`report\`, \`email_template\`, \`translation\`, \`hook\` | ${YES} |`, + ); + const cellRes = run(FIXTURE_REGISTRY, oneWrongInCell); + check( + 'a wrong type inside a 6-name cell is caught, per type not per row', + cellRes.leg1.length === 1 && cellRes.leg1[0].kind === 'mismatch' && cellRes.leg1[0].type === 'hook', + JSON.stringify(cellRes.leg1.map((f) => `${f.kind}:${f.type}`)), + ); + + // ---- 8. STRUCTURAL refusals. Each must be RED, none may read as clean. + const structural = [ + ['renamed heading', FIXTURE_TABLE_AFTER.replace(HEADING, '## Overlay whitelist')], + ['no table', `${HEADING}\n\nJust prose now.\n\n## Next\n`], + [ + 'swapped columns', + FIXTURE_TABLE_AFTER.replace(`| Type | \`${COL_FLAG}\` | Rationale |`, `| \`${COL_FLAG}\` | Type | Rationale |`), + ], + [ + 'renamed column', + FIXTURE_TABLE_AFTER.replace(`| Type | \`${COL_FLAG}\` | Rationale |`, `| Type | \`overridable\` | Rationale |`), + ], + ['qualified verdict', FIXTURE_TABLE_AFTER.replace(`| \`agent\` | ${NO} |`, `| \`agent\` | ${NO} (mostly) |`)], + ['prose in type cell', FIXTURE_TABLE_AFTER.replace('| `agent` |', '| everything else |')], + ['non-type name in cell', FIXTURE_TABLE_AFTER.replace('| `agent` |', '| `Agent Type` |')], + ]; + for (const [name, text] of structural) { + const res = readTable(text); + check(`structure/${name} is refused`, res.findings.length > 0, 'read clean'); + } + + // ---- 9. REGISTRY structural refusals -- the route-around clause. + const registryStructural = [ + ['spread element', FIXTURE_REGISTRY.replace(" { type: 'agent'", ' ...EXTRA_ENTRIES,\n { type: \'agent\'')], + [ + 'non-literal flag', + FIXTURE_REGISTRY.replace(`{ type: 'agent', label: 'AI Agent', supportsOverlay: false, ${COL_FLAG}: false`, + `{ type: 'agent', label: 'AI Agent', supportsOverlay: false, ${COL_FLAG}: AGENT_OVERRIDE`), + ], + [ + 'computed type value', + FIXTURE_REGISTRY.replace("{ type: 'agent'", '{ type: TYPES.agent'), + ], + ['not an array', `export const ${REGISTRY_CONST}: X[] = buildRegistry();`], + ['no declaration', 'export const SOMETHING_ELSE = [];'], + ['empty array', `export const ${REGISTRY_CONST}: X[] = [];`], + [ + 'no true-set left', + FIXTURE_REGISTRY.replace(new RegExp(`${COL_FLAG}: true`, 'g'), `${COL_FLAG}: false`), + ], + ]; + for (const [name, text] of registryStructural) { + const res = readRegistry(text, 'fixture.ts'); + check(`registry/${name} is refused`, res.findings.length > 0, 'read clean'); + } + + // ---- 10. A duplicated type in the table is caught. + const dup = run( + FIXTURE_REGISTRY, + FIXTURE_TABLE_AFTER.replace(`| \`job\` | ${NO} |`, `| \`job\` | ${NO} |\n| \`job\` | ${NO} |`), + ); + check( + 'duplicate table row is caught', + dup.leg1.some((f) => f.kind === 'duplicate' && f.type === 'job'), + JSON.stringify(dup.leg1.map((f) => f.kind)), + ); + + // ---- 11. A retired type left behind in the table is caught. + const retired = run(FIXTURE_REGISTRY, FIXTURE_TABLE_AFTER.replace('| `job` |', '| `validation` |')); + check( + 'a table row for an undeclared type is caught', + retired.leg1.some((f) => f.kind === 'unknown-type' && f.type === 'validation'), + JSON.stringify(retired.leg1.map((f) => `${f.kind}:${f.type}`)), + ); + + // ---- 12. An entry that OMITS the optional flag counts as false, not as a + // structural refusal (the schema's documented default). + const omitted = readRegistry( + FIXTURE_REGISTRY.replace(`{ type: 'agent', label: 'AI Agent', supportsOverlay: false, ${COL_FLAG}: false, loadOrder: 90 }`, + `{ type: 'agent', label: 'AI Agent', supportsOverlay: false, loadOrder: 90 }`), + 'fixture.ts', + ); + check('omitted flag defaults to false', omitted.findings.length === 0 && + omitted.entries.some((e) => e.type === 'agent' && e.allowOrgOverride === false), + JSON.stringify(omitted.findings)); + + if (failures.length > 0) { + console.error('\n✗ check-overlay-whitelist-table self-test failed:\n'); + for (const f of failures) console.error(` - ${f}`); + console.error(''); + process.exit(1); + } + console.log( + '✓ check-overlay-whitelist-table self-test: positive control reproduces the 4 known ' + + 'divergences (leg 1: flow, permission, position; leg 2: translation), the corrected ' + + 'table is green on both legs, and 21 structural/parser cases are refused.', + ); +} + +// --------------------------------------------------------------------------- +// main +// --------------------------------------------------------------------------- + +function main() { + if (process.argv.includes('--self-test')) return selfTest(); + + const registryText = readFileSync(join(ROOT, REGISTRY_FILE), 'utf8'); + const docText = readFileSync(join(ROOT, DOC_FILE), 'utf8'); + + const reg = readRegistry(registryText); + const tab = readTable(docText); + + const structural = [ + ...reg.findings.map((f) => ({ ...f, file: REGISTRY_FILE })), + ...tab.findings.map((f) => ({ ...f, file: DOC_FILE })), + ]; + + if (structural.length > 0) { + console.error('\n✗ the overlay whitelist could not be READ — refusing to report a verdict:\n'); + for (const f of structural) console.error(` ${f.file}:${f.line} [${f.kind}] ${f.message}`); + console.error( + '\nA gate that cannot read its input must not print a clean line: that is exactly\n' + + 'the failure this gate exists to prevent, one level up. Re-point the gate, or\n' + + 'restore the shape it reads.\n', + ); + process.exit(1); + } + + const { leg1, leg2, tableTypes } = compare(reg.entries, tab.rows); + const trueSet = reg.entries.filter((e) => e.allowOrgOverride).map((e) => e.type); + + if (process.argv.includes('--list')) { + console.log(`${REGISTRY_FILE} — ${reg.entries.length} types, ${trueSet.length} overridable:\n`); + for (const e of reg.entries.slice().sort((a, b) => a.type.localeCompare(b.type))) { + const row = tableTypes.get(e.type); + console.log( + ` ${(e.allowOrgOverride ? YES : NO)} ${e.type.padEnd(20)} ` + + `registry:${String(e.line).padEnd(5)} ` + + `${row ? `table:${row.line}` : 'table: (not listed)'}`, + ); + } + console.log(''); + } + + if (leg1.length === 0 && leg2.length === 0) { + console.log( + `✓ ${DOC_FILE}: the overlay whitelist table agrees with ${REGISTRY_CONST} in both ` + + `directions — leg 1 (table → registry) 0 divergence(s) over ` + + `${tableTypes.size} type(s) named in ${tab.rows.length} row(s); leg 2 ` + + `(registry → table) 0 divergence(s) over ${trueSet.length} \`${COL_FLAG}: true\` ` + + `type(s) [${trueSet.join(', ')}] out of ${reg.entries.length} declared.`, + ); + return; + } + + console.error(`\n✗ ${DOC_FILE} — the overlay whitelist table disagrees with ${REGISTRY_CONST}:\n`); + console.error(` LEG 1 table → registry: ${leg1.length} divergence(s)`); + for (const f of leg1) console.error(` ${DOC_FILE}:${f.line} [${f.kind}] ${f.message}`); + console.error(`\n LEG 2 registry → table: ${leg2.length} divergence(s)`); + for (const f of leg2) console.error(` ${REGISTRY_FILE}:${f.line} [${f.kind}] ${f.message}`); + console.error(` +VERDICT: ${leg1.length + leg2.length} DIVERGENCE(S) + +⛔ Fix the TABLE, not the registry. \`${REGISTRY_CONST}\` is the single + machine-readable whitelist and the page says so in its own first sentence. + Flipping a flag there to make this gate green is an ADR-0005 whitelist + CHANGE — it requires the ADR revised, not a file edit + (scripts/adr-anchors/packages__spec__src__kernel__metadata-plugin.zod.ts.json). + A gate "fixed" by editing the thing it measures has inverted its own point. + + leg 1 [mismatch] the row's ${YES}/${NO} contradicts the registry -> correct the cell. + leg 1 [unknown-type] the table names a type the registry does not declare -> + the type was retired and the row outlived it, or it is misspelt. + leg 1 [duplicate] the same type is listed on two rows -> keep one. + leg 2 [missing-row] an overridable type the table never names -> add it. This is + the \`translation\` shape from #11664: invisible to any gate + that only checks the rows it can see. +`); + process.exit(1); +} + +// Exports bindings, so an import for those exports alone must run nothing (#10667). +if (isEntrypoint(import.meta.url)) { + main(); +}