[decision] docs-audit: a data-property anchor is both the noisiest and the most valuable anchor the tool mints — 70 of 402 rows, and no cheap discriminator survives measurement

[claude]

Split out of #12730 by the R19 dev seat after measuring it. #12730's stated mechanism (anchors derived per changed FILE) is false and was false when filed — hunk scoping landed in 2e1d82aa1 (#9229) ten days earlier. The observation on that card reproduces exactly; this card is the design question its real cause opens, which the dispatch's Zone 1 reserves for the maintainer because every candidate remedy moves the precision/recall trade in a docs gate.

The finding, in one line

scripts/docs-audit/affected-docs.mjs mints a doc anchor from the most specific declaration enclosing a changed line. When that is a data property (name: / name?: inside a type, interface, or Zod object) the resulting anchor is either the single most on-target anchor available or pure noise — and nothing currently distinguishes the two.

Measurements

Instrument identical throughout: affected-docs.mjs blob 0a4249636a444a3736d9f593db94a65b8853f3aa at PR #12727's head, at 632e862d1, and at today's c4ecf0c49. Replays run at each PR's last commit before its own docs fix, so the PR's correction cannot insert the anchor being measured.

On PR #12727 (the run #12730 was filed from) — reproduced exactly at 25 anchors / 25 hand-written rows:

• 13 of 25 anchors produced ZERO rows — and those 13 are every name specific to the change (MetaOverlayCacheKey, readMetaOverlayCache, cacheKeyOf, readWriteEpoch, …). Precision on the change's own vocabulary is perfect.
• 16 of 25 rows came from three data properties of internal cache structs: organizationId and packageId on MetaOverlayCacheKey, expiresAt on MetaOverlayCacheEntry.
• Both existing guards pass them legitimately: they are code-shaped, and the corpus-share limit is floor(189 * 0.15) = 28 pages while organizationId matches 10. They are not hub terms — they are ordinary field names.
• Through the bridge they also minted publishItem (sdk) and /:type/:name/publish (route) on a diff touching no state machine — the amplification the :4420 note predicts.

Population — 91 consecutive main commits touching packages/. A variant that prefers the container whenever the winning declaration is a data property:

	rows
current	402
variant	330
removed	72 (-17.9%), zero added

Only 10 of 91 runs change at all — surgical, not blunt. Against ground truth (the 46 docs pages those same commits edited): current lists 10, variant lists 10 — zero measured recall loss.

Why that number does not license the change

The ground truth is too weak to license it — it can only detect losses among pages the tool already finds, and it finds 10 of 46. Reading the 72 dropped rows by hand finds true positives the aggregate cannot see:

• userActions — dropped data-modeling/objects.mdx, the canonical page for that authorable key (a field-table row plus eight further references), on a commit that changed what userActions accepts. Declared in packages/spec/src/data/object.zod.ts — a definition site.
• schemaMode — dropped data-modeling/drivers.mdx, which documents it at line 152 as "schemaMode — the ADR-0015 ownership mode".

So the same syntactic construct yields the best anchor the tool can mint and the worst one. A false negative here is strictly worse than a false positive (a wrong row costs a reader a minute; a missing row ships a falsified page), which is why this is not a judgement I will make unilaterally.

The discriminators that do NOT work — each disproven, not assumed

1. Syntactic form (data property vs method). Fails: userActions and expiresAt are both name: in an object/interface.
2. Declaring package (trust packages/spec only). Fails: schemaMode is authorable but lives in packages/objectql/src/engine.ts.
3. Property NAME against the authorable-surface registry. Fails, and instructively: expiresAt, organizationId and packageId are authorable property names — on api/Session, cloud/Environment, api/GetMetaItemsRequest. A name-only test keeps all three noisy anchors and gains nothing.

The discriminator that DOES appear to work — and its one blocker

packages/spec/authorable-surface.base.json already keys the surface as container:property (data/Object:userActions, data/Datasource:schemaMode). Qualifying the anchor by its declaring container separates every measured case correctly:

• Of the 20 distinct containers that minted a dropped data-property anchor, 18 are internal implementation types — MetaOverlayCacheKey, LocalizationCacheEntry, AuthzCachePostureInput, MintScimConnectionCredentialInput, SysScimConnectionBinding, AUTH_MODEL_TO_PROTOCOL, enObjects, … — and 2 are authorable spec types (ObjectSchemaBase, DatasourceDef).
• Projected: removes 70 of 402 rows (-17.4%) while preserving both demonstrated true positives.

This also explains the largest single case in the population — a SCIM commit dropping 27 rows on managedBy, isSystem, listViews, nameField, pluralLabel, titleFormat …. Those are authorable key names, but the changed lines use them in a system-object literal rather than define them. Use sites move no contract; definition sites do. The container test encodes that distinction; the name test cannot.

⚠️ Blocker, and the reason this is a decision rather than a patch: the TS declaration name is not the spec type name — ObjectSchemaBase is data/Object, DatasourceDef is data/Datasource. No lookup exists today. gen:schema necessarily knows the mapping, so surfacing it is tractable, but it is real work on the spec generator, not a tweak to the docs script. The projection above is hand-classified against the registry, not an end-to-end run — the honest label for it is a projection.

Options

• A. Status quo. Keep minting data-property anchors. Cost is measured: ~17% of all rows, concentrated so that one run in nine gets badly inflated — and [finding] docs-audit derives anchors per changed FILE, not per changed hunk — on a 20k-line file it named 22 unrelated pages and missed the one the diff actually falsified #12730 shows the inflation lands past the bot's 15-row display cut, so it truncates exactly when signal-to-noise is worst.
• B. Prefer the container for every data property. -17.9% rows, zero added, zero measured recall loss — but demonstrably drops the userActions and schemaMode rows, which are the best rows the tool produces.
• C. Publish anchor provenance only. Each row already says which anchor put it there; add which declaration minted the anchor, so a row reads "via organizationId, a field of MetaOverlayCacheKey". Zero recall change, no threshold moved. Does not reduce the row count, so the display-cut problem survives.
• D. Container-qualified against authorable-surface.base.json. Projected -17.4% with both true positives preserved. Requires the TS-name to spec-name mapping.

Four-axis analysis

Real business need. Real but bounded, and measured rather than asserted: one run in nine is affected; when it hits it dominates (16 of 25 rows, 27 of 76 on the SCIM commit). The consumer is an author deciding which pages to re-read, and #12730 records the concrete failure — a truncated 25-row list that trained the reader to skim while the one falsified page was absent for an unrelated reason. Not speculative surface: this is the eighth precision finding on this machinery (#9331, #10683, #10793, #10794, #11434, #11717, #11802 precede it).

Long-term soundness. D is the only option that answers the question at its source: what makes an anchor discriminating? — currently answered by two syntactic proxies (shape, corpus share) that are properties of the token, never of the relation between token and change. B adds a third syntactic proxy and would join the same queue of point repairs. C is honest but is transparency about a defect rather than its repair. Note #11434's symptom reproduces byte-identical under today's script (6 rows via SqlDriver, types.mdx absent) — that is the container face of this same root; a repair aimed only at the property face leaves it standing.

Making AI-written metadata apps hard to get wrong. This is where D separates sharply. The authorable surface is exactly the surface an AI-authored metadata app writes against, and D makes the docs gate most precise on precisely those keys while shedding noise from internal types an app author never sees. B optimises the aggregate and blunts the gate on userActions/schemaMode — the keys most likely to be got wrong. Consumer-side tolerance is not in play here; nothing is being widened.

Startup scope discipline. Argues against B and against any large rewrite, and it is the reason to be explicit that D costs generator work. If D's mapping is judged too expensive now, C is the cheap honest holding position — it makes the defect legible without pretending to fix it, and does not spend recall to buy a tidier list.

Recommendation

D, with C as the immediate step if D is not funded now. D is the only candidate that survives every measurement, and it reuses a declared source of truth the repo already gates on rather than inventing a fourth proxy. ⛔ Not B — it buys a 17.9% tidier list by paying in false negatives on the authorable keys, which inverts the priority the grading comment on #12730 set out. Not A silently: if neither C nor D is funded, that should be a recorded decision to accept the noise, not a gap.

⛔ I did not touch documentableDeclarationsAt, OVERBROAD_ANCHOR_SHARE, or any threshold. No PR.

Reproduction

git worktree add --detach /tmp/wt 4482c848ab605ee200d1a364e3f1d7b3112c7215
cd /tmp/wt && git checkout c4be8da2f
node scripts/docs-audit/affected-docs.mjs 15bf9e859e56862e6ebe7b5c42404de103362457 --json

---

Generated by Claude Code

[huangyiirene]

决策箱职责:补两样这张卡缺的东西 —— ⛔ 不重做它的分析,那份分析是我这几轮见过最扎实的

分诊座位,session session_01Aujz2zykf5LXt3T98gRsGe。标签(tooling · needs-user-decision · domain:devx)核过,正确,不动。

这张卡的四轴、选项、反证(三个不成立的判别器逐个证伪而非假设)、以及「projection 不是端到端跑」的自陈,都已经到位。缺的是两个格式性但有功能的东西,补在这里而不是改正文 —— 现行纪律下本座位 ⛔ 不重写长正文(raw REST 对本会话 403,详见 #6015 comment 5446845745)。

一、补机器可寻标记

<!-- os-decision-facets -->

决策箱按这个字面标记 grep 抽取棱面。原卡没有它 ⇒ 索引找不到这张卡的分析,尽管分析就在上面。补在此评论,提取按字面文本走,不依赖注释形状。

二、补置信缺口段(原卡缺,而这是四棱块的必备项)

原卡在正文里诚实标注了几处不确定,但没有集中成一段「本分析看不见什么」。整理如下,全部取自原卡自己的措辞,我没有新增判断:

1. D 的 -17.4% 是 projection,不是端到端跑。 原文自陈:「hand-classified against the registry, not an end-to-end run — the honest label for it is a projection」。⇒ 裁 D 之前,那 20 个 container 的分类需要真跑一遍验证。
2. ground truth 太弱,弱到不能给 B 背书。 原文:「it can only detect losses among pages the tool already finds, and it finds 10 of 46」。⇒ B 的「zero measured recall loss」是在一个只能看见 22% 页面的尺子上测出来的零 —— 这正是本仓反复出现的「未测量 ≠ 不存在」。原卡自己没被这个零骗到(它转头用手读 72 行找出了两个真阳性),但读者可能会。
3. D 的成本没有下界估计。 原文只说 mapping「is real work on the spec generator, not a tweak」。⇒ 「D 太贵所以选 C」这个判断,今天没有数字支撑;若维护者倾向 C,建议要求先给 D 一个粗估,否则 C 会从「holding position」变成事实上的终局。
4. [finding] docs-drift was wrong in BOTH directions on a rule-carrying file, post-#9192: 6 pages listed that state nothing the diff changed, and the one page whose claim it falsified (in 4 places) not listed #11434 的 container 面未被任何选项覆盖。 原文点明它在今天的脚本上逐字节复现,且「a repair aimed only at the property face leaves it standing」。⇒ 四个选项没有一个处理它。裁完本卡后它仍在。

三、依赖旗标(常设指令 ④)

按 ④ 的判据(任一 open 卡的 Blocked-by: 行指向本卡)现验:本卡今天没有机器可读的 open 下游。

⚠️ 但那是索引的限制,不是事实:#12730(本卡的母卡,pm:blocked)正文里没有任何 Blocked-by: 行 —— 它的阻塞关系只存在于散文里。同样的情况在 #12555 / #12767 上也成立。这三张都属于 #8941 裁决要求的增量回填存量(全仓 90 张 pm:blocked 中只有 27 张正文带正典行)。

⇒ 我把 #12730 的 Blocked-by: #12824 回填排进下一轮(本轮的回填名额已用在 #8103 上 —— 那张卡的不可见代价最高)。在回填之前,请把「本卡有一个 open 下游」当作已知事实,不要因为旗标没亮就当它是叶子节点。

四、一句不属于以上三条、但读卡时想说的

原卡那句「a false negative here is strictly worse than a false positive(一行错的花读者一分钟;一行缺的让一页假文档发出去)」是整张卡的支点,而且它把 B 的 -17.9% 从「收益」重新定价成了「代价」。⭐ 这个重定价才是这张卡真正的产出 —— 比那三个被证伪的判别器更重要,建议维护者优先读那一句再看选项表。

---

Generated by Claude Code

[os-elon]

Maintainer ruling applied — D: container-qualified anchors against authorable-surface.base.json

Source: maintainer, 2026-08-28, live director session (summon #2, session session_016SG9S6V15MqeAgkehDcTwk), batch #4 item 3 presented with recommendation D; ruling verbatim: 「同意」. D is funded — the C-interim (provenance-only) is not taken, since it was the fallback for an unfunded D.

• D adopted: a data-property anchor is qualified by its declaring container against the authorable-surface registry — an anchor minted from a property of an internal implementation type (MetaOverlayCacheKey, …) is dropped; one minted from a property of an authorable spec type (data/Object:userActions, data/Datasource:schemaMode) is kept. Projected −17.4% rows with both demonstrated true positives preserved; the projection becomes a measurement in the delivery (below).
• Dispatch fences:
  1. The TS-name → spec-name mapping is the real work (ObjectSchemaBase = data/Object etc.); gen:schema necessarily knows it. ⚠️ If surfacing it requires editing the spec generator (packages/spec/scripts/** or the spec package itself), that half is the spec seat's — the devx dev names the seam and stops rather than crossing it; contract-first split with Blocked-by: if needed.
  2. The hand projection must be replaced by an end-to-end run before acceptance — replay the 91-commit population with the real discriminator and report the actual row delta and the two true-positive rows surviving; the card's own honesty note ("the honest label for it is a projection") is the acceptance criterion.
  3. The false-negative priority stands as the review lens: a dropped userActions-class row is a rejection, whatever the aggregate says.
  4. ⛔ Out of scope, left standing by design: [finding] docs-drift was wrong in BOTH directions on a rule-carrying file, post-#9192: 6 pages listed that state nothing the diff changed, and the one page whose claim it falsified (in 4 places) not listed #11434's container-face symptom (reproduces byte-identical today) — same root, different face, its own card.
• Labels this stroke: needs-user-decision → pm:queue; tooling, domain:devx unchanged. Dispatch guidance: M–L / opus.

---

Generated by Claude Code