Skip to content

The docs-accuracy audit verifies that a documented type exists and matches, but not that it is the type the page's seam resolves to — it passed audit-service.mdx with "None" #9590

Description

@os-steve

Observation found while implementing #9534. Not a defect in any shipped surface — a gap in the docs-accuracy audit method, filed so the class is on record rather than rediscovered.

The measurement

docs/audits/2026-06-handwritten-docs-accuracy-followups.md:226 records the verdict for the page #9534 later turned out to be wrong about:

guides/runtime-services/audit-service.mdx

  • None. Independently re-verified every claim against live code: (1) canonical source packages/services/service-settings/src/settings-service.types.ts exists; (2) doc record(entry) signature lines 16-26 matches SettingsAuditSink.record EXACTLY [...] fixCount 0 was correct; no edits were made so no regression is possible.

Every one of those statements was true. The signature did match SettingsAuditSink.record field-for-field. The canonical source did exist. And the page was still wrong, because it is filed under runtime-services/ and written as services.audit — and the kernel slot named audit resolves to a different type entirely (recordAuthEvent / 'login' | 'logout', from plugin-audit).

The gap

The audit answers "does the documented symbol exist, and does its shape match the type the page names?" It does not answer "is the type the page names the one this page's seam actually hands you?"

That second question is the one a reader's getService('audit') asks. Nothing in the pass connected the page's location and accessor (runtime-services/, services.audit) back to registerService('audit', ...).

This defect class is specifically invisible to the checks that exist:

  • a grep for phantom identifiers finds nothing — every symbol on the page was real;
  • a signature diff against the named type passes — it was an exact match;
  • check:docs-audit-scope confirms the page is in scope, not that the verdict was right.

The audit also recorded, correctly, that services.* is a chapter-wide descriptive convention rather than a literal registry key (see the versioning.mdx entry: services.storage maps to the real key file-storage). That ruling is sound, and it is also what made the seam question look answered — the accessor was classified as notation, so nothing went looking for a slot behind it.

Possible directions (not a recommendation)

  • For pages under runtime-services/, resolve the accessor to a registration site and check the documented type against that registrant's type. This is mechanizable for the subset where services.X does correspond to a literal key.
  • Where the accessor is genuinely only notation (services.storagefile-storage), the mapping itself is worth declaring somewhere a check can read, instead of living in an audit note.
  • At minimum, record "which seam does this page describe" as an explicit question in the audit method, so a "None" verdict states that it was asked.

No claim here about which is worth building — filing the class, not the fix.

Refs: #9534 · PR #9587 · docs/audits/2026-06-handwritten-docs-accuracy-followups.md

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions