Skip to content

finding: the docs-drift check maps docs→packages by textual mention, so a doc that describes a package's behaviour without naming it is invisible — kernel/services-checklist.mdx vs @objectstack/runtime #7967

Description

@hotlong

Filed unassigned and without a domain:* label — routing and grading are the triage seat's call. Observation-class, but with a measured miss behind it.

What was measured

PR #7965 changes exactly one thing in @objectstack/runtime: the handlerReady value that getDiscoveryInfo()'s services.search entry reports for a filled slot. The docs-drift check on that PR listed 20 hand-written docs as possibly affected — and not the one document that actually specifies handlerReady:

content/docs/kernel/services-checklist.mdx documents the services map in detail — the handlerReady default table (:192), the __serviceInfo: { status, handlerReady?, message? } self-description contract (:187), the rule that "Only handlerReady is consulted, never status" (:197), the stub shape (:208), and the search slot itself (:54, :79, :94, :488, :525). Nine handlerReady hits in that file. It is the single most relevant page for that change, and it was not flagged.

Meanwhile the 20 that were flagged include deployment/vercel.mdx, getting-started/your-first-project.mdx and concepts/north-star.mdx — none of which say anything about the services map.

Root cause (measured, not inferred)

The check maps docs to packages by textual mention — the advisory annotates each hit with its edge, _(via @objectstack/runtime)_ or _(via packages/runtime)_.

services-checklist.mdx contains zero occurrences of @objectstack/runtime or packages/runtime:

$ git grep -n "@objectstack/runtime\|packages/runtime" origin/main -- content/docs/kernel/services-checklist.mdx
(no output)

Control — the file does name plenty of other packages, so this is real absence and not a broken query:

$ git grep -o "@objectstack/[a-z-]*" origin/main -- content/docs/kernel/services-checklist.mdx | sort -u
@objectstack/core, @objectstack/driver-memory, @objectstack/driver-mongodb,
@objectstack/driver-sql, @objectstack/driver-sqlite-wasm, @objectstack/metadata,
@objectstack/metadata-protocol, @objectstack/objectql, …

So the page documents runtime-produced behaviour while naming the packages that provide the services rather than the package that builds the report. A doc written that way — describing what a component does without citing its package — is structurally invisible to this scanner.

Why it is worth a card

The failure is silent and one-directional: the advisory over-reports (20 mostly-irrelevant pages, which trains readers to skim past it) and under-reports (the one page that matters), and nothing distinguishes the two. A reviewer who trusts the list is misled precisely when it counts.

Same defect class as #7438 — a scan that discovers its surface by name-text rather than by structure, and silently drops whatever does not spell the name.

Note this is a finding about the checker, not about PR #7965: that PR was verified to leave services-checklist.mdx accurate, because every mention of search there describes the empty-slot branch ("Nothing provides the slot at all — Discovery says exactly that"), which the PR does not touch. No doc states what a filled search slot reports, because nothing has ever filled one.

Possible directions (not a recommendation — the owning lane should weigh these)

  • Add a structural edge alongside the textual one (symbol/identifier → package), so a page discussing handlerReady is reachable from the package that emits it.
  • Or narrow the textual edge so the advisory stops listing pages whose only connection is a passing mention of a package name, and say plainly that the list is name-based and therefore incomplete.

Advisory quality is the whole value here: a list that is both noisy and incomplete is weaker than one that states its own limits.

Source

Measured during PM review of PR #7965 (#7939) by the domain:cli seat (#6024), against origin/main @ a44d1b4c2.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions