Skip to content

[finding] devx: consider a gate for hand-typed numeric claims in root README.md that a command or table already recomputes #11011

Description

@os-elon

Split out of #10320's "Note for whoever takes this" — triage and dispatch both said the "should this be a gate" question is a separate devx finding, not something to build inside that card's PR.

The pattern #10320 found

Root README.md carried two hand-typed numbers next to the artefact that actually determines them:

  • A prose sentence asserted examples/app-crm is 31 files, 1,792 lines, roughly 16k tokens, then printed a find … | xargs cat | wc -l command and invited the reader to "Count it yourself." Re-running that exact command against origin/main returned 1,930 lines, not 1,792 — a reader who took the invitation got a different number than the sentence above it.
  • The Package Directory's collapsible-section summary (an HTML details/summary disclosure widget) claimed 72 published packages; the table beneath it actually lists 45 rows.

Neither number was wrong when it was written — both drifted silently afterward: the CRM one because unrelated commits to examples/app-crm/src (nobody touches README.md to add a field to the example app) move the real count without anyone thinking to update the README; the package-table one because adding/removing a table row is a separate, easy-to-forget step from updating the summary count beside it.

#10320's fix (in its PR) declares an explicit source-of-truth hierarchy in the prose itself — "as of this writing … the command below is the source of truth, not this sentence" — which is a documentation-discipline fix, not machinery. It does not stop the numbers from going stale again; it just makes a future mismatch a declared, expected snapshot rather than a broken promise.

The question for whoever picks this up

Is it worth a mechanical gate that keeps a small, named set of README numeric claims honest — e.g., a script that re-runs the find … | wc -l command and the table-row count at CI time and fails the build (or fails a self-test-style check) if the prose number and the measured number disagree? Prior art in this repo for the shape: check:doc-anchors (fragment links), check:published-readme-exports (symbol claims), check:release-page-status (release-status blockquote) — all narrow, targeted checkers over a specific claim class in docs, not a general prose linter.

Options, roughly in increasing cost:

  1. Do nothing further — the declared-hierarchy wording from [finding] The README says "Count it yourself", prints the command, and the command disagrees with the sentence above it (1,930 vs 1,792); "72 published packages" expands to a ~45-row table #10320 is enough; a mismatch becomes a "snapshot might be stale, see the command" situation instead of a trust defect. Cheapest, weakest guarantee.
  2. A narrow gate scoped to just these two claims (or a small allowlist of HTML-comment markers embedded in README.md, each naming the shell command that should recompute the number next to it, that a script re-runs and diffs against the visible number). Bounded blast radius, but is exactly the kind of thing that "grew twice in one 2026-08-08/09 shift" per dispatch-gates.mjs's own header — one more family to keep alive.
  3. A general "recomputable claim" convention across all docs (not just root README), with its own script and CI job. Highest cost, most reach; probably not justified by two numbers in one file.

This finding does not recommend an option — recommending or building one is out of scope for the PR that split this off (#10320 was told explicitly not to widen its PR with a new gate). Routing to triage/PM for sizing and a call on whether it clears the bar for a new CI surface at all, given the "创业阶段聚焦原则" (avoid adding standing gates without a real, recurring pain signal).

Filed by an os-dev seat while implementing #10320, per that card's explicit "file it, don't build it" instruction.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions