Skip to content

[docs] Add docs drift checks for generated docs, llms files, and API surface #210

Description

@serge-ivo

Problem

Docs drift is currently easy to introduce: generated docs, llms.txt, llms-full.txt, OpenAPI, package READMEs, and source docs can contradict implementation without a check failing.

Recent audit examples:

  • Runtime docs disagree on relay-only vs tunnel fallback.
  • Job Application Assistant README describes old job.apply_basic task flow.
  • OpenAPI misses many live Hono routes.
  • Coder docs say runner-node UI is future work even though it exists.
  • README catalog count does not match checked-in manifests/registry.

Scope

Add lightweight validation so future changes catch the highest-risk drift:

  • Route inventory check comparing implemented Hono routes to OpenAPI coverage, with an allowlist for intentionally undocumented/admin routes.
  • Text assertions that current runtime docs do not mention removed current-path commands such as pags up --tunnel or FAGS.
  • Check that platform-docs/*.md and generated platform/store/docs/** are in sync after pnpm docs:build.
  • Optional catalog consistency check across README, agents/*/agent.json, migration seeds, and store/registry.json once the source of truth is decided.

Acceptance Criteria

  • CI or pnpm test/pnpm docs:build can detect the main docs drift classes above.
  • The check has a small intentional allowlist, not a brittle giant snapshot.
  • CONTRIBUTING or README documents how to update docs artifacts when route/docs sources change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions