The Docs Drift Check advisory is computed against a tree the reader cannot identify from the comment, and the comment's own re-derivation instruction points at a different tree. A reader who follows that instruction can get a different list and conclude the tool emitted a wrong row.
Mechanism
.github/workflows/docs-drift-check.yml runs on pull_request, and actions/checkout@v7 checks out the merge commit for that event — base branch merged with the PR head. scripts/docs-audit/affected-docs.mjs then reads every page with readFileSync from that working tree (line 1849, docTexts) and emits a row only when a live regex matches that text.
So the advisory's page set is a fact about merge(base-at-run-time, head). Nothing in the posted comment names that commit. The comment closes by telling the reader:
Re-derive: node scripts/docs-audit/affected-docs.mjs --json origin/BASEREF
Run in a task worktree cut from an older main, that command reads a differentcontent/docs tree. Any page that gained or lost an anchor token on main after the branch was cut produces a row the reader cannot reproduce — and the tool gives them no way to see that they are looking at a different tree.
Measured
On PR #9324 (origin/main, all numbers re-measured 2026-08-18):
| tree | viewName in content/docs/getting-started/build-with-claude-code.mdx | row emitted? |
|---|
PR head 4e5989b9a | 0 | no |
merged a4c11ad48 (what CI read) | 1 | yes |
The token was added to that page by 70775ccac (#9262, an unrelated docs commit) at 08:40Z; the PR branch was cut before that and the advisory ran at 13:51Z. Both runs are correct about their own tree. Re-running today's affected-docs.mjs on each tree reproduces exactly this split — the merged tree yields the seven-row advisory byte-for-byte, the head tree yields six rows with that one absent.
Cost
This is not hypothetical, and it is not cheap. The agent on #9190 read the advisory, grepped its own worktree, found nothing, and reported the row as false. That observation became #9331, filed as "the anchor set itself can go stale" — a defect class that does not exist in this tool, since the anchor set is derived fresh per run with no cache, index or snapshot anywhere. Confirming that took a full investigation round.
What would close it
State the commit the advisory was computed against, in the comment. affected-docs.mjs can emit it in its JSON (git rev-parse HEAD), and the workflow can render it beside the re-derivation command, so a reader who gets a different answer is told why instead of concluding the tool lied. That is the same "say what the run could not see" posture the file already takes everywhere else — this is one more thing the run knows and does not say.
Not claimed: that the row set is wrong. It is right for the tree it was computed on. What is missing is that the tree is unnamed.
Scope
Found while measuring #9331 and deliberately not fixed there: #9331's file surface is scripts/docs-audit/affected-docs.mjs and its self-test, and the rendering half of this lives in .github/workflows/docs-drift-check.yml. Filed unassigned for triage.
Related: #9331 (the card this falsified) · #9192 (the precision-first rework) · #9433 · #9503
Generated by Claude Code
Generated by Claude Code
The
Docs Drift Checkadvisory is computed against a tree the reader cannot identify from the comment, and the comment's own re-derivation instruction points at a different tree. A reader who follows that instruction can get a different list and conclude the tool emitted a wrong row.Mechanism
.github/workflows/docs-drift-check.ymlruns onpull_request, andactions/checkout@v7checks out the merge commit for that event — base branch merged with the PR head.scripts/docs-audit/affected-docs.mjsthen reads every page withreadFileSyncfrom that working tree (line 1849,docTexts) and emits a row only when a live regex matches that text.So the advisory's page set is a fact about merge(base-at-run-time, head). Nothing in the posted comment names that commit. The comment closes by telling the reader:
Run in a task worktree cut from an older
main, that command reads a differentcontent/docstree. Any page that gained or lost an anchor token onmainafter the branch was cut produces a row the reader cannot reproduce — and the tool gives them no way to see that they are looking at a different tree.Measured
On PR #9324 (
origin/main, all numbers re-measured 2026-08-18):viewNameincontent/docs/getting-started/build-with-claude-code.mdx4e5989b9aa4c11ad48(what CI read)The token was added to that page by
70775ccac(#9262, an unrelated docs commit) at 08:40Z; the PR branch was cut before that and the advisory ran at 13:51Z. Both runs are correct about their own tree. Re-running today'saffected-docs.mjson each tree reproduces exactly this split — the merged tree yields the seven-row advisory byte-for-byte, the head tree yields six rows with that one absent.Cost
This is not hypothetical, and it is not cheap. The agent on #9190 read the advisory, grepped its own worktree, found nothing, and reported the row as false. That observation became #9331, filed as "the anchor set itself can go stale" — a defect class that does not exist in this tool, since the anchor set is derived fresh per run with no cache, index or snapshot anywhere. Confirming that took a full investigation round.
What would close it
State the commit the advisory was computed against, in the comment.
affected-docs.mjscan emit it in its JSON (git rev-parse HEAD), and the workflow can render it beside the re-derivation command, so a reader who gets a different answer is told why instead of concluding the tool lied. That is the same "say what the run could not see" posture the file already takes everywhere else — this is one more thing the run knows and does not say.Not claimed: that the row set is wrong. It is right for the tree it was computed on. What is missing is that the tree is unnamed.
Scope
Found while measuring #9331 and deliberately not fixed there: #9331's file surface is
scripts/docs-audit/affected-docs.mjsand its self-test, and the rendering half of this lives in.github/workflows/docs-drift-check.yml. Filed unassigned for triage.Related: #9331 (the card this falsified) · #9192 (the precision-first rework) · #9433 · #9503
Generated by Claude Code
Generated by Claude Code