test(uat): tier 8 — do the docs still describe this repo? - #355
Merged
Conversation
Two of the five defects this suite has found were doc rot, and both were found by reading one page closely. This tier does mechanically, across every page, what reading cannot: it checks that the things the documentation NAMES still exist and that the commands it PRINTS can still run. 8.2 is #350 generalised, and the reason to have this tier at all: a documented `dotnet run --project X` cannot run if X multi-targets and the command omits -f/--framework. The command still looks correct on the page, so a human reread never catches it and a csproj change breaks it silently. All 103 documented `dotnet run --project` invocations now name a real project and select a framework where one is required -- which also establishes that #350 was the only instance of its class, rather than the first one found. Nothing new is broken: 219 shipped paths across 12 follow-along pages, 8 compose files and 47 script paths all resolve. 8.1 is deliberately scoped to pages a reader is told to follow. Swept across all ~120 pages it produced ~85 hits and zero real defects, because documentation legitimately names paths that do not exist: plans and inventories name paths they intend to CREATE (65 hits came from one extraction inventory), pages name files the reader creates or a run generates (skipped here by asking git whether the path is ignored, which is exactly what those are), and CONTRIBUTING names `src/Nexo.CLI` in the NEGATIVE to warn that the CLI is not there. A check that cries wolf gets muted, so it asks the narrower question that matters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two of the five defects this suite has found were documentation rot, and both were found by reading one page closely. Tier 8 does mechanically, across every page, what reading cannot.
The check worth having
8.2 is #350 generalised. A documented
dotnet run --project Xcannot run ifXmulti-targets and the command omits-f. The command still looks correct on the page — which is why a human reread never caught it, and why a.csprojchange broke it silently for weeks.All 103 documented
dotnet run --projectinvocations now name a real project and select a framework where one is required. That result is worth stating in both directions: nothing new is broken, and it establishes that #350 was the only instance of its class rather than the first one found.Results
dotnet run --projectcommandsdocker compose -ffilesNo new defects. Tier 8 is a guard, not a finding.
Why 8.1 is scoped, and why that is the honest choice
Swept across all ~120 markdown pages, the path-existence check produced ~85 hits and zero real defects. Documentation legitimately names paths that do not exist:
deploy/compose/.env,deploy/compose/local/) or a run generates (spikes/**/generated/). These are skipped by asking git whether the path is ignored — which is precisely what a reader-created or generated path is.CONTRIBUTINGnamessrc/Nexo.CLIin the negative, to warn that the CLI is not there. The check flagged correct documentation of a deliberately non-existent path.A check that cries wolf gets muted, and a muted gate is worse than no gate. So 8.1 asks the narrower question that actually matters: on the pages a tester or operator is told to follow, does every path they are pointed at exist? The scope and the skip count are both printed, so the coverage is never silently smaller than it looks.
🤖 Generated with Claude Code