diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 55f8dade..cf856d7c 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -37,7 +37,7 @@ Auto-review on push is configured (via the branch ruleset's `copilot_code_review **A review with no inline comments is still a completed review, not a failure, and not a reason to ask the maintainer to re-trigger.** Copilot very often posts a single formal review (GraphQL `state: COMMENTED`) whose body ends with "...reviewed N of N changed files ... and generated no comments" and adds **zero** inline threads. That review carries the head `commit.oid` and fully satisfies the loop, and it is the clean-pass success case. Never read "no inline comments" as "the review didn't run," and never re-request or escalate to the maintainer because comments are absent. -**Read the low-confidence findings, which are not inline threads.** A review body can carry a collapsed `
` block of findings Copilot withheld from the inline threads, and those findings appear nowhere in `reviewThreads`, so a loop that polls threads alone never sees them and reports a clean pass. **Match the block on more than one phrasing.** Its heading has appeared both as `Suppressed comments (N)` and as "Comments suppressed due to low confidence", so a filter keyed on either one alone silently reports zero suppressed findings on a review that has them, the same false clean this rule exists to prevent, one level up in the detection. They have been right repeatedly, including a rule stated more broadly than its check enforced and a check that skipped fenced blocks in every rule but one. Read the body of every review, investigate each suppressed finding on the same footing as an inline one, and answer it in the PR conversation, since a suppressed finding has no thread to reply on or resolve. +**Read the low-confidence findings, which are not inline threads.** A review body can carry a collapsed `
` block of findings Copilot withheld from the inline threads, and those findings appear nowhere in `reviewThreads`, so a loop that polls threads alone never sees them and reports a clean pass. **Match the block on more than one phrasing.** Its heading has appeared both as `Suppressed comments (N)` and as "Comments suppressed due to low confidence", so a filter keyed on either one alone silently reports zero suppressed findings on a review that has them, the same false clean this rule exists to prevent, one level up in the detection. **The section moves as well as it is worded, so match the heading wherever it sits.** It has appeared as its own `
` wrapper with a matching ``, as a bare heading in the body, and as a Markdown heading nested inside the `Review details` wrapper, whose `` names the wrapper and not the section. A filter reading a wrapper's `` reports zero on the nested shape, and the count it needs is the heading's own `(N)` rather than the wrapper's. They have been right repeatedly, including a rule stated more broadly than its check enforced and a check that skipped fenced blocks in every rule but one. Read the body of every review, investigate each suppressed finding on the same footing as an inline one, and answer it in the PR conversation, since a suppressed finding has no thread to reply on or resolve. ```sh # `test` with an alternation, not `contains` on one phrasing: the heading wording has changed. diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc index 40f5d762..89ff90de 100644 --- a/.markdownlint-cli2.jsonc +++ b/.markdownlint-cli2.jsonc @@ -3,9 +3,9 @@ // Prose paragraphs and data-heavy tables or URLs are intentionally long. // Reflowing at 80 columns hurts readability and churns diffs. "MD013": false, - // MD033 (inline HTML) stays enabled so native markdown wins. + // MD033 (inline HTML) stays enabled so native Markdown wins. // HTML comments, used as reference-link dividers, pass it. - // The details and summary elements are allowed for GitHub collapsibles, which have no markdown equivalent. + // The details and summary elements are allowed for GitHub collapsibles, which have no Markdown equivalent. // Every other element still flags. "MD033": { "allowed_elements": ["details", "summary"] }, // Require fenced code blocks over the legacy 4-space-indented style. diff --git a/AUDIT.md b/AUDIT.md index 96afc418..02e9dd29 100644 --- a/AUDIT.md +++ b/AUDIT.md @@ -12,7 +12,7 @@ The verdict vocabulary is [`WORKFLOW.md`][workflow]'s: **operational / not opera This audit is not occasional. Run it whenever you **create, adopt, or materially change** a fleet repo, and on demand for any known repo: - **Onboarding a repo is complete only when it either passes this audit** (operational on every applicable check) **or carries a committed `reports//audit.md` plus a tracking issue** enumerating every residual delta. A repo that is partially set up but never audited is itself a **defect**, the exact state this process prevents. The create-to-conformance counterpart is [`STANDUP.md`][standup]. Because both read the same manifests, a repo stood up by that file passes this audit by construction. -- **Touching a repo** (any conformance-affecting change) ends by re-running the applicable checks and **reconciling the registry entry to reality**: `status`, `types`, `releaseTrigger`, `workflowModel`, `driftNotes`. The registry records reality, not intent. [`spec/validate.py`][validate] proves the catalog is self-consistent, not that it matches the live repo. Closing that gap is this audit's job. The deterministic subset (settings, rulesets, secret names, file presence, per-scope markdown section presence, workflow interface conformance, verbatim content, branch facts) is mechanized in [`spec/audit.py`][audit-runner]: owner-initiated, run on demand when onboarding a repo, on suspected drift, or before fleet-wide changes. A required section missing from a carried markdown file is a **drift finding**, not a letter, because a heading rename reads as missing and equivalence is judged by hand. A carried `interface` workflow (spec/fidelity-model.md) is checked by name and wiring (required jobs, the ruleset-bound check name, the artifact-name handoff, and the forbidden `artifact-ids:` fork), all at **drift**, since the body is owned and a rename is a hint to verify. A carried `verbatim` unit, whether a whole file (`.markdownlint-cli2.jsonc`) or a canonical workflow job region (the `github-release` job), is content-hashed against the hub's canonical after line-ending normalization. A mismatch is classified **stale** (matches a past hub revision, re-vendor) or **modified** (matches none, the repo changed fixed content), both at **drift**, since equivalence is intent-governed and a byte diff is a hint to review. +- **Touching a repo** (any conformance-affecting change) ends by re-running the applicable checks and **reconciling the registry entry to reality**: `status`, `types`, `releaseTrigger`, `workflowModel`, `driftNotes`. The registry records reality, not intent. [`spec/validate.py`][validate] proves the catalog is self-consistent, not that it matches the live repo. Closing that gap is this audit's job. The deterministic subset (settings, rulesets, secret names, file presence, per-scope Markdown section presence, workflow interface conformance, verbatim content, branch facts) is mechanized in [`spec/audit.py`][audit-runner]: owner-initiated, run on demand when onboarding a repo, on suspected drift, or before fleet-wide changes. A required section missing from a carried Markdown file is a **drift finding**, not a letter, because a heading rename reads as missing and equivalence is judged by hand. A carried `interface` workflow (spec/fidelity-model.md) is checked by name and wiring (required jobs, the ruleset-bound check name, the artifact-name handoff, and the forbidden `artifact-ids:` fork), all at **drift**, since the body is owned and a rename is a hint to verify. A carried `verbatim` unit, whether a whole file (`.markdownlint-cli2.jsonc`) or a canonical workflow job region (the `github-release` job), is content-hashed against the hub's canonical after line-ending normalization. A mismatch is classified **stale** (matches a past hub revision, re-vendor) or **modified** (matches none, the repo changed fixed content), both at **drift**, since equivalence is intent-governed and a byte diff is a hint to review. ## 1. Scope and Ground-Truth Branch @@ -28,6 +28,7 @@ Look up the repo in [`registry/repos.json`][repos] and read its `types[]`. If th - `pyproject.toml` / `setup.py` -> `python`, a `pypa/gh-action-pypi-publish` workflow -> `pypi`. - `Dockerfile` + a docker build/push workflow -> `docker`, an `upstream-version.json` tracker -> `upstream-wrapper`. - `custom_components/*/manifest.json` + `hacs.json` -> `homeassistant`, a codegen workflow -> `codegen`, no `build-*` task -> `source-only`, governance-only -> `docs`. +- `hugo.yaml` / `hugo.toml` / `config/_default/hugo.yaml` -> `hugo`. A repo may carry it alongside `source-only`, since a site deploy leaf is not a `build-*` task and both declarations stay true. ## 3. Applicability Gate @@ -39,6 +40,8 @@ Which carried files and sections a repo is expected to have is decided by its sc For each applicable type in [`spec/project-types.json`][project-types] and every cross-cutting dimension, evaluate each check at its stated verdict tier: +**Every check under a project type is judged by hand. The cross-cutting dimensions are only partly mechanized, and the line between the two halves is not where a reader assumes.** [`spec/audit.py`][audit-runner] evaluates **no** check belonging to a type in `spec/project-types.json`, and it reads that file for one purpose only, to resolve the id a registry `driftNote` names (section 8) against the catalog and against the repo's declared types. Resolving an id is not running the check it names. What the runner does mechanize is the deterministic subset in section 0, and that subset lands on several `crossCutting` checks without being organized by them: branch protection and the ruleset diffs, secret names, Dependabot ecosystems, the cspell single source, section presence, and `driftNotes` freshness. So read a clean run precisely. It is evidence for that subset, it is **no** evidence for any of a type's checks, and it is partial evidence across the cross-cutting dimensions. The three are easy to conflate, because adding a check under a type changes what an auditor must judge and changes no tool's output, so the check reports nothing until someone evaluates it, and silence from a tool that was never looking reads exactly like a pass. Cite the `file:line` each check was judged against, since that citation is the only durable record that the judgment happened. + - **letter** - the exact file, section, config, or construct is present. - **intent** - an equivalent outcome holds even if the form differs. @@ -50,6 +53,7 @@ A check with `intentRef`/`workflowRef` points at the prose section that owns the - **python** - ruff and pyright present (intent), canonical in `pyproject.toml` (letter), and a standalone `.ruff.toml` / `pyrightconfig.json` is a drift finding. - **console** - smoke runtime matrix is a strict subset, and per-runtime outputs aggregate to one `release-asset-*`, gated `!smoke`. - **docker** - registry layer cache (`buildcache-`, never `type=gha`), the size-limited Docker Hub README is published via the docker-readme task, and the image always re-pushes on publish. +- **hugo** - the build fails on a generator warning, the URL-parity gate asserts a length floor before comparing, the rendered output is untracked, the generator is pinned by version and checksum and declared once, a vendored tree records its upstream ref, and the deploy asserts what the host serves (the release id and the environment). Retention is bounded by a declared count with one side recorded as owning the prune, which is the deploy where its credential can observe the destination and the host where that credential is confined write-only, so grade which shape the repo uses rather than looking for a prune step. Deploy credentials are per-environment, which `spec/secrets.json` cannot express, so a clean **repo-setup** verdict says nothing about whether the environments are configured. - **branch-model** - `main` and `develop` both exist and are protected, and the live rulesets match [`repo-config/*.json`][repo-config] by normalized diff (below). - **repo-setup** - every required secret for the repo's publish mechanisms is configured, and no forbidden secret is present (per [`spec/secrets.json`][secrets]). - **linter-parity** - one config per linter (`.markdownlint-cli2.jsonc`, `cspell.json`, ruff/pyright, editorconfig/csharpier, actionlint) drives the editor extension, the CLI, and CI, and CI runs each. @@ -129,7 +133,7 @@ What a downstream repo does instead is unchanged and is where its context is wor **Re-running the audit needs a full hub clone with git history.** The verbatim stale-vs-modified classification walks the canonical's history (`git log` / `git show` from the hub root), so a shallow clone or a files-only checkout cannot answer "matches a past hub revision" and those findings are unreliable there. A downstream agent verifying one finding without the full history can instead compare against the current hub canonical on `main` (the whole file for a file-level unit, or the named `## heading` block for a verbatim section), which decides current-match but not stale-vs-modified. An agent picking up such an issue **re-runs the audit first and acts on the live result, not the pasted findings**: a repo moves between filing and pickup, so a stale block leads an agent to "fix" what is already fixed (re-requesting secrets that exist, attempting a no-op forward-sync). State the findings as evidence for *why* the issue was filed, never as the current state. -**Reconcile `driftNotes` in the same pass.** A registry `driftNote` records a *current* deviation from the baseline. Once the deviation is resolved the note is deleted, not left describing finished work, since hand-maintained prose drifts silently otherwise. `spec/audit.py` flags this: when a repo audits clean but a note still asserts outstanding work ("pending", "not yet", "missing", "behind", ...), it raises a drift finding naming the note. +**Reconcile `driftNotes` in the same pass.** A registry `driftNote` records a *current* deviation from the baseline. Once the deviation is resolved the note is deleted, not left describing finished work, since hand-maintained prose drifts silently otherwise. `spec/audit.py` flags two shapes of note, neither of them gated on the rest of the audit being clean. A note asserting outstanding work in prose ("pending", "not yet", "missing", "behind", ...) is contradicted outright by a clean audit, and where findings are open it is raised as a question of which one it means, because gating the check on a clean audit meant one standing finding a repo could not clear exempted its whole note list, and the repo carrying open findings is where a stale note is most likely. A note naming the check that would retire it, as an id in parentheses and matched with them (`(hugo.generator.pinned)`, the bare id is not detected), is the mechanically checkable shape and is surfaced on **every** run: the audit resolves the id against the catalog and confirms the repo declares its type, then hands the check itself to the auditor, since section 4 above is judged by hand. **So a note naming a check id is retired by a person, not by a run.** Write it that way anyway. The id says exactly what would close the note, and the surfaced finding puts that decision in front of whoever runs the audit rather than leaving the note to sit until someone rereads it. ## 9. Escalate diff --git a/CODESTYLE.md b/CODESTYLE.md index dfdfaa9f..2414a1ea 100644 --- a/CODESTYLE.md +++ b/CODESTYLE.md @@ -2,7 +2,7 @@ This is the single code-style guide for the fleet. The **General** section applies to every language. Each **language section** (.NET, Python) is self-contained: a repo follows only the section(s) for the languages it ships and ignores the rest. A repo keeps the whole file rather than trimming it. An unused-language section costs nothing, the same whole-file model as [`.editorconfig`][root], whose inert `[*.cs]` block a non-.NET repo keeps. -Cross-cutting *process* rules (PR titles, branching, US English, markdown style, comments philosophy, workflow YAML, PR review etiquette, and the verification discipline that defines the pre-push lint gate) live in [GOVERNANCE.md][governance] and are not repeated here. +Cross-cutting *process* rules (PR titles, branching, US English, Markdown style, comments philosophy, workflow YAML, PR review etiquette, and the verification discipline that defines the pre-push lint gate) live in [GOVERNANCE.md][governance] and are not repeated here. ## General @@ -33,10 +33,11 @@ Each language defines a **clean-compile** verification: the combination of build These apply repo-wide, in every directory: -1. **Markdown linting**: All `.md` files must be lint-clean (error and warning free) via the VS Code `markdownlint` extension. [`.markdownlint-cli2.jsonc`][markdownlint-cli2] at the repo root is the single source of truth, and the davidanson `markdownlint` extension and a command-line `markdownlint-cli2` run both read it, so the IDE and CLI stay in lock-step. Rules it deliberately disables (e.g. `MD013` line-length) are **intentional**, so do not "fix" them. `MD033` inline HTML stays **enabled**: HTML comments are permitted (markdownlint does not flag them), `details` and `summary` are allowed because a GitHub collapsible has no markdown equivalent, every other element is flagged, and anything with a native markdown equivalent uses the markdown. Fix violations at the source rather than disabling rules. +1. **Markdown linting**: All `.md` files must be lint-clean (error and warning free) via the VS Code `markdownlint` extension. [`.markdownlint-cli2.jsonc`][markdownlint-cli2] at the repo root is the single source of truth, and the davidanson `markdownlint` extension and a command-line `markdownlint-cli2` run both read it, so the IDE and CLI stay in lock-step. Rules it deliberately disables (e.g. `MD013` line-length) are **intentional**, so do not "fix" them. `MD033` inline HTML stays **enabled**: HTML comments are permitted (markdownlint does not flag them), `details` and `summary` are allowed because a GitHub collapsible has no Markdown equivalent, every other element is flagged, and anything with a native Markdown equivalent uses the Markdown. Fix violations at the source rather than disabling rules. 2. **Spelling**: All spelling must be clean via the CSpell VS Code integration, and words must be correctly spelled in **US English** (the repo-wide convention, per [GOVERNANCE.md][governance]). The shared `cspell.json` sets `"language": "en-US"` so British spellings are flagged, where a bare `"en"` accepts both US and British and silently passes the wrong spelling. Project-specific terms go in the shared `cspell.json` `words` list, the single source of truth the extension, CLI, and CI all read. The `.code-workspace` must **not** carry its own `cspell.words`/`cSpell.words` block, and when externalizing words into `cspell.json`, delete any word list left in the workspace (a leftover one duplicates the list and silently drifts). -3. **Spelling CI scope**: The enforced CI spell-check gate covers **`README.md` and `HISTORY.md` only**, because these are the files every repo visitor sees, so they must be clean. It is deliberately **not** all `**/*.md`: repos carry many markdown files full of technical terms, and gating every one of them would mean endlessly padding `cspell.json` just to keep CI green. Broad, live spell-checking across any file (source, markdown, text) is the **cspell editor extension's** job, so typos still surface to whoever is editing. A repo owner **may** widen their own CI file list, but README + HISTORY are the default; keep the CI workflow, the `Lint: Spelling` VS Code task, and the GOVERNANCE.md cspell one-liner on the same file list. The list is explicit (not a glob), so a repo that ships no `HISTORY.md` (e.g. one with no changelog) must drop it from all three surfaces and gate on `README.md` alone, since cspell errors on a listed file that does not exist. Markdown *linting* (item 1) stays repo-wide `**/*.md`, which does not choke on technical terms. +3. **Spelling CI scope**: The enforced CI spell-check gate covers **`README.md` and `HISTORY.md` only**, because these are the files every repo visitor sees, so they must be clean. It is deliberately **not** all `**/*.md`: repos carry many Markdown files full of technical terms, and gating every one of them would mean endlessly padding `cspell.json` just to keep CI green. Broad, live spell-checking across any file (source, Markdown, text) is the **cspell editor extension's** job, so typos still surface to whoever is editing. A repo owner **may** widen their own CI file list, but README + HISTORY are the default; keep the CI workflow, the `Lint: Spelling` VS Code task, and the GOVERNANCE.md cspell one-liner on the same file list. The list is explicit (not a glob), so a repo that ships no `HISTORY.md` (e.g. one with no changelog) must drop it from all three surfaces and gate on `README.md` alone, since cspell errors on a listed file that does not exist. Markdown *linting* (item 1) stays repo-wide `**/*.md`, which does not choke on technical terms. 4. **`HISTORY.md` mirrors the README opening**: `HISTORY.md` is the maintainer-curated changelog and opens as the README's twin, carrying the same `# ` (without the README's ToC-omit comment) and the same intro paragraph copied verbatim, then a `## Release History` section. The mirrored opening keeps the project identity consistent for a reader who lands on the changelog directly. The audit checks that the title and intro match the README, with HTML comments stripped. +5. **"Markdown" is the format's name**: The format is a proper noun, so prose capitalizes it, meaning a Markdown file, a Markdown link, and the Markdown a surface renders. Lowercase is for the strings a machine reads and for nothing else: a tool or package name (`markdownlint`, `markdownlint-cli2`, `yzhang.markdown-all-in-one`), a settings key (`markdown.extension.toc.levels`), a heading anchor (`#markdown-and-spelling`), an identifier in code, and a file extension. A hyphenated compound in prose is prose, so it capitalizes too (Markdown-only), which is the boundary a mechanical sweep gets wrong, since it reads the hyphen as the mark of an identifier. What this settles is the mix rather than either spelling, because a file carrying both gives the next author no default to follow and a reviewer a finding to raise on whichever one it wrote last. The rule lives here because every repo carries this file, so the convention arrives with it rather than being re-decided per repo. ## .NET diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 32738f45..c8f53703 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -25,6 +25,7 @@ A state-changing GitHub call is the highest-blast-radius thing an agent does her - **Write only within the owner of the current project's repository.** Every state-changing call targets this project's `origin` or another repository under the same owner, which is the fleet the maintainer already administers. A broad or logged-in identity is capability, not permission: a token that *can* reach another owner's repository does not authorize writing to it. Writing under a **different owner** needs explicit human permission naming that repository, granted deliberately rather than assumed from a token's reach, and a "harmless test" write is still a write, so there is no probe exception. That boundary is where the harm sits, since the incident this rule exists for was a stray comment on a stranger's repository, not work across the maintainer's own projects. Reads from anywhere are fine. - **Never fabricate, guess, or reuse an identifier passed to a write.** Every id a state-changing call consumes (a node id, a numeric id, a thread or comment id) is captured from a live query in the **same** session into a variable and passed from there. Do not hand-type an id, guess it, recall it from memory or an earlier session, or copy it from documentation or an example. Ids commonly resolve **globally**, so a wrong-but-valid id does not fail. It writes to the wrong target, in someone else's repository. If a query returns no id, stop rather than invent one to proceed. - **A write is never a probe, and a write's output is never suppressed.** Never fire a state-changing call to see whether it works: decide it should happen, make it happen, and read the result. Never append output-discarding redirection or a force-success tail to a mutation (for example `>/dev/null`, `2>/dev/null`, `&>/dev/null`, `|| true`, `|| :`, `|| echo`), because the write's output is exactly what must be read. A write that appears to fail is **verified, not assumed harmless**, because the operation may have succeeded on the server while the client reported an error, so confirm the actual state before retrying or moving on. The ban targets hiding a *failure*. An ad-hoc call's response is the only signal you get, so `>/dev/null 2>&1`, `|| true`, and `|| echo`, which swallow the error stream or force success, are never acceptable on one. A committed script under `set -e` is a narrow exception: it may send a write's *stdout* to `/dev/null` to drop the success-response noise, because stderr stays visible and a failed write still aborts loudly (`repo-config/configure.sh` does exactly this). The exception is stdout-only suppression inside a reviewed, fail-loud script, never `2>&1` or a force-success tail, and never an ad-hoc command. +- **A refused write is reported, never re-shaped, and the maintainer's say-so does not lift a refusal by the harness.** These are two different permissions and only one of them is the maintainer's to give. When the agent harness refuses a write, the maintainer authorizing it in conversation does not change the outcome, and the identical call is refused again, so a second attempt is not worth making and reading the second refusal as a flake is how an agent starts hunting for another shape of the same request. **That hunt is the failure this rule exists to stop.** Re-expressing a refused `gh` command as a raw `gh api -X POST` reaches the same endpoint with the same identity and the same blast radius, having defeated the one control that stopped it, and it is the more dangerous version because the agent believes it has permission. So a refused write is never re-attempted through a different API surface, a different tool, or a rephrasing, and it is never routed around by the agent writing itself a permission rule, which is self-authorization whatever the maintainer said. Two routes remain, both of them the maintainer's: they add the permission rule themselves, or they run the command themselves. Raise it as a blocked decision naming those two (see "Communicating with the User"), and where the work needs the result rather than the call, say what the agent will verify once the maintainer has run it. **A refusal is also a fact about the contract, not just about the session**: where a required verification can only be performed by a write the agent is refused, the document requiring it says so and names who runs it, since a check that is mandatory and unperformable is quietly dropped and then reported as done. ## Representative Data in Agent-Authored Text @@ -55,6 +56,7 @@ Agent-authored text illustrates with data the agent constructed, never with data - **Executing a `develop -> main` promotion safely, two traps both learned the hard way:** - **Never delete `develop`.** A promotion PR's head *is* `develop`, so `gh pr merge --delete-branch` (and the repo's "Automatically delete head branches" toggle, which is why that toggle is [kept off](./repo-config/settings.json)) deletes `develop` itself. Merge a promotion with a plain `gh pr merge --merge`, no `--delete-branch`. If `develop` is ever lost this way, restore it to the merged PR's head SHA, which is still reachable as the merge commit's second parent: `gh api -X POST "repos/<owner>/<repo>/git/refs" -f ref=refs/heads/develop -f sha="$(gh pr view <n> --json headRefOid --jq .headRefOid)"`. - **Spurious EOL-only conflicts resolve by taking `develop`.** When develop declared workflow YAML as LF while main is still CRLF, `develop -> main` conflicts *whole-file* on those paths. develop's `required_linear_history` + PR rulesets forbid resolving on `develop` (no merge commit, no force-push), so resolve on a throwaway branch off `main`: `git checkout -b promote/develop-to-main origin/main && git merge origin/develop`, take develop's side for the EOL-conflicted files (`git checkout --theirs <file>`) **after confirming each is content-identical modulo EOL or that develop is a strict superset** (`diff <(git show :2:f|tr -d '\r') <(git show :3:f|tr -d '\r')`), then open that branch -> `main`. Verify no genuine main-only content is dropped (build/test where the repo supports it). +- **Issue-closing keywords (`Closes #N`, `Fixes #N`) go in the `develop -> main` promotion PR, not the feature -> `develop` PR.** GitHub auto-closes an issue only when the closing keyword merges into the **default branch** (`main`). A feature -> `develop` PR merges into `develop`, so the keyword never fires there. Reference the issue in the `develop` PR body if useful, but put the actual closing keyword on the promotion PR. If a `develop` PR merges with the keyword on it, the keyword does nothing and the issue stays open, so put it on the promotion PR body instead. Close the issue by hand citing the squash SHA only when the promotion has already merged without it. - **Both rulesets intentionally omit "Require branches to be up to date before merging".** The flag is off on `main` and on `develop`, for related but distinct reasons. - *Main:* the check is graph-based: it asks whether main's tip commit is reachable from develop, not whether the two branches have the same content. After any develop -> main release, main's tip is a brand-new merge commit that develop's history doesn't contain. Forward-only develop never adds it (no back-merge of main into develop), so the check would fail on every subsequent release. Other technical workarounds (rebasing develop onto main, or rewriting develop's history) exist but contradict the squash-only develop ruleset and the linearity invariant. - *Develop:* the check stalls bot auto-merge when two bot PRs against develop land within the same window. As soon as the first merges, the second flips to `mergeStateStatus: BEHIND` and GitHub's auto-merge will not fire while strict is on. The merge-bot only *enables* auto-merge on `opened`/`reopened` (see below) and never auto-updates bot branches, and Dependabot's rebase isn't real-time, so the second PR sits OPEN with all checks green indefinitely. Squash mechanics still rebase the diff onto develop's tip on merge, `required_linear_history` still enforces linearity, textual conflicts still block `mergeable: CONFLICTING`, and the required `Check pull request workflow status job` still gates merges. The only thing lost is pre-merge detection of *semantic-but-not-textual* conflicts, which the post-merge develop CI run catches anyway. @@ -82,6 +84,7 @@ The **two-phase model is the default**: PRs build fast, publishing is batched. S - *Files attached to the GitHub Release* (zips, binaries, packaged libraries): one leaf task per output, each uploading `release-asset-<branch>-<name>`. A data-only repo (e.g. a symbol library) has exactly one such task: validate -> `zip` -> upload `release-asset-<branch>-library`. It deletes the nuget/pypi/executable/docker jobs and the `publish-pypi` job, keeps `github-release` as-is. This is also where the .NET `build-executable-task` lives, and it is *not* a generic file step but specifically `dotnet publish` of the console app; replace it wholesale, don't adapt it. - *Package-registry pushes* (NuGet.org, PyPI): the leaf task both builds **and** publishes to its registry. NuGet pushes from inside `build-nugetlibrary-task` (`dotnet nuget push --skip-duplicate`) *and* also uploads a `release-asset-*` (.7z) for the GitHub release. PyPI is split: `build-pypilibrary-task` only builds + uploads the `pypilibrary-build-<branch>` artifact, and the separate `publish-pypi` job in `publish-release.yml` does the OIDC Trusted-Publishing upload (so `id-token: write` is granted only at that one entry point), and PyPI contributes **no** `release-asset-*`. - *Image-registry pushes* (Docker Hub): `build-docker-task` pushes multi-arch tags directly and contributes **no** `release-asset-*`. The image tag is build-layer-owned, so drive it from whatever version source fits (NBGV `SemVer2`, an upstream-release pin, or a per-image matrix). To publish the Docker Hub repository overview, `publish-docker-readme-task.yml` pushes `Docker/README.md` via `peter-evans/dockerhub-description` (single-repo by default, matrix per image for multi-image repos), wired into `publish-release.yml` and gated to `main`. + - *Filesystem on a host the project owns* (a static site, a config tree): a deploy leaf builds the tree and ships it over the repo's own transport, contributing **no** `release-asset-*`. It is a **separate `workflow_dispatch`** from the release, so a redeploy of an unchanged commit mints no tag, and its credentials come from a **per-environment GitHub Environment** rather than the repository secret store. Its last step asserts what the host actually serves, the release id and the environment, never that the transport exited zero. Retention at the destination is bounded by a declared count, and one side is recorded as owning the prune: the deploy where its credential can observe the destination, the host where that credential is deliberately write-only. - *Source-only / no build* (validate + tag + release): this seam does not apply. A source-only repo carries **no** `build-release-task.yml` (its `appliesTo` excludes it), so there are no leaf tasks and no `get-version`/`github-release`/`date-badge` jobs to curate. Its whole release is the standalone [`publish-release.yml`](./.github/workflows/publish-release.yml) on `workflow_dispatch`: a `validate` job (the repo's reusable validation task) gates a publish job that **inlines** NBGV for the tag and `action-gh-release` for the release (tag + auto source archive + README + LICENSE). - `get-version-task.yml` installs the .NET SDK only because NBGV needs the runtime to compute the version/tag, which is heavyweight but expected even for a non-.NET repo, and acceptable as-is. - **No-op republish guarantee.** A weekly/dispatch publish where NBGV `SemVer2` is **unchanged** (no new commit since the last publish) re-pushes **nothing** to GitHub Releases (the `github-release` job's `release-exists` check skips the create step), NuGet (`dotnet nuget push --skip-duplicate`), or PyPI (`gh-action-pypi-publish` `skip-existing: true`), since all three key on the version string. **Docker always re-pushes** by design: it picks up upstream base-image refreshes (e.g. `ubuntu:rolling`) that aren't visible in the repo. Boundary: `version.json` has **no `pathFilters`**, so *any* commit, including a CI/workflow-only or docs-only change, advances the NBGV git height and therefore `SemVer2`, and the next publish *does* create a fresh release for it even when the shipped binary is byte-identical. This is accepted NBGV behavior; `pathFilters` are intentionally not added. @@ -89,7 +92,6 @@ The **two-phase model is the default**: PRs build fast, publishing is batched. S - **Bump `version.json` only for functional changes, by maintainer instruction.** Raise the major/minor when the work being introduced warrants a new semantic version (a new feature, a behavior or API change, a breaking change), and do it in the PR that introduces that work (typically on `develop`). Do **not** bump on a fixed cadence or mechanically after a release. NBGV advances the patch (git height) on every commit automatically, so a release always gets a fresh build version without any `version.json` edit. - **No post-release bump, and no develop-ahead requirement.** NBGV advances the patch (git height) on every commit, so a release always gets a fresh build version with no `version.json` edit and there is no `bump-version-X.Y` PR after a release. A `develop -> main` promotion carries whatever `version.json` is current: a promotion with a functional bump releases that new version on `main`, and a maintenance-only promotion carries the unchanged `version.json` and `main` advances only its NBGV height. - **Docs reference the 2-digit `major.minor` line, never a 3-digit build.** `README.md`, `HISTORY.md`, and release notes name the version as `Version 1.0` (the `version.json` floor). NBGV owns the patch/build position, so a concrete three-part number in a doc is both wrong (the real build height differs) and a maintenance trap. "Correcting" `1.0` to `1.0.0` is a defect, not a fix, and it has blocked a release. -- **Issue-closing keywords (`Closes #N`, `Fixes #N`) go in the `develop -> main` promotion PR, not the feature -> develop PR.** GitHub auto-closes an issue only when the closing keyword merges into the **default branch** (`main`). A feature/develop PR merges into `develop`, so the keyword never fires there. Reference the issue in the develop PR body if useful, but put the actual closing keyword on the promotion PR. If a develop PR merges with the keyword on it, the keyword does nothing and the issue stays open, so put it on the promotion PR body instead. Close the issue by hand citing the squash SHA only when the promotion has already merged without it. - **Wrapper repos that track an upstream release.** A repo wrapping an upstream release uses `check-upstream-version-task.yml`: a resolver command prints the upstream version(s) as a **JSON object of `name -> version`**, written to a committed state file at the **repo root beside `version.json`** (default `upstream-version.json`, since it is a build-input version source, not GitHub-platform config, so it does not belong under `.github/`), and opens a rolling App-signed bump PR per branch that the merge-bot auto-merges (`merge-upstream-version`). The object carries one key for the common single-version case (`{"version":"X"}`) or N keys for a wrapper that pins several upstream components (e.g. an image plus a companion tool), and the build reads each component by key, and the bump PR's title/body name only the keys that actually moved. Call it from a scheduled entry-point workflow and matrix only the branches that ship the version (a CI-only version uses `["develop"]`). A merged bump ships on the **next publish**, not immediately, which is the two-phase latency tradeoff. ## Operational Repositories @@ -139,11 +141,11 @@ Clarify devcontainer setup steps in README ## Documentation Style Conventions -- **Carried files carry no coordination references.** In the files the fleet carries (`AGENTS.md`, `GOVERNANCE.md`, `CODESTYLE.md`, `WORKFLOW.md`, `.github/copilot-instructions.md`, the `repo-config/` and `spec/` files, the carried `AUDIT.md`), two things are banned. **Any reference to the template repo**, in prose or in a link: the coordination flow is machinery a consumer of that repo should never have to see, and naming where a file came from is the derived-from framing the present-tense rule below independently forbids. Where a carried file must express a template-level behavior ("report a rule discrepancy upstream"), state the behavior rather than the destination. The maintainer supplies the destination out of band. And **a sibling fleet repo named as an illustrative example** ("repo X does it this way", "see repo Y's adoption"), which couples the repos and rots as they diverge. To point at a current good example, name it in the onboarding or conformance issue, never in a carried doc. **A contextually relevant link to a related project is not a coordination reference, and is expected.** Where another repo is part of this repo's subject matter (the image that consumes this config, the builder that generates this hardware, a library this depends on), link it normally. The test is whether the link serves a reader of *this* repo's content, not whether the target happens to be in the fleet. This rule governs carried template content. A repo's own `README.md` and topical docs are its own content, not carried verbatim, and it does not reach them. This pairs with the present-tense rule below: state the current shape, not a history of which repo it came from. +- **Carried files carry no coordination references.** In the files the fleet carries (`AGENTS.md`, `GOVERNANCE.md`, `CODESTYLE.md`, `WORKFLOW.md`, `.github/copilot-instructions.md`, the `repo-config/` and `spec/` files, the carried `AUDIT.md`), two things are banned. **Any reference to the template repo**, in prose or in a link: the coordination flow is machinery a consumer of that repo should never have to see, and naming where a file came from is the derived-from framing the present-tense rule below independently forbids. Where a carried file must express a template-level behavior ("report a rule discrepancy upstream"), state the behavior rather than the destination. **The one exception is a verbatim section, and `AGENTS.md` "Fleet Bootstrap" is why it exists.** That section's whole function is to name where the canonical rules live, for an agent in a repository whose carried copies are stale, partial, or absent, which is exactly when no other file present can say it. Its bytes are fixed fleet-wide, so a repository cannot edit the reference out without failing the verbatim check instead, and a rule banning it would be unsatisfiable rather than merely strict. The exception is scoped to the verbatim region and never leaks past it: the same document's own prose is governed normally, so a repository that names the destination in a paragraph it owns is still in breach. A reference that reaches a verbatim section is a defect in the canonical, fixed once at the source rather than reported against every repository carrying it. The maintainer supplies the destination out of band. And **a sibling fleet repo named as an illustrative example** ("repo X does it this way", "see repo Y's adoption"), which couples the repos and rots as they diverge. To point at a current good example, name it in the onboarding or conformance issue, never in a carried doc. **A contextually relevant link to a related project is not a coordination reference, and is expected.** Where another repo is part of this repo's subject matter (the image that consumes this config, the builder that generates this hardware, a library this depends on), link it normally. The test is whether the link serves a reader of *this* repo's content, not whether the target happens to be in the fleet. This rule governs carried template content. A repo's own `README.md` and topical docs are its own content, not carried verbatim, and it does not reach them. This pairs with the present-tense rule below: state the current shape, not a history of which repo it came from. ### Markdown -- **Reference-style links in human-facing docs.** Every markdown file uses reference-style links only, **except** the four agent-instruction files that keep inline links: `AGENTS.md`, `GOVERNANCE.md`, `OPERATIONS.md`, and `.github/copilot-instructions.md`. Those are read one section at a time rather than end to end, so an inline target resolves where it is read while a definition at the bottom of the file is never reached. The exception is that closed list rather than a category to argue from, so every other markdown file follows the rule whatever its audience. Under the rule: every URI (internal path, anchor, external URL, or shield image) is defined at the **bottom of the file**, split into groups by type under an HTML-comment header (e.g. `<!-- Shields -->`, `<!-- Workflow -->`, `<!-- Repo -->`, `<!-- External -->`) with each group's definitions alphabetized by reference name. **Reference names are contextual and encode the target and its group**: `foo-shield` for a shield image, `foo-link` for an external URL, and a bare `foo` for a local path or anchor (e.g. `[license-shield]`, `[releases-link]`, `[repo-config]`), never numeric (`[1]`) or opaque. No inline `[text](uri)` targets in prose. **A URL inside a fenced code block stays inline**, because reference links do not resolve in code blocks, so do not extract it, and exclude fenced code from any link-integrity check (bracket literals like `["a", "b"]` otherwise read as undefined references). **Removing a link also removes its reference definition**, since an orphaned definition fails the no-unused-defs rule. The one exception is the Table of Contents, whose entries stay inline anchor links (see Table of Contents below). +- **Reference-style links in human-facing docs.** Every Markdown file uses reference-style links only, **except** the four agent-instruction files that keep inline links: `AGENTS.md`, `GOVERNANCE.md`, `OPERATIONS.md`, and `.github/copilot-instructions.md`. Those are read one section at a time rather than end to end, so an inline target resolves where it is read while a definition at the bottom of the file is never reached. The exception is that closed list rather than a category to argue from, so every other Markdown file follows the rule whatever its audience. Under the rule: every URI (internal path, anchor, external URL, or shield image) is defined at the **bottom of the file**, split into groups by type under an HTML-comment header (e.g. `<!-- Shields -->`, `<!-- Workflow -->`, `<!-- Repo -->`, `<!-- External -->`) with each group's definitions alphabetized by reference name. **Reference names are contextual and encode the target and its group**: `foo-shield` for a shield image, `foo-link` for an external URL, and a bare `foo` for a local path or anchor (e.g. `[license-shield]`, `[releases-link]`, `[repo-config]`), never numeric (`[1]`) or opaque. No inline `[text](uri)` targets in prose. **A URL inside a fenced code block stays inline**, because reference links do not resolve in code blocks, so do not extract it, and exclude fenced code from any link-integrity check (bracket literals like `["a", "b"]` otherwise read as undefined references). **Removing a link also removes its reference definition**, since an orphaned definition fails the no-unused-defs rule. The one exception is the Table of Contents, whose entries stay inline anchor links (see Table of Contents below). - **Table of Contents.** Generate it with the Markdown All in One extension, which fills and auto-updates the list on save, so leave the `## Table of Contents` heading for the extension to populate and never hand-author or hand-edit the entries. Exclude a heading with an inline `<!-- omit from toc -->` marker on it (the badge/build header block and the `## Table of Contents` heading itself carry it). The workspace sets which heading levels appear. - One logical paragraph per line, with no hard-wrap line-length limit. For an intentional hard line break within a block (stacked badges, status, or license lines), end the line with a trailing backslash (`\`). This explicit form is preferred over trailing whitespace and is not treated as a paragraph split. - Headings follow the title-case-with-short-bind-words rule from the PR-title section. @@ -313,8 +315,9 @@ Anti-pattern: don't keep flipping the code on the same style point. Flip the rul ## Communicating with the User -- **Reference every pull request as a clickable link.** When you mention a PR (in chat, a summary, or a report), render it as a markdown link to the PR (`[#123](https://github.com/OWNER/REPO/pull/123)`), never a bare `#123`. The same applies to issues and commits. +- **Reference every pull request as a clickable link.** When you mention a PR on a surface that renders Markdown (chat, a summary, a report), render it as a Markdown link to the PR (`[#123](https://github.com/OWNER/REPO/pull/123)`), never a bare `#123`. The same applies to issues and commits. **The form follows the surface.** Some surfaces link neither a Markdown link nor a bare URL, an interactive prompt's question and option text among them, and pasting a full URL into one of those does not rescue it, since the reader gets a string to copy, which is the outcome this rule exists to prevent. There the reference is a bare `#123`, and the clickable link goes in the message that accompanies the prompt. The test is whether the reader can click it where it is read, not whether it was written in the syntax that works elsewhere. - **Ask for input as a numbered list.** When you need the user to decide or answer, present the questions, and any options, as a numbered list so they can reply per number. A single inline question is fine, and two or more are always numbered. +- **Raise work blocked on the user as a direct interactive prompt.** When progress needs a decision, an authorization, or an answer only the user can give, ask for it through the interface's own prompt mechanism, at the point the work stops. Never leave it as prose in a summary: a handoff buried in a paragraph is a handoff that did not happen, because a summary reads as a report of finished work and the one line still waiting on the user is the easiest in it to skim past. The blocked item is the message, not a closing remark on a message about something else. **The options offered are the actions themselves**, and the one that unblocks the work names the action it authorizes ("squash and merge it"), so selecting it is the go-ahead rather than a note to act on later. Offering only ways to wait is the same failure in interactive clothing, since a prompt whose every choice is inaction reports the block rather than clearing it, and where the agent may not perform the authorized action itself, the option says who does it. This supersedes the numbered-list rule above wherever an interactive prompt is available, and the numbered list is the fallback where none is. ## Workflow YAML Conventions @@ -398,7 +401,7 @@ Contributors commit to this repo with signed commits. The SSH-signing setup live Every repo's GitHub repository details (the About panel) follow a fixed convention so the fleet stays consistent and self-describing. -- **Description** matches the README's first non-empty line after the `#` H1 heading, as plain text, stripping markdown links (`[text](url)` and `[text][ref]` become `text`) since a description is not rendered. The README is the source of truth: set the description from it (`gh api -X PATCH repos/<owner>/<repo> -f description=...`), never the reverse. When the current description is *more specific* than the README (a chip revision or variant the README omits), surface the drift to the maintainer rather than silently discarding the detail, and the fix is to sharpen the README so the description follows it. Keep the line at most **100 characters**, Docker Hub's short-description cap and the tightest surface it feeds. For a repo that publishes a Docker image, the **Docker Hub short description** mirrors the same README intro line, so one canonical sentence carries to the README, the About panel, and Docker Hub alike. +- **Description** matches the README's first non-empty line after the `#` H1 heading, as plain text, stripping Markdown links (`[text](url)` and `[text][ref]` become `text`) since a description is not rendered. The README is the source of truth: set the description from it (`gh api -X PATCH repos/<owner>/<repo> -f description=...`), never the reverse. When the current description is *more specific* than the README (a chip revision or variant the README omits), surface the drift to the maintainer rather than silently discarding the detail, and the fix is to sharpen the README so the description follows it. Keep the line at most **100 characters**, Docker Hub's short-description cap and the tightest surface it feeds. For a repo that publishes a Docker image, the **Docker Hub short description** mirrors the same README intro line, so one canonical sentence carries to the README, the About panel, and Docker Hub alike. - **Topics** are optional, and any that are present match the repo's actual content. Do not invent topics to fill the field. - **Include in the home page**: Releases on, Deployments off, Packages off. These toggles are UI-only, since the REST and GraphQL APIs neither read nor write them, so they are set by hand and cannot be audited through `gh`. diff --git a/OPERATIONS.md b/OPERATIONS.md index 30e98be3..84a98154 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -6,7 +6,7 @@ How this repository is run. It ships no application code, so its operations are ### Run the gates the way CI runs them -CI passes explicit `--check` lists, and a bare `python3 scripts/prose_lint.py [file]` runs `DEFAULT_RULES`, which omits `comment-wrap` and `comment-case`. A bare run therefore under-reports against what CI checks, and a clean result from it proves less than it appears to. Run the CI invocations: +CI passes explicit `--check` lists, and a bare `python3 scripts/prose_lint.py [file]` runs `DEFAULT_RULES`, which is those two lists together. What differs is the exit code rather than the coverage: CI gates on `charset`, `dupword` and `spelling` and reports the other five warn-only, where a bare run exits non-zero on any of the eight. `sentence-split` is in neither and is asked for by name. Run the CI invocations: ```sh python3 scripts/test_prose_lint.py @@ -43,7 +43,7 @@ python3 spec/audit.py [RepoName] # one repo python3 spec/audit.py --issue [RepoName] ``` -Findings are a point-in-time snapshot read live over the API. Re-run before acting on one, and quote the run stamp in any issue derived from it. The deterministic subset lives here, and the full letter-and-intent verdict is [AUDIT.md](./AUDIT.md). +Findings are a point-in-time snapshot read live over the API. Re-run before acting on one, and quote the run stamp in any issue derived from it. The deterministic subset lives here, and the full letter-and-intent verdict is [AUDIT.md](./AUDIT.md). No project-type check in `spec/project-types.json` runs here, and the cross-cutting ones are covered only in part, so read a clean run as evidence for the subset above and not for AUDIT.md section 4. ### Apply or verify repository configuration @@ -87,7 +87,7 @@ docker run --rm --pull=always -v "$PWD":/workdir --workdir /workdir davidanson/m docker run --rm --pull=always -v "$PWD":/workdir --workdir /workdir ghcr.io/streetsidesoftware/cspell:latest --no-progress README.md HISTORY.md ``` -Both commands are the canonical invocations from [GOVERNANCE.md](./GOVERNANCE.md). markdownlint reads every markdown file, while cspell reads `README.md` and `HISTORY.md` only. That narrower spelling scope is deliberate, since gating every markdown file would mean padding `cspell.json` with technical terms without end, and broad live spell-check is the editor extension's job. Widening it here produces noise that no gate acts on. +Both commands are the canonical invocations from [GOVERNANCE.md](./GOVERNANCE.md). markdownlint reads every Markdown file, while cspell reads `README.md` and `HISTORY.md` only. That narrower spelling scope is deliberate, since gating every Markdown file would mean padding `cspell.json` with technical terms without end, and broad live spell-check is the editor extension's job. Widening it here produces noise that no gate acts on. The `editorconfig-checker` action is setup-only. Using it alone silently skips the check, so CI invokes the checker itself rather than relying on the action. diff --git a/STANDUP.md b/STANDUP.md index 70958f6f..209c16ad 100644 --- a/STANDUP.md +++ b/STANDUP.md @@ -92,7 +92,7 @@ Copy every [`spec/files.json`][files] entry whose `appliesTo` matches the repo's - `OPERATIONS.md`: how the repo is run, under the headings `Runbooks`, `Backup and Recovery`, `Logs and Debugging`, `Tool Usage`, and `Configuration Layout`. - `TODO.md`: the repo's running backlog, per [`spec/readme-structure.md`][readme-structure]. It keeps open work out of the README's section order, where it does not belong and changes on a different cadence from everything around it. -**`OPERATIONS.md` is required on every repo**, not optional, so it appears in the baseline above with `appliesTo: "*"`. It is presence-checked only, the same footing as `README.md` and `HISTORY.md`, so its content is entirely the repo's own and a repo with little to say still carries the file as a stub, meaning those five headings with no content under them, for which this repo's own `OPERATIONS.md` is the worked example. Do not read the `operational` workflow model into the requirement, because that selector describes where config lives rather than whether the repo has runbooks, and a repo that publishes to a package registry or deploys a site has operational surface under either model. It is the operational analogue of `ARCHITECTURE.md`, and it is where an `AGENTS.md` split puts the repo-specific half, so real runbooks (a deploy procedure, a rollback, a retention policy, a credential rotation) go there rather than into a carried file. It is agent-instruction content, so it takes the inline-link exception the markdown rules name rather than the reference-style default. `ARCHITECTURE.md` and `TODO.md` stay advisory and are required by no selector, so a repo with nothing to say in one carries no file rather than an empty one. +**`OPERATIONS.md` is required on every repo**, not optional, so it appears in the baseline above with `appliesTo: "*"`. It is presence-checked only, the same footing as `README.md` and `HISTORY.md`, so its content is entirely the repo's own and a repo with little to say still carries the file as a stub, meaning those five headings with no content under them, for which this repo's own `OPERATIONS.md` is the worked example. Do not read the `operational` workflow model into the requirement, because that selector describes where config lives rather than whether the repo has runbooks, and a repo that publishes to a package registry or deploys a site has operational surface under either model. It is the operational analogue of `ARCHITECTURE.md`, and it is where an `AGENTS.md` split puts the repo-specific half, so real runbooks (a deploy procedure, a rollback, a retention policy, a credential rotation) go there rather than into a carried file. It is agent-instruction content, so it takes the inline-link exception the Markdown rules name rather than the reference-style default. `ARCHITECTURE.md` and `TODO.md` stay advisory and are required by no selector, so a repo with nothing to say in one carries no file rather than an empty one. Choose the destination while scaffolding rather than after. Repo-specific content left in a carried file is drift, which the audit lists as an undeclared section to reconcile, and reconciling it later means moving prose that downstream readers have already started trusting in the wrong place. @@ -129,8 +129,8 @@ Run [`AUDIT.md`][audit] end to end. The repo is stood up only when it is **opera When a repo matches no existing type, the work is onboarding a **type**, not just a repo: -1. Add the type to [`spec/project-types.json`][project-types] (`detect[]`, plus `checks` with verdict tiers and intent refs) and any per-type files to [`spec/files.json`][files], then add its publish mechanism to [`spec/secrets.json`][secrets] if new. -2. Add the reference workflow leaf to [`catalog/snippets/workflows/`][workflows] and document the type's [`WORKFLOW.md`][workflow] walkthrough. +1. Add the type to [`spec/project-types.json`][project-types] (`detect[]`, plus `checks` with verdict tiers and intent refs) and any per-type files to [`spec/files.json`][files], then add its publish mechanism to [`spec/secrets.json`][secrets] if new. Add the type's token to [`spec/scope-model.md`][scope-model] and the type itself to [`spec/type-model.md`][type-model] in the same change, which that file's own rule requires. A type publishing to a **new destination** also needs the target added to the closed `target` enum in [`registry/repos.schema.json`][repos-schema] and mapped in `targetMechanisms`, or the first repo declaring it fails `spec/validate.py` with an unknown-target error. +2. Add the reference workflow leaf to [`catalog/snippets/workflows/`][workflows] and document the type's [`WORKFLOW.md`][workflow] walkthrough. A leaf must not be named `build-*-task.yml` unless the type really is a build target, since `source-only.detect` is literally "no `build-*-task.yml`" and the name alone would make that declaration false for any repo carrying both. 3. Add the type to the [conformance matrix][matrix] and run the cold-start self-test until a context-free agent stands it up to operational. ## Self-Test: Cold-Start Conformance @@ -165,9 +165,11 @@ The same [`AUDIT.md`][audit] run is the on-demand audit for any known repo, and [repo-config-carry]: ./docs/repo-config-carry.md [repo-config-readme]: ./repo-config/README.md [repos]: ./registry/repos.json +[repos-schema]: ./registry/repos.schema.json [scope-model]: ./spec/scope-model.md [secrets]: ./spec/secrets.json [section-model]: ./spec/section-model.md [spec]: ./spec/ +[type-model]: ./spec/type-model.md [validate]: ./spec/validate.py [workflow]: ./WORKFLOW.md diff --git a/TODO.md b/TODO.md index a99ead4a..04c96d48 100644 --- a/TODO.md +++ b/TODO.md @@ -19,7 +19,7 @@ Each of these was checked line by line against the current tree and has nothing The gates in [`scripts/`][scripts] are hub-only and are the fleet's main defense against agent-authored drift, so a gate that reports clean without having read anything is the worst failure available here. The `gh-write-guard` hook in [`host-setup/agent-safety/`][agent-safety] is grouped with them, since it is the same kind of instrument even though it ships per host rather than per repo. These entries are ordered so the scope floor lands before anything relies on a clean verdict. - Make [`prose_lint.py`][prose-lint] assert a floor on its own scope, applying to itself the rule [`GOVERNANCE.md`][governance] already states: a gate that finds nothing is indistinguishable from a gate with nothing to find. A `--diff` run that resolves a non-empty diff and then matches **zero** files has almost certainly failed to scope rather than found a clean change, so it should say so instead of exiting 0. One session produced four separate routes to that same false clean: an unresolvable base widening to a whole-tree scan, a multi-line `paths` input read only to its first newline, a diff taken in one repository while scanning another, and a path under no repository at all. Each was fixed with its own guard, which is the wrong shape, because the fifth route will need a fifth guard and will be found the same way the first four were, by a reviewer rather than by the gate. A floor assertion covers the family. Note the honest limit before building it: a change touching only files the gate does not read (an image, a lock file) legitimately scopes to zero, so the assertion compares against the diff's own file list rather than against zero alone. This lands first, because every entry below it produces verdicts that are only worth reading once the gate can prove it read something. -- Clear the [#519][issue-519] prose backlog, starting with `catalog/snippets/**`. Two of the four changes that issue planned have already landed and its headline numbers are stale, so re-measure before quoting anything from it. `comment-wrap` and `comment-case` are now in `DEFAULT_RULES`, and `reports/` is now exempt as a generated tree, which is why the current figures are **534** whole-tree under the default rules and **184** under `catalog/snippets` alone rather than the 668 and 119 the issue records. The snippets lead the fix order for the reason the issue gives, that a non-conformant snippet seeds its violations into every repo that adopts it and the downstream repo is then flagged for content it was handed. What remains undecided is whether the gate itself becomes a carried file rather than a hub-only one, which is the same question the entry below on hub-only machinery asks from the other direction. +- Clear the [#519][issue-519] prose backlog, starting with `catalog/snippets/**`. Two of the four changes that issue planned have already landed and its headline numbers are stale, so re-measure before quoting anything from it. `comment-wrap` and `comment-case` are now in `DEFAULT_RULES`, and `reports/` is now exempt as a generated tree, which is why the current figures are **559** whole-tree under the default rules and **184** under `catalog/snippets` alone rather than the 668 and 119 the issue records. The whole-tree figure is the one that moves, having been 534 when this entry was written and 520 by the time [#570][issue-570] scoped the semicolon exemption to the sentence it lives in, which reported 44 semicolons the rule had been silent on. Re-measure it rather than quoting it, since a fix to the gate moves it as readily as a fix to the prose does. The snippets lead the fix order for the reason the issue gives, that a non-conformant snippet seeds its violations into every repo that adopts it and the downstream repo is then flagged for content it was handed. What remains undecided is whether the gate itself becomes a carried file rather than a hub-only one, which is the same question the entry below on hub-only machinery asks from the other direction. - Gate the pattern-detectable half of the representative-data rule in [`prose_lint.py`][prose-lint], meaning an absolute home path (`/home/<name>`, `/Users/<name>`, `C:\Users\<name>`) or a bare drive letter sitting in committed prose, a code comment, or a fixture. [`GOVERNANCE.md`][governance] "Representative Data in Agent-Authored Text" states the rule and says why a check is a floor rather than an answer, so the check is introduced as covering the easy half or it gets read as closing the rule, which is the specific way it would make things worse. Three things to settle when writing it. The exemption carries the whole burden, since the rule's own wording, the `host-setup/` docs, and the audit's examples all quote path shapes in order to describe them, and a wrong exemption hands out a work list that damages correct documents. The leak that motivated the rule was in a pull request comment, which no committed-file linter reads at all, so say what surface the gate covers rather than letting its name imply the rule. And a home path in an operational repo's runbook may be the literal path an operator types, which is the repo's own content rather than an agent quoting the maintainer's environment, so decide whether the finding is scoped by file, by repo type, or left for the author to judge. - Teach the `sha-pin` check in [`repo_gate.py`][repo-gate] to verify a pin **resolves**, not merely that it is shaped like a SHA. Forty hex characters is a format any fabricated string satisfies, and an agent hand-writing a plausible SHA into a workflow is a real failure mode rather than a hypothetical one. A resolvability check also catches the neighboring case, a pin whose commit was reachable only from a branch that has since been squashed and deleted, which breaks a downstream gate long after the change that caused it. Scope the network call to same-owner repositories, where the fleet's own actions live, and skip rather than fail when the host is offline so the local gate stays usable. Note that the existing `gh-write-guard` hook cannot cover this, since it watches Bash and an editor tool writing the same string into a file never reaches it. - Add a check that a pull request's **description** does not contradict its own branch. Three stale descriptions in one session generated six review findings between them, each one a reviewer noticing that the body named a commit, a branch, or a behavior the branch no longer carried. The cheap and precise form is to extract SHAs and `uses:` refs quoted in the body and confirm each still appears in the head tree, since those are the claims that go stale silently and the ones a reviewer actually catches. Prose claims are out of scope, and deliberately so: judging those needs a similarity heuristic, which [`spec/section-model.md`][section-model] already rejects for exactly the reason it would fail here. @@ -30,22 +30,22 @@ The gates in [`scripts/`][scripts] are hub-only and are the fleet's main defense Everything here changes a file the fleet holds a copy of, so each entry costs a hub edit plus a sweep, and the sweep is the part that gets skipped. -- Re-vendor the changed `verbatim` content across the fleet, which is one sweep covering three files. `repo-config/configure.sh` is carried `verbatim` with `appliesTo: "*"` and the hub swept it to one sentence per line. In `AGENTS.md`, "Context and Delegation Discipline" carries the wait rule's failure clause and "Where the Rules Live" carries a row for the new section named below. In `GOVERNANCE.md`, "Verification Discipline" carries the rule that a launched process is not a result, "PR Review Etiquette" carries the five outcomes that close a finding, and "Representative Data in Agent-Authored Text" is an entirely new carried section that no downstream repo holds, which the audit reports as a missing section rather than as drift. Every repo already holding a copy of a changed section is byte-mismatched against the hub until it takes the new one, which the audit reports as stale rather than modified. This sweep is also the follow-through [#489][issue-489] and [#379][issue-379] were waiting on, and the `.editorconfig` line in [#353][issue-353] rides the same visit to each repo. Regenerate [reports/divergences.md][divergences-report] before using it as the work list, since the committed copy is dated 2026-07-22 and therefore predates the router split, which shows in it naming "Repository Boundaries and Write Safety", "Git and Commit Rules" and "Verification Discipline" as `AGENTS.md` sections when all three now live in `GOVERNANCE.md`. A stale ledger is the same hazard as a stale exemption, in that it hands out a work list measured against a tree that no longer exists. +- Re-vendor the changed `verbatim` content across the fleet, which is one sweep covering five files. `repo-config/configure.sh` is carried `verbatim` with `appliesTo: "*"` and the hub swept it to one sentence per line. In `AGENTS.md`, "Context and Delegation Discipline" carries the wait rule's failure clause and "Where the Rules Live" carries a row for the new section named below. In `GOVERNANCE.md`, "Verification Discipline" carries the rule that a launched process is not a result, "PR Review Etiquette" carries the five outcomes that close a finding, "Repository Boundaries and Write Safety" carries the rule that a refused write is reported rather than re-shaped, and "Representative Data in Agent-Authored Text" is an entirely new carried section that no downstream repo holds, which the audit reports as a missing section rather than as drift. Three further `GOVERNANCE.md` sections differ by a single word each, "Documentation Style Conventions", "Communicating with the User" and "Repository Details", where the format's name was capitalized to the convention `CODESTYLE.md` "Markdown and Spelling" now states, so they are byte-mismatched for a reason a reader of the diff would otherwise call cosmetic. Two comment lines in `.markdownlint-cli2.jsonc` took the same capitalization, and that file is `verbatim` and `whole`, so every downstream copy is byte-mismatched on a config nothing else changed about. `CODESTYLE.md` carries the new item and is the fifth file, at `intent` rather than `verbatim` fidelity, so it reaches the fleet as a rule each repo adopts in its own copy rather than as bytes to match, and the same mixed spelling is waiting in every downstream tree. Every repo already holding a copy of a changed section is byte-mismatched against the hub until it takes the new one, which the audit reports as stale rather than modified. This sweep is also the follow-through [#489][issue-489] and [#379][issue-379] were waiting on, and the `.editorconfig` line in [#353][issue-353] rides the same visit to each repo. Regenerate [reports/divergences.md][divergences-report] before using it as the work list, since the committed copy is dated 2026-07-22 and therefore predates the router split, which shows in it naming "Repository Boundaries and Write Safety", "Git and Commit Rules" and "Verification Discipline" as `AGENTS.md` sections when all three now live in `GOVERNANCE.md`. A stale ledger is the same hazard as a stale exemption, in that it hands out a work list measured against a tree that no longer exists. - Drop the unused `GITHUB_TOKEN` grants from the merge-bot workflow, per [#521][issue-521], which the tree confirms is still open, since all four jobs still carry `contents: write` and `pull-requests: write` while every write in the file authenticates with the App token. The finding is least privilege on the highest-blast-radius shape in the fleet, a `pull_request_target` workflow holding an App private key, where the grant is not exploitable today only because no step consumes it. Three things to settle: whether to drop the job-level blocks or set a workflow-level `permissions: {}`, whether the audit should compare permissions at all given the file is `interface` fidelity with only a `requiredJobKeys` contract, and the re-vendor, since every repo carries the file. Note that [`spec/files.json`][files] now declares this workflow at `appliesTo: "*"`, which closes the separate gap [#456][issue-456] raised, that D8.1 graded a file the file spec never required. - State the rule that an agent never assumes a Docker image is present locally, however recently it pulled one, because a background prune can remove an image between two commands of the same session and the run then fails on something that worked minutes earlier. The prescription is the always-pull flag by default, and an explicit `docker pull <image>:<tag>` before use where the flag does not apply. The four documented lint invocations already carry `--pull=always`, in [`GOVERNANCE.md`][governance] "Running the Linters Locally", mirrored in [`OPERATIONS.md`][operations] and both VS Code task snippets, so what is missing is the rule rather than the one-liners: an agent composing an ad-hoc `docker run` drops the flag precisely because it believes the image is already cached. Two things to settle when writing it. Where it lives, since that section is scoped to the four lint tools while the rule covers any container an agent starts, which argues for placing it with the Docker or verification guidance instead, and whether it is carried to the fleet, since every repo runs the same images from the same instructions. Note the limit before writing it, so the flag does not read as the whole answer: `docker run` against a registry tag re-pulls an absent image on its own, so the cases that actually break are a locally built tag with no registry to pull from, and any command that branches on the image being present (`docker image inspect`, `docker images`) rather than simply running it. - Decide where a carried file may name hub-only machinery, since `GOVERNANCE.md` "PR Review Etiquette" points at `scripts/pr_review.py` and the fleet carries the section but not the script. A downstream reader follows that pointer to a path their repo does not have. Either the script joins the carried set, or the rule states the behavior and drops the tool name the way the coordination-reference rule already requires for the template repo itself. - Revisit [#305][issue-305], which asks for a propagation path for the rule docs and is the oldest open issue here, because most of the machinery it asked for has since been built and the issue has not been re-scoped to match. The fidelity model, the `spec/files.json` manifest, [`spec/divergences.json`][divergences] with its generated [reports/divergences.md][divergences-report], and [`AUDIT.md`][audit-doc] section 10 together give the canonical-versus-adapted split and the audit path the issue proposed. What is genuinely still missing is the push half, since every one of those detects drift while the sweep that fixes it is manual, which is exactly the entry above. Either re-scope the issue to the push half or close it against the entry above and the audit-automation entry below. -- Reduce the surface that is copied downstream at all, since every carried file is a copy that breaks when it is not the current one. `repo-config/configure.sh` is the worked example from this session: a defect in its check mode was fixed once in the hub and immediately turned into a re-vendor owed by every repository carrying it, and until each takes the fix they run different logic while reporting the same command. A repository that misses the sweep does not fail loudly, it quietly audits itself with an older script. The alternative is for the hub to host the tool and for each downstream repository to call it rather than hold a copy, so one edit reaches the fleet and no stale copy exists to detect, re-vendor, or classify. This is the same argument as the reusable-workflow entry below and should be settled with it rather than separately, since both are about consuming hub code at runtime instead of vendoring it. Three things to settle. What the fleet actually needs vendored, which is the content a repository is audited against and could not run without, against what only needs to be reachable. How a consumed tool is pinned, because consuming hub code at runtime is a floating dependency unless it names a commit, and the action-pinning rule already requires exactly that. And what a repository does when the hub is unreachable, since a vendored copy still works offline and a called one does not, which is the real cost being traded away. +- Reduce the surface that is copied downstream at all, since every carried file is a copy that breaks when it is not the current one. `repo-config/configure.sh` is the worked example from this session: a defect in its check mode was fixed once in the hub and immediately turned into a re-vendor owed by every repository carrying it, and until each takes the fix they run different logic while reporting the same command. A repository that misses the sweep does not fail loudly. It quietly audits itself with an older script. The alternative is for the hub to host the tool and for each downstream repository to call it rather than hold a copy, so one edit reaches the fleet and no stale copy exists to detect, re-vendor, or classify. This is the same argument as the reusable-workflow entry below and should be settled with it rather than separately, since both are about consuming hub code at runtime instead of vendoring it. Three things to settle. What the fleet actually needs vendored, which is the content a repository is audited against and could not run without, against what only needs to be reachable. How a consumed tool is pinned, because consuming hub code at runtime is a floating dependency unless it names a commit, and the action-pinning rule already requires exactly that. And what a repository does when the hub is unreachable, since a vendored copy still works offline and a called one does not, which is the real cost being traded away. - Investigate replacing copy-pasted workflow content with cross-repo reuse, now that this repo is public. A public repository's composite actions and reusable workflows can be consumed by any other repository regardless of owner type, so the organization account this pattern was assumed to require is not needed, and the constraint that shaped the current vendor-everything model no longer holds. The catalog under [`catalog/snippets/workflows/`][workflows] is copied into each repo today, which means a fix to a shared job is a sweep across the fleet rather than one edit here, and it is the mechanism by which a defect in a snippet seeds itself into every repo that adopted it. Scope the investigation to which jobs are genuinely identical across repos against which only look similar, since a reusable workflow that needs a long input list to cover per-repo variation is worse than the copy it replaces. Settle the ref policy in the same pass, because consuming hub code at CI time is a floating dependency unless it is pinned, and [`GOVERNANCE.md`][governance] "Action pinning" requires a commit SHA for every action with one documented exception. Note that `uses:` does not accept expressions, so a per-branch ref cannot be selected in the workflow file and any branch-dependent behavior belongs inside the consumed action instead. ## Audit and Spec Model - Teach the audit to see content sitting in the wrong file, per [#523][issue-523], which the tree confirms is untouched, since [`spec/audit.py`][audit] still checks file presence, declared-section presence, verbatim hashes, and workflow interface conformance and nothing that reads a heading against a destination. The case that found it is a repo whose `.github/copilot-instructions.md` carried 311 lines under nine headings that [`spec/section-model.md`][section-model] assigns to `ARCHITECTURE.md` and `OPERATIONS.md`, reported as a missing-file letter while the misplacement that caused it was invisible. The shape that fits the engine is structural, collecting the level-two headings of an `intent` file, subtracting the ones [`spec/files.json`][files] declares for it, and comparing the remainder against the headings other destinations declare. The similarity-based version is explicitly rejected by `section-model.md`, and a detector built on it produces findings whose remedy is to delete content. Four things to settle, all in the issue: whether an undeclared heading is a finding at all given a repo may add locally, whether the destination mapping becomes declared data rather than prose, whether it reaches the advisory `ARCHITECTURE.md`, and how many repos are affected, which is measured before the check is designed rather than after it starts reporting. - State what a `driftNote` may contain at the moment one is written, not only what happens to it once the deviation clears, and widen the check that guards it. [`AUDIT.md`][audit-doc] section 8 says a note records a current deviation and is deleted once resolved, which is a rule about a note's end of life and leaves an agent authoring one with no rule about its content. PhotoCleaner is the worked example, in [#537][pr-537]: asked to self-audit and reconcile its notes, the agent wrote a completion record ("baseline onboarding completed"), a historical fact (the date a past version first published), and a description of the repo conforming to the README spec. All three are prose about the past or about compliance rather than a current deviation, and none of them is what the field is for. Read that as an incomplete instruction rather than an agent ignoring one, since nothing it was given forbids any of the three. Two things to settle. The rule wants stating positively at authoring time, that a note names a deviation that is true right now and would still be true if nobody read it, so history, completion, and conformance are each out by construction, and a repo doing exactly what the spec requires earns no note at all. And the detector needs the same widening, because [`spec/audit.py`][audit] currently flags a stale note by matching words that assert *outstanding* work ("pending", "not yet", "missing", "behind"), which is the opposite polarity to all three of PhotoCleaner's notes, and it only runs that check when the repo audits clean, which PhotoCleaner did not until the round that found this. A guard that fires only on one polarity, and only in the state where the problem has already gone, cannot see the failure it exists to catch. Note the limit before building it, since it is the same limit the section model already records: judging whether a sentence describes a current deviation is a semantic call, so the honest target is the shapes that are mechanically recognizable, a past-tense completion verb and an explicit date among them, with the rest left to the rule. -- Declare the canonical short description in [`registry/repos.json`][repos] instead of deriving it by parsing the README, which makes every check and every push deterministic. Today [`spec/audit.py`][audit] extracts the intro line after the H1 and compares the GitHub About panel and the Docker Hub short description against it, so the source of truth lives inside prose. That forces anything wanting the value to parse markdown, and PhotoCleaner#32 shows the cost, since a workflow step that reads the intro at publish time needs nine guards against headings, block quotes, all four list markers, ordered lists, HTML, tables, code, links and the length cap, and every one of them fails the release rather than the tagline. Adding a `description` field turns the README intro into a third mirror rather than the source, so the audit compares all three against one declared value, the Docker workflow pushes a literal with no parser, and `repo-config/configure.sh` can set the GitHub About panel from the same field it already sets every other repository setting from. The 100-character cap stays, since Docker Hub's short description is still the tightest surface. Add the field to `registry/repos.schema.json` as well, which sets `additionalProperties: false`, and make it optional at first so the audit falls back to the README intro while repos adopt it one at a time. The disposition is recorded on [#509][issue-509], and the ask on the Docker repos is only that the parsing step is not propagated further while the field is pending. +- Declare the canonical short description in [`registry/repos.json`][repos] instead of deriving it by parsing the README, which makes every check and every push deterministic. Today [`spec/audit.py`][audit] extracts the intro line after the H1 and compares the GitHub About panel and the Docker Hub short description against it, so the source of truth lives inside prose. That forces anything wanting the value to parse Markdown, and PhotoCleaner#32 shows the cost, since a workflow step that reads the intro at publish time needs nine guards against headings, block quotes, all four list markers, ordered lists, HTML, tables, code, links and the length cap, and every one of them fails the release rather than the tagline. Adding a `description` field turns the README intro into a third mirror rather than the source, so the audit compares all three against one declared value, the Docker workflow pushes a literal with no parser, and `repo-config/configure.sh` can set the GitHub About panel from the same field it already sets every other repository setting from. The 100-character cap stays, since Docker Hub's short description is still the tightest surface. Add the field to `registry/repos.schema.json` as well, which sets `additionalProperties: false`, and make it optional at first so the audit falls back to the README intro while repos adopt it one at a time. The disposition is recorded on [#509][issue-509], and the ask on the Docker repos is only that the parsing step is not propagated further while the field is pending. - Rework [`spec/readme-structure.md`][readme-structure] to match the hand-crafted PlexCleaner README, which is the shape the maintainer wants, and make the result auditable rather than advisory. Four concrete divergences are already identified, measured against PlexCleaner `README.md`, this repo's `README.md`, and the current spec. First, the distribution bullet is labeled by deliverable: PlexCleaner ships executables and calls the channel **Binary Releases**, while the spec fixes the label as **Versioned Releases** for every repo, so the label belongs in a per-channel table rather than as one string. Second, the license shield sits in the top **Build Status** block here and at the very bottom of PlexCleaner, inside a closing `## License` section that reads `Licensed under the [MIT License]` followed by the shield, immediately before the link definitions. Third, the Release Notes section closes with `See Release History for complete release notes and older versions.` in PlexCleaner against `See Release History for the full history.` here, and the PlexCleaner form is the wanted one. Note that PlexCleaner writes that link inline, which the reference-style rule forbids, so adopt the wording and keep the reference form. Fourth, the channel bullets and their shields vary by deliverable, meaning GitHub binaries, Docker Hub, NuGet, and PyPI each carry a different bullet label and a different shield set, which is what a per-type table has to encode for the `readme-structure` audit dimension to check a repo against its own declared types. - Decide whether the canonical README section order follows PlexCleaner, which is a separate question from the four divergences above and affects every repo plus the `readme-structure` audit. PlexCleaner places **Questions or Issues** immediately after the Table of Contents, where the spec orders it ninth, and it carries sections the spec names nowhere, including Performance Considerations, Runtime Metrics, Custom Plugins, Testing, Development Tooling, Feature Ideas, and Sample Media Files. Under the recurrence rule in [`spec/section-model.md`][section-model] those last ones are correctly repo-specific and stay undeclared, so the open question is only the position of the sections the spec already names. -- Declare locally-required secrets the way GitHub-stored ones are already declared, and make a gitignored `secrets/` directory the fleet standard that holds them. [`spec/secrets.json`][secrets] covers only the Actions and Dependabot stores, so a repo that deploys somewhere has no declared way to say what it needs at runtime, and the required set is discoverable only by reading the deploy. The pattern already runs in the fleet in two shapes: HomeAutomation-Config keeps a gitignored secrets directory of env files and Docker secret files, and ESPHome-Config keeps a gitignored `secrets.yaml` beside a committed `_secrets.yaml`. The committed file carries the required names with dummy values, so the shape of the requirement is in git while the values never are, which is the same split the GitHub side already gets from `requiredSecrets[]`. Blog needs it immediately, since it deploys on the proxmox host through HomeAutomation-Config's Docker Compose stack and carries the copy destinations and the internal URI. The hub carries neither the directory nor a `.gitignore` entry for one today, so adopting it here comes first. +- Declare locally-required secrets the way GitHub-stored ones are already declared, and make a gitignored `secrets/` directory the fleet standard that holds them. [`spec/secrets.json`][secrets] covers only the Actions and Dependabot stores, so a repo that deploys somewhere has no declared way to say what it needs at runtime, and the required set is discoverable only by reading the deploy. The pattern already runs in the fleet in two shapes: HomeAutomation-Config keeps a gitignored secrets directory of env files and Docker secret files, and ESPHome-Config keeps a gitignored `secrets.yaml` beside a committed `_secrets.yaml`. The committed file carries the required names with dummy values, so the shape of the requirement is in git while the values never are, which is the same split the GitHub side already gets from `requiredSecrets[]`. Blog needs it immediately, since it deploys on the proxmox host through HomeAutomation-Config's Docker Compose stack and carries the copy destinations and the internal URI. The hub carries neither the directory nor a `.gitignore` entry for one today, so adopting it here comes first. Note that the **GitHub** side has the same missing axis, surfaced by the `hugo` type: a deploy's credentials are per-environment GitHub Environment secrets and variables, `stores` is a closed enum of `actions` and `dependabot`, and `spec/audit.py` seeds `required_by_store` with those two keys and indexes it unguarded, so adding an `environments` value to the enum raises `KeyError` for every repo whose `publish[]` maps to that mechanism. The type landed with an optional `environments` block added to [`spec/secrets.schema.json`][secrets-schema] and described in `spec/secrets.json`'s note, so a repo that needs it may declare its per-environment names and the shape is legal rather than an invention. The hub's own `spec/secrets.json` carries no such block, having no environments of its own, and no tool reads one where it exists. That is honest but it is not a gate, so a clean audit says nothing about whether an environment is configured. Solving it properly means a store vocabulary that does not crash on an unknown key and a per-environment secrets and variables fetch, and it is the same axis as the local half above rather than a separate problem. - Canonicalize Python linter-config placement on `pyproject.toml`, since one cataloged repo uses a standalone `.ruff.toml` plus `pyrightconfig.json`. Track it as a drift finding and fix it downstream. - Populate [reports/][reports] for the cataloged repos that still have no audit, since a registry `status` of `cataloged` asserts a result that only a committed report evidences. Eight repos have one. This is paced by maintainer capacity rather than blocked on anything, since repos are brought up to spec as they are worked on, so the entry records the outstanding set rather than a defect. - Revisit automating the audit, which was explored and deliberately deferred, recorded here so the reasoning is not re-derived from scratch. Three shapes were considered: a scheduled hub-driven audit publishing each report as a workflow artifact, the same thing committing the report back the way the codegen bot updates its own files, and a pull-request hook in each downstream repo that audits itself against the current hub. Three things blocked all of them. Until the fleet reaches stasis with every repo onboarded, a scheduled run reports mostly noise, since a repo mid-onboarding is expected to be non-conformant. The hub has to be stable before downstreams can audit against it, because a hub change lands as fleet-wide findings the same day. And the downstream half is a catch-22, since a self-auditing pull-request hook is CI instrumentation the repos that most need it do not yet carry. The agreed outcome was the on-demand audit that [`AUDIT.md`][audit-doc] describes today. Worth reopening once the fleet is onboarded and the hub goes a stretch without carried-content changes, and the artifact shape is the one to try first, since it produces evidence without committing anything and so cannot generate review load while the noise level is still unknown. @@ -53,7 +53,7 @@ Everything here changes a file the fleet holds a copy of, so each entry costs a ## Onboarding and New Types - Give [`STANDUP.md`][standup] an explicit branch-bootstrap step, because it currently says nothing about how `main` and `develop` come into being and an agent therefore commits onto whichever branch it finds. Note that section 0A now covers the human-gated prerequisites from [#490][issue-490], so this is the remaining half of that gap rather than the whole of it. The sequence that avoids every cleanup problem is: create `main` and leave it carrying nothing, create `develop` from `main` and leave it carrying nothing, create the first feature branch from `develop` and do the whole standup there, add the repo to GitHub and apply the repo config while still on that branch, then open a normal pull request to `develop` when the repo is ready. Nothing ever has to be cleaned off `main` or `develop`, because nothing ever reached them without review. Note the mechanical wrinkle when writing this up: a git branch cannot exist without a commit, so "carrying nothing" means exactly one signed empty root commit (`git commit --allow-empty`), and section 0's signing window applies to that commit like any other. The alternative of committing onto `develop` and squashing afterwards does not work: `non_fast_forward` is set on both `develop` payloads, so the history cannot be rewritten without disabling the ruleset, and Blog was correctly blocked when it tried. Worth stating that the protection is uneven, since a `release` repo's `repo-config/develop.json` carries a `pull_request` rule that blocks a direct commit outright, while `repo-config/operational/develop.json` carries only `deletion`, `non_fast_forward` and `required_signatures`, so on an operational repo a direct commit succeeds and only the instruction stands between the agent and an unfixable history. The reason it matters on a public repo is that the exploratory standup history is where PII, secrets and noise commits accumulate, and squashing a feature branch is the one chance to leave them out. -- Add the static-site generator type that [#456][issue-456] has been gathering answers for, once Blog reports its measured shape rather than its expected one. The intake is answered in detail across that issue: dispatch-only publish with the tag cut last after a live check, rsync over SSH into a per-release directory with an atomic symlink swap, one secret plus per-environment variables, `staging` and `production` environments, theme as a Dependabot-tracked submodule, built output never committed, LF line endings under the operational model, and imported content excluded from the prose gates by an explicit provenance table. Two decisions are already settled against the first draft, that the deploy **is** a publish because the pipeline versions it with NBGV and readers consume it, and that the type is named for the generator rather than the transport with the checks phrased so they do not name the generator. The URL parity gate is the check of record, asserting a floor on the golden list length before comparing, since a truncated list makes every check below it pass vacuously. Blog is interim-classified until the type exists, so the deferral clears when this lands. Note two caveats the issue also records and nothing else states: `source-only.detect` is literally `["no build-*-task.yml"]`, so a Hugo build leaf must not be named `build-site-task.yml` or the declaration becomes a false letter, and the `copilot_code_review` rule in both ruleset payloads does not gate a merge today because gated Copilot review is an invite-only beta, which deserves a sentence near the merge gate so no repo reads the rule as the enforcement and relaxes the manual discipline actually holding the line. +- The static-site generator type from [#456][issue-456] and [#558][issue-558] has landed as `hugo`, with the deploy destination as the `self-hosted` publish target and the `deploy-ssh` mechanism. Correct the record while reading this, because three details the intake predicted are wrong against what Blog actually runs, and planning from the prediction would encode requirements the repo does not meet: the theme is **vendored with no recorded upstream ref**, not a Dependabot-tracked submodule; the generator is **pinned by version and SHA256**, not run at latest; and the deploy is a **separate dispatch** from the GitHub release, not a tag cut last after the live check. What was settled and did hold: the deploy is a publish, the type is named for the generator with the generic checks phrased so they do not name it, and the URL parity gate asserting a floor on the golden list length before comparing is the check of record. What remains: promote the generator-agnostic `hugo.*` checks to a shared type when a second generator arrives, which `spec/type-model.md` "Generators" makes a registry edit by construction. Also still true and stated nowhere else, the `copilot_code_review` rule in both ruleset payloads does not gate a merge today because gated Copilot review is an invite-only beta, which deserves a sentence near the merge gate so no repo reads the rule as the enforcement and relaxes the manual discipline actually holding the line. - Finish onboarding hardening (from [#310][issue-310]): make the [`AUDIT.md`][audit-doc] audit a required onboarding step and run the per-type cold-start self-tests tracked in [reports/conformance-matrix.md][matrix]. `STANDUP.md` is already in place. - Add a linter-only Python project type for codegen and boilerplate Python, code that runs during another tool's build to emit generated source (for example ESPHome codegen that produces enriched C++ at compile time), so it ships no unit tests and no coverage and needs only the linter. Keep it distinct from the existing `python` type, which is utility code that can and should carry unit tests and coverage, as in PlexCleaner. Until it exists, ESPHome-Config stays `source-only` and its `+python` reclassification is deferred, so accept its one outstanding validation finding meanwhile. - Add a fleet-standard clang-format config for the `cpp` type, a catalog snippet plus a CODESTYLE C++ section defining the style, the C++ analogue of the shared ruff config, so the `cpp` clang-format check references one canonical style rather than each repo inventing its own. Base it on the ESPHome-Config agent's proposed `.clang-format`. @@ -61,7 +61,7 @@ Everything here changes a file the fleet holds a copy of, so each entry costs a ## Host and Environment - State the rule that an agent works only in its own checkout, in its own directory, on its own feature branch, and never writes to a repository because it happened to be on disk. A clone left in a scratch directory is convenient exactly when it is most dangerous, since the operations that cross the boundary are the ordinary ones rather than the reckless ones. A blanket `git add -A` sweeps another agent's uncommitted work into your commit, a `git reset --hard` deletes it outright, and a branch switch carries it along into a change it has nothing to do with. This session produced the first of those: a blanket add swept a second agent's in-progress registration work into an unrelated commit and pushed it, and the work survived only because it was rescued and restored by hand afterwards. Care does not make a shared working tree safe, because every one of those commands is correct in isolation and wrong only because of who else is in the directory. Two things to settle when writing it. Whether the rule is stated as one checkout per agent or one checkout per task, since a single agent moving between two repositories in one directory hits the same hazard as two agents sharing one. And what an agent does when it finds a foreign checkout anyway, where the answer is to leave it alone and clone its own rather than to work carefully in it. -- Extend the same rule to **reading**, which is the more common case and the easier one to think is harmless. A clone on disk is not the branch it names. It is whatever that clone was last fetched to, so reading it to answer what a repository carries answers instead what that clone last saw. The failure is silent, because the read succeeds and returns a plausible answer. This session produced two instances. A detection pass over local clones reported one repository as still drifted on a file whose fix had already merged, because that clone's refs predated the merge, and the correction came from a fetch rather than from the read. A second pass then reported a repository as missing a file it does carry, because the local checkout sat on a branch behind the one being measured. State the working practice with the rule: read the live ref through the API where a claim will be acted on, or fetch immediately before reading, and name the ref and commit in any finding derived from a local read so a reader can tell what was actually measured. Note the exception worth keeping, which is that a local clone is the right tool for anything needing history or a build, since those are what an API read cannot give. +- Extend the same rule to **reading**, which is the more common case and the easier one to think is harmless. A clone on disk is not the branch it names. It is whatever that clone was last fetched to, so reading it to answer what a repository carries answers a different question, which is what that clone last saw. The failure is silent, because the read succeeds and returns a plausible answer. This session produced two instances. A detection pass over local clones reported one repository as still drifted on a file whose fix had already merged, because that clone's refs predated the merge, and the correction came from a fetch rather than from the read. A second pass then reported a repository as missing a file it does carry, because the local checkout sat on a branch behind the one being measured. State the working practice with the rule: read the live ref through the API where a claim will be acted on, or fetch immediately before reading, and name the ref and commit in any finding derived from a local read so a reader can tell what was actually measured. Note the exception worth keeping, which is that a local clone is the right tool for anything needing history or a build, since those are what an API read cannot give. - Finish the [#365][issue-365] rollout, which has two halves and one of them is invisible from here. Four machines are still unchecked, WSL2 Ubuntu, the MacBook Air, and both ThinkPads, plus any headless or cron environment that runs Claude Code with the token. A ticked row means the `CLAUDE.md` text and not the hook, since only running `host-setup/agent-safety/install.sh` deploys both layers, and the proxmox host proved that distinction the hard way by carrying the documentary half alone for eight days on the very machine where the incident originated. Note also the follow-up left open on this host and never applied, that the superseded `## Safety rules` section from [#364][issue-364] still sits above the canonical `agent-safety v1` block in `~/.claude/CLAUDE.md`, so the two overlap. Removing the superseded section is a judgment call on a per-machine file, which is why it was surfaced rather than applied. - Fill in the remaining cells of the [#483][issue-483] host tooling matrix, which is the reference that makes the per-machine chore above repeatable, so the two close together rather than either alone. Linux, Windows and WSL2 are filled in and verified. **macOS** needs someone on that platform, the **Proxmox** question is whether that host is a shell for agent work or also runs containers, which decides whether `docker` is required there at all, and the engine-inside-the-distro variant of the WSL2 `docker` cell is still unverified since that host only exercised the Docker Desktop integration. Honor the issue's own rule when filling a cell, that an unverified install command is worse than a blank, because a blank prompts a question while a wrong command produces a broken host and a false sense that setup succeeded. @@ -91,6 +91,8 @@ Everything here changes a file the fleet holds a copy of, so each entry costs a [issue-519]: https://github.com/ptr727/ProjectTemplate/issues/519 [issue-521]: https://github.com/ptr727/ProjectTemplate/issues/521 [issue-523]: https://github.com/ptr727/ProjectTemplate/issues/523 +[issue-558]: https://github.com/ptr727/ProjectTemplate/issues/558 +[issue-570]: https://github.com/ptr727/ProjectTemplate/issues/570 <!-- Pull requests --> @@ -115,6 +117,7 @@ Everything here changes a file the fleet holds a copy of, so each entry costs a [repos]: ./registry/repos.json [scripts]: ./scripts/README.md [secrets]: ./spec/secrets.json +[secrets-schema]: ./spec/secrets.schema.json [section-model]: ./spec/section-model.md [standup]: ./STANDUP.md [workflows]: ./catalog/snippets/workflows/ diff --git a/WORKFLOW.md b/WORKFLOW.md index df2ffff3..dcc1e97e 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -132,6 +132,7 @@ Pick each output's path by **where the artifact goes**: - **File on the GitHub release** (zip, binary, packaged library): one leaf per output uploading `release-asset-<branch>-<name>`. The repo keeps `expect_release_assets: true` (its default). - **Package-registry push** (NuGet, PyPI): the leaf builds and publishes to its registry. NuGet pushes from the leaf *and* uploads a `release-asset-*`. PyPI is **split**: the leaf only builds + uploads its build artifact, a separate publish job does the OIDC upload (so `id-token: write` is granted at one entry point, behind an environment gate) and contributes **no** `release-asset-*`. - **Image-registry push** (Docker): the leaf pushes the default branch multi-arch (amd64+arm64) and any other branch `amd64`-only (arm64 emulation is reserved for the released image), and contributes no `release-asset-*`. +- **Filesystem on a host the project owns** (a static site, a config tree): the leaf builds the tree, ships it to the host, and contributes no `release-asset-*`. The transport is the repo's own. What the contract fixes is that the deploy is a **separate `workflow_dispatch`** from the release, so a redeploy of an unchanged commit mints no tag and a host rebuild, a rollback, or proving a branch on a non-production environment costs nothing; that its credentials come from a **per-environment GitHub Environment** rather than the repository secret store; and that the deploy ends by asserting **what the host serves** rather than the transport's exit status (D4.6). Retention at the destination is bounded by a declared count with one side recorded as owning the prune, which is the deploy where its credential can observe the destination and the host where that credential is deliberately write-only (D5.6). - **No file target via the release task** (Docker-only, PyPI-only): the release is tag + source zip + README + LICENSE. The repo's **caller MUST pass `expect_release_assets: false`** to the release task (the input is never set by a publisher that ships file targets, which keeps the default `true`). This is the one case where the otherwise-verbatim publisher is edited. With the default `true` and no assets, the release-create step fails on `fail_on_unmatched_files`. A **source-only** repo has no release task at all. Its standalone `publish-release.yml` inlines `action-gh-release`, so `expect_release_assets` does not apply (see Section 6). ## 4. Behavioral Contract: Expected Outcomes @@ -169,6 +170,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D4.3 Release contents.** Output: every release is a tag on the built commit plus the auto source zip, README, and LICENSE; file-producing targets attach `release-asset-*`; `prerelease` equals `branch != default`. A no-file-target repo that uses the release task (Docker-only, PyPI-only) reaches the tag-only shape **only** with `expect_release_assets: false` set by the caller (which relaxes `fail_on_unmatched_files` and skips the asset download). With the default `true` and no assets the release-create step fails. A source-only repo reaches the same shape through its inlined `action-gh-release` instead, with no release task or `expect_release_assets`. - **D4.4 No-op republish.** Input: a re-run whose version is unchanged. Output: nothing is re-pushed, because the release-create step is skipped when the tag exists (refreshed only on `workflow_dispatch`), and the paired asset-delete is skipped with it. Registry pushes are no-ops. The NuGet/PyPI publish steps are **not** statically gated on existence. They run and the **server** dedupes (`dotnet nuget push --skip-duplicate` turns a 409 into success; PyPI `skip-existing: true`). **Docker always re-pushes** the image (base-image refresh), independently of the release-create skip, within the same run. *Prevents: duplicate releases and wasted pushes.* - **D4.5 A build failure blocks every publish target.** Input: a real publish where one enabled build fails. Output: nothing publishes. `github-release` needs every build, so a failed build skips it (no tag, no release), and the terminal registry pusher (Docker) needs every other build and guards its `if` with `!failure() && !cancelled()`, so a failed build skips docker too (no image push) while a disabled or unchanged target (skipped, not failed) still lets docker build on smoke. *Prevents: a partial publish, e.g. a Docker image pushed while the executable build failed and no release was cut.* A repo pushing two registry targets at once would need a build/publish split behind an all-builds gate, which none does today. +- **D4.6 Deploy verification names the release.** Input: a deploy to a filesystem on a host the project owns that completes without error. Output: a check against the running host asserts **which release is answering**, not merely that it answers. The artifact stamps its own version into the configuration it ships, and the check compares that against the version just installed, **waiting for convergence to a bounded timeout** rather than sampling once, because content goes live the instant a pointer moves while server rules wait on an asynchronous reload. The same check asserts **which environment** answered, since several environments serve a byte-identical artifact and a proxy rule aimed at the wrong one answers healthily under the right hostname. An unreachable host is reported distinctly from an HTTP status. *Prevents: a green deploy over a host still serving the previous release's configuration; a URL contract checked against the wrong environment; a dead config watcher read as a routing fault.* ### D5 - Resource Cleanup @@ -177,6 +179,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D5.3 Best-effort.** Output: cleanup is `continue-on-error`, tolerates a failed listing, and deletes **all** matching ids. *Prevents: a cleanup hiccup reddening a job whose publish succeeded.* - **D5.4 Retention backstop.** Output: **every** `upload-artifact` sets `retention-days: 1`. - **D5.5 Never blanket-delete.** Output: cleanup MUST NOT enumerate and delete the run's whole artifact set. *Prevents: destroying diagnostic/log artifacts and auto-emitted build-records.* +- **D5.6 A durable destination's retention is bounded and owned.** Input: a deploy that installs a release beside the retained ones on a host the project owns. Output: retention is bounded by a **declared count**, and the side owning the prune is **written down**. Where the deploy credential can observe the destination, the deploy asserts the count converged and fails when it does not. Where the credential is deliberately write-only, so it can neither delete nor read back, the prune belongs to the **host** and that ownership is recorded there: widening the credential to reach the destination would trade a real confinement boundary for a check, which is the wrong trade. The release the live pointer resolves to is never a prune candidate, whatever the sort order says. A prune that runs against a local scratch tree, or that is best-effort, or that no side is recorded as owning, satisfies none of this. Unlike D5.1 through D5.4, this destination is durable rather than a run-scoped artifact, so no retention backstop expires it. *Prevents: a destination growing without bound until the disk fills, which surfaces as a site outage rather than as a failed deploy; and the split-ownership version of the same, where each side assumes the other prunes.* ### D6 - Seam / Architecture Conformance @@ -232,6 +235,7 @@ Read the workflow files plus `version.json` and assert the structural fact behin - **NuGet:** the publish step is gated `if: inputs.push` only (not on an existence check) and uses `--skip-duplicate`. `*.nupkg` push also carries the paired `.snupkg` to the symbol server where symbols are enabled. The `release-asset` zip carries the package(s). - **PyPI:** `publish-pypi` declares `environment: { name: pypi }`. `id-token: write` appears only on that job (absent from the build/PR path). `skip-existing: true` is set on the publish action. The build artifact is deleted after publish. The `pypi` environment has a deployment-branch rule. - **Docker:** a Docker-only repo's caller passes `expect_release_assets: false`. The leaf reads the external state file for the tag instead of `SemVer2` (wrapper repos only, since a plain Docker repo correctly tags off `SemVer2` and records this N/A). The readme/date-badge jobs are gated main-only. The docker-readme task validates `repositories` XOR `manifest`+`manifest-jq`. The buildcache follows D9.4. +- **Static site deployed to a host:** the generator is pinned by version **and** by a checksum verified before install, declared once across the workflows that install it. The deploy is a dispatch carrying an environment choice, with concurrency keyed on the **environment** and `cancel-in-progress: false`, and production gated to the default branch while any ref may reach a non-production environment. The reusable callee re-asserts the environment name in a job of its own. The upload targets a per-release directory and carries no delete flag at the environment root; the pointer flip is a separate step. The terminal check asserts the golden-list length floors first, then the environment, then the release id, then the URL contract. Retention is bounded by a declared count and one side is recorded as owning the prune: the deploy asserts it where the credential can observe the destination, and the host owns it where the credential is confined write-only (D5.6). ### 5B. End-to-End Trace Scenarios (No Execution, Deterministic from the YAML) @@ -250,6 +254,8 @@ For each *applicable* scenario, evaluate every job's `if:`/`needs:` against the | S9 | re-run publish, version unchanged | release-create **skipped**, `release-asset-*` delete **skipped**; NuGet/PyPI pushes no-op (server dedupe); **PyPI build-artifact still deleted** (its publish ran); **Docker still re-pushes** the image; no duplicate release | D4.4, D5.2 | | S10 | branch/version classification disagree | validate-release **fails loud**, build/publish skip | D2.2 | | S11 | scheduled upstream-version bump (wrapper) | resolver detects a change -> commits the state file -> opens a `<prefix>-<branch>` PR -> merge-bot auto-merges -> the `main` pin publishes via the gate (a develop pin does not auto-publish, shipping instead via a develop dispatch or promotion) | D8.3, D3.5 | +| S12 | deploy dispatch naming an environment | the ref gate runs **first** (production from the default branch only, any ref to a non-production environment); validation runs; the callee re-asserts the environment name; a release installs under its own id; the pointer flips as a separate step; retention is bounded by whichever of the two D5.6 shapes the repo uses, so a deploy whose credential can observe the destination asserts the count converged and one confined write-only leaves it to the host; the live check asserts the environment and the release id, waiting out the reload, then the URL contract; **no tag and no release are created** | D2.1, D4.6, D5.6, D7.1 | +| S13 | deploy dispatch of a production environment from a non-default ref | **fails fast**, before anything is installed or written | D2.1, D2.3 | ### 5C. Live Probe (Where Warranted) @@ -257,6 +263,7 @@ For each *applicable* scenario, evaluate every job's `if:`/`needs:` against the - Drive a `smoke: true` push-probe of the build task for **both** the default and a non-default branch and assert the version classification (clean vs prerelease) and that the gate passes, **without publishing**. *Caveat: the Docker leg logs in to the registry even on smoke and reads the buildcache, so it needs `DOCKER_HUB_*` secrets and cannot run on a fork PR (same-repo only).* - Per registry: after a real publish, query NuGet.org for the expected version + prerelease classification (and the `.snupkg` on the symbol server), and confirm a re-run added no duplicate. For PyPI inspect the `Compute PyPI version step` log and the built `dist/*` filenames for `.dev0` off `develop` vs a plain version on the default branch. - Inspect the latest real publish's logs for `PublicRelease`/`SemVer2` per leg and confirm the artifact lifecycle (uploaded, consumed, deleted, with none left behind). +- **The deploy ref gate (S13) is verified only by tripping it, and the dispatch that trips it is the maintainer's to run.** Dispatch the production environment from a non-default ref and expect the run to fail at the gate. The evidence is four things, and each of them matters: the gate job's conclusion, its error text naming the expected and the received ref, every downstream job recorded as **skipped** rather than passed, and the deployment count against the production environment unchanged. Capture all four, because a gate that fails open and a gate nobody tripped produce the same empty run history, so "we have never seen it fail" is not evidence about the one control standing between a mis-dispatch and the live site. **The agent prepares the command and reads all four back afterwards. It does not fire it.** An agent harness may refuse to dispatch a production deploy, which is the harness working as intended, and the refusal is neither re-shaped into a raw API call nor talked around (GOVERNANCE.md "Repository Boundaries and Write Safety"). The same split applies to any probe that acts on the deploy host directly, an outbound SSH exercising a forced command among them. ### Assessment @@ -277,6 +284,7 @@ Each type maps the *applicable* S-scenarios onto its targets. The differences ar - **Docker image.** The leaf pushes the default branch multi-arch (amd64+arm64) and any other branch `amd64`-only, with a per-branch registry buildcache (`buildcache-<branch>`; a multi-image repo adds a per-image tag) (`cache-to` only the built branch and only on push, `cache-from` both branches); no `release-asset-*`, so a Docker-only repo's caller passes `expect_release_assets: false`; the readme (`peter-evans/dockerhub-description`, `DOCKER_HUB_ACCESS_TOKEN`) and date-badge jobs run **only** when the default branch publishes; the docker-readme task validates `repositories` XOR `manifest`+`manifest-jq` and a multi-image repo derives its publish matrix from the manifest. Docker **always re-pushes** the image, independently of a skipped release-create (S9). A **wrapper** repo tracks an upstream release: the upstream tracker writes a `name -> version` state file and the merge-bot auto-merges the bump PR (S11), and the leaf MUST read that file for the immutable tag instead of `SemVer2` (the tracker ships without this consumer wiring). Test: S7 default leg pushes `latest` + the version tag and updates readme/badge. Non-default pushes the develop tag (amd64 only). S9 still re-pushes. S11 ships the bumped upstream version next publish. 5C Docker probe needs `DOCKER_HUB_*` secrets and same-repo (not fork) runs. - **Data / asset library.** A single new leaf: validate -> zip -> upload `release-asset-<branch>-library` (`retention-days: 1`, upload gated `!smoke`, mirroring the nugetlibrary leaf's shape). Because no such leaf ships, you **add a target** (D6.4): a new `enable_library` input + `build-library` job + `github-release` `needs:` entry in the release task, and a `library` paths-filter entry + `changes` output + `smoke-build` enable-forward in the PR workflow (without it, D1.1 never smoke-builds the library). Keep `expect_release_assets: true` (it has a file target, unlike Docker). The .NET `unit-test` job is replaced by a type-appropriate validator with the aggregator **and** `smoke-build` both re-pointed to it (D1.2/D1.5). `version.json` + the NBGV `get-version` step are retained (they own the tag). Test: S1 smoke runs validate+zip and uploads nothing; S7 attaches the zip, prerelease on the non-default leg; S9 on a *scheduled* re-run release-create + asset-delete skip (the existing zip is untouched, no registry push), while a `workflow_dispatch` re-run **refreshes** the release and re-runs the asset-delete (the asset is re-uploaded then re-deleted). N/A: the nuget/pypi/docker/executable 5A addenda and their scenario clauses. - **Source-only / no build.** There is no `build-release-task.yml` (its `appliesTo` excludes source-only) and no package/image leaf, so nothing is edited down. The release is a standalone dispatch-only `publish-release.yml` that inlines NBGV for the tag and `action-gh-release` for the release: tag + source zip + README + LICENSE, with no reusable release task and no asset download. With no target the paths-filter matches nothing, so `smoke-build` is **structurally always skipped**, and validation is carried solely by the (replaced, non-.NET) validation job that the aggregator and `smoke-build`'s own `needs:` must both point at (D1.2; or drop the never-running `smoke-build` job). NBGV and `version.json` are still retained (they own the tag). Its publish job gates on the repo's reusable validation task (`needs:` the same `workflow_call` job the PR workflow runs), so a dispatch cannot release a ref that fails validation. Applicable scenarios: S1 (validation only), S5/S6 (publish gating), S7 (tag-only release), S8 (dispatch guard), S9 (no-op republish), S10 (classification gate). N/A: S2-S4 (assume a smoke-built target), the artifact-lifecycle and registry clauses of S7/S9, the D5/D6 artifact items, and all per-type 5A addenda, all recorded N/A, not failed. +- **Static site deployed to a host the project owns.** Two independent surfaces, and keeping them apart is the point. The **release** is the source-only shape above, unchanged: a dispatch-only `publish-release.yml` where NBGV and `version.json` own the tag, producing tag + source zip + README + LICENSE. The **deploy** is its own `workflow_dispatch` carrying an `environment` choice input, so redeploying an unchanged commit mints no tag, which matters because redeploying is routine. It runs a ref gate **first**, before anything is installed or written (production from the default branch only; any ref may reach a non-production environment, since proving a branch before it merges is what that environment is for), then the **same** reusable validation task the PR gate runs, so a dispatch cannot deploy a ref that fails validation, then calls `deploy-site-task.yml` with `secrets: inherit`. Concurrency is keyed on the environment with `cancel-in-progress: false`, because a cancelled deploy leaves a release uploaded and unflipped. The task pins the generator by version and checksum, and re-asserts the environment name in a job of its own, because the `environment:` binding resolves before any step runs and a `workflow_call` caller is not bound by the dispatch choice list a human sees. Its environment-bound job then: checks out full history (a shallow clone silently changes page metadata), installs the verified generator, derives the release id **once** and exports it (deriving it twice yields ids seconds apart, and the live check then asserts a version nothing installed), builds the tree, installs the deploy credential from the environment, uploads into a per-release directory hard-linked against the current release and carrying **no** delete flag (at an environment root a delete removes the rollback targets), flips the pointer as a separate atomic step so a failed transfer cannot half-publish, and finally checks the running host (D4.6). Retention (D5.6) is bounded by a declared count with one side recorded as owning it: a deploy whose credential can observe the destination prunes and asserts the count here, while a credential confined **write-only** can neither delete nor read back, so there the prune is a host-side timer and the repo's runbook records that ownership. Widening the credential to bring the prune in-pipeline would trade a real confinement boundary for a check, and is the wrong trade; what the guarantee rejects is neither side owning it. One thing the pipeline cannot assert and the server config must: a non-public environment serving a byte-identical copy must not be indexed, and that default belongs on the side that is harmless in production, since a non-public container missing the value is still behind its gate while a production container inheriting it deindexes the site silently. Applicable scenarios: S1 (validation), the source-only release set S7/S8/S9/S10, and S12/S13 (the deploy dispatch). N/A: S2-S4, every registry scenario, and D5.1-D5.4 (the pipeline uploads no workflow artifact at all, so D5.6 is what applies in their place), all recorded N/A, not failed. - **Operational (workflow model, not a build target).** A `workflowModel: operational` repo layers a direct-commit `develop` onto the **source-only** release shape (above). Two workflows: (1) a **lint/validation** PR workflow feeding the required `Check pull request workflow status job`, built from the generic linters (editorconfig/EOL, markdownlint, cspell, actionlint) plus a domain validator (Home Assistant `hass --script check_config`, `esphome config`, a firmware build), with **no unit tests**; its triggers differ from the `release` model: `push` to `develop` (advisory feedback on direct commits) plus `pull_request` to `main` (the enforced promotion gate) plus `workflow_dispatch`. (2) the standard **source-only publisher** on `workflow_dispatch` only (`releaseTrigger: dispatch-only`): NBGV + `version.json` own the tag, and a manual dispatch cuts a GitHub release (tag + source zip + README + LICENSE, via the standalone publisher's inlined `action-gh-release`). Applicable scenarios: S1 (validation) on the promotion PR, plus the source-only release set: S7 (tag-only release), S8 (dispatch guard), S9 (no-op republish), S10 (classification). N/A: the auto-publish paths (S5/S6 bot-push and schedule, neither of which an operational repo has) and every build/registry scenario. See the branch-model note in Section 3 and [GOVERNANCE.md "Branching Model"][governance-branching-model]. <!-- Workflow --> diff --git a/catalog/snippets/workflows/README.md b/catalog/snippets/workflows/README.md index c0340390..d42e1e5e 100644 --- a/catalog/snippets/workflows/README.md +++ b/catalog/snippets/workflows/README.md @@ -13,6 +13,8 @@ The reusable build/publish workflow tasks a code-shipping repo runs. They are ** | `build-docker-task.yml` | Multi-arch image build + push, registry layer cache | D4.4, D6, D9.4; section 6 Docker walkthrough | | `build-datebadge-task.yml` | BYOB date/last-build badge on the default branch | D4; section 3 Release Model | | `publish-docker-readme-task.yml` | Push the size-limited Docker Hub overview | D2.4; section 6 Docker walkthrough | +| `deploy-site.yml` | Dispatch entry point for a site deploy: environment choice, per-environment concurrency, ref gate, shared validation | D2.1, D2.3, D7.1 | +| `deploy-site-task.yml` | Build a site and ship it to a filesystem on a host the project owns, then verify against the running host | D4.6, D5.6, D7.2 (section 6 static-site walkthrough) | | `check-upstream-version-task.yml` | Upstream-version tracker for wrapper repos | D3.5, D8.3 | | `run-codegen-pull-request-task.yml` | Deterministic codegen executor (per-branch PR) | D8.2 | | `run-periodic-codegen-pull-request.yml` | Scheduled codegen trigger over both branches | D8.2 | diff --git a/catalog/snippets/workflows/deploy-site-task.yml b/catalog/snippets/workflows/deploy-site-task.yml new file mode 100644 index 00000000..ed42da8f --- /dev/null +++ b/catalog/snippets/workflows/deploy-site-task.yml @@ -0,0 +1,203 @@ +name: Deploy site task + +# Reusable deploy of a built site to a filesystem on a host the project owns. +# It is a separate dispatch from the release, so a redeploy of an unchanged commit mints no tag. +# That matters because redeploying is routine: a host rebuild, a rollback, or proving a branch on a non-production environment. +# +# The transport here is rsync over SSH, but the transport is not the guarantee. +# What the contract fixes is the dispatch gate, the per-environment GitHub Environment credentials, and that success is asserted against what the host serves rather than against the transport's exit status. +# See WORKFLOW.md D4.6 and D5.6. +# +# Two steps call repo-owned scripts instead of inlining their work. +# The bundle assembly builds the site, gates it on the repo's URL contract, precompresses, and stamps the release id into the server configuration it ships. +# The live verification asserts the golden-list floors, the environment, the release id, and then the URL contract. +# Both are long, both are repo-specific, and both have to be runnable by hand outside CI. + +on: + workflow_call: + inputs: + # Selects the GitHub Environment, and is also the path segment the release lands under on the host. + environment: + description: The GitHub Environment to deploy to. + required: true + type: string + outputs: + # The caller records what shipped; without this a rollback has to read the host to find out. + release-id: + value: ${{ jobs.deploy.outputs.release-id }} + site-url: + value: ${{ jobs.deploy.outputs.site-url }} + +# Pinned by version and by checksum, because the site is reproducible only if the generator is. +# A floating installer action defeats the point, and a minor bump changes rendered output. +# Update both values together, and keep them equal to the pin the validation task installs. +# +# The transport's options are pinned rather than left to the runner's OpenSSH defaults, and declared once so two transfers cannot drift apart. +# StrictHostKeyChecking=yes refuses an unknown or changed host key outright, where the default asks and a non-interactive runner then resolves that ambiguously. +# UserKnownHostsFile names the file the deploy key step writes, so the check reads the pinned value rather than whatever the runner image happens to carry. +# BatchMode=yes makes every prompt an immediate failure, so a credential problem surfaces as a failed step rather than as a job that hangs to its timeout. +# IdentitiesOnly=yes stops the agent offering other keys, so the deploy authenticates as the confined account or not at all. +env: + HUGO_VERSION: 0.164.0 + HUGO_SHA256: 8325f3653032d0fc536503691f4833dc4eb6c6be02ee62466758f3f37a7f2fcd + SSH_TRANSPORT: >- + ssh -i ~/.ssh/deploy + -o IdentitiesOnly=yes + -o StrictHostKeyChecking=yes + -o UserKnownHostsFile=~/.ssh/known_hosts + -o BatchMode=yes + +jobs: + + # A job of its own, because the environment binding on the deploy job resolves before any step runs. + # A workflow_call caller is not bound by the dispatch choice list a human sees, so an unknown name would otherwise bind nothing and run anyway. + assert-environment: + name: Assert environment name job + runs-on: ubuntu-latest + permissions: {} + + steps: + + - name: Assert environment is known step + env: + ENVIRONMENT: ${{ inputs.environment }} + run: | + set -Eeuo pipefail + case "$ENVIRONMENT" in + production | staging) ;; + *) + echo "::error::environment must be production or staging, got '$ENVIRONMENT'." + exit 1 + ;; + esac + + # Every host-specific value comes from the environment, so this file names no host, path, or address. + deploy: + name: Deploy site job + runs-on: ubuntu-latest + needs: [assert-environment] + environment: ${{ inputs.environment }} + permissions: + contents: read + outputs: + release-id: ${{ steps.release.outputs.id }} + site-url: ${{ vars.SITE_BASE_URL }} + + steps: + + # Full history, because a shallow clone silently changes page metadata where the generator reads git info. + - name: Checkout code step + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + fetch-depth: 0 + + - name: Install generator step + run: | + set -Eeuo pipefail + deb="hugo_extended_${HUGO_VERSION}_linux-amd64.deb" + curl -sSLf -o "$deb" "https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/${deb}" + echo "${HUGO_SHA256} ${deb}" | sha256sum --check --strict + sudo dpkg --install "$deb" + hugo version + + # Derived once and used three times: the directory name, the stamp in the shipped server config, and the value the live check expects. + # Deriving it twice yields ids seconds apart, and the check then asserts a version nothing installed. + # + # The run id and attempt make it unique by construction rather than by argument. + # A bare second-resolution timestamp is unique here only because a multi-minute install precedes this step and the concurrency group serializes an environment, which are both incidental. + # Reordering this step earlier, an obvious refactor, would silently make two ids collide, and a colliding id is the one value the live check cannot tell apart. + # The timestamp stays the prefix, so the id reads chronologically and a name-sorted prune keeps working. + - name: Resolve release id step + id: release + run: | + set -Eeuo pipefail + echo "id=$(date -u +%Y%m%d-%H%M%S)-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" >> "$GITHUB_OUTPUT" + + # Assembled to a scratch path, since the environment's deploy root is on the far host. + - name: Assemble release bundle step + env: + SITE_BASE_URL: ${{ vars.SITE_BASE_URL }} + run: | + set -Eeuo pipefail + bash deploy/make-release.sh "${RUNNER_TEMP}/bundle" "${{ steps.release.outputs.id }}" + + # The known-hosts value is a variable rather than a secret: it is integrity-critical but not confidential, and a variable keeps it diff-visible. + # StrictHostKeyChecking is never disabled. + - name: Install deploy key step + env: + DEPLOY_SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_SSH_PRIVATE_KEY }} + DEPLOY_SSH_KNOWN_HOSTS: ${{ vars.DEPLOY_SSH_KNOWN_HOSTS }} + run: | + set -Eeuo pipefail + mkdir -p ~/.ssh + chmod 700 ~/.ssh + printf '%s\n' "$DEPLOY_SSH_PRIVATE_KEY" > ~/.ssh/deploy + chmod 600 ~/.ssh/deploy + printf '%s\n' "$DEPLOY_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts + chmod 600 ~/.ssh/known_hosts + + # The destination is anchored at the key's confinement root, so no host path appears here. + # The hard-link source points at current, which still resolves to the previous release until the flip, so CI carries no state. + # A missing target is a warning rather than an error, so a first deploy into a fresh environment needs no special case. + # The mkpath flag creates the releases parent, which the transport will not create on its own. + # No delete flag: at an environment root it silently removes the releases a rollback needs. + - name: Upload release step + env: + DEPLOY_SSH_USER: ${{ vars.DEPLOY_SSH_USER }} + DEPLOY_SSH_HOST: ${{ vars.DEPLOY_SSH_HOST }} + RELEASE_ID: ${{ steps.release.outputs.id }} + ENVIRONMENT: ${{ inputs.environment }} + run: | + set -Eeuo pipefail + rsync -az --mkpath --no-g --chmod=D2755,F644 \ + --link-dest="/${ENVIRONMENT}/current/" \ + -e "$SSH_TRANSPORT" \ + "${RUNNER_TEMP}/bundle/releases/${RELEASE_ID}/" \ + "${DEPLOY_SSH_USER}@${DEPLOY_SSH_HOST}:/${ENVIRONMENT}/releases/${RELEASE_ID}/" + + # A separate step from the upload, so a failed transfer cannot half-publish a site. + # The pointer is relative, so one bundle works at any remote root. + # The transport replaces a symlink through a temporary and a rename, so it is never absent to a request in flight. + - name: Flip current step + env: + DEPLOY_SSH_USER: ${{ vars.DEPLOY_SSH_USER }} + DEPLOY_SSH_HOST: ${{ vars.DEPLOY_SSH_HOST }} + ENVIRONMENT: ${{ inputs.environment }} + run: | + set -Eeuo pipefail + rsync -a --no-recursive \ + -e "$SSH_TRANSPORT" \ + "${RUNNER_TEMP}/bundle/current" \ + "${DEPLOY_SSH_USER}@${DEPLOY_SSH_HOST}:/${ENVIRONMENT}/" + + # Retention is bounded and owned, and this step is only one of the two shapes D5.6 accepts. + # It applies where the deploy credential can observe the destination, so the count can be asserted after pruning and the deploy fails when it does not converge. + # A prune whose pattern stops matching reports success while the disk fills, which is the same shape as a gate that stops gating. + # The release current resolves to is never a candidate, whatever the sort order says. + # This destination is durable, so no artifact retention backstop expires it. + # + # Delete this step where the credential is confined write-only, as a forced rsync command with no send capability is. + # Such a key can neither delete nor read back, and widening it to satisfy a check would trade a real confinement boundary for the check. + # There the prune belongs to a host-side timer, and D5.6 is met by recording that ownership on the host side of the repo's runbook rather than by a step here. + # What D5.6 rejects is neither side owning it, since each then assumes the other prunes. + - name: Prune old releases step + env: + DEPLOY_SSH_USER: ${{ vars.DEPLOY_SSH_USER }} + DEPLOY_SSH_HOST: ${{ vars.DEPLOY_SSH_HOST }} + ENVIRONMENT: ${{ inputs.environment }} + KEEP_RELEASES: '10' + run: | + set -Eeuo pipefail + bash deploy/prune-releases.sh "${DEPLOY_SSH_USER}@${DEPLOY_SSH_HOST}" "${ENVIRONMENT}" "${KEEP_RELEASES}" + + # The only step that observes the running site (WORKFLOW.md D4.6). + # An upload succeeds against a container serving nothing, and a flip succeeds against a server that never reloads its rules. + # The check asserts which environment answered, then which release its rules came from, then the URL contract. + # The release value is a response header the artifact stamps into its own served configuration, so it settles without parsing a payload and survives an auth gate in front of the site. + - name: Verify URL contract step + env: + EXPECT_SITE_ENV: ${{ inputs.environment }} + EXPECT_RELEASE: ${{ steps.release.outputs.id }} + run: | + set -Eeuo pipefail + bash checks/check-live-urls.sh "${{ vars.SITE_BASE_URL }}" diff --git a/catalog/snippets/workflows/deploy-site.yml b/catalog/snippets/workflows/deploy-site.yml new file mode 100644 index 00000000..010c0e56 --- /dev/null +++ b/catalog/snippets/workflows/deploy-site.yml @@ -0,0 +1,63 @@ +name: Deploy site action + +# Dispatch entry point for a site deploy, and deliberately not the release publisher. +# A deploy ships an already-validated tree to one environment; the GitHub release is a separate dispatch that cuts the tag. +# Splitting them means a redeploy of an unchanged commit mints no tag, which is what makes a rollback or a host rebuild cheap. + +on: + workflow_dispatch: + inputs: + # A choice rather than free text, so a typo cannot bind an environment that does not exist. + # The callee re-asserts the name anyway, since a workflow_call caller is not bound by this list. + environment: + description: Environment to deploy to. + required: true + type: choice + options: + - staging + - production + +# Keyed on the environment rather than the ref, so two environments deploy independently. +# Runs queue rather than cancel, because a cancelled deploy leaves a release uploaded and unflipped. +concurrency: + group: ${{ github.workflow }}-${{ inputs.environment }} + cancel-in-progress: false + +jobs: + + # First, so a mis-dispatched production deploy fails before anything is installed or written. + # Production is constrained to the default branch; a non-production environment deploys from any ref, since proving a branch before it merges is what that environment is for. + assert-ref: + name: Assert deploy ref job + runs-on: ubuntu-latest + permissions: {} + + steps: + + # Compared against the full ref rather than ref_name, because tags and branches are separate namespaces that share a short name. + # A tag named for the default branch would satisfy a ref_name comparison while being an arbitrary commit, which is a bypass of the one gate protecting production. + - name: Assert production deploys from the default branch step + run: | + set -Eeuo pipefail + if [ "${{ inputs.environment }}" = "production" ] && [ "${{ github.ref }}" != "refs/heads/${{ github.event.repository.default_branch }}" ]; then + echo "::error::Deploy production from the default branch, got ${{ github.ref }}." + exit 1 + fi + + # The same reusable validation task the pull request gate runs, so a dispatch cannot deploy a ref that fails validation. + validate: + name: Validate job + needs: [assert-ref] + permissions: + contents: read + uses: ./.github/workflows/validate-task.yml + + deploy: + name: Deploy job + needs: [validate] + permissions: + contents: read + uses: ./.github/workflows/deploy-site-task.yml + with: + environment: ${{ inputs.environment }} + secrets: inherit diff --git a/registry/repos.json b/registry/repos.json index 62bf599c..10b90d77 100644 --- a/registry/repos.json +++ b/registry/repos.json @@ -309,17 +309,17 @@ "name": "Blog", "url": "https://github.com/ptr727/Blog", "status": "cataloged", - "types": ["source-only"], + "types": ["hugo", "source-only"], "groundTruthBranch": "main", "workflowModel": "release", "lineEndings": "lf", "hasDevelop": true, - "publish": [{ "target": "github-release", "mechanism": "none" }], + "publish": [{ "target": "github-release", "mechanism": "none" }, { "target": "self-hosted", "mechanism": "static-secret" }], "requiredSecrets": [], "consumerModel": "pull", "releaseTrigger": "dispatch-only", "configLayout": { "rulesetsDir": "repo-config", "pythonConfig": null }, - "driftNotes": ["Hugo static site migrated off WordPress.com, stood up 2026-08-01; release model with a dispatch-only publisher that cuts the tag and a source archive.", "lineEndings lf on a release repo, where the rule grants the native-platform default to operational repos only: every consumer is Linux (the Hugo build, the Caddy container, and the rsync deploy).", "types source-only rather than the docs the hub proposed, because both docs predicates are false for a repo that builds a site and gates a URL contract. Interim until the static-site-generator type lands, which #456 is holding for Blog's measured deploy shape.", "content/ is an imported WordPress archive, so the prose, spelling, and style gates are scoped to exclude it.", "The VPS deploy does not exist yet, so the publish target is the GitHub release only. Revisit publish[] once deploy-site.yml is proven."] + "driftNotes": ["Hugo static site migrated off WordPress.com, stood up 2026-08-01; release model with a dispatch-only publisher that cuts the tag and a source archive.", "lineEndings lf on a release repo, where the rule grants the native-platform default to operational repos only: every consumer is Linux (the Hugo build, the Caddy container, and the rsync deploy).", "content/ is an imported WordPress archive, so the prose, spelling, and style gates are scoped to exclude it.", "Deploy credentials are per-environment GitHub Environment secrets and variables, which spec/secrets.json has no vocabulary for, so requiredSecrets stays empty and the names are declared in this repo's own secrets.json environments block; listing them here would make the audit demand them in the repository actions store."] } ] } diff --git a/registry/repos.schema.json b/registry/repos.schema.json index bd14c2ff..fac95ffd 100644 --- a/registry/repos.schema.json +++ b/registry/repos.schema.json @@ -27,7 +27,7 @@ "workflowModel": { "enum": ["release", "operational"] }, "lineEndings": { "enum": ["lf", "crlf"] }, "mechanism": { "enum": ["oidc", "static-secret", "none"] }, - "target": { "enum": ["nuget", "pypi", "docker", "github-release"] }, + "target": { "enum": ["nuget", "pypi", "docker", "github-release", "self-hosted"] }, "repo": { "type": "object", "required": ["name", "url", "status"], diff --git a/reports/blog/audit.md b/reports/blog/audit.md index 8a0b41ce..b6775807 100644 --- a/reports/blog/audit.md +++ b/reports/blog/audit.md @@ -1,37 +1,57 @@ # Audit: Blog -- **Audited branch:** main (`6855ddb`) -- **Types:** source-only (from registry) +- **Audited branch:** main (`2b132e4`) +- **Types:** `hugo`, `source-only` (from registry) - **Verdict:** operational -- **Date:** 2026-08-03 -- **Run stamp:** `audit run 2026-08-03T22:34:03Z | hub 3a7cc64` +- **Date:** 2026-08-05 +- **Run stamp:** `audit run 2026-08-05T21:57:38Z | hub 01507a0` -First audit of this repo. It was stood up on 2026-08-01 and reached a proven release path the same day, but it was never added to [`registry/repos.json`][repos], so no hub tool had measured it until now and [`reports/divergences.md`][divergences] under-reported the fleet by exactly this repo. The registry entry lands with this report. Nothing here is a defect: every finding is the hub advancing after the carry, which is the propagation job [#536][pr-536] exists to make possible. +Second full run, replacing the 2026-08-03 report in whole rather than editing it, per the run-stamp discipline in [`AUDIT.md`][audit] section 8. The previous run graded the repo as `source-only` against a hub that had no static-site type, and it recorded that it was due a re-run once the deploy existed. That is what this is. The `hugo` type and the `self-hosted` composable target have since landed, the deploy has run against both environments, and every `hugo.*` check is judged here for the first time. -## Develop Drift - -`develop` vs `main`: ahead 1, behind 1. The behind-1 is the promotion merge commit, which is the benign ancestry artifact merge-commit promotions always leave. The ahead-1 is a Dependabot action-SHA bump the merge bot landed after promotion. A second run with `--branch develop` reports the identical nine findings, so `develop` carries no conformance content `main` lacks. +The three deviations the first run recorded against the repo are closed. [ptr727/Blog#27][blog-27] (the remote release tree was never pruned), [ptr727/Blog#28][blog-28] (the vendored theme recorded no upstream ref), and [ptr727/Blog#29][blog-29] (the generator pin was duplicated across two workflows) each have a fix on `main` and are cited as evidence below. No defect is open. Everything remaining is the hub having advanced past what this repo carries, plus one finding that is the hub's own to fix rather than the repo's. -## Already Owed by the Hub's Own Develop - -This audit reads hub `main` (`3a7cc64`), which AUDIT.md section 1 makes the ground truth. Re-running it from a tree at hub `develop` (`362aec8`) reports two additional re-vendors and two DEFECTs, all four from [#545][pr-545] taking `bypass_actors` out of the three ruleset payloads about an hour before this run. They are recorded here rather than counted, because measuring a repo against un-promoted hub content reports work in flight as a conformance failure. What the re-run observed was: +## Develop Drift -- `repo-config/develop.json` and `repo-config/main.json` become stale carries, re-vendored the same mechanical way as the rest. -- The live `develop` and `main` rulesets then diverge from the payloads, because both still carry the `RepositoryRole` admin bypass the new payloads no longer declare. Closing that is a repository-settings change on a protected branch, so it is the maintainer's to apply, not an agent's. +`develop` vs `main`: ahead 0, behind 6. `git diff origin/develop origin/main` is empty, so the two branches carry identical trees and the six commits are the promotion's PR commits plus its merge commit. That is the benign ancestry artifact a merge-commit promotion always leaves. No drift finding: `main` carries no content `develop` lacks. ## Dimensions | Dimension | Letter | Intent | Verdict | Evidence (file:line) | | --- | --- | --- | --- | --- | -| branch-model | pass | pass | pass | Both branches protected, exactly one ruleset per name, and both match `repo-config/develop.json` and `repo-config/main.json` by normalized diff. The required-status-check context is the fleet canonical `Check pull request workflow status job` (`repo-config/main.json:48`, `repo-config/develop.json:51`) and matches the aggregator's own name (`.github/workflows/test-pull-request.yml:26`) | -| repo-setup | pass | pass | pass | The baseline App-token pair is present in both the Actions and Dependabot stores, no forbidden name is configured, and no stale secret. `requiredSecrets` is empty, correct for a source-only repo publishing only a GitHub release | -| linter-parity | pass | pass | pass | One config per linter, each driving CI: markdownlint-cli2, cspell, actionlint, editorconfig-checker, shellcheck and shfmt, then the Hugo build and the URL-contract check (`.github/workflows/validate-task.yml:34-101`) | -| recurring-violations | drift | pass | drift | Three prose findings across the repo's own files, none in carried content. Line endings are LF and bound by git rather than only by the editor (`.gitattributes:7,16`) | -| readme-structure | pass | pass | pass | Sections present and in spec order, with Installation and Usage legitimately N/A for source-only. The intro is 77 characters, link-free, single-sentence (`README.md:3`), and the GitHub About description mirrors it exactly | -| workflow (WORKFLOW.md 5A/5B) | pass | pass | pass | The PR gate runs the reusable validation and gates the merge on one always-run aggregator (`test-pull-request.yml:25-26`). The publisher is dispatch-only, refuses a ref that is not `main` or `develop` (`publish-release.yml:34`), pins the dispatch-time commit so a later push cannot release unvalidated, versions with NBGV (`publish-release.yml:52-53`), and derives prerelease from the ref (`publish-release.yml:64`). Proven end to end: release `1.0.11`, 2026-08-01 | -| agent-instruction-set | drift | drift | drift | Eight verbatim units behind the canonical, two of them absent rather than stale, plus one undeclared section. Enumerated below | - -csharp, nuget, pypi, python, console, docker: N/A. The repo builds a Hugo site and ships no package or image. It carries Python helper scripts under `checks/`, which is not enough to make it a `python` repo: there is no package, no `pyproject.toml`, and no ruff or pyright surface for the dimension to check. +| hugo | pass | pass | pass | All nine checks pass, enumerated below | +| branch-model | pass | pass | pass | Both branches protected, exactly one ruleset per name, and both match `repo-config/develop.json` and `repo-config/main.json` by normalized diff. The required-status-check context is the fleet canonical `Check pull request workflow status job` (`repo-config/main.json:41`, `repo-config/develop.json:44`) and matches the aggregator's own name (`.github/workflows/test-pull-request.yml:26`). General settings diff clean against `repo-config/settings.json`, and the two state-dependent settings hold: the repo is public with discussions on, and `default_branch` is `main` | +| repo-setup | pass | pass | pass | No forbidden secret, no stale secret. `requiredSecrets` is empty and correct: the GitHub release needs no credential, and the deploy's credentials are per-environment. See the caveat below, because a clean verdict here says less than it looks like it does | +| linter-parity | pass | pass | pass | One config per linter, each driving CI: markdownlint-cli2, cspell, actionlint, editorconfig-checker, then shellcheck and shfmt, config validation, the site build, and the URL contract (`.github/workflows/validate-task.yml:28-88`) | +| recurring-violations | drift | pass | drift | 17 prose findings across 7 of the repo's own files, none in carried or vendored content. Line endings are LF and bound by git rather than only by the editor (`.gitattributes`). Enumerated below | +| readme-structure | pass | pass | pass | Sections present and in spec order, with Getting Started legitimately omitted and Installation and Usage N/A for a repo that ships no installable artifact. Build and Distribution carries all three subsections (`README.md:12,17,22`). The intro is 76 characters, link-free, and a single sentence (`README.md:3`), the GitHub About description mirrors it exactly, and `HISTORY.md:1-3` matches the README title and intro | +| workflow (WORKFLOW.md 5A/5B) | pass | pass | pass | Enumerated below | +| agent-instruction-set | drift | pass | drift | Seven verbatim units behind the canonical, all of them stale copies rather than absences, so each states its rule in an older form. Enumerated below | + +`csharp`, `nuget`, `pypi`, `python`, `console`, `docker`: N/A. The repo builds a static site and ships no package or image. It carries Python helper scripts under `checks/`, which is not enough to make it a `python` repo, since there is no package, no `pyproject.toml`, and no ruff or pyright surface for the dimension to check. + +**What a clean `repo-setup` does not cover.** The deploy's credentials are GitHub Environment secrets and variables rather than repository secrets, and [`spec/secrets.json`][secrets] has no vocabulary for an environment scope. The registry therefore declares an empty `requiredSecrets`, which is correct rather than a gap, because listing the names there would make the audit demand them in the repository Actions store where they deliberately are not. The consequence is that this dimension passing is no evidence at all that either environment is configured. What proves that is a deploy run reaching its verification step, which has happened for both environments and is outside what the audit can see. + +## Hugo Dimension, Check by Check + +| Check | Verdict | Evidence | +| --- | --- | --- | +| `hugo.build.strict` | pass | `hugo --gc --minify --panicOnWarning` at `.github/workflows/validate-task.yml:83` and the identical command at `deploy/make-release.sh:94`, so the gate and the deploy build the same way rather than in two variants | +| `hugo.urls.parity` | pass | `checks/check-url-parity.py:16-30` asserts a floor on each of the three lists before comparing, and `checks/check-live-urls.sh:18-25` does the same against the running site. The floors sit under the current counts (328 render, 917 redirect, 778 legacy media), so a list may grow but cannot collapse into a vacuous pass | +| `hugo.output.uncommitted` | pass | `public/` and `resources/_gen/` are gitignored (`.gitignore:5-6`) and neither is tracked. The Markdown glob excludes the imported archive, the vendored theme, and the render (`.github/workflows/validate-task.yml:31-35`), and cspell is scoped to `README.md` and `HISTORY.md` | +| `hugo.generator.pinned` | pass | Version and SHA-256 are pinned together at `.github/actions/install-hugo/action.yml:26-27`, verified by `sha256sum --check --strict` before install (line 33), and the extended build is asserted from the binary rather than inferred from the file name (lines 36-38). The pin is declared once, in the composite action both callers use (`validate-task.yml:76`, `deploy-site-task.yml:79`), and is not exposed as an overridable input. This closes [ptr727/Blog#29][blog-29], which is the stronger fix: the check asks that something assert two copies agree, and removing the second copy makes agreement structural | +| `hugo.vendored.provenance` | pass | `themes/README.md:10-16` records the upstream repository, the exact commit, its upstream date, its `git describe` form, and the license, and lines 23-26 record the two local edits against it. The record sits outside the vendored directory deliberately, so replacing that directory on an update does not take the record with it. This closes [ptr727/Blog#28][blog-28] | +| `hugo.deploy.environment` | pass | The deploy job binds `environment: ${{ inputs.environment }}` (`deploy-site-task.yml:62`) and takes the host, user, base URL, and known-hosts entry from that environment's variables and secrets, so the workflow file names no host, path, or address. A separate `assert-environment` job re-asserts the name (lines 38-55), which is what the check asks for, because the environment binding resolves before any step runs and a `workflow_call` caller is not bound by the dispatch choice list a human sees | +| `hugo.deploy.atomic` | pass | The release installs under its own immutable id beside the retained ones, and `--delete` is omitted on the upload precisely because at an environment root it would remove rollback targets (`deploy-site-task.yml:123,132-136`). `--link-dest` points at `current`, which still resolves to the previous release at that moment. The pointer moves as a separate step (lines 140-150), and locally the same flip is a temporary link renamed over the old one (`deploy/make-release.sh:206-210`), which is a single rename rather than a replace in place | +| `hugo.deploy.verified` | pass | The terminal step observes the running host rather than the transport's exit status (`deploy-site-task.yml:155-163`). `checks/check-live-urls.sh` asserts which environment answered from a response header (lines 130-139) and that the rules answering are the release just installed (lines 154-182), polling to a bounded timeout because the config reload is asynchronous. Unreachability is reported distinctly from an HTTP status in three separate places (lines 101-105, 145-152, 163-167), and the preflight separates a bad credential from a vanished site (lines 109-126) | +| `hugo.deploy.retention` | pass | `OPERATIONS.md:170` declares the count and names the host timer that owns the prune, and records that the release `current` resolves to is retained unconditionally without consuming one of the ten. `OPERATIONS.md:172` records why nothing prunes on the deploy path, which is that the deploy key then needs no delete capability. This is the second of the two shapes D5.6 allows, and it is the correct one here: the credential is confined write-only and cannot observe the destination, so an in-pipeline assertion would mean widening it. Note that `deploy/make-release.sh:212-230` does prune, but in CI it runs against `${RUNNER_TEMP}/bundle`, so it is a local scratch prune and satisfies nothing on its own. The host timer is what the verdict rests on. This closes [ptr727/Blog#27][blog-27] | + +## Workflow Assertions (5A and 5B) + +- **D1, PR fast feedback.** The pull request gate runs the reusable validation and gates the merge on one always-run aggregator whose name is the ruleset-bound context (`test-pull-request.yml:15-19,25-29`). +- **D4, release and publish.** The publisher is dispatch-only, refuses a ref that is not `main` or `develop` (`publish-release.yml:34-40`), pins the dispatch-time commit so a push landing after dispatch cannot release unvalidated (line 46), versions with NBGV (line 53), and derives prerelease from the ref (line 64). It runs the same validation gate first (lines 15-19). Proven end to end: release `1.0.11` on 2026-08-01 from `main`, and prerelease `1.0.17-g4b2def3ee9` on 2026-08-04 from `develop`. +- **D4, deploy.** The deploy is a separate `workflow_dispatch` from the release, so a redeploy of an unchanged commit mints no tag (`deploy-site.yml:3-12`). Production is refused from any ref other than `main`, compared against the full ref rather than `ref_name`, because a tag and a branch sharing a short name are separate namespaces and the `ref_name` form would accept a tag named `main` pointing anywhere (`deploy-site.yml:25-37`). The gate runs first, before anything is installed or written. +- **D7, concurrency and permissions.** Both dispatch workflows queue rather than cancel, each for a stated reason: a cancelled deploy leaves a release uploaded and unflipped, and a cancelled publish leaves a half-created GitHub release (`deploy-site.yml:14-17`, `publish-release.yml:6-10`). Both assertion jobs declare `permissions: {}`, since neither reads the repository. +- **Dependabot.** `github-actions` is declared and dual-targets `main` and `develop`, which is the whole implied set: the site has no package manifest, and the theme is vendored rather than pulled by a manager. ## Defects @@ -39,43 +59,97 @@ None. ## Drift Findings -**Carried content behind the canonical.** Six of these are stale copies and two never arrived, and the distinction matters because a stale copy still states the rule in an older form while an absent one states nothing at all: +### Carried Content Behind the Canonical -1. `AGENTS.md > Fleet Bootstrap` is **absent**. Added by [#536][pr-536] after this repo carried its baseline, so the repo holds no statement of where the canonical rules live or how to route by its own state. -2. `GOVERNANCE.md > Representative Data in Agent-Authored Text` is **absent**. The rule against agent-authored text quoting the maintainer's own data is not present in this repo in any form. -3. `AGENTS.md > Context and Delegation Discipline` is stale. Missing the rule that a wait separates three outcomes and says which one it reached. -4. `AGENTS.md > Where the Rules Live` is stale. Missing the table row routing to `Representative Data in Agent-Authored Text`, consistent with finding 2. -5. `GOVERNANCE.md > Git and Commit Rules` is stale. Missing "Commit means commit and push". -6. `GOVERNANCE.md > Verification Discipline` is stale. Missing "A launched process is not a result, and a cause nobody observed is not a diagnosis". -7. `GOVERNANCE.md > PR Review Etiquette` is stale. Missing the whole `### Every Finding Ends in an Action` subsection, so the repo carries the review loop without the five outcomes that close a finding. -8. `repo-config/configure.sh` is stale. It predates the payload-driven check mode ([#540][pr-540], [#543][pr-543]), so its check mode compares the pull-request merge methods and the required-status-check contexts by name rather than comparing every parameterized rule's whole parameters object in both directions. It passes clean over drift the current canonical would catch. +Seven verbatim units are stale, and none is absent. Every one is the hub advancing after this repo's last re-vendor, which is the propagation this model expects rather than anything the repo did. They fall into two groups. -**Undeclared section.** `AGENTS.md > Project Conventions` (`AGENTS.md:9`) is not a section [`spec/files.json`][files] declares. Its four rules are genuinely repo-specific (the append-only URL contract, never populating media over HTTP, `content/` as an archive, and a gate proving itself by failing) and none duplicates a verbatim section. Left in place pending the spec question raised below, which is what the section model asks for when reconciliation is not obvious. +**The Markdown capitalization settlement ([#566][pr-566]).** Three units differ from the canonical only in the case of the word Markdown in prose: `GOVERNANCE.md > Documentation Style Conventions`, `GOVERNANCE.md > Repository Details`, and `.markdownlint-cli2.jsonc` (in two comments). No rule changed. -**Prose.** Three findings from `scripts/prose_lint.py` run over the repo excluding `content/`, `public/`, `themes/`, and `resources/`: a comment opening in lowercase (`.github/workflows/merge-bot-pull-request.yml:20`), a semicolon in prose (`OPERATIONS.md:142`), and a COPYRIGHT SIGN that the character-set tiers do not classify (`layouts/rss.xml:51`). +**Four rule additions this repo has not yet carried.** These are substantive, and the repo currently states each rule in a form that is missing the new part: -## Proposed Registry / Spec Updates +1. `GOVERNANCE.md > Repository Boundaries and Write Safety` is missing the rule that a refused write is reported and never re-shaped, and that the maintainer's say-so does not lift a refusal by the harness ([#569][pr-569]). +2. `GOVERNANCE.md > Communicating with the User` is missing both the form-follows-surface qualifier on clickable links and the whole rule that work blocked on the user is raised as a direct interactive prompt whose options are the actions themselves ([#561][pr-561]). +3. `GOVERNANCE.md > Release Model` is missing the filesystem-deploy leaf bullet, which is the one that describes this repo's own deploy shape, including the retention rule the `hugo.deploy.retention` verdict above is judged against ([#558][issue-558], [#560][pr-560]). +4. `GOVERNANCE.md > Branching Model` is missing the issue-closing-keyword rule, which moved here out of Release Model, so this repo carries it in the old location and will lose it on the next Release Model re-vendor if the two are done separately ([#563][issue-563]). + +Item 4 is the one to sequence carefully. A re-vendor that takes the new Release Model without also taking the new Branching Model drops the rule entirely rather than leaving it stale. + +### Prose -- Add the `Blog` registry entry. Applied in this change: `source-only`, `release` workflow model, `dispatch-only` release trigger, `lineEndings: "lf"`, and `driftNotes` recording the two declared deviations plus the interim classification. -- Revisit `publish[]` once the VPS deploy exists. It declares the GitHub release only, because that is the only channel that currently ships. [#456][issue-456] holds the static-site type pending this repo's measured deploy shape. +17 findings from `scripts/prose_lint.py` over the repo with `content/`, `public/`, `themes/`, and `resources/` excluded. All 17 are in files this repo authors. + +| Check | Count | Locations | +| --- | --- | --- | +| semicolon | 6 | `OPERATIONS.md:80,176,186`, `TODO.md:31,104`, `deploy/README.md:110` | +| spelling | 5 | `OPERATIONS.md:231`, `deploy/README.md:197,219,231`, `deploy/make-release.sh:51` | +| comment-wrap | 4 | `.github/workflows/deploy-site-task.yml:35`, `checks/check-live-urls.sh:77,99,149` | +| comment-case | 1 | `.github/workflows/deploy-site-task.yml:139` | +| charset-unknown | 1 | `layouts/rss.xml:51` | + +Each was read rather than counted. The six semicolons all join independent clauses, which is the form the rule bans. The five spellings are British forms with US equivalents. The four wraps are comment sentences continuing onto a second line. The one case finding opens a comment sentence on a lowercase command name. The `charset-unknown` finding is not a violation and is carried to Escalations below, unchanged from the previous run. + +## Non-Findings + +**`GOVERNANCE.md > Repository Onboarding and Conformance` is absent, and correctly so.** [`spec/section-model.md`][section-model] line 51 declares it hub-only and not carried, so its absence here is conformance rather than drift. Recorded because a section-count comparison against the hub surfaces it and it reads like a gap. + +**The `Project Conventions` section is gone.** The previous run raised it as an undeclared `AGENTS.md` section and as escalation 1. The repo has since moved that content into the topical docs that own it, and its `AGENTS.md` now carries exactly the three declared sections. The escalation is resolved and is not repeated below. ## Escalations -Two spec questions, raised rather than resolved, per AUDIT.md section 9. +Three, raised rather than resolved, per [`AUDIT.md`][audit] section 9. The first is new and is a hub defect rather than a repo one. + +### 1. The Template-Reference Check Contradicts the Byte-Locked Fleet Bootstrap Section + +[`spec/audit.py`][audit-runner] lines 691-695 flag any carried `AGENTS.md`, `GOVERNANCE.md`, or `.github/copilot-instructions.md` that contains the hub's name anywhere in the file, on the reasoning that the coordination flow is machinery a consumer should not see. That reasoning is sound and the check catches real cases. + +It has no exemption for `AGENTS.md > Fleet Bootstrap`, which the hub declares `verbatim` in [`spec/files.json`][files] and whose first sentence names the hub repository by path. Naming it is the section's entire function: it is the byte-locked entry point that tells an agent where the canonical rules live when nothing else present says so. Blog's line 11 is byte-identical to the hub's own `AGENTS.md:11`, and it is the file's only occurrence of the name. + +So the finding on this repo is unclearable by construction. The only way to satisfy the check is to delete or alter a section the hub byte-locks, which the verbatim check would then flag instead. This is not specific to Blog. It fires on any repo that has carried the current canonical, and it will therefore spread across the fleet as the carry propagates rather than staying a single-repo curiosity. The fix belongs in the check: scan the file with the `Fleet Bootstrap` block excised, so a reference outside that section is still caught. The finding above is recorded as a hub defect and is not counted against this repo's verdict. + +### 2. A Downstream Repo Holding a Report at the Hub's Own Report Path + +Blog carries `reports/Blog/audit.md`, a self-audit dated 2026-08-05 against hub `3b802b9`. [`AUDIT.md`][audit] section 8 states that the hub authors the report and that a report written by the repo being audited is a claim rather than evidence, and it forbids the route it anticipated, which is a downstream repo opening a pull request against the hub. It says nothing about a downstream repo keeping such a report in its own tree, and `spec/files.json` neither declares nor forbids the path. + +Two things follow. The paths differ from the hub's only in the case of one directory, `reports/Blog/audit.md` there against `reports/blog/audit.md` here, so a reader with both trees open has two documents at what reads as the same location, reaching verdicts on the same repository, and only one of them is evidence under the model. The case difference makes that worse rather than better, because it is invisible on a case-insensitive filesystem and is the kind of distinction a reader treats as a typo rather than as a boundary. The downstream copy is also stale by exactly the mechanism section 8 describes: it declares `types: ["source-only"]`, which the registry superseded before the file was written, so its own header disagrees with the registry it cites. + +The question for the spec is whether a repo self-auditing in its own tree is sanctioned, and if it is, under what name. The content is genuinely useful, because the repo checks things the hub cannot see, its environment secrets among them. The collision is with the path and with the word audit, not with the practice. Raised rather than resolved, since the answer changes `spec/files.json` for every repo rather than just this one. + +### 3. The Character-Set Tiers Do Not Classify the COPYRIGHT SIGN + +Unchanged from the previous run and still open. `layouts/rss.xml:51` emits the sign as U+00A9 in generated feed output, where the alternative is an ASCII transliteration in a machine-read document. The tiers in `GOVERNANCE.md` "Character Set" have no entry for it, so the linter reports `charset-unknown` rather than pass or fail. Every repo generating a feed or a rendered document hits the same gap. + +## Proposed Registry / Spec Updates + +- **Registry: no change.** The entry already declares `types: ["hugo", "source-only"]`, both publish targets, `dispatch-only`, and `lineEndings: "lf"`. Its four `driftNotes` all describe current, live deviations, none asserts outstanding work, and none names a check id, so nothing is retired by this run. +- **Conformance matrix: the `hugo` row is updated with this run's date and findings** in the same change as this report. +- **`spec/audit.py`: exempt the `Fleet Bootstrap` block from the template-reference scan**, per escalation 1. This is a hub defect with a fleet-wide blast radius and is the one item here that should not wait. +- **`spec/files.json` or `AUDIT.md` section 8: decide the downstream self-report question**, per escalation 2. +- **`GOVERNANCE.md` "Character Set": classify U+00A9**, per escalation 3. + +## Convergence + +Per [`AUDIT.md`][audit] section 10, the drift above is applied to the target by pull request, one focused PR per drift class, and the maintainer merges. Two classes are open here: + +1. **Re-vendor the seven stale units**, taking Branching Model and Release Model together for the reason in item 4 above. +2. **Fix the 17 prose findings**, which are all in this repo's own authored files. -1. **The hub contradicts itself on whether a repo may carry its own `AGENTS.md` section.** [`AGENTS.md`][agents] states that "a project's project-specific conventions and public-API/behavioral contracts (e.g. a 'Library API Conventions' section) live in that project's own `AGENTS.md`", while [`spec/section-model.md`][section-model] states that "a downstream repo's extra section the hub does not declare is drift to reconcile, not a local liberty" and lists four destinations for repo-specific content, none of them `AGENTS.md`. A repo following the first is flagged by the audit for violating the second. Blog is the case: its `Project Conventions` section is exactly the shape the first sanctions. -2. **The character-set tiers do not classify the COPYRIGHT SIGN.** `layouts/rss.xml:51` emits the sign as U+00A9 in generated RSS output, where the alternative is an ASCII transliteration in a machine-read feed. The tiers in GOVERNANCE.md "Character Set" have no entry for it, so the linter reports `charset-unknown` rather than pass or fail, and every repo generating a feed or a rendered document will hit the same gap. +Neither is started. The repo also carries one open item of its own, [ptr727/Blog#33][blog-33], which is a retest of the deploy transport's SSH options against the real host. That is the repo's to close and is recorded here only so a reader is not surprised by it. <!-- Repo --> -[agents]: ../../AGENTS.md -[divergences]: ../divergences.md +[audit]: ../../AUDIT.md +[audit-runner]: ../../spec/audit.py [files]: ../../spec/files.json -[repos]: ../../registry/repos.json +[secrets]: ../../spec/secrets.json [section-model]: ../../spec/section-model.md <!-- External --> -[issue-456]: https://github.com/ptr727/ProjectTemplate/issues/456 -[pr-536]: https://github.com/ptr727/ProjectTemplate/pull/536 -[pr-540]: https://github.com/ptr727/ProjectTemplate/pull/540 -[pr-543]: https://github.com/ptr727/ProjectTemplate/pull/543 -[pr-545]: https://github.com/ptr727/ProjectTemplate/pull/545 +[blog-27]: https://github.com/ptr727/Blog/issues/27 +[blog-28]: https://github.com/ptr727/Blog/issues/28 +[blog-29]: https://github.com/ptr727/Blog/issues/29 +[blog-33]: https://github.com/ptr727/Blog/issues/33 +[issue-558]: https://github.com/ptr727/ProjectTemplate/issues/558 +[issue-563]: https://github.com/ptr727/ProjectTemplate/issues/563 +[pr-560]: https://github.com/ptr727/ProjectTemplate/pull/560 +[pr-561]: https://github.com/ptr727/ProjectTemplate/pull/561 +[pr-566]: https://github.com/ptr727/ProjectTemplate/pull/566 +[pr-569]: https://github.com/ptr727/ProjectTemplate/pull/569 diff --git a/reports/conformance-matrix.md b/reports/conformance-matrix.md index d7e1e89d..4be38a37 100644 --- a/reports/conformance-matrix.md +++ b/reports/conformance-matrix.md @@ -1,39 +1,40 @@ # Conformance Matrix -Tracks, per supported repo **shape** - the project type(s) plus the workflow model (`operational` is a `workflowModel` overlay, not a `spec/project-types.json` type) - whether a **context-free agent stands it up cold** from the hub docs to an audit-passing state ([`STANDUP.md`][standup] "Self-Test"), and the date (`YYYY-MM-DD`; `-` = not yet audited) of the shape's most recent audit ([`AUDIT.md`][audit]). A shape that cannot be stood up cold is a documentation defect, not an agent failure - iterate the docs until it can. +Tracks, per supported repo **shape**, whether a **context-free agent stands it up cold** from the hub docs to an audit-passing state ([`STANDUP.md`][standup] "Self-Test"), and the date of the shape's most recent audit ([`AUDIT.md`][audit]). A shape is the project type(s) plus the workflow model, where `operational` is a `workflowModel` overlay rather than a `spec/project-types.json` type. The date is `YYYY-MM-DD`, and a `-` cell means not yet audited. A shape that cannot be stood up cold is a documentation defect rather than an agent failure, so iterate the docs until it can. -`Cold-standup` values: `passing` (a fresh agent reaches operational), `gaps` (reaches partial; the note records the first doc gap), `not-tested` (self-test not yet run for this shape). +`Cold-standup` values: `passing` (a fresh agent reaches operational), `gaps` (reaches partial, and the note records the first doc gap), `not-tested` (self-test not yet run for this shape). -The primary shapes are stood up as whole repos; the **composable targets** (`nuget`, `pypi`, `docker`) layer a publish leaf onto a base repo and are exercised as part of a base shape's standup, not alone. +The primary shapes are stood up as whole repos. The **composable targets** (`nuget`, `pypi`, `docker`) layer a publish leaf onto a base repo and are exercised as part of a base shape's standup, not alone. ## Primary Shapes | Shape | Reference repo | Cold-standup | Last audited | First gap / notes | |---|---|---|---|---| -| `python` + `source-only` | Financial-Modeling | not-tested | - | Reference for the source-release (dispatch-only) profile; the downstream standup issue is open. | -| `source-only` + `release` | Blog | not-tested | 2026-08-03 | Hugo static site (#456), stood up 2026-08-01 and cataloged 2026-08-03. It cut release `1.0.11` from a dispatch-only publisher, so `releaseTrigger` is `dispatch-only` and `publish[]` carries the GitHub release, both recorded from what ran rather than from what was predicted. The VPS deploy has still not run, so its target has no declared type yet. | +| `python` + `source-only` | Financial-Modeling | not-tested | - | Reference for the source-release (dispatch-only) profile. The downstream standup issue is open. | +| `hugo` + `source-only` + `release` | Blog | not-tested | 2026-08-05 | Hugo static site (#456, #558), stood up 2026-08-01 and cataloged 2026-08-03. Release and deploy are independent surfaces: a dispatch-only publisher cuts the tag, and a separate dispatch deploys to a `self-hosted` filesystem per environment. Reclassified off the interim `source-only`-alone declaration when the type landed. **The 2026-08-05 run is the first to judge the `hugo` checks**, since the 2026-08-03 one predated the type and graded the repo as `source-only` alone. All nine pass, the repo is operational, and the three deviations the first run recorded are closed (ptr727/Blog#27, ptr727/Blog#28, ptr727/Blog#29). Two drift classes stay open, both of them the hub having advanced: seven stale verbatim carries and 17 prose findings. This is the shape `hugo.deploy.retention` was written against: its deploy credential is confined write-only, so the deploy can neither prune nor read the destination back, and the prune is a host-side timer its runbook records as host-owned. The first draft of that check demanded an in-pipeline assertion and would have pushed a correct design to widen a deliberately narrow credential. The run also surfaced a hub defect rather than a repo one, carried as escalation 1 of the report: the template-reference check had no exemption for the byte-locked `Fleet Bootstrap` section, whose first sentence must name the hub, so it fired unclearably on every repo that had carried the current canonical. Fixed since, across all three surfaces that stated the rule: the scan in `spec/audit.py` excises a file's verbatim sections before looking for the name, and `GOVERNANCE.md` "Documentation Style Conventions" and `recurring.norepoxref` both carry the exception and its boundary. It cleared exactly two findings fleet-wide and kept the other eleven. | | `csharp` + `console` | - | not-tested | - | | | `csharp` + `docker` | - | not-tested | - | | -| `csharp` + `python` | PlexCleaner | not-tested | - | First mixed-language shape (#339). Python is a stdlib-only `uvx` **scripts** profile subtree (`RegressionTests/`): no `uv.lock`, `pyproject.toml` lint/type config only, mypy checker, `python.uvlock.pinned` + `python.coverage.codecov` N/A; `codecov.yml` stays required for the C# side. Both language rule-sets apply (CODESTYLE.md "Two profiles"). | -| `homeassistant` | - | not-tested | - | Standalone-config conventions (home-assistant/core); scored by the `ha.*` checks. | +| `csharp` + `python` | PlexCleaner | not-tested | - | First mixed-language shape (#339). Python is a stdlib-only `uvx` **scripts** profile subtree (`RegressionTests/`): no `uv.lock`, `pyproject.toml` lint/type config only, mypy checker, `python.uvlock.pinned` + `python.coverage.codecov` N/A, and `codecov.yml` stays required for the C# side. Both language rule-sets apply (CODESTYLE.md "Two profiles"). | +| `homeassistant` | - | not-tested | - | Standalone-config conventions (home-assistant/core), scored by the `ha.*` checks. | | `eda` | - | not-tested | - | Data-zip release, pull consumer. | | `upstream-wrapper` | - | not-tested | - | Tag from a committed state file, not SemVer2. | | `codegen` | - | not-tested | - | Deterministic matrix over both branches. | -| `docs` | ProjectTemplate | not-tested | - | Governance hub; CI is lint-only. | +| `docs` | ProjectTemplate | not-tested | - | Governance hub, and CI is lint-only. | | `operational` config | - | not-tested | - | `workflowModel: operational`, direct signed commits to `develop`, promotion-PR gate. Its `develop` ruleset carries no `pull_request` rule, so the branch discipline rests on the instruction rather than the gate. | ## Composable Targets | Target | Exercised via | Cold-standup | Notes | |---|---|---|---| -| `nuget` | a `csharp` library base | not-tested | OIDC Trusted Publishing; no stored key. | -| `pypi` | a `python` library base | not-tested | OIDC; `environment: pypi`, `skip-existing: true`. | -| `docker` | any base with a Dockerfile | not-tested | Registry layer cache; always re-push. | +| `nuget` | a `csharp` library base | not-tested | OIDC Trusted Publishing, with no stored key. | +| `pypi` | a `python` library base | not-tested | OIDC, with `environment: pypi`, `skip-existing: true`. | +| `docker` | any base with a Dockerfile | not-tested | Registry layer cache, and always re-push. | +| `self-hosted` | a `hugo` base | not-tested | rsync over SSH into a per-environment release directory, with an atomic pointer flip. Retention takes either D5.6 shape: the deploy asserts the count where its credential can observe the destination, and the host owns it where that credential is confined write-only, which is the case on the first member. Credentials are per-environment GitHub Environment secrets and variables rather than repository secrets, so `spec/secrets.json` declares the mechanism with an empty `requires` and the audit cannot see whether the environments are configured. | ## Updating a Row 1. Run the [`STANDUP.md`][standup] self-test for the shape (fresh agent, docs only). -2. Run [`AUDIT.md`][audit] against the result; set `Cold-standup` and `Last audited`. +2. Run [`AUDIT.md`][audit] against the result, then set `Cold-standup` and `Last audited`. 3. If the result is not `passing`, record the first doc gap and fix it in the hub (docs or manifests), then re-run. <!-- Repo --> diff --git a/scripts/README.md b/scripts/README.md index dd555d23..3bc339b4 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -36,25 +36,27 @@ The default rule set covers comment shape (`comment-wrap` and `comment-case`) al A wide scan skips the trees this repo generates rather than authors, currently `reports/`, which [`spec/audit.py`][audit] writes. A finding there is the audit engine's phrasing rather than an author's, so no edit to that tree can fix it, and leaving them in made the repo's own number mostly generated output. Naming such a path directly still reads it (`prose_lint.py reports`), so nothing becomes uncheckable. -In markdown an HTML comment carrying no sentence punctuation is treated as a structural marker rather than commentary, so it takes neither a capital nor a sentence split. The reference-link group headers, the ToC-omit directive, and the `agent-safety` install markers are each matched verbatim by a tool, so rewriting one to satisfy the rule breaks whatever reads it. A markdown comment that does punctuate a sentence is prose and is judged as prose. +In Markdown an HTML comment carrying no sentence punctuation is treated as a structural marker rather than commentary, so it takes neither a capital nor a sentence split. The reference-link group headers, the ToC-omit directive, and the `agent-safety` install markers are each matched verbatim by a tool, so rewriting one to satisfy the rule breaks whatever reads it. A Markdown comment that does punctuate a sentence is prose and is judged as prose. -The `spelling` rule covers the US English convention where cspell does not reach. That gate reads README and HISTORY only, deliberately, because gating every markdown file would mean endlessly padding `cspell.json` with technical terms, so a British spelling anywhere else in the tree had nothing checking it. The banned words are generated from stems rather than listed one by one, since an inflected spelling is as wrong as its base and a hand-listed family drifts as soon as one form is added without the others. Two words are deliberately absent: `analyses` is the US plural of `analysis` as much as it is a British verb form, and `cancelled` is a GitHub Actions job status rather than prose. +The `spelling` rule covers the US English convention where cspell does not reach. That gate reads README and HISTORY only, deliberately, because gating every Markdown file would mean endlessly padding `cspell.json` with technical terms, so a British spelling anywhere else in the tree had nothing checking it. The banned words are generated from stems rather than listed one by one, since an inflected spelling is as wrong as its base and a hand-listed family drifts as soon as one form is added without the others. Two words are deliberately absent: `analyses` is the US plural of `analysis` as much as it is a British verb form, and `cancelled` is a GitHub Actions job status rather than prose. -**Outside markdown `spelling` and `dupword` read the comments, not the source lines**, reusing the extraction the `comment-wrap` rule already does. An identifier, a string literal, or a lookup table is code, and judging it as prose would make this script report its own table of banned words. Each comment on a line is judged on its own rather than joined with its neighbors, because two comments are two sentences and joining them reads the second's opening word as a repeat of the first's last. +**Outside Markdown `spelling` and `dupword` read the comments, not the source lines**, reusing the extraction the `comment-wrap` rule already does. An identifier, a string literal, or a lookup table is code, and judging it as prose would make this script report its own table of banned words. Each comment on a line is judged on its own rather than joined with its neighbors, because two comments are two sentences and joining them reads the second's opening word as a repeat of the first's last. `dupword` gates CI, so its scope decides what a correct file is allowed to contain. A repeated token outside a comment is usually correct authoring rather than a typo: `class="gallery gallery-cols-1"` is the ordinary way two CSS class names share a prefix, and `rel`, `srcset`, `sizes` and the `data-*` attributes all take value lists of the same shape. There is no edit that satisfies the rule without changing the rendered page, so a blocking gate that reads those lines rejects correct work. The cost of the narrower scope is stated plainly rather than hidden: a duplicated word in HTML body text, or in a YAML or JSON string value, is no longer caught. Narrowing to the comment is preferred over exempting an attribute, since an exemption list covers only the attributes its author thought of. **Scope** is every text file git tracks, binaries skipped by a NUL-byte check, with no extension allowlist: an allowlist covers what its author thought of and silently stops covering whatever is added next, which is the same reason the line-endings rule already requires `git ls-files` over a raw `find`. `--list-files` prints the discovered set for auditing. -A double-quoted span in markdown is treated as a quotation and not scanned for prose rules, so a rule that states its own counter-example does not report the document that documents it. Outside markdown a double quote is structural, so the prose inside it still counts. +A double-quoted span in Markdown is treated as a quotation and not scanned for prose rules, so a rule that states its own counter-example does not report the document that documents it. Outside Markdown a double quote is structural, so the prose inside it still counts. The `semicolon` and `dash` rules ban a construction rather than a detectable subset of it, so each flags by default and the exceptions are the ones the rule names: a semicolon inside a list that already carries commas, and for the dash a compound word, a leading list marker, a range, and the `- **Label** - explanation` separator that opens a governed bullet. -**The semicolon rule reads the list where it lives.** The comma qualifies the list as a whole rather than one separator's position, so an enumeration whose commas fall in a later item keeps every semicolon it carries. Reading it positionally split one series in two, flagging the openers of the same list it then exempted the tail of, which would have restructured the enumerated guarantees the exemption exists to protect. A markdown table row is judged one cell at a time, since a row is a record of fields and a comma in one column cannot excuse a semicolon in another, and a bullet's `**Label**:` is dropped before the line is read, because it opens the bullet rather than announcing a list, the same construct the label dash is exempted for. What this misses is a sentence that reads as a list without being one: a colon early in a long line still excuses a splice later on it, which reading the diff catches. +**The semicolon rule reads the list where it lives.** The comma qualifies the list as a whole rather than one separator's position, so an enumeration whose commas fall in a later item keeps every semicolon it carries. Reading it positionally split one series in two, flagging the openers of the same list it then exempted the tail of, which would have restructured the enumerated guarantees the exemption exists to protect. A Markdown table row is judged one cell at a time, since a row is a record of fields and a comma in one column cannot excuse a semicolon in another, and a bullet's `**Label**:` is dropped before the line is read, because it opens the bullet rather than announcing a list, the same construct the label dash is exempted for. The colon is written inside the emphasis as often as outside it, so `**Label:**` is dropped on the same grounds, matching only one spelling having left the other announcing a list it never announced. -**Both are markdown-only for now.** A shell script carries 78 statement separators that are not prose at all, so telling a comment from code is a precondition for reaching source files. Until then a semicolon or dash in a code comment is missed, which reading the diff by eye still catches. +**The sentence is the unit the exemption is judged on, because that is where a list lives.** The whole bullet decided it once, so a colon anywhere before the first semicolon marked the bullet a list and exempted every semicolon after it, however plainly one joined two independent clauses, and the two did not have to be near each other or related at all. Measured over this repo when it was fixed, the exemption was covering 62 spans holding 120 semicolons across 9 files while the rule reported none of them, so the gate read as clean over the docs it exists to check. Scoping it to the sentence reported 43 further semicolons and silenced none, with a 44th from dropping the other spelling of the label colon, and the sentence boundary is the run-on rule's, so an initial or an abbreviation ends nothing and a terminator closing inside emphasis or a bracket (`.**`, `.)`) still ends a sentence. The colon arm was measured before being kept rather than dropped: dropping it flagged 14 further lines, and those were genuine colon-introduced lists whose items carry commas, which is the standard use the rule names. -The `comment-wrap` rule covers comments in every syntax the fleet's project types carry, not only the hash ones: `//` and `/* */` for C#, C, C++ and JSONC, `/* */` alone for CSS, `<!-- -->` for XML, csproj and markdown, `<# #>` for PowerShell, `;` for INI, and `#` for Python, shell, YAML and TOML. +**Both are Markdown-only for now.** A shell script carries 78 statement separators that are not prose at all, so telling a comment from code is a precondition for reaching source files. Until then a semicolon or dash in a code comment is missed, which reading the diff by eye still catches. + +The `comment-wrap` rule covers comments in every syntax the fleet's project types carry, not only the hash ones: `//` and `/* */` for C#, C, C++ and JSONC, `/* */` alone for CSS, `<!-- -->` for XML, csproj and Markdown, `<# #>` for PowerShell, `;` for INI, and `#` for Python, shell, YAML and TOML. JSON is treated as JSONC, because that is what ships: VS Code tasks, launch, devcontainer and workspace files all carry comments under a plain `.json` name. A marker inside a string literal is not a comment, so each line is scanned with quoted spans blanked first, and Python uses `tokenize` so a trailing comment is seen exactly. A documentation comment (`///`, `/**`, a docstring) is left to CODESTYLE, which permits the paragraphs this rule forbids. diff --git a/scripts/pr_review.py b/scripts/pr_review.py index 09ef7390..6d883241 100644 --- a/scripts/pr_review.py +++ b/scripts/pr_review.py @@ -37,6 +37,10 @@ SUMMARY = re.compile(r'<summary>(.*?)</summary>', re.DOTALL | re.IGNORECASE) TAGS = re.compile(r'</?(?:details|summary)>', re.IGNORECASE) COUNT = re.compile(r'\((\d+)\)') +# What makes a line a heading rather than prose mentioning the phrase, in either markup. +# A line that is neither still qualifies where it carries a count. +# The section has appeared as a bare line too, and a count is what prose does not carry. +HEADING = re.compile(r'\s*(?:#{1,6}\s|<summary)', re.IGNORECASE) # How many of the newest reviews and comments both queries read. # A narrow window drops the reviewer's answer behind ordinary discussion, reporting no answer. @@ -207,28 +211,44 @@ def live_state(owner: str, repo: str, num: int) -> tuple[str, bool, dict | None] def heading_of(block: str) -> str: - """The block's `<summary>`, or its opening where the wrapper carries none.""" - m = SUMMARY.search(block) - return m.group(1) if m else block[:200] + """The block's own heading, unwrapped from `<summary>` where it wears one. + + A block starts at its heading, so the heading is the first line whatever markup it carries. + Reading further would let a finding's own text supply a count the heading never carried, and + a wrong count reads exactly like a right one. + """ + head = block.lstrip() + m = SUMMARY.search(head) + return m.group(1) if m and head.lower().startswith('<summary') else head.split('\n', 1)[0] def suppressed_blocks(body: str) -> list[str]: - """Return the review body's low-confidence sections, matched on their heading. + """Return the review body's low-confidence sections, each sliced from its own heading. + + The section has worn three shapes so far: its own `<details>` wrapper, a bare heading in the + body, and a Markdown heading nested inside the `Review details` wrapper. The wrapper is the + part that keeps moving, so each region is scanned line by line for the heading rather than + read for a wrapper's summary: a nested heading is not a summary, and stripping the wrappers + to look for it outside deletes the very region it sits in. That is the pair that reported + `suppressed=0` over a body carrying `### Suppressed comments (2)`. The heading carries the match rather than the body text, since a review whose prose discusses - suppressed findings is not itself carrying any. The fallback covers the day the `<details>` - wrapper moves, and takes a heading with a count so ordinary prose is not read as one. + suppressed findings is not itself carrying any. A region ends the block, so a section is not + read on into the file table that follows it. """ if not body: return [] - blocks = [b for b in DETAILS.findall(body) if SUPPRESSED.search(heading_of(b))] - if blocks: - return blocks - outside = DETAILS.sub('', body).splitlines() - for i, line in enumerate(outside): - if SUPPRESSED.search(line) and COUNT.search(line): - return ['\n'.join(outside[i:])] - return [] + # Each wrapper's contents, plus what is left outside them all, so a heading is found anywhere. + # The regions do not overlap, since what the sub deletes is exactly what the findall keeps. + regions = DETAILS.findall(body) + [DETAILS.sub('', body)] + blocks = [] + for region in regions: + lines = region.splitlines() + for i, line in enumerate(lines): + if SUPPRESSED.search(line) and (HEADING.match(line) or COUNT.search(line)): + blocks.append('\n'.join(lines[i:])) + break + return blocks def finding_count(block: str) -> int: diff --git a/scripts/prose_lint.py b/scripts/prose_lint.py index a158b5bb..1353ca5d 100644 --- a/scripts/prose_lint.py +++ b/scripts/prose_lint.py @@ -605,15 +605,35 @@ def strip_quoted(s: str) -> str: return re.sub(r'"[^"\n]*"', '""', s) -# A bullet's `**Label**:` opens the text the same way `- **Label** -` does, so its colon -# introduces the bullet rather than a list, and reading it as one excused the splice after it. -LABEL_COLON = re.compile(r'^\s*(?:[-*]|[0-9]+\.)\s+\*\*[^*]+\*\*\s*:') +# A bullet's `**Label**:` opens the text the same way `- **Label** -` does. +# Its colon introduces the bullet rather than a list, and reading it as one excused the splice. +# The colon is written inside the emphasis as often as outside it, and both spell one construct. +# Matching `**Label**:` alone left `**Label:**` announcing a list it never announced. +LABEL_COLON = re.compile(r'^\s*(?:[-*]|[0-9]+\.)\s+\*\*[^*]+?(?:\*\*\s*:|:\s*\*\*)') + +# A sentence boundary inside one line, so a list exemption is scoped to the sentence holding it. +# The guards are the run-on rule's, so an initial or an abbreviation ends nothing. +# The trailing class is the emphasis or bracket a Markdown sentence closes inside. +# Reading a bare `. ` instead left `.**` and `.)` joining a bullet's every sentence into one span. +SENTENCE_BREAK = re.compile(r'(?<!\b[A-Z])(?<!\be\.g)(?<!\bi\.e)(?<!\bvs)(?<!\betc)' + r'[.!?][*_`"\')\]]*\s+') + + +def sentences(span: str) -> list[str]: + """The span split at its sentence boundaries, empty pieces dropped. + + A list lives inside one sentence, so the sentence is the unit an exemption may be judged on. + Judged over a whole bullet instead, a colon anywhere before the first semicolon marked the + bullet a list and exempted every semicolon after it, however plainly one joined two clauses. + The colon and the semicolon did not have to be near each other, or related at all. + """ + return [s for s in SENTENCE_BREAK.split(span) if s.strip()] def list_spans(s: str) -> list[str]: """Split a line into the spans that each hold their own list. - A markdown table row is a record of fields rather than one sentence, so judging the row whole + A Markdown table row is a record of fields rather than one sentence, so judging the row whole let a comma in one column excuse a semicolon in another. """ cells = s.strip().strip('|').split('|') if s.lstrip().startswith('|') else [s] @@ -821,7 +841,7 @@ def comment_wrap_findings(path: Path, raw: str, lines: list[str]) -> list[tuple[ if not body or NOT_PROSE.search(body) or BARE_URI.match(body.strip()): prev_body = '' continue - # An unpunctuated markdown HTML comment is a structural marker, not commentary. + # An unpunctuated Markdown HTML comment is a structural marker, not commentary. # It is a label, so it takes neither a capital nor a sentence split. # A tool matches each one verbatim, so rewriting it breaks whatever reads it. # Group headers, the ToC-omit directive, and the agent-safety markers are the cases. @@ -860,7 +880,7 @@ def check_file(path: Path, rules: set[str]) -> list[tuple[int, str, str]]: lines = raw.split('\n') if {'comment-wrap', 'comment-case'} & rules: out.extend(f for f in comment_wrap_findings(path, raw, lines) if f[1] in rules) - # Outside markdown the prose lives in the comments, and both rules judge prose, not code. + # Outside Markdown the prose lives in the comments, and both rules judge prose, not code. # A source line holds identifiers and literals, and an attribute value may legally repeat. # Reading it rejects correct work, `class="gallery gallery-cols-1"` being the reported case. comments: dict[int, list[str]] = {} @@ -885,22 +905,24 @@ def check_file(path: Path, rules: set[str]) -> list[tuple[int, str, str]]: txt = strip_inline_code(line) prose = strip_quoted(txt) if path.suffix == '.md' else txt - # Both prose rules are markdown-only until a comment can be told from code. + # Both prose rules are Markdown-only until a comment can be told from code. # A shell script carries 78 statement separators that are not prose at all. if path.suffix == '.md': if 'semicolon' in rules: for span in list_spans(prose): - # A list keeps its semicolons, announced by a colon or a second separator. - # The comma is a property of the list rather than of one separator's position, - # so an enumeration whose commas fall in a later item keeps every semicolon it - # carries. Reading it positionally split such a list, flagging the openers of - # the same series it then exempted the tail of. - listish = span.count(';') > 1 or ':' in span.split(';')[0] - if listish and ',' in span: - continue - for _ in SEMICOLON.finditer(span): - out.append((i, 'semicolon', - 'semicolon in prose -> a comma or two sentences')) + # The sentence is the unit, since the list an exemption protects lives in one. + # Judged over a whole bullet, one colon exempted every semicolon after it. + for sentence in sentences(span): + # A list keeps its semicolons, announced by a colon or a second separator. + # The comma qualifies the list rather than one separator's position. + # An enumeration whose commas fall in a later item keeps every semicolon. + # Read positionally, it split one series and flagged that series' openers. + listish = sentence.count(';') > 1 or ':' in sentence.split(';')[0] + if listish and ',' in sentence: + continue + for _ in SEMICOLON.finditer(sentence): + out.append((i, 'semicolon', + 'semicolon in prose -> a comma or two sentences')) if 'dash' in rules: skip = LABEL_DASH.match(prose) for m in DASH.finditer(prose): diff --git a/scripts/test_pr_review.py b/scripts/test_pr_review.py index 8e563e28..419cef3c 100644 --- a/scripts/test_pr_review.py +++ b/scripts/test_pr_review.py @@ -44,6 +44,22 @@ def collapsed(heading: str = 'Comments suppressed due to low confidence (1)', f'{finding}\n\n</details>\n') +def nested(heading: str = '### Suppressed comments (2)', + finding: str = '**a.py:12**\n* The retry count is off by one.') -> str: + """The section as a Markdown heading nested inside the `Review details` wrapper. + + The live shape as of 2026-08-05: the section is no longer its own `<details>` wrapper with a + matching `<summary>`, it is a Markdown heading inside the wrapper that also carries the + round's file and effort metadata, which trails the findings rather than preceding them. + """ + return ('### Ready to approve\n\nThe change is narrow.\n\n' + '<details>\n<summary>File summaries</summary>\n\n' + '| File | Description |\n\n</details>\n\n' + f'<details>\n<summary>Review details</summary>\n\n{heading}\n\n{finding}\n\n' + '- **Files reviewed:** 1/1 changed files\n' + '- **Review effort level:** Lite\n</details>\n') + + def thread(tid: str, resolved: bool = False, login: str = pr_review.REVIEWER, body: str = 'A finding.', path: str = 'a.py', line: int = 1) -> dict: return {'id': tid, 'isResolved': resolved, @@ -361,6 +377,47 @@ def test_prose_that_merely_discusses_the_phrase_is_not_a_block(self) -> None: self.assertIn('suppressed=0', out) self.assertNotIn('SUPPRESSED', out) + def test_a_heading_nested_inside_the_review_details_wrapper_reports(self) -> None: + """The shape that reported `suppressed=0` over a body carrying two findings. + + The reviewer moved the section inside the `Review details` wrapper as a Markdown heading, + so the wrapper's summary reads `Review details` and matches nothing, and the fallback that + exists for a moved wrapper scans the body with every wrapper deleted, which deletes the + region the heading now sits in. The two misses compound into a clean round over findings + no thread will ever carry, which is the one failure this whole digest exists to prevent. + """ + self.answer(payload([review(body=nested())])) + out, _ = pr_review.digest('o', 'r', 7) + self.assertIn('suppressed=2', out) + self.assertIn('The retry count is off by one.', out) + + def test_the_nested_count_is_the_heading_s_own_rather_than_the_wrapper_s(self) -> None: + """The wrapper's summary carries no count, so reading it floors two findings to one.""" + self.answer(payload([review(body=nested(heading='### Suppressed comments (3)'))])) + out, _ = pr_review.digest('o', 'r', 7) + self.assertIn('suppressed=3', out) + # The block starts at its own heading, so the wrapper's summary is not the finding's header. + self.assertNotIn('Review details', out) + + def test_both_the_wrapper_shape_and_the_nested_shape_report_in_one_run(self) -> None: + """Both appear across the rounds of a single pull request, so neither replaces the other. + + Retargeting the parse from the old shape to the new one would report the same false clean + one round later, on whichever shape the reviewer happened not to emit that time. + """ + self.answer(payload([review(oid=OLD, body=collapsed(heading='Suppressed comments (2)')), + review(body=nested())])) + out, _ = pr_review.digest('o', 'r', 7) + self.assertIn('suppressed=4', out) + self.assertIn('(on_head=2 earlier=2)', out) + + def test_the_file_summary_wrapper_beside_a_nested_section_is_still_not_a_finding(self) -> None: + """A body carries several wrappers, and scanning them all must not read the table as one.""" + self.answer(payload([review(body=nested())])) + out, _ = pr_review.digest('o', 'r', 7) + self.assertNotIn('File summaries', out) + self.assertNotIn('| File | Description |', out) + def test_a_human_review_carrying_the_phrase_is_not_a_copilot_finding(self) -> None: self.answer(payload([review(login='ptr727', body=collapsed()), review()])) out, _ = pr_review.digest('o', 'r', 7) diff --git a/scripts/test_prose_lint.py b/scripts/test_prose_lint.py index 9822f302..ffe6d9df 100644 --- a/scripts/test_prose_lint.py +++ b/scripts/test_prose_lint.py @@ -169,10 +169,10 @@ def test_the_tiers_do_not_overlap(self) -> None: class TestDupword(BaitCase): - """A doubled word, read from markdown prose and from the comments of every other syntax. + """A doubled word, read from Markdown prose and from the comments of every other syntax. The scope matters more here than for the other prose rules, because this one gates CI. Outside - markdown a repeated token is far more often correct code than a typo: `class="gallery + Markdown a repeated token is far more often correct code than a typo: `class="gallery gallery-cols-1"` is the ordinary HTML idiom, and no edit fixes it without changing the page. """ @@ -321,6 +321,56 @@ def test_a_bullet_label_colon_does_not_announce_a_list(self) -> None: self.kinds('- **Async**: avoid blocking calls; use await, always\n', {'semicolon'})) + def test_a_colon_in_an_earlier_sentence_does_not_exempt_a_later_splice(self) -> None: + """The exemption belongs to the sentence, not the bullet, and the two were unrelated. + + Read over the whole bullet, an enumeration in one sentence excused every semicolon after + it, so the gate went silent across 120 semicolons in the docs it exists to check. + """ + self.assertEqual(['semicolon'], self.kinds( + '- **A rule.** The evidence is three things, and each matters: the first, the second, ' + f'and the third. {SPLICE_BAIT}.\n', {'semicolon'})) + + def test_a_sentence_closing_inside_emphasis_or_a_bracket_still_ends(self) -> None: + """`.**` and `.)` end a sentence, and reading a bare `. ` joined a whole bullet into one.""" + for opener in ('- **A label: with a list, of two.**', + 'A label (with a list, of two.)'): + with self.subTest(opener=opener): + self.assertEqual(['semicolon'], + self.kinds(f'{opener} {SPLICE_BAIT}.\n', {'semicolon'})) + + def test_a_series_in_one_sentence_does_not_exempt_the_next(self) -> None: + """The second-separator arm is scoped the same way, a series belonging to its sentence.""" + self.assertEqual(['semicolon'], self.kinds( + 'It covers each target, and excludes the rest; it runs on push; it gates. ' + f'{SPLICE_BAIT}.\n', {'semicolon'})) + + def test_a_colon_introduced_list_whose_items_carry_commas_keeps_its_semicolon(self) -> None: + """The colon arm earns its place: dropping it flagged this, the use the rule names. + + Measured over the tree, dropping it reported 14 further lines, and the shapes below are + what they were, so the arm is scoped rather than removed. + """ + for text in (('Match the heading style: title case with short bind words (a, an, the, of); ' + 'hyphenated compounds capitalize both parts.\n'), + ('- **Python** (the script profile): lint, format, and type check; ' + 'format-on-save and import organization via the formatter.\n')): + with self.subTest(text=text.split(':')[0]): + self.assertEqual([], self.kinds(text, {'semicolon'})) + + def test_a_bullet_label_colon_inside_the_emphasis_is_the_same_opener(self) -> None: + """`- **D3:**` and `- **D3**:` are one construct, and only one spelling was stripped.""" + self.assertEqual(['semicolon'], self.kinds( + '- **D3:** each run builds one branch, so it classifies the version directly; the ' + 'gate literal, the expression, and the config all name the same branch.\n', + {'semicolon'})) + + def test_an_abbreviation_does_not_end_a_sentence(self) -> None: + """Splitting at `e.g.` cuts a list in half and flags the separator the exemption protects.""" + self.assertEqual([], self.kinds( + 'Pinned by path: a script, a hook (e.g. a shebang); vanilla files stay as they are.\n', + {'semicolon'})) + def test_prose_rules_do_not_reach_code_files(self) -> None: """A shell script carries statement separators, not prose, until comments can be extracted.""" for name in ('bait.sh', 'bait.py', 'bait.yml'): @@ -620,7 +670,7 @@ def test_an_unpunctuated_markdown_marker_is_a_label_not_a_sentence(self) -> None # A punctuated HTML comment is commentary, so it stays judged as prose. self.assertEqual(['comment-wrap'], self.flag('a.md', 'Prose.\n\n<!-- One thing. Another thing. -->\n')) - # Outside markdown the carve-out does not apply, since there the marker case does not arise. + # Outside Markdown the carve-out does not apply, since there the marker case does not arise. self.assertEqual(['comment-case'], self.flag('a.py', '# lowercase opening\n')) def test_a_block_opener_inside_a_line_comment_is_text(self) -> None: @@ -1116,7 +1166,7 @@ def test_a_binary_file_is_not_scanned(self) -> None: class TestSentenceSplit(BaitCase): - """One sentence per line, the markdown counterpart of the comment-wrap rule.""" + """One sentence per line, the Markdown counterpart of the comment-wrap rule.""" def test_a_sentence_continuing_onto_the_next_line_is_flagged(self) -> None: self.assertEqual(['sentence-split'], @@ -1152,7 +1202,7 @@ def test_the_rule_is_markdown_only(self) -> None: class TestSpelling(BaitCase): - """US English, read from markdown prose and from the comments of every other syntax. + """US English, read from Markdown prose and from the comments of every other syntax. The rule runs on whatever file it is handed, README and HISTORY included. It complements the cspell gate rather than dividing the tree with it: cspell reads those two files and this reads diff --git a/spec/audit.py b/spec/audit.py index 03c707d3..11e819d1 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -4,7 +4,7 @@ Compares each cataloged registry repo against the ground truth in this repo - general settings (repo-config/settings.json), branch rulesets (normalized diff vs the model's payloads), secret names (spec/secrets.json; values are never read), baseline/per-type file presence and per-scope -markdown section presence on the ground-truth branch (spec/files.json, spec/scope-model.md), and +Markdown section presence on the ground-truth branch (spec/files.json, spec/scope-model.md), and branch-model facts (main/develop existence, develop behind main). Owner-initiated: run it when onboarding a repo, when drift is suspected, or before fleet-wide changes. Read-only - it never modifies a target. @@ -51,6 +51,11 @@ # Deliberately specific, so a note recording a permanent deviation must not match. # An example of one that must not match is a note reading that there is no get-version-task and validate-task is relied on instead. PENDING_MARKERS = ["pending", "not yet", "owed", "todo", "still", "behind", "missing", "absent"] +# A driftNote may name the check that would retire it, as a parenthesized id: "(hugo.generator.pinned)". +# Deliberately unanchored. +# Both notes this form was introduced for end the sentence after the paren, so an end-anchored pattern matches neither of the two it was written to catch. +# That is the shape a matcher fails at silently: it reports nothing, and a fleet with no such note in it reports exactly the same. +CHECK_ID_RE = re.compile(r"\(([a-z][a-z0-9-]*(?:\.[a-z0-9-]+)+)\)") def load(rel): @@ -113,6 +118,58 @@ def normalize_ruleset(payload): return json.dumps(sub, sort_keys=True) +def check_id_owner(spec, cid): + """The project type owning a check id, and whether the catalog defines the id at all. + + Returns (None, True) for a cross-cutting check, which every repo carries and no repo declares. + The catalog is a required input rather than an optional one: resolving against an absent + catalog would report every id as undefined, which is a louder failure than reporting none. + """ + types = spec.get("types") + if not types: + raise KeyError("spec['types'] (spec/project-types.json) is required to resolve a driftNote check id") + for name, t in types.get("types", {}).items(): + if any(c.get("id") == cid for c in t.get("checks", [])): + return name, True + for dim in types.get("crossCutting", {}).values(): + if any(c.get("id") == cid for c in dim.get("checks", [])): + return None, True + return None, False + + +def driftnote_findings(entry, spec, open_count): + """Freshness findings over one repo's registry driftNotes, given how many findings the audit already has. + + Neither shape below is gated on the rest of the audit being clean, and the gate that used to wrap both + is the defect this replaces: one standing finding a repo cannot clear exempted its whole note list, so + the repo with open findings, where a stale note is most likely, was the one never checked. + + A note naming a check id declares what would retire it, so it is surfaced on every run. What the audit + cannot do is decide it, since no per-type check is mechanized, so the id is resolved against the + catalog here and the check itself is left to the auditor. A pending-marker note is a prose claim that + work is outstanding, so a clean audit contradicts it outright while an unclean audit only asks which of + the open findings it means. Narrow markers keep a permanent-deviation note ("relies on validate-task") + from tripping. + """ + out = [] + for note in entry.get("driftNotes", []): + quoted = f"\"{note[:70]}{'...' if len(note) > 70 else ''}\"" + for cid in CHECK_ID_RE.findall(note): + owner, known = check_id_owner(spec, cid) + if not known: + out.append(("DRIFT", f"registry: driftNote names check '{cid}', which spec/project-types.json does not define - fix the id or drop the note: {quoted}")) + elif owner and owner not in entry.get("types", []): + out.append(("DRIFT", f"registry: driftNote names check '{cid}', whose type '{owner}' this repo does not declare: {quoted}")) + else: + out.append(("DRIFT", f"registry: driftNote names check '{cid}', which this audit does not evaluate by id (AUDIT.md section 4) - judge it by hand and delete the note once it passes: {quoted}")) + marker = next((w for w in PENDING_MARKERS if re.search(rf"\b{re.escape(w)}\b", note, re.I)), None) + if marker and not open_count: + out.append(("DRIFT", f"registry: driftNote says '{marker}' but the audit is clean - verify and reconcile: {quoted}")) + elif marker: + out.append(("DRIFT", f"registry: driftNote says '{marker}' while {open_count} finding(s) are open - confirm it describes one of them rather than closed work: {quoted}")) + return out + + def repo_slug(entry): # url is https://github.com/<owner>/<repo> return "/".join(entry["url"].rstrip("/").split("/")[-2:]) @@ -208,8 +265,48 @@ def extract_section(text, heading): return "\n".join(out) if capturing else None +# Carried files scanned for a coordination reference (GOVERNANCE.md "Documentation Style Conventions"). +TEMPLATE_REF_SCANNED = ("AGENTS.md", "GOVERNANCE.md", ".github/copilot-instructions.md") + + +def strip_sections(text, names): + """`text` with each named `## <heading>` region removed, located by position rather than by content. + + Region rules match extract_section (a fenced `## ` is not a boundary, a sibling H2 ends the region), so + the two agree on where a section starts and stops. + Positional removal is the point: deleting the extracted text instead would also delete an identical + passage anywhere else in the document, including one quoted inside the prose the caller means to read. + That failure is silent and it fails open, since the removed duplicate takes its content out of the scan. + """ + want = {n.strip().lower() for n in names} + out, dropping, fenced = [], False, False + for ln in normalize(text).split("\n"): + stripped = ln.strip() + if stripped.startswith("```") or stripped.startswith("~~~"): + fenced = not fenced + elif not fenced and stripped.startswith("## "): + dropping = stripped[2:].strip().lower() in want # a sibling H2 always ends the previous region + if not dropping: + out.append(ln) + return "\n".join(out) + + +def template_ref_outside_verbatim(text, verbatim_names, hub_name): + """True when `hub_name` appears in `text` outside every one of its verbatim sections. + + A verbatim section's bytes are the hub's canonical and are checked byte-for-byte elsewhere, so a hub + reference inside one cannot be removed downstream: the repo would have to fail the verbatim check to + clear this one. `AGENTS.md > Fleet Bootstrap` is the standing case, since naming the hub is that + section's entire function - it is the byte-locked entry point stating where the canonical rules live, + and a repo holding no current copy of anything else is exactly who reads it. Excising the verbatim + regions before scanning keeps the check pointed at the prose a repo actually owns. A hub reference that + reaches a verbatim section is the hub's defect to fix once in the canonical, never each repo's to clear. + """ + return hub_name.lower() in strip_sections(text, verbatim_names).lower() + + def heading_texts(markdown): - """Lowercased heading texts in a markdown document, for case-insensitive section-presence matching.""" + """Lowercased heading texts in a Markdown document, for case-insensitive section-presence matching.""" return {m.group(1).strip().lower() for line in markdown.splitlines() for m in (_HEADING.match(line),) if m} @@ -626,17 +723,6 @@ def audit_repo(entry, spec, branch=None): for name in sorted(present - claimed_names): findings.append(("DRIFT", f"secrets: {name} in the {store} store is claimed by no applicable mechanism (stale?)")) - # --- Carried files must not reference the template repo --- - # The coordination flow is machinery a consumer should not see, so a carried file states the behavior rather than the destination. - # This reads AGENTS.md, GOVERNANCE.md and .github/copilot-instructions.md, where a stale "report drift upstream" paragraph once spread. - # Skip the hub itself, whose own carried files are the source, where naming the repo they live in is correct. - # A downstream repo naming it is still flagged, which is the point. - if entry.get("name") != HUB_NAME: - for path in ("AGENTS.md", "GOVERNANCE.md", ".github/copilot-instructions.md"): - doc = gh(f"repos/{slug}/contents/{path}?ref={ground}", ok404=True) - if doc and doc.get("content") and HUB_NAME.lower() in base64.b64decode(doc["content"]).decode("utf-8", "replace").lower(): - findings.append(("DRIFT", f"carried: {path} references the template repo by name or link (the coordination flow is machinery this repo's readers should not see; state the behavior, not the destination)")) - # --- Dependabot ecosystem coverage --- # A repo's tree implies Dependabot ecosystems it must track: github-actions when it ships workflows # (the action versions they reference otherwise go stale, and a merge-bot then has no PRs to auto-merge), @@ -662,7 +748,7 @@ def audit_repo(entry, spec, branch=None): # --- File and section presence on the ground-truth branch --- # appliesTo is matched against the repo's full selector set (types + workflowModel + releaseTrigger + # consumerModel), so the release/operational develop ruleset is two data entries, not a code swap. - # Required sections union across same-path entries. A carried markdown file must contain each heading + # Required sections union across same-path entries. A carried Markdown file must contain each heading # scoped to this repo. A rename reads as missing and equivalence is judged by hand, so a missing section # is DRIFT (a hint to verify), never a LETTER. sel = repo_selectors(entry, spec["registry"].get("defaults", {})) @@ -719,7 +805,7 @@ def audit_repo(entry, spec, branch=None): findings.append(("DRIFT", f"verbatim: could not read {path} content on {ground} to compare (no inline content returned); verify by hand")) else: findings.extend(check_verbatim(path, text, item.get("reference") or path)) - # Heading-based presence is only meaningful for markdown. A "section" named on a non-md file (e.g. a + # Heading-based presence is only meaningful for Markdown. A "section" named on a non-md file (e.g. a # tasks.json task group) is an intent marker judged per AUDIT.md, not a heading grep. needed = wanted_sections[path] verbatim_needed = verbatim_secs[path] @@ -754,6 +840,19 @@ def audit_repo(entry, spec, branch=None): for h in sorted(h2s - declared): findings.append(("DRIFT", f"section: '{h}' in {path} is not a declared section - reconcile it (a duplicate of a verbatim section, or repo-specific content that moves to a topical doc), or confirm it is intentional (spec/section-model.md)")) + # --- Carried files must not reference the template repo --- + # The coordination flow is machinery a consumer should not see, so a carried file states the behavior rather than the destination. + # A stale "report drift upstream" paragraph once spread this way. + # Skip the hub itself, whose own carried files are the source, where naming the repo they live in is correct. + # A downstream repo naming it in prose it owns is still flagged, which is the point. + # Verbatim sections are excised first, and template_ref_outside_verbatim carries why that is not a loophole. + # Sited here, in the file loop, so the scan reuses the content already fetched for the section checks and reads the same selector-resolved verbatim list they were judged against. + if path in TEMPLATE_REF_SCANNED and entry.get("name") != HUB_NAME: + if text is None: + findings.append(("DRIFT", f"carried: could not read {path} content on {ground} to scan for a coordination reference (no inline content returned); verify by hand")) + elif template_ref_outside_verbatim(text, verbatim_secs[path], HUB_NAME): + findings.append(("DRIFT", f"carried: {path} references the template repo by name or link outside its verbatim sections (the coordination flow is machinery this repo's readers should not see; state the behavior, not the destination)")) + # --- HISTORY.md mirrors the README opening --- # spec/readme-structure.md "HISTORY.md": the changelog opens as the README's twin - same H1 title and the # same intro paragraph. Checked only when both files were readable (absence is already a file LETTER above). @@ -783,7 +882,7 @@ def audit_repo(entry, spec, branch=None): findings.append(("LETTER", "readme: no intro line after the H1 - the README opens with the title then a one-line description, which doubles as the About description (spec/readme-structure.md)")) else: if strip_md_links(intro_line) != intro_line: - findings.append(("LETTER", "readme: the intro line carries markdown links - keep it link-free plain text, it doubles as the repo About description (spec/readme-structure.md)")) + findings.append(("LETTER", "readme: the intro line carries Markdown links - keep it link-free plain text, it doubles as the repo About description (spec/readme-structure.md)")) want = strip_md_links(intro_line).strip() if len(want) > 100: findings.append(("LETTER", f"readme: the intro line is {len(want)} characters, over the 100-char limit (Docker Hub's short-description cap, the tightest surface it feeds) - tighten it to one short sentence (spec/readme-structure.md)")) @@ -821,13 +920,7 @@ def audit_repo(entry, spec, branch=None): findings.append(("LETTER", f"cspell: {ws_name} carries a cSpell word list while cspell.json is the single source of truth - delete the workspace copy (CODESTYLE.md Markdown and Spelling)")) # --- Registry driftNotes freshness --- - # Gated on everything else passing: a clean repo has no outstanding work for a pending-marker note to - # describe. Narrow markers keep a permanent-deviation note ("relies on validate-task") from tripping. - if not findings: - for note in entry.get("driftNotes", []): - marker = next((w for w in PENDING_MARKERS if re.search(rf"\b{re.escape(w)}\b", note, re.I)), None) - if marker: - findings.append(("DRIFT", f"registry: driftNote says '{marker}' but the audit is clean - verify and reconcile: \"{note[:70]}{'...' if len(note) > 70 else ''}\"")) + findings.extend(driftnote_findings(entry, spec, len(findings))) # Stamp the commit actually read for the ground-truth branch. Never fall back to another branch: # a stamp naming develop while carrying main's sha would misattribute every finding. @@ -964,6 +1057,36 @@ def _selftest(): else: print(" ok section: heading in region, fenced ## kept, sibling H2 ends, None if absent, whitespace-tolerant locate, re-cased heading rehashes") + # Coordination-reference scan: the hub name inside a verbatim section is exempt, outside one is not. + # The first case is the real AGENTS.md shape, where the byte-locked Fleet Bootstrap block must name the hub and a repo therefore cannot clear a finding against it. + # The CRLF case matters because extract_section normalizes EOLs while carried files are CRLF on this fleet, so excision must survive that. + boot = "## Fleet Bootstrap\n\nThe canonical rules live in `github.com/acme/Hub`.\n" + owned = "## Where the Rules Live\n\nReport a rule discrepancy to acme/Hub.\n" + clean_doc = "# AGENTS\n\n" + boot + "\n## Where the Rules Live\n\nState the behavior, not the destination.\n" + dirty_doc = "# AGENTS\n\n" + boot + "\n" + owned + tref = [ + ("hub name only inside the verbatim section", clean_doc, {"Fleet Bootstrap"}, False), + ("hub name in prose the repo owns", dirty_doc, {"Fleet Bootstrap"}, True), + ("same document with nothing declared verbatim still flags", clean_doc, set(), True), + ("CRLF document excises the same way", clean_doc.replace("\n", "\r\n"), {"Fleet Bootstrap"}, False), + ("a re-cased verbatim heading still excises", clean_doc.replace("## Fleet Bootstrap", "## fleet bootstrap"), {"Fleet Bootstrap"}, False), + ("no hub reference at all", "# AGENTS\n\n## Where the Rules Live\n\nNothing to see.\n", {"Fleet Bootstrap"}, False), + # A second region under the same heading is excised by name, which is right: both are that section. + ("the heading appearing twice excises both", "# AGENTS\n\n" + boot + "\n## Notes\n\n" + boot, {"Fleet Bootstrap"}, False), + # A fenced copy is not a heading, so it is prose the repo owns and the reference in it must flag. + # This is the case that made positional excision necessary, because removing the extracted text instead would delete the fenced copy along with the real region and the scan would fail open. + # That is the one arrangement a repo could otherwise use to carry the reference in a document it owns. + ("a fenced copy of the section is prose, not the section", "# AGENTS\n\n## Notes\n\n```\n" + boot + "```\n\n" + boot, {"Fleet Bootstrap"}, True), + ] + tref_ok = True + for label, doc, verb, want in tref: + got = template_ref_outside_verbatim(doc, verb, "acme/Hub") + if got != want: + ok = tref_ok = False + print(f" FAIL template-ref: {label} (expected {want}, got {got})") + if tref_ok: + print(f" ok template-ref: {len(tref)} cases, verbatim regions excised before the hub-name scan") + # Issue generator: findings land in the right buckets and the title carries the count. fe = {"name": "Widget", "types": ["python"]} it, ib = render_issue(fe, [("LETTER", "file: X absent"), ("DRIFT", "verbatim: Y differs"), ("ERROR", "gh failed")], @@ -995,7 +1118,7 @@ def _selftest(): ok = False print(" FAIL description: strip_md_links behavior") else: - print(" ok description: markdown links reduce to their text, plain text passes through") + print(" ok description: Markdown links reduce to their text, plain text passes through") # cspell duplication: a workspace cSpell word list is detected, and a mere cspell.json mention is not. ws_dup = '{ "settings": { "cSpell.words": ["foo"] } }' ws_ok = '{ "settings": { "editor.rulers": [100] }, "note": "words live in cspell.json" }' @@ -1050,6 +1173,49 @@ def _selftest(): else: print(f" ok ground branch: registry={entry.get('groundTruthBranch')} override={branch} -> {want}") + # The driftNote freshness rules, driven directly rather than through audit_repo. + # That exercises both the clean and the unclean audit without standing up a whole conformant repo to reach one branch. + note_spec = {"types": {"types": {"hugo": {"checks": [{"id": "hugo.generator.pinned"}]}, + "docker": {"checks": [{"id": "docker.cache.registry"}]}}, + "crossCutting": {"setup": {"checks": [{"id": "setup.driftnotes.current"}]}}}} + note_cases = [ + # The end-of-sentence period is the case an end-anchored pattern misses, and it is how both notes this form was introduced for are written. + ("check id followed by a period", ["hugo"], ["Pinned in two workflows (hugo.generator.pinned)."], 0, ["does not evaluate"]), + ("check id mid-sentence", ["hugo"], ["The pin (hugo.generator.pinned) is duplicated."], 0, ["does not evaluate"]), + ("cross-cutting id, which no repo declares", ["hugo"], ["Notes go stale (setup.driftnotes.current)."], 0, ["does not evaluate"]), + ("check id of an undeclared type", ["hugo"], ["Cache layer (docker.cache.registry)."], 0, ["this repo does not declare"]), + ("check id absent from the catalog", ["hugo"], ["Theme record (hugo.vendored.provenence)."], 0, ["does not define"]), + # A version string in parentheses is the false positive the id shape has to exclude. + ("parenthesized version, not a check id", ["hugo"], ["Generator held at (0.164.0) by the composite action."], 0, []), + ("permanent deviation, no marker and no id", ["hugo"], ["Relies on validate-task, having no get-version-task."], 0, []), + ("marker note on a clean audit", ["hugo"], ["The sibling doc is pending fleet ratification."], 0, ["but the audit is clean"]), + # The case the old gate suppressed outright: one unclearable finding, and the note never checked. + ("marker note with findings open", ["hugo"], ["The sibling doc is pending fleet ratification."], 1, ["while 1 finding(s) are open"]), + ] + for label, types, notes, open_count, wanted in note_cases: + got = driftnote_findings({"types": types, "driftNotes": notes}, note_spec, open_count) + texts = [t for _, t in got] + if len(texts) != len(wanted) or not all(w in t for w, t in zip(wanted, texts)): + ok = False + print(f" FAIL driftNote {label} -> {texts}, want {wanted}") + else: + print(f" ok driftNote {label}: {len(texts)} finding(s)") + + # A spec with no catalog and an entry with a check-id note fails loudly and names what is missing. + # No live caller pairs the two, since main() always loads the catalog. + # The pairing stays an error rather than becoming a fallback, because resolving ids against an absent catalog reports every one of them undefined. + # That is a work list which destroys correct notes. + try: + driftnote_findings({"types": ["hugo"], "driftNotes": ["A note (hugo.build.strict)."]}, {"registry": {}}, 0) + ok = False + print(" FAIL missing catalog: no error raised") + except KeyError as e: + if "project-types.json" not in str(e): + ok = False + print(f" FAIL missing catalog: error does not name the file -> {e}") + else: + print(" ok missing catalog: raises and names spec/project-types.json") + # A ground-truth branch that does not resolve is one error, not a baseline's worth of letters. # Every `?ref=` read would 404 and report each carried file absent, describing the ref, not the repo. # The branch facts are already read at that point, so they are reported rather than dropped. @@ -1094,7 +1260,8 @@ def render_issue(entry, findings, ground, audited_sha, run_utc, hub_sha): w(f"Generated from the hub audit of `{name}` ({types}). Run stamp `audit run {run_utc} | hub {hub_sha}`, " f"against `@ {stamp}` (the format AUDIT.md section 8 says a derived artifact quotes). Regenerate with " f"`spec/audit.py --issue {name}`. Findings are a point-in-time snapshot - re-run the audit before acting. " - f"This lists what the audit mechanically detects. The full letter and intent verdict lives in AUDIT.md.") + f"This lists what the audit mechanically detects. No check belonging to a project type in `spec/project-types.json` is run " + f"here, and the cross-cutting dimensions are covered only in part, so the full letter and intent verdict lives in AUDIT.md section 4.") w("") if not findings: w("The deterministic checks are clean - nothing to converge.") @@ -1149,6 +1316,7 @@ def main(argv=None): "settings": load("repo-config/settings.json"), "secrets": load("spec/secrets.json"), "files": load("spec/files.json"), + "types": load("spec/project-types.json"), } issue_mode = a.issue wanted = {n.lower() for n in a.names} @@ -1199,7 +1367,7 @@ def main(argv=None): stamp = f" @ {ground}@{audited_sha[:7]}" if audited_sha else "" print(f"== {entry['name']} ({', '.join(entry.get('types', []))}; {model}){stamp} ==") if not findings: - print(" clean (deterministic checks; the full letter+intent verdict is AUDIT.md's)") + print(" clean (deterministic checks only; no project-type check in spec/project-types.json runs here, and the cross-cutting ones are covered only in part - AUDIT.md section 4)") for kind, text in findings: print(f" {kind:6} {text}") if kind in ("DEFECT", "LETTER", "ERROR"): diff --git a/spec/fidelity-model.md b/spec/fidelity-model.md index 8af3786c..c76acb49 100644 --- a/spec/fidelity-model.md +++ b/spec/fidelity-model.md @@ -10,9 +10,9 @@ Carried content is a class with virtual functions. The **fixed** part is the int Each [`spec/files.json`][files] entry declares one `fidelity`, defaulting to `presence`. -- **presence** - the unit exists (a file, or a markdown section heading). The audit's baseline check. +- **presence** - the unit exists (a file, or a Markdown section heading). The audit's baseline check. - **intent** - carried faithfully but judged by meaning, not bytes. A downstream copy legitimately differs (a governed divergence or a paraphrase), and equivalence is a human call via `intentRef`. The audit asserts nothing beyond presence. -- **verbatim** - byte-identical to the hub's canonical after line-ending, action-pin, and job-needs normalization. The audit content-hashes the downstream copy against canonical. It applies to a whole file, a workflow job region (a job selected by key), or a markdown section region (a `## heading` block selected by name). The section granularity lets one file be **intent overall while a few of its sections are verbatim** - a universal rule block stays byte-identical fleet-wide even though the rest of the document is a repo-adapted paraphrase, so a stale section or a missing rule is caught while its heading still passes the presence check. +- **verbatim** - byte-identical to the hub's canonical after line-ending, action-pin, and job-needs normalization. The audit content-hashes the downstream copy against canonical. It applies to a whole file, a workflow job region (a job selected by key), or a Markdown section region (a `## heading` block selected by name). The section granularity lets one file be **intent overall while a few of its sections are verbatim** - a universal rule block stays byte-identical fleet-wide even though the rest of the document is a repo-adapted paraphrase, so a stale section or a missing rule is caught while its heading still passes the presence check. - **interface** - an overridable body that must honor a named contract. The audit checks the contract by name and wiring, never the body. Fidelity is a declared field defaulting to `presence`, never inferred from `whole`/`placeholders`. `.editorconfig` and `.markdownlint-cli2.jsonc` are both whole with no placeholders yet sit at opposite fidelity, because the discriminator is governance, not field shape. diff --git a/spec/fidelity_honesty.py b/spec/fidelity_honesty.py index 8cd21af7..d880ae1c 100644 --- a/spec/fidelity_honesty.py +++ b/spec/fidelity_honesty.py @@ -179,7 +179,7 @@ def _fmt(repos): def render_report(spreads, promote, gaps, ledger): - """Join the live passes against the curated ledger into the checked-in burn-down markdown. + """Join the live passes against the curated ledger into the checked-in burn-down Markdown. A recorded disposition still matching a live divergence is a burn-down row. A live divergence (a verbatim hand-modification, or a manifest gap) with no disposition reads UNTRIAGED. A disposition is diff --git a/spec/files.json b/spec/files.json index ee4db8bb..3be522ab 100644 --- a/spec/files.json +++ b/spec/files.json @@ -30,6 +30,8 @@ { "path": ".github/workflows/test-pull-request.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["check-workflow-status"], "requiredCheckName": "Check pull request workflow status job" }, "intentRef": "GOVERNANCE.md#workflow-yaml-conventions", "appliesTo": "*" }, { "path": ".github/workflows/merge-bot-pull-request.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["merge-dependabot", "disable-auto-merge-on-maintainer-push"] }, "intentRef": "WORKFLOW.md#d8---bots--automation", "appliesTo": "*" }, { "path": ".github/workflows/build-release-task.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["get-version", "validate-release", "github-release"], "artifactNameToken": "release-asset-", "requireTokensInJob": { "github-release": ["pattern:", "merge-multiple:"] }, "forbidTokensInJob": { "github-release": ["artifact-ids:"] }, "verbatimJobs": ["github-release"] }, "reference": "catalog/snippets/workflows/build-release-task.yml", "intentRef": "GOVERNANCE.md#release-model", "appliesTo": ["csharp", "console", "docker", "nuget", "pypi", "eda"] }, + { "path": ".github/workflows/deploy-site.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["assert-ref", "validate", "deploy"], "requireTokensInJob": { "deploy": ["deploy-site-task.yml", "secrets: inherit"] } }, "reference": "catalog/snippets/workflows/deploy-site.yml", "intentRef": "WORKFLOW.md#d4---release--publish", "appliesTo": ["hugo"] }, + { "path": ".github/workflows/deploy-site-task.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["assert-environment", "deploy"], "requireTokensInJob": { "deploy": ["environment:"] } }, "reference": "catalog/snippets/workflows/deploy-site-task.yml", "intentRef": "WORKFLOW.md#d4---release--publish", "appliesTo": ["hugo"] }, { "path": ".vscode/tasks.json", "sections": ["clean-compile task group"], "reference": "catalog/snippets/configs/vscode-tasks.json", "appliesTo": ["csharp"] }, { "path": ".vscode/tasks.json", "sections": ["clean-compile task group"], "reference": "catalog/snippets/configs/vscode-tasks-python.json", "appliesTo": ["python"] }, { "path": "codecov.yml", "fidelity": "intent", "reference": "catalog/snippets/configs/codecov.yml", "intentRef": "WORKFLOW.md", "appliesTo": ["csharp", "python"] }, diff --git a/spec/project-types.json b/spec/project-types.json index 62703949..652e4aaa 100644 --- a/spec/project-types.json +++ b/spec/project-types.json @@ -107,6 +107,22 @@ { "id": "sourceonly.nbgv.retained", "verdict": "letter", "assert": "version.json is retained and NBGV is inlined in publish-release.yml to compute the tag.", "workflowRef": "WORKFLOW.md#d3---versioning-and-classification" } ] }, + "hugo": { + "detect": ["hugo.yaml", "hugo.toml", "config/_default/hugo.yaml"], + "intentRefs": ["WORKFLOW.md"], + "note": "Named for the generator rather than for the transport, because what a repo builds and where the result lands are separate axes. The destination is publish[] ({ target, mechanism }), so a repo changes transport without changing type. Every assert below is phrased without naming the generator except hugo.build.strict, where a generator-specific flag is the letter, so promoting the generic ones to a shared type when a second generator arrives is a registry edit. Deploy credentials are per-environment GitHub Environment secrets and variables, which spec/secrets.json cannot yet express, so a repo does not list them in its registry requiredSecrets: spec/audit.py resolves that list against the repository actions store and would report an environment-scoped name as missing.", + "checks": [ + { "id": "hugo.build.strict", "verdict": "letter", "assert": "The site build fails on a generator warning rather than rendering around it (hugo --gc --minify --panicOnWarning), and the pull request gate and the deploy run the same build command rather than two variants.", "workflowRef": "WORKFLOW.md#d1---pr-fast-feedback-smoke" }, + { "id": "hugo.urls.parity", "verdict": "letter", "assert": "A URL contract gate compares the built tree against a committed list of the URLs that must render and the URLs that must redirect, and asserts a minimum length on each list before comparing it, since a truncated list makes every assertion below it pass vacuously. This is the type's check of record, standing in for the unit tests a site does not have.", "workflowRef": "WORKFLOW.md#6-per-project-type-test-walkthroughs" }, + { "id": "hugo.output.uncommitted", "verdict": "letter", "assert": "The rendered output is produced in CI only: its roots are gitignored and untracked, and they are excluded from the prose, spelling, and Markdown gates along with any vendored third-party tree. A committed render is drift rather than a deliverable.", "intentRef": "GOVERNANCE.md#documentation-style-conventions" }, + { "id": "hugo.generator.pinned", "verdict": "letter", "assert": "The generator is pinned by exact version and by a checksum of the downloaded artifact, verified before install, never installed from a floating action or a latest tag, since the site is reproducible only if the generator is. The pin is declared once, and where two workflows need it something asserts the two copies agree.", "workflowRef": "WORKFLOW.md#d9---style--static-see-section-2" }, + { "id": "hugo.vendored.provenance", "verdict": "letter", "assert": "A vendored third-party tree records the upstream repository and the exact ref or commit it was taken from, or is carried by a mechanism that pins it, so a bot or a tracker can move it. An unpinned copy with no recorded origin cannot be updated, diffed against upstream, or audited for a security fix.", "intentRef": "GOVERNANCE.md#release-model" }, + { "id": "hugo.deploy.environment", "verdict": "letter", "assert": "The deploy job binds a GitHub Environment and takes every host-specific value and its credential from that environment, so the workflow file names no host, path, or address. A reusable callee re-asserts the environment name in a job of its own, because the environment binding resolves before any step runs and a workflow_call caller is not bound by the dispatch choice list a human sees.", "workflowRef": "WORKFLOW.md#d7---concurrency-permissions-safety" }, + { "id": "hugo.deploy.atomic", "verdict": "intent", "assert": "A release installs beside the retained ones under its own immutable id, and is published by moving a single pointer through a temporary and a rename, so no request observes a half-written site and the previous release stays on disk as a rollback target. The transport never deletes at the environment root.", "workflowRef": "WORKFLOW.md#d4---release--publish" }, + { "id": "hugo.deploy.verified", "verdict": "letter", "assert": "The deploy's terminal step observes the running host rather than the transport's exit status. It asserts which release is answering, by comparing a version the artifact stamps into the configuration it ships against the id just installed, polling to a bounded timeout because the reload is asynchronous, and it asserts which environment answered. An unreachable host is reported distinctly from an HTTP status.", "workflowRef": "WORKFLOW.md#d4---release--publish" }, + { "id": "hugo.deploy.retention", "verdict": "letter", "assert": "Retention at the destination is bounded by a declared count, and the side that owns the prune is written down rather than assumed. Where the deploy credential can observe the destination, the deploy asserts the count converged and fails when it does not. Where the credential is deliberately write-only and cannot observe it, the prune belongs to the host and that ownership is recorded, since widening the credential to reach it would trade a real boundary for a check. The release the live pointer resolves to is never a prune candidate. A prune that runs against a local scratch tree, or that is best-effort, or that no side is recorded as owning, satisfies none of this.", "workflowRef": "WORKFLOW.md#d5---resource-cleanup" } + ] + }, "docs": { "detect": ["governance-only repo"], "checks": [ @@ -155,7 +171,7 @@ { "id": "recurring.charset", "verdict": "letter", "assert": "ASCII only in agent-authored text: no em-dash (use a spaced hyphen), no smart quotes, no stray non-ASCII.", "intentRef": "GOVERNANCE.md#character-set" }, { "id": "recurring.spelling", "verdict": "letter", "assert": "US English spelling; the shared cspell.json sets language en-US (a bare en accepts British spellings too).", "intentRef": "CODESTYLE.md#markdown-and-spelling" }, { "id": "recurring.eol", "verdict": "letter", "assert": "Line endings follow .editorconfig, which carries a global [*] end_of_line default plus LF pins for execution-sensitive files (shell, Dockerfiles, shebang-executable .py by path), plus workflow YAML in .github/workflows/* enforced by editorconfig-checker in CI. A per-extension-only form lacking the global [*] default is a drift finding. The global default is CRLF for release repos, or the consuming application's native platform for an operational (config) repo as recorded in the registry lineEndings field (LF for Linux-native/container config e.g. ESPHome/Home Assistant, CRLF for a Windows-native editor e.g. Vantage/Design Center) - do not re-normalize such a repo to CRLF. Edits preserve the file's endings.", "intentRef": "GOVERNANCE.md#line-endings" }, - { "id": "recurring.norepoxref", "verdict": "intent", "assert": "A carried file (AGENTS.md, GOVERNANCE.md, CODESTYLE.md, WORKFLOW.md, .github/copilot-instructions.md, repo-config/README.md, repo-config/develop.json, repo-config/main.json, spec/secrets.json, the carried AUDIT.md) carries no coordination reference: no reference to the template repo in prose or link (the coordination flow is machinery a consumer should not see - state the behavior, not the destination), and no sibling fleet repo named as an illustrative example of a rule or adoption. A contextually relevant link to a related project is NOT a coordination reference and is expected (the image that consumes this config, a library this depends on) - the test is whether the link serves a reader of this repo's content. The rule governs carried template content. A repo's own README.md and topical docs are its own content. spec/audit.py mechanically checks AGENTS.md, GOVERNANCE.md and .github/copilot-instructions.md for the template name, and skips the hub itself, whose copies are the source.", "intentRef": "GOVERNANCE.md#documentation-style-conventions" } + { "id": "recurring.norepoxref", "verdict": "intent", "assert": "A carried file (AGENTS.md, GOVERNANCE.md, CODESTYLE.md, WORKFLOW.md, .github/copilot-instructions.md, repo-config/README.md, repo-config/develop.json, repo-config/main.json, spec/secrets.json, the carried AUDIT.md) carries no coordination reference: no reference to the template repo in prose or link (the coordination flow is machinery a consumer should not see - state the behavior, not the destination), and no sibling fleet repo named as an illustrative example of a rule or adoption. A contextually relevant link to a related project is NOT a coordination reference and is expected (the image that consumes this config, a library this depends on) - the test is whether the link serves a reader of this repo's content. The rule governs carried template content. A repo's own README.md and topical docs are its own content. A verbatim section is the one exception, because AGENTS.md 'Fleet Bootstrap' must name the hub to do its job and its bytes are fixed fleet-wide, so a repo cannot clear such a finding except by failing the verbatim check instead. The exception stops at the region boundary, and the same file's own prose is judged normally. spec/audit.py mechanically checks AGENTS.md, GOVERNANCE.md and .github/copilot-instructions.md for the template name outside their verbatim sections, and skips the hub itself, whose copies are the source.", "intentRef": "GOVERNANCE.md#documentation-style-conventions" } ] }, "readme-structure": { diff --git a/spec/readme-structure.md b/spec/readme-structure.md index 3cc7f214..43ad9ea4 100644 --- a/spec/readme-structure.md +++ b/spec/readme-structure.md @@ -4,7 +4,7 @@ The preferred `README.md` shape for a fleet project. The audit's `readme-structu ## Sections and Order -1. **Title (`# <Name>`)** - the H1 **is the repository name** (a hyphenated name may render its hyphens as spaces: `Financial-Modeling` -> `Financial Modeling`), then a one-line description as the next paragraph. That description is a **single sentence, link-free plain text, at most 100 characters**, and it is the one canonical short description. It doubles as the GitHub About description (GOVERNANCE.md "Repository Details") and, for a repo that publishes a Docker image, the Docker Hub short description. Both render no markdown, and Docker Hub caps the short description near 100 characters, the tightest surface, which sets the limit. The audit checks the H1 name, the length, the link-free form, and the mirrors. +1. **Title (`# <Name>`)** - the H1 **is the repository name** (a hyphenated name may render its hyphens as spaces: `Financial-Modeling` -> `Financial Modeling`), then a one-line description as the next paragraph. That description is a **single sentence, link-free plain text, at most 100 characters**, and it is the one canonical short description. It doubles as the GitHub About description (GOVERNANCE.md "Repository Details") and, for a repo that publishes a Docker image, the Docker Hub short description. Both render no Markdown, and Docker Hub caps the short description near 100 characters, the tightest surface, which sets the limit. The audit checks the H1 name, the length, the link-free form, and the mirrors. 2. **Build and Distribution (`##`)** - a bullet per distribution channel the project actually ships, each linking where it lives: **Source Code** (the GitHub repo), **Versioned Releases** (GitHub Releases), **Docker Images** (Docker Hub), **NuGet Packages** (NuGet.org), **PyPI Packages** (PyPI.org). List only the channels the project uses. It carries three sub-sections: - **Build Status (`###`)** - the CI/build status shields (release build, Docker build, last commit, last build). - **Releases (`###`)** - the version shields (GitHub release, GitHub pre-release, Docker latest/develop, NuGet, PyPI), one per channel the project publishes. diff --git a/spec/scope-model.md b/spec/scope-model.md index a0eafe05..3747fcd7 100644 --- a/spec/scope-model.md +++ b/spec/scope-model.md @@ -15,7 +15,7 @@ A selector is one token from one of four **disjoint** namespaces. Because the na | Namespace | Tokens | Source of truth | | --- | --- | --- | -| project type | `csharp` `nuget` `pypi` `python` `console` `docker` `homeassistant` `eda` `codegen` `upstream-wrapper` `source-only` `docs` | [`spec/project-types.json`][project-types] | +| project type | `csharp` `nuget` `pypi` `python` `cpp` `console` `docker` `homeassistant` `eda` `codegen` `upstream-wrapper` `source-only` `hugo` `docs` | [`spec/project-types.json`][project-types] | | workflow model | `release` `operational` | [`registry/repos.schema.json`][repos-schema] | | release trigger | `two-phase` `publish-on-merge` `dispatch-only` `none` | [`registry/repos.schema.json`][repos-schema] | | consumer model | `push` `pull` | [`registry/repos.schema.json`][repos-schema] | diff --git a/spec/secrets.json b/spec/secrets.json index d5534a09..722b156c 100644 --- a/spec/secrets.json +++ b/spec/secrets.json @@ -1,6 +1,6 @@ { "$schema": "./secrets.schema.json", - "note": "Secrets the audit cross-checks. `baseline` applies to every fleet repo (the App-signed merge-bot runs everywhere). `mechanisms` are per-target/per-feature additions: a repo requires the baseline plus the mechanisms whose publish target (`targetMechanisms`) or feature workflow (`featureMechanisms`, detected by the mechanism's `workflowNeeds` action) it uses. Baseline secrets are implicit and are NOT repeated in a repo's registry `requiredSecrets`, which lists only the domain-specific additions. `typeMechanisms` are per-language requirements: a `csharp` or `python` repo must carry the mapped mechanism (codecov) regardless of opt-in. A configured secret that no applicable mechanism claims is a stale-secret finding; a present `forbids` secret is a defect.", + "note": "Secrets the audit cross-checks. `baseline` applies to every fleet repo (the App-signed merge-bot runs everywhere). `mechanisms` are per-target/per-feature additions: a repo requires the baseline plus the mechanisms its declared publish target (`targetMechanisms`) or declared type (`typeMechanisms`) maps to. All three mappings resolve from the registry entry rather than from workflow content: nothing reads a repo's Actions files to infer a mechanism, and `workflowNeeds` records what a mechanism needs to appear in a workflow for a human or agent reading the audit, rather than being a detector. `featureMechanisms` is shape-validated but claims nothing today, since the one feature it names (codecov) is claimed through `typeMechanisms` at build profile instead. Baseline secrets are implicit and are NOT repeated in a repo's registry `requiredSecrets`, which lists only the domain-specific additions. `typeMechanisms` are per-language requirements: a `csharp` or `python` repo must carry the mapped mechanism (codecov) regardless of opt-in. A configured secret that no applicable mechanism claims is a stale-secret finding; a present `forbids` secret is a defect. `environments`, where a repo carries it, lists the per-environment GitHub Environment secrets and variables its deploy needs. It is operator documentation rather than part of the mechanism audit: no tool reads it, because neither `spec/validate.py` nor `spec/audit.py` can enumerate an environment-scoped store, so a clean audit is not evidence that an environment is configured. `environmentSecrets` names what one environment carries and another does not, so a name audit does not read a single-environment credential as missing everywhere else.", "baseline": { "requires": ["CODEGEN_APP_CLIENT_ID", "CODEGEN_APP_PRIVATE_KEY"], "forbids": ["CODEGEN_APP_ID"], @@ -37,13 +37,22 @@ "workflowNeeds": ["codecov/codecov-action"], "stores": ["actions", "dependabot"], "note": "A csharp or python repo lists CODECOV_TOKEN in its registry requiredSecrets (unlike the implicit baseline secrets). Both stores: a workflow run triggered by a Dependabot PR reads the Dependabot secrets store, so without that copy the coverage upload silently skips on every bot PR. Coverage upload is report-only by default (fail_ci_if_error: false, so a Codecov hiccup never fails the gate). A repo may enforce a stricter threshold (e.g. 99%+) on top of that default." + }, + "deploy-ssh": { + "kind": "static-secret", + "requires": [], + "forbids": [], + "workflowNeeds": ["environment:", "IdentitiesOnly=yes"], + "stores": [], + "note": "A deploy to a filesystem on a host the project owns, reached over SSH. requires and stores are empty deliberately rather than for want of credentials: the key and the host values are per-environment GitHub Environment secrets and variables, which this file has no vocabulary for and neither validate.py nor audit.py can enumerate. Listing the names would force them into the repo's registry requiredSecrets, which the audit resolves against the repository actions store, so a correctly configured repo would report every one of them as missing. A repo declares them in its own environments block below instead. The key is confined at the far end by an authorized_keys forced command rooted at the deploy tree, so the workflow names no host path." } }, "targetMechanisms": { "nuget": "nuget-oidc", "pypi": "pypi-oidc", "docker": "docker-hub", - "github-release": null + "github-release": null, + "self-hosted": "deploy-ssh" }, "featureMechanisms": { "codecov": "codecov" diff --git a/spec/secrets.schema.json b/spec/secrets.schema.json index dfebcf34..d3f6451d 100644 --- a/spec/secrets.schema.json +++ b/spec/secrets.schema.json @@ -44,6 +44,22 @@ "typeMechanisms": { "type": "object", "additionalProperties": { "type": ["string", "null"] } + }, + "environments": { + "type": "object", + "additionalProperties": false, + "properties": { + "note": { "type": "string" }, + "names": { "type": "array", "items": { "type": "string" } }, + "secrets": { "type": "array", "items": { "type": "string" } }, + "variables": { "type": "array", "items": { "type": "string" } }, + "secretsNote": { "type": "string" }, + "variablesNote": { "type": "string" }, + "environmentSecrets": { + "type": "object", + "additionalProperties": { "type": "array", "items": { "type": "string" } } + } + } } } } diff --git a/spec/section-model.md b/spec/section-model.md index 63615f5f..e416df8b 100644 --- a/spec/section-model.md +++ b/spec/section-model.md @@ -76,7 +76,7 @@ A repo's own content is not carried, and the hub declares where it goes rather t `CODESTYLE.md` is carried by every repo already. `ARCHITECTURE.md` stays **declared but advisory**: every code repo has an architecture, and how much of it earns a separate document is contextual, so mandating it would produce empty files where the design needs none. Declared and required are separate questions, and only a universal need answers both. -`OPERATIONS.md` is agent-instruction content, so it takes the inline-link exception the markdown rules name, not the reference-style default. `ARCHITECTURE.md` is not on that closed list and follows the reference-style rule. +`OPERATIONS.md` is agent-instruction content, so it takes the inline-link exception the Markdown rules name, not the reference-style default. `ARCHITECTURE.md` is not on that closed list and follows the reference-style rule. ## Migrating a repo onto the split diff --git a/spec/type-model.md b/spec/type-model.md index d5214525..7d4db591 100644 --- a/spec/type-model.md +++ b/spec/type-model.md @@ -48,6 +48,14 @@ Language types carry the style and type-check requirements for their language, g - **cpp** - C/C++ present for style only. The check of record is **clang-format** (a shared config driving the editor, the CLI, and CI, a `parity.lang` arm), feeding the operational lint CI. Deeper semantic and static analysis is intentionally out of scope here - for a codegen or config repo the C++ is scaffolded and completed by its downstream toolchain (an ESPHome compile), which does the compilation-time checking, and clang-tidy would need a compile database the repo does not have. A repo's `.h` is read as C++ by context (Arduino/ESPHome), since the extension alone is ambiguous. +## Generators + +A **generator** type is what a repo builds its deliverable *with*, where the deliverable is not code: a static-site generator, a documentation builder. It is named for the generator (`hugo`) rather than for the transport that ships the result, because what a repo builds and where the result lands are separate axes. The destination lives in the registry `publish[]` entry (`{ target, mechanism }`), so a repo changes transport without changing type, and a second transport is a new **mechanism** rather than a new type. Baking the transport into the type is what makes the set explode combinatorially: one generator over two transports would otherwise need two types. + +There is no `static-site` to `hugo` hierarchy while the type has one member. Instead each check's `assert` is phrased without naming the generator wherever the requirement generalizes (the URL-contract gate and its length floor, the rendered output never committed, the generator pinned by version and digest, vendored-dependency provenance), and names it only where a generator-specific construct *is* the letter, such as a build flag. When a second generator joins the fleet, promoting the generic checks to a shared type is then a registry edit rather than a rewrite, which is the property the phrasing rule exists to preserve. Paying for that abstraction at one member would be the more expensive mistake. + +A generator type declares no `profiles`. Build versus lint-only is a depth of *language* presence, so a profile on a generator type would assert nothing, and `spec/validate.py` rejects a declared profile whose type does not define one. + ## Changing the type set carries review weight The set of types, their profiles, and each type's checks is governed, like the section and fidelity models. diff --git a/spec/validate.py b/spec/validate.py index 2afa7971..5495f08b 100644 --- a/spec/validate.py +++ b/spec/validate.py @@ -302,13 +302,13 @@ def check_selector(where, applies_to): errors.append(f"files.json: {path} section object missing a non-empty string 'name': {elt!r}") check_selector(f"{path} section '{elt.get('name', '?')}'", elt.get("appliesTo", "*")) # A section may carry its own fidelity (intent default, or verbatim for a universal rule block - # checked byte-for-byte). verbatim is meaningful only on a markdown file, where the heading + # checked byte-for-byte). verbatim is meaningful only on a Markdown file, where the heading # delimits the region. The hub's own file is the canonical, so no reference is needed. sfid = elt.get("fidelity", "intent") if sfid not in ("intent", "verbatim"): errors.append(f"files.json: {path} section '{elt.get('name', '?')}' fidelity '{sfid}' invalid (expected intent or verbatim)") elif sfid == "verbatim" and not path.endswith(".md"): - errors.append(f"files.json: {path} section '{elt.get('name', '?')}' is verbatim but {path} is not markdown (heading regions apply to .md only)") + errors.append(f"files.json: {path} section '{elt.get('name', '?')}' is verbatim but {path} is not Markdown (heading regions apply to .md only)") elif not isinstance(elt, str): errors.append(f"files.json: {path} section entry {elt!r} must be a string or object")