Add a checked-in divergence ledger and generated burn-down report by ptr727 · Pull Request #390 · ptr727/ProjectTemplate

ptr727 · 2026-07-22T18:53:01Z

Makes the fidelity-honesty tool's findings durable and systematically tracked, instead of re-analyzed every run with dispositions living only in memory.

What

spec/divergences.json (+ divergences.schema.json) — the curated disposition ledger. Each known divergence carries a disposition (re-vendor | track | accepted | upstream-candidate | investigate), a reason, and an optional tracking link.
spec/fidelity_honesty.py --report — joins the live fleet passes against the ledger → reports/divergences.md. Recorded divergence still live → burn-down row; live divergence with no disposition → ⚠ UNTRIAGED; recorded disposition no longer live → resolved. Verbatim stale copies are listed separately as the mechanical re-vendor set (the audit already flags them).
spec/validate.py — validates the ledger (paths are manifest units, repos exist in the registry, dispositions are in the vocabulary, gaps aren't already tracked) so a typo can't silently drop or misfile a row.
reports/divergences.md — the generated, checked-in report, beside conformance-matrix.md. Regenerate anytime; git dates it, no embedded timestamp (so regeneration diffs are clean).

Why it survives memory loss

Dispositions are checked-in structured state; the report is regenerated on demand; and new drift with no recorded disposition surfaces itself as UNTRIAGED. That's the "no re-analysis, no lost state" property.

Seeded dispositions (from the current fleet)

re-vendor: .markdownlint-cli2.jsonc — 3 hand-modified copies.
upstream-candidate: configure.sh / NxWitness — a forked design (repo-specialized + a check/5D-audit mode the hub canonical lacks); adopt the check mode into the hub, then re-vendor.
investigate: 5 other configure.sh forks; pyproject.toml, publish-release.yml, validate-task.yml manifest gaps (fidelity + appliesTo calls).
accepted: .editorconfig-checker.json's two outliers (both carry a legitimate repo-specific Exclude list — Vantage/ subtree, .fseq files — so the unit is correctly intent, not verbatim); LICENSE (repo-owned, not template-governed).

Verification

validate.py green (ledger checks active); audit.py --selftest passes (tool import still safe); report regenerates and markdownlint-clean.

🤖 Generated with Claude Code

The fidelity-honesty tool finds divergences on demand, but its output was ephemeral - re-run every time, and the dispositions (which divergences are drift to fix, which are accepted, which are upstream candidates) lived only in memory. This makes that state durable and systematically tracked. - spec/divergences.json (+ schema): the curated disposition ledger. Each known divergence carries a disposition - re-vendor, track, accepted, upstream-candidate, or investigate - a reason, and an optional tracking link. - spec/fidelity_honesty.py --report: joins the live fleet passes against the ledger and writes reports/divergences.md. A recorded divergence still present is a burn-down row; a live divergence with no disposition renders UNTRIAGED; a recorded disposition no longer live renders resolved. Verbatim stale copies are listed separately as the mechanical re-vendor set (the audit already flags them). - spec/validate.py: validates the ledger (paths are manifest units, repos exist in the registry, dispositions are in the vocabulary, gaps are not already tracked) so a typo cannot silently drop or misfile a burn-down row. - reports/divergences.md: the generated, checked-in report, beside conformance-matrix.md. Seeded from the current fleet: markdownlint (3 hand-modified), configure.sh (NxWitness is a forked check-mode design -> upstream-candidate; 5 others to triage), and the manifest gaps (LICENSE accepted; pyproject.toml/publish-release/validate-task pending). .editorconfig-checker.json's two outliers are accepted - both carry a legitimate repo-specific Exclude list, so the unit is correctly intent, not verbatim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds a durable, checked-in divergence disposition ledger and a generated Markdown burn-down report so fidelity-honesty findings are tracked over time rather than re-triaged on each run.

Changes:

Add spec/divergences.json (with schema) as a curated ledger of known divergences and manifest gaps, including dispositions and rationale.
Extend spec/fidelity_honesty.py with --report to generate a checked-in reports/divergences.md by joining live fleet state with the ledger.
Extend spec/validate.py to validate the divergence ledger when present.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
spec/validate.py	Adds validation for the divergence ledger entries against the manifest and registry.
spec/fidelity_honesty.py	Adds ledger loading and report generation logic to produce the checked-in burn-down report.
spec/divergences.schema.json	Introduces a JSON Schema for the divergence ledger structure.
spec/divergences.json	Adds the initial curated divergence dispositions and manifest-gap entries.
reports/divergences.md	Adds the generated divergence report output committed to the repo.

… and inline links - validate.py: isinstance-guard the disposition/gap path before the set-membership test, so a non-string (unhashable) path reports a clear error instead of crashing validation with TypeError. Trim the ledger comment to two terse lines. - fidelity_honesty.py: recast the docstrings, the generated report header and prose, and a comment to avoid clause-joining semicolons (repo prose convention). The report header now uses a reference-style link, and a tracking URL renders as plain text - reports/*.md use reference-style links only, never inline. - divergences.json: recast the note to sentences (no clause-joining semicolons). - reports/divergences.md: regenerated; markdownlint-clean, no inline links or semicolons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

spec/validate.py:299

The divergences.json validation assumes the loaded JSON is an object with list-valued 'dispositions'/'gaps'. If the file is malformed (e.g., top-level array, or dispositions is not an array), validate.py will raise (AttributeError/TypeError) and crash instead of reporting a clear validation error in the normal errors list.

 if (ROOT / "spec/divergences.json").exists():
div = load("spec/divergences.json")
repo_names = {r.get("name") for r in repos["repos"] if isinstance(r, dict)}
manifest_paths = {i.get("path") for i in baseline if isinstance(i, dict)}
for d in div.get("dispositions", []):

… document intent-skip - fidelity_honesty.py: main() called manifest_gap_pass() twice in non-report mode (once hoisted for the shared path, once in the stdout section). Drop the second call and reuse the hoisted result - one gh/tree round-trip instead of two. - Generated report: the [ledger] reference-link definition now sits at the bottom of the document, per the AGENTS.md Markdown convention (definitions at the end). - Document why untriaged auto-detection is verbatim-only: an intent unit is judged by meaning, so a byte diff is expected, not an anomaly. Flagging every intent divergence would bury the report under the normal spread. Intent divergences are tracked only when a human records a disposition, which still renders in the burn-down. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

If spec/divergences.json parsed to a non-object root (a list from a bad edit), the div.get() calls would raise AttributeError and crash validation. Report a clear error instead, matching how validate.py shape-checks the other spec files (CI runs no JSON-schema validation). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

- validate.py: type-check the dispositions and gaps containers before iterating, so a null or non-array value reports a clear error instead of raising TypeError. - fidelity_honesty.py: load_ledger() normalizes a non-object root and non-array dispositions/gaps to empty sections, so --report degrades gracefully instead of raising AttributeError. validate.py still reports the malformation loudly in CI. Verified with negative tests: a list root, and null dispositions/gaps, both degrade cleanly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

- fidelity_honesty.py: emit the report's header and mechanical-re-vendor paragraphs as a single line each, matching the Markdown convention of one logical paragraph per line (no hard-wrap). Condense the verbatim-only untriaged comment to two terse lines and drop a rule citation from a comment. - reports/divergences.md: regenerated; the wrapped paragraphs are now single lines, markdownlint-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

…resolved The burn-down join marked a disposition resolved whenever none of its repos appeared in differs/stale - which silently swept in unavailable copies (deleted, renamed, or too large to fetch inline), dropping a real burn-down item we simply cannot see, and omitted unavailable repos from the rendered row. Split the spread into three sets per unit (still-divergent, confirmed-match, unavailable). A disposition is resolved only when every recorded repo now matches the canonical; a repo that went unavailable keeps the disposition on the burn-down, annotated "unavailable, unverified". Verified with an offline unit test over all three buckets. The report is byte-identical for the current ledger (no disposition repo is unavailable today). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

- validate.py: a disposition/gap reason is validated as a non-empty string, not merely truthy, so a number or object no longer slips through against the ledger schema. - fidelity_honesty.py: load_ledger's docstring said an absent ledger makes "every live divergence" untriaged, but auto-detection is verbatim-only plus manifest gaps. Correct it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

…generator render_report used unchecked dict indexing (d["path"], d["repos"], g["path"]) that would crash with KeyError/TypeError if the ledger were hand-edited into a malformed-but-valid-JSON shape. Filter dispositions and gaps to well-formed entries at the top, so --report degrades cleanly (drops the bad entry) instead of crashing. validate.py still reports the malformation loudly in CI. Verified with an offline test mixing malformed and well-formed entries. The report is byte-identical for the current ledger (the filter keeps every valid entry). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

spec/fidelity_honesty.py:166

In render_report(), the "well-formed" ledger filter only checks that repos is a list, not that it is a non-empty list of strings. If a malformed-but-valid JSON ledger contains something like "repos": [[], {}] or "repos": [], the later covered.setdefault(...).update(d["repos"]) / set(d["repos"]) can still raise (unhashable elements) or silently treat the entry as resolved. Since this function is explicitly trying to degrade gracefully on malformed ledgers, tighten the filter to require a non-empty list of strings here.

 # Keep only well-formed entries so --report degrades cleanly on a hand-malformed ledger instead of
# raising KeyError/TypeError downstream. validate.py reports the malformation loudly in CI.
dispositions = [d for d in ledger.get("dispositions", [])
if isinstance(d, dict) and isinstance(d.get("path"), str)
and isinstance(d.get("repos"), list) and isinstance(d.get("disposition"), str)
and isinstance(d.get("reason"), str)]

…ence ledger (#391) Merge-commit promotion of #386-#390 + #392 (fidelity-honesty tool, Python Scripts-profile config, manifest hardening, divergence ledger + burn-down report, report fail-loud guard). Copilot-clean; a human merge, so no release fires.

CopilotAI review requested due to automatic review settings July 22, 2026 18:53

Copilotstarted reviewing on behalf of ptr727 July 22, 2026 18:53 View session