Skip to content

skills(checklist-test): the documented QA-run title convention matches 0 of 23 real records #9617

Description

@claude

Found while building the qa-run roll-up (#9486 / PR #9616), which had to parse these titles for real.

The drift

.claude/skills/checklist-test/SKILL.md:126 documents the run-record title as five ·-separated fields with a standalone judged/total:

QA run · SELECTOR · JUDGED/TOTAL · FRAMEWORK-SHA8 · DATE

No record has ever been written that way. Measured over the complete live corpus on 2026-08-18 — all 23 qa-run issues, open and closed:

  • 23 / 23 use five fields in which the judged/total is folded into a parenthetical on the selector, and a counts field the documented convention does not mention at all:
QA run · SELECTOR (JUDGED/TOTAL) · FRAMEWORK-SHA8 · DATE · COUNTS

The card for #9486 described a third shape (six fields, standalone judged/total and counts). Zero records follow that one either.

Why it matters

The documented line is the only written spec for a field that is now parsed by tooling. PR #9616 reads these titles to produce the QA roll-up, and had to be written against the measured shape rather than the documented one. The next person to write against SKILL.md will produce a parser that silently matches nothing — or, worse, one that mis-assigns fields by position and reports confident nonsense.

(FULL area) is also doing real damage as a phrasing: it declares no total, and at least one record titled (FULL area) judged 13 of its area's 33 items (#9351, records-forms). The roll-up now renders that as 13/33* by reading the item count from docs/qa/platform-checklist/areas/, but the title alone reads as complete coverage.

Suggested resolution — needs a maintainer, not a dev

.claude/skills/** is a governed, human-merge-only surface, so this is filed rather than fixed.

Two directions, and they are not equivalent:

  1. Update SKILL.md to document what runs actually write. Cheap, immediate, keeps every existing record valid. Leaves five phrasings of the same field as the permanent contract.
  2. Tighten the convention and have runs emit one shape. A parser could then be strict rather than tolerant, which is the contract-first direction. Costs a migration decision for the 23 existing records — most cleanly, "new records only", since the roll-up already reports what it cannot parse.

Recommendation: 2, with the roll-up's "Not parsed" section as the enforcement surface — it already prints any record it cannot read, so a tightened convention becomes visible the first time someone deviates instead of silently degrading. 1 alone leaves the tolerant-consumer shape that hides authoring errors.

Filed unassigned for triage.


Generated by Claude Code

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions