ci(2364): a PR whose title names a ticket must link it (backend#2364)

[LukasWodka]

Summary

Merging a fix never closed or advanced its ticket. The board automation is present, wired and green — and structurally unable to act, because the input it needs is never produced.

closingIssuesReferences was 0 on 7 of 7 sampled merged PRs: release-train#109, release-train#108, .github#304, .github#300, backend#2266, client#774, docs#131.

The house convention puts the ticket in the PR title (fix(2302): summary). GitHub creates a closing link only from a keyword in the body. A title reference is inert, so kanban-closure-router.yml fires, finds no linked issue, and correctly does nothing. Every kanban workflow reported success. They were right to.

This adds a closing-ref job that parses the real title and asserts the real link graph contains what it names.

Related

Closes tracebloc/backend#2364

What it checks

Exactly one thing: title-to-link agreement. If the title names a ticket, the PR's link graph must contain that ticket.

It deliberately does not require that a PR have a ticket. A title naming none is reported and passes — chore(global_model): delete dead TF FLOPS path (backend#2333) names no ticket and links backend#2288 correctly. Requiring a ticket per PR is a separate decision.

Derived, not restated (rule 1). Two live reads — the real title, the real closingIssuesReferences (the same graph the router consumes). No list of tickets, repos or authors anywhere in the file.

The four title forms are measured, not imagined:

form	measured on
fix(2218): — bare number in the scope	client-runtime#365 → tracebloc/backend#2218
fix(#349):	client-runtime#351 → client-runtime#349
feat(kanban): … (backend#2348) — repo without owner	.github#309 → tracebloc/backend#2348
fix(hop): … (release-train#110) — same repo	release-train#112

A bare #Noutside parentheses is deliberately not read as a ticket. backend#2309 is titled test(platform): pin the three early-close call sites #2271 rewrote — #2271 is prose about a PR, and its real closing ref is backend#2308. Scanning loose #N would have reddened a compliant PR. Precision over recall.

A bare number resolves against no repo: fix(2218) in client-runtime means backend#2218, fix(90) in release-train means release-train#90. Both are real and measured, so inventing a repo for a bare number would be this file holding a mapping of its own. A bare number is satisfied by that number in any repo; nothing stronger is claimed.

The cross-repo trap, as its own verdict

A bare Closes #2302 inside release-train resolves against release-train, not backend. Two outcomes, reported separately because the remedies differ:

• MISSING — no such local issue, nothing links at all. A line has to be added.
• WRONG_REPO — a local issue with that number exists, so the PR links and on merge closes the wrong issue. .github is where this bites: its own issues are in the 300s and it references backend tickets in the 300s and 2000s alike, so Closes #304 in .github links .github#304. A line has to be rewritten to the full owner/repo#num form.

Fails closed — "cannot tell" is its own reported state (rule 3)

Each exits 2. Never a pass, and never counted as a finding against the author, because these are malfunctions of the check:

• an absent or blank title
• the GraphQL read failing, returning errors[], or pullRequest: null
• totalCount > len(nodes)
• a node with no repository.nameWithOwner or no integer number
• the query no longer asking for totalCount (which would make the test above inert) — checked before any read

The truncation test is load-bearing for a reason beyond pagination: a link to an issue the token cannot read comes back missing from nodes while totalCount still counts it. That is indistinguishable from "not linked" and would redden every compliant cross-repo PR. Hence the org-scoped App token, and hence this refusal if the read is ever partial — it blames the check, not the author.

SOFT_FAIL governs findings only; every malfunction above exits 2 either way. Same split action-pins-soft-fail keeps in code-quality.yml.

Fixtures are the measured shape (the backend#2114 lesson)

#2114 encoded login.endswith("[bot]") while the real API returns {"is_bot": true, "login": "app/dependabot"} — the test passed while the code could not fire. Every fixture here was captured with gh api graphql and re-verified against the live API before this commit:

{"number":365,"title":"fix(2218): three age checks compared local wall-clock against UTC",
"isDraft":false,"closingIssuesReferences":{"totalCount":1,
"nodes":[{"number":2218,"repository":{"nameWithOwner":"tracebloc/backend"}}]}}

Tests

102 selftest assertions; 34 mutations, 0 stale, 0 uncaught.make check green.

• Rule 9 — the mutation harness edits the real scripts/closing-ref-gate.py and re-runs the real suite. There is no inline copy of any rule; that shape drifted from production twice here (.github#114, feat(code-quality): opt-in black --check format job (diff-scoped) #115). classify() is the single function both the assertions and the mutations go through.
• Rule 5 — every anchor must match exactly once. Zero matches is stale and fails the run exactly like an uncaught mutation, which is the assertion that the anchor actually applied — an inert mutation and real coverage look identical in a log otherwise.
• Rule 10 — expect_unreadable takes the substring the refusal must contain, so a case named for a truncated page cannot pass on an unrelated exception. Two mutations are caught precisely by "refused, but for the wrong reason".
• Rule 6 — the commit-type vocabulary is parsed out of org-standards.md (found: chore ci docs feat fix sec) and every declared type is asserted to parse; the derivation fails closed if it finds fewer than six. Independently written observed types (test refactor perf build style) are asserted too. Mutation coverage cannot see a vocabulary gap.
• The draft fail-open is pinned in both directions, because a fail-open nobody can break is a fail-open nobody notices.

Arming — not required anywhere (rule 4)

Measured, not assumed:closing-ref / set-status / set-pr-status is a required status check nowhere — 19 repos × develop/staging/main/master × both protection systems (classic protection and rulesets, since a ruleset-only branch 404s on the classic API). So a finding is loud and blocks no merge.

Blast radius on open PRs is near zero: callers trigger on opened, reopened, ready_for_review, converted_to_draft — not synchronize — so a push does not re-run this on an existing PR.

Measured against every open PR in the fleet with the gate's own logic: 9 pass, 11 draft (exempt), 5 name no ticket, 13 would report a finding. Naming them exactly, as the ones that would go red:

.github#306, .github#311, backend#2083, backend#2128, client#785, client#791, client#798, e2e-test-agent#208, e2e-test-agent#209, e2e-test-agent#210, frontend-app#878, rfcs#44, rfcs#45

Each is fixed by adding one line to the PR body — there is no backlog to clear and nothing to migrate. Making it a required context is a separate, later decision that belongs after that backlog, not in the change that introduces the check.

Stated limitation: there is no edited trigger, so after an author adds the Closes line the job does not re-run by itself — the check has to be re-run manually. The gate's own failure message says so. Adding edited means touching 19 per-repo callers and belongs in its own PR.

Placement — why a job in an existing reusable

Three constraints, and set-pr-status.yml is the only host satisfying all three:

1. It needs a cross-repo token. Almost every house ticket is in tracebloc/backend, and nearly every repo is private. This file already mints an org-scoped installation token (repository_selection: all) and all 19 callers pass secrets: inherit. code-quality.yml cannot host it: it is deliberately secretless with contents: read, and a called workflow may not hold more than its caller grants — adding pull-requests: read there would fail all 16 callers at startup, with no jobs, before any could be updated.
2. It must run on the same event that puts a card in Code review — this file's trigger set by construction.
3. A new reusable would need a repo-inventory.yml row for all 19 repos, and that file is guarded by conformance-gate.yml — a required check every other merge invalidates. A job in an existing reusable needs no row: the inventory tracks callers, one row per reusable, and set-pr-status.yml already has its rows.

Guarded files touched: none. Not repo-inventory.yml, scripts/caller-drift.py, scripts/tests/caller-drift-selftest.py, .github/workflows/caller-drift.yml, or .github/workflows/conformance-gate.yml — so this does not queue behind the org audit.

closing-ref is a separate job from set-status deliberately: a finding must not stop the card being written, and a failed board write must not hide the finding. Steps inside one job short-circuit; jobs do not.

Nothing from the PR is interpolated into a run: block — the title is read from the API by the checker itself, so a title containing shell is data, never code.

Also

The PR template's Related section and checklist now say the link is required when the title names a ticket, and spell the cross-repo form. The old note claiming closing keywords "do not fire on merge" predated the fleet's move to develop as the default branch and is corrected.

Test plan

• make check green (ruff, shellcheck, house-rules, action-pins, mint-scope, actionlint, mutations-dry, all selftests)
• closing-ref-gate-selftest.py — 102 assertions
• closing-ref-gate-mutations.py — 34 mutations, 0 stale, 0 uncaught
• Fixture shapes re-verified against the live GraphQL API
• Required-check status measured across 19 repos and both protection systems
• This PR complies with its own gate: title names #2364 and backend#2364; body links tracebloc/backend#2364

---

Note

Medium Risk
Org-wide CI on 19 callers that mints an App token and reads cross-repo issue graphs. Not a required check, so it cannot block merges, but a logic bug could still close or fail to close the wrong tickets once authors follow its advice.

Overview
Title-named tickets must now be linked in the PR body. House titles put the ticket in the subject (fix(2218): …); GitHub only creates closingIssuesReferences from a body keyword, so kanban-closure-router was green while cards never moved.

A new closing-ref job on set-pr-status.yml (same event as the board write, org-scoped App token with issues/pull_requests read) parses the live title and asserts the live link graph. Drafts and titles that name no ticket pass. Missing vs wrong-repo (bare Closes #N in the wrong repo) are separate findings. API/permission failures exit 2 and are not softened. The job is not a required check.

The PR template now requires an owner-qualified Closes line when the title names a ticket, and corrects the old note that closing keywords do not fire on develop. Selftest + mutation harness are wired into make check.

^{Reviewed by Cursor Bugbot for commit 9b0e6d2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1e99fbf. Configure here.}

[LukasWodka]

bugbot run

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c97832a. Configure here.}

[saadqbal]

Good gate, and I'd take it. One inline note at closing-ref-gate.py:156 — a latent fragility rather than a live defect, and I verified it isn't one before writing it up. Commenting rather than approving because that thread is mine and open.

The head commit's fix is the sharpest thing in the PR and worth naming explicitly: a remedy that advised Closes tracebloc/backend#N for every bare title number would, on a release-train ticket, link the wrong issue, turn this check green, and close the wrong ticket on merge. A remedy that manufactures the defect the gate exists to prevent is a much worse failure than no remedy, and dropping the or "backend" default in favour of telling the author to confirm the owning repo is the right answer. Keeping the explanation of why GitHub ignores a title reference — closing links come only from a keyword in the body — in the message the author actually reads is what makes this actionable rather than annoying.

Completeness is derived and fails closed.connections_missing_totalcount reads the real QUERY text so the self-check cannot drift from what's actually asked for, PAGE_CAP is likewise computed from the query rather than restated, and require_complete raises Unreadable when totalCount is absent or the page is short. The docstring on Unreadable is the line I'd point at: "Never a pass, never the author's fault." That's the correct two-sided posture for a gate — an unreadable state must not approve, and must not blame the person who opened the PR either.

The type class is deliberately loose in the right direction.[A-Za-z]+ rather than the declared feat|fix|docs|sec|ci|chore list, so a type the org adds later doesn't silently stop the scope being read — with the selftest walking the declared list out oforg-standards.md and asserting each member parses. That's the direction that can't go stale: the file holds no copy of the vocabulary, and the test fails if the real list grows a member this parser can't read.

The .github note is small, and it's only interesting because you already treated the leading dot as a requirement one pattern down — so the two patterns disagreeing is the kind of thing that reads as an oversight later even though it works today.

[saqlainsyed007]

Reviewed at high effort — approving.

The gate logic is correct and precisely scoped. classify handles the three cases right — a bare number matches any repo, a repo-qualified ref matches on repo (and owner if given), and the number-seen-but-repo-wrong case is WRONG_REPO vs MISSING. parse_title reads the four measured title forms and the scope/parenthetical overlap is de-duplicated. It fails closed on an unreadable title or a partial link graph (Unreadable), and correctly PASSES drafts and titles that name no ticket (precision over recall, stated). The rollout is opt-in per repo (if: inputs.closing-ref-check), so it isn't a fleet-wide surprise. The PR-template edits also fix a now-false note (closing keywords do fire on merge now that every train repo's default branch is develop).

I checked @saadqbal's open thread and agree with their own conclusion: it is not a live defect. SCOPE_REPO_RE's repo class [A-Za-z0-9][A-Za-z0-9._-]* can't match a leading dot, but a scope is always inside parentheses and PAREN_REPO_RE ([A-Za-z0-9.]) matches anywhere in the title, so a .github#N scope is still detected via that path. Worth aligning the two character classes as a one-char consistency nit so the requirement isn't stated for one pattern and silently relied-upon-elsewhere for the other — but non-blocking, and your call.

Green: all checks pass, mergeable, no open Bugbot threads.

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9b0e6d2. Configure here.}

[saadqbal]

Approving on 9b0e6d2d. Fix verified in the source and by running it:

SCOPE_REPO_RE = ^(?:([A-Za-z0-9][A-Za-z0-9._-]*)/)?([A-Za-z0-9.][A-Za-z0-9._-]*)#(\d+)$
PAREN_REPO_RE = \(\s*(?:([A-Za-z0-9][A-Za-z0-9._-]*)/)?([A-Za-z0-9.][A-Za-z0-9._-]*)#(\d+)\s*\)

The two repo classes now agree, and parse_title reports the right provenance:

'ci(.github#300): arm the stale sweep' -> [('.github', 300, 'scope')] # was 'parenthetical'
'ci(backend#300): arm the stale sweep' -> [('backend', 300, 'scope')] # unchanged
'ci(300): bare' -> [(None, 300, 'scope')] # unchanged

So the label is accurate for the one input where it wasn't, and — the part I actually cared about — detection of a .github scope no longer depends on PAREN_REPO_RE happening to reach into the scope position. Narrowing that pattern later can't now silently stop the gate seeing tickets in the repo it lives in.

Reproducing my measurement before applying it, and quoting the before/after in the thread, is the right instinct — I'd have wanted that even if the answer had come back the other way.

Also nice that you added the selftest case; a fix whose only evidence is a reviewer's shell session is one refactor from regressing.

One thing I noticed and am explicitly not asking for: a title naming the same ticket twice at different qualification levels — ci(.github#300): … (tracebloc/.github#300) — yields two refs, because the dedupe key is (owner, repo, number) and "" ≠ "tracebloc" for the owner. So the remedy would name that ticket twice. It's cosmetic, and it behaved identically before this change (two refs, both labelled parenthetical), so my fix didn't introduce it. Mentioning it only in case you'd rather normalise the owner into the key while you're in here — not a request, and I wouldn't hold anything for it.