Skip to content

QEP-4: Automation Registry and Label Coordination (Draft) - #8

Open
mmcky wants to merge 3 commits into
mainfrom
qep-0004
Open

QEP-4: Automation Registry and Label Coordination (Draft)#8
mmcky wants to merge 3 commits into
mainfrom
qep-0004

Conversation

@mmcky

@mmcky mmcky commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

QEP-4: Automation Registry and Label Coordination

Status: Draft · Type: standard · Discussion: grew out of the QEP-2 field reports

QEP-2 standardises the human label vocabulary and a closed set of bot diagnostics, but field use surfaced a second kind of bot label it can't describe: labels automations use to coordinateaction-translation routing the Translation-Sync rollup, status-report letting the weekly-reports workflow find its own issues. These contracts evolve with the tooling, so they shouldn't be baked into a QEP label-by-label. This QEP resolves that with a registrar model: the QEP allocates the boundary, the automation owns the interior.

What it defines

  • The registry — one entry per automation: a durable origin label (full name, e.g. action-translation, on every artifact it creates), an optional task prefix (short, capability-named, /-separated, e.g. translate/), a family grey from the #ededed#bdbdbd band, a service area, and a contract link. Registration requires a documented consumer you can point at; adding/removing an entry is a substantive amendment, while everything inside a namespace never touches a QEP.
  • Three label tiersautomated (org-wide union query) and the origin label are durable; task labels are transient. Origin marks authorship, not participation.
  • Mailbox coordination semantics — worker-owns: apply = enqueue (open to org automations and maintainers by default), owner-removes = complete, human-removes = cancel, dead-letter on can't-act, and completion is inert (the anti-loop rule). Owners provision and garbage-collect their zones.
  • The token gateGITHUB_TOKEN events don't cascade, so bot→bot enqueueing requires a deliberately provisioned App/PAT: the human decision sits exactly where coordination power is granted. Humans trigger via the UI freely, floored by triage permission.
  • Tooling rulessync never touches a registered zone; prune skips registered zones mechanically instead of relying on a human recognising a routing key.

Founding registrations

Automation Origin label Task prefix Family grey
action-translation action-translation translate/ #e5e5e5
reports-activity (weekly reports) status-report #d5d5d5

Relationship to QEP-2

QEP-2 keeps automated + the three diagnostics as core vocabulary and is now fully self-contained: under the cross-QEP convention adopted in QuantEcon/meta#9, an in-review QEP is never forward-referenced by the standards it extends — instead this QEP carries the amendments to QEP-2. The draft's new Amendments to QEP-2 section preserves, verbatim, the registry hooks originally drafted inside QEP-2 during co-development (the related:/Related pairing, the origin/task-label policy sentences, the prune skip-rule, the grey-band registration clause); the acceptance PR re-applies them in their review-settled form, bumping QEP-2's version if it is Accepted by then (Rollout step 2). The machine-readable registry (qep-0004-automations.yml, following QEP-2's companion-file pattern) ships at acceptance per the Rollout — the appendix CI check needs a small extension for the registry schema first.

Decision deadline

To be set by the author once the draft has had a round of review.

🤖 Generated with Claude Code

Registry of QuantEcon automations and the label-coordination policy
designed in the QEP-2 field-report thread: registering an automation
grants a durable origin label and optionally a /-separated task
namespace whose interior is the owner's contract, free to evolve
without QEP amendment. Defines the mailbox semantics (apply=enqueue,
owner-removes=complete, human-removes=cancel), the anti-loop rule,
token mechanics, the grey-band colour allocation, and the prune/sync
skip rules. Founding registrations: action-translation (translate/)
and reports-activity's status-report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 02:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds QEP-4 as a new Draft standard proposal defining a registry model for automation-owned coordination labels (origin labels + optional task namespaces), and registers it in the README index so it becomes part of the QEP registry.

Changes:

  • Add QEP-4 document (Draft, standard) describing registry fields, label tiers, and coordination semantics (“mailbox model”).
  • Add QEP-4 entry to the README index table.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
README.md Adds QEP-4 to the registry index with Draft status and unversioned .
qeps/qep-0004-automation-registry-and-label-coordination.md Introduces the QEP-4 draft content, including founding registrations and tooling implications.
Comments suppressed due to low confidence (1)

qeps/qep-0004-automation-registry-and-label-coordination.md:105

  • This reference to QEP-2 uses the same non-existent qep-0002-standard-github-labels.md relative link, which will be broken when rendered. Update it to a valid target (e.g., the QEP-2 PR) to avoid dead links in the table.
| Family marker | `automated` | "was this made by a machine?" (org-wide union query) | Durable — every bot artifact ([QEP-2](qep-0002-standard-github-labels.md)) |

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread qeps/qep-0004-automation-registry-and-label-coordination.md
…er mirror)

The parity check that keeps the related frontmatter and the Related
header-table row in lockstep lands with the qep-0002 branch's check.mjs;
this branch carries the data it will validate once both merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per the cross-QEP convention adopted in #9, QEP-2 holds no
forward references to this in-review QEP; the registry hooks it drafted
(Related pairing, origin/task-label policy, prune skip-rule, grey-band
registration) are preserved here verbatim and re-applied to QEP-2 by
this QEP's acceptance PR, in whatever form review settles. Rollout gains
the matching amend-QEP-2 step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jstac pushed a commit that referenced this pull request Aug 7, 2026
## QEP-2: Standard GitHub Label Set and Labelling Policy

**Status: Draft** · **Type: standard** · **Discussion:
[QuantEcon/meta#324](QuantEcon/meta#324

This is the **DRAFT** of QEP-2, re-recording the label-set decision from
[QuantEcon/meta#324](QuantEcon/meta#324) as a
worked `Standard`-type QEP — step 2 of QEP-1's rollout. It's opened as a
draft so we can keep iterating on the wording before it goes up for a
decision; the substantive calls in #324 are all settled, and the draft
has since been revised from field testing (see below).

### What it defines

- **The standard label set** — 19 core labels (universal) + 2
lecture-extension labels (`new-lecture`, `editor`) = 21, with standard
names, colours, and descriptions. The core 19 also becomes the org-level
default for new repos.
- **The labelling policy** — one Type label per issue at triage, plus
optional cross-cutting modifiers (priority, `security`); unlabelled =
needs-triage/normal-priority (and the clarification that "no labels" ≠
"no priority label"); priority is a heat-scale that marks only outliers;
`question` vs `discuss` and a `discuss` closing convention; automation
labels are bot-applied.
- **The one behavioural change** — retire the status labels (`in-work` /
`ready` / `review` / `blocked`) in favour of native GitHub (Draft PRs,
"Ready for review", review requests, issue dependencies). Only
`do-not-merge` survives.
- **The machine-readable appendix** — the normative schema ships as a
co-located
[`qeps/qep-0002-labels.yml`](https://github.com/QuantEcon/qeps/blob/qep-0002/qeps/qep-0002-labels.yml),
the single source of truth that tooling consumes, versioned with the
QEP.
- **Cross-QEP process** — QEP-2 is fully self-contained: it carries **no
forward references** to the in-review automation-registry QEP. Under the
convention adopted in #9, [QEP-4
(#8)](#8) preserves the registry
hooks drafted here (Related pairing, origin/task-label policy, prune
skip-rule, grey-band registration) and re-applies them by **amending
QEP-2 in its own acceptance PR** — cross-references land on `main`
atomically, never as dangling links. The CI plumbing still ships here:
`check.mjs` keeps `related:` frontmatter and the **Related** header row
in lockstep (used when #8 lands), and `check-labels.mjs` enforces
table↔yml parity for any `qep-NNNN-*.yml` companion plus the
post-acceptance version bump.

### Decisions from #324, folded into this draft

| Item | Resolution (from #324) |
|---|---|
| **Q1 — `infrastructure`** | **Approved** as a distinct Type label,
split from `maintenance` |
| **Add `discuss`** (@jstac) | In — core label, reuses meta's pink
`#f904a0`, split cleanly from `question` |
| **Recolour `documentation`** (@xuanguang-li) | Done — moved to tan
`#b08968`, off the blue "new content" family |
| **`good first issue` / `help wanted` spacing** (@longye-tian) | Keep
the spaces — GitHub's Contribute-page algorithm needs the exact names |
| **Add `blocked`?** | No — native issue dependencies + Draft/"Blocked
by #N" cover it |
| **Q2 — rollout** | **Pilot `lecture-python-programming` first**, then
roll out more widely |
| **Q3 — retire status labels** | Settled — unanimous in the thread |
| **Guide clarifications** (@shlff) | Folded into the policy section
(unlabelled vs no-priority; `discuss` exit) |
| **Tooling** (@mmcky) | `qe gh labels sync` additive; a **separate**
`qe gh labels prune` reviews non-standard labels one-by-one |

### Revisions from field testing (`QuantEcon/actions`)

We applied the draft set to a full maintenance audit of
[`QuantEcon/actions`](https://github.com/QuantEcon/actions)
([#103–#110](QuantEcon/actions#110)); the
field report is in the comments below. The
`infrastructure`/`maintenance` split and the "grouping labels →
Milestones" rule both held up well; the following revisions came out of
the report, the follow-up review, and reviewer feedback on the draft:

| Change | Outcome |
|---|---|
| **`security`** | Added as a **cross-cutting modifier**, like priority:
exactly one Type label, plus optionally `security` (e.g. `bug` +
`security` + `high-priority`). Makes `org:QuantEcon label:security` a
standing query and signals a different review bar. Type-level and
repo-local alternatives are recorded in the QEP |
| **`infrastructure` colour** | Settled at `#1d3c78` — the placeholder
`#006b75` clashed with the `help wanted` teal |
| **`infrastructure` vs `maintenance`** | The boundary test is now
repo-agnostic: *would it appear in a release note?* |
| **`testing`** | Added to the deliberately-not-labelled list — test
work reads as `infrastructure` or `maintenance` |
| **Source of truth** | The QEP carries the schema itself as
`qep-0002-labels.yml`; a change to it is a substantive amendment (bumps
`version`) per QEP-1. CI (`check-labels.mjs`) fails any PR where the
tables and the yml disagree, and requires the version bump once the QEP
is Accepted. `QuantEcon/cli` reads the yml from this repo and carries no
copy |
| **Rename map** | `high priority` → `high-priority` added as a sync
example. Alias/variant/prune knowledge deliberately stays in the CLI —
the yml records only the standard itself |
| **Automation policy** | Diagnostics are **at most one** (routine
output diagnoses nothing). Coordination labels (origin labels,
`/`-namespaced task labels) are spun out to [QEP-4
(#8)](#8), which amends QEP-2 with
the registry hooks when accepted — see *Cross-QEP process* above. Grey =
low-salience state, with a `#ededed`–`#bdbdbd` band allocation rule |
| **Tier-neutral descriptions** | `description` is machine-synced to
every adopting repo (incl. tooling repos and the org default), so `bug`
/ `enhancement` / `documentation` drop their lecture wording — the
lecture guidance moves to the human-only *When to use* column.
`build-failure` is scoped while we're at it: scheduled checks label the
issues they open; a failing check on a PR carries no label (the red ✗ is
the signal) |

### Notes

- With the appendix as source of truth, `QuantEcon/cli`'s `labels.yml`
is retired rather than updated — the sync tool reads
`qep-0002-labels.yml` from this repository.
- @shlff's optional `qep-candidate` (meta-only) label is **deliberately
out of scope** here — he flagged it for #325, not this rollout.

### Decision

No formal deadline window — #324 already reached consensus by its 17
June deadline, so this is largely a formalisation. The author will
circulate the draft to the team by email for a final look before it
moves out of Draft.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@mmcky mmcky mentioned this pull request Aug 8, 2026
mmcky added a commit that referenced this pull request Aug 8, 2026
QEP-2 merged in #2 while its status still read **Draft**, so `main`
currently holds an in-effect standard that its own record describes as a
draft. This PR completes step 5 of QEP-1's decision process, which was
skipped at merge time.

The flip is applied in the three places
[AGENTS.md](https://github.com/QuantEcon/qeps/blob/main/AGENTS.md) keeps
in sync: the YAML frontmatter, the in-document header table, and the
README index row. CI parity passed before this PR only because all three
agreed on the wrong value.

**No version bump.** A newly accepted QEP carries no `version` —
implicitly v0 until first substantively amended — so the README
`Version` column stays at the em dash. QEP-4 will be the thing that
moves QEP-2 to v1, when its acceptance PR re-applies the registry hooks
it parked per the convention in #9.

The decision itself is not reopened here: @jstac approved the label set
and the labelling policy on #2, and @mmcky confirmed the merge. Only the
bookkeeping is outstanding.

## Changes

| Location | Was | Now |
|---|---|---|
| `qeps/qep-0002-standard-github-labels.md` frontmatter | `status:
Draft` | `status: Accepted` |
| `qeps/qep-0002-standard-github-labels.md` header table | Draft |
Accepted |
| `README.md` index row | Draft | Accepted |

Nothing else changes — no normative content is touched, so this is an
editorial commit under AGENTS.md's substantive/editorial split.

## Downstream

Merging this changes the README row QEP-3 (#7) and QEP-4 (#8) must
carry. Both branches are already stale against `main` (they predate
QEP-1 v2 and QEP-2's merge, and neither has the QEP-2 row at all), so
each will be brought up to date in its own revision round.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mmcky

mmcky commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

One boundary question for this QEP's review, surfaced while auditing the CLI issues against the now-accepted QEP-2 (QuantEcon/project-qep2#1).

QEP-2's automation contract opens "Every bot-opened issue or PR carries automated, plus at most one diagnostic". Applied literally that pulls in Dependabot, which labels its own PRs dependencies alone. We settled it on 2026-08-08 as: the contract binds automations we own; a third-party bot labelling its own output is outside it, and QEP-2's authority there stops at defining the label dependencies — name, colour, description. So labels: [dependencies] in .github/dependabot.yml is correct as-is (QuantEcon/cli#4, QuantEcon/cli#5).

That is a working decision, not something QEP-2 says. Since this QEP owns bot behaviour and the registrar model, it is the right place to make the boundary explicit — who registers, and what the contract asks of automations nobody here can configure beyond a labels: key. Worth settling here rather than as a QEP-2 amendment, given the same line decides how origin labels apply to third-party emitters.

quantecon-services pushed a commit that referenced this pull request Sep 8, 2026
…tream amendments

Folds three ruled changes into this amendment and drops one it originally
proposed, so v3 lands as one coherent round instead of a v3 that a v4 would
immediately partly delete. Ruled 2026-09-08; discussion on #26, #9 and #22.

DROPPED FROM THIS AMENDMENT: the one-week default comment window. The ruling
went further than shortening it.

THE DECISION DEADLINE GOES. Step 3 of "How a QEP is decided" is struck and the
acceptance trigger becomes "no objection is outstanding" rather than a date —
lazy consensus with no clock, so a QEP nobody objects to still lands without
needing the lead, and nothing is accepted by the mere passage of time. Roles,
the Summary's "clear deadline" promise, and the README's Proposing steps all
follow.

The evidence is recorded under Alternatives considered: on 2026-09-08 none of
the four open QEP pull requests had a live deadline — #8 had never set one in
47 days, #13's and #23's had passed, #18's was deliberately deferred — while
nothing in CI ever read a PR description. The field was required by this
document and checked by nothing. Enforcing it in CI was considered and
declined: it would have turned three open PRs red for a rule they predated.

THE README INDEX IS GENERATED, post-merge from frontmatter and ordered by
number, so a PR carries no row of its own and two QEP PRs cannot collide on one
line of one table. The mechanism shipped ahead of this amendment in #30,
together with the ordered-list check; Automation now names both, and a stale
index is a warning rather than a failure. Numbering states what already
happened informally: a number is reserved when its draft PR opens, released if
that PR closes unmerged, so index gaps are normal while drafts are open.

A DOWNSTREAM QEP CARRIES ITS UPSTREAM AMENDMENTS — #9's sentence, widened by
the case it did not anticipate. Where the superseded standard is not itself a
QEP, the Adoption section must name the contract, the obligation and a date by
which the handover completes: QEP-6 asserts precedence over an external tracker
contract with no date on it, and the qe skills depend on that sentence at
runtime. Closes #9.

Also brings the branch up to date with main, which had moved on by QEP-3's
merge (this branch predated it, so it carried neither the file nor its index
row).

Verified green under both the pre-#30 checks on this branch and #30's new ones,
including the ordered-list rule over the rewritten Adoption entry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hwm5shrGXmaq4r3Lp9kS33
@mmcky mmcky mentioned this pull request Sep 10, 2026
mmcky added a commit that referenced this pull request Sep 14, 2026
Adds **QEP-5: Code of AI Use** as a Draft. Discussion: #12.

AI-assisted contributions are welcome at QuantEcon — our maintainers use
these tools daily and in the open. This QEP asks every contributor to
accept six norms in exchange: a human is in the loop (chooses the task,
reads the result, submits it, answers review), meaningful AI involvement
is disclosed with a hand-added `Assisted-by:` trailer, the contributor
owns and can explain the work, claims about tests and benchmarks are
verified before they are made, nobody crawls the tracker, duplicates, or
farms pull requests, and a lapse is put right by amending the pull
request.

It is **policy, not infrastructure**. The norms bind from acceptance,
like the Code of Conduct, without anyone signing anything and without
anything being built. The problem it answers is cost, not safety: an
unreviewed submission is an *extractive contribution* — it costs more to
review than it returns.

## Scope

The [discussion issue](#12)
originally proposed a contributor registry and a pull-request gate
alongside the policy. Both are deliberately **out of scope**, for two
reasons set out in *What this is not* and *Alternatives considered*: the
norms bind whether or not anyone attests to them, so a registry would
notify rather than bind; and GitHub's native volume controls (org-level
pull-request limits in particular) cover the single-account case for
free. The full registry design is retained in
QuantEcon/infrastructure#5; two constraints from it are carried into the
QEP so they survive. At acceptance, #12 closes with a pointer there and
the revival trigger — the native controls proving insufficient — rather
than a new discussion issue.

## Revised 2026-09-10

Rebased onto main (the hand-added README index row is gone; the index is
generated post-merge). The text was revised against the two comments
below and @oyamad's suggestions from the 8 Sep review round; the comment
of 2026-09-10 on this PR lists what changed. The revision was drafted
with AI assistance and the document now carries its own *Disclosure*
section.

## Revised 2026-09-11

Revised against @jstac's comment of 10 Sep; the comment of 2026-09-11 on
this PR lists what changed. In short: a marker-fenced *Code in brief*
above the full norms, three visibility channels in *Adoption* (a single
`Assisted-by: <tool> (<model>)` or `Assisted-by: none` line in the
template, the `AGENTS.md` block, and a comment-only ask on external PRs
that fires only when the line is missing), norm 2 letting the tool add
the trailer, the compliance study in *Alternatives*, and an *Amendments
to QEP-2* section for the `good first issue` reservation. Rollout is
tracked on #33, with QuantEcon/cli#50 and QuantEcon/meta#385 for the two
mechanisms.

## Points worth arguing about

**Norm 1 binds ownership, not attendance.** LLVM's test: a human in the
loop, not a human at the keyboard. Consequence: a cron-scheduled
pipeline whose operator discloses, owns, and answers review is
compliant; one that files and vanishes is not. The draft as first
written would have made QuantEcon.py#923 a violation of a change we
merged.

**Norm 3's test carve-out is now bounded.** Tests stand in for
line-by-line reading only where a test would fail if the change were
wrong. Deletions, behaviour-preserving refactors, and changes the issue
asked to be judged site by site must be read — #923 is the
counter-example, where dead-code removal satisfied the original
carve-out with no reading at all.

**Norm 5 adopts LLVM's `good first issue` prohibition, with the label as
the fence.** It is deliberately *not* widened to `bug` or audit issues:
a maintainer who wants any issue reserved for hands-on learning labels
it, and Adoption makes that deliberate. The alternative — a fuzzy
"tractable enumerated issue" category — has no boundary a contributor
can see.

**Scope now covers issues, review comments, and proposals**, in one
sentence after the norms. LLVM covers these and the cost argument is
identical. This is the one change with no prior discussion on the
thread.

**Organisation-operated automation is carved out**, applying the
accountability rule rather than waiving it: a named maintainer owns the
output, the automation is *declared in a public register*, and it
carries QEP-2's `automated` label. That register is what QEP-4 (#8)
would supply; the two drafts should agree before either is accepted.

## Field test

Per the practice proposed in #14, the draft's disclosure ask was tried
informally on **one pull request in one repository** (QuantEcon.py#923,
merged 9 Sep). That exercised norms 1, 2, and 3 and the enforcement
path. Norms 4, 5 (the `good first issue` rule), and 6, the
organisation-automation carve-out, and the scope beyond pull requests
are **untested**.

## Process

Draft. #23 (QEP-1 v3) removes the decision deadline and requires
acceptance by a Core Maintainer other than the author, with no objection
outstanding; this PR follows that rule whether or not #23 has merged
first. Before a decision, the draft goes to the whole team, and possibly
the community, as @oyamad suggested.

`qep-checks` parity: new QEP, unversioned (implicitly v0); no README
row.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants