Skip to content

docs: add a pr-triage skill and document pull request triage - #5742

Closed
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:pr-triage-skill
Closed

docs: add a pr-triage skill and document pull request triage#5742
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:pr-triage-skill

Conversation

@andygrove

@andygrove andygrove commented Sep 6, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #.

Rationale for this change

Comet has around 170 open pull requests. Almost none of them carried labels, so
there was no way for a reviewer to filter the backlog down to their area of
expertise: a query like is:pr is:open label:area:shuffle returned nothing
useful.

The project already has a documented triage process for issues
(docs/source/contributor-guide/bug_triage.md) and a bug-triage skill that
automates it. Pull requests had no equivalent.

PR triage differs from issue triage in ways that are easy to get wrong:

  • Priority labels are for issues, not PRs.
  • The PR title rarely names the subsystem. perf: reuse zstd compression contexts across shuffle blocks never says "shuffle"; the changed files do.
  • A fix: prefix does not always mean bug. A PR adding support for a type
    Comet previously fell back on is an enhancement, since nothing was broken.

Capturing these rules means the next triage pass is consistent with this one
rather than re-deriving the conventions from scratch.

What changes are included in this PR?

  • Add .ai/skills/pr-triage/SKILL.md. The skill enumerates open PRs, fetches
    each PR's changed files to derive its area, applies exactly one type label
    (bug or enhancement) plus the supporting labels (performance,
    correctness, crash, test, build, documentation) and the area:*
    labels, and prints a report. It does not review, comment on, or edit PRs, and
    it asks a human before creating any new area label rather than inventing one.
  • Add a "Pull Request Triage" section to the bug triage guide covering the three
    rules above.
  • Bring the area label table in the guide up to date with the labels that
    already exist in the repository but were undocumented: area:Iceberg,
    area:udf, area:memory, and area:joins. Also list the pre-existing
    array expressions, map expressions, json expressions, and
    temporal expressions labels as area indicators.

area:memory and area:joins were created while triaging the current backlog:
seven open PRs cover memory pools, reservations, and OOM handling, and three
cover join operators and dynamic filter pushdown, none of which any existing
area described.

How are these changes tested?

Documentation and agent-skill changes only, with no code paths to test. The
skill was exercised against the live backlog before being written up: all 170
open non-Dependabot PRs now carry a type label, and 151 carry at least one area
label. The
remainder are plan-rule, AQE, EXPLAIN, and caching changes that the guide
explicitly says should be left without one.

prettier passes on both changed files.

Add `.ai/skills/pr-triage/SKILL.md`, a skill that labels the open pull
request backlog with a type label and the `area:*` labels for the
subsystems each PR touches, so that reviewers can filter the backlog to
their area of expertise.

The skill derives the area from the PR's changed files rather than its
title, since titles rarely name the subsystem, and it asks a human before
creating any new area label.

Also document PR triage in the contributor guide and bring the area label
table up to date with the labels that exist in the repository:
`area:Iceberg`, `area:udf`, `area:memory`, and `area:joins`.
@andygrove andygrove added enhancement New feature or request documentation Improvements or additions to documentation labels Sep 6, 2026

@sunchao sunchao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness

Comet has an issue-triage guide and skill, but no corresponding PR workflow. This change adds that workflow and documents its differences from issue triage. The type definitions preserve the distinction between repairing broken behavior and adding support where Comet previously fell back. Supporting labels distinguish silent wrong results from exceptions and crashes.

There is one P2 finding in Step 3: the prescribed changed-file query returns only the first 100 files with GitHub CLI 2.93.0. A wider PR can therefore miss area labels for subsystems that appear only in the omitted paths. The later nonempty-label check cannot detect this. Please fetch the complete file list before classifying the area, as described inline.

I reviewed head 21ef3c034b93d6cd3e0ea708b7826dd5a7a83ec5 against base 7f1e00189b1ed86f1cb5acd872d97fce694482b1. The complete discussion snapshot at 2026-09-08 03:19:10 UTC contains no reviews or comments. The head has four successful checks, including Preflight, and twelve skipped checks. The authored diff passes the whitespace check. Validation was limited to source and CLI behavior inspection. I did not run the triage commands against the backlog.

Performance

The change affects documentation and contributor tooling, so it does not alter query execution or introduce a runtime performance claim. The workflow bounds concurrent PR requests at six and batches each PR's label additions into one edit. Complete file pagination will add reads for wide PRs, which is necessary to support the area-classification contract. Spark tests and runtime benchmarks do not apply to these two changed files.

Design

The guide remains the authority for type and area definitions, while the skill provides the operational sequence: discover labels and PRs, inspect changed files, classify, apply, verify, and report. It explicitly leaves Dependabot PRs alone, allows PRs without a matching area, and requires the user's choice before creating a new area label. Its permitted PR changes are limited to labels, with the result reported to the user. These boundaries fit the stated contributor workflow.

Abstraction & complexity

The implementation adds one skill and extends the existing guide without introducing a new framework or changing the issue-triage workflow. The path mapping gives the operator useful subsystem hints, while the guide holds the shared classification rules. Label additions preserve unrelated labels, and an incorrect opposite type label is removed explicitly. The file-completeness correction belongs in the existing collection step and does not require a larger abstraction.

Comment on lines +92 to +93
'gh pr view {} --repo apache/datafusion-comet --json files \
--jq "[.files[].path]|join(\" \")" > '"$SCRATCH"'/files/{}.txt 2>/dev/null' \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness

[P2] Fetch the complete changed-file list before assigning areas

With GitHub CLI 2.93.0, gh pr view --json files uses files(first: 100), and the number lookup makes one request without paging the files. For a PR changing more than 100 files, this silently omits the remaining paths. Any subsystem touched only by those paths will be missed, even though Step 7 can still report success because the PR already has other labels. Please use a paginated file query and check that collection completed before deriving the area labels, reporting an incomplete read instead of treating the first page as the whole diff.

@andygrove

Copy link
Copy Markdown
Member Author

superseded by #5762

@andygrove andygrove closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants