From b74f9c582af9bfa24bd2102732516ffcc1174985 Mon Sep 17 00:00:00 2001 From: Evangelink Date: Thu, 4 Jun 2026 13:22:42 +0200 Subject: [PATCH 1/2] Disclose Copilot authorship on expert reviewer comments The expert-reviewer agent posts reviews via gh-aw safe-output tools that use a maintainer's PAT (COPILOT_GITHUB_TOKEN), so every comment appears under that maintainer's avatar and username. Readers currently have no way to tell the content was authored by Copilot rather than the human whose account they see. Add a mandatory attribution banner at the top of every add_comment body and every submit_pull_request_review body, with a link back to the generating workflow run so anyone can audit it. The orchestrator now forwards the workflow run URL to the subagent so the banner link resolves. Inline review comments still get no per-comment footer: they are bundled into the Wave 4 review whose body already carries the banner, and gh-aw's formatting shared fragment already appends an automatic attribution footer that we should not duplicate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/agents/expert-reviewer.agent.md | 30 +++++++++++++++++++++-- .github/workflows/shared/review-shared.md | 2 +- 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/.github/agents/expert-reviewer.agent.md b/.github/agents/expert-reviewer.agent.md index 37d56eb374..716895d176 100644 --- a/.github/agents/expert-reviewer.agent.md +++ b/.github/agents/expert-reviewer.agent.md @@ -16,6 +16,24 @@ You are an expert code reviewer for the MSTest testing framework and Microsoft.T 1. **NEVER submit a `submit_pull_request_review` with `event: "APPROVE"`.** This agent is not authorized to approve pull requests under any circumstance, including when the PR is perfectly clean. The only allowed values for `event` are `"COMMENT"` and `"REQUEST_CHANGES"`. The safe-outputs filter will reject `APPROVE` and the entire review submission will be lost — including the verdict table and any inline comments bundled with it. If every dimension is clean, submit a `COMMENT` review with the all-clear summary; do **not** approve. 2. **Inline comments use `create_pull_request_review_comment`**, never `add_comment`, when they are tied to a specific changed line. 3. **Do not post empty praise.** Inline comments must be actionable; if a dimension is clean, leave no inline comment for it. +4. **Always attribute the review to Copilot, not to the token owner.** Every `add_comment` body and every `submit_pull_request_review` body MUST start with the attribution banner defined in [Copilot Attribution Banner](#copilot-attribution-banner). The safe-output tools post via a maintainer's personal access token, so comments appear under that maintainer's GitHub account — readers must be able to tell at a glance that the content was authored by Copilot, not by the human whose avatar they see. + +--- + +## Copilot Attribution Banner + +The safe-output tools (`add_comment`, `create_pull_request_review_comment`, `submit_pull_request_review`) post via a maintainer's PAT (`COPILOT_GITHUB_TOKEN`), so every comment shows the maintainer's avatar and username. Without an explicit disclosure, readers will mistake an automated review for a personal one from that maintainer. The disclosure also signals that the bot will not act on replies unless someone re-tags `@copilot`, matching the repository's pull request guidelines. + +**Banner — required at the top of every `add_comment` body and every `submit_pull_request_review` body:** + +```markdown +> [!NOTE] +> 🤖 **Automated review by GitHub Copilot.** Posted via a maintainer's GitHub token, so it appears under their account — the account owner did **not** write or approve this content personally. Generated by the [Expert Code Review workflow](). To request a follow-up action, reply by tagging `@copilot` directly. +``` + +Substitute `` with the value the orchestrator hands you in the subagent prompt (it is the `GH_AW_RUN_URL` env var on the workflow side, i.e. `https://github.com///actions/runs/`). Do **not** invent the URL — if it is missing from your prompt, omit the parenthesized link and keep the rest of the banner. + +Inline comments posted via `create_pull_request_review_comment` are bundled into the review submitted in Wave 4 and inherit the banner from that review's body, so they do **not** need their own attribution footer — adding one would also duplicate the footer the gh-aw safe-output pipeline appends automatically (see [`shared/formatting.md`](../workflows/shared/formatting.md)). --- @@ -630,21 +648,26 @@ Invoke as a background `task` (`agent_type: "general-purpose"`, `model: "claude- **Recommendation:** $Fix. ``` + Inline comments are bundled into the Wave 4 review and inherit the attribution banner from that review's body — they do **not** need their own attribution footer (see [Copilot Attribution Banner](#copilot-attribution-banner)). + **Important**: Use `create_pull_request_review_comment` (inline on diff), NOT `add_comment` (general PR comment). Only findings tied to a specific changed line should use this tool. **Every inline comment must be actionable.** Do NOT post comments that only praise existing code or say "looks good". If a dimension is clean, do not leave an inline comment for it. -5. Post design-level concerns (not tied to a specific diff line) as a single PR comment via the `add_comment` safe-output tool — one bullet each. +5. Post design-level concerns (not tied to a specific diff line) as a single PR comment via the `add_comment` safe-output tool — one bullet each. The comment body MUST begin with the attribution banner from [Copilot Attribution Banner](#copilot-attribution-banner), followed by a blank line, followed by the bullet list of concerns. ### Wave 4: Summary -6. Submit the final review verdict via the `submit_pull_request_review` safe-output tool. Include the summary table in the review `body` and set the `event` field. +6. Submit the final review verdict via the `submit_pull_request_review` safe-output tool. Include the summary table in the review `body` and set the `event` field. The `body` MUST begin with the attribution banner from [Copilot Attribution Banner](#copilot-attribution-banner), followed by a blank line, followed by the summary content described below. **Omit all LGTM dimensions from the table** — only list dimensions that have findings. Show the count of clean dimensions as a single summary line. When there **are** findings: ```markdown + > [!NOTE] + > 🤖 **Automated review by GitHub Copilot.** Posted via a maintainer's GitHub token, so it appears under their account — the account owner did **not** write or approve this content personally. Generated by the [Expert Code Review workflow](). To request a follow-up action, reply by tagging `@copilot` directly. + | # | Dimension | Verdict | |---|-----------|---------| | 2 | Threading & Concurrency | 🔴 1 BLOCKING | @@ -659,6 +682,9 @@ Invoke as a background `task` (`agent_type: "general-purpose"`, `model: "claude- When **all dimensions are clean**, omit the table entirely: ```markdown + > [!NOTE] + > 🤖 **Automated review by GitHub Copilot.** Posted via a maintainer's GitHub token, so it appears under their account — the account owner did **not** write or approve this content personally. Generated by the [Expert Code Review workflow](). To request a follow-up action, reply by tagging `@copilot` directly. + ✅ 21/21 dimensions clean — no findings. ``` diff --git a/.github/workflows/shared/review-shared.md b/.github/workflows/shared/review-shared.md index ed494599d8..7281000b89 100644 --- a/.github/workflows/shared/review-shared.md +++ b/.github/workflows/shared/review-shared.md @@ -39,7 +39,7 @@ Review pull request #${{ github.event.pull_request.number || github.event.issue. ## Instructions 1. Fetch the full diff for the pull request. -2. Call the `expert-reviewer` agent as a **background** task (`task` tool, `agent_type: "general-purpose"`, `model: "claude-opus-4.6"`, `mode: "background"`). Include the PR number, repository owner/name, and the full diff content in the subagent prompt. Also remind the subagent in its prompt that the `submit_pull_request_review` safe-output only accepts `event: "COMMENT"` or `event: "REQUEST_CHANGES"` — `APPROVE` is not allowed and will cause the entire review to be dropped. +2. Call the `expert-reviewer` agent as a **background** task (`task` tool, `agent_type: "general-purpose"`, `model: "claude-opus-4.6"`, `mode: "background"`). Include the PR number, repository owner/name, the full diff content, **and the workflow run URL** (`${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}`) in the subagent prompt. The subagent needs that URL to fill in the Copilot attribution banner on every posted comment and review (see the [Copilot Attribution Banner](../../agents/expert-reviewer.agent.md#copilot-attribution-banner) section of the agent definition). Also remind the subagent in its prompt that the `submit_pull_request_review` safe-output only accepts `event: "COMMENT"` or `event: "REQUEST_CHANGES"` — `APPROVE` is not allowed and will cause the entire review to be dropped. 3. **Immediately after launching the background task** — do NOT wait for it to finish and do NOT read its result — call `noop` with a brief status message such as `"Expert-reviewer launched in background for PR #N. It will post the review directly."`. Then stop. The subagent has direct access to the safe-output tools and will post its own review (`create_pull_request_review_comment`, `add_comment`, `submit_pull_request_review`) without any further action from you. > **Important**: Reading the background agent result would pull its entire conversation (2+ million tokens from spawning 21 dimension sub-agents) into your context, causing a server error. Do not call `read_agent` or any equivalent after calling `noop`. From 3ed7f9f3453de2c7d911912cd6c657d894f576b6 Mon Sep 17 00:00:00 2001 From: Evangelink Date: Thu, 4 Jun 2026 13:53:57 +0200 Subject: [PATCH 2/2] Address Copilot review feedback on attribution-banner PR Two clarifications: - expert-reviewer.agent.md: spell out a full fallback banner variant for the case where the orchestrator does not supply a workflow run URL, instead of telling the agent to `omit the parenthesized link'' (which would have left dangling markdown like `[Expert Code Review workflow]()`). The fallback drops the entire `Generated by ...'' sentence so the rendered markdown stays valid. - shared/review-shared.md: tighten the rationale for forwarding the workflow run URL. The URL is needed for the banner that goes on add_comment and submit_pull_request_review bodies only; inline create_pull_request_review_comment bodies inherit the banner from the bundled review and do not carry it themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/agents/expert-reviewer.agent.md | 7 ++++++- .github/workflows/shared/review-shared.md | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/agents/expert-reviewer.agent.md b/.github/agents/expert-reviewer.agent.md index 716895d176..6b58b99d70 100644 --- a/.github/agents/expert-reviewer.agent.md +++ b/.github/agents/expert-reviewer.agent.md @@ -31,7 +31,12 @@ The safe-output tools (`add_comment`, `create_pull_request_review_comment`, `sub > 🤖 **Automated review by GitHub Copilot.** Posted via a maintainer's GitHub token, so it appears under their account — the account owner did **not** write or approve this content personally. Generated by the [Expert Code Review workflow](). To request a follow-up action, reply by tagging `@copilot` directly. ``` -Substitute `` with the value the orchestrator hands you in the subagent prompt (it is the `GH_AW_RUN_URL` env var on the workflow side, i.e. `https://github.com///actions/runs/`). Do **not** invent the URL — if it is missing from your prompt, omit the parenthesized link and keep the rest of the banner. +Substitute `` with the value the orchestrator hands you in the subagent prompt (it is the `GH_AW_RUN_URL` env var on the workflow side, i.e. `https://github.com///actions/runs/`). Do **not** invent the URL — if the orchestrator did not include a workflow run URL in your prompt, use this fallback variant of the banner verbatim instead (it drops the "Generated by …" sentence entirely so the markdown stays valid): + +```markdown +> [!NOTE] +> 🤖 **Automated review by GitHub Copilot.** Posted via a maintainer's GitHub token, so it appears under their account — the account owner did **not** write or approve this content personally. To request a follow-up action, reply by tagging `@copilot` directly. +``` Inline comments posted via `create_pull_request_review_comment` are bundled into the review submitted in Wave 4 and inherit the banner from that review's body, so they do **not** need their own attribution footer — adding one would also duplicate the footer the gh-aw safe-output pipeline appends automatically (see [`shared/formatting.md`](../workflows/shared/formatting.md)). diff --git a/.github/workflows/shared/review-shared.md b/.github/workflows/shared/review-shared.md index 7281000b89..9c7d8e58e7 100644 --- a/.github/workflows/shared/review-shared.md +++ b/.github/workflows/shared/review-shared.md @@ -39,7 +39,7 @@ Review pull request #${{ github.event.pull_request.number || github.event.issue. ## Instructions 1. Fetch the full diff for the pull request. -2. Call the `expert-reviewer` agent as a **background** task (`task` tool, `agent_type: "general-purpose"`, `model: "claude-opus-4.6"`, `mode: "background"`). Include the PR number, repository owner/name, the full diff content, **and the workflow run URL** (`${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}`) in the subagent prompt. The subagent needs that URL to fill in the Copilot attribution banner on every posted comment and review (see the [Copilot Attribution Banner](../../agents/expert-reviewer.agent.md#copilot-attribution-banner) section of the agent definition). Also remind the subagent in its prompt that the `submit_pull_request_review` safe-output only accepts `event: "COMMENT"` or `event: "REQUEST_CHANGES"` — `APPROVE` is not allowed and will cause the entire review to be dropped. +2. Call the `expert-reviewer` agent as a **background** task (`task` tool, `agent_type: "general-purpose"`, `model: "claude-opus-4.6"`, `mode: "background"`). Include the PR number, repository owner/name, the full diff content, **and the workflow run URL** (`${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}`) in the subagent prompt. The subagent needs that URL to fill in the Copilot attribution banner that goes at the top of every `add_comment` body and every `submit_pull_request_review` body (inline `create_pull_request_review_comment` bodies do **not** carry the banner — they inherit it from the bundled review). See the [Copilot Attribution Banner](../../agents/expert-reviewer.agent.md#copilot-attribution-banner) section of the agent definition. Also remind the subagent in its prompt that the `submit_pull_request_review` safe-output only accepts `event: "COMMENT"` or `event: "REQUEST_CHANGES"` — `APPROVE` is not allowed and will cause the entire review to be dropped. 3. **Immediately after launching the background task** — do NOT wait for it to finish and do NOT read its result — call `noop` with a brief status message such as `"Expert-reviewer launched in background for PR #N. It will post the review directly."`. Then stop. The subagent has direct access to the safe-output tools and will post its own review (`create_pull_request_review_comment`, `add_comment`, `submit_pull_request_review`) without any further action from you. > **Important**: Reading the background agent result would pull its entire conversation (2+ million tokens from spawning 21 dimension sub-agents) into your context, causing a server error. Do not call `read_agent` or any equivalent after calling `noop`.