Skip to content

Harden PR Sous Chef and Contribution Check prompts to require explicit add_comment targets - #35371

Merged
pelikhan merged 2 commits into
mainfrom
copilot/aw-failures-fix-safe-outputs
May 28, 2026
Merged

Harden PR Sous Chef and Contribution Check prompts to require explicit add_comment targets#35371
pelikhan merged 2 commits into
mainfrom
copilot/aw-failures-fix-safe-outputs

Conversation

CopilotAI commented May 28, 2026

Copy link
Copy Markdown
Contributor

safe_outputs was deterministically failing in scheduled runs because agents emitted add_comment items without a numeric target while both workflows use add-comment.target: "*". That caused post-step validation to fail even when agent execution otherwise succeeded.

  • Prompt contract: PR Sous Chef

    • Updated .github/workflows/pr-sous-chef.md to require pr_number on every add_comment.
    • Added an explicit prohibition against emitting targetless add_comment items when target: "*" is configured.
    • Added a concrete JSON example using the loop PR number.
  • Prompt contract: Contribution Check

    • Updated .github/workflows/contribution-check.md to require copying issue_number verbatim from comment-dispatcher payloads into add_comment.
    • Added an explicit “never emit without numeric target” rule and concrete example.
  • Regression guardrails

    • Extended pkg/cli/contribution_check_workflow_contract_test.go to assert numeric-target requirements remain in the workflow prompt.
    • Added pkg/cli/pr_sous_chef_workflow_contract_test.go to enforce the same contract for PR Sous Chef.
{"type":"add_comment","pr_number":12345,"body":"<!-- gh-aw-pr-sous-chef-nudge -->\n@copilot review all comments and address unresolved review feedback."}
{"type":"add_comment","issue_number":35304,"body":"Thanks for the PR — here are the next changes to make..."}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
CopilotAI changed the title [WIP] Fix safe_outputs validation error for add_comment itemsHarden PR Sous Chef and Contribution Check prompts to require explicit add_comment targetsMay 28, 2026
CopilotAI requested a review from gh-aw-botMay 28, 2026 04:04
@pelikhan
pelikhan marked this pull request as ready for review May 28, 2026 04:13
CopilotAI review requested due to automatic review settings May 28, 2026 04:13
@pelikhan
pelikhan merged commit 0b956ce into mainMay 28, 2026
@pelikhan
pelikhan deleted the copilot/aw-failures-fix-safe-outputs branch May 28, 2026 04:13

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens two workflow prompts to require explicit numeric targets on add_comment safe outputs, preventing validation failures under add-comment.target: "*". Adds regression tests asserting the new prompt contract is preserved.

Changes:

  • Update pr-sous-chef.md to require pr_number on every add_comment and add a concrete example.
  • Update contribution-check.md to require copying issue_number verbatim and add a concrete example.
  • Add/extend Go contract tests to enforce the new prompt language stays in place.
Show a summary per file
FileDescription
.github/workflows/pr-sous-chef.mdAdds explicit pr_number requirement, prohibition, and JSON example.
.github/workflows/contribution-check.mdAdds verbatim issue_number requirement and JSON example.
pkg/cli/contribution_check_workflow_contract_test.goAsserts new prompt strings are present.
pkg/cli/pr_sous_chef_workflow_contract_test.goNew contract test for PR Sous Chef prompt.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 0

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw-failures] safe_outputs rejects add_comment items missing item_number when target:* is configured (PR Sous Chef, Contribution Check)

4 participants

@pelikhan@gh-aw-bot