feat(harness): add verdict prefix to reviewer comments - #1153

Merged
jesseturner21 merged 4 commits into
mainfrom
feat/reviewer-verdict-prefix
May 7, 2026
Merged

feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom
feat/reviewer-verdict-prefix

Conversation

@aidandaly24

@aidandaly24aidandaly24 commented May 7, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.

Related Issue

N/A — internal improvement to the AI reviewer harness.

Documentation PR

N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • I ran npm run test:unit and npm run test:integ
  • I ran npm run typecheck
  • I ran npm run lint
  • If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

No code changes — prompt-only update to .github/harness/prompts/system.md.

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24 requested a review from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot added size/xs PR size: XS agentcore-harness-reviewing AgentCore Harness review in progress labels May 7, 2026
@agentcore-cli-automation

agentcore-cli-automation commented May 7, 2026

Copy link
Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:

  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).

A few ways to resolve this:

  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.

@agentcore-cli-automation

agentcore-cli-automation commented May 7, 2026

Copy link
Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.

Options:

  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.

Either is fine, but the two prompts should agree.

@github-actionsgithub-actionsBot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 7, 2026
@github-actions

github-actionsBot commented May 7, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Coverage Report

StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action

@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot added size/s PR size: S and removed size/xs PR size: XS labels May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot added size/xs PR size: XS and removed size/s PR size: S labels May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot added size/xs PR size: XS and removed size/xs PR size: XS labels May 7, 2026
@jesseturner21
jesseturner21 merged commit a3f504a into mainMay 7, 2026
16 of 17 checks passed
@jesseturner21
jesseturner21 deleted the feat/reviewer-verdict-prefix branch May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

size/xsPR size: XS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
feat(harness): add verdict prefix to reviewer comments - #1153
Merged
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
May 7, 2026
Merged
feat(harness): add verdict prefix to reviewer comments#1153
jesseturner21 merged 4 commits into
mainfrom 
feat/reviewer-verdict-prefix
Conversation
@aidandaly24
@aidandaly24aidandaly24
commented
May 7, 2026

Copy link


Copy Markdown

Contributor
Description
Adds a "Comment Format" section to the harness reviewer's system prompt requiring every PR comment to begin with one of three verdicts: APPROVED, APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES. This makes review outcomes immediately visible in the PR timeline and programmatically parseable.
Related Issue
N/A — internal improvement to the AI reviewer harness.
Documentation PR
N/A
Type of Change
  •  Bug fix
  •  New feature
  •  Breaking change
  •  Documentation update
  •  Other (please describe):
Testing
How have you tested the change?
  •  I ran npm run test:unit and npm run test:integ
  •  I ran npm run typecheck
  •  I ran npm run lint
  •  If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots
No code changes — prompt-only update to .github/harness/prompts/system.md.
Checklist
  •  I have read the CONTRIBUTING document
  •  I have added any necessary tests that prove my fix is effective or my feature works
  •  I have updated the documentation accordingly
  •  I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  •  My changes generate no new warnings
  •  Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the

terms of your choice.
@aidandaly24
feat(harness): add verdict prefix requirement to reviewer system prompt
c77ded1
The PR reviewer agent now must begin every comment with APPROVED,
APPROVED WITH MINOR COMMENTS, or REQUESTING CHANGES so outcomes
are immediately visible and parseable.
@aidandaly24
aidandaly24
requested a review
from a teamMay 7, 2026 16:14
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
agentcore-harness-reviewing
AgentCore Harness review in progress
labels
May 7, 2026
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The new Comment Format section conflicts with the existing guidance in .github/harness/prompts/review.md, which tells the reviewer to post a comment on the PR for each issue. The new section says "When posting your final review comment… you MUST begin the comment with one of these exact verdicts" — but it's unclear whether:
  • Every posted comment should be prefixed with a verdict (resulting in multiple REQUESTING CHANGES comments on a single PR, each with the same verdict), or
  • Only a single summary comment should carry the verdict (in which case review.md needs to be updated to tell the reviewer to emit that summary comment, and clarify how it relates to the per-issue comments).
A few ways to resolve this:
  1. Change review.md so the reviewer always posts exactly one summary comment (optionally in addition to per-issue inline comments) and put the verdict on that summary.
  2. Clarify in system.md that every top-level PR comment must begin with a verdict, and accept that multi-issue PRs will have repeated REQUESTING CHANGES prefixes.
  3. Switch to GitHub review objects (gh pr review --request-changes/--approve) instead of free-form comments, which encodes the verdict natively and sidesteps the prefix convention.
@agentcore-cli-automation
agentcore-cli-automation
commented
May 7, 2026

Copy link


Copy Markdown

The APPROVED WITH MINOR COMMENTS verdict is inconsistent with .github/harness/prompts/review.md, which explicitly instructs the reviewer to "Skip style nits and minor suggestions — only flag things that actually need to change." Under that guidance there should never be a case where the reviewer has non-blocking suggestions to post, so this verdict has no reachable use case.
Options:
  1. Drop APPROVED WITH MINOR COMMENTS and keep only APPROVED / REQUESTING CHANGES, matching the binary behavior review.md currently prescribes.
  2. Relax the "skip nits" rule in review.md so the reviewer is allowed to post optional suggestions, and keep the three-verdict scheme.
Either is fine, but the two prompts should agree.
@github-actionsgithub-actionsBot
removed
the 
agentcore-harness-reviewing
AgentCore Harness review in progress
label
May 7, 2026
@github-actions
github-actionsBot
commented
May 7, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Coverage Report
StatusCategoryPercentageCovered / Total
🔵Lines43.19%9034 / 20915
🔵Statements42.47%9592 / 22585
🔵Functions40.02%1558 / 3893
🔵Branches40%5811 / 14527
Generated in workflow #2587 for commit 986a5cd by the Vitest Coverage Report Action
@aidandaly24
feat(harness): use gh pr review for formal review verdicts
a866627
Instead of plain comments with text prefixes, the reviewer agent now
submits formal GitHub PR reviews via `gh pr review` with --approve,
--request-changes, or --comment. This integrates with branch protection
rules and makes the review status machine-readable.
@github-actionsgithub-actionsBot
added

size/s
PR size: S
and removed

size/xs
PR size: XS
labels
May 7, 2026
@aidandaly24
refactor(harness): move review verdict to review.md as one-liner
5da8cbe
Keep system.md as pure workspace context. The review task prompt now
ends with a single directive to submit a formal PR review rather than
a plain comment.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/s
PR size: S
labels
May 7, 2026
@aidandaly24
Update PR review submission instructions
986a5cd
Clarified instructions for submitting PR reviews.
@github-actionsgithub-actionsBot
added

size/xs
PR size: XS
and removed

size/xs
PR size: XS
labels
May 7, 2026
@jesseturner21
jesseturner21
merged commit a3f504a
into
mainMay 7, 2026

16 of 17 checks passed

@jesseturner21
jesseturner21
deleted the

feat/reviewer-verdict-prefix

branch
May 7, 2026 16:27
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


2 more reviewers

@jesseturner21jesseturner21jesseturner21 approved these changes
@agentcore-cli-automationagentcore-cli-automationagentcore-cli-automation approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

size/xsPR size: XS

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@aidandaly24@agentcore-cli-automation@jesseturner21