Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit

[Copilot]

A strict workflow that must set tools.bash: false (e.g. tools.github.min-integrity: none) still received the <mcp-clis> prompt section, which tells the agent to invoke safeoutputs from bash and that it is not an MCP tool. With no shell tool in the harness, a model following that instruction can finish without emitting any safe output — silently, when failure reporting is disabled.

Changes

• Prompt generation (pkg/workflow/mcp_cli_mount.go): buildMCPCLIPromptSection returns nil when shell execution is fully refused (bash: false or bash: []). The <mcp-clis> section and GH_AW_MCP_CLI_SERVERS_LIST are dropped, leaving the <safe-output-tools> MCP guidance as the only advertised path. The mount step is untouched, so engine.command harnesses keep their CLI wrappers on PATH.
• Validation (pkg/workflow/tools_validation.go): tools.cli-proxy: true is now rejected when bash is disabled — CLI wrappers are only reachable from a shell.
• Strict mode (pkg/workflow/strict_mode_network_validation.go): a disabled tools.bash now requires an explicit tools.cli-proxy: false, mirroring the existing min-integrity: none ⇒ explicit tools.bash rule, so the incompatibility is visible in the source.
• Codemod (pkg/cli/codemod_cli_proxy_bash.go): cli-proxy-false-when-bash-disabled inserts cli-proxy: false into the tools: block, or rewrites an existing cli-proxy: true, via gh aw fix.
• Repository workflows: ai-moderator (which paired bash: false with cli-proxy: true), smoke-agent-all-none, and smoke-agent-public-none updated and recompiled. Two existing tests using bash: false / bash: [] gained the now-required explicit setting.
• Docs (reference/tools.md), changeset, and tests for the prompt suppression, validation matrix, strict rule, and codemod.

tools:
bash: falsecli-proxy: false # added by `gh aw fix`; required in strict modegithub:
min-integrity: nonesafe-outputs:
add-labels:

Resulting lock diff for such a workflow: mcp_cli_tools_with_safeoutputs_prompt.md and GH_AW_MCP_CLI_SERVERS_LIST no longer appear in GH_AW_PROMPT_CONFIG.

Note: the strict-mode rule is a compile-time behavior change for existing workflows that disable bash without stating cli-proxy: false; the codemod is the intended migration path.

• FixesCopilot strict no-Bash workflows receive CLI-only safeoutputs instructions #53532

---

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.7 AIC · ⌖ 7.81 AIC · ⊞ 6.4K · ◷
_{Comment /souschef to run again}

---

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 7.29 AIC · ⌖ 7.92 AIC · ⊞ 8.8K · ◷
_{Comment /souschef to run again}

---

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.6 AIC · ⌖ 7.87 AIC · ⊞ 6.4K · ◷
_{Comment /souschef to run again}

---

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

Run: https://github.com/github/gh-aw/actions/runs/32109331559> Generated by 👨‍🍳 PR Sous Chef · gpt54 · 30 AIC · ⌖ 11.4 AIC · ⊞ 8.8K · ◷

_{Comment /souschef to run again}

---

Run: https://github.com/github/gh-aw/actions/runs/32123781702> Generated by 👨‍🍳 PR Sous Chef · gpt54 · 41.3 AIC · ⌖ 7.88 AIC · ⊞ 8.8K · ◷

_{Comment /souschef to run again}

[github-actions]

Thanks for working on this! 👋 This PR addresses a real pain point in #53532 where strict Copilot workflows with Bash disabled still receive CLI-only safeoutputs instructions, creating a mismatch between what the prompt directs and what tools are actually available.

A few things to keep in mind as you complete this implementation:

• Add test coverage — The implementation checklist includes validation, codemods, and workflow updates. Make sure to add tests for compiler validation, codemod correctness, and prompt generation with bash disabled.
• Document the behavior — Update docs to explain the new validation rule and when cli-proxy: false is required.

This is in great shape once the implementation is complete and tests are added!

Generated by ✅ Contribution Check · auto · 52.7 AIC · ⌖ 5.23 AIC · ⊞ 9.2K · ◷

[github-actions]

❌ Design Decision Gate 🏗️ failed during design decision gate check.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

[github-actions]

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #53548

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

[github-actions]

The implementation is clean and well-structured.

• Validation split is correct: validateCLIProxyBashCompatibility (non-strict) rejects only cli-proxy: true + bash disabled. validateStrictTools additionally requires the explicit cli-proxy: false declaration — the two layers are properly separated.
• Prompt gating (buildMCPCLIPromptSection → data.BashDisabled) prevents confusing CLI-only instructions from reaching agents that have no shell — the key behavioral fix.
• Codemod insertion logic correctly finds the first field of the tools: block and inserts before it, preserving natural YAML ordering.
• Duplicate helper logic (isBashExplicitlyRefused / frontmatterRefusesBash) is acceptable given the package boundary — both are identical and correct.
• Test coverage is thorough across validator, codemod, prompt section, and strict mode.

No blocking issues found. ✅

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 38 AIC · ⌖ 11.7 AIC · ⊞ 5.7K

[github-actions]

Comment Memory

reviewed_at: 2026-08-18T00:00:00Z
review_event: REQUEST_CHANGES
top_themes:
- strict-mode rule conflicts with Pi engine requirements
files_reviewed:
- .changeset/cli-proxy-incompatible-with-bash-disabled.md
- .github/workflows/ai-moderator.md
- .github/workflows/smoke-agent-all-none.md
- .github/workflows/smoke-agent-public-none.md
- docs/src/content/docs/reference/tools.md
- pkg/cli/codemod_cli_proxy_bash.go
- pkg/cli/codemod_cli_proxy_bash_test.go
- pkg/cli/fix_codemods.go
- pkg/workflow/compiler_orchestrator_workflow.go
- pkg/workflow/compiler_string_api.go
- pkg/workflow/mcp_cli_mount.go
- pkg/workflow/strict_mode_network_validation.go
- pkg/workflow/tools_validation.go
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 14.9 AIC · ⌖ 6.78 AIC · ⊞ 4.5K · ◷
_{Comment /review to run again}

[github-actions]

Request changes

This patch introduces one merge-blocking regression: it makes tools.bash: false incompatible with Pi workflows by unconditionally requiring tools.cli-proxy: false in strict mode, even though Pi explicitly requires tools.cli-proxy: true and relies on CLI-mounted MCP tools instead of shell access.

Blocking theme

• The new strict-mode rule conflicts with existing engine requirements and rejects a valid no-shell configuration for Pi.
• The change needs an engine-aware exception or the compatibility rule needs to be narrowed so it only applies to engines that actually depend on bash for CLI access.

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 14.9 AIC · ⌖ 6.78 AIC · ⊞ 4.5K
_{Comment /review to run again}

[github-actions]

Ponytail review: scoped to complexity/over-engineering only.

pkg/cli/codemod_cli_proxy_bash.go:L219-237: yagni: frontmatterRefusesBash duplicates tools_validation.go's isBashExplicitlyRefused (identical bool/[]any switch logic, different map depth). Export/reuse one helper instead of maintaining two copies.

Everything else (codemod line-rewriting, validation, prompt-section gating) is proportionate to the bug being fixed — no speculative abstractions or unused flexibility found.

net: -14 lines possible.

Generated by ✂️ Ponytail Reviewer for #53548 · auto · 28.2 AIC · ⌖ 5.51 AIC · ⊞ 7.3K
_{Comment /ponytail to run again}

Comments that could not be inline-anchored

pkg/cli/codemod_cli_proxy_bash.go:237

L219-237: yagni: frontmatterRefusesBash duplicates tools_validation.go's isBashExplicitlyRefused (identical bool/[]any switch logic, just a different map nesting depth). Export/reuse one helper instead of maintaining two copies.

[github-actions]

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — commenting on two minor issues; no blocking changes.

📋 Key Themes & Highlights

Key Themes

• Logic duplication: frontmatterRefusesBash (codemod package) and isBashExplicitlyRefused (workflow package) implement the same semantics independently — drift risk if one is updated.
• Test boundary clarity: TestValidateCLIProxyBashCompatibility doesn't explicitly assert that bash: false with no cli-proxy key is valid — this leaves the function's narrow scope (only blocking cli-proxy: true) unspecified in the tests.
• Codemod insertion order: setCLIProxyFalseInTools inserts before the first field rather than after bash:, so the generated change puts cli-proxy: above bash: which is slightly surprising to readers.

Positive Highlights

• ✅ Root cause addressed properly — both prompt suppression and validation guards prevent re-introduction.
• ✅ Strict-mode rule mirrors the existing min-integrity: none ⇒ explicit bash pattern — consistent design.
• ✅ Codemod is well-tested (idempotency, both insert and rewrite paths covered).
• ✅ Changeset, docs, and all affected workflows updated atomically.
• ✅ validateCLIProxyBashCompatibility is correctly called in both ParseWorkflowString and validateWorkflowToolConfigurations so neither compilation path is missed.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 55.2 AIC · ⌖ 10.2 AIC · ⊞ 7.8K
_{Comment /matt to run again}

[Copilot]

Pull request overview

Prevents shell-disabled workflows from receiving unusable MCP CLI instructions and adds validation plus migration support.

Changes:

• Suppresses MCP CLI prompts when Bash is disabled.
• Rejects incompatible CLI proxy settings and adds a codemod.
• Updates tests, documentation, and affected workflows.

Show a summary per file

File	Description
pkg/workflow/tools_validation.go	Adds Bash/CLI proxy compatibility validation.
pkg/workflow/tools_validation_test.go	Tests compatibility validation.
pkg/workflow/strict_mode_validation_test.go	Tests strict-mode requirements.
pkg/workflow/strict_mode_network_validation.go	Requires explicit CLI proxy disablement.
pkg/workflow/mcp_cli_mount.go	Omits CLI prompts without Bash.
pkg/workflow/mcp_cli_mount_test.go	Tests prompt omission.
pkg/workflow/compiler_string_api.go	Applies validation to string parsing.
pkg/workflow/compiler_orchestrator_workflow.go	Applies validation during compilation.
pkg/workflow/compiler_orchestrator_workflow_test.go	Updates disabled-Bash fixture.
pkg/workflow/bash_anonymous_validation_test.go	Updates strict Bash fixture.
pkg/cli/fix_codemods.go	Registers the migration codemod.
pkg/cli/fix_codemods_test.go	Verifies codemod registration and order.
pkg/cli/compile_guard_policy_test.go	Updates guard-policy fixture.
pkg/cli/codemod_cli_proxy_bash.go	Implements CLI proxy migration.
pkg/cli/codemod_cli_proxy_bash_test.go	Tests codemod behavior.
docs/src/content/docs/reference/tools.md	Documents the incompatibility.
.github/workflows/smoke-agent-public-none.md	Explicitly disables CLI proxy.
.github/workflows/smoke-agent-public-none.lock.yml	Recompiles the public smoke workflow.
.github/workflows/smoke-agent-all-none.md	Explicitly disables CLI proxy.
.github/workflows/smoke-agent-all-none.lock.yml	Recompiles the all-none smoke workflow.
.github/workflows/ai-moderator.md	Disables MCP CLI mounting.
.github/workflows/ai-moderator.lock.yml	Recompiles the moderator workflow.
.changeset/cli-proxy-incompatible-with-bash-disabled.md	Records the behavioral change.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 23/23 changed files
• Comments generated: 3
• Review effort level: Balanced

[gh-aw-bot]

@copilot Please address the blocking review feedback below, refresh the branch if needed, then run the pr-finisher skill and report back with validation results and any remaining blockers.

• Unresolved review threads (newest first):
  • github-actions discussion
• Failed checks from the compact queue:
  • impacted-go-tests
  • agent
  • Unit tests (A-C)
• Branch refresh was requested for this PR.
• After fixes, please run the pr-finisher skill.

Run: https://github.com/github/gh-aw/actions/runs/32093730091

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.7 AIC · ⌖ 7.81 AIC · ⊞ 6.4K · ◷
_{Comment /souschef to run again}

[gh-aw-bot]

@copilot Please review the remaining addressed feedback on this PR, confirm which threads can now be resolved, and run the pr-finisher skill if any follow-up edits are still needed.

• Unresolved review threads with author follow-up (newest first):
  • github-actions[bot] — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
  • Copilot — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
  • github-actions[bot] — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
  • github-actions[bot] — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
  • github-actions[bot] — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
  • Copilot — Omit CLI-only safeoutputs instructions when bash is disabled; make cli-proxy/bash incompatibility explicit #53548 (comment)
• No failed checks were listed in the compact queue for this run.
• Branch refresh was requested for this PR, but local make fmt could not run in this sandbox because the installed Go toolchain is older than the repo requirement (go >= 1.26.5).
• Please confirm whether CI now only needs maintainer review/re-trigger and whether the addressed threads can be resolved.

Run: https://github.com/github/gh-aw/actions/runs/32112664745

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 22.3 AIC · ⌖ 7.94 AIC · ⊞ 8.8K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot Please review the remaining addressed feedback on this PR, confirm which threads can now be resolved, and run the pr-finisher skill if any follow-up edits are still needed.

• Unresolved re...

No follow-up edits are needed, so I did not run pr-finisher. The six listed review threads now all report is_resolved: true via the review-thread API and can remain resolved; if any UI still shows them as unresolved, that appears stale.

Branch status: clean at d76a88f, with origin/main at merge-base 21b4067. CI status: referenced run 32112664745 has 8/8 jobs successful and no failed logs; the PR check list only showed the current Copilot follow-up in progress plus prior Copilot successes.

Local validation in this pass: make fmt passed with Go 1.26.5, and BASE_REF=origin/main make agent-report-progress passed.

[gh-aw-bot]

@copilot Please confirm the current state of this PR, resolve any remaining addressed review feedback in the UI if applicable, and run the pr-finisher skill only if further edits are actually needed.

• Recent follow-up indicates no code changes are currently needed.
• No failed checks were listed in the compact queue for this run.
• Branch refresh was requested for this PR.
• Please confirm whether only maintainer review / stale blocking review cleanup / normal CI completion remain.

Run: https://github.com/github/gh-aw/actions/runs/32115672254

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 20 AIC · ⌖ 7.91 AIC · ⊞ 8.8K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot Please confirm the current state of this PR, resolve any remaining addressed review feedback in the UI if applicable, and run the pr-finisher skill only if further edits are actually needed...

Confirmed current state: no follow-up edits are needed, so I did not run pr-finisher. The review-thread API reports all 6 previously listed addressed threads as is_resolved: true; I don’t have any remaining applicable UI thread resolution to perform from here.

Branch is clean at d76a88f, refreshed against fetched origin/main21b4067 (merge-base=21b4067). Latest referenced PR Sous Chef run 32115672254 has 8/8 jobs successful and no failed logs. Current PR checks show the active Copilot follow-up run in progress plus prior Copilot successes.

Local validation in this pass passed: BASE_REF=origin/main make agent-report-progress.

Remaining items appear to be maintainer/action-system side only: maintainer review or stale blocking review cleanup if GitHub still gates on it, and normal CI/check completion or re-trigger if branch protection requires it.

[gh-aw-bot]

@copilot Please confirm the current state of the addressed review feedback on this PR, refresh the branch if needed, then run the pr-finisher skill only if further edits are actually required.

• No failed checks were listed in the compact queue for this run.
• make fmt could not run in this sandbox, so no formatter push was attempted here.
• Prior follow-up suggests the work is complete; please confirm whether only stale blocking review cleanup and normal CI completion remain.

Run: https://github.com/github/gh-aw/actions/runs/32120796211

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 21.2 AIC · ⌖ 7.92 AIC · ⊞ 8.8K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot Please confirm the current state of the addressed review feedback on this PR, refresh the branch if needed, then run the pr-finisher skill only if further edits are actually required.

• No...

Confirmed current state: no follow-up edits are needed, so I did not run pr-finisher.

Addressed review feedback: the review-thread API still reports all 6 prior addressed threads as is_resolved: true.

Branch: clean at d76a88f, refreshed against fetched origin/main21b4067 (merge-base=21b4067).

CI/checks: latest referenced PR Sous Chef run 32120796211 has 8/8 jobs successful and no failed logs. Current PR checks show the active Copilot follow-up run in progress plus prior Copilot successes.

Local validation: make fmt ran successfully with Go 1.26.5; it produced unrelated Prettier-only local changes, which I reverted rather than pushing. Clean validation then passed with BASE_REF=origin/main make agent-report-progress.

Remaining items appear to be maintainer/action-system side only: stale blocking review cleanup if GitHub still gates on it, and normal CI/check completion or re-trigger if branch protection requires it.

[pelikhan]

@copilot add workflow integration tests

[Copilot]

@copilot add workflow integration tests

Added workflow integration coverage in d4fb33d: one test compiles a no-bash Codex safe-outputs workflow and asserts MCP-only safe-output guidance remains while CLI prompt/env is omitted; another verifies no-shell cli-proxy and gh-proxy configurations are rejected. Validation passed: go test -tags=integration ./pkg/workflow -run 'TestNoBash', BASE_REF=origin/main make agent-report-progress, code review, CodeQL (skipped as test-only/trivial), and secret scan.

[gh-aw-bot]

@copilot Please address the remaining review feedback on this PR, refresh the branch if needed, then run the pr-finisher skill and report back with validation results and any blockers.

• Remaining unresolved feedback appears to be the current CHANGES_REQUESTED review state.
• No failed checks were listed in the compact queue for this run.
• Branch refresh was requested for this PR.
• After fixes, please identify any addressed review threads or stale blocking reviews that could now be resolved.

Run: https://github.com/github/gh-aw/actions/runs/32134497286

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 41.8 AIC · ⌖ 8.03 AIC · ⊞ 8.8K · ◷
_{Comment /souschef to run again}

[github-actions]

PR Triage

Category: bug | Risk: high | Priority: medium (score 58)

• Impact 30/50 — fixes safeoutputs CLI/bash-disabled prompt inconsistency for strict workflows
• Urgency 12/30 — CHANGES_REQUESTED from github-actions, needs rework
• Quality 16/20 — extensive discussion, large diff (26 files)

Recommended action: defer — address CHANGES_REQUESTED first

Generated by 🔧 PR Triage Agent · auto · 53.1 AIC · ⌖ 2.52 AIC · ⊞ 8.3K · ◷

[Copilot]

@copilot Please address the remaining review feedback on this PR, refresh the branch if needed, then run the pr-finisher skill and report back with validation results and any blockers.

• Remaining ...

Current state after this pass: branch refreshed in 2481165 and pushed; origin/main (1485547) is now the merge-base. The review-thread API reports all 6 prior addressed threads as resolved, so the remaining CHANGES_REQUESTED review appears stale and can be dismissed/superseded by reviewer or bot review.

Validation passed locally: make fmt, make lint, make test-unit, make test, make recompile, and BASE_REF=origin/main make agent-report-progress. Prior run 32134497286 had 8/8 jobs successful and no failed job logs; the current agent-pushed HEAD has no check runs yet, so CI needs normal human/action-system re-trigger/completion before merge.

Code review found only an unrelated upstream origin/main test-documentation suggestion in /home/runner/work/gh-aw/gh-aw/pkg/linters/internal/astutil/astutil_test.go; no PR follow-up edit made. CodeQL timed out and explicitly instructed not to rerun.

done

[github-actions]

🎉 This pull request is included in a new release.

Release: v0.87.1