Uh oh!
There was an error while loading. Please reload this page.
Harden daily regression audit against tool-denial loops and prevent premature denial-issue closure - #53370
Merged
Merged
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
CopilotAI
changed the title
[WIP] Fix tool-denial guardrail issue in daily regression auditHarden daily regression audit against tool-denial loops and prevent premature denial-issue closureAug 17, 2026
pelikhan
approved these changes
Aug 17, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Hardens regression-audit workflows against tool-denial loops and adds evidence requirements before closing related issues.
Changes:
- Replaces Python JSON parsing with
jq. - Adds denial retry limits.
- Requires workflow-fix evidence before issue closure.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/daily-regression-audit-kiro.md | Updates parsing and denial guidance. |
.github/workflows/daily-regression-audit-kiro.lock.yml | Regenerates the compiled audit workflow. |
.github/workflows/aw-failure-investigator.md | Adds the closure-evidence gate. |
.github/workflows/aw-failure-investigator.lock.yml | Regenerates the compiled investigator workflow. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Balanced
Comment on lines
+93
to
+94
| If a tool/command is denied for this step, do not retry near-identical variants: after 2 denied attempts for the | ||
| same intent, stop and call `missing-tool` with the denied command and required capability. |
Comment on lines
+408
to
+409
| exists, do **not** close the issue as completed; keep it open and add/update a tracking comment with the missing workflow | ||
| fix evidence. |
Contributor
🎉 This pull request is included in a new release. Release: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Daily Regression Audit workflow was repeatedly hitting the Copilot SDK tool-denial guardrail due to blocked inline JSON parsing patterns, causing failed runs with no actionable output. In parallel, failure triage automation could close denial-limit issues without evidence that the affected workflow was actually changed.
Daily Regression Audit: remove blocked parsing path
python3tojqindaily-regression-audit-kiro.python3 -c/heredoc JSON parsing for MCP/local outputs.Daily Regression Audit: denial-budget-aware behavior
missing-toolwith the denied capability.Failure Investigator: closure evidence requirement
.mdor.lock.yml.Compiled workflow outputs updated
.lock.ymlfiles to keep runtime policy/prompt behavior in sync with markdown sources.