Uh oh!
There was an error while loading. Please reload this page.
Fix chronic Smoke Copilot AOAI dispatch failure and cap per-run AI credits - #53817
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Thanks for the fix! 🎉 This PR is well-aligned with the project's focus on agentic workflows and reliability. You've addressed a chronic issue affecting multiple duplicate open issues (#53235, #53263, #53129, #48838) where What stands out:
The PR is ready for review and merge.
|
PR Triage
Fixes a chronic 0%-success smoke workflow that burns ~1.5M tokens per run — high urgency. Currently a draft; recommend undrafting for expedited human review.
|
There was a problem hiding this comment.
Pull request overview
Fixes AOAI smoke workflow dispatch failures and adds AI-credit guardrails.
Changes:
- Supplies the required dispatch ref and
inputs.message. - Caps AI usage at 60 credits while retaining the 15-minute timeout.
- Adds regression coverage and recompiles workflows.
Show a summary per file
| File | Description |
|---|---|
pkg/cli/smoke_copilot_aoai_workflow_contract_test.go | Verifies dispatch inputs and resource caps. |
.github/workflows/smoke-copilot-aoai-entra.md | Fixes dispatch instructions and adds the credit cap. |
.github/workflows/smoke-copilot-aoai-entra.lock.yml | Applies the compiled Entra workflow changes. |
.github/workflows/smoke-copilot-aoai-apikey.md | Fixes dispatch instructions and adds the credit cap. |
.github/workflows/smoke-copilot-aoai-apikey.lock.yml | Applies the compiled API-key workflow changes. |
Review details
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 5/5 changed files
- Comments generated: 0
- Review effort level: Balanced
🎉 This pull request is included in a new release. Release: |
Four open duplicate issues track
Smoke Copilot - AOAI (apikey)failing at 0% success while burning ~1.5M tokens per run. All four share one root cause: the prompt asks the agent to dispatchhaiku-printerwithout naming that workflow's requiredmessageinput, so the API rejects the dispatch and one bad item hard-fails the wholesafe_outputsjob — after 11/13 outputs had already succeeded.smoke-copilot.mdwas already hardened against this; the AOAI variants were missed.Changes
Dispatch instruction (
smoke-copilot-aoai-apikey.md,smoke-copilot-aoai-entra.md— same latent bug in the twin): mirror the fixed wording fromsmoke-copilot.md, requiring an explicitrefand a non-emptyinputs.message.Cost guardrail:
max-ai-credits: 60on both AOAI workflows. Healthy runs cost ~27 AIC; the failing run cost ~50 AIC, so this leaves headroom while bounding runaway burn regardless of future root causes. The existingtimeout-minutes: 15runtime cap is left alone — a healthy agent job already uses ~12.5 min, so tightening it would produce false failures.Regression test:
pkg/cli/smoke_copilot_aoai_workflow_contract_test.gofollows the existing*_workflow_contract_test.gopattern and pins the dispatch-input wording plus both caps for the two variants.Recompiled the two
.lock.ymlfiles (maxAiCredits: 60in the AWF api-proxy config).Notes for reviewers
max-ai-creditspropagates to theevalsjob as well (existing compiler behavior), lowering its budget from the 400 default to 60. Evals run asmallmodel over the log and are far below that, but flagging it in case 60 is considered too tight.safe_outputstolerate partial success. This PR deliberately fixes the caller instead, leaving the fail-fast semantics unchanged.