Skip to content

Fix chronic Smoke Copilot AOAI dispatch failure and cap per-run AI credits - #53817

Merged
pelikhan merged 2 commits into
mainfrom
copilot/deep-report-consolidate-duplicate-failures
Aug 19, 2026
Merged

Fix chronic Smoke Copilot AOAI dispatch failure and cap per-run AI credits#53817
pelikhan merged 2 commits into
mainfrom
copilot/deep-report-consolidate-duplicate-failures

Conversation

CopilotAI commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Four open duplicate issues track Smoke Copilot - AOAI (apikey) failing at 0% success while burning ~1.5M tokens per run. All four share one root cause: the prompt asks the agent to dispatch haiku-printer without naming that workflow's requiredmessage input, so the API rejects the dispatch and one bad item hard-fails the whole safe_outputs job — after 11/13 outputs had already succeeded.

##[error]1 safe output(s) failed:
- dispatch_workflow: Failed to dispatch workflow "haiku-printer": Required input 'message' not provided

smoke-copilot.md was already hardened against this; the AOAI variants were missed.

Changes

  • Dispatch instruction (smoke-copilot-aoai-apikey.md, smoke-copilot-aoai-entra.md — same latent bug in the twin): mirror the fixed wording from smoke-copilot.md, requiring an explicit ref and a non-empty inputs.message.

    -11. Workflow dispatch: call `dispatch_workflow` for `haiku-printer` with an original testing/automation haiku.+11. Workflow dispatch: call `dispatch_workflow` for `haiku-printer`, set top-level `ref` to `${{ github.event.repository.default_branch }}`, and include `inputs.message` with an original testing/automation haiku (non-empty string).
  • Cost guardrail: max-ai-credits: 60 on both AOAI workflows. Healthy runs cost ~27 AIC; the failing run cost ~50 AIC, so this leaves headroom while bounding runaway burn regardless of future root causes. The existing timeout-minutes: 15 runtime cap is left alone — a healthy agent job already uses ~12.5 min, so tightening it would produce false failures.

  • Regression test: pkg/cli/smoke_copilot_aoai_workflow_contract_test.go follows the existing *_workflow_contract_test.go pattern and pins the dispatch-input wording plus both caps for the two variants.

  • Recompiled the two .lock.yml files (maxAiCredits: 60 in the AWF api-proxy config).

Notes for reviewers

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
CopilotAI changed the title [WIP] Consolidate chronic duplicate Smoke Copilot AOAI failuresFix chronic Smoke Copilot AOAI dispatch failure and cap per-run AI creditsAug 18, 2026
CopilotAI requested a review from pelikhanAugust 18, 2026 20:34
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the fix! 🎉

This PR is well-aligned with the project's focus on agentic workflows and reliability. You've addressed a chronic issue affecting multiple duplicate open issues (#53235, #53263, #53129, #48838) where Smoke Copilot - AOAI (apikey) was failing at 0% success while burning ~1.5M tokens per run due to missing required workflow dispatch inputs.

What stands out:

  • Focused fix — mirrors the already-proven hardening from smoke-copilot.md to the AOAI variants, fixing the root cause (missing inputs.message in dispatch call).
  • Regression test — adds pkg/cli/smoke_copilot_aoai_workflow_contract_test.go following the existing *_workflow_contract_test.go pattern, pinning the dispatch wording and both cost caps.
  • Cost guardrail — sets max-ai-credits: 60 on both workflows (healthy runs ~27 AIC, failing runs ~50 AIC), bounding runaway token burn without false negatives on healthy runs.
  • Lock files recompiled — both .lock.yml files updated with maxAiCredits: 60 in AWF config.
  • Clear notes for reviewers — documents that the 60 credit cap propagates to evals job (existing compiler behavior) and explains the deliberate choice to fix the caller rather than make safe_outputs tolerate partial success.

The PR is ready for review and merge.

Generated by ✅ Contribution Check · auto · 49.4 AIC · ⌖ 6.33 AIC · ⊞ 9.2K ·

@github-actions

Copy link
Copy Markdown
Contributor

PR Triage

  • Category: bug · Risk: medium
  • Score: 70/100 (impact 35, urgency 25, quality 10)
  • Recommended action: fast_track

Fixes a chronic 0%-success smoke workflow that burns ~1.5M tokens per run — high urgency. Currently a draft; recommend undrafting for expedited human review.

Generated by 🔧 PR Triage Agent · auto · 58.5 AIC · ⌖ 2.15 AIC · ⊞ 8.3K ·

@pelikhan
pelikhan marked this pull request as ready for review August 19, 2026 00:56
CopilotAI balanced review requested due to automatic review settings August 19, 2026 00:56
@pelikhan
pelikhan merged commit 894c977 into mainAug 19, 2026
@pelikhan
pelikhan deleted the copilot/deep-report-consolidate-duplicate-failures branch August 19, 2026 00:56

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes AOAI smoke workflow dispatch failures and adds AI-credit guardrails.

Changes:

  • Supplies the required dispatch ref and inputs.message.
  • Caps AI usage at 60 credits while retaining the 15-minute timeout.
  • Adds regression coverage and recompiles workflows.
Show a summary per file
FileDescription
pkg/cli/smoke_copilot_aoai_workflow_contract_test.goVerifies dispatch inputs and resource caps.
.github/workflows/smoke-copilot-aoai-entra.mdFixes dispatch instructions and adds the credit cap.
.github/workflows/smoke-copilot-aoai-entra.lock.ymlApplies the compiled Entra workflow changes.
.github/workflows/smoke-copilot-aoai-apikey.mdFixes dispatch instructions and adds the credit cap.
.github/workflows/smoke-copilot-aoai-apikey.lock.ymlApplies the compiled API-key workflow changes.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.87.2

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Consolidate 4 chronic duplicate Smoke Copilot AOAI failures; add runtime/token cap

3 participants

@pelikhan