Skip to content

Fix create-agentic-workflow agent prompting patterns - #6999

Merged
pelikhan merged 3 commits into
mainfrom
copilot/fix-agentic-workflow-prompting
Dec 19, 2025
Merged

Fix create-agentic-workflow agent prompting patterns#6999
pelikhan merged 3 commits into
mainfrom
copilot/fix-agentic-workflow-prompting

Conversation

CopilotAI commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Fix create-agentic-workflow agent prompting patterns

✅ All Requirements Implemented

1. Workflow .md file location in .github/workflows folder

  • Status: Already present, verified in multiple locations (lines 33, 225, 283, 330-331)
  • Enhancement: Added explicit instruction in prompt.md to always create workflows in .github/workflows/ folder

2. Do not recommend GitHub remote mode

  • Status: Already present with clear warning at line 198
  • Enhancement: Added same warning to prompt.md for consistency

3. Use fuzzy schedule syntax for daily/weekly schedules

  • Status: ✅ FIXED
  • Changes: Updated from cron: daily to schedule: daily, added schedule: weekly support
  • Impact: Clearer syntax, better scheduling documentation, removed unnecessary weekend caveats

4. Copilot is the default, do not add to template

  • Status: ✅ FIXED
  • Changes: Removed engine: copilot from example templates, updated guidance text
  • Impact: Cleaner templates, explicit instruction not to include default engine

5. Add close-older-issue/discussion flag for daily reporting workflows

  • Status: ✅ FIXED
  • Changes: Added guidance in "Safe Outputs" section and security best practices
  • Impact: Prevents clutter from daily reports by auto-closing older items

6. Use skip-if-match for daily improver workflows that create PRs

  • Status: ✅ FIXED
  • Changes: Added guidance with example filter pattern in "Safe Outputs" section
  • Impact: Prevents duplicate PR creation from scheduled workflows

7. Always add workflow_dispatch trigger

  • Status: ✅ FIXED
  • Changes: Updated all trigger examples, added explicit "ALWAYS include" instruction
  • Impact: All workflows can be manually triggered for testing/debugging

Files Modified

  • .github/agents/create-agentic-workflow.agent.md
  • pkg/cli/templates/create-agentic-workflow.agent.md
  • pkg/cli/templates/create-agentic-workflow.prompt.md (deleted - duplicate file)

Testing & Verification

  • ✅ Build succeeded: make build
  • ✅ Unit tests passed: make test-unit
  • ✅ No breaking changes
  • ✅ Templates properly synced

Additional Improvements

  1. Enhanced GitHub tool documentation to show read-only best practices
  2. Clarified safe-outputs usage vs direct GitHub tool mutations
  3. Improved scheduling documentation with weekly schedule support
  4. Made workflow_dispatch more prominent in all examples
  5. Removed duplicate .prompt.md file per feedback
Original prompt

Fix prompting issues in the create-agentic-workflow custom custom agent.md:

  • the agentic workflow .md file should ALWAYS be in .github/workflows folder

  • do not recommend GitHub remote

  • for daily, weekly schedules, use fuzzy schedule syntax:

    schedule: daily

  • copilot is the default, do not add to template

  • if a workflow is a form of daily reporting, add the close-older-issue/discussion: true flag

  • if a workflow is a daily improver that creates a pull request, use skip-if-match with a filter to avoid opening 2 PRs at the same time

  • always add workflow_dispatch

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@pelikhan
Fix prompting issues in create-agentic-workflow custom agent
fb628a6
- Update scheduling guidance to use fuzzy schedule syntax: schedule: daily/weekly
- Remove recommendation for GitHub remote mode (already present)
- Remove engine: copilot from templates (it's the default)
- Add guidance to always include workflow_dispatch trigger
- Add guidance for close-older-issues/discussions flag for daily reporting workflows
- Add guidance for skip-if-match filter for daily improver workflows that create PRs
- Update GitHub tool examples to show read-only operations with safe-outputs for writes
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
pelikhan
pelikhan reviewed Dec 19, 2025
View reviewed changes
Comment threadpkg/cli/templates/create-agentic-workflow.prompt.md Outdated
@@ -49,9 +49,11 @@ Analyze the user's response and map it to agentic workflows. Ask clarifying ques
- 💡 If you detect the task requires **browser automation**, suggest the **`playwright`** tool.

@pelikhanpelikhanDec 19, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot delete this file, it is a duplicate of the .agent.md file.

CopilotAIDec 19, 2025

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Deleted the duplicate file in d6b7fc5.

CopilotAI changed the title [WIP] Fix prompting issues in create-agentic-workflowFix create-agentic-workflow agent prompting patternsDec 19, 2025
CopilotAI requested a review from pelikhanDecember 19, 2025 19:47
@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

🎉 Yo ho ho! Smoke Copilot Safe Inputs found the treasure and completed successfully! ⚓💰

@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

🤖 DIAGNOSTIC COMPLETE: Smoke Copilot No Firewall STATUS: ALL_UNITS_OPERATIONAL. MISSION_SUCCESS.

@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

✅ Firewall validation complete... Smoke Codex Firewall confirmed network sandboxing is operational. 🛡️

@github-actions

github-actionsBot commented Dec 19, 2025•
edited by agentic-workflows-dev Bot
Loading

Copy link
Copy Markdown
Contributor

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

🎬 THE END — Smoke ClaudeMISSION: ACCOMPLISHED! The hero saves the day! ✨

@github-actions

github-actionsBot commented Dec 19, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Smoke Test Results

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP - Retrieved PR data successfully
  • ✅ File Writing - Created test file at /tmp/gh-aw/agent/smoke-test-copilot-20380827743.txt
  • ✅ Bash Tool - Read file back successfully
  • ✅ Playwright MCP - Navigated to github.com, verified title contains "GitHub"
  • ❌ Safe Input gh Tool - Skipped (workflow completed with other tests)

Status: PASS (4/4 core tests)

🤖 DIAGNOSTIC REPORT GENERATED BY Smoke Copilot No Firewall fer issue #6999 🗺️

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Smoke Test Results (Claude Sonnet 4.5)

Last 2 merged PRs:

Tests:
✅ GitHub MCP (list PRs)
✅ File writing (/tmp/gh-aw/agent)
✅ Bash verification
✅ Playwright navigation (GitHub homepage loaded)
✅ Cache memory write (/tmp/gh-aw/cache-memory)
❌ Safe Input gh tool (not available)

Status: PASS (5/6 tests passed)

💥 [THE END] — Illustrated by Smoke Claude fer issue #6999 🗺️

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Smoke Test Results for Copilot Engine

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP (read PRs)
  • ✅ File Writing (/tmp/gh-aw/agent/)
  • ✅ Bash Tool
  • ✅ GitHub MCP Default Toolset (get_me correctly fails - not in default toolsets)
  • ✅ Cache Memory (/tmp/gh-aw/cache-memory/)

Overall: PASS

cc @pelikhan

📰 BREAKING: Report filed by Smoke Copilot fer issue #6999 🗺️

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Recent merged PRs: Document CLI version changes: Codex 0.74.0+ and GitHub MCP Server v0.26.0+; [WIP] Fix markdown lint issues and inconsistencies
OpenAI domain access: ❌ (DNS blocked in sandbox)
GitHub MCP list merged PRs: ✅
File write/read /tmp/gh-aw/agent/smoke-test-codex-firewall-20380827747.txt: ✅
Blocked domain (example.com) blocked: ✅
Network: SANDBOXED
Overall: FAIL

🔥 Firewall tested by Smoke Codex Firewall fer issue #6999 🗺️

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Smoke Test Results

Last 2 Merged PRs: Unable to retrieve (GitHub MCP not configured)
File Creation: ✅ Created /tmp/gh-aw/agent/smoke-test-copilot-20380827723.txt
Bash Tool: ✅ Verified file contents successfully
Serena MCP: ❌ Not configured/accessible
safeinputs-gh: ❌ Tool not available in this context

Overall Status: PARTIAL PASS (3/5 tests passed)

Ahoy! This treasure was crafted by 🏴‍☠️ Smoke Copilot Safe Inputs fer issue #6999 🗺️

@github-actions

github-actionsBot commented Dec 19, 2025

Copy link
Copy Markdown
Contributor

Smoke test results:

🔮 The oracle has spoken through Smoke Codex fer issue #6999 🗺️

@pelikhan
pelikhan marked this pull request as ready for review December 19, 2025 19:51
@pelikhan
Delete duplicate create-agentic-workflow.prompt.md file
d6b7fc5
The .prompt.md file is a duplicate of .agent.md and is not used anywhere in the codebase.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
pelikhan merged commit a6bda33 into mainDec 19, 2025
4 checks passed
@pelikhan
pelikhan deleted the copilot/fix-agentic-workflow-prompting branch December 19, 2025 19:56
@github-actionsgithub-actionsBot mentioned this pull request Dec 19, 2025
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pelikhanpelikhanAwaiting requested review from pelikhan

Assignees

@pelikhanpelikhan

Copilot code reviewCopilot

Labels

smokesmoke-claudesmoke-copilotsmoke-copilot-no-firewall

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pelikhan