Skip to content

Add github-token support to all safe-output handlers with comprehensive test coverage - #14819

Closed
mnkiefer with Copilot wants to merge 7 commits into
mainfrom
copilot/investigate-safe-output-compile
Closed

Add github-token support to all safe-output handlers with comprehensive test coverage#14819
mnkiefer with Copilot wants to merge 7 commits into
mainfrom
copilot/investigate-safe-output-compile

Conversation

CopilotAI commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Safe-outputs configurations with per-handler github-token values failed schema validation. Only 4 of 21 handlers supported the field despite all having BaseSafeOutputConfig.GitHubToken.

# Previously failed compilationsafe-outputs:
create-issue:
github-token: ${{ secrets.AGENT_GITHUB_TOKEN }} # ❌ Unknown propertyupdate-project:
github-token: ${{ secrets.PROJECT_GITHUB_TOKEN }} # ✅ Worked

Changes

  • Schema: Added github-token property to 17 handler schemas (add-comment, create-issue, close-issue, update-issue, etc.)
  • Config generation: Updated all handler builders to include github-token in GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG JSON
  • Test coverage: Added comprehensive tests for all token precedence scenarios:
    • TestHandlerManagerMultipleNonProjectTokens - Multiple handler-level tokens
    • TestGitHubTokenPrecedenceAllLevels - All precedence combinations (5 scenarios)
    • TestSafeOutputsLevelGitHubToken - Safe-outputs level token with handler overrides

Token Precedence

The system now supports github-token at three levels with proper precedence:

  1. Handler level (highest priority) - Per-handler override
  2. Safe-outputs level - Default for all handlers
  3. Top-level (lowest priority) - Workflow-wide default

Example Usage

Safe-outputs level token with handler overrides:

safe-outputs:
github-token: ${{ secrets.SAFE_OUTPUTS_GITHUB_TOKEN }} # Default for all handlerscreate-issue:
github-token: ${{ secrets.AGENT_GITHUB_TOKEN }} # Handler overrideupdate-project:
github-token: ${{ secrets.PROJECT_GITHUB_TOKEN }} # Handler overrideproject: "https://github.com/orgs/my-mona-org/projects/1"

All three levels:

github-token: ${{ secrets.TOP_LEVEL_TOKEN }} # Workflow-wide defaultsafe-outputs:
github-token: ${{ secrets.SAFE_OUTPUTS_GITHUB_TOKEN }} # Safe-outputs defaultcreate-issue:
github-token: ${{ secrets.AGENT_GITHUB_TOKEN }} # Handler overrideupdate-project:
project: "https://github.com/orgs/my-mona-org/projects/1"# Uses safe-outputs token

Behavior

All handlers now accept github-token in frontmatter. Individual tokens are preserved in handler config JSON. The github-script step uses the highest-priority token (project handlers > safe-outputs level > top-level) for Octokit authentication.

Enables workflows to use different tokens for different permissions scopes (e.g., standard token for issues, PAT for Projects v2).


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

CopilotAIand others added 2 commits February 10, 2026 18:11
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@pelikhan

pelikhan commented Feb 10, 2026

Copy link
Copy Markdown
Collaborator

@copilot also support safe-outputs level github token:

safe-outputs:
github-token: ${{ secrets.SAFE_OUTPUT_GITHUB_TOKEN }}
create-issue:
github-token: ${{ secrets.AGENT_GITHUB_TOKEN }}
update-project:
github-token: ${{ secrets.PROJECT_GITHUB_TOKEN }}
Add tests for all scenarios and combinations.

CopilotAIand others added 2 commits February 10, 2026 18:22
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
CopilotAI changed the title [WIP] Investigate safe output combination compilation issueAdd github-token support to all safe-output handlersFeb 10, 2026
CopilotAI requested a review from mnkieferFebruary 10, 2026 18:26

@pelikhanpelikhan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a second thought, this is not correct in the new unified javascript handler since they all use the same token aside from agent task and project.

- Test safe-outputs level token with handlers
- Test handler-level overriding safe-outputs level
- Test all three levels (handler > safe-outputs > top-level)
- Test project handlers with safe-outputs token
- Test example from issue (safe-outputs + handler tokens)
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI commented Feb 10, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot also support safe-outputs level github token:

...

Added comprehensive test coverage in commit 76373d2. The safe-outputs level github-token was already supported in the code and schema - I've added tests for all scenarios:

✅ Safe-outputs level token (already worked)
✅ Handler-level override (from original PR)
✅ All three levels together (new test)
✅ Your exact example scenario (new test)

Token precedence: handler-level > safe-outputs-level > top-level

All tests pass:

  • TestGitHubTokenPrecedenceAllLevels - 5 scenarios covering all combinations
  • TestSafeOutputsLevelGitHubToken - Your example configuration
  • Plus existing tests for safe-outputs and top-level tokens

…UB_TOKEN
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
CopilotAI changed the title Add github-token support to all safe-output handlersAdd github-token support to all safe-output handlers with comprehensive test coverageFeb 10, 2026
CopilotAI requested a review from pelikhanFebruary 10, 2026 18:35
@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Triage Results

Category: bug | Risk: high | Priority: 73/100

Scores Breakdown

  • Impact: 50/50 - Critical bug fix for safe-outputs configuration schema. Affects 17 of 21 handlers that were incorrectly missing github-token support, blocking valid configurations.
  • Urgency: 10/30 - Recent PR (0.4 hours old) with bug severity. Currently draft with pending CI.
  • Quality: 13/20 - Good quality with test coverage and comprehensive documentation updates. Includes schema changes and validation tests. CI pending.

📋 Recommended Action: fast_track

Rationale: This is a high-priority bug fix that resolves schema validation errors preventing users from configuring per-handler GitHub tokens. The changes span schemas, compiler logic, and documentation, requiring thorough review. The systematic approach (updating 17 handler schemas) and test coverage support fast-track review.

Next Steps:

  1. ✅ Wait for CI to complete
  2. Review schema changes for consistency across all handlers
  3. Verify backward compatibility with existing configurations
  4. Test with real safe-outputs configurations
  5. Consider merging once CI passes and review is complete

⚠️ Note: Schema changes require binary rebuild (make build). Ensure compiled workflows are regenerated after merge.


Triaged by PR Triage Agent on 2026-02-10T18:33:07Z

AI generated by PR Triage Agent

@pelikhan

Copy link
Copy Markdown
Collaborator

not making this change currently.

@github-actions
github-actionsBot deleted the copilot/investigate-safe-output-compile branch April 2, 2026 13:09
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@pelikhan@mnkiefer