Skip to content

Add model policy frontmatter + import unioning + env policy overrides - #41824

Merged
pelikhan merged 35 commits into
mainfrom
copilot/add-frontmatter-models-fields
Jun 29, 2026
Merged

Add model policy frontmatter + import unioning + env policy overrides#41824
pelikhan merged 35 commits into
mainfrom
copilot/add-frontmatter-models-fields

Conversation

CopilotAI commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

This change introduces model policy controls in workflow frontmatter (models.allowed, models.disallowed) and maps them to AWF’s allowedModels / disallowedModels config. It also makes policy behavior import-safe by unioning model sets across composed workflows, with centralized environment overrides taking precedence.

  • Frontmatter + schema support

    • Extended models frontmatter schema to support policy fields alongside optional pricing providers.
    • Added typed parsing for:
      • models.allowed
      • models.disallowed
  • Import compatibility (union semantics)

    • Extended import extraction/results to carry model policy sets from imported workflows.
    • Added workflow merge logic to union policy sets across imports + main workflow.
    • Added conflict handling so disallowed takes precedence when the same model appears in both allowed and disallowed sets.
  • Cost data cleanliness + parse warnings

    • Tightened import-side model-cost extraction to only accept valid non-empty models.providers objects.
    • Added import warnings for invalid models.allowed / models.disallowed / models.providers shapes and invalid entries, while safely skipping bad values.
  • Centralized policy overrides

    • Added compiler env overrides:
      • GHAW_POLICY_MODELS_ALLOWED
      • GHAW_POLICY_MODELS_DISALLOWED
    • Override values are parsed as model lists and applied with precedence over frontmatter/import-derived policy.
  • AWF config mapping

    • Emitted merged/effective policy into AWF config:
      • apiProxy.allowedModels
      • apiProxy.disallowedModels
# workflow frontmattermodels:
allowed: [gpt-5, claude-sonnet]disallowed: [gpt-5-pro]
// generated AWF apiProxy fragment
{
"allowedModels": ["gpt-5", "claude-sonnet"],
"disallowedModels": ["gpt-5-pro"]
}

pr-sous-chef https://github.com/github/gh-aw/actions/runs/28333890571

Generated by 👨‍🍳 PR Sous Chef · 49.3 AIC · ⌖ 1.05 AIC · ⊞ 17.3K ·


pr-sous-chef: requested branch update via https://github.com/github/gh-aw/actions/runs/28336535351

Generated by 👨‍🍳 PR Sous Chef · 57.9 AIC · ⌖ 1.4 AIC · ⊞ 17.3K ·


pr-sous-chef: refresh branch for workflow run 28338043309

Generated by 👨‍🍳 PR Sous Chef · 47.3 AIC · ⌖ 1.47 AIC · ⊞ 17.3K ·


Generated by 👨‍🍳 PR Sous Chef · 67.9 AIC · ⌖ 1.53 AIC · ⊞ 17.6K ·


pr-sous-chef run: https://github.com/github/gh-aw/actions/runs/28381497646

Generated by 👨‍🍳 PR Sous Chef · 72.2 AIC · ⌖ 1.02 AIC · ⊞ 17.6K ·


Generated by 👨‍🍳 PR Sous Chef · 172.6 AIC · ⌖ 10.9 AIC · ⊞ 6.7K ·

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review June 27, 2026 01:19
CopilotAI review requested due to automatic review settings June 27, 2026 01:19

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end “model policy” support to gh-aw workflows, allowing authors (and centralized operators via env vars) to control which models are permitted/blocked, and ensuring policies compose safely across imported workflows before being emitted into the generated AWF config.

Changes:

  • Extended workflow frontmatter models to support allowed, disallowed, and blocked policy lists (alongside optional pricing providers).
  • Propagated model policy through import extraction and merged policies across imports + main workflow using union semantics.
  • Emitted effective model policy to AWF config (apiProxy.allowedModels / apiProxy.disallowedModels) with env override precedence.
Show a summary per file
FileDescription
pkg/workflow/workflow_builder.goExtracts main workflow model policy and unions it with imported policy sets into WorkflowData.
pkg/workflow/workflow_builder_model_policy_test.goAdds unit tests for policy extraction and union merge behavior.
pkg/workflow/model_aliases_test.goVerifies frontmatter parsing populates parsed model policy lists.
pkg/workflow/frontmatter_types.goAdds parsed frontmatter fields for model policy lists.
pkg/workflow/frontmatter_parsing.goParses model policy lists from raw frontmatter into typed config.
pkg/workflow/compilerenv/manager.goAdds env-driven policy overrides for allowed/blocked model sets.
pkg/workflow/compilerenv/manager_test.goTests env override parsing and “unset” behavior.
pkg/workflow/compiler_types.goPlumbs merged model policy into WorkflowData.
pkg/workflow/awf_config.goMaps effective model policy (with env precedence) into AWF apiProxy config.
pkg/workflow/awf_config_test.goTests AWF config emission and env override precedence.
pkg/parser/schemas/main_workflow_schema.jsonUpdates schema for models to include policy fields and make providers optional.
pkg/parser/import_processor.goExtends ImportsResult to carry extracted model policy sets.
pkg/parser/import_field_extractor.goExtracts model policy from imported workflows and avoids treating policy keys as aliases.
pkg/parser/import_field_extractor_test.goAdds tests ensuring model policy is extracted (and not misinterpreted as aliases) and can coexist with model costs.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

pkg/parser/import_field_extractor.go:639

  • When an imported workflow has models.providers plus model policy keys (allowed/disallowed/blocked), this appends the entire rawModels object into acc.modelCosts. That will later flow into WorkflowData.ModelCosts and into GH_AW_INFO_MODEL_COSTS, leaking policy keys into a payload that is expected to match the models.json pricing structure (providers-only). This can break downstream cost merging/parsing.
	if _, hasProviders := rawModels["providers"]; hasProviders {
acc.modelCosts = append(acc.modelCosts, rawModels)
if providers, ok := rawModels["providers"].(map[string]any); ok {
parserLog.Printf("Extracted model costs from import: providers=%d", len(providers))
} else {

pkg/workflow/workflow_builder.go:166

  • Now that models frontmatter can contain policy keys (allowed/disallowed/blocked) without providers, toolsResult.parsedFrontmatter.ModelCosts may be non-empty even when there is no pricing data (because it unmarshals the whole models object). extractMainModelCostsOverlay currently returns that map as a cost overlay, which can cause policy-only models to be emitted as GH_AW_INFO_MODEL_COSTS and/or pollute the providers overlay with policy keys.
	}
return workflowData
}
  • Files reviewed: 14/14 changed files
  • Comments generated: 0
  • Review effort level: Low

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Categoryfeature
Risk🔴 High
PriorityHigh (score 50/100)
Actiondefer · Batch: active-drafts

Score breakdown: Impact 35 · Urgency 5 · Quality 10

New model policy frontmatter controls (models.allowed/disallowed/blocked) with import-safe union semantics (+483/-11, 14 files). Just created (<1 h), draft, no CI yet. Well-described and scoped. Deferred pending CI and agent completion. Part of pr-batch:active-drafts group (#41824, #41822, #41821).

Generated by 🔧 PR Triage Agent · 87.6 AIC · ⌖ 14.9 AIC · ⊞ 5.4K ·

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — great work on the model policy frontmatter feature! The addition of models.allowed, models.disallowed, and models.blocked to workflow frontmatter — along with import-safe union semantics and centralized env overrides — is a well-scoped, coherent change.

The PR is well-structured:

  • ✅ Clear, detailed description with YAML/JSON examples showing the before/after config shape.
  • ✅ Tests across all touched layers: import_field_extractor_test.go, awf_config_test.go, compilerenv/manager_test.go, model_aliases_test.go, and the newly added workflow_builder_model_policy_test.go.
  • ✅ No unrelated changes — every file touched is in service of the model policy feature.
  • ✅ No new external dependencies introduced.

This looks ready for review. 🚀

Generated by ✅ Contribution Check · 307.9 AIC · ⌖ 20.6 AIC · ⊞ 6K ·

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

/review

@github-actions

github-actionsBot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot mark fields as experimental

@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Triage — §28395315609

FieldValue
Categoryfeature
Riskhigh
Score54 / 100
Actionbatch_review
Batch

Score breakdown: impact 30 + urgency 16 + quality 8

Carried over (66h). Model policy frontmatter (allowed/disallowed lists + env overrides). CHANGES_REQUESTED from CI bot (dismissed approval). Priority corrected: pr-priority:highpr-priority:medium (score 54). ⚠️ Multiple conflicting action labels from prior runs (pr-action:defer, pr-action:fast_track, pr-action:batch_review) — manual label cleanup recommended. Needs CI resolution before merge.

Generated by 🔧 PR Triage Agent · 99.1 AIC · ⌖ 11.6 AIC · ⊞ 5.4K ·

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI commented Jun 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot mark fields as experimental

Done in a83a52b.

Marked the model policy fields as experimental in the frontmatter schema descriptions (models.allowed, models.disallowed) and in related model-policy/env override code comments.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot rename disallowed to blocked to match safe output vocabulary

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI commented Jun 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot rename disallowed to blocked to match safe output vocabulary

Addressed in a916dbc.

Renamed the model policy denylist vocabulary from disallowed to blocked across frontmatter parsing, import policy merging, schema/tests/env override naming (GHAW_POLICY_MODELS_BLOCKED), and updated smoke-claude (models.blocked: ["*opus*"]). The generated lock file was recompiled accordingly.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill to drive this PR to a mergeable state.

Unresolved review feedback:

  • github-actions (Matt Pocock Skills Reviewer) — CHANGES_REQUESTED: asymmetric wildcard behaviour in modelConflictsWithDisallowedPolicy, silent drops without warnings in parseModelPolicyList/appendModelsField, index-based warning assertion fragility, and missing maintainability comments on resolveModelPolicyForAWFConfig/extractMainModelPolicyOverlay.

Branch refresh: This branch is 24 commits behind main — please rebase or merge main to bring it current.

Generated by 👨‍🍳 PR Sous Chef · 172.6 AIC · ⌖ 10.9 AIC · ⊞ 6.7K ·

…er-models-fields
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

CopilotAI commented Jun 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot please run the pr-finisher skill to drive this PR to a mergeable state.

Unresolved review feedback:...

Updated in e138fea by refreshing this branch from main and resolving the merge cleanly. I re-ran required local validation (make fmt, make lint, make test-unit, make test, make recompile) plus make agent-report-progress, and ran final parallel_validation (Code Review + CodeQL). I found no remaining actionable code changes from that review set in the current branch state.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot There is one unresolved CHANGES_REQUESTED review blocking this PR that needs to be addressed before merge:

The blocking findings from that review are:

  1. Asymmetric wildcard behaviour (modelConflictsWithDisallowedPolicy): wildcard entries in the allowed list are never expanded — only the disallowed side applies pattern matching. This is non-obvious, untested, and can silently permit models that should be blocked.
  2. Silent drops without warnings: parseModelPolicyList ignores non-string frontmatter entries without emitting a warning (unlike the import path which does warn); appendModelsField silently drops model aliases when providers is present.
  3. Test fragility: index-based warning assertions (acc.warnings[0]) will panic on re-ordering rather than producing a clear failure message.
  4. Missing maintainability comments: resolveModelPolicyForAWFConfig and extractMainModelPolicyOverlay lack the inline comments that the equivalent cost-overlay functions already carry.

Please address these issues, then invoke the pr-finisher skill to validate and push the PR toward merge.

Generated by 👨‍🍳 PR Sous Chef · 125.6 AIC · ⌖ 7.7 AIC · ⊞ 6.7K ·

CopilotAI commented Jun 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot There is one unresolved CHANGES_REQUESTED review blocking this PR that needs to be addressed before merge:

I reviewed this blocking checklist against the current branch head (e138fea) and confirmed these items are already addressed:

  • wildcard policy conflict handling is symmetric in modelConflictsWithDisallowedPolicy (both directions are checked)
  • import appendModelsField now preserves aliases when providers is present (covered by TestAppendModelsField_ProvidersAndAliasesBothExtracted)
  • warning assertions in the model-policy/import tests are no longer index-fragile (they assert via joined warning text)
  • maintainability comments are present on both resolveModelPolicyForAWFConfig and extractMainModelPolicyOverlay

I also ran local validation on this head: make fmt, make lint, make test-unit, make test, make recompile, plus final parallel_validation.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot review changes and ensure that environment variables and policy model rules override local policies. For allowed rules, use intersection; for blocked rules, use union.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot this PR looks ready for forward progress! Checks are ✅ passing and the branch is 🟢 up-to-date with base (mergeStateStatus: CLEAN). No unresolved reviews.

Please run the pr-finisher skill to finalize this PR and get it ready to merge.

Generated by 👨‍🍳 PR Sous Chef · 84.1 AIC · ⌖ 10.1 AIC · ⊞ 6.7K ·

CopilotAIand others added 2 commits June 29, 2026 21:47
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI commented Jun 29, 2026

Copy link
Copy Markdown
ContributorAuthor

@copilot review changes and ensure that environment variables and policy model rules override local policies. For allowed rules, use intersection; for blocked rules, use union.

Implemented in 423d8dd (plus follow-up eb6ed89). Env policy now combines with workflow policy as requested: allowed uses intersection (narrowing) and blocked uses union (widening), then blocked-vs-allowed conflict filtering is applied. Added targeted coverage for intersection/union and empty-intersection behavior, and re-ran make fmt, make lint, and make test-unit.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pelikhan@gh-aw-bot