Uh oh!
There was an error while loading. Please reload this page.
chore: remove user-invokable and disable-model-invocation from the gh-aw schema - #38328
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
user-invokable and disable-model-invocation from the gh-aw schema🧠 Matt Pocock Skills Reviewer was skipped during the skills-based review. |
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #38328 does not have the implementation label and has only 2 new lines of code in business logic directories (threshold is 100). |
🧪 Test Quality Sentinel completed test quality analysis. No test functions were added or modified in PR #38328. The only test change is a deletion of a table-driven test row (user-invokable case) from TestFilterIgnoredFields in pkg/parser/schema_utilities_test.go — which correctly removes coverage for the schema field being deleted. Test Quality Sentinel skipped. |
✅ PR Code Quality Reviewer completed the code quality review. |
There was a problem hiding this comment.
Pull request overview
Removes acceptance of two Copilot custom-agent frontmatter fields (user-invokable, disable-model-invocation) that are not meaningful in gh-aw, ensuring they now fail schema validation instead of being silently tolerated, and updates editor autocomplete metadata accordingly.
Changes:
- Clear
constants.IgnoredFrontmatterFieldssouser-invokableis no longer filtered prior to schema validation. - Update tests/docs to reflect the removal of the ignored-field behavior.
- Remove stale
disable-model-invocationentries from the editor autocomplete data and its generator.
Show a summary per file
| File | Description |
|---|---|
| pkg/constants/constants.go | Empties IgnoredFrontmatterFields so unknown frontmatter keys are no longer silently ignored. |
| pkg/parser/schema_utilities_test.go | Removes the user-invokable filtering test case to match new validation behavior. |
| pkg/constants/README.md | Updates documented value of IgnoredFrontmatterFields. |
| docs/scripts/generate-autocomplete-data.js | Removes disable-model-invocation from root key sort order for generated autocomplete data. |
| docs/public/editor/autocomplete-data.json | Removes stale disable-model-invocation autocomplete entry and sort ordering. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 5/5 changed files
- Comments generated: 1
| "mode": { | ||
| "type": "string", | ||
| "desc": "Integration mode: 'cli' (recommended) installs @playwright/cli via npm for token-efficient CLI invocations — use play...", | ||
| "desc": "Integration mode: 'cli' (recommended) installs @playwright/cli via npm for token-efficient CLI invocations \u2014 use play...", |
There was a problem hiding this comment.
Two blocking correctness bugs — must fix before merge
1. Codemod produces a field that no longer exists in the schema (HIGH)
pkg/cli/codemod_infer_field.go is not modified by this PR but still unconditionally emits disable-model-invocation as the migration target for the deprecated infer field. After this PR, that field is gone from the schema. Any user who runs gh aw fix --write on a workflow that still uses infer: will get a workflow file that fails schema validation immediately. See inline comment on generate-autocomplete-data.js.
2. contribution-checker.agent.md will fail validation (HIGH)
.github/agents/contribution-checker.agent.md still declares user-invokable: false. Emptying IgnoredFrontmatterFields removes the silent-filter that previously stripped this field before schema validation. The schema uses additionalProperties: false, so that file will now fail validation. See inline comment on constants.go.
Non-blocking observations
Stale docs (MEDIUM): docs/src/content/docs/reference/frontmatter-full.md:1679 still shows disable-model-invocation: true as a valid example. This is misleading once the field is removed from the schema.
Outdated ADR (MEDIUM): docs/adr/33808-remove-deprecated-infer-field.md asserts that disable-model-invocationis the canonical field and that the codemod must produce it. That ADR should be superseded or annotated as obsolete.
🔎 Code quality review by PR Code Quality Reviewer · ⌖ 13.9 AIC
| // IgnoredFrontmatterFields are fields that should be silently ignored during frontmatter validation | ||
| // NOTE: user-invokable is a GitHub Copilot custom agent field that is not part of the gh-aw schema | ||
| var IgnoredFrontmatterFields = []string{"user-invokable"} | ||
| var IgnoredFrontmatterFields = []string{} |
There was a problem hiding this comment.
Breaks validation for contribution-checker.agent.md: removing user-invokable from IgnoredFrontmatterFields means the field is no longer silently stripped before schema validation, but .github/agents/contribution-checker.agent.md still declares user-invokable: false in its frontmatter.
💡 Details
Schema validation runs filterIgnoredFields(frontmatter) before passing the map to the JSON schema validator. The schema uses additionalProperties: false, so any key that is not explicitly defined will produce a validation error. Until contribution-checker.agent.md is updated (or until user-invokable is added to the schema as a valid field), any compile/validate run against that file will fail.
Either:
- Remove
user-invokable: falsefrom.github/agents/contribution-checker.agent.mdin this PR, or - Keep
user-invokableinIgnoredFrontmatterFieldsuntil all in-repo consumers are cleaned up.
# Confirms the affected file
grep -r "user-invokable" .github/ --include="*.md"# .github/agents/contribution-checker.agent.md:3:user-invokable: false| 'timeout-minutes', 'concurrency', 'environment', 'container', 'services', | ||
| 'network', 'sandbox', 'plugins', 'if', 'steps', 'post-steps', | ||
| 'features', 'disable-model-invocation', 'secrets', | ||
| 'features', 'secrets', |
There was a problem hiding this comment.
Codemod infer-to-disable-model-invocation now produces invalid frontmatter: pkg/cli/codemod_infer_field.go is untouched by this PR and still rewrites infer: <bool> → disable-model-invocation: <!bool>. Since disable-model-invocation is removed from the schema here, running gh aw fix --write on any workflow that contains infer: will inject an unknown field that immediately fails schema validation.
💡 Details
getInferToDisableModelInvocationCodemod() (line 86 of codemod_infer_field.go) unconditionally writes:
newLine:=fmt.Sprintf("%sdisable-model-invocation: %v", indent, disableValue)After this PR that field is gone from the schema, so the post-codemod file will fail ValidateMainWorkflowFrontmatterWithSchemaAndLocation. The codemod either needs to be:
- Updated to delete the
inferfield without producing a replacement (if the concept is gone entirely), or - Retargeted at whatever the new canonical field is.
The ADR at docs/adr/33808-remove-deprecated-infer-field.md also documents disable-model-invocation as the permanent canonical form; that ADR needs to be superseded or noted as obsolete.
| "enum": [true, false], | ||
| "leaf": true | ||
| }, | ||
| "secrets": { |
There was a problem hiding this comment.
Bundled CLI template files still use disable-model-invocation: true — will fail schema validation: Two embedded data files compiled into the CLI binary are not updated by this PR:
pkg/cli/data/agentic_workflow_designer_skill.md:4—disable-model-invocation: truepkg/cli/data/agentic_workflows_agent.md:4—disable-model-invocation: true
💡 Details
Removing disable-model-invocation from the schema without updating these embedded files means any code path that parses or validates these bundled templates will produce a schema error at runtime. Since these files are embedded in the binary via //go:embed or similar, they will silently break scaffolding/agent-creation flows that depend on them.
Both lines must be removed (or the field must be replaced with its functional equivalent) before this PR is safe to merge.
pelikhan
commented
Jun 10, 2026
@copilot run pr-finisher skill Custom agents.md should not be validated against the main JSOK schema. |
``
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in 8218c9f. |
These two Copilot custom-agent frontmatter fields have no meaning in gh-aw and should not be accepted. Additionally, custom agent files (
.github/agents/*.md) are now explicitly excluded from main workflow schema validation, since they follow the Copilot agent format rather than the gh-aw workflow format.Changes
pkg/constants/constants.go— emptiesIgnoredFrontmatterFields;user-invokableis no longer silently swallowed before schema validation and will now produce anadditionalPropertieserror like any other unknown fieldpkg/parser/schema_utilities_test.go— drops theuser-invokablefilter test casedocs/public/editor/autocomplete-data.json— removesdisable-model-invocationfromrootandsortOrder(field was already absent frommain_workflow_schema.json; the autocomplete data was stale)docs/scripts/generate-autocomplete-data.js— removesdisable-model-invocationfromROOT_SORT_ORDERpkg/constants/README.md— updates the documented value ofIgnoredFrontmatterFieldspkg/parser/schema_validation.go— skips main workflow schema validation for custom agent files (.github/agents/*.md); these files use the Copilot agent format and must not be validated against the gh-aw schemapkg/parser/schema_location_test.go— adds test coverage confirming that Copilot-specific fields in custom agent files do not produce schema validation errorsNo migration path or codemod provided; these fields should not be used in gh-aw workflow files.