Default `runs-on` for non-reusable custom jobs when omitted by Copilot · Pull Request #48397 · github/gh-aw

Copilot · 2026-07-27T15:42:42Z

A workflow in gh-aw-test failed to compile because a custom job using restore-memory: true omitted runs-on, and the generated .lock.yml emitted that job without a runner. GitHub Actions rejects such jobs with Required property is missing: runs-on.

Root cause
- Non-reusable custom jobs did not inherit any runner when jobs.<id>.runs-on was absent.
- restore-memory path made this easy to hit in deterministic setup jobs.
Compiler behavior change
- In non-reusable custom job compilation, when job.RunsOn is empty:
  - use workflow-level data.RunsOn if present
  - otherwise fallback to runs-on: ubuntu-latest
- Reusable-workflow call jobs (uses:) are unchanged.
Regression coverage
- Added a targeted test for restore-memory custom jobs without explicit runs-on to ensure generated job YAML always includes a runner.

jobs:
setup:
restore-memory: truesteps:
- run: echo "ok"

With this change, the compiled setup job includes a valid runs-on stanza inherited from workflow defaults.

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 9.88 AIC · ⌖ 7.74 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

pr-sous-chef run: https://github.com/github/gh-aw/actions/runs/30293408015

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 28.1 AIC · ⌖ 8.01 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 15.6 AIC · ⌖ 7.45 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 31.8 AIC · ⌖ 8.29 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Run: https://github.com/github/gh-aw/actions/runs/30302221322

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 13 AIC · ⌖ 6.16 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Run: https://github.com/github/gh-aw/actions/runs/30304405147

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 15.2 AIC · ⌖ 6.68 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 15.5 AIC · ⌖ 7.12 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

run: https://github.com/github/gh-aw/actions/runs/30308333914

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 18.8 AIC · ⌖ 7.74 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Run: https://github.com/github/gh-aw/actions/runs/30311684823

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.1 AIC · ⌖ 6.51 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 13.9 AIC · ⌖ 6.46 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Run: https://github.com/github/gh-aw/actions/runs/30316592253

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 14.3 AIC · ⌖ 9.66 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Defaults non-reusable custom jobs to the workflow runner or ubuntu-latest when runs-on is omitted.

Changes:

Adds custom-job runner fallback logic.
Adds regression coverage for restore-memory jobs.

Show a summary per file

File	Description
`pkg/workflow/compiler_custom_jobs.go`	Applies default runner configuration.
`pkg/workflow/compiler_custom_job_memory_test.go`	Tests omitted `runs-on` behavior.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 1
Review effort level: Medium

github-actions · 2026-07-27T17:16:22Z

✅ Test Quality Sentinel completed test quality analysis.

github-actions · 2026-07-27T17:16:37Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

github-actions · 2026-07-27T17:17:24Z

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #48397 does not have the 'implementation' label and has 48 new lines of code in business logic directories (below the 100-line threshold).

github-actions · 2026-07-27T17:19:02Z

✅ PR Code Quality Reviewer completed the code quality review.

github-actions

Skills-Based Review 🧠

Applied /tdd — requesting one change on test coverage.

📋 Key Themes & Highlights

Key Themes

Partial branch coverage: the new test only exercises the ubuntu-latest fallback; the data.RunsOn inheritance branch is untested.
Existing comment (Copilot, line 481): data.RunsOn may be multi-line, which the direct assignment does not indent correctly — that concern is already filed.

Positive Highlights

✅ Clean, targeted fix with a clear fallback chain
✅ Regression test added in the right file alongside similar memory-job tests
✅ PR description clearly explains root cause, compiler behavior change, and test coverage

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 27.7 AIC · ⌖ 4.63 AIC · ⊞ 6.7K
_{Comment /matt to run again}

github-actions · 2026-07-27T17:21:45Z


 func (c *Compiler) configureCustomJobSteps(job *Job, jobName string, configMap map[string]any, data *WorkflowData) error {
+	if job.RunsOn == "" {
+ job.RunsOn = data.RunsOn


[/tdd]data.RunsOn inheritance path is untested — only the ubuntu-latest fallback is exercised by the new test.
The new test omits a workflow-level runs-on:, so the job.RunsOn = data.RunsOn branch (line 481) is never taken. A regression there would go undetected.
💡 Suggested additional test case
Add a second test that sets a workflow-level runs-on: self-hosted in the frontmatter and asserts the compiled setup job inherits runs-on: self-hosted instead of the ubuntu-latest fallback. This covers both branches of the new logic.
@copilot please address this.

github-actions · 2026-07-27T17:23:47Z

🧪 Test Quality Sentinel Report

⚠️Test Quality Score: 65/100 — Acceptable

Analyzed 15 test(s): 10 design, 5 implementation, 1 violation.

📊 Metrics (15 tests)

Metric	Value
Analyzed	15 (Go: 15, JS: 0)
✅ Design	10 (67%)
⚠️ Implementation	5 (33%)
Edge/error coverage	9 (60%)
Duplicate clusters	0
Inflation	YES (594 test lines / 194 prod lines ≈ 3.1:1)
🚨 Violations	1 (implementation ratio > 30%)

Test	File	Classification	Issues
`TestCustomJobRestoreMemoryCacheMemory`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryUsesDefaultRunsOn`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryRepoMemory`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryCommentMemory`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryMultipleTypes`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryStepOrder`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryErrorWhenNotConfigured`	compiler_custom_job_memory_test.go	design_test (error path)	—
`TestCustomJobRestoreMemoryOnlyEmitsRestoreSteps`	compiler_custom_job_memory_test.go	design_test	—
`TestCustomJobRestoreMemoryStandaloneJob`	compiler_custom_job_memory_test.go	design_test	—
`TestExtractRestoreMemoryConfig`	compiler_custom_job_memory_test.go	design_test (table-driven, 7 rows)	—
`TestGenerateCacheMemoryRestoreLines`	compiler_custom_job_memory_test.go	implementation_test	Internal function test
`TestGenerateCacheMemoryRestoreLinesNilData`	compiler_custom_job_memory_test.go	implementation_test	Nil-guard only
`TestBuildCacheRestoreKeysEmptyForSinglePartKey`	compiler_custom_job_memory_test.go	implementation_test	Internal key-builder detail
`TestGenerateRepoMemoryRestoreLinesNilData`	compiler_custom_job_memory_test.go	implementation_test	Nil-guard only
`TestGenerateCommentMemoryRestoreLinesNilData`	compiler_custom_job_memory_test.go	implementation_test	Nil-guard only

⚠️ Flagged Tests (5)

TestGenerateCacheMemoryRestoreLines (compiler_custom_job_memory_test.go:542) — implementation_test, low_value. Tests an internal helper function directly; behavior is fully covered by the higher-level TestCustomJobRestoreMemoryCacheMemory.

TestGenerateCacheMemoryRestoreLinesNilData (compiler_custom_job_memory_test.go:566) — implementation_test, low_value. Nil-guard on an internal function already covered indirectly when no cache config is set.

TestBuildCacheRestoreKeysEmptyForSinglePartKey (compiler_custom_job_memory_test.go:573) — implementation_test, low_value. Tests an internal helper with no observable behavioral impact at the compiler level.

TestGenerateRepoMemoryRestoreLinesNilData (compiler_custom_job_memory_test.go:581) — implementation_test, low_value. Nil/empty guards on an internal helper.

TestGenerateCommentMemoryRestoreLinesNilData (compiler_custom_job_memory_test.go:589) — implementation_test, low_value. Same nil-guard pattern as the repo-memory variant.

Inflation flag — compiler_custom_job_memory_test.go (594 lines) vs compiler_custom_job_memory.go (194 lines) → ratio ≈ 3.1:1 (threshold 2:1). Consolidating nil-guard tests would bring the ratio into range.

Verdict

❌ Failed. 33% implementation tests (threshold: 30%). The ten behavioral contract tests covering restore-memory compilation are excellent. The five internal-helper tests push the ratio just over threshold — consolidating them would bring this into passing range.

🧪 Test quality analysis by Test Quality Sentinel · sonnet46 · 69.4 AIC · ⌖ 7.87 AIC · ⊞ 8.1K · ◷
_{Comment /review to run again}

github-actions

❌ Test Quality Sentinel: 65/100. 33% implementation tests (threshold: 30%). Review flagged tests in the comment above.

github-actions

Review: Default `runs-on` for non-reusable custom jobs

Overall: The fix is correct and targeted. One non-blocking reliability concern worth addressing:

Multi-line runs-on indentation (line 481):

data.RunsOn is a pre-rendered YAML snippet. When the workflow declares a complex runner (group/labels object), the snippet spans multiple lines. Inheriting it via job.RunsOn = data.RunsOn and emitting through Job.Write's fmt.Fprintf(b, " %s\n", job.RunsOn) mis-indents continuation lines (4 spaces instead of 6). No test exercises this path.

Consider using normalizeRunsOnSnippet or formatIndentedYAMLField (both used elsewhere) before the assignment.

Test gap: The new test only exercises the fallback path (ubuntu-latest). A second case with an explicit workflow-level runs-on label array would validate the inheritance path for non-trivial runner forms.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 41.8 AIC · ⌖ 5.3 AIC · ⊞ 5K

github-actions · 2026-07-27T17:24:37Z


 func (c *Compiler) configureCustomJobSteps(job *Job, jobName string, configMap map[string]any, data *WorkflowData) error {
+	if job.RunsOn == "" {
+ job.RunsOn = data.RunsOn


The inherited data.RunsOn snippet can be a multi-line YAML block (e.g., a matrix runner or group/labels object). Assigning it directly to job.RunsOn and emitting it via fmt.Fprintf(b, " %s ", job.RunsOn) in Job.Write will mis-indent continuation lines—they will render at 4 spaces instead of the required 6.
Consider normalizing with the same helper used elsewhere (e.g., normalizeRunsOnSnippet or formatIndentedYAMLField) before assigning, or document that data.RunsOn is guaranteed to be a pre-indented snippet.
@copilot please address this.

github-actions

REQUEST_CHANGES — fix is mostly correct but has a correctness bug and a gap in test coverage.

### Findings

High — broken YAML for multi-line runs-on inheritance (compiler_custom_jobs.go:481)

data.RunsOn is a raw multi-line YAML snippet. Every other site that feeds it into a Job struct calls c.indentYAMLLines(data.RunsOn, " ") first. This line assigns it bare, so array/object runner forms will produce invalid YAML with unindented continuation lines. The hardcoded "runs-on: ubuntu-latest" fallback is fine; only the data.RunsOn branch needs the indent call.

Medium — test misses the actual bug path (compiler_custom_job_memory_test.go:46)

The new test exercises the ubuntu-latest hardcoded fallback (no workflow-level runs-on present), not the data.RunsOn inheritance path. A test with a complex (array/object) workflow-level runner would catch the indentation regression.

🔎 Code quality review by PR Code Quality Reviewer · sonnet46 · 45.1 AIC · ⌖ 4.83 AIC · ⊞ 5.7K
_{Comment /review to run again}

Comments that could not be inline-anchored

pkg/workflow/compiler_custom_jobs.go:481

Missing indentation for multi-line runs-on fallback: assigning data.RunsOn directly will produce broken YAML when the workflow uses an array or object runs-on form.

<details>
<summary>💡 Details and fix</summary>

data.RunsOn stores a raw YAML snippet such as:

runs-on:
group: my-runnerslabels: [ubuntu-latest]

When job.RunsOn is written by jobs.go it is emitted as %s — a single 4-space prefix on the first line only. Multi-line continuation lines are not…

pkg/workflow/compiler_custom_job_memory_test.go:46

Test only covers ubuntu-latest fallback — the actual regression (multi-line runs-on) is untested: the new test verifies runs-on: ubuntu-latest is present, but the correctness bug is in the data.RunsOn branch, not the hardcoded fallback branch.

<details>
<summary>💡 Details</summary>

The test at line 46 asserts runs-on: ubuntu-latest — this hits the job.RunsOn = "runs-on: ubuntu-latest" fallback (line 483) which is fine. It does not exercise the job.RunsOn = data.RunsOn branc…

github-actions · 2026-07-27T17:29:45Z

Great work, @Copilot! 🎉 This fix looks solid — you have addressed a critical compiler bug where custom jobs without explicit runs-on would fail GitHub Actions validation.

What looks good:

Targeted fix: The logic is clear — inherit workflow-level runs-on, fall back to ubuntu-latest.
Regression test: The new TestCustomJobRestoreMemoryUsesDefaultRunsOn directly covers the restore-memory scenario that triggered this bug.
Minimal scope: Just 2 files, 48 lines. Easy to review and merge.

This PR is ready for review! 🚀

Generated by ✅ Contribution Check · sonnet46 · 63.8 AIC · ⌖ 7.55 AIC · ⊞ 6.2K · ◷

github-actions · 2026-07-27T18:54:09Z

🤖 PR Triage

Field	Value
Category	bug
Risk	low
Score	70/100 (impact:28, urgency:25, quality:17)
Action	auto_merge

Rationale: Fixes compiler omitting runs-on for custom jobs using restore-memory: true, causing GHA to reject the workflow. 2-file change, CI fully passing. Auto-merge candidate.

Generated by 🔧 PR Triage Agent · sonnet46 · 54.1 AIC · ⌖ 8.18 AIC · ⊞ 5.7K · ◷

Done

pelikhan · 2026-07-27T20:50:51Z

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-07-27T21:04:47Z

@copilot run pr-finisher skill

Applied the fix in commit Fix multi-line runs-on indentation for inherited custom job runner.

Changed job.RunsOn = data.RunsOn to job.RunsOn = c.indentYAMLLines(data.RunsOn, " ") so continuation lines of multi-line YAML (arrays/objects) are correctly indented at job depth.
Added TestCustomJobRestoreMemoryInheritsArrayRunsOn which exercises a group/labels object runner inherited from the workflow level and asserts every non-empty line inside the job section is indented with at least 4 spaces.

github-actions · 2026-07-28T00:58:25Z

🤖 PR Triage

Field	Value
Category	bug
Risk	🟢 low
Score	68/100 (impact:32 urgency:22 quality:14)
Action	✅ `auto_merge`

Adds default runs-on for non-reusable custom jobs when omitted, preventing compilation failures. Small change (104 adds, 0 dels, 2 files). CI passing (23 success, 5 skipped). Auto-merge candidate.

Generated by 🔧 PR Triage Agent · sonnet46 · 63.9 AIC · ⌖ 5.94 AIC · ⊞ 5.7K · ◷

github-actions · 2026-07-29T10:49:33Z

🎉 This pull request is included in a new release.

Release: v0.83.5

Default runs-on for non-reusable custom jobs
ddeb37d
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI assigned Copilot and pelikhan Jul 27, 2026

CopilotAI changed the title ~~Default missing runs-on for non-reusable custom jobs~~Default runs-on for non-reusable custom jobs when omittedJul 27, 2026

Copilotcreated this pull request from a session on behalf of pelikhan July 27, 2026 15:43 View session

Copilotfinished work on behalf of pelikhan July 27, 2026 15:43

CopilotAI requested a review from pelikhan July 27, 2026 15:43

pelikhan marked this pull request as ready for review July 27, 2026 16:55

CopilotAI review requested due to automatic review settings July 27, 2026 16:55

Copilotstarted reviewing on behalf of pelikhan July 27, 2026 16:56 View session

CopilotAI reviewed Jul 27, 2026
View reviewed changes

Comment threadpkg/workflow/compiler_custom_jobs.go Outdated

github-actionsBot requested changes Jul 27, 2026
View reviewed changes

github-actionsBot mentioned this pull request Jul 27, 2026
[aw] No-Op Runs #43499
Closed

github-actionsBot requested changes Jul 27, 2026
View reviewed changes

github-actionsBot reviewed Jul 27, 2026
View reviewed changes

github-actionsBot previously requested changes Jul 27, 2026
View reviewed changes

github-actionsBot mentioned this pull request Jul 27, 2026
[Contribution Check Report] Contribution Check — 2026-07-27 #48248
Closed

github-actionsBot mentioned this pull request Jul 27, 2026
[pr-sous-chef] PR Sous Chef run report #48239
Closed

github-actionsBot added pr-action:auto_merge pr-agent:copilot pr-priority:high pr-risk:low pr-type:bug labels Jul 27, 2026

Copilotstarted work on behalf of pelikhan July 27, 2026 20:51 View session

github-actionsBotand others added 2 commits July 27, 2026 20:59

Merge branch 'main' into copilot/investigate-aw-compilation-issue
1423936

Fix multi-line runs-on indentation for inherited custom job runner
b33bda2
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilotfinished work on behalf of pelikhan July 27, 2026 21:05

github-actionsBot mentioned this pull request Jul 28, 2026
[pr-sous-chef] pr-sous-chef run report #48464
Closed

pelikhan merged commit c24715d into mainJul 28, 2026
28 checks passed

pelikhan deleted the copilot/investigate-aw-compilation-issue branch July 28, 2026 01:26

Uh oh!

Conversation

CopilotAI commented Jul 27, 2026• edited by github-actions BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

github-actionsBot commented Jul 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Jul 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Jul 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Jul 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

github-actionsBotJul 27, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 27, 2026

🧪 Test Quality Sentinel Report

Verdict

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Review: Default runs-on for non-reusable custom jobs

Uh oh!

github-actionsBotJul 27, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Comments that could not be inline-anchored

Uh oh!

github-actionsBot commented Jul 27, 2026

Uh oh!

github-actionsBot commented Jul 27, 2026

🤖 PR Triage

Uh oh!

pelikhan commented Jul 27, 2026

Uh oh!

CopilotAI commented Jul 27, 2026

Uh oh!

github-actionsBot commented Jul 28, 2026

🤖 PR Triage

Uh oh!

Uh oh!

github-actionsBot commented Jul 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CopilotAI commented Jul 27, 2026•
edited by github-actions Bot
Loading

github-actionsBot commented Jul 27, 2026•
edited
Loading

github-actionsBot commented Jul 27, 2026•
edited
Loading

github-actionsBot commented Jul 27, 2026•
edited
Loading

github-actionsBot commented Jul 27, 2026•
edited
Loading

Review: Default `runs-on` for non-reusable custom jobs