Uh oh!
There was an error while loading. Please reload this page.
Enable Helix Job Monitor for the runtime pipeline - #129690
Merged
Merged
Conversation
Contributor
Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR wires up Helix Job Monitor support across the runtime Azure DevOps pipelines by introducing a useHelixMonitor parameter through the relevant YAML templates, enabling it for the runtime pipeline test legs, and adding the Helix Job Monitor .NET tool/dependency so the monitor job can run.
Changes:
- Add a
HelixJobMonitorjob toeng/pipelines/runtime.yml(Build stage) using the sharedeng/commontemplate. - Plumb a
useHelixMonitorparameter through Helix-related pipeline templates and pass/p:EnableHelixJobMonitor=...through the various “send to helix” paths. - Add
Microsoft.DotNet.Helix.JobMonitortool/dependency and pin it in.config/dotnet-tools.json.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| eng/Version.Details.xml | Adds the Microsoft.DotNet.Helix.JobMonitor toolset dependency entry. |
| eng/Version.Details.props | Adds version properties for the Helix Job Monitor dependency. |
| eng/pipelines/runtime.yml | Adds the Helix Job Monitor job and enables useHelixMonitor: true across runtime pipeline test legs. |
| eng/pipelines/libraries/run-test-job.yml | Introduces useHelixMonitor parameter and forwards it into the libraries Helix submission step. |
| eng/pipelines/libraries/helix.yml | Accepts useHelixMonitor and passes /p:EnableHelixJobMonitor=... into sendtohelix.proj. |
| eng/pipelines/installer/helix.yml | Adds useHelixMonitor, but currently passes UseHelixMonitor into /eng/common/templates/steps/send-to-helix.yml (template parameter mismatch; needs fix). |
| eng/pipelines/coreclr/nativeaot-post-build-steps.yml | Adds useHelixMonitor pass-through to the shared runtime test+Helix submit template. |
| eng/pipelines/common/templates/wasm-runtime-tests.yml | Adds and forwards useHelixMonitor into the Helix submission path. |
| eng/pipelines/common/templates/wasm-library-tests.yml | Adds and forwards useHelixMonitor into the Helix submission path. |
| eng/pipelines/common/templates/wasm-library-aot-tests.yml | Adds and forwards useHelixMonitor into nested wasm templates. |
| eng/pipelines/common/templates/wasm-coreclr-library-tests.yml | Adds and forwards useHelixMonitor into the Helix submission path. |
| eng/pipelines/common/templates/simple-wasm-build-tests.yml | Adds and forwards useHelixMonitor into the Helix submission path for WBT scenarios. |
| eng/pipelines/common/templates/runtimes/send-to-helix-step.yml | Adds useHelixMonitor and includes /p:EnableHelixJobMonitor=... in the computed send parameters. |
| eng/pipelines/common/templates/runtimes/run-test-job.yml | Adds useHelixMonitor parameter and forwards it to send-to-helix-step.yml. |
| eng/pipelines/common/templates/runtimes/build-runtime-tests-and-send-to-helix.yml | Adds useHelixMonitor parameter and forwards it to send-to-helix-step.yml. |
| eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml | Hard-enables useHelixMonitor: true for the CoreCLR Wasm.Build.Tests Helix submission. |
| eng/pipelines/common/templates/browser-wasm-build-tests.yml | Adds and forwards useHelixMonitor for non-CoreCLR WBT Helix submissions. |
| .config/dotnet-tools.json | Adds the microsoft.dotnet.helix.jobmonitor tool entry and pins its version. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
- Add useHelixMonitor parameter to all helix pipeline templates - Enable Helix Monitor in runtime.yml for all test legs - Add helix-job-monitor.yml template job at the top of the Build stage - Add Microsoft.DotNet.Helix.JobMonitor dependency (same version as Arcade) - Pass EnableHelixJobMonitor MSBuild property through send-to-helix Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
premunforce-pushed
the
prvysoky/helix-job-monitor-rebased
branch
from
June 22, 2026 13:35
20899d1 to
624f97dCompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Jun 22, 2026
Open
3 tasks
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 19 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml:145
- This template hard-codes
useHelixMonitor: truewhen invoking the libraries Helix template. Since other Helix templates were updated to accept auseHelixMonitorparameter (defaulting to false) andruntime.ymlis now explicitly setting it, this hard-codedtruemakes the behavior inconsistent and harder to roll back/experiment with.
Consider adding a useHelixMonitor parameter to this template (default false) and threading it through instead of hard-coding true.
testRunNamePrefixSuffix: CoreCLR_$(_BuildConfig)_$(_hostedOs)
extraHelixArguments: /p:BrowserHost=$(_hostedOs)
scenarios:
- buildwasmapps
useHelixMonitor: true
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 22 changed files in this pull request and generated no new comments.
Suppressed comments (3)
eng/pipelines/common/variables.yml:50
- enableHelixJobMonitor is set to true unconditionally, which turns on the new Helix monitor path for all runtime.yml runs immediately. The linked arcade rollout plan for HRJ/Job Monitor describes a phased rollout (including a dual-mode period). Consider defaulting this to false (or scoping it to specific pipelines/stages) so the feature can be enabled gradually and rolled back via variable override if needed.
- name: enableHelixJobMonitor
value: true
global.json:18
- The PR description says the Helix Job Monitor dependency is the same version as Arcade, but global.json keeps Microsoft.DotNet.Arcade.Sdk at 11.0.0-beta.26365.101 while Microsoft.DotNet.Helix.Sdk is bumped to 11.0.0-beta.26381.1 (and dotnet-tools.json pins the monitor tool to 11.0.0-beta.26381.1). Please clarify whether this version skew is intentional; if not, align the Arcade SDK version with the Helix/JobMonitor build you’re taking.
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "11.0.0-beta.26365.101",
"Microsoft.DotNet.Helix.Sdk": "11.0.0-beta.26381.1",
"Microsoft.DotNet.SharedFramework.Sdk": "11.0.0-beta.26365.101",
eng/Version.Details.xml:73
- This change also updates Microsoft.DotNet.GenAPI (version+SHA) and moves Microsoft.DotNet.Helix.Sdk’s source URI from dotnet/dotnet to dotnet/arcade. If these dependency updates are required for Helix Job Monitor enablement, it would help to call that out explicitly (or otherwise keep this PR focused by avoiding unrelated toolset churn).
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="11.0.0-beta.26381.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>93eebf1a31a5eaafd44326f1a81ca107913e098c</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.JobMonitor" Version="11.0.0-beta.26381.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>93eebf1a31a5eaafd44326f1a81ca107913e098c</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.GenAPI" Version="11.0.0-beta.26363.117">
<Uri>https://github.com/dotnet/dotnet</Uri>
<Sha>b25932e8d128b5242f1d89691f8e8171b82774c0</Sha>
</Dependency>
…nitor-rebased # Conflicts: # eng/pipelines/runtime.yml
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 22 changed files in this pull request and generated no new comments.
Suppressed comments (3)
eng/Version.Details.xml:72
- This change updates
Microsoft.DotNet.GenAPIto11.0.0-beta.26363.117(and a different SHA) as part of the Helix/JobMonitor update. Since the PR description doesn't mention GenAPI and this appears unrelated to Helix Job Monitor enablement, it would be good to either justify why this version shift is required here or split it into a separate dependency update PR.
<Dependency Name="Microsoft.DotNet.GenAPI" Version="11.0.0-beta.26363.117">
<Uri>https://github.com/dotnet/dotnet</Uri>
<Sha>b25932e8d128b5242f1d89691f8e8171b82774c0</Sha>
eng/pipelines/runtime.yml:71
- The Helix job monitor template defaults
allowNoHelixJobstofalse(see /eng/common/core-templates/job/helix-job-monitor.yml). Since this job is enabled here wheneverenableHelixJobMonitoris true, the Build stage can legitimately produce zero Helix submissions (e.g., PRs where all Helix legs are skipped by path conditions), which would make the monitor job fail. Consider opting in toallowNoHelixJobs: truehere (or gate the job on a condition that guarantees at least one Helix submission).
- ${{ if eq(variables['enableHelixJobMonitor'], true) }}:
- template: /eng/common/core-templates/job/helix-job-monitor.yml
parameters:
helixAccessToken: $(HelixApiAccessToken)
timeoutInMinutes: 540
global.json:18
- PR description says the new
Microsoft.DotNet.Helix.JobMonitordependency is "same version as Arcade", butglobal.jsonstill pinsMicrosoft.DotNet.Arcade.Sdkto11.0.0-beta.26365.101while Helix SDK/JobMonitor are11.0.0-beta.26381.1. If the intent is version parity with Arcade, the Arcade SDK version (and any derived dependency metadata) likely needs to be updated too, or the PR description should be adjusted to match the actual versioning approach.
"Microsoft.DotNet.Arcade.Sdk": "11.0.0-beta.26365.101",
"Microsoft.DotNet.Helix.Sdk": "11.0.0-beta.26381.1",
"Microsoft.DotNet.SharedFramework.Sdk": "11.0.0-beta.26365.101",
…nitor-rebased # Conflicts: # eng/Version.Details.props # eng/Version.Details.xml # eng/common/core-templates/job/helix-job-monitor.yml # global.json
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 22 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/Version.Details.props:12
- This change introduces a duplicate property. The file already defines MicrosoftDotNetHelixSdkPackageVersion later in the same PropertyGroup, so the later (older) value will win and the Helix SDK version update won’t actually take effect.
<!-- dotnet-arcade dependencies -->
<MicrosoftDotNetHelixJobMonitorPackageVersion>11.0.0-beta.26381.1</MicrosoftDotNetHelixJobMonitorPackageVersion>
<MicrosoftDotNetHelixSdkPackageVersion>11.0.0-beta.26381.1</MicrosoftDotNetHelixSdkPackageVersion>
<!-- dotnet-dotnet dependencies -->
<MicrosoftCodeAnalysisPackageVersion>5.10.0-1.26379.102</MicrosoftCodeAnalysisPackageVersion>
eng/pipelines/runtime.yml:71
- The Helix job monitor is added unconditionally when enableHelixJobMonitor is true, but the template defaults allowNoHelixJobs=false. If a Build stage run ends up producing zero Helix jobs (e.g., path-based conditions skip all Helix legs), the monitor job may fail the stage. Consider opting into allowNoHelixJobs for runtime.yml to avoid spurious failures.
- ${{ if eq(variables['enableHelixJobMonitor'], true) }}:
- template: /eng/common/core-templates/job/helix-job-monitor.yml
parameters:
helixAccessToken: $(HelixApiAccessToken)
timeoutInMinutes: 540
mmitche
approved these changes
Aug 4, 2026
premun
enabled auto-merge (squash)
August 4, 2026 17:09
akoeplinger
approved these changes
Aug 4, 2026
mmitche
commented
Aug 4, 2026
Member
/ba-g Unrelated test feailures |
Uh oh!
There was an error while loading. Please reload this page.
rcj1 added a commit
that referenced
this pull request
Aug 9, 2026
## Summary Disable the Helix Job Monitor for cDAC cross-platform dump generation. This restores the previous synchronous behavior where the submitting job: 1. Waits for Helix work to complete. 2. Downloads `dumps.tar.gz` into `artifacts/helixresults`. 3. Publishes the dumps for the dependent cross-platform test jobs. ## Root cause After #129690 enabled the Helix Job Monitor globally, `SendHelixJob` began returning immediately after submission. The cDAC dump-generation flow requires `DownloadFilesFromResults`, but the monitor does not download those artifacts for the submitting job. Consequently, `Publish Dump Artifacts` runs before `artifacts/helixresults` exists, although the Helix work items later complete successfully. Fixes #131979 > [!NOTE] > This content was created with assistance from AI.
mmitche added a commit
that referenced
this pull request
Aug 11, 2026
## Summary - default `enableHelixJobMonitor` to `false` in the shared pipeline variables - explicitly enable it in `eng/pipelines/runtime.yml` - preserve the standalone monitor job and fire-and-forget submissions only for the runtime pipeline ## Motivation The original rollout in #129690 intended to enable the Helix Job Monitor only for the runtime pipeline. However, the switch was set to `true` in the shared variables template imported by other pipelines. This caused standalone pipelines such as [`runtime-coreclr superpmi-collect` build 3043381](https://dev.azure.com/dnceng/internal/_build/results?buildId=3043381&view=results) to configure Helix submissions for out-of-band monitoring without adding a monitor job. Those submissions returned after submission rather than waiting for completion. Defaulting the shared switch off and opting in from `runtime.yml` keeps the complete feature scoped to its intended pipeline. ## Validation - `git diff --check` - verified `enableHelixJobMonitor` is enabled only by `eng/pipelines/runtime.yml` > [!NOTE] > This pull request description was generated with GitHub Copilot. Copilot-Session: 0c07f370-3ea0-4352-8610-0ab0cf1f7b0a
jtschuster pushed a commit
to jtschuster/runtime
that referenced
this pull request
Aug 11, 2026
) ## Summary Disable the Helix Job Monitor for cDAC cross-platform dump generation. This restores the previous synchronous behavior where the submitting job: 1. Waits for Helix work to complete. 2. Downloads `dumps.tar.gz` into `artifacts/helixresults`. 3. Publishes the dumps for the dependent cross-platform test jobs. ## Root cause After dotnet#129690 enabled the Helix Job Monitor globally, `SendHelixJob` began returning immediately after submission. The cDAC dump-generation flow requires `DownloadFilesFromResults`, but the monitor does not download those artifacts for the submitting job. Consequently, `Publish Dump Artifacts` runs before `artifacts/helixresults` exists, although the Helix work items later complete successfully. Fixes dotnet#131979 > [!NOTE] > This content was created with assistance from AI.
jtschuster pushed a commit
to jtschuster/runtime
that referenced
this pull request
Aug 11, 2026
## Summary - default `enableHelixJobMonitor` to `false` in the shared pipeline variables - explicitly enable it in `eng/pipelines/runtime.yml` - preserve the standalone monitor job and fire-and-forget submissions only for the runtime pipeline ## Motivation The original rollout in dotnet#129690 intended to enable the Helix Job Monitor only for the runtime pipeline. However, the switch was set to `true` in the shared variables template imported by other pipelines. This caused standalone pipelines such as [`runtime-coreclr superpmi-collect` build 3043381](https://dev.azure.com/dnceng/internal/_build/results?buildId=3043381&view=results) to configure Helix submissions for out-of-band monitoring without adding a monitor job. Those submissions returned after submission rather than waiting for completion. Defaulting the shared switch off and opting in from `runtime.yml` keeps the complete feature scoped to its intended pipeline. ## Validation - `git diff --check` - verified `enableHelixJobMonitor` is enabled only by `eng/pipelines/runtime.yml` > [!NOTE] > This pull request description was generated with GitHub Copilot. Copilot-Session: 0c07f370-3ea0-4352-8610-0ab0cf1f7b0a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dotnet/arcade#16731