Uh oh!
There was an error while loading. Please reload this page.
[cdac] Share one coreclr build between cDAC dump + stress test legs - #129237
Closed
max-charlamb wants to merge 3 commits into
Closed
[cdac] Share one coreclr build between cDAC dump + stress test legs#129237max-charlamb wants to merge 3 commits into
max-charlamb wants to merge 3 commits into
Conversation
Adds a per-platform CdacBuild stage that builds coreclr (Checked) + libs + tools.cdac + the cDAC dump+stress test subsets, then publishes artifacts/bin as a CdacBuildArtifacts_<plat> tar. Subsequent commits will switch CdacDumpTests, CdacStressTests, and the CdacXPlat stages to consume this artifact instead of each running their own full build. The matrix uses cdacDumpPlatforms (the broader 8-platform set); CdacStressTests' 4-platform subset is still satisfied since per-platform artifacts are independent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CdacDumpTests, CdacStressTests, CdacXPlatDumpGen, and CdacXPlatDumpTests now download the per-platform CdacBuildArtifacts_<plat> tar published by CdacBuild instead of each running their own full clr+libs+tools.cdac+... build. Each test leg still calls ./build.sh with a minimal subset (tools.cdac<test>tests) so .dotnet is initialized and MSBuild can re-link the test csproj against the downloaded artifact -- no coreclr or libs work is repeated. Adds a cdacTestConfig parameter to prepare-cdac-helix-steps.yml (default $(_BuildConfig) for back-compat); dump-test stages pass cdacTestConfig: Debug so the dotnet build invocations that build the debuggees and prepare the Helix payload pull in the Debug-configured managed cDAC assemblies that the DumpTests project loads directly. The stress stage keeps the Release native cDAC shim that ships in the shared testhost. Cross-platform job-name expansion uses dependsOnGlobalBuilds rather than a hand-rolled build_<plat>_<config>_CdacBuild string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…elism
Moves CdacBuild + CdacDumpTests + CdacStressTests + CdacXPlatDumpGen
into a single `Cdac` stage. Within that stage, every test job uses
job-level `dependsOnGlobalBuilds: - nameSuffix: CdacBuild` which the
global-build-job template expands to a per-platform
`build_<plat>_<config>_CdacBuild` dep. This is the canonical
dotnet/runtime pattern (runtime.yml:1591,1622,1653) and means
`linux_x64` test legs start as soon as `linux_x64`'s build finishes --
no waiting on slower platforms' builds.
Previously each test stage had a stage-level `dependsOn: CdacBuild`
which is all-or-nothing in AzDO -- every platform's tests had to wait
for the slowest platform's build to finish before starting.
The single-leg / xplat / stress / schedule selection moves from
stage-level `${{ if }}` to job-level `${{ if }}` inside the Cdac stage.
CdacXPlatDumpTests stays as a separate stage with stage-level
`dependsOn: Cdac` because its cross-platform synchronization (every
target platform's test needs every source platform's dumps) is
intrinsic. Its previous job-level `dependsOnGlobalBuilds: CdacBuild`
is dropped -- AzDO doesn't support cross-stage job-level deps, and
the stage-level dep already gates everything correctly.
Also fixes a duplicate `parameters:` block in CdacXPlatDumpTests
that was a merge artifact from the previous commit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>Contributor
There was a problem hiding this comment.
Pull request overview
This PR restructures the runtime-diagnostics.yml pipeline to avoid redundant CoreCLR/Libraries builds across the cDAC dump tests, stress tests, and x-plat dump generation legs by introducing a per-platform “shared build” job that downstream per-platform jobs depend on.
Changes:
- Collapses the prior multi-stage cDAC flow into a single
Cdacstage with per-platform job-level dependencies on a sharedCdacBuildjob. - Switches downstream legs (dump tests, stress tests, x-plat dump gen/tests) to download
artifacts/binfrom the shared build instead of rebuilding CoreCLR+libs. - Extends
prepare-cdac-helix-steps.ymlwith acdacTestConfigparameter to allow rebuilding the managed cDAC/DumpTests payload inDebugduring payload prep.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| eng/pipelines/runtime-diagnostics.yml | Refactors cDAC pipeline stages into a shared per-platform build plus dependent per-platform test legs; wires artifact upload/download and job-level deps. |
| eng/pipelines/cdac/prepare-cdac-helix-steps.yml | Adds cdacTestConfig parameter and uses it for payload-prep MSBuild invocations to support Debug-managed cDAC binaries. |
Comment on lines
+298
to
+303
| - template: /eng/pipelines/common/platform-matrix.yml | ||
| parameters: | ||
| jobTemplate: /eng/pipelines/common/global-build-job.yml | ||
| buildConfig: release | ||
| platforms: ${{ parameters.cdacDumpPlatforms }} | ||
| jobParameters: |
This was referenced Jun 10, 2026
max-charlamb
commented
Jun 10, 2026
MemberAuthor
Superseded by #129244 -- consolidated all the cDAC follow-up changes into a single PR. |
max-charlamb added a commit
that referenced
this pull request
Jun 14, 2026
…d fix (#129244) > [!NOTE] > This PR was prepared with assistance from GitHub Copilot CLI. Consolidated follow-ups to #128872 covering the cDAC stages in `runtime-diagnostics.yml` plus a small datacontract doc cleanup. ## Commits (oldest first) 1. **`efe2cdeee14`** — `[cdac] StackWalk.md: drop stress-harness detail from datacontract` The new "Signature-Based Scanning (currently deferred)" section reads like stress-harness README content. Replaced with a one-sentence note + link to `tests/StressTests/known-issues.md` (which already documents the stub, the sentinel, and the ICallingConvention work needed to re-enable the scan). 2. **`0e4caf52fd5`** — `[cdac] Fix DataGeneratorTests build: override Target.ReadNInt` #128784 added `public abstract TargetNInt ReadNInt(ulong address)` to `Target` but didn't update the test-only `TestTarget` mock, so `DataGeneratorTests` no longer compiles. Adds the missing override, mirroring `ReadNUInt`. 3. **`ebc2c062fbf`** — `[cdac pipeline] Refactor runtime-diagnostics: share one build, run unit tests, parallelize per-platform` See the commit message for the full detail. Highlights: - **Shared `CdacBuild`** stage so coreclr+libs is built ONCE per platform (Checked) and consumed by Dump / Stress / XPlatDumpGen / XPlatDumpTest legs. - **Stress vs dump cDAC split**: stress uses the Release native cDAC shim (in the shared testhost); dumps use Debug managed cDAC assemblies, with libraries pinned at Release + pre-flight verification. - **Per-platform parallelism** via job-level `dependsOnGlobalBuilds: CdacBuild` (the canonical `runtime.yml` pattern). `CdacXPlatDumpTest` adds an explicit per-source-platform `dependsOn` so the whole thing fits in one stage. - **Stress tests now run on every trigger** (previously gated to not Schedule). - **`CdacUnitTests`** runs the cDAC managed-side unit tests + DataGeneratorTests on linux_x64 Debug using a minimal `-s tools.cdactests -test` (modeled after `eng/pipelines/coreclr/ilasm.yml:52`). Also adds the previously-unbuilt DataGeneratorTests project to the `tools.cdactests` subset. - **3 new reusable templates** (`sos-test-leg.yml`, `cdac-helix-test-leg.yml`, `download-cdac-build-artifact.yml`) compress ~250 lines of repeated boilerplate. - Top-of-file overview header summarizing every stage. `Build` -> `SOSTests`, `Cdac` -> `CdacTests`. - `runtime-diagnostics.yml` shrinks from 581 -> 351 lines. ## Files changed - `docs/design/datacontracts/StackWalk.md` - `eng/Subsets.props` - `eng/pipelines/runtime-diagnostics.yml` - `eng/pipelines/cdac/prepare-cdac-helix-steps.yml` - `eng/pipelines/cdac/cdac-helix-test-leg.yml` (new) - `eng/pipelines/cdac/download-cdac-build-artifact.yml` (new) - `eng/pipelines/diagnostics/sos-test-leg.yml` (new) - `src/native/managed/cdac/tests/DataGenerator/TestTarget.cs` ## Status Draft. Local YAML parses; needs an end-to-end pipeline run on this branch to validate per-platform parallelism + Debug cDAC payload prep + `CdacUnitTests` minimal-subset build all work as expected. Supersedes #129237 and #129241 (will close those). --------- Co-authored-by: Max Charlamb <maxcharlamb@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was prepared with assistance from GitHub Copilot CLI.
Summary
Refactors
runtime-diagnostics.ymlso the cDAC dump tests and stress tests share one coreclr+libs build per platform, and so per-platform pipelines run in parallel (no waiting on slower platforms).Before
Each test leg rebuilt coreclr from scratch.
After
linux_x64dump tests start as soon aslinux_x64build is done; no waiting onosx_arm64to finish building.runtime.ymldependsOnGlobalBuildsmechanism (seeruntime.yml:1591,1622,1653).How the Release / Debug cDAC split works
The user requested:
-rc checked).These coexist because they live in different artifact locations:
-c Release -rc checked→ native shim ships in the Release testhost.cdacTestConfig: $(_BuildConfig)(Release) — they pick up the Release native shim from the testhost.cdacTestConfig: Debugparameter — payload-prep MSBuild walks the DumpTests project graph and rebuilds the managed cDAC assemblies at Debug in a separate output directory.Files changed
eng/pipelines/runtime-diagnostics.yml— collapsed 5 stages into 2, wired job-level deps.eng/pipelines/cdac/prepare-cdac-helix-steps.yml— addedcdacTestConfigparameter (defaults to$(_BuildConfig)for back-compat).Build/ SOS stage untouched — different subset, different platforms.)Validation