Uh oh!
There was an error while loading. Please reload this page.
Move CI to GitHub Actions, remove the Azure pipeline - #2905
Merged
Conversation
Runs in parallel with azure-pipelines.yml so we can compare the two before cutting over. Build once on Windows, then the same 8-leg matrix as Azure: PowerShell 7 on ubuntu/macOS/windows (latest + one previous GA), Windows PowerShell 5.1 on windows (latest + previous). Coverage renders in each run's job summary and a consolidated table, and uploads to Codecov with a flag per leg. GA-only images: floating *-latest plus one pinned previous GA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Add https://github.com/dorny/test-reporter to the test matrix jobs?
Didn't know. 🙁 Good thing we don't use exclusive features here. |
- Publish NUnit3 test results per matrix leg with dorny/test-reporter, a GitHub-native test report (Azure test-tab parity). test.ps1 gains an env-selectable result format (PESTER_TESTRESULT_FORMAT); Azure keeps the default NUnit 2.5, only the pilot switches to NUnit3. - Add Dependabot for github-actions (weekly, grouped, incl. major) to keep the actions current and ahead of Node.js runtime deprecations. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dorny/test-reporter shells out to git and fails with exit 128 when the leg has no repo checkout (the legs are artifact-only). Add a shallow checkout before the artifact download; the artifact overlays it, so the tests still run against the published build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Azure PublishTestResults@2 supports NUnit3, so test.ps1 can just emit the modern format for both consumers. Drops the PESTER_TESTRESULT_FORMAT env indirection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fflaten
commented
Jul 19, 2026
Contributor
Test result summary not showing, not sure why. Looks like we need to extract the test-report to a follow-up workflow into separate workflow to handle forks (https://github.com/dorny/test-reporter#recommended-setup-for-public-repositories). And maybe limit the summary to failed only using |
The pilot ran alongside Azure since July and matches it, so this removes azure-pipelines.yml and makes the workflow the CI. Releasing stays on Azure DevOps in azure-pipelines-publish.yml, it needs the signing service connection and the publishing variable groups. Test results now render from test-report.yml, a workflow_run workflow, instead of from inside the matrix legs. That is the setup dorny/test-reporter recommends for public repos, and the reason the report was not showing: a run triggered by a pull request from a fork gets a read only token and cannot create a check run. The separate workflow runs in the base repo and can. It lists only failed suites and tests, a full pass list gets truncated by GitHub. Legs no longer check out the repo, that checkout only existed to give test-reporter git context. The gate to require in branch protection is "Done". 🤖
The runner forces Node 20 actions onto Node 24 and warns about it on every job. setup-dotnet v6, upload-artifact v7, download-artifact v8, codecov-action v7. The github-script warning came from codecov-action v5, it is gone with v7. download-artifact v8 fails the job when the download digest does not match, instead of only warning. 🤖
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves CI to GitHub Actions and deletes
azure-pipelines.yml. Releasing stays on Azure DevOps (azure-pipelines-publish.yml), it needs the signing service connection and the publishing variable groups.Why now: Azure DevOps public projects are being retired, existing ones convert to private in 2027 and the free grant drops from 10 parallel jobs to 1. GitHub Actions is free for public repos with 20.
The workflow builds once on
windows-latest, then runs the same 8 leg matrix Azure ran: PowerShell 7 on ubuntu/macOS/windows (latest plus one previous GA), Windows PowerShell 5.1 on windows (latest plus previous). Coverage shows in each run's job summary and in a consolidated table, and uploads to Codecov with a flag per leg. OneDonejob gates the matrix, same idea as the AzureDonestage, so branch protection requires one check instead of listing all 8 legs.@fflaten the test report is fixed. It moved out of the matrix into
test-report.yml, which is the setup dorny/test-reporter recommends for public repos, and also the reason nothing was showing: a run triggered by a pull request from a fork gets a read only token and cannot create a check run. Aworkflow_runworkflow runs in the base repo and can. It lists only failed suites and tests now. One catch,workflow_runonly fires for the copy of the workflow on the default branch, so the report cannot be seen on this PR, it starts working once this is merged.Before merging, main still requires
Pester PR (Build build)andPester PR (Done Done). Neither can report once the pipeline file is gone, so the required check has to change toDoneat the same time, and the Pester PR pipeline in Azure DevOps should stop triggering on pull requests.Next: the same workflow on
rel/5.x.x, and aDonejob in the v4 workflow onrel/4.x.x, then one branch policy requiringDonecovers main and both release branches.Images are GA only, floating
*-latestplus one pinned previous GA.macos-14is deprecating (removed Nov 2026) and will need swapping before then.🤖