Also fail on Pester block/container failures (not just failed tests) by nohwnd · Pull Request #128 · psake/PowerShellBuild

nohwnd · 2026-07-01T21:47:27Z

⚠️ This PR was generated by an AI agent (GitHub Copilot CLI). Please review accordingly.

I've been seeing this commonly across Pester 5 build scripts and I'm fixing it where it looks like it could hide real failures.

In Pester 5 a run reports three independent failure counts:

FailedCount — failed tests (It)
FailedBlocksCount — failed BeforeAll / AfterAll
FailedContainersCount — files that error during discovery / fail to load

Gating on only FailedCount means a BeforeAll that throws, or a test file that fails to load, leaves FailedCount = 0 and the build passes green despite real failures. Pester itself derives the overall pass/fail from the sum of all three.

This change includes all three counts in the gate. Display/log messages are left unchanged.

In Pester 5 a run has three independent failure counts. Gating only on FailedCount misses FailedBlocksCount (BeforeAll/AfterAll) and FailedContainersCount (files that fail to discover/load), so a failing BeforeAll or an unloadable test file passes the build green. This adds those counts to the gate. Generated with AI (GitHub Copilot CLI). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

tablackburn · 2026-07-07T19:34:58Z

@nohwnd should we also make this change to

PowerShellBuild/PowerShellBuild/Public/Test-PSBuildPester.ps1

Line 107 in efebfa4

if ($testResult.FailedCount-gt0) {

?

nohwnd · 2026-07-07T19:58:18Z

Yes. You can also check the result property.

tablackburn · 2026-07-07T20:15:30Z

@nohwnd do you want to include that change in this PR or do you want me to take care of it? I'm good either way.

Test-PSBuildPester gated only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories (failed tests, failed blocks, failed containers). This applies the shipped-function half of the fix discussed in #128; that PR covers the repository's own psakeFile.ps1. Verified against crash fixtures: a discovery-crash file yields FailedCount=0 / Result=Failed and now throws; a BeforeAll crash still throws; a healthy suite still passes. Full suite: 301 passed, 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa

tablackburn

Thanks for the fix! This is correct for the repo's own build file, and we've verified the bug class is real here — while migrating this repo's test suite to Pester 6 (#132), a test file died during discovery and the build still reported success (FailedCount=0, Container failed: 1, exit 0).

Per the discussion above, the shipped-function half (Test-PSBuildPester.ps1) is now covered in #133 using the Result property you suggested, so this PR can stay scoped as-is. Left one optional inline suggestion if you'd like the two gates to match.

Generated by Claude Code

Address review feedback on #133: the Unreleased entry linked #128, but this change lands via #133. Point the entry at #133 and note the companion relationship to #128 explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa

Replace the three-counter sum merged in #128 with the aggregate Result property, matching the gate in Test-PSBuildPester. Same behavior, one source of truth for what "failed" means. Suggested by nohwnd in the #128 discussion. Verified both directions locally: a discovery-crash container fails the build (exit 1), and a healthy run under Pester 5.8.0 passes (301 passed, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa

Test-PSBuildPester gated pass/fail only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories. Also switch this repo's own psakeFile.ps1 Pester gate (the three-counter sum from #128) to the same Result check so both gates share one definition of "failed". Companion to #128. Same bug lineage as #31, #52, and #57.

nohwnd · 2026-07-08T07:35:08Z

Thanks for merging, because see you using Claude here, just pointing out there is now skill for v6 migration, as well as for should -* to should-* migration.

https://pester.dev/docs/migrations/v5-to-v6#migrate-with-an-ai-assistant

Add tests/Test-PSBuildPester.tests.ps1 (#102). Every invocation runs in a Start-Job subprocess with a pinned inner Pester version, and the scenarios run against both installed Pester majors (5.x and 6.x) to verify the shipped function keeps supporting Pester 5 consumers. Crash fixtures are generated into $TestDrive at runtime, never checked in. Scenarios: healthy suite, failing test (regression #52), BeforeAll crash and discovery crash (#128/#133 gate), NUnit result output, and code coverage output path/format (regression Add a second, named Pester dependency (PesterLegacy, 5.9.0) to requirements.psd1 so CI installs both majors side by side. Fix two latent Test-PSBuildPester bugs the tests exposed, test-first (all 14 tests failed before the fixes, pass after): - The finally block called Remove-Module with an empty -Name when the optional ModuleName parameter was omitted, raising a parameter- binding error that -ErrorAction SilentlyContinue cannot suppress. - An unconditional Import-Module Pester -MinimumVersion 5.0.0 loaded the newest installed Pester on top of an already-loaded one, which crashes with a Pester.dll version conflict when two versions are installed side by side. The function now respects a loaded Pester (with a clear error below 5.0.0) and imports the newest installed version only when none is loaded. Closes#102 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa

) Add tests/Test-PSBuildPester.tests.ps1 (#102). Every invocation runs in a Start-Job subprocess with a pinned inner Pester version, and the scenarios run against both installed Pester majors (5.x and 6.x) to verify the shipped function keeps supporting Pester 5 consumers. Crash fixtures are generated into $TestDrive at runtime, never checked in. Scenarios: healthy suite, failing test (regression #52), BeforeAll crash and discovery crash (#128/#133 gate), NUnit result output, and code coverage output path/format (regression #62). Pester 5.9.0 is installed side by side with 6.0.0 through the new install-only requirements.pester-matrix.psd1; the bootstrap must not import it, because importing two Pester majors into one session crashes with a Pester.dll version conflict. Fix two latent Test-PSBuildPester bugs the tests exposed, test-first: - The finally block called Remove-Module with an empty -Name when the optional ModuleName parameter was omitted, raising a parameter- binding error that -ErrorAction SilentlyContinue cannot suppress. - An unconditional Import-Module Pester -MinimumVersion 5.0.0 loaded the newest installed Pester on top of an already-loaded one. The function now respects a loaded Pester (with a clear error below 5.0.0) and imports the newest installed version only when none is loaded. Closes#102

nohwnd mentioned this pull request Jul 1, 2026
Build scripts commonly gate only on FailedCount (ignores failed blocks/containers) pester/Pester#2833
Open

tablackburn mentioned this pull request Jul 7, 2026
Tests: Test-PSBuildPester #102
Closed

tablackburn mentioned this pull request Jul 7, 2026
chore: Pin Pester 6.0.0 for the repository test suite #132
Merged
3 tasks

Merge branch 'main' into fix/pester5-count-blocks-containers
5544777

tablackburn mentioned this pull request Jul 7, 2026
fix: Fail the build when a Pester run fails for any reason #133
Merged
5 tasks

tablackburn reviewed Jul 7, 2026
View reviewed changes

Comment threadpsakeFile.ps1

tablackburn approved these changes Jul 7, 2026
View reviewed changes

tablackburn merged commit f59efcc into psake:mainJul 7, 2026
6 checks passed

tablackburn mentioned this pull request Jul 10, 2026
test: Add Test-PSBuildPester integration tests and fix latent bugs #137
Merged
4 tasks

tablackburn mentioned this pull request Jul 21, 2026
[Tracking] PowerShellBuild v1.0.0 roadmap #120
Open
24 tasks

Navigation Menu

Uh oh!

Uh oh!

Also fail on Pester block/container failures (not just failed tests) - #128

Also fail on Pester block/container failures (not just failed tests)#128
tablackburn merged 2 commits into
psake:mainfrom
nohwnd:fix/pester5-count-blocks-containers

nohwnd commented Jul 1, 2026

Uh oh!

tablackburn commented Jul 7, 2026

Uh oh!

nohwnd commented Jul 7, 2026

Uh oh!

tablackburn commented Jul 7, 2026

Uh oh!

tablackburn left a comment

Uh oh!

Uh oh!

Uh oh!

nohwnd commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

nohwnd commented Jul 1, 2026

Uh oh!

tablackburn commented Jul 7, 2026

Uh oh!

nohwnd commented Jul 7, 2026

Uh oh!

tablackburn commented Jul 7, 2026

Uh oh!

tablackburn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nohwnd commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants