Skip to content

Add native C++ coverage via OpenCppCoverage - #1049

Merged
jasonleenaylor merged 1 commit into
mainfrom
native-coverage-opencppcoverage
Aug 10, 2026
Merged

Add native C++ coverage via OpenCppCoverage#1049
jasonleenaylor merged 1 commit into
mainfrom
native-coverage-opencppcoverage

Conversation

@jasonleenaylor

@jasonleenaylorjasonleenaylor commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

test.ps1 -Coverage now also collects native C++ coverage: Invoke-CppTest.ps1 gains a -Coverage switch that runs testGenericLib.exe and TestViews.exe under OpenCppCoverage (debugger-based, needs only the build's PDBs, never rewrites binaries in the shared Output/<Configuration> folder) and exports Output/<Configuration>/TestResults/native.<exe>.cobertura.xml. CI uploads the files under Codecov flag native, separate from managed.

Specifics:

  • Cobertura paths are rewritten repo-relative so Codecov matches files on any checkout path (no fixes: needed).
  • Without OpenCppCoverage installed (choco install opencppcoverage, or env OpenCppCoveragePath for a portable copy), native tests run bare with a [WARN]; CI then fails at the native upload step (fail_ci_if_error), never at the test step.
  • Invoke-CppTest's post-summary grace kill is disabled under coverage — the export is written after the Unit++ summary prints and a kill there would truncate it; covered runs rely on the overall timeout, which now also stops the child exe.
  • Shared plumbing consolidated into FwBuildHelpers.psm1: Set-TestAssertDialogEnvironment, Disable-CrashDialog, Get-UnitppSummary, Find-OpenCppCoverage, Convert-CoberturaPathsToRepoRelative.
  • New composite action .github/actions/codecov-upload owns the Codecov attempt/retry/verify sequence and the pinned action SHA; both the managed and native uploads use it.
  • CI installs OpenCppCoverage 0.9.9.0 (pinned) before the test step; codecov.yml ignores the native Unit++ test sources (Src/Generic/Test, Src/views/Test).

Measured: identical test results under coverage (testGenericLib [31-0-0], TestViews [301-0-0], exit 0); ~4x native wall time (TestViews ~5s -> ~20s); ~51% native line coverage across Views.dll, DebugProcs.dll, and the statically linked Generic sources; ReportGenerator 5.3.11 parses the files alone and merged with the managed report. All CI checks pass, including the native Codecov upload.

🤖 Generated with Claude Code


This change is Reviewable

@github-actions

This comment has been minimized.

@github-actions

github-actionsBot commented Aug 7, 2026

Copy link
Copy Markdown

NUnit Tests

1 files ±0 1 suites ±0 11m 38s ⏱️ +6s
5 761 tests ±0 5 680 ✅ ±0 81 💤 ±0 0 ❌ ±0 
5 770 runs ±0 5 689 ✅ ±0 81 💤 ±0 0 ❌ ±0 

Results for commit 6bd6f61. ± Comparison against base commit 64bba89.

♻️ This comment has been updated with latest results.

@codecov-commenter

codecov-commenter commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.02%. Comparing base (8dd2468) to head (6bd6f61).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #1049 +/- ##
==========================================
+ Coverage 36.43% 38.02% +1.58% 
==========================================
Files 1354 1500 +146 Lines 295676 349932 +54256 Branches 40223 40228 +5 ==========================================
+ Hits 107742 133047 +25305 - Misses 158661 187606 +28945 - Partials 29273 29279 +6 

see 151 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jasonleenaylor
jasonleenaylor marked this pull request as draft August 7, 2026 15:04
@jasonleenaylor
jasonleenaylorforce-pushed the native-coverage-opencppcoverage branch from 9c95a75 to 2849a1bCompareAugust 7, 2026 22:29
@jasonleenaylorjasonleenaylor changed the title Add native C++ coverage via OpenCppCoverage under a separate Codecov flagAdd native C++ coverage via OpenCppCoverage (Codecov flag: native)Aug 8, 2026
@jasonleenaylor
jasonleenaylorforce-pushed the native-coverage-opencppcoverage branch from 2849a1b to c49ae6dCompareAugust 8, 2026 03:31
test.ps1 -Coverage passes -Coverage to Invoke-CppTest.ps1, which wraps
the test exe with OpenCppCoverage and exports
Output/<Configuration>/TestResults/native.<exe>.cobertura.xml with
paths rewritten repo-relative so Codecov can match files. CI uploads
the files under Codecov flag "native", separate from "managed". When
the tool is missing the native tests run bare with a warning, and CI
fails at the upload step instead of the test step.
The post-summary grace kill is disabled under coverage because the
export is written after the Unit++ summary prints; covered runs rely
on TimeoutSeconds. Shared plumbing (assert-dialog env, crash-dialog
suppression, Unit++ summary parsing) moved into FwBuildHelpers.psm1,
and both Codecov uploads now go through the composite action
.github/actions/codecov-upload, which owns the retry/verify sequence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jasonleenaylor
jasonleenaylorforce-pushed the native-coverage-opencppcoverage branch from c49ae6d to 6bd6f61CompareAugust 8, 2026 04:46
@jasonleenaylorjasonleenaylor changed the title Add native C++ coverage via OpenCppCoverage (Codecov flag: native)Add native C++ coverage via OpenCppCoverageAug 8, 2026
@jasonleenaylor
jasonleenaylor marked this pull request as ready for review August 10, 2026 15:43

@thejambithejambi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thejambi reviewed 7 files and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on jasonleenaylor).

@jasonleenaylor
jasonleenaylor merged commit a3d91e9 into mainAug 10, 2026
7 checks passed
@jasonleenaylor
jasonleenaylor deleted the native-coverage-opencppcoverage branch August 10, 2026 22:13
thejambi pushed a commit that referenced this pull request Aug 25, 2026
* Add NativeCoverage to test.ps1 -Coverage to invoke OpenCppCoverage
* Upload Output/<Configuration>/TestResults/native.<exe>.cobertura.xml
under Codecov flag "native", separate from "managed".
* If the tool is missing the native tests run bare with a warning
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jasonleenaylor@codecov-commenter@thejambi