Skip to content

test_runner: print coverage and diagnostic info with dot reporter - #61423

Open
mag123c wants to merge 3 commits into
nodejs:mainfrom
mag123c:fix-dot-reporter-coverage
Open

test_runner: print coverage and diagnostic info with dot reporter#61423
mag123c wants to merge 3 commits into
nodejs:mainfrom
mag123c:fix-dot-reporter-coverage

Conversation

@mag123c

@mag123cmag123c commented Jan 18, 2026

Copy link
Copy Markdown
Contributor

This PR adds coverage and diagnostic output to the dot reporter in test_runner.

Currently, when using the dot reporter with coverage enabled, if a coverage threshold fails the program exits with code 1 but prints nothing about why. Users have no way to know what went wrong. The spec reporter already handles this correctly.

Changes:

  • dot.js: handle test:diagnostic events at error level (coverage threshold failure messages)
  • dot.js: handle test:coverage events (coverage report table)

The dot reporter now behaves consistently with the spec reporter for coverage output.

Fixes: #60884

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-botnodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Jan 18, 2026
@mag123c
mag123cforce-pushed the fix-dot-reporter-coverage branch from 83840c7 to 313a274CompareJanuary 18, 2026 06:34
@codecov

codecovBot commented Jan 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.13%. Comparing base (1ba3ce4) to head (f27934e).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
lib/internal/test_runner/reporter/dot.js33.33%15 Missing and 1 partial ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #61423 +/- ##
==========================================
- Coverage 90.14% 90.13% -0.02% 
==========================================
Files 743 743 Lines 242415 242438 +23 Branches 45651 45655 +4 ==========================================
- Hits 218531 218517 -14 - Misses 15386 15424 +38 + Partials 8498 8497 -1 
Files with missing linesCoverage Δ
lib/internal/test_runner/reporter/dot.js74.60% <33.33%> (-25.40%)⬇️

... and 37 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pmarchini

Copy link
Copy Markdown
Member

Hey @mag123c, could you please add some tests?

@@ -170,4 +170,25 @@ for (const coverage of coverages) {
assert.strictEqual(result.status, 1);
assert(!findCoverageFileForPid(result.pid));
});

test(`test failing ${coverage.flag} with dot reporter`, () => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I would suggest adding a snapshot test too, like as we have done in test/test-runner/test-output-coverage-width-100-uncovered-lines.mjs 👀

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pmarchini Thanks! I did!

@meixgmeixg added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 4, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 4, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

When using the dot reporter with coverage enabled, coverage threshold
failures and coverage reports were not printed, only an exit code was
returned. This made it impossible to know why the test run failed.
This change adds handling for test:diagnostic and test:coverage events
to the dot reporter, matching the behavior of the spec reporter.
Fixes: nodejs#60884
Signed-off-by: mag123c <diehreo@gmail.com>
Signed-off-by: mag123c <diehreo@gmail.com>
Signed-off-by: mag123c <diehreo@gmail.com>
@mag123c
mag123cforce-pushed the fix-dot-reporter-coverage branch from 1f59f26 to f27934eCompareJuly 26, 2026 06:28
@mag123c

mag123c commented Jul 26, 2026

Copy link
Copy Markdown
ContributorAuthor

One thing worth flagging for whoever picks this up: several newer PRs now touch the same handler in lib/internal/test_runner/reporter/dot.js#64695, #64578, #64331, and #62234. They overlap with this one to varying degrees, and it would probably save everyone time to decide which should land rather than reviewing them independently. Happy to close this in favour of another if that is the cleaner path.

@avivkelleravivkeller added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Jul 26, 2026
@github-actionsgithub-actionsBot added request-ci-failed An error occurred while starting CI via request-ci label, and manual interventon is needed. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Jul 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor
Failed to start CI
- Validating Jenkins credentials
✔ Jenkins credentials valid
- Querying data for job/node-test-pull-request/71552/
[SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/30192660637

@mcollinamcollina added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci-failed An error occurred while starting CI via request-ci label, and manual interventon is needed. labels Jul 26, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 26, 2026
@github-actionsgithub-actionsBot added the request-ci-failed An error occurred while starting CI via request-ci label, and manual interventon is needed. label Jul 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor
Failed to start CI
- Validating Jenkins credentials
✔ Jenkins credentials valid
- Querying data for job/node-test-pull-request/71552/
[SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/30195826270

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.needs-ciPRs that need a full CI run.request-ci-failedAn error occurred while starting CI via request-ci label, and manual interventon is needed.test_runnerIssues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_runner: print failed coverage reports with dot runner

7 participants

@mag123c@nodejs-github-bot@pmarchini@meixg@avivkeller@atlowChemi@mcollina