Skip to content

test: allow coverage threshold to be enforced - #25675

Closed
bcoe wants to merge 1 commit into
nodejs:masterfrom
bcoe:thresholds
Closed

test: allow coverage threshold to be enforced #25675
bcoe wants to merge 1 commit into
nodejs:masterfrom
bcoe:thresholds

Conversation

@bcoe

@bcoebcoe commented Jan 24, 2019

Copy link
Copy Markdown
Contributor

If COV_ENFORCE_THRESHOLD is set, tests run in coverage mode will
exit with an error, if line coverage is below the percentage
threshold.

will rebase against #25671, once it's ready to land.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@bcoebcoe added build Issues and PRs related to build files or the CI. test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory. blocked PRs that are blocked by other issues or PRs. labels Jan 24, 2019
@bcoe
bcoe requested review from mhdawson and refackJanuary 24, 2019 02:17
@nodejs-github-bot

nodejs-github-bot commented Jan 24, 2019

Copy link
Copy Markdown
Collaborator

@bcoe build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/2365/pipeline

@refack

refack commented Jan 24, 2019

Copy link
Copy Markdown
Contributor

Thinking about the next step of enabling this by default in CI...
Should this get the value from the repo (a file we update from time to time)? Maybe read dynamically from https://coverage.nodejs.org/?
Should we run it daily (and notify some derived list of authors)? Or like "Codecov" run it for every PR?

@refackrefack added the coverage Issues and PRs related to native coverage support. label Jan 24, 2019
@bcoe

bcoe commented Jan 24, 2019

Copy link
Copy Markdown
ContributorAuthor

@refack, I think for starters @mhdawson was planning on making this a configuration variable, which would be set on the build server he's configuring here.

@mhdawson

mhdawson commented Jan 24, 2019

Copy link
Copy Markdown
Member

Will test this out on the build I'm trying out and the approve if looks good. Would like #25671 to land first as well.

@mhdawson

mhdawson commented Jan 30, 2019

Copy link
Copy Markdown
Member

#25671 landed, if you can rebase I'll run it against the coverage sanity job just to double check and then we'd be good to land.

@mhdawson

mhdawson commented Jan 30, 2019

Copy link
Copy Markdown
Member

I have found one other test that is failing under coverage on the docker ubuntu 16 machines we'll use for this job and am testing a PR now, but it should not conflict with this one once rebased.

@bcoe
bcoeforce-pushed the thresholds branch 2 times, most recently from c9408bf to 79ffeecCompareJanuary 30, 2019 23:49
@bcoe
test: allow coverage threshold to be enforced
62decf4
If COV_ENFORCE_THRESHOLD is set, tests run in coverage mode will
exit with an error, if line coverage is below the percentage
threshold.
@bcoe

bcoe commented Jan 31, 2019

Copy link
Copy Markdown
ContributorAuthor

@mhdawson rebased 👍

@mhdawson

mhdawson commented Jan 31, 2019

Copy link
Copy Markdown
Member

run of coverage sanity job we are working on with this update: https://ci.nodejs.org/view/all/job/node-test-commit-coverage/24/

@mhdawson

mhdawson commented Jan 31, 2019

Copy link
Copy Markdown
Member

New CI with naming fixed: https://ci.nodejs.org/view/all/job/node-test-commit-coverage/26/

mhdawson
mhdawson approved these changes Jan 31, 2019
View reviewed changes

@mhdawsonmhdawson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as long as CI runs pass

@bcoe

bcoe commented Jan 31, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

ci: https://ci.nodejs.org/job/node-test-pull-request/20493/

@bcoebcoe added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed blocked PRs that are blocked by other issues or PRs. labels Feb 1, 2019
@bcoe

bcoe commented Feb 1, 2019

Copy link
Copy Markdown
ContributorAuthor

Landed in a861add

@bcoebcoe closed this Feb 1, 2019
@bcoe
bcoe deleted the thresholds branch February 1, 2019 18:21
bcoe added a commit that referenced this pull request Feb 1, 2019
@bcoe
test: allow coverage threshold to be enforced
a861add
If COV_ENFORCE_THRESHOLD is set, tests run in coverage mode will
exit with an error, if line coverage is below the percentage
threshold.
PR-URL: #25675
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
addaleax pushed a commit that referenced this pull request Feb 1, 2019
@bcoe@addaleax
test: allow coverage threshold to be enforced
704a440
If COV_ENFORCE_THRESHOLD is set, tests run in coverage mode will
exit with an error, if line coverage is below the percentage
threshold.
PR-URL: #25675
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
@targostargos mentioned this pull request Feb 14, 2019
Merged
This was referenced Feb 15, 2019
Closed
Closed
Closed
Closed
Closed
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@refackrefackrefack approved these changes
@mhdawsonmhdawsonmhdawson approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.buildIssues and PRs related to build files or the CI.coverageIssues and PRs related to native coverage support.testIssues and PRs related to the tests.toolsIssues and PRs related to the tools directory.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bcoe@nodejs-github-bot@refack@mhdawson