Add test-coverage & codecov target and update circleci by vdemeester · Pull Request #109 · docker/cli

vdemeester · 2017-05-18T08:13:15Z

This is more or less what we used on some other docker project.

Not sure if test-coverage should be on his own or be just test
@dnephin I added parellel here too 👼
@nandhini915 the codecov token is set on circleci I guess ? 👼

Signed-off-by: Vincent Demeester vincent@sbr.pm

dnephin · 2017-05-18T14:35:02Z

Looks like it's failing because of missing dep.

Do you think we need this in the Makefile, or only the circleci?

I think parallel might actually slow it down, because we only get 1 CPU on CI.

I'm going to push some changes and you can let me know what you think.

nandhini915 · 2017-05-18T14:46:21Z

@vdemeester I don't think the token would be required (since docker/cli is a public repo), but yes for private repos we'd setup codecov token in circleci.

codecov-io · 2017-05-18T15:06:17Z

Codecov Report

❗ No coverage uploaded for pull request base (master@883d28c). Click here to learn what that means.
The diff coverage is n/a.

@@ Coverage Diff @@## master #109 +/- ##
=========================================
Coverage ? 46.16% =========================================
Files ? 161 Lines ? 10998 Branches ? 0 =========================================
Hits ? 5077 Misses ? 5631 Partials ? 290

dnephin

LGTM

although this now includes some of my changes

aaronlehmann · 2017-05-19T21:15:56Z

We run codecov on PRs against the swarmkit and distribution repositories. I don't have anything against it. But I also can't recall a time when the coverage information has been useful. We've pushed back against PRs that didn't have adequate tests, but that came from reviewing the PR and noticing that the tests were insufficient. I don't think we've ever made the decision based on the codecov report. In the past we tried to automatically fail CI for PRs that decreased the coverage ratio, and this worked very poorly. There was random fluctuation in the results because of nondeterminism in some code and tests (for example, the order in which two goroutines execute might affect which code paths are taken). It also penalized changes that added extensive error checking, when those errors were not possible to trigger in practice (and therefore the code paths did not show up in code coverage).

cc @stevvooe

dnephin · 2017-05-19T21:22:05Z

I don't think we have any plans to make it fail CI, I agree that is unreliable.

For me the benefit to this integration is that it makes it easier to see the test coverage for a PR. Test coverage may be easy to see in a PR that adds new features, but test coverage in a PR that refactor lots of code is less obvious.

The valuable metric to me is patch coverage, not total coverage.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Signed-off-by: Daniel Nephin <dnephin@docker.com>

mlaventure

LGTM

stevvooe · 2017-05-22T20:49:01Z

+#!/usr/bin/env bash
+set -eu -o pipefail
+
+go test -tags daemon -v $@


Why aren't these just in the Makefile? These kinds of systems are extremely hard to maintain.

What are "these kinds of systems" ? And what makes them hard to maintain?

A mess of bash scripts. It contributes to a horrid build experience and is very hard to modify.
This particular example is absurd. Why is this in its own file?

stevvooe · 2017-05-22T20:49:26Z

+ -covermode=atomic \
+ ${pkg}
+
+ if test -f profile.out; then


Please include a comment as to why this required.

Please address this. Why is this not an argument to the original command?

What is "this"? The profile.out filename?
https://lk4d4.darth.io/posts/multicover/

The file is given as one argument, then copied to the other. Why not give the original file to the command and have it write directly? Are you trying to append it to a global coverage file?

The file isn't copied. The file is appended to the global coverage file yes. That's what it's doing.

vdemeester

LGTM @dnephin's commit 👍

stevvooe · 2017-05-23T16:46:24Z

@vdemeester I would hope that you would at least address my comments before merging.

This Makefile setup gets a hard REJECT.

thaJeztah · 2017-05-23T16:59:11Z

@dnephin @vdemeester can this PR be discussed so that things are addressed or consensus reached?

dnephin · 2017-05-23T17:04:43Z

Yes, I'm still trying to understand the concerns with this PR. It was important to get it merged so that we have coverage on other PRs. If there are things that need to be changed, we can change them.

thaJeztah · 2017-07-21T10:46:44Z

This didn't make it into 17.06 - moving milestone

…-master-f2123b3fe41e7e126e9c8f4794214a50f71e866b [master] sync to upstream master f2123b3

[17.06] backport (cli) Don't prune volumes on docker system prune

vdemeester requested review from dnephin and nandhini915 May 18, 2017 08:13

GordonTheTurtle added the status/0-triage label May 18, 2017

vdemeester force-pushed the codecov-in-circle branch from 17ccd6f to f6fb2a4Compare May 18, 2017 08:15

vdemeester added status/2-code-review and removed status/0-triage labels May 18, 2017

vdemeester force-pushed the codecov-in-circle branch 3 times, most recently from 8fc32af to c436887Compare May 18, 2017 08:48

dnephin force-pushed the codecov-in-circle branch from 281e431 to 6334b4dCompare May 18, 2017 14:43

dnephin force-pushed the codecov-in-circle branch 3 times, most recently from f5e6f42 to a248158Compare May 18, 2017 15:06

dnephin approved these changes May 18, 2017
View reviewed changes

nandhini915 approved these changes May 18, 2017
View reviewed changes

dnephin requested a review from tiborvass May 18, 2017 15:18

Add test-coverage & codecov target and update circleci
d79d903
Signed-off-by: Vincent Demeester <vincent@sbr.pm>

dnephin force-pushed the codecov-in-circle branch from a248158 to 0019076Compare May 19, 2017 21:23

dnephin requested review from cpuguy83 and mlaventure May 19, 2017 21:23

Upload coverage report from outside of test container.
f6d148c
Signed-off-by: Daniel Nephin <dnephin@docker.com>

dnephin force-pushed the codecov-in-circle branch from 0019076 to f6d148cCompare May 19, 2017 21:37

mlaventure approved these changes May 22, 2017
View reviewed changes

stevvooe reviewed May 22, 2017
View reviewed changes

vdemeester commented May 23, 2017
View reviewed changes

dnephin merged commit e47cc91 into docker:masterMay 23, 2017

GordonTheTurtle added this to the 17.06.0 milestone May 23, 2017

vdemeester deleted the codecov-in-circle branch May 23, 2017 16:01

aaronlehmann mentioned this pull request Jun 6, 2017
Update vendoring of docker/docker #155
Merged

thaJeztah modified the milestones: 17.07.0, 17.06.0 Jul 21, 2017

trapier pushed a commit to trapier/cli that referenced this pull request Sep 30, 2019
Merge pull requestdocker#109from GordonTheTurtle/sync-with-upstream…
7bf41b1
…-master-f2123b3fe41e7e126e9c8f4794214a50f71e866b [master] sync to upstream master f2123b3

nobiit pushed a commit to nobidev/docker-cli that referenced this pull request Nov 19, 2025
Merge pull requestdocker#109from andrewhsu/dont-prune-vol
1e1c6b2
[17.06] backport (cli) Don't prune volumes on docker system prune

Uh oh!

Conversation

vdemeester commented May 18, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dnephin commented May 18, 2017

Uh oh!

nandhini915 commented May 18, 2017

Uh oh!

codecov-io commented May 18, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dnephin left a comment

Choose a reason for hiding this comment

Uh oh!

aaronlehmann commented May 19, 2017

Uh oh!

dnephin commented May 19, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mlaventure left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vdemeester left a comment

Choose a reason for hiding this comment

Uh oh!

stevvooe commented May 23, 2017

Uh oh!

thaJeztah commented May 23, 2017

Uh oh!

dnephin commented May 23, 2017

Uh oh!

thaJeztah commented Jul 21, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

vdemeester commented May 18, 2017•
edited
Loading

codecov-io commented May 18, 2017•
edited
Loading

dnephin commented May 19, 2017•
edited
Loading