Collect code coverage

[jelbourn]

We should collect and surface our unit test code coverage. While a high coverage does not necessarily tell you much about the quality of your tests, a low coverage does tell you that something is amiss.

[joshwiens]

@jelbourn - I'll pick this one up.

I assume I should work out the particulars with @hansl again?

[jelbourn]

@d3viant0ne sounds good! If @hansl is willing to chat about it on gitter that works for me, but we can also do the discussion here.

As a starting point, I'm okay with just printing out the coverage as part of the Travis jobs. Then, as a follow-up, we can send that data off somewhere (probably some firebase instance) and incorporate it into a dashboard.

As far as actual implementation goes, I'm not sure what the best avenue will be. The karma-coverage plugin uses the instanbul tool, but by itself doesn't support TypeScript. So there's exploration to happen in determining which tools are viable.

[hansl]

@d3viant0ne

Supporting TypeScript in karma with coverage is the problematic part of this plan, actually. Otherwise it should be straightforward. There is a few results for searching "code coverage karma typescript" on Google, but nothing out of the box. It is likely that we will need a custom made solution. For now, making a personal repo (or even just adding files to Material2) and using that should be great. That would be super useful for other TypeScript developers!

I'm not hanging often on gitter so let me know what would be a good time and I'll try to show up.

[joshwiens]

@hansl @jelbourn - I've done TypeScript coverage using remap-istanbul on a few projects but the implementation has always been very project specific.

It is also worth noting that nyc is also actively working to support TypeScript coverage.

My question is .. Do we want to do this in two parts?

Phase 1: Simple implementation of karma-coverage outputting to the command line when running "npm test"

This I could PR right now, karma doesn't exactly make coverage reporting hard :)
There is also a feature in remap-istanbul#master related to relative paths I would need to make it work.

Phase 2: Tailored, more complex implementation using remap-istanbul for dev TS coverage which also reports coverage in CI to a 3rd Party service && comments on the pull request. There are a few offerings (mentioned above) that are getting close on TypeScript coverage, perhaps the prudent path forward is the patient one.

While I personally think reporting coverage based on the transpiled javascript should be taken with a grain of salt ( branches in particular ) it does show you where most of the holes are. While it's not something I would keep for historical purposes, I do believe there is a short term benefit for code quality.

[jelbourn]

@d3viant0ne sounds good, though you could break it down even smaller:

1. Just like you said, print out the basic js coverage w/ npm test
2. Add the mapping back to the TypeScript source
3. Run on Travis, just printing out the results
4. Actually send the data somewhere more useful (TBD)

Thanks for looking into this!

[joshwiens]

note: After a bit of discussion, the "test" task will be left alone. Code coverage will be generated via it's own npm task