Feature request: decouple test running from coverage analysis from stdout feedback

[adamcameron]

(I am not sure the title is right, but bear with me)

I followed the recommendation at #321 (comment) to exclude certain files from code coverage analysis.

This adds a coverage reporter to the reporters collection (ie: $reporters->add('coverage', $coverage)).

Having done that, whenever I do a test run now, I always get a coverage summary emitted to stdout. Even if I'm not passing the code coverage command line params. EG whether I do vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --coverage=4 or just vendor/bin/kahlan, I get the summary.

Now obviously I can see why this happens: as per above I have added code to the config to always add the Coverage reporter, and I guess I could put a condition around the code in the config file to check if I've asked for coverage on the command-line. But that seems very... um... heavy-handed. TBH having code-as-config at all seems questionable to me, and needing to pile conditional logic into the code in the config seems... even more questionable still. Before too long I'm gonna need to unit test my config file! ;-)

The issue I guess is that the mechanism to exclude files from code coverage analysis seems to be very clunky. I should just be able to either annotate the files themselves with @codeCoverageIgnore (ie: like PHPUnit does), or provide the already-loaded coverage analyser with a file manifest or something. I should not have to configure a whole new Coverage reporter with the exception list as a config param. In code.

Secondly, whilst I might want to run code coverage, I don't want a summary of it bled to stdout, given I've told it to go to a file (as per above: --lcov="var/tmp/lcov/coverage.info"). I looked at the code and this summary info seems to be emitted from Kahlan\Reporter\Coverage::stop. That seems to be the thing doing the report. I had a cursory glance through the code looking for a place to say "actually nothing to stdout pls", but couldn't find it. I guess I could extend Collector as a SilentCollector (or use a decorator or something), but this seems like a lot of work to coerce Kahlan to just put the code coverage output where I've actually asked it to (only) go: into that coverage.info file.

I'm now wondering if there's some coupling between coverage analysis and reporting on that analysis that I'm not quite getting. It seems odd to me that I am having to tell Kahlan*Reporter*\Coverage (my emphasis obviously) about files to not analyse. Especially given it seems that the rest of the job of a Reporter is to just report on the results. I should be telling the analyser to not analyse those files, shouldn't I? I'm so confused now.

For now I'll stick the conditional logic in the config file I guess.

Cheers!

[adamcameron]

Hang on perhaps some of this can be circumvented with a clarification. The docs for --coverage say this:

Generate code coverage report. The value specify the level of detail for the code coverage report (0-4). If a namespace, class, or method definition is provided, it will generate a detailed code coverage of this specific scope (default `''`)

I note that if I use this:

vendor/bin/kahlan --lcov="var/tmp/lcov/coverage.info" --coverage=0 # note the ZERO there
genhtml --output-directory public/lcov/ var/tmp/lcov/coverage.info

I still get a (accurate, complete) code coverage report, when I browse to lcov/index.html.

It seems to me that the --coverage param only addresses reporting, and the analysis is done whether or not the reporting is done? So is code coverage analysis always done by Kahlan, and that param just addresses whether or not Kahlan reports anything about it?

[jails]

The code coverage analysis is not always done by Kahlan.
It's only done when the --coverage options is set (but indeed this option is automatically set when --lcov --clover or --istanbul is set).

For the code coverage annotation I'm closing this since it's a duplicate of #333