Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Measure true coverage (of mcpyrate tests) #27

Open
Technologicat opened this issue May 8, 2021 · 0 comments
Open

Measure true coverage (of mcpyrate tests) #27

Technologicat opened this issue May 8, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@Technologicat
Copy link
Owner

As of 8b2b47c, coverage measurement includes code covered not only by tests, but also by demos.

Demos don't necessarily test anything other than that they don't crash, so the coverage report is overly optimistic.

We need a flag in runtests.py for whether it should run tests and demos (for testing), or tests only (for coverage measurement).

Results are likely horrifying at this point, but an honest result is better than an overly optimistic estimate.

Despite the abysmal test coverage, mcpyrate is actually working pretty well, it can run all of unpythonic just fine. It's only the automated tests that are lacking (see #2). Obviously we need a lot more tests, but producing an accurate coverage result is a good first step so we can actually see what needs to be done.

@Technologicat Technologicat added the enhancement New feature or request label May 8, 2021
@Technologicat Technologicat added this to the 3.x milestone May 8, 2021
@Technologicat Technologicat self-assigned this May 8, 2021
@Technologicat Technologicat changed the title Measure true coverage Measure true coverage (of mcpyrate tests) May 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant