Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve test coverage #2

Open
Technologicat opened this issue Oct 17, 2020 · 4 comments
Open

Improve test coverage #2

Technologicat opened this issue Oct 17, 2020 · 4 comments
Labels
enhancement New feature or request
Milestone

Comments

@Technologicat
Copy link
Owner

Add tests for any parts that are nontrivial and/or can be tested with reasonable effort.

  • Quasiquote system
  • Corner cases
    • Multiple macro invocations in the same with
    • Decorator macro that edits the decorator list

May need a small utility to clear .pyc caches, to make the test reproduceable.

Too bad we can't use unpythonic.test.fixtures to test this.

@Technologicat Technologicat added the enhancement New feature or request label Oct 17, 2020
@Technologicat Technologicat added this to the 3.0.0 milestone Oct 17, 2020
@Technologicat Technologicat modified the milestones: 3.0.0, 3.1.0 Nov 12, 2020
@Technologicat
Copy link
Owner Author

We could also run the demos as part of the tests, to check that mcpyrate doesn't crash on them - so that we can guarantee them to be compatible with the mcpyrate version being tested.

However, we also need more real unit tests for the mcpyrate codebase itself, in order for the test run of the demos not to exaggerate its coverage. Tests should do more than just assert that there are no crashing statements ;)

@Technologicat Technologicat modified the milestones: 3.1.0, 3.2.0 Feb 11, 2021
@Technologicat
Copy link
Owner Author

Also, at the very least, we need a test runner script to run all demos and see if any of them crashed.

@Technologicat Technologicat modified the milestones: 3.2.0, 3.3.0 Apr 7, 2021
@Technologicat
Copy link
Owner Author

Technologicat commented May 8, 2021

As of 9bf1461 (release 3.3.0 and later), runtests.py runs both tests and demos.

As of 42d93d4 (release 3.2.0 and later), macropython -c . clears .pyc caches in the current directory. The test runner calls this functionality programmatically, and runs everything twice - once from a clean slate, and once with existing bytecode caches.

Both tests and demos are discovered automatically, so new ones are easy to add. See CONTRIBUTING.md (and runtests.py itself) for details.

I'm leaving the issue open for now, because as of release 3.5.0, we still need a lot more tests. Porting our sister proejct unpythonic to use mcpyrate has already lead to discovering (and fixing!) some bugs in mcpyrate that would have otherwise gone unnoticed.

@Technologicat Technologicat changed the title Add automated tests Improve test coverage May 8, 2021
@Technologicat
Copy link
Owner Author

Now that we have a proper CI process (see #5), what remains is to bump up the test coverage significantly. Issue title changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant