-
-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Updating visual tests docs for 2.x versions #7827
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
perminder-17
merged 6 commits into
dev-2.0
from
updating-contributor-docs-for-visual-testing-for-dev-2.0
Jul 4, 2025
+22
−12
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
d82a7ea
Updating visual tests docs for 2.x versions
perminder-17 8280fe7
Fixing testing docs
perminder-17 44b5c4b
fixing trailing spaces
perminder-17 c46ff27
Add files via upload
perminder-17 dc23014
adding screenshots.
perminder-17 44f5649
rephrasing
perminder-17 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -25,9 +25,13 @@ As you can see, the subfolders in the `unit` subfolder roughly correspond to the | |
|
||
## Testing frameworks | ||
|
||
p5.js uses [Mocha](https://mochajs.org) as a test runner. It is responsible for running the test code as well as providing a solid framework for reporting test results (i.e., the very long output you see in the terminal when you run the tests!) | ||
p5.js 2.0 runs its tests with [Vitest](https://vitest.dev), which provides Mocha-compatible globals (`suite`, `test`, etc.). It is responsible for running the test code as well as providing a solid framework for reporting test results (i.e., the very long output you see in the terminal when you run the tests!) | ||
|
||
However, Mocha by itself doesn’t do any testing; for that, we need an assertion library. An assertion library is a collection of handy functions that lets us test various properties of our code, such as whether two values are equal, two values are of the same type, whether a function throws an error, and many more. p5.js uses [Chai's `assert` (and `expect`)](https://www.chaijs.com/api/assert/) to write individual statements about how the code should behave. | ||
However, Vitest still needs an assertion layer, and it bundles Chai for that. An assertion library is a collection of handy functions that lets us test various properties of our code, such as whether two values are equal, two values are of the same type, whether a function throws an error, and many more. p5.js uses [Chai's `assert` (and `expect`)](https://www.chaijs.com/api/assert/) to write individual statements about how the code should behave. You can also import `assert` and `expect` straight from Vitest (Vitest bundles Chai internally): | ||
|
||
```js | ||
import { assert, expect } from 'vitest' | ||
``` | ||
|
||
|
||
## Writing unit tests | ||
|
@@ -40,7 +44,7 @@ To start writing unit tests, first pick a unit. A unit can be a function or a va | |
- it should be `true` if multiple keys are pressed | ||
- it should be `false` if no keys are pressed | ||
|
||
In the example below, `suite()` and `test()` are both built-in functions provided by Mocha as part of the test environment. If you look into the test file (eg. `test/unit/events/keyboard.js`) you may find additional built-in functions such as `setup()` and `teardown()` as well. Each `suite()` describes a unit of p5.js that you are writing a test for (a function, a variable, etc). Each `test()` in a `suite()` is an individual test case that checks a single feature/behavior of the unit being tested. The first argument passed to `suite()` and `test()` is a description of the suite/test, and its purpose is to give clear output in the terminal for the test case. | ||
In the example below, `suite()` and `test()` are both built-in functions provided by Vitest’s Mocha-compatible API. If you look into the test file (eg. `test/unit/events/keyboard.js`) you may find additional built-in functions such as `setup()` and `teardown()` as well. Each `suite()` describes a unit of p5.js that you are writing a test for (a function, a variable, etc). Each `test()` in a `suite()` is an individual test case that checks a single feature/behavior of the unit being tested. The first argument passed to `suite()` and `test()` is a description of the suite/test, and its purpose is to give clear output in the terminal for the test case. | ||
|
||
- `p5.prototype.keyIsPressed` is the unit being tested in this suite. | ||
- There are three tests in this suite: | ||
|
@@ -81,7 +85,7 @@ setup(function(done) { | |
|
||
The `p` parameter, which is used in instance mode to access various p5.js variables and functions, is then assigned to `myp5`. This setup allows you to use `myp5` to access p5.js variables and functions anywhere in the testing code to test their functionalities. | ||
|
||
Remember that, as previously mentioned, Mocha is a test runner but by itself doesn’t do any testing; we need Chai for that. Consider the following: | ||
Remember, Vitest supplies the runner; Chai (bundled by Vitest) supplies the assertions. Consider the following: | ||
|
||
```js | ||
test('keyIsPressed is a boolean', function() { | ||
|
@@ -182,10 +186,9 @@ Here are the conventions and best practices that p5.js uses for unit tests which | |
|
||
## Running tests | ||
|
||
The most straightforward way to run the tests is by using the `npm test` command in your terminal. However, `npm test` usually takes a long time to run simply because of the large number of test cases p5.js has. It can also sometimes be a bit repetitive to make some changes, run `npm test`, make some more changes, and run `npm test` again. Here are some tricks that can help streamline this process: | ||
The most straightforward way to run the tests is by using the `npm test` command in your terminal. However, `npm test` usually takes a long time to run simply because of the large number of test cases p5.js has. We don’t need to run `npm test` over and over after each change. As soon as we save our code, we can see in the browser or terminal whether the tests pass or not. | ||
|
||
- Use the `npx grunt watch:main` command to automatically build and test p5.js whenever you save changes to p5.js source files. This will save you from having to manually run tests whenever you are making frequent changes to the codebase. | ||
- You can mark certain test suites to be skipped or be the only suite that is run with the `.skip` and `.only` syntax. | ||
- You can also mark certain test suites to be skipped or be the only suite that is run with the `.skip` and `.only` syntax. | ||
|
||
```js | ||
// This test suite will not run | ||
|
@@ -199,7 +202,11 @@ The most straightforward way to run the tests is by using the `npm test` command | |
}); | ||
``` | ||
|
||
- You can also run `grunt yui:dev` to launch a local server with the reference and a test runner. Once live, you can go to [`http://127.0.0.1:9001/test/test.html`](http://127.0.0.1:9001/test/test.html) to run tests live in your browser. This can be useful if you want to use a debugger in the middle of a test case or if you want to log complex objects. If you want to filter tests by the name of the test case, you can add a search term after a `?grep=` URL parameter, e.g.: [`http://127.0.0.1:9001/test/test.html?grep=framebuffer`](http://127.0.0.1:9001/test/test.html?grep=framebuffer) | ||
- When you run `npm test`, Vitest launches two browser windows. The first window (usually at http://localhost:63315) renders and executes all unit and visual tests; it provides a built‐in interface where you can click on filters (for failed, skipped, passed tests, etc.) to narrow down the results. The layout in this window is self-explanatory. Each test suite and spec is listed with its status, and you can easily expand or collapse sections to see details, rerun individual tests, or inspect error messages. | ||
|
||
- The second window hosts Vitest’s DevTools/BiDi control channel, where you’ll see raw JSON frames exchanged between Vitest and the browser. That second window isn’t an extra test suite, it simply facilitates communication and result reporting, so you can safely ignore it unless you’re debugging Vitest’s runner itself. | ||
|
||
|
||
|
||
|
||
## Visual tests | ||
|
@@ -217,13 +224,11 @@ visualTest('2D objects maintain correct size', function(p5, screenshot) { | |
}); | ||
``` | ||
|
||
If you need to add a new test file, add it to that folder, then add the filename to the list in `test/visual/visualTestList.js`. Additionally, if you want that file to be run automatically as part of continuous integration on every pull request, add the filename to the `visual` list in `test/unit/spec.js`. | ||
No manual registration necessary. Any file placed in `test/unit/visual/cases` is auto-discovered by Vitest. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor: "To add a new test file, place it into |
||
|
||
When you add a new test, running `npm test` will generate new screenshots for any visual tests that do not yet have them. Those screenshots will then be used as a reference the next time tests run to make sure the sketch looks the same. If a test intentionally needs to look different, you can delete the folder matching the test name in the `test/unit/visual/screenshots` folder, and then re-run `npm test` to generate a new one. | ||
|
||
To manually inspect all visual tests, run `grunt yui:dev` to launch a local server, then go to http://127.0.0.1:9001/test/visual.html to see a list of all test cases. | ||
|
||
|
||
Running `npm test` launches Vitest’s interactive UI at http://localhost:63315, where you can view the list of all test cases including visual tests as well. | ||
In a continuous integration (CI) environment, optimizing test speed is essential. It is advantageous to keep the code concise, avoid unnecessary frames, minimize canvas size, and load assets only when essential for the specific functionality under test. | ||
To address scenarios involving operations like asynchronous 3D model rendering, consider returning a promise that resolves upon completing all the necessary tests, ensuring efficiency in your visual testing approach. Here's an example of how you can asynchronous 3D model rendering in your visual tests: | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.