Skip to content

Latest commit

 

History

History
19 lines (12 loc) · 2.5 KB

testing.md

File metadata and controls

19 lines (12 loc) · 2.5 KB
Raw

Testing

We try to develop unit tests wherever possible. They test specific components of our project by respecting the F.I.R.S.T Principles of Unit Testing:

  • Fast – Tests that need too long time would annoy the workflow
  • Isolated – Tests should not affect each other. Continuous Integration testing helps a lot at fulfilling this principle.
  • Repeatable – Tests should be independent from the current environment and other side effects.
  • Self-Validating – A test should either pass or fail, no manual inspection is needed to determine the state.
  • Timely – With new features, also new tests have to be implemented

Tests are located in the tests/ directory (surprise!), one test per file. They can be called manually except of the GitHub tests in tests/github/ (see below).

We use GitHub Actions for continuous integration (CI): It runs our specified tests with every GitHub commit. The exit code returned by a test script tells GitHub Actions whether it has passed or failed: Tests with 0 exit code (in Node.js: process.exit(0);) have passed, all others (non-zero exit codes) have failed.

The GitHub repository is configured such that pull requests with failing tests can't be merged. To implement optional tests that don't prevent a PR from being merged, just make it always pass (with exit code 0) and warn the developers via console output or a GitHub comment.

Some tests rely on the information what files have been changed in a PR (GitHub tests, located in tests/github/). They use the helper module pull-request.js to identify the current pull request and commit hash and to add comments. They can't be called manually (because there is no pull request locally). The pull-request.js module can also be used by other non-GitHub tests to create warning comments.

Running the test suite against the whole fixture library would be too much effort in most cases. The test fixture collection (tests/test-fixtures.json) is a set of test fixtures that cover all defined Fixture features (while keeping the set as small as possible). Running tests only against these fixtures ensures a broad coverage with minimal effort. The set can be generated by calling cli/build-test-fixtures.js or simply npm run build.