Skip to content

Pytest

Jump to bottom
shamik edited this page Oct 12, 2024 · 9 revisions

Pytest

Instead of unit tests, python has pytest, which requires less boilerplate code to set up and has a much easier learning curve than unit tests. Official documentation

Install pytest

pip install pytest

First Pytest

Each test must start or end with the substring test, e.g. test_my_name. pytest will run all files of the form test_*.py or *_test.py in the current directory and its subdirectories. Example here. Voila there you have written your first pytest.

There's no need to create a class and all tests can be functions. A simple assert statement always works because of pytest's advanced assertion introspection.

To run the pytest

Since the test is in a tests directory and is in the test_main.py file, we need to mention the same along with the test name(test_my_name) to run it. If we don't then it will run all the tests in the tests directory and any other directory/sub-directory, which contains tests. Try it by downloading this folder and running just pytest.

pytest tests/test_main.py::test_my_name

More useful commands below.

Pytest fixture

Let's say we have a bunch of tests all testing get_age, refer here with the same date "1992/09/08" instead of splitting the dates every time in every test we can create it once and access it for all the tests. This is what a pytest fixture does.

Creating a separate file conftest.py and sharing fixtures

Official link for conftest.py

A single test directory can have several conftest.py inside sub-directories and pytest will initialise them accordingly in the respective tests. More on this in the link above.

For our example, creating a single conftest.py and including all fixtures in them works as good as creating a separate fixture inside the relevant test file. Refer here for the conftest.py file.

Safely setting up and tearing down fixture

In the conftest.py, refer here, the fixtures are being set up before the relevant tests and being torn down accordingly too. Notice that each fixture is changing a single state, for instance. the input_value fixture is creating an input and yielding the input and deleting the input after the function has used it. Irrespective of the test failing the fixture is tearing down the state. This is different from a return statement and is the prefered way to safely set up and tear down a fixture even if there are exceptions. More on this here.

Using built-in pytest fixtures

tmp_path

This is a built-in pytest fixture that provides a temporary directory unique to each test function invocation as a Posix path. It is automatically available to any test or fixture that requests it as an argument.

def test_a(tmp_path):
    assert isinstance(tmp_path.as_posix(), str)

monkeypatch

This is another built-in pytest fixture that allows you to safely set and restore attributes, dictionaries, and environment variables. It is also automatically available to any test or fixture that requests it as an argument.

@pytest.fixture(autouse=True)
def tmp_homedir(self, tmp_path, monkeypatch):
    monkeypatch.setenv("HOME", str(tmp_path))
    return tmp_path

def test_a(self, tmp_homedir):
    assert not list(tmp_homedir.iterdir())
  • @pytest.fixture(autouse=True): This decorator marks the method as a fixture that will be automatically used by all tests in the class.
  • monkeypatch: Another built-in pytest fixture that allows you to modify objects, dictionaries, and environment variables temporarily.
  • monkeypatch.setenv("HOME", str(tmp_path)): Sets the HOME environment variable to the path of the temporary directory.
  • test_a(self, tmp_homedir): A test method that takes tmp_homedir as an argument.
  • tmp_homedir: The fixture defined earlier, providing the temporary home directory path.
  • assert not list(tmp_homedir.iterdir()): Asserts that the temporary home directory is empty.

For a more detailed code refer here.

Using pytest markers

Pytest markers are used to select, skip, set any metadata to the tests. If custom markers aren't configured in a pytest configuration file, refer here for creating a pytest configuration file, then it will throw up warnings.

Getting default and configured pytest markers

pytest --markers

Custom marker slow and others

Creating two custom markers for testing i.e. 1. slow and 2. others. The code below is included in pytest.ini.

markers =
    slow: marks tests as slow (deselect with '-m "not slow"')
    others: marking certain tests as others (deselect with '-m "not others"')

Now executing pytest with these markers won't raise any warnings.

Native markers such as xfail, skip and skipif

xfail

In case you know that a test is expected to fail because the feature that you are creating isn't ready but you already have a test for it, then you can mark that test with @pytest.mark.xfail. This can also be used while using the parametrize marker, refer here or here.

skip

If you want to skip a test without providing any reason, then affix the specific test with @pytest.mark.skip. This can also be used while using the parametrize marker, refer here.

skipif

The same as above but will be executed only on certain condition. E.g. @pytest.mark.skipif(sys.version_info >= (3, 0), reason ="needs to run on python2"). This will skip if the python version is >=3. This can also be used while using the parametrize marker, refer here.

Working with pytest.ini

Specific pytest settings can be configured in this file and generally resides in the root directory of the repo. This file will take precedence overall other pytest configuration files even when empty. The default determined root directory by pytest will always be printed while running pytests. For instance, working with custom markers, defining test paths/directories, minimum version of pytest, etc. Refer here

Running Pytest with a parameter to control maximum failed tests

Sometimes, it's not worth running all the tests in a repo, while there are failures in certain number of them. For such instances, if it's important to stop running the tests even if a single/# of tests fail then it can be set with the --maxfail parameter.

# will stop running the tests if it encounters a single failed test
pytest -v --maxfail 1 <filename/dir>

Mocking with pytest-mock(thin-wrapper around the mock package for easier use with pytest)

This is a package for using mocking tests in pytest.

Installation

pip install pytest-mock

Mocking tests in pytest

Any test, which has the mocker object as an argument to any test, e.g. test_get_operating_system_mocked(mocker) can mock a specific method, class or an object. The following piece of code shows the same and the entire code can be viewed here:

def test_operation_system_is_linux_mocked(mocker):
    # Mock the operatng system function and return False for testing Linux
    mocker.patch('test_main.is_windows', return_value=False)
    assert get_operating_system() == 'Linux'

Here the mocker object patches the is_windows method in test_main file and returns False, so when the get_operating_system() method is called it returns False, which returns Linux and thus the assertion passes. As the pytest-mock library is a wrapper on the mock library it supports all the methods of it and for a detailed list of methods, refer here.

Useful pytest commands

Printing a print statement to the console while running pytest

pytest -s <filename/dir>

Running tests which contain only a certain substring

pytest -v -k "substring" <filename/dir>

Running tests which do not contain a certain substring

pytest -v -k "not substring" <filename/dir>

To display verbose content of the tests; displays all the test names and the status

pytest -v <filename/dir>

Run tests in a module

pytest test_mod.py

Run tests in a directory

pytest testing/

To run a specific test within a module

pytest test_mod.py::test_func

Another example specifying a test method nested inside a test class

pytest test_mod.py::TestClass::test_method

Running pytest in jupyter notebook

import ipytest
ipytest.autoconfig()

Execute a cell containing the tests in a different cell.

%%ipytest

def test_a():
    assert True

Include mock tests

https://realpython.com/python-mock-library/#what-is-mocking

Clone this wiki locally