Unit tests are a great way for developers to get immediate feedback when introducing new features, behavior, fixing bugs, and to prevent future regressions.
There are many resources describing the benefit and usage of what you can do with unit tests. But, for python, there are few resources devoted to best practices when creating a great pytest test suite.
Based on developer experience, we standardized test suite "best practices" for Carta that you can implement just as easily.
Folder structure for tests follow a defined pattern. The rule to follow is that for any given python file the corresponding test file should be easy to find.
Good test folder structure mimics the root directory. For instance, If the python file you are adding behavior to is located at /apps/vehicles/models.py the corresponding tests for the classes in that file are located at /tests/unit/apps/vehicles/models/.
Inside of the file /apps/vehicles/models.py there is a Car class. The tests for this class are easily found in the file: /apps/vehicles/models/test_car.py
Inside of the file /app/vehicles/helpers.py there are only top level function definitions and no classes. The tests for these functions could be found in /app/vehicles/test_helpers.py.
Inside of test_car.py there will be classes which mimic the public interfaces of the object. For instance, if the Car class has the method start() there will be a class in the test file called, class TestStart.
The functions that belong to the TestStart class correspond to the code under test.
# bad
class TestThatTheCarStarts:
# good
# describing Car.start()
# located in car.py
class TestStart:Be clear about what behavior you are describing. When describing a context, start its description with "when" or "with".
# bad
def test_car_does_not_start_if_no_gas_in_the_tank(self):
car = Car(fuel=None)
with pytest.raises(RuntimeError):
car.start()
# good
def test_when_tank_is_empty_then_car_does_not_start(self):
car = Car(fuel=None)
with pytest.raises(RuntimeError):
car.start()The "one expectation" tip is more broadly expressed as "each test should make only one assertion". This helps you on finding possible errors, going directly to the failing test, and to make your code readable.
In isolated unit specs, you want each example to specify one (and only one) behavior. Multiple expectations in the same example is a code smell that you may be specifying multiple behaviors.
# bad
class TestGetCarInfo():
def test_default_instance_has_factory_settings(self):
car = Car()
info = car.get_car_info()
assert info['color'] == 'gray'
assert info['has_gps'] is False
assert info['interior_material'] is not 'leather'
# good
class TestGetCarInfo():
def test_default_instance_color_is_gray(self):
car = Car()
assert car.get_car_info()['color'] == 'gray'
def test_default_instance_does_not_have_gps(self):
car = Car()
assert car.get_car_info()['has_gps'] is False
def test_default_instance_does_not_have_leather_interior(self):
car = Car()
assert car.get_car_info()['interior_material'] is not 'leather'While writing tests we often need to setup some variables or data structures, then call the method we want to test and after that assert that the results match what is expected. The Arrange, Act, Assert pattern enforces clear distinctions between these activities on a test file. All tests should be divided in up to three sections, creating a consistent reading experience.
- First, arrange all the setup conditions for the test
- Empty line
- Second, act on the behavior being tested
- Empty line
- Third, assert expected results
# bad
def test_when_tank_is_full_then_returns_the_topped_off_amount(self)
fuel_tank = FuelTank(max_volume=50)
car = Car(fuel_tank=fuel_tank)
topped_off = car.fill_tank(55)
assert topped_off == 5
# good
def test_when_tank_is_full_then_returns_the_topped_off_amount(self)
fuel_tank = FuelTank(max_volume=50)
car = Car(fuel_tank=fuel_tank)
topped_off = car.fill_tank(55)
assert topped_off == 5Decorators become very nasty to read once you have more than one. The pytest-mock library description sums up the problem nicely.
- test functions must receive the mock objects as parameter, even if you don’t plan to access them directly.
- The argument order depends on the order of the decorated patch functions.
- receiving the mocks as parameters doesn’t mix nicely with pytest’s approach of naming fixtures as parameters, or
pytest.mark.parametrize; - you can’t easily undo the mocking during the test execution;
# bad
@patch.object(FuelTank, 'get_fuel_left', lambda self: 0)
def test_when_no_fuel_exists_returns_empty_1(self):
mocked_tank = FuelTank()
car = Car(fuel_tank=mocked_tank)
assert car.get_remaining_fuel() == 'empty'
# good
def test_when_no_fuel_exists_returns_empty(self):
mocked_tank = FuelTank()
mocked_tank.get_fuel_left = MagicMock(return_value=0)
car = Car(fuel_tank=mocked_tank)
assert car.get_remaining_fuel() == 'empty'When needing to mock a property, use the pytest-mock library. Decorators create too much code and become hard to read when needing to mock multiple properties. The mocker fixture provided by pytest-mock makes things clean and easy to read.
# bad
@patch.object(Car, 'condition', None)
@patch.object(Car, 'year', None)
@patch.object(Car, 'miles', None)
@patch.object(Car, 'base_price', None)
def test_returns_a_value(self):
car = Car()
car.condition = 'very good'
car.year = 2005
car.miles = 65000
car.base_price = Decimal(25000)
assert car.get_value() == Decimal('7812.5')
# good
def test_better_returns_a_value(self, mocker):
car = Car()
mocker.patch.object(Car, 'condition', 'very good')
mocker.patch.object(Car, 'year', 2005)
mocker.patch.object(Car, 'miles', 65000)
mocker.patch.object(Car, 'base_price', Decimal(25000))
assert car.get_value() == Decimal('7812.5')Don't do it in the class directly, or the mock will remain after the test
finishes and will affect all the following tests that uses that class.
With mocker.patch.object, the class will be rolled back to its original state at
the end of the test.
# really bad (please, be carefull not to do this)
def test_returns_a_value(self, mocker):
Car.condition = 'very good'
Car.year = 2005
Car.miles = 65000
Car.base_price = Decimal(25000)
Car.break = mocker.MagicMock()
Car.a_class_or_static_method = mocker.MagicMock()
car = Car()
assert car.get_value() == Decimal('7812.5')
# good
def test_returns_a_value(self, mocker):
mocker.patch.object(Car, 'a_class_or_static_method', mocker.MagicMock())
car = Car(
condition='very good',
year=2005,
miles=65000,
base_price=Decimal(25000),
)
car.break = mocker.MagicMock() # This is an instance, so it's fine to assign directly
assert car.get_value() == Decimal('7812.5')This is an exception where there's no discussion, just follow it and our test suite will be fine.