Don't persist fixtures in test database between tests #2595
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #2523.
Replaces the process-level universe fixture that puts instances in the database that persist through the session and builds fixtures to retrieve copies of them with standard pytest-style fixture functions. This is work-in-progress and there's still quite a lot to do but I'd like some feedback on the approach.
The commits could made clearer by separating out moving things around within fixtures.py from other changes and so on. But I haven't spent the time to do that yet. There's still likely to be a Big Commit that changes the approach and fixes the tests. I don't currently think it's worth the effort of adding the new fixtures one at a time in separate commits, and those intermediate steps with a hybrid approach would look a bit confusing.
This means the universe of instances isn't always the same. Fixtures declare which other fixtures they depend upon. Several tests have been modified to explicitly declare things they depend on. I've provided fixtures for "instantiate lots of codelists and versions" where there are tests that want that such as the API tests. There were comments describing things like "this codelist has 4 versions" -- I haven't updated these yet to something more accurate. I also noted those weren't always up-to-date before these changes.
Coding system releases instances are still loaded into the core database once per session (setup_coding_systems). I think this can be improved by having a function-level fixture that loads the coding system release for a specific coding system and depends on the session-level fixtures that load data to the secondary coding system databases.
There are still a handful of test failures I need to resolve.
Making this change has knock-on implications I haven't resolved/investigated yet:
I've changed the password hasher in tests to a faster less secure one as we now create users frequently (conftest.py).
The tests currently run a few second faster than before but probably due to the test errors and failures above. But I'm hopeful this approach is approximately as fast as before. We could look at multi-worker pytest-xdist.
We could possibly benefit by having FactoryBoy-style factories for different objects. create_codelists is a factory fixture, for example.
I've removed some asserts that didn't make sense in isolation and that we don't want to run every time the fixture is called (previously they would only have been called once). We could add these back in as tests of the fixtures.
There were two tests where the use of codes fixtures seemed to be the wrong way around between them, I haven't fully understood these yet: