Update test code standards #255
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
liamfarley13
commented
Nov 6, 2023
liamfarley13
commented
- update test code standards doc
- typo
- typo
- update test code standards
- pr comments for test code standards
- update test code standards
rsj-github
approved these changes
Nov 7, 2023
| ### Unit Testing | ||
| #### Test a single Method in isolation | ||
| * Test naming should follow a convention agreed by the team. An example is **Method_Action_Assertion** e.g. ValidateObject_WhenObjectPropteryIsNull_ThrowsArgurmentException | ||
| * Tests should be isolated, i.e. have no dependencies on other tests nor on order of execution. |
There was a problem hiding this comment.
nor test data setup by a previous test case
| * Test naming should follow a convention agreed by the team. An example is **Method_Action_Assertion** e.g. ValidateObject_WhenObjectPropteryIsNull_ThrowsArgurmentException | ||
| * Tests should be isolated, i.e. have no dependencies on other tests nor on order of execution. | ||
| * Unit tests should take very little time to run. Milliseconds not seconds! | ||
| * Unit tests are standalone, can be run in isolation, and have no dependencies on any outside factors such as a file system or database. |
There was a problem hiding this comment.
maybe spell out that this is targeted on the smallest atomic unit eg method/function and not class with usually has multiple functions
| * Running a unit test should be consistent with its results, it always returns the same outcome if you do not change anything in between runs. | ||
| * The test should be able to automatically detect if it passed or failed without any human interaction. | ||
| * A unit test should not take a disproportionately long time to write compared to the code being tested. If you find testing the code taking a large amount of time compared to writing the code, consider a design that is more testable. | ||
| * Unit Test coverage should aim to cover around 80% of code. |
There was a problem hiding this comment.
they should focus on the most complex functionality in order to ensure these are tested thoroughly
Comment on lines
-29
to
-44
| * Consider whether using these are required - | ||
| * will it add value, | ||
| * are they clearly defined and allow for ease in maintenance, | ||
| * will a stakeholder be reading the tests, | ||
| * or can the extra technical layer be avoided? | ||
| * If using, then need to consider their accessibility | ||
| * using (see livingdocs [https://docs.specflow.org/projects/specflow-livingdoc/en/latest/sbsguides/sbsazdo.html] with in the Azure Devopsm framework, or | ||
| * publish the tests to a website for easy access using Pickles (see our [Pickles Example Project](test-repositories.md)) | ||
| * If considered necessary then: | ||
| * The feature name should reflect the area being tested | ||
| * Is there a need for a background task, moving repeated steps into a Background that will be run before eash scenario | ||
| * The type of Scenario that could be used eg Single 'Scenario' or data driven 'Scenario Outline' | ||
| * The scenario name should reflect the purpose of the test, e.g. “Ensure two numbers are added correctly” rather than “add two numbers” or “add” | ||
| * Similar scenarios should be in one feature file – a feature file should only contain similar scenarios | ||
| * Ensure only one result is tested per scenario, try to avoid having too many assertions, better to split into different scenario | ||
| * Perform technical setup and teardown in the Steps classes, not in the Gherkin. |
There was a problem hiding this comment.
As we're not actively promoting the use of specflow etc (in fact we seem to be warning against it). I didn't really seems like this document was a good place for it
nevillejrbrown
requested changes
Nov 7, 2023
| #### Multiple components integration with real dependencies (i.e. database integration) | ||
| * Determine test data ahead of time and ensure this test data is reusable. | ||
| * Ensure integration tests have as few external dependencies as possible outside of the test code. | ||
| * Fakes, mocks and stubs are permitted, provided the test is asserting integration with a real dependency (e.g. database or message broker integration) |
There was a problem hiding this comment.
"provided the test is asserting integration with a real dependency" - I'm not sure what this means...?
|
|
||
| * Only use for e2e tests or explicit UI features at this level | ||
| * Interactions with UIs should be abstracted into a separate service/facade, not alongside the test code (e.g., Page Object Model) | ||
| ### Unit Testing |
There was a problem hiding this comment.
There is already a unit test policy here which is owned by the Lead Devs. We shouldn't duplicate. Maybe leave out this section and add a reference?
| * Running a unit test should be consistent with its results, it always returns the same outcome if you do not change anything in between runs. | ||
| * The test should be able to automatically detect if it passed or failed without any human interaction. | ||
| * A unit test should not take a disproportionately long time to write compared to the code being tested. If you find testing the code taking a large amount of time compared to writing the code, consider a design that is more testable. | ||
| * Unit Test coverage should aim to cover around 80% of code. |
There was a problem hiding this comment.
The Lead Devs chose not to put a number on coverage. I don't think we should put one in here
| @@ -1,49 +1,73 @@ | |||
| # Test Automation Standards | |||
|
|
|||
| ## C# and Java | |||
| ## Test Code | |||
There was a problem hiding this comment.
This doc strays into the same area as the microservice testing doc, i.e. talking about the different types of testing and how much of each you should do. There's a lot of duplication. Can you please consolidate into a single doc?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.