Adds Module 6: Testing Code #125
Conversation
Co-Authored-By: David Mascharka <[email protected]>
Co-Authored-By: David Mascharka <[email protected]>
|
Thanks so much for the careful read through @davidmascharka ! Hugely appreciated |
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
| ) | ||
| ``` | ||
|
|
||
| This setup file dictates that a user must have Python 3.6+ installed - we will bar Python 3.5 and below so that we are free to make use of [f-strings](https://www.pythonlikeyoumeanit.com/Module2_EssentialsOfPython/Basic_Objects.html#Formatting-strings) in our code, which were introduced in Python 3.6. Additionally, we will require pytest and hypothesis for running tests; the Hypothesis library will be introduced in a later section. |
There was a problem hiding this comment.
It's been a while I guess, but Python 3.6 is now past end-of-life and so you could just point to https://devguide.python.org/versions/ with a strong recommendation to avoid unsupported versions, and ideally stick to versions for which bugfixes are being issued (i.e. only support older if you're publishing code for others to use, and they can't feasibly update their Python).
| ----> 1 assert count_vowels("aA bB yY", include_y=True) == 4 | ||
|
|
||
| AssertionError: assert 2 == 4 | ||
| + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True |
There was a problem hiding this comment.
| + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True | |
| + where 2 = <function count_vowels at 0x000001B91B913708>('aA bB yY', include_y=True) |
| It is often the case that the process of *describing our data* is by far the heaviest burden that we must bear when writing tests. This process of assessing "what variety of values should I test?", "have I thought of all the important edge-cases?", and "how much is 'enough'?" will crop up with nearly every test that we write. | ||
| Indeed, these are questions that you may have been asking yourself when writing `test_count_vowels_basic` and `test_merge_max_mappings` in the previous sections of this module. | ||
|
|
||
| [Hypothesis](https://hypothesis.readthedocs.io/) is a powerful Python library that empowers us to write a _description_ (specification, to be more precise) of the data that we want to use to exercise our test. |
There was a problem hiding this comment.
I don't think there's a clear distinction between description and specification here, so I'd just delete the parenthetical.
|
|
||
| Hypothesis' [given decorator](https://hypothesis.readthedocs.io/en/latest/details.html#the-gory-details-of-given-parameters) is responsible for: | ||
|
|
||
| - drawing values from Hypothesis' so-called "strategies" for describing input data for our test function |
There was a problem hiding this comment.
| - drawing values from Hypothesis' so-called "strategies" for describing input data for our test function | |
| - drawing values from Hypothesis' "strategies" for describing input data for our test function |
"so-called" has two possible meanings - either to to introduce a new word or phrase, or that the following description is unsuitable or incorrect. Apparently the first is more common in the US but the latter in Commonwealth countries, so to avoid ambiguity I'd just drop it.
|
|
||
|
|
||
| Let's see the `given` decorator in action by writing a simple "test" for which `x` should be integers between 0 and 10, and `y` should be integers between 20 and 30. | ||
| To do this we will make use of the `integers` Hypothesis strategy. |
There was a problem hiding this comment.
| To do this we will make use of the `integers` Hypothesis strategy. | |
| To do this we will make use of the `st.integers()` Hypothesis strategy. |
| <!-- #region --> | ||
| **Exploring other Core Strategies: Solution** | ||
|
|
||
| Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False. |
There was a problem hiding this comment.
| Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False. | |
| Dictionaries of arbitrary size whose keys are positive-values integers and whose values are `True` or `False`. |
| # We need to shuffle the ordering of our characters so that | ||
| # our input string isn't unnaturally patterned; e.g. always | ||
| # have its vowels at the end | ||
| shuffle(letters) | ||
| in_string = "".join(letters) |
There was a problem hiding this comment.
Trouble! Hypothesis will seed the random-number generator, so you shouldn't show using random functions in PBTs.
Or, preferably, use the st.random() strategy to provide a Random instance that you use to do the shuffle, and explain why that's a good idea.
| # This is only one of many ways that you could have created this | ||
| # strategy | ||
| cat_or_dog = st.booleans().map(lambda x: "cat" if x else "dog") |
There was a problem hiding this comment.
I'd prefer to show st.sampled_from(["cat", "dog"]) as the primary example, with the remapped booleans as a curiosity.
| ``` | ||
|
|
||
| The value doesn't change at all because we would have to exceed 64-bits allotted to represent a floating point number. | ||
| Thus `x < x + 1` fails to hold for `x = 2.0 ** 53`. |
There was a problem hiding this comment.
| Thus `x < x + 1` fails to hold for `x = 2.0 ** 53`. | |
| Thus `x < x + 1` fails to hold when `2.0 ** 53 <= x < inf`. |
| **Describing objects that evaluate to `False`** | ||
|
|
||
| ```python | ||
| falsies = st.sampled_from([False, 0, 0.0, "", [], tuple(), {}]) |
There was a problem hiding this comment.
| falsies = st.sampled_from([False, 0, 0.0, "", [], tuple(), {}]) | |
| falsies = st.sampled_from([False, 0, 0.0, "", [], (), {}]) |
I'd also add a comment explicitly noting that the same objects will be generated every time, so this strategy is unsafe if you mutate them. Instead, you could rely on the fact that calling the respective type objects with no arguments returns the empty/falsy value: st.sampled_from([bool, int, float, str, list, tuple, dict]).map(lambda type_: type_()).
@davidmascharka (and anyone else!) I am in the process of completing an early draft of Module 6. I will make note of the sections that are ready for review/edits below. Thank you for your help!
I have pushed a temporary branch that has the updated html for this WIP - so that you can read the content as it would appear on the website. The branch is here. You can clone/download it and view
index.htmlin your browser to get to the PLYMI home page.Ready for Review