Practice writing specific tests
We have previously looked at automated tests, and how to make them specific, fast and deterministic in the presence of concurrency. But there are many other properties that we want our tests to have; some of my favorites are:
- Inspiring, ie. each passing test gives you confidence.
- Specific, ie. the cause of failure is obvious from the test.
- Predictive, ie. no failure means all is good.
(For more properties see Kent Beck’s excellent post Test Desiderata.)
These all have a common theme of specificity. (1) states that a test should not test zero properties. (2) states that a test should not test multiple properties. (3) states that collectively our tests should be complete.
Writing specific tests is difficult because we need to spot how code can fail. To help with this we have several heuristics, such as testing the boundaries, or Myers heuristic: “You can test many positive cases together, but only one negative at a time.”
Practice
To help practice writing specific tests I have constructed a GitHub repo with nine different versions of the same function. They all look natural, but all have bugs, except for one. Sort of a Where’s Waldo but with bugs. The function has a fairly simple purpose:
- take an array of floats,
- find the average, and
- subtract the average from each element.
The exercise is straight forward: construct a test suite to weed out all the bad implementations. Keep in mind:
- A good test tests something — ie. fails at least one of the erroneous programs.
- A good test is specific — ie. fails as few implementations as possible. (Only one of my tests fail more than one implementation.)
- A good test is correct — ie. never fails the working implementation.
- A good suite should be complete — ie. fail all but the working implementation.
My final test suite has 11 test cases, how many does your solution have? Comment below.
If you like this sort of thing, you are probably proud of being technically proficient, then you will probably also like my refactoring book:
