On Thursday’s SIGIST meeting, it was great to have such a positive reaction to my workshop and closing talk.
The Fallibility axiom (p41) tells us our sources of knowledge are undependable. The tester is a human being and prone to error. The system is being tested because we are uncertain of its behaviour or reliability. As a consequence, the plan for any test worth running cannot be relied upon to be accurate before we follow it.
Predictions of test status (e.g. coverage achieved or test pass- rate) at any future date or time are notional.
The planning quandary is conveniently expressed in the testing uncertainty principle:
Consequently, if a plan defines completion of testing using test exit criteria to be met at a specified date (expressed in terms of tests run and the status of those tests) it is wise to regard them as planning assumptions, rather than hard targets.
Whichever outcome arises, we still need to think very carefully about what they actually mean in our project.