Seven Principles of Testing

For the purposes of the ISTQB CTFL, there are seven core principles of testing:

Testing shows the presence of defects, not their absence.

We can never prove through testing that there are no defects, only that there are defects. Through testing we reduce the probability of undiscovered defects remaining to be found by end users. Just because there are no found defects, doesn’t mean there are none present.

Exhaustive testing is impossible.

Testing all possible combinations is not feasible except for the most manini of cases. Don’t try to test exhaustively; rather, test smart using risk analysis, appropriate test techniques, and prioritization.

Early testing saves time and money.

Get involved as early as possible in testing. Shift Left on the timetable for involvement. Testing earlier in the SDLC reduces cost changes.

Defects cluster together.

It’s possible a smaller number of modules contain most defects during pre-release testing. Use the defect clustering principle to help with testing focus. Don’t waste time on an older established feature when the new feature will likely contain the next cluster. Note this doesn’t mean not review for regression… use common sense.

Beware of the pesticide paradox.

Just as how over time the same pesticide is no longer effective for insects, running the same tests over and over will not produce the same level of results as the tests over time will be less effective. One exception is for automated regression testing in which the paradox may be beneficial to help reduce regression defects. Furthermore, tests need regular review and maintenance to continue to find new defects.

Testing is context dependent.

Testing for all items will not be the same… it depends on the context of the system under test. For example, safety-critical ICS are tested more thoroughly with a lower margin for error than a simple static website (such as this one). Additionally, different project types will utilize different types and levels of testing.

Absence-of-errors is a fallacy.

It is not possible of feasible for testers to run all possible tests and find all possible defects. The above principles allude to that. If the expectation is that this is possible then all parties involved will be sorely disappointed. To continue, just finding and fixing a large number of defects will not guarantee system success. The system could still be difficult to use, not fulfill requirements, or otherwise still rank sub-par against competitors.