Many test managers, test leads or testers focus on the tactical details of testing (aspects specific to the task or test level) while ignoring the larger strategic (higher level) issues related to testing that other project participants care about.
The Test Manager must work to optimize testing so that good business value is delivered:
- Testing excessively does not deliver good business value, because the testing will impose unreasonable delays and cost more than it saves.
- Testing too little does not deliver good business value because too many defects will be delivered to users
The optimum business value of testing lies between those two extremes and the Test Manager must help testing stakeholders understand this and the value delivered by testing.
Testing delivers value to the organization, project, and/or operation in both quantitative and qualitative ways:
Qualitative values such as:
- improved reputation for quality
- smoother and more-predictable releases
- increased confidence, protection from liability
- reducing risk of loss of whole missions or lives
Quantitative values such as:
- include finding defects that are prevented or fixed prior to release
- finding defects that are known prior to release (not fixed but documented, with workarounds)
- reducing risk by running tests
- delivering information on project, process, and product status
Cost of quality
The Test Manager should understand which of these values apply for their organization, project, and/or operation, and be able to communicate about testing in terms of these values.
A well-established method for measuring the quantitative value and efficiency of testing is called cost of quality (or, sometimes, cost of poor quality).
It involves classifying project and operational costs into four categories related to product defect costs:
- Costs of prevention training developers to write more maintainable or secure code
- Costs of detection writing test cases, configuring test environments, and reviewing requirements
- Costs of internal failure fixing defects detected during testing or reviews, prior to delivery
- Costs of external failure support costs associated with defective software delivered to customers
This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.