Software testing Defect Management (or Incident Management) must happen across all test levels and it is critical for understanding where the quality of the product/project. Software testing basics consider the dealing with the software testing issues or defects required to reach the test levels exit criteria. A large part of the time of a Test Manager is spent performing Defect Management.
Defect Management is the process of recognizing and recording defects, classifying them, investigating them, taking action to resolve them, and disposing of them when resolved.
An organization’s defect management process and the tool used to manage this work are of critical importance:
- Information gathered by effective management of defects allows to gain insight on the state of a project throughout the development life cycle
- By collecting and analyzing data over time, this can help locate areas of potential improvement for testing and development processes
Defect report = Documentation of the occurrence, nature, and status of a defect.
Whatever the specific information determined as necessary for defect reports, it is critical that testers enter information that is complete, concise, accurate, objective, relevant and timely.
When a software testing defect is detected (static testing), or a failure is observed (dynamic testing) data should be gathered by the person(s) involved and included in the defect report.
Information from a defect report should suffice for:
- Management of the report
- Assessment of project status (terms of product quality and test progress)
- Assessment of process capability
Core information gathered should be consistent across the software testing life cycle and ideally across all projects in order to allow for meaningful comparison of process defect data within and across all projects.
Defect information should support:
- defect density analysis
- trend analysis of defects detected and resolved
- average time from defect detection to resolution
- failure intensity like mean time between failures
- Even though the collection of data can assist in:
- test progress monitoring
- exit criteria evaluation
There can be a lot of challenges in properly assessing project status, test progress and process capability when defect report data is of low quality and not properly managed.
Various standards as ISO 9126 being replaced by ISO 25000, IEEE 829, IEEE 1044 and Orthogonal Defect Classification exist to help the Test Manager determine which information to gather for defect reporting.
A defect report typically progresses through a workflow and moves through a sequence of states as it continues through the defect life cycle. In most of these states a single defect participant owns the report and is responsible for carrying out a task which will cause the defect report to be moved to the next state.
Once the defect report reaches a terminal state like closed, cancelled, could not reproduce or deferred then it does not have an owner anymore as no further action is required.
Software Defect workflow and states
The INITIAL state
- One or more testers gather the information necessary for the person responsible for resolving the defect to reproduce the anomaly
- Also be referred to as the “open” or “new” state
The RETURNED state
- The receiver of the report has rejected the report or is asking the tester to supply further details
- May indicate a shortfall in the initial information-gathering process or of the testing itself.
- This may also be referred to as the “rejected” or “clarification” state
The CONFIRMATION state
- The tester will run a confirmation test to determine whether the fix has solved the problem.
- The defect can then be “Closed” or “Re-Open”
- This may also be referred to as the “resolved” or “verification” state
This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.