- Provide developers and other parties with information about any adverse event that occurred
- enables to identify specific effects and to isolate the problem with a minimal reproducing test
- enables to correct the potential defect(s) as needed or to otherwise resolve the problem
- Provide test managers a means of tracking:
- the quality of the work product
- the impact on the testing
- Provide ideas for development and test process improvement
When logging defects, you should consider:
- the context or component/system under test
- the test level
- the software development life cycle
Software defects may be reported during:
- static analysis
- dynamic testing
- use of a software product
Testing defects may be reported for issues in:
- code or working system
- user stories
- development or test documents
- user manuals or installation guides
Any defects identified should be investigated and should be tracked from discovery and classification to their resolution. This is not just a target, but a software testing goal.
- Dynamic defect reports may sometimes differ. Defects found during static testing (particularly reviews) will be documented in a different way.
- An example of contents of a defect report can be found in ISO standard (ISO/IEC/IEEE 29119-3) and refers to it as incident reports
- The defect management tools used may automatically manage and fill in some details
A defect report is the documentation of the occurrence, nature, and status of a defect.
Whatever the specific information determined as necessary for defect reports, it is critical that testers enter information that is complete, concise, accurate, objective, relevant and timely.
When a defect is detected (static testing), or a failure is observed (dynamic testing) data should be gathered by the person(s) involved and included in the defect report.
Information from a software defect report should suffice for:
- Management of the report
- Assessment of project status (terms of product quality and test progress)
- Assessment of process capability
Core information gathered should be consistent across the life cycle and ideally across all projects in order to allow for meaningful comparison of process defect data within and across all projects.
Defect data may also include the following (on top of the already presented data for a Defect Report):
- The role of the author (end user, tester, business analyst, technical support, etc.)
- Type of testing performed (usability, performance, regression, etc.)
- Sub-system or component where the issue lies besides the System under test
- Project activity occurring when the issue was found
- Type of defect (based on defect taxonomy)
- Quality characteristics affected by the defect
- Project or product in which the problem lies
- Current owner (assignee)
- Business Impact
- Risks, costs, opportunity and benefits for fix/no fix
- Description of how the defect was resolved
Defect information should support:
- defect density analysis
- trend analysis of defects detected and resolved
- average time from defect detection to resolution
- failure intensity like mean time between failures
Even though the collection of data can assist in:
- test progress monitoring
- exit criteria evaluation
There can be a lot of challenges in properly assessing project status, test progress and process capability when defect report data is of low quality and not properly managed.
Various standards as ISO 9126 being replaced by ISO 25000, IEEE 829, IEEE 1044 and Orthogonal Defect Classification exist to help the Test Manager determine which information to gather for defect reporting.
A defect report typically progresses through a workflow and moves through a sequence of states as it continues through the defect life cycle. In most of these states a single defect participant owns the report and is responsible for carrying out a task which will cause the defect report to be moved to the next state.
Once the defect report reaches a terminal state like closed, cancelled, irreducible or deferred then it does not have an owner anymore as no further action is required.
Defect workflow and states
The INITIAL state
One or more testers gather the information necessary for the person responsible for resolving the defect to reproduce the anomaly also be referred to as the “open” or “new” state.
The RETURNED state
The receiver of the report has rejected the report or is asking the tester to supply further details. May indicate a shortfall in the initial information-gathering process or of the testing itself. This may also be referred to as the “rejected” or “clarification” state.
The CONFIRMATION state
The tester will run a confirmation test to determine whether the fix has solved the problem. The defect can then be “Closed” or “Re-Open”. This may also be referred to as the “resolved” or “verification” state.
This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.
In late 2019 we have launched A Test Manager’s Guide eBook that stands as the base for this article. You can check out more useful test management lessons by enrolling for free to view Chapter 1 – Back to the basics.