How to improve testing process

Just as organizations use software testing to improve software, software testing process improvement techniques can be selected and used to improve the process of developing software and the resulting software deliverable. This is why a test manager must know how to improve the test process and knowing this must become a software testing basic for the up and coming test manager.

Only limited attention is given to the test process in the various software process improvement models, such as CMMI® even through testing accounts for a major part of total project costs. This is why specific test improvement models were developed:

• Test Maturity Model integration (TMMi®)
• Systematic Test and Evaluation Process (STEP)
• Critical Testing Processes (CTP)
• TPI Next®

The software testing models presented are not intended to be a recommendation for use, but are presented here to provide a representative view of how the models work and what they include. The Test Manager role is to decide which model best fits the organization or project.

Process improvements are relevant to software development as well as to the testing process:

• Learning from one’s own mistakes makes it possible to improve the process that organizations are using to develop and test software
• the Deming improvement cycle: Plan, Do, Check, Act, has been used for many decades, and is still relevant

One premise for how to improve the test process is based on the belief that the quality of a system is highly influenced by the quality of the process used. Improved quality in the software industry reduces the need for resources to maintain the software and thus provides more time for creating more and better solutions in the future.

Test Process improvement models are split into:

Process reference model

• provides a maturity measurement as part of the assessment
• Evaluate an organization’s capability compared with the model
• Evaluate the organization within the framework
• Provide a road map for improving the process

Content reference model

• provides business-driven evaluations of an organization’s opportunities to improve
• Including bench marking against industry averages using objective measurements
• This evaluation can be used to create a road map for improving the process

Test process improvement can also be accomplished without models by using analytical approaches and retrospective meetings.

Test process improvement approach

Industry standard models are helping to develop cross-organization metrics and measures that can be used for comparison. Several sets of recommended processes are: STEP, TMMi, TPI Next, CTP. All of these models allow an organization to determine where it stands in terms of its current test processes.

Stages Models: TMMI and CMMI Provide standards for comparison across different companies and organizations.

Continuous models (CTP, STEP and TPI Next) allow an organization to address its highest priority issues with more freedom in the order of implementation.

Once an assessment is performed:

• TMMi and TPI Next suggest a road map for improving the test process
• STEP and CTP provide the means to determine where its greatest process improvement return on investment will come from and leave it to the organization to select the appropriate road map.

Once it has been agreed that test processes should be reviewed and improved, the process improvement implementation steps to be adopted for this activity could be as defined in the IDEAL model:

• • Initiating the improvement process
  • Diagnosing the current situation
  • Establishing a test process improvement plan
  • Acting to implement improvement
  • Learning from the improvement program

1. Initiating the improvement process

• Before the process improvement activities start
• Objectives, goals, scope and coverage of the process improvements are agreed on by the stakeholders
• The choice of the process improvement model is taken (publicly available or developed internally)
• Success criteria should be defined and a method by which they will be measured throughout the improvement activity should be determined.

2. Diagnosing the current situation

• The agreed assessment approach is undertaken
• A test assessment report is created which contains an appraisal of:
  • current testing practice
  • a list of possible process improvements

3. Establishing a test process improvement plan

• The list of possible process improvements is prioritized
• The prioritization could be based on:
  • return on investment
  • risks
  • alignment with organizational strategy
  • measurable quantitative or qualitative benefits
• Having established the priority order, a plan for the delivery of the improvements is developed

4. Acting to implement improvement

• The test process improvement plan for the delivery of the improvements is implemented
• This could include any training or mentoring required, piloting of processes and ultimately their full deployment

5. Learning from the improvement program

• Having fully deployed the process improvements, it is essential to verify which benefits were received.
• It is also important to check which of the success criteria for the process improvement activity have been met
• Depending on the process model used, this stage of the process is where monitoring of the next level of maturity starts and a decision is made to either start the improvement process again, or to stop the activity at this point.

This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.