Software testing tools return on investment

Test tool return on investment (2)

The Test Manager must ensure that all tools introduced in the organization add value and have a positive test tool return on investment. This is mainly done by performing a cost-benefit analysis before acquiring or building a tool and considering both recurring and non-recurring costs when calculating ROI. These are covering at least monetary, resources, time and risks that may impact the value of the tool.

The test team usually used more than one tool and the ROI will be obtained by evaluating the entire software testing tool set and for larger projects or for organizations, a long term and comprehensive software testing tool strategy is recommended.

Nothing comes without risk and a Test Manager should also consider the following when assessing ROI:

  • Immaturity of the organization (not ready)
  • Artifacts created by the tool may be difficult to maintain, requiring multiple revisions when the software under test is changed
  • Reduction of Test Analysts involvement in testing tasks may reduce the value of testing

Besides this, the Test manager role is to also:

  • look at the benefits that may accrue from the use of the tool
  • consider a prospective tool from several different viewpoints like: business; project; user

Non-recurring costs

  • Defining tool requirements to meet the objectives and goals
  • Evaluating and selecting the correct tool and tool vendor
  • Purchasing, adapting or developing the tool
  • Performing the initial training for the tool
  • Integrating the tool with other tools
  • Procuring the hardware/software needed to
  • support the tool

Recurring costs

  • Owning the tool (licensing, maintenance costs of the tool and of the tool artifacts, ongoing training, etc.)
  • Porting the tool to different environments
  • Adapting the tool to future needs
  • Improving the quality and processes to ensure optimal use of the selected tools

Opportunity costs

  • usually inherent in the tool
  • The time spent on acquiring, administering, training and using the tool could have been spent on actual testing tasks
  • More testing resources may be needed up front until the tool goes on line

Benefits that may accrue can be a reduction in

  • Repetitive work
  • Test cycle time (automated regression)
  • Test execution costs
  • Human error in different phases of testing because:
    • Test data may be more valid using data generation tools
    • Test result comparisons may be more accurate using comparison tools
    • Test data entry may be more correct by entering with a scripting tool
  • Effort required to access information about tests:
    • Tool generated reports and metrics
    • Reuse of test assets such as test cases, test scripts and test data

Other viewpoints for a tool prospect

When looking from other viewpoints at a tool prospect, the ones we should most likely consider are:

  • Business viewpoint (Positive ROI is required)
    • we need to assure that the existing and non test tool work together with the new test tool
    • look at improving the processes and inter connectivity between tool usage
  • Project viewpoint (Tool must be effective)
    • The tool may require an appreciable amount of time in order to start earning positive ROI
    • In many cases, ROI may occur in the second release or during maintenance rather than during the initial project when it was implemented
  • Tool user viewpoint (must support in doing tasks)
    • learning curve must be considered to ensure the users will be able to learn the tool quickly with minimal stress
    • When first introduced, test tools require training and mentoring for the users

Test Tool Return on Investment by Lucian C on Scribd

For a broader article on this topic, you can check our Test Manager School or further browse our software testing blog.

This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.

In late 2019 we have launched A Test Manager’s Guide eBook that stands as the base for this article. You can check out more useful test management lessons by enrolling for free to view Chapter 1 – Back to the basics.