The software testing manager must know which test tool success factors should be assessed for each new test tool added within the project or organization, but to also consider the importance of integrating the software testing tools in the software development life cycle within the organization.
Some of the most common software testing tools success factors are:
- Rolling out the tool to the rest of the organization incrementally
- Adapting and improving processes to fit with the use of the tool
- Providing training, coaching, and mentoring for tool users
- Defining guidelines for the use of the tool
- Implementing a way to gather usage information from the actual use of the tool
- Monitoring tool use and benefits
- Providing support to the users of a given tool
- Gathering lessons learned from all users
This can be achieved by defining reporting requirements for the tools during the selection process. These requirements must be properly implemented during tool configuration to ensure that the information tracked by the tools can be reported in ways that will be understandable to the stakeholders.
Simply acquiring a software testing tool does not guarantee success. Each new tool introduced into an organization will require effort to achieve real and lasting benefits.
In order to have a smooth &successful implementation, there are a number of things that should be considered when selecting and integrating test execution and test management tools into an organization.
Potential test tool benefits
Potential benefits of using tools to support test execution (test automation) include:
- Reduction in repetitive manual work (save time)
- running regression tests
- environment set up/tear down tasks
- re-entering the same test data
- checking against coding standards
- Greater consistency and repeat-ability
- test data is created in a coherent manner
- tests are executed by a tool in the same order with the same frequency
- tests are consistently derived from requirements
- More objective assessment
- static measures
- Easier access to information about testing
- statistics and graphs about test progress
- defect rates and performance
Potential testing tool risks
Potential RISKS of using tools to support test execution:
- Expectations for the tool may be unrealistic
- The time, cost and effort for the initial introduction of a tool may be under-estimated
- The time and effort needed to achieve significant and continuing benefits may be under-estimated
- The effort required to maintain the test assets generated by the tool may be under-estimated
- The tool may be relied on too much (seen as a replacement for test design or execution, or use of automated testing where manual would be better)
- Version control of test assets may be neglected
- An open source project may be suspended
- Relationships and interoperability issues between critical tools may be neglected, such as requirements management tools, configuration management tools, defect management tools and tools from multiple vendors
- The tool vendor may go out of business, retire the tool, or sell the tool to a different vendor
- The vendor may provide a poor response for support, upgrades, and defect fixes
- A new platform or technology may not be supported by the tool
- There may be no clear ownership of the tool
This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.
In late 2019 we have launched A Test Manager’s Guide eBook that stands as the base for this article. You can check out more useful test management lessons by enrolling for free to view Chapter 1 – Back to the basics.