Test Management in context

test management in context

A central responsibility of a software test manager is to secure and utilize resources (people, software, hardware, infrastructure, etc.) to carry out value-adding processes. These processes are often part of a project or a Program aimed at delivering software or a system for internal or external use. This is why we need software test management in context of the project or product under work in order to improve the business value of testing.

For this, the Test Manager role is to arrange:

  • the software testing process
  • associated activities
  • work products

According to the needs and circumstances of:

  • stakeholders
  • activities (ex: software development life cycle)
  • work products (ex: requirements, specifications)

Understanding testing stakeholders

People are stakeholders of software testing when they have an interest in the testing activities, the testing work products, or the quality of the final system or deliverable. Such interests can be:

  • direct or indirect involvement in the testing activities
  • direct or indirect receipt of testing work products
  • direct or indirect effect by the quality of the deliverable produced by the project or program

Below is an example (not comprehensive) of stakeholders and their roles:

  • Developers, development leads, and development managers
    • implement the software under test
    • receive test results
    • must take action based on those results
  • Database architects, system architects, and designers
    • design the software
    • receive test results
    • must take action on those results
  • Marketing and business analysts
    • determine the features and the level of quality inherent in those features
    • involved in defining needed test coverage
    • reviewing test results
    • making decisions based on test results
  • Senior management, product managers and project sponsors
    • often involved in defining needed test coverage
    • reviewing test results
    • making decisions based on test results
  • Project managers
    • responsible for managing their projects to success
    • balance quality, schedule, feature, and budget priorities
    • often procure the resources required for the test activities
    • collaborate with the Test Manager in test planning and control
  • Technical support, customer support and help desk staff
    • support the users and customers
  • Direct and indirect users
    • use the software directly
    • receive outputs or services produced or supported by the software

Test Managers must:

  • identify the specific testing stakeholders for their project or Program
  • understand the precise nature of the stakeholder relationship with testing and how the test team serves the needs of the stakeholders

Test Managers should identify the other software development life cycle activities and work products that affect testing and/or are affected by testing.

Software Delivery Life cycle Activities and Work Products

Software testing exists in the context of a larger set of software development lifecycle activities.

The Test Manager must:

  • plan and guide the testing activities with an understanding of how these other activities and their work products affect testing
  • identify the other software development life cycle activities and work products that affect testing and/or are affected by testing

Software testing is closely interconnected and related to:

Requirements engineering and management

  • The Test Manager must:
    • consider requirements during the scoping and estimation of test effort
    • remain aware of changes to the requirements
    • exercising test control actions to adjust to those changes
  • Technical Test Analysts and Test Analysts should participate in requirements reviews

Configuration management, release management, and change management

  • The Test Manager must:
    • establish the test object delivery processes and mechanisms
    • capture test object delivery processes and mechanisms in the test plan
  • and may ask Test Analysts and Technical Test Analysts to create build verification tests and to ensure version control during test execution

Project management

  • The Test Manager must:
    • provide schedule and resource requirements to the Project Manager
    • work with the Project Manager to understand changes in the project plan and exercise test control actions to adjust to those changes

Software development and maintenance

  • The Test Manager should:
    • work with Development Managers to coordinate the delivery of test objects, including content and dates of each test release
    • participating in defect management

Technical support

  • The Test Manager should:
    • work with the Technical Support Manager to ensure proper delivery of test results during test closure to make sure that the support team is aware of aware of known failures and workarounds
    • work with the Technical Support Manager to analyze production failures in order to implement test process improvements

Production of technical documentation

  • The Test Manager should:
    • work with the Technical Documentation Manager to ensure delivery of documentation for testing in a timely fashion as the management of defects found in those documents

Test management in context foe aligning testing with Life cycle Activities

Testing should be an integral part of the project, regardless of the software development models used.

Sequential models: Waterfall, V-model and W-model

  • All of the work products and activities for a given phase are completed before the next phase begins
  • Test planning, test analysis, test design and test implementation proceed in an overlapping fashion with project planning, business/requirements analysis, software design and development
  • Test execution proceeds sequentially according to the test levels

Test Management in context 2

Iterative or incremental models: Rapid Application Development (RAD) and Rational Unified Process (RUP)

  • The features to be implemented are grouped together and then the project phases and their work products and activities occur for each group
  • The phases may be done either sequentially or can overlap
  • The iterations themselves may be sequential or overlapping
  • During project initiation, high-level test planning and test analysis occurs in parallel with the project planning and business/requirements analysis
  • Detailed test planning, test analysis, test design, and test implementation occurs at the beginning of each iteration, in an overlapping fashion
  • Test execution often involves overlapping test levels
  • Each test level begins as early as possible and may continue after subsequent, higher levels started

Agile: SCRUM and Extreme Programming (XP)

  • These are iterative life cycles where the iterations are very short (often two to four weeks)
  • The work products and activities for each iteration are concluded before the next iteration starts
  • Testing is similar to iterative models, but with a higher degree of overlap of the various testing activities with the development activities, including considerable overlap of test execution (at various levels) with the development activities
  • All of the activities in an iteration, including the test activities should be complete before the next iteration starts
  • In an Agile project, the role of the Test Manager usually changes from a direct managerial role to a technical authority/advisory role

Test Management in context 3

Spiral

  • Prototypes are used early in the project to confirm feasibility and to experiment with design and implementation decisions
  • The order of prototyping is based on the level of business priority and technical risk
  • These prototypes are tested to determine what aspects of the technical problems remain unsolved.
  • Once the main technical problems are resolved, the project proceeds according to either a sequential or iterative model

In order to properly align testing activities within the life cycle, the Test Manager must have a detailed understanding of the life cycle models used.

For example, in V-model we have:

  • System Test planning activities occur concurrently with project planning
  • System Test control continues until system test execution and closure are complete
  • System Test analysis and design activities occur concurrently with requirements specification, system architectural and component design specification.
  • System Test implementation activities might start during system design and would typically occur concurrently with development and unit test. Work on system test implementation activities stretch often until just days before the start of execution
  • System Test execution begins when system test entry criteria is met and execution continues until the system test exit criteria are met.
  • Evaluation of the system test exit criteria and reporting of system test results occur throughout test execution, generally with greater frequency and urgency as project deadlines approach.
  • System Test closure activities occur the system test exit criteria is met and system test execution is declared complete. Sometimes it can be delayed until after acceptance testing is over

In an iterative or incremental life cycle, the same tasks must be performed but the timing and the extent may vary:

  • rather than being able to implement the entire test environment at the beginning of the project, it may be more efficient to implement only the part needed for the current iteration
  • the farther ahead the planning occurs, the farther ahead the scope of the test process can extend
  • test execution and reporting may also be influenced by the lifecycle being used by the team
    • effective to produce complete reports and to conduct post-iteration review sessions before the start of the next iteration
    • by treating each iteration as a mini-project, the team is given an opportunity to correct and adjust based on what occurred before
    • as iterations may be short tasks might be conducted in a manner in which testing can be reported based on task completion
    • process issues experienced in one iteration can easily affect and recur in the next iteration if corrective measures are not taken

Test strategy and test management in context

The Test Strategy may capture general information about how to align testing with other life cycle activities and The Test Manager should perform project-specific alignment for each test level and for any selected combination of software development life cycle and test process during the planning stage.

Depending on the needs of the organization, project and product, additional test levels beyond those defined may be required, such as:

  • Hardware-software integration testing
  • System integration testing
  • Feature interaction testing
  • Customer product integration testing

Each of these test levels should have the following elements clearly defined:

  • Test objectives, with achievable goals
  • Test scope and test items
  • Test basis, along with a means of measuring coverage of that basis (trace ability)
  • Entry and exit criteria
  • Test deliverable, including results reporting
  • Applicable test techniques with a way of ensuring the appropriate degrees of test coverage
  • Measurements and metrics relevant to the test objectives, entry and exit criteria, and results reporting (including coverage measurements)
  • Test tools to be applied for specific test tasks
  • Resources
  • Responsible individuals and groups, both inside and outside the test team
  • Compliance with organizational, regulatory or other standards (if and where applicable)

Test Management in Context by Lucian C on Scribd

For a broader article on this topic, you can check our Test Manager School or further browse our software testing blog.

This article is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2.

In late 2019 we have launched A Test Manager’s Guide eBook that stands as the base for this article. You can check out more useful test management lessons by enrolling for free to view Chapter 1 – Back to the basics.