User Acceptance Testing tips and tricks

A Test manager should use tips and tricks for UAT ( user acceptance testing ), especially for Major Programs in software testing. This is why we need to highlight that there are 10 areas of concern related to software testing on which a Test Manager should focus. These areas cover both preparation and execution of the fundamentals of UAT:

1. Stakeholder Engagement
2. Communications
3. Governance
4. Objectives, Scope and Expectations
5. Planning and Logistics
6. Preparation – System, Data and Scenarios
7. Preparation – People
8. Scheduling and Management
9. Execution and Defect Management
10. Reporting

Before we start and dive into the user acceptance testing process, it would be best to properly have a UAT definition: A type of acceptance testing performed to determine if intended users accept the system. This is a part of acceptance testing and which is a software test level that focuses on determining whether to accept the system.

1. Stakeholder Engagement

Bring stakeholders along the UAT journey: Align expectations of key stakeholders and obtain agreement on UAT objectives, testing success criteria, test scope, testing timelines, as well as test resource requirements. The idea is to have no surprise during user acceptance testing.

This is why a test manager should:

• Manage stakeholders by engaging and bringing the right people along for UAT at the right time
• Engage with stakeholders to make sure they agree to the goals and objectives of UAT
• Make sure everyone has a view on what is the exit criteria
• Make sure the business & IT stakeholders know what is expected of them
• Be transparent about UAT readiness

2. Communications

Implement effective communication channels: Implement open, honest and fast communication mechanisms and structures for UAT including stakeholder engagement, daily/weekly progress reporting and defect tracking and resolution.

• Develop a Communications Plan
• Socialize the communications plan
• Provide daily forums for open review of testing progress and defect prioritization
• Communication channels should be in place to provide access to critical resources
• Engage business users early in the program

3. UAT Governance

Implement operational governance processes that are understood by all software testing stakeholders: Clear definition of test roles and testing responsibilities in advance of execution, as well as agreed detailed entry and exit criteria

• Implement UAT governance processes and that are understood by stakeholders
• Reassess and confirm Entry and Exit criteria defined in Test Strategy and Acceptance Test Plan documents
• Adhere to Exit and Entry Criteria or the agreed acceptance thresholds that are within acceptable risk profiles
• Clearly define roles and responsibilities and publish a UAT Testing Organization Chart to the UAT teams involved in the process
• Create and publish the RACI matrix
• Define and map how defects are logged, triaged, escalated, rejected and approved.

4. User Acceptance Testing Objectives, Scope and Expectations

Be firm on objectives, clear on Scope and transparent on Expectations: Set clear objectives and scope with your stakeholders. Use the Requirements Trace-ability Matrix to validate the scope and make sure the criteria and thresholds for acceptance are understood.

• The Programs and Stakeholders need to agree with the objectives of UAT. Without a clear definition expectations may be misaligned and defects recorded against unexpected areas.
• UAT should be testing processes, data and authorizations.
• Agree the scope of UAT for each entity/area and where possible share the testing to reduce duplication.
• Agree prioritization of test scripts and expectations of test volumes up front
• SLAs for defect resolution must be understood

5. Planning and Logistics

Plan UAT execution strategy and approach alongside scenario definition: Major Programs require additional considerations and effort when preparing the approach and organizing the location of testing.

• Assign a UAT Test Manager to oversee integrated user acceptance testing preparation and execution activities
• Plan the execution strategy in parallel with preparation of test scenarios and test scripts
• Agree the Testing Approach and Testing locations and assign Testing Leads and support personnel to manage the daily operations of UAT Testing in each location.
• Identify the ‘critical resources’ and prepare a critical resource plan to support Testing and Defect resolution and agree process for contact and communication between key resources.
• Plan dry run testing of management meetings and processes
• Testing Hub preparation requires significant planning and preparation.

6. Preparation – Systems, Data and Scenarios

Preparing for success:  Planning and reserving time for systems, interfaces, data, authorization and scenario readiness early in order to prove UAT readiness will save time and effort during test execution.

• Publish clear timelines and dependencies for delivery of UAT Test environment.
• Confirm that time has been allocated for UAT readiness
• Preparation of test management tool for UAT testing must include access logons as well as timely loading of scripts into the system.
• Clearly articulate each scenario and its objective prior to developing the full test script
• Set up Scenario/Script control processes for review and review for insufficient or missing data.

7. Preparation – People

Preparing for success: Developing processes and ensuring your people are trained as well as user acceptance testing approach and results recording will produce more thorough testing and more accurate defect recording.

• Training of testers is critical to efficient testing and operations
• Choose testers with relevant understanding of processes and commitment to support the project through to go-live and hyper care
• Train Subject Matter Experts and SPOCs in the Defect logging and escalation process
• Establishing rules of engagement for testing helps the team maintain focus on testing, identification of defects and resolution rather than just the defects.
• Check that the SI has the relevant commercial arrangements in place to support testing

8. UAT Test Scheduling and Management

Understand priorities for scheduling: Prepare action plans and guidance on how to prioritize and reschedule testing sequences and unblock test scripts in order to keep testing on track.

• Stakeholders and acceptance test Teams must agree to prioritization for scheduling and sequencing of testing.
• Put in place a triage process to prioritize the assessment of defects blocking Testing progress and to quickly assess the extent of the impact on other testing scenarios
• Put in place processes and tools to facilitate rescheduling if defects or other factors stop progress of testing.
• Implement a mechanism to track test scenarios/scripts back to requirements

9. Testing Execution and Defect Management

Prioritize and manage defects: Putting in place processes to identify quickly blocking defects or high priority/severity defects helps focus the Design team on root cause analysis assessment and permits the test teams to progress or refocus on new scripts.

• Assign a Defect Manager responsible for the process of defect management and reporting across all defect types. This person would be responsible for execution progress and scheduling and retest.
• Develop processes to identify and assess root cause of impact of Critical defects, including Data, Authorizations and show stopper defects.
• Develop and trial run UAT testing processes to validate execution and results recording as well as defect triage and assessment.
• UAT Defect Manager actively monitor defect assignment and assessment to prepare a view on resolution progress and timing of retest
• Develop and agree criteria for defect severity

10. User Acceptance Testing Reporting

Plan and Prepare for Test Reporting: Accurate defect and testing status reporting will build stakeholder confidence in control and content of testing and will visibility of actual testing status.

• Schedule and lock in daily reporting meetings.
• Prepare the format of regular reporting before start of execution
• Build reporting process and tools to support Defect Reporting and testing progress reporting prior to starting UAT.
• Confirm that reports can be generated in a timely and consistent manner.
• Defect reporting should be generated from the test management system
• Agree reporting framework, hierarchy and feeding mechanism between teams and dry run reporting generation.