2. 1. INTRODUCTION ............................................................................................................................. 4
1.1. TESTING APPROACH .......................................................................................................................... 5
1.2. “V” MODEL ....................................................................................................................................... 6
1.3. STLC THREADS................................................................................................................................. 7
2. THREAD: TEST DEFINITION ......................................................................................................... 8
2.1. ACTIVITY: PERFORM TEST ANALYSIS ........................................................................................................ 9
2.2. ACTIVITY: DEFINE TESTING TECHNIQUES ................................................................................................ 10
2.3. ACTIVITY: VALIDATE REQUIREMENTS ..................................................................................................... 11
2.4. ACTIVITY: DEVELOP TESTING STRATEGY ................................................................................................. 12
3. THREAD: TEST DESIGN ............................................................. ERROR! BOOKMARK NOT DEFINED.
3.1. ACTIVITY: DEFINE TRACEABILITY TRIAGE ..................................................... ERROR! BOOKMARK NOT DEFINED.
3.2. ACTIVITY: INFRASTRUCTURE REQUIREMENTS ................................................ ERROR! BOOKMARK NOT DEFINED.
3.3. ACTIVITY: TEST DATA REQUIREMENTS........................................................ ERROR! BOOKMARK NOT DEFINED.
4. THREAD: TEST DEVELOPMENT ............................................... ERROR! BOOKMARK NOT DEFINED.
4.1. ACTIVITY: DEVELOP TEST SUITES................................................................ ERROR! BOOKMARK NOT DEFINED.
4.2. ACTIVITY: GENERATE TEST DATA ............................................................... ERROR! BOOKMARK NOT DEFINED.
4.3. ACTIVITY: SET UP TEST SYSTEM ................................................................. ERROR! BOOKMARK NOT DEFINED.
5. THREAD: TEST EXECUTION ...................................................... ERROR! BOOKMARK NOT DEFINED.
5.1. ACTIVITY: EXECUTE TEST SCRIPTS............................................................... ERROR! BOOKMARK NOT DEFINED.
5.2. ACTIVITY: ITERATE TEST EXECUTION ........................................................... ERROR! BOOKMARK NOT DEFINED.
5.3. ACTIVITY: TEST REPORTING AND METRICS ................................................... ERROR! BOOKMARK NOT DEFINED.
6. THREAD: TEST EXIT .................................................................. ERROR! BOOKMARK NOT DEFINED.
6.1. ACTIVITY: CREATE EXIT REPORT ................................................................. ERROR! BOOKMARK NOT DEFINED.
6.2. ACTIVITY: DISTRIBUTE EXIT REPORT ........................................................... ERROR! BOOKMARK NOT DEFINED.
7. THREAD: DEFECT MANAGEMENT ............................................ ERROR! BOOKMARK NOT DEFINED.
7.1. ACTIVITY: DEFECT PREVENTION ................................................................. ERROR! BOOKMARK NOT DEFINED.
7.2. ACTIVITY: DEFECT DETECTION ................................................................... ERROR! BOOKMARK NOT DEFINED.
7.3. ACTIVITY: ROOT CAUSE ANALYSIS .............................................................. ERROR! BOOKMARK NOT DEFINED.
7.4. ACTIVITY: DEFECT RESOLUTION ................................................................. ERROR! BOOKMARK NOT DEFINED.
7.5. ACTIVITY: PROCESS IMPROVEMENT ............................................................ ERROR! BOOKMARK NOT DEFINED.
8. TEST MANAGEMENT ................................................................. ERROR! BOOKMARK NOT DEFINED.
8.1. ESTABLISH AND DISTRIBUTE TEST PLAN ....................................................... ERROR! BOOKMARK NOT DEFINED.
8.2. OBTAIN COMMITMENT TO TEST PLAN ........................................................ ERROR! BOOKMARK NOT DEFINED.
8.3. RECONCILE WORK AND STAFF.................................................................... ERROR! BOOKMARK NOT DEFINED.
8.4. MONITOR PROGRESS AGAINST PLAN .......................................................... ERROR! BOOKMARK NOT DEFINED.
8.5. MANAGE DEFECTS TO CLOSURE ................................................................ ERROR! BOOKMARK NOT DEFINED.
9. SERVICE OFFERING ................................................................... ERROR! BOOKMARK NOT DEFINED.
9.1. FUNCTIONAL TESTING.............................................................................. ERROR! BOOKMARK NOT DEFINED.
9.2. PERFORMANCE TESTING .......................................................................... ERROR! BOOKMARK NOT DEFINED.
3. 9.3. AUTOMATION TESTING ............................................................................ ERROR! BOOKMARK NOT DEFINED.
9.4. INTEGRATION TESTING............................................................................. ERROR! BOOKMARK NOT DEFINED.
9.5. SYSTEM TESTING .................................................................................... ERROR! BOOKMARK NOT DEFINED.
9.6. REGRESSION TESTING .............................................................................. ERROR! BOOKMARK NOT DEFINED.
9.7. USABILITY TESTING ................................................................................. ERROR! BOOKMARK NOT DEFINED.
10. TOOL BOX ............................................................................... ERROR! BOOKMARK NOT DEFINED.
4. 1. INTRODUCTION
Software testing is a core quality control component of any software development lifecycle and essential
to effective software quality assurance. Testing is the means of achieving two major objectives –
Verification (You built it right) and Validation (You built the right thing). By themselves, Verification and
Validation do not guarantee software quality; various test management threads, and other aspects of
software engineering are required.
Quality Assessment Hand Book focuses on defining QAT’s testing approach and reflects its years of
experience and resulting best practices in this area. Like any other framework, this Hand Book is designed
to be flexible and customizable so that it can be tailored for use on a variety of testing initiatives.
Specifically, this is designed to support testing on the following types of initiatives:
IT Managed Development Projects
Both production support changes and enhancement requests on IT Managed Applications
Stand-alone Testing Service for any software developed by teams outside IT or other third-party
5. 1.1. TESTING APPROACH
This document identifies the QAT’s testing methodology as implemented across all projects. Core
elements of QAT’s approach to testing are -
Adaptable to all project types and methodologies by carefully and thoughtfully integrating with
project management
Support of Process Improvement Goals as defined by various functional teams in IT
Risk Based Testing to detect the software’s technical risks, and to control critical path failures
Defect Management to include collection of defect measures and causal analysis to identify,
remove and prevent defects as close to injection as possible
6. 1.2. “V” MODEL
The V-Model is a structured testing framework emphasizing quality from the initial requirements stage
through the final testing stage. The framework can be used with any project management or system
development methodology. It focuses on testing throughout the development life-cycle, early
development of test requirements, and early detection of errors. Each major deliverable in the
development process is assessed, verified, validated and tested.
“V” - Model
7. 1.3. STLC THREADS
STLC Threads
Our STLC can be classified into threads of work performed across the phases of SDLC. The goal of this
process is to depict the relationship and integration between testing and software development activities.
The SDLC phases shown on the diagram are for illustrative purposes and are not defined within this
document. A quick look at our STLC threads -
Test Definition– Defines the overall scope, strategy and performance objectives of the testing to
be performed
Test Design – Decide environment and set up, testing techniques to be used, test data, and test
sets & scenarios for automation and manual testing
Test Development –Build test scripts, and create test data; Create and configure test systems
Test Execution –Execute scripts, log defects, report metrics
Test Exit –Deliver Test Exit Report to confirm the software is ready to be deployed
Defect Management – Manage defect prevention, tracking, from detection till closure in
compliance to the approach defined in this document
8. 2. THREAD: TEST DEFINITION
This thread defines overall test scope, Testing Strategy and performance objectives of the tests to be
performed. A Testing Strategy driven by the risk analysis is delivered at the end of the thread that, details
on planning and executing the testing phase in perfection.
The Testing Strategy provides the roadmap for the overall testing effort. Its primary goal is to provide a
management-level view of all testing to be accomplished and the sequencing and dependency between
the various types and iterations of testing.
This thread contains the following activities:
Perform test analysis
Define testing techniques
Validate requirements
Develop Testing Strategy
9. 2.1. ACTIVITY: PERFORM TEST ANALYSIS
The purpose of this activity is to identify and assess the software’s technical risks and overall impact on
the business context so the optimal testing scope can be determined and planned.
Ultimately the question of how much testing is enough is a business decision to be made by the business
owner that owns the application software. The optimal level of testing cannot be decided without the
assistance of the business area owners.
10. 2.2. ACTIVITY: DEFINE TESTING TECHNIQUES
The purpose of this activity is to assess how every item on scope would be tested and what technique
could be applied to test a particular feature of the item. Test scope and specifications (from the Test
Analysis) grouped by levels, modules etc. are required to focus on testing. Document the test structure
with applicable testing techniques, test data, and approach for each area. Ensure the structure allows the
specification coverage to be measured.
From your own experience, or other experience you have access to, identify existing techniques that will
either meet the requirements of the test approach, or can be adapted to meet them .
11. 2.3. ACTIVITY: VALIDATE REQUIREMENTS
Defects introduced early in the effort to engineer a system due to poorly identified requirements are
generally seen as a major factor leading to high system and software costs, especially if the defective
requirements are undetected until later development phases in the lifecycle. The ultimate savings due to
error detection, diagnosis, and correction before a trial system is produced are generally great.
Traceability matrix is created by associating requirements with the work products that satisfy them and is
used to ensure all the requirements are met. The traceability matrix helps in locating affected system
components when there is a requirement change or a defect identified. This thread addresses
requirements validation through Requirements Traceability Matrix ensuring that the
set of requirements is correct, complete, and consistent
a system can be created that satisfies the requirements
and the solution can be tested to prove that it satisfies the requirements
12. 2.4. ACTIVITY: DEVELOP TESTING STRATEGY
Develop a Testing Strategy to provide a roadmap for testing that defines the testing scope, objective,
process and plan. Other areas that will be addressed in the Testing Strategy document include:
Test Infrastructure & Tool Requirements
Test Data Development & Management Process
Defect Management Process - Refer the thread Defect Management for additional information
on this topic.
Software Risk Profile
