Testing Methodologies and Strategies

Testing Methodologies
Dr.R.DHANALAKSHMI, M.Tech , Ph.D
Professor / CSE
KCG College of Technology , Chennai

2
Structural
Development
Agent Oriented
Development
Object Oriented
Development
Analysis
Design
Implementation
Testing
Maintenance
Analysis
Design
Implementation
Testing
Maintenance
OO
OO
OO
OO
Testing
Analysis
Design
Implementation
Maintenance
AO
AO
AO
AO
Software
Engineering
Software
Development
Paradigm
Data, Function
Class , Object
Agent

Software Testing is a process of evaluating a
system by manual or automatic means and verify
that it satisfies specified requirements or identify
differences between expected and actual results
Software Testing

Software Testing
 The objective of software testing is to find
greatest possible number of errors with a
manageable amount of effort applied over a
realistic time span

Verification and Validation
 Verification refers to the set of activities that ensure
that software correctly implements a specific function.
 Validation refers to a different set of activities that
ensure that the software that has been built is traceable
to customer requirements.
Verification : "Are we building the product right ?"
Validation : "Are we building the right product ?"

Integration Testing Strategies
 Incremental Integration
 Top-down Integration
 Depth-First Integration
 Breadth-First Integration
 Bottom-Up Integration
 Non-Incremental Integration

Depth-First IntegrationDepth-First IntegrationDepth-First Integration
Integrating Path : M1 – M2 – M5 – M8

Breadth-First Integration
Integrating Path : M2 – M3 – M4

Validation Testing
 Testing focuses on user-visible actions and user-
recognizable output from the system.
 Validation succeeds when software functions in
a manner that can be reasonably expected by the
customer.
 The alpha test is conducted at the developer's
site by a customer.
 The beta test is conducted at one or more
customer sites by the end-user of the software.

System Testing
 System testing is actually a series of different
tests whose primary purpose is to fully exercise
the computer-based system.
 Software is incorporated with other system
elements (e.g., hardware, people, information),
and a series of system integration and validation
tests are conducted.

Types of System Testing
 Recovery testing is a system test that forces the
software to fail in a variety of ways and verifies
that recovery is properly performed.
 Security testing attempts to verify that
protection mechanisms built into a system will,
in fact, protect it from improper penetration.
 Performance testing is designed to test the
run-time performance of software within the
context of an integrated system.
 Stress testing is designed to confront programs
with abnormal situations.

Software Testing Strategy - Spiral

Testing Strategy
 Any Testing strategy must incorporate
 Test planning
 Test case design
 Test execution
 Resultant data collection and evaluation

Debugging
 Debugging is not testing but always occurs as a
consequence of testing.
 Debugging occurs as a consequence of successful
testing.
 When a test case uncovers an error, debugging is
the process that results in the removal of the
error.

Testing Types
White Box Testing
Black Box Testing
Grey Box Testing

WHITE-BOX TESTING
Static testing
Desk Checking
Code Walkthrough
Code Inspection
Structural Testing
Code Coverage
Statement coverage
Path coverage
Condition coverage
Function coverage
Code complexity
Cyclomatic complexity
Unit / code
Functional
testing

Static Testing
 Static testing is a type of testing which requires
only the source code of the product, not the
binaries and executables

Desk checking
 Manually done by the author of the code for
correctness checking
 Desk checking done before compiling and
executing
 Done by comparing the code with the design or
specification
 Completely relies on author’s thoroughness and
skills
 No structured method or formalism to ensure
completeness
 No maintaining of a log or checklist

Code walkthrough
 It is a group-oriented method
 It is a informal method
 Set of people look at the program and question
the author.
 The author explains the logic of the code and
answers the questions.
 Unanswered questions are noted down and
author finds the answer.

Code Inspection
 It is a formal method
 The focus is to detect all faults, violations and other side
effects
 Code inspection starts when author made sure the code
is ready after desk checking and walkthrough.
 Members of the meeting:
 The author of the code
 Moderator: formally runs the inspection according to the
process
 Inspectors: Provides review comments for the code
 Scribe: takes detailed notes during the meeting and circulates
them to the inspection team

Structural testing
 Structural testing takes into account the code, code
structure, internal design and how they are coded
 Structural testing run by computer on the built product
whereas in static testing is done by humans just using
the source code.
 A given portion of the code is exercised if a test case
causes the program to execute that portion of the code
when running the test

Unit/code functional testing
 Quick check by developer before subjecting the code to more
extensive code coverage testing or code complexity testing.
 Developer performs this tests knowing the input variable and
the corresponding expected output variables
 Developer can build a “debug version” of the product by
putting “intermediate print statements” and making sure
the program is passing thro the right loops and iterations the
right no. of times
 Testing the product under a debugger or an IDE by setting
“break points” at any functions or instructions.
 All the above fall under the “debugging” category rather than
“testing” category

Code coverage testing
 Code coverage testing involves designing and
executing test cases and finding out the
percentage of code that is covered by testing.
 Program constructs
 Sequential control flow
 Two-way decision statements (if then else)
 Multi-way decision statements ( switch)
 Loops ( while do, repeat until and for)

BASIS PATH TESTING
 Basis path testing is a white-box testing technique
 Basis path testing can be achieved by performing
the following steps
 Flow graph notation
 Independent Path Identification
 Cyclomatic Complexity
 Deriving Test cases

Flow Graph Notation
 Flow graph is a simple notation for the representation of
control flow.
 Each circle, called a flow graph node, represents one or
more procedural statements.
 The arrows on the flow graph, called edges or links,
represent flow of control and are analogous to flowchart
arrows.
 Areas bounded by edges and nodes are called regions.
 When counting regions, we include the area outside the
graph as a region

Greatest of Three Numbers
Start
Input a, b
If a>b
Print “A is
greater”
Print “B is
greater”
Stop
YES
NO
A
B
C
E
D
F

Independent Path
 Path 1: A - B – C – D – F
 Path 2: A – B – C – E – F
A
B
C
E
D
F

Cyclomatic Complexity
 Cyclomatic complexity is a software metric that provides a
quantitative measure of the logical complexity of a
program.
 Cyclomatic complexity defines the number of
independent paths in the basis set of a program and
provides us with an upper bound for the number of
tests that must be conducted to ensure that all
statements have been executed at least once.
 An independent path is any path through the program that
introduces at least one new set of processing statements
or a new condition.

Cyclomatic complexity
Complexity is computed in one of three ways:
1. The number of regions of the flow graph correspond to the
cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph, G, is defined as
V(G) = E - N + 2
where E is the number of flow graph edges, N is the number of
flow graph nodes.
3. Cyclomatic complexity, V(G), for a flow graph, G, is also defined as
V(G) = P + 1
where P is the number of predicate nodes contained in the flow graph
G.

A
B
C
E
D
F
Region1
Region2
No.of.Vertex(V) = 6 (A,B,C,D,E,F)
No.of.Edges(E) = 6
{(A,B),(B,C), (C,D), (C,E), (E,F), (D,F)}
No.of. Regions(R) = 2 (Region1,
Region2)
No.of Predicates (P) = 1 (C)

The flow graph has two regions i.e. R=2.
2. V (G) = E – N + 2
= 6 edges - 6 nodes + 2
= 2
3. V (G) = P + 1
= 1 predicate nodes + 1
= 2.

Graph Matrices
 A graph matrix is a square matrix whose size (i.e.,
number of rows and columns) is equal to the
number of nodes on the flow graph. Each row
and column corresponds to an identified node,
and matrix entries correspond to connections
(an edge) between nodes.

Condition coverage
 All the conditions may not get evaluated, even
though the right path is chosen.
 Example:
If (mm<1 || MM>12)
false
true
Valid month
invalid month
No. of Paths = 2
A – B
A – C
The path A – B has two conditions to be tested and
therefore need two test cases

BLACK-BOX TESTING
 Black-box testing, also called behavioral testing,
focuses on the functional requirements of the
software.
 Black-box testing enables the software engineer
to derive sets of input conditions that will fully
exercise all functional requirements for a
program.

BLACK-BOX TESTING
 Black-box testing attempts to find errors in the
following categories:
(1) incorrect or missing functions,
(2) interface errors,
(3) errors in data structures or external database
access,
(4) behavior or performance errors,
(5) initialization and termination errors.

Equivalence Partitioning
 Equivalence partitioning is a black box testing technique
to minimize number of permutation and combination of
input data.
 In equivalence partitioning, data is selected in such a way
that it gives as many different out put as possible with the
minimal set of data.
 If software behaves in an identical way for a set of value,
then the set is termed as equivalence class or a partition.
 It can be assumed safely that functionality of the software
will be same for any data value from the equivalence class
or partition.

Equivalence Partitioning
 Steps in Equivalence Partitioning
 Identifying equivalence classes or partition
 Picking one value from each partition for the complete
coverage.
 Equivalence classes may be defined according to the
following guidelines:
1. If an input condition specifies a range, one valid and two invalid
equivalence classes are defined.
2. If an input condition requires a specific value, one valid and two
invalid equivalence classes are defined.
3. If an input condition specifies a member of a set, one valid and
one invalid equivalence class are defined.
4. If an input condition is Boolean, one valid and one invalid class
are defined.

Example
 For example, consider a very simple function for
awarding grades to the students. This program
follows the following guideline to award grades
 Marks 00 – 39 ------------ Grade D
 Marks 40 – 59 ------------ Grade C
 Marks 60 – 70 ------------ Grade B
 Marks 71 – 100 ------------ Grade A

Example
 Based on the equivalence partitioning techniques,
partitions for this program could be as follows
 Marks between 0 to 39 - Valid Input
 Marks less than 0 - Invalid Input
 Marks more than 100 - Invalid Input
 Non numeric input - Invalid Input
Now even if you take only one data value from these partitions,
your coverage will be good.

Boundary Value Analysis
 Boundary value analysis leads to a selection of test cases
that exercise bounding values.
 Boundary value analysis is a test case design technique
that complements equivalence partitioning.
 Rather than selecting any element of an equivalence class,
BVA leads to the selection of test cases at the "edges" of
the class.
 Boundary value analysis is the technique of making sure
that behavior of system is predictable for the input and
output boundary conditions.
 Defects could be introduced at the boundaries very easily.

Example
 " Input should be greater than equal to 10 and
less than 50"
 Probably you will write something like
if (input >=10 AND input <50) then
do some
else
do some thing else.

Example
According to this input values from 10 to 49 are valid, but if
you make mistake in specifying the conditions, following
things can happen
 input >10 AND input <50 -------> Input value 10 in invalid now.
 input <=10 AND input <50 -------> Input value 9 is valid now.
 input >=10 AND input <=50 -----> Input value 50 is valid now
 input >=10 AND input >50 -----> Input value 49 is invalid now.
we should have following test cases for boundaries
9, 10, 11 and 48, 49, 50
lower_boundary - 1, lower_boundary, lower_boundary + 1 and
upper_boundary - 1, upper_boundary, upper_boundary + 1

Guidelines for BVA
1. If an input condition specifies a range bounded by
values a and b, test cases should be designed with values
a and b and just above and just below a and b.
2. If an input condition specifies a number of values, test
cases should be developed that exercise the minimum
and maximum numbers. Values just above and below
minimum and maximum are also tested.
3. If internal program data structures have prescribed
boundaries (e.g., an array has a defined limit of 100
entries), be certain to design a test case to exercise the
data structure at its boundary.

Testing OO systems
 Unit testing a class
 Integration testing of classes
 System testing
 Regression testing

Unit testing a class
 A class is intended for heavy reuse. A defect in a class affect
every instance of reuse
 Defects get introduced at the time a class gets defined. A delay
in catching these defects makes them go into the clients of
these classes. Thus fixing of defect would have to be reflected
in multiple places, giving rise to inconsistencies
 As class is a combination of data and method, they should
work in sync at a unit level. If not, it may cause defects that are
very difficult to narrow down
 Inheritance puts more context into the class. Thus unless the
class is thoroughly tested, defects arising out of these contexts
may magnified many times later in the cycle

Testing objects at unit level
 Test the object through its life cycle from birth
to death ( instantiation to destruction)
 Test the simple method first and then the more
complex methods
 Test the method from private to public method
 Send a message to every method at least once to
ensure that every method is tested at least once.

Integration Testing
 Object-oriented software does not have
hierarchical control structure, so top-down and
bottom-up integration has little meaning
 Integrating operations one at a time into a class
is often impossible because of direct and indirect
interactions of the components that make up a
the class

Integration testing strategies
 Thread-based testing
 Integrates set of classes required to one input or event for the
system.
 Each thread is integrated and tested individually
 Use-based testing
 Begins the construction of the system by testing independent
classes
 After independent classes are tested, dependent classes that
use the independent classes are tested
 Cluster testing
 Cluster of collaborating classes are exercised by designing test
cases that attempt to uncover errors in the collaboration

72
Agent 1 Agent 2 Agent x
Agent Based System
Goal 1
Goal 2
Goal y
Plan 1
Plan 2
Plan z
Task1
Task 2
Task r
Consists of
Responsibility 1
Activity 1
Activity 2
Activity p
Role1 Role 2 Role m
Activities
Responsibility 2
Responsibility n
includes
Roles
Composed of
Goals
Plans
Responsibilities
Agent Testing

"What is software quality?"
 Software quality assurance (SQA) is an umbrella activity that
is applied throughout the software process.
 software quality is defined as
 Conformance to explicitly stated functional and performance
requirements, explicitly documented development standards,
and implicit characteristics that are expected of all
professionally developed software.
 Quality of design refers to the characteristics that designers
specify for an item.
 Quality of conformance is the degree to which the design
specifications are followed during manufacturing.

McCall’s Quality Factors
 The factors that affect software quality can be
categorized in two broad groups:
(1) factors that can be directly measured (e.g., defects
per function-point)
(2) factors that can be measured only indirectly (e.g.,
usability or maintainability).

FURPS
 Hewlett-Packard [GRA87] developed a set of software
quality factors that has been given the acronym FURPS—
Functionality, Usability, Reliability, Performance, and
Supportability.
 The FURPS quality factors draw liberally from earlier work,
defining the following attributes for each of the five major
factors:
 Functionality is assessed by evaluating the feature set and
capabilities of the program, the generality of the functions
that are delivered, and the security of the overall system.
 Usability is assessed by considering human factors, overall
aesthetics, consistency, and documentation.
 Reliability is evaluated by measuring the frequency and
severity of failure, the accuracy of output results, the mean-
time-to-failure (MTTF), the ability to recover from failure,
and the predictability of the program.

FURPS
 Performance is measured by processing speed,
response time, resource consumption, throughput, and
efficiency.
 Supportability combines the ability to extend the
program (extensibility), adaptability, serviceability—
these three attributes represent a more common term,
maintainability—in addition, testability, compatibility,
configurability (the ability to organize and control
elements of the software configuration, Chapter 9), the
ease with which a system can be installed, and the ease
with which problems can be localized.

ISO 9126 Quality Factors
 The ISO 9126 standard was developed in an attempt to
identify the key quality attributes for computer
software. The standard identifies six key quality
attributes:
 Functionality. The degree to which the software satisfies
stated needs as indicated by the following subattributes:
suitability, accuracy, interoperability, compliance, and
security.
 Reliability. The amount of time that the software is
available for use as indicated by the following
subattributes: maturity, fault tolerance, recoverability.
 Usability. The degree to which the software is easy to
use as indicated by the following subattributes:
understandability, learnability, operability.

ISO 9126 Quality Factors
 Efficiency. The degree to which the software makes
optimal use of system resources as indicated by the
following subattributes: time behavior, resource
behavior.
 Maintainability. The ease with which repair may be made
to the software as indicated by the following
subattributes: analyzability, changeability, stability,
testability.
 Portability. The ease with which the software can be
transposed from one environment to another as
indicated by the following subattributes: adaptability,
installability, conformance, replaceability.

Testing Methodologies and Strategies

Testing Methodologies and Strategies

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Testing Methodologies and Strategies

Ähnlich wie Testing Methodologies and Strategies (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Testing Methodologies and Strategies