2. Summary
• White‐box (or Glass‐box) tes,ng: general
characteris,cs
• Statement coverage
• Decision coverage
• Condi,on coverage
• Decision and condi,on coverage
• Coverage of a linearly independent set of
paths
SoNware Engineering a.a. 2008/2009
2
White‐box tes,ng
3. White‐box tes,ng: general characteris,cs
• Also called
– Glass‐box tes,ng
– Structural tes,ng (compared to func,onal)
• Exploits the control structure, that means the
control or flow graph of programs in order to
design test cases
• Typically performed during coding
SoNware Engineering a.a. 2008/2009
3
White‐box tes,ng
7. Statement coverage cont.d
Java example: A
/*A*/ float x = InOut.readFloat(), B
x != 0
y = InOut.readFloat();
/*B*/ if (x != 0) x == 0
C
/*C*/ y += 10;
/*D*/ y = y/x;
/*E*/ System.out.println(x + ' ' + y); D
• The test case E
{x != 0; y = any}
• Covers the statements Control flow graph for the example
SoNware Engineering a.a. 2008/2009
7
White‐box tes,ng
8. Statement coverage cont.d
• Possible problem
– Even in a case of a program without itera,ons, the
execu,on of every statement does not guarantee
that all possible paths are exercised
• Possible nega,ve consequence:
– In previous java example program the error of
division by 0 is not iden,fied (statement D)
SoNware Engineering a.a. 2008/2009
8
White‐box tes,ng
9. Decision coverage
• Properly includes statement coverage
• Vice versa does not hold
• Focus on all possible decisions in a program
• Decisions are in statements
– if, switch, while, for, do
• Goal
– To iden,fy a set of test cases sufficient for
guaranteeing that each decision will have value
“true” at least once and value “false” at least once
SoNware Engineering a.a. 2008/2009
9
White‐box tes,ng
11. Decision coverage cont.d
• Possible problem
• If decisions are composed of several condi,ons (AND,
OR), decision coverage can be not sufficient
Esempio Java:
/*A*/ float z = InOut.readFloat(),
w = InOut.readFloat();
t = InOut.readFloat();
/*B*/ if (z == 0 || w > 0)
What is the
/*C*/ w = w/z; problem here?
/*D*/ else w = w + 2/t;
/*E*/ System.out.println(z+' '+w+' '+t);
SoNware Engineering a.a. 2008/2009
11
White‐box tes,ng
12. Decision coverage cont.d
• Test cases:
– {t = 0; z= 5 ; w= 5} decision TRUE
– {t = 0; z= 5 ; w= ‐5} decision FALSE
• Cover decision
• The second one iden,fies the division by zero in
D but
• The risk of division by zero in C is not iden,fies
• We need a criterion that considers both structure
and components of decisions
SoNware Engineering a.a. 2008/2009
12
White‐box tes,ng
13. Condi,on coverage
• Does not properly include decision coverage
• It is not properly included by decision coverage
• Focus on all possible condi,ons in a program
• Condi,ons are the operands of non‐atomic
boolean expressions
• Goal:
– To iden,fy a set of test cases sufficient for
guaranteeing that every condi,on (atomic boolean
expression) included in the program’s decisions have
value “true” at least once and value “false” at least
once
SoNware Engineering a.a. 2008/2009
13
White‐box tes,ng
15. Condi,on coverage
• Test cases
– {t=0;z=0;w=‐5} 1a condi,on =T, 2a condi,on =F
– {t=0;z=5;w= 5} 1a condi,on=F, 2a condi,on=T
• Cover condi,ons
• The first one iden,fies the division by zero in
C
• The risk of division by zero in D is not
iden,fied as D is never excercised
• Decision is always “true”
SoNware Engineering a.a. 2008/2009
15
White‐box tes,ng
16. Decision and condi,on coverage
• Properly includes
– Decision coverage
– Condi,on coverage
• Goal
– To iden,fy a set of test cases sufficient for
guaranteeing that
• Each decision is “true” at least once and “false” at least
once
• All condi,ons composing decisions is “true” at least
once and “false” at least once
SoNware Engineering a.a. 2008/2009
16
White‐box tes,ng
17. Decision and condi,on coverage
cont.d
/*A*/ float z = InOut.readFloat(),
w = InOut.readFloat();
t = InOut.readFloat();
/*B*/ if (z == 0 || w > 0)
/*C*/ w = w/z;
/*D*/ else w = w + 2/t;
/*E*/ System.out.println(z+' '+w+' '+t);
Test cases
{t=0;z=0;w=5} 1a c.=T, 2a c.=T, dec.=T
{t=0;z=5;w=‐5} 1a c.=F, 2a c.=F, dec.=F
Cover decisions and condi,ons and iden,fy both division by zero
in statements C and D
SoNware Engineering a.a. 2008/2009
17
White‐box tes,ng
18. Summary of test cases so far
Test t z w cond.1 cond.2 decis. Finds Finds
case error error
in C? in D?
C1 0 0 5 T T T yes no
C2 0 5 5 F T T no no
C3 0 0 ‐5 T F T yes no
C4 0 5 ‐5 F F F no yes
SoNware Engineering a.a. 2008/2009
18
White‐box tes,ng
19. Coverage Associated Finds error Finds error
criterion test cases in C? in D?
Decision C2 + C4 no yes
Condi,on C2 + C3 yes no
Decision and C1 + C4, or yes yes
condi,on C2 + C3 + C4
SoNware Engineering a.a. 2008/2009
19
White‐box tes,ng
20. Methodological issue
• There are different combina,on of test cases
that guarantee decision and condi,on
coverage but it is not always trivial to find one
• We need a simple and effec,ve method for
iden,fying test cases that cover both decisions
and condi,ons
SoNware Engineering a.a. 2008/2009
20
White‐box tes,ng
21. Control flow graph’s paths coverage
• Based on
– Criteria of decision and condi,on coverage
– Program’s control flow graph
• Goal
– To provide a simple method for guaranteeing
decision and condi,on coverage
SoNware Engineering a.a. 2008/2009
21
White‐box tes,ng
22. Control flow graph
Predicate nodes arc
while (A) {
A if (B) { C }
node else { D }
B E
}
D C F;
E
region Example:
F
V(G) = 7 ‐ 6 + 2 = 3
V(G) = 3
Cycloma,c number
• V(G) = E ‐ N + 2 (E = number of arcs, N = number of nodes)
• V(G) = R (R = number of regions)
SoNware Engineering a.a. 2008/2009
22
White‐box tes,ng
23. Control flow graph
un,l
Sequence of
statements
Simple if case
while
SoNware Engineering a.a. 2008/2009
23
White‐box tes,ng
24. Control flow graph without condi,ons
/*A*/ float z = InOut.readFloat(), A
w = InOut.readFloat();
B
t = InOut.readFloat(); False True
/*B*/ if (z == 0 || w > 0)
/*C*/ w = w/z; D C
/*D*/ else w = w + 2/t;
/*E*/ System.out.println(z+' '+w+' '+t);
E
The control flow graph should be more
detailed
SoNware Engineering a.a. 2008/2009
24
White‐box tes,ng
25. Control flow graph with complex decisions
if (a || b) if (a && b)
x; x;
else y; else y;
... ...
a a
a==F a==T a==T a==F
b b
b==F b==T b==F
b==T
y x x y
... ...
SoNware Engineering a.a. 2008/2009
25
White‐box tes,ng
27. Control flow graph, condi,ons and
decisions
• Boolean values (“true” and “false”) associated
with condi,ons and decisions have
correponding arcs
• To design a set of test cases such that all arcs
of the control flow graph are traversed implies
condi,on and decision coverage
SoNware Engineering a.a. 2008/2009
27
White‐box tes,ng
29. For the previous example
/*A*/ float z = InOut.readFloat(), A
w = InOut.readFloat();
t = InOut.readFloat(); z
/*B*/ if (z == 0 || w > 0) != 0 == 0
/*C*/ w = w/z;
/*D*/ else w = w + 2/t; <= 0 w > 0
/*E*/ System.out.println(z+' '+w+' '+t);
D C
E
Three test cases
C2. {t=0; z=5; w= 5} V=7‐6+2=3
C3. {t=0; z=0; w= ‐5} The three paths
C4. {t=0; z=5; w=‐5} 1. A‐z‐w‐C‐E (C2)
Cover all arcs, hence decisions and condi,ons 2. A‐z‐C‐E (C3)
3. A‐z‐w‐D‐E (C4)
SoNware Engineering a.a. 2008/2009
29
White‐box tes,ng
30. How to choose paths
• Pragma,c rule
– Experiments show that the the number of errors
increase with increasing of the cycloma,c
complexity
– Choose a number of paths equal to the
cycloma,c complexity
– You have a propor,on between number of test
cases and complexity of the code
SoNware Engineering a.a. 2008/2009
30
White‐box tes,ng
31. Linearly independent paths
• A good criterion for choosing paths is based on the
concept of linear independency
• A maximal set of linearly independent paths is called a
base
– It is NOT unique
• A base contains a number of paths equal to the
cycloma,c complexity
• Intui,vely, every path input‐output can be obtained as
a linear combina,on of paths of a base
• By choosing a base, a certain extend of reliability id
guaranteed with respect of combina,ons of errors that
hide each other
SoNware Engineering a.a. 2008/2009
31
White‐box tes,ng
32. Example of paths
A
1 while (A) { Paths
B if (B) { C }
2 3 else { D }
α. A‐F
7
D C 6 E β. A‐B‐D‐E‐A‐F
}
4 5
F; γ. A‐B‐C‐E‐A‐F
E
F
The matrix rank
Arcs/ 1 2 3 4 5 6 7 is 3 (the
paths
maximum)
α 0 0 0 0 0 0 1
{α β γ} is a base
β 1 1 0 1 0 1 1
γ 1 0 1 0 1 1 1
SoNware Engineering a.a. 2008/2009
32
White‐box tes,ng
33. Remarks
• Every path corresponds to a binary row vector (0/1)
• Vice versa does not hold
• A matrix represen,ng paths has maximum 2E dis,nct
rows (E = n° archi) A
• McCabe proved that for each graph G: B
1
1. The matrix rank cannot be greater than V(G) (cycloma,c 2 3
complexity) 7 D C 6
2. There exists always a matrix made of paths with rank
equal to V(G) 4 5
E
• A base is any set of paths (rows) with maximum rank
• Note: MacCabe’s result is purely topological
F
/*B*/ if(5>7)
/*C*/ i=9
• Arc B‐C is never covered
• In other cases is necessary to re‐iterate the cycles in
order to traverse all arcs
SoNware Engineering a.a. 2008/2009
33
White‐box tes,ng
35. Addi,onal considera,ons
• Paths coverage method does not consider
explicitly loops itera,ons
• In some cases it is necessary to perform
itera,ons in order to traverse all arcs
• It might be interes,ng to consider some
itera,on in any case but the number of test
cases increases exponen,ally
• It is important to decide the type of itera,ons
SoNware Engineering a.a. 2008/2009
35
White‐box tes,ng
36. Pragma,c choices for loops
• To limit the number of itera,ons to n
– It is called “loop coverage”
• To execute only certain loops
• To limit the number of paths to be traversed based on
weighted arcs and func,on to be maximized
– Probability of execu,on
– Resources occupancy
• To limit the number of paths by iden,fying the paths
that define and use program variables (Data Flow
tes,ng)
– Defini,on of a variable value
– Use of such value in a test
SoNware Engineering a.a. 2008/2009
36
White‐box tes,ng
37. Data Flow Tes,ng B
C
...
A int x,y,a,b; D
B scanf(“%d %d”,&x, &y);
C a=x; E
D b=y a!=b a==b
E while (a!=b)
F
F if(a>b) a>b a<=b
G a=a‐b;
G H
H else b=b‐a;
I prin~(“%d”,a);
....
I
For x and y it is not needed to execute the loop (loop coverage = 0)
For the defini,on of a and b the loop has to be iterated once (loop coverage = 1)
For the defini,on and usage of a and b the loop has to be iterated twice (loop coverage = 2)
SoNware Engineering a.a. 2008/2009
37
White‐box tes,ng
