SlideShare ist ein Scribd-Unternehmen logo
1 von 39
Downloaden Sie, um offline zu lesen
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction

Constraint-based Fitness Function
for Search-based Software Testing

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

Abdelilah Sakti

Yann-Ga¨l Gu´h´neuc
e
e e

Gilles Pesant

Department of Computer and Software Engineering
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e

May 21, 2013
CPAIOR, Yorktown Heights

Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

2 / 14

What is the cost of a programming error ?
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

2 / 14

What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

2 / 14

What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

What is the cost of a programming error ?
Every year inadequate infrastructure software costs the U.S.
economy around 60 billion $.
Between 1985 and 1987 : the radiotherapy
machine Therac-25 sent to patients a X-ray dose
100 times greater than expected.
At least five deaths ;
Several other patients were severely affected
by radiation.

Only a high quality software can reduce errors cost.
Testing is an important technique for validating and
checking the correctness of software.
Despite that the testing may cost more than 50% of the
budget of critical software, it remains an inevitable
phase in the life cycle of software.

2 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

3 / 14

Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.

Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.

3 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

Software testing
Formal Methods
Is a way of modelling and verifying a software by using
mathematics or logic techniques ;
Aims to prove and argue that a software respect some
properties (e.g., an undesired behaviour will never
occur).
Some complex programmes are difficult or impossible to
model.

Informal or Pragmatic Techniques
Performed by executing a real implementation on a
small subset of all possible inputs data that satisfies
some criteria (e.g., all statements, all branches...) ;
Aims to show the presence of errors.
One main challenge is automating inputs data generation.

3 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

4 / 14

Search Based Software Testing (SBST)
Constraint-based
Fitness Function
for Search-based
Software Testing

Search Based Software Testing (SBST)

Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Fitness Evaluation

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

The Fitness Evaluation is the core of SBST.

4 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing

Search Based Software Testing (SBST)

Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
What is the cost of a
programming error ?
Software testing
Search Based Software
Testing (SBST)

Fitness Evaluation

Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

The Fitness Evaluation is the core of SBST.
An evolution in the fitness evaluation may bring a significant
enhancement to SBST
4 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

5 / 14

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

tinyy == z

tinyy > 0

tinyx == 10

Evaluation
Conclusions

5 / 14

tinyTarget
tinyEND
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

tinyy > 0

tinyx == 10

Evaluation
Conclusions

5 / 14

tinyTarget
tinyEND
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation
Conclusions

5 / 14

tinyTarget
tinyEND
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
3

fSE
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
3+ 90
91

fSE
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
3+ 90
91

fSE
90
91
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
3+ 90
91

fSE
90
91

+ 31
32
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0
Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
3+ 90
91

90
91

+ 31
32

fSE
+0
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0

i2

Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
90
3+ 91 = 3.9890
2 + 21 =2.9545
22

fSE
31
+ 32 +0= 1.9577
+ 21 + 20 =1.9068
22
21

90
91

0
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

Motivating Example
tinyx, y, z

1
2

Introduction

3

Motivating
Example

5

Fitness Function
Based on Branch
Hardness

4
6

int sample(int x,int y,int z){
if(y==z)
if(y>0)
if(x==10)
...//Target
}

i1 : (10,−30,60)
i2 : (30, −20, −20)

tinyy == z

i1
tinyy > 0

i2

Critical Branch of i1

tinyx == 10

Evaluation

tinyTarget

Conclusions

tinyEND

Test candidate
i1
i2

5 / 14

fAL
90
3+ 91 = 3.9890
2 + 21 =2.9545
22

fSE
31
+ 32 +0= 1.9577
+ 21 + 20 =1.9068
22
21

90
91

0

Both fitness functions favour i2 over i1 because either they
do not consider non-executed branches or they ignore some
branches’ future.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

6 / 14

Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

6 / 14

Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

6 / 14

Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

6 / 14

Fitness Function Based on Branch Hardness
Combines SBST with static analysis of the
non-executed branches.
Prioritizes branches according to how hard it is to
satisfy them.
Defines the difficulty to satisfy a constraint in terms of
its arity and its projection tightness (the ratio of the
approximate number of solutions to the size of its
search space).
1. The lower the arity of the constraint, the less freedom
we have to choose some of its variables in order to
evolve the test candidate.
2. A projection tightness close to 0 will indicate high
constrainedness and hardness to satisfy a constraint.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

7 / 14

Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B 2 ·

1
arityc

+ B · (1 − tightness) + 1 .
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness

Branch-Hardness Metrics
Difficulty Coefficient (DC)
DC is a possible representation of the hardness of a
branch ;
Each constraint has its own DC that is determined
according to its arity and tightness ;
DC(c) = B 2 ·

1
arityc

+ B · (1 − tightness) + 1 .

Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

Difficulty Level (DL)
DL is based on DC ranking (r) ;
DL is a representation of a relative hardness level of a
constraint in a set of constraints ;
DL(c, C) =

7 / 14

|C|,
2r−1 · (|C| + 1),

if r = 0
.
if r > 0
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

8 / 14

Branch-Hardness Fitness Functions
DC Fitness Function (fDC )
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC (i, C) =

c∈C

DC(c) · η(i, c) .
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness

Branch-Hardness Fitness Functions
DC Fitness Function (fDC )
DC is used as a penalty coefficient for breaking a
constraint ;
The target of this fitness function is determining a
standard-branch-distance.
fDC (i, C) =

c∈C

DC(c) · η(i, c) .

Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

DL Fitness Function (fDL )
DL is used as a constant penalty for breaking a
constraint in a set of constraints to satisfy ;
fDL (i, C) =
where (i, c) =

8 / 14

c∈C

(i, c) + η(i, c),

0,
if η(i, c) = 0
.
DL(c, C), if η(i, c) = 0
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

9 / 14

Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].

DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness

Applying fDC on our motivating example
Assume that all domains are equal to [−99, 100].

DC for each branch
1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ;
2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ;
3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.

Overview
Branch-Hardness Metrics
Branch-Hardness Fitness
Functions
Applying fDC on our
motivating example

Evaluation
Conclusions

(fDC ) for each test candidate
90
fDC (i1 , C) = 60.95· 91 +106· 31 +110.95· 0 = 162.9677 ;
32
1

fDC (i2 , C) = 60.95· 0 +106· 21 +110.95· 20 = 206.8485.
1
22
21
Contrary to fAL and fSE this fitness function makes the
adequate choice by choosing the test candidate i1 instead of
i2 .

9 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results

Conclusions

10 / 14

Evaluation
Subjects and configuration

analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL .
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results

Conclusions

Evaluation
Subjects and configuration

analysed meta-heuristics and fitness functions
Two widely used meta-heuristic algorithms are
analysed : Simulated Annealing (SA) and Evolutionary
Algorithm (EA).
fAL and fSE from the literature ;
fSEL a natural combination of fAL and fSE ;
Our fitness functions fDC and fDL .

Subjects
440 synthetic test targets that were randomly generated.
20 executions for every combination of fitness function
and meta-heuristic algorithm.
If test data was not found after 25000 (respectively
100000) fitness evaluations for EA (respectively SA),
the search was terminated.

10 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing

Evaluation
EA Results

Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

EA: Comparing all fitnesses on 440 test targets
Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level

Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness

80

60

Evaluation
Subjects and configuration
EA Results

40

SA Results

Conclusions
20

0
0

5000

10000

15000

Evaluations

11 / 14

20000

25000
Constraint-based
Fitness Function
for Search-based
Software Testing

Evaluation
SA Results

Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant

SA: Comparing all fitnesses on 440 test targets
Approach Level
Symbolic Enhanced
Symbolic Enhanced with levels
Difficulty Coefficient
Difficulty Level

Introduction
80

Fitness Function
Based on Branch
Hardness
Evaluation
Subjects and configuration
EA Results
SA Results

Conclusions

Branch Coverage %

Motivating
Example

60

40

20

0
0e+00

2e+04

4e+04

6e+04

Evaluations

12 / 14

8e+04

1e+05
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

13 / 14

Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

Conclusions
Two new metrics to measure the difficulty to satisfy a
constraint in the context of test case generation for
software testing are defined ;
Two new fitness functions for SBST are defined ;
Our new fitness functions are significantly more effective
and efficient than with the largely used fitness functions
from the literature.
The obtained results are promising but more
experiments must be performed.

Future work
Performing more experiments on real world programs ;
Extending our approach by defining new ways to
compute projection tightness for constraints involving
other data types than integer.
13 / 14
Constraint-based
Fitness Function
for Search-based
Software Testing
Abdelilah Sakti,
Yann-Ga¨l
e
Gu´h´neuc, Gilles
e e
Pesant
Introduction
Motivating
Example
Fitness Function
Based on Branch
Hardness
Evaluation
Conclusions

14 / 14

Thank you
Questions ?

Weitere ähnliche Inhalte

Was ist angesagt?

Verification and validation of knowledge bases using test cases generated by ...
Verification and validation of knowledge bases using test cases generated by ...Verification and validation of knowledge bases using test cases generated by ...
Verification and validation of knowledge bases using test cases generated by ...Waqas Tariq
 
AI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsAI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsChakkrit (Kla) Tantithamthavorn
 
Towards formulating dynamic model for predicting defects in system testing us...
Towards formulating dynamic model for predicting defects in system testing us...Towards formulating dynamic model for predicting defects in system testing us...
Towards formulating dynamic model for predicting defects in system testing us...Journal Papers
 
Testing and test case generation by using fuzzy logic and n
Testing and test case generation by using fuzzy logic and nTesting and test case generation by using fuzzy logic and n
Testing and test case generation by using fuzzy logic and nIAEME Publication
 
Automated exam question set generator using utility based agent and learning ...
Automated exam question set generator using utility based agent and learning ...Automated exam question set generator using utility based agent and learning ...
Automated exam question set generator using utility based agent and learning ...Journal Papers
 
Effectiveness of test case
Effectiveness of test caseEffectiveness of test case
Effectiveness of test caseijseajournal
 
SOFTWARE ENGINEERING UNIT 6 Ch 13
SOFTWARE ENGINEERING UNIT 6 Ch 13SOFTWARE ENGINEERING UNIT 6 Ch 13
SOFTWARE ENGINEERING UNIT 6 Ch 13Azhar Shaik
 
Gui path oriented test generation algorithms paper
Gui path oriented test generation algorithms paperGui path oriented test generation algorithms paper
Gui path oriented test generation algorithms paperIzzat Alsmadi
 
A survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsA survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsAhmed Magdy Ezzeldin, MSc.
 
JUnit with_mocking
JUnit with_mockingJUnit with_mocking
JUnit with_mockingZeeshan Khan
 
Integrating goals after prioritization and
Integrating goals after prioritization andIntegrating goals after prioritization and
Integrating goals after prioritization andijseajournal
 
Software Testing Outline Performances and Measurements
Software Testing Outline Performances and MeasurementsSoftware Testing Outline Performances and Measurements
Software Testing Outline Performances and Measurementsijtsrd
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testingjfrchicanog
 
Model based fault diagnosis techniques 2e
Model based fault diagnosis techniques 2eModel based fault diagnosis techniques 2e
Model based fault diagnosis techniques 2eSpringer
 
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET Journal
 

Was ist angesagt? (19)

Verification and validation of knowledge bases using test cases generated by ...
Verification and validation of knowledge bases using test cases generated by ...Verification and validation of knowledge bases using test cases generated by ...
Verification and validation of knowledge bases using test cases generated by ...
 
AI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsAI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOps
 
Towards formulating dynamic model for predicting defects in system testing us...
Towards formulating dynamic model for predicting defects in system testing us...Towards formulating dynamic model for predicting defects in system testing us...
Towards formulating dynamic model for predicting defects in system testing us...
 
Testing and test case generation by using fuzzy logic and n
Testing and test case generation by using fuzzy logic and nTesting and test case generation by using fuzzy logic and n
Testing and test case generation by using fuzzy logic and n
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.ppt
 
Automated exam question set generator using utility based agent and learning ...
Automated exam question set generator using utility based agent and learning ...Automated exam question set generator using utility based agent and learning ...
Automated exam question set generator using utility based agent and learning ...
 
Effectiveness of test case
Effectiveness of test caseEffectiveness of test case
Effectiveness of test case
 
SOFTWARE ENGINEERING UNIT 6 Ch 13
SOFTWARE ENGINEERING UNIT 6 Ch 13SOFTWARE ENGINEERING UNIT 6 Ch 13
SOFTWARE ENGINEERING UNIT 6 Ch 13
 
Gui path oriented test generation algorithms paper
Gui path oriented test generation algorithms paperGui path oriented test generation algorithms paper
Gui path oriented test generation algorithms paper
 
A survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithmsA survey of fault prediction using machine learning algorithms
A survey of fault prediction using machine learning algorithms
 
JUnit with_mocking
JUnit with_mockingJUnit with_mocking
JUnit with_mocking
 
Integrating goals after prioritization and
Integrating goals after prioritization andIntegrating goals after prioritization and
Integrating goals after prioritization and
 
Wcre13a.ppt
Wcre13a.pptWcre13a.ppt
Wcre13a.ppt
 
Software Testing Outline Performances and Measurements
Software Testing Outline Performances and MeasurementsSoftware Testing Outline Performances and Measurements
Software Testing Outline Performances and Measurements
 
20 54-1-pb
20 54-1-pb20 54-1-pb
20 54-1-pb
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testing
 
Model based fault diagnosis techniques 2e
Model based fault diagnosis techniques 2eModel based fault diagnosis techniques 2e
Model based fault diagnosis techniques 2e
 
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
 
1.basics of software testing
1.basics of software testing 1.basics of software testing
1.basics of software testing
 

Ähnlich wie Cpaior13.ppt

Application Testing For Software Testing
Application Testing For Software TestingApplication Testing For Software Testing
Application Testing For Software TestingRobyn Champagne
 
Development of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural networkDevelopment of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural networkIJAAS Team
 
MIT521 software testing (2012) v2
MIT521   software testing  (2012) v2MIT521   software testing  (2012) v2
MIT521 software testing (2012) v2Yudep Apoi
 
Generation of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing PrincipleGeneration of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing Principleiosrjce
 
Test case-point-analysis (whitepaper)
Test case-point-analysis (whitepaper)Test case-point-analysis (whitepaper)
Test case-point-analysis (whitepaper)KMS Technology
 
Different Methodologies For Testing Web Application Testing
Different Methodologies For Testing Web Application TestingDifferent Methodologies For Testing Web Application Testing
Different Methodologies For Testing Web Application TestingRachel Davis
 
Dipali_Jambhale_7+ Yrs_Testing_Professional
Dipali_Jambhale_7+ Yrs_Testing_ProfessionalDipali_Jambhale_7+ Yrs_Testing_Professional
Dipali_Jambhale_7+ Yrs_Testing_ProfessionalDipa Tripathi
 
Chapter 9 Testing Strategies.ppt
Chapter 9 Testing Strategies.pptChapter 9 Testing Strategies.ppt
Chapter 9 Testing Strategies.pptVijayaPratapReddyM
 
Machine Learning Based Approach for Predicting Fault in Software Engineering ...
Machine Learning Based Approach for Predicting Fault in Software Engineering ...Machine Learning Based Approach for Predicting Fault in Software Engineering ...
Machine Learning Based Approach for Predicting Fault in Software Engineering ...ijcnes
 
Unit Testing Essay
Unit Testing EssayUnit Testing Essay
Unit Testing EssayDani Cox
 
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...A Literature Review Study Of Software Defect Prediction Using Machine Learnin...
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...Sean Flores
 
A Complexity Based Regression Test Selection Strategy
A Complexity Based Regression Test Selection StrategyA Complexity Based Regression Test Selection Strategy
A Complexity Based Regression Test Selection StrategyCSEIJJournal
 
Enabling and Supporting the Debugging of Field Failures (Job Talk)
Enabling and Supporting the Debugging of Field Failures (Job Talk)Enabling and Supporting the Debugging of Field Failures (Job Talk)
Enabling and Supporting the Debugging of Field Failures (Job Talk)James Clause
 
A Research Study On Importance Of Testing And Quality Assurance In Software D...
A Research Study On Importance Of Testing And Quality Assurance In Software D...A Research Study On Importance Of Testing And Quality Assurance In Software D...
A Research Study On Importance Of Testing And Quality Assurance In Software D...Martha Brown
 
Thesis+of+abdelilah+sakti.ppt
Thesis+of+abdelilah+sakti.pptThesis+of+abdelilah+sakti.ppt
Thesis+of+abdelilah+sakti.pptPtidej Team
 
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...gerogepatton
 
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...gerogepatton
 

Ähnlich wie Cpaior13.ppt (20)

Cpaior13.ppt
Cpaior13.pptCpaior13.ppt
Cpaior13.ppt
 
Application Testing For Software Testing
Application Testing For Software TestingApplication Testing For Software Testing
Application Testing For Software Testing
 
Development of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural networkDevelopment of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural network
 
Software testing
Software testingSoftware testing
Software testing
 
MIT521 software testing (2012) v2
MIT521   software testing  (2012) v2MIT521   software testing  (2012) v2
MIT521 software testing (2012) v2
 
Generation of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing PrincipleGeneration of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing Principle
 
D017642026
D017642026D017642026
D017642026
 
Test case-point-analysis (whitepaper)
Test case-point-analysis (whitepaper)Test case-point-analysis (whitepaper)
Test case-point-analysis (whitepaper)
 
Different Methodologies For Testing Web Application Testing
Different Methodologies For Testing Web Application TestingDifferent Methodologies For Testing Web Application Testing
Different Methodologies For Testing Web Application Testing
 
Dipali_Jambhale_7+ Yrs_Testing_Professional
Dipali_Jambhale_7+ Yrs_Testing_ProfessionalDipali_Jambhale_7+ Yrs_Testing_Professional
Dipali_Jambhale_7+ Yrs_Testing_Professional
 
Chapter 9 Testing Strategies.ppt
Chapter 9 Testing Strategies.pptChapter 9 Testing Strategies.ppt
Chapter 9 Testing Strategies.ppt
 
Machine Learning Based Approach for Predicting Fault in Software Engineering ...
Machine Learning Based Approach for Predicting Fault in Software Engineering ...Machine Learning Based Approach for Predicting Fault in Software Engineering ...
Machine Learning Based Approach for Predicting Fault in Software Engineering ...
 
Unit Testing Essay
Unit Testing EssayUnit Testing Essay
Unit Testing Essay
 
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...A Literature Review Study Of Software Defect Prediction Using Machine Learnin...
A Literature Review Study Of Software Defect Prediction Using Machine Learnin...
 
A Complexity Based Regression Test Selection Strategy
A Complexity Based Regression Test Selection StrategyA Complexity Based Regression Test Selection Strategy
A Complexity Based Regression Test Selection Strategy
 
Enabling and Supporting the Debugging of Field Failures (Job Talk)
Enabling and Supporting the Debugging of Field Failures (Job Talk)Enabling and Supporting the Debugging of Field Failures (Job Talk)
Enabling and Supporting the Debugging of Field Failures (Job Talk)
 
A Research Study On Importance Of Testing And Quality Assurance In Software D...
A Research Study On Importance Of Testing And Quality Assurance In Software D...A Research Study On Importance Of Testing And Quality Assurance In Software D...
A Research Study On Importance Of Testing And Quality Assurance In Software D...
 
Thesis+of+abdelilah+sakti.ppt
Thesis+of+abdelilah+sakti.pptThesis+of+abdelilah+sakti.ppt
Thesis+of+abdelilah+sakti.ppt
 
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
 
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
Software Testing: Issues and Challenges of Artificial Intelligence & Machine ...
 

Mehr von Ptidej Team

From IoT to Software Miniaturisation
From IoT to Software MiniaturisationFrom IoT to Software Miniaturisation
From IoT to Software MiniaturisationPtidej Team
 
Presentation by Lionel Briand
Presentation by Lionel BriandPresentation by Lionel Briand
Presentation by Lionel BriandPtidej Team
 
Manel Abdellatif
Manel AbdellatifManel Abdellatif
Manel AbdellatifPtidej Team
 
Azadeh Kermansaravi
Azadeh KermansaraviAzadeh Kermansaravi
Azadeh KermansaraviPtidej Team
 
CSED - Manel Grichi
CSED - Manel GrichiCSED - Manel Grichi
CSED - Manel GrichiPtidej Team
 
Cristiano Politowski
Cristiano PolitowskiCristiano Politowski
Cristiano PolitowskiPtidej Team
 
Will io t trigger the next software crisis
Will io t trigger the next software crisisWill io t trigger the next software crisis
Will io t trigger the next software crisisPtidej Team
 
Thesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptThesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptPtidej Team
 
Thesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptThesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptPtidej Team
 

Mehr von Ptidej Team (20)

From IoT to Software Miniaturisation
From IoT to Software MiniaturisationFrom IoT to Software Miniaturisation
From IoT to Software Miniaturisation
 
Presentation
PresentationPresentation
Presentation
 
Presentation
PresentationPresentation
Presentation
 
Presentation
PresentationPresentation
Presentation
 
Presentation by Lionel Briand
Presentation by Lionel BriandPresentation by Lionel Briand
Presentation by Lionel Briand
 
Manel Abdellatif
Manel AbdellatifManel Abdellatif
Manel Abdellatif
 
Azadeh Kermansaravi
Azadeh KermansaraviAzadeh Kermansaravi
Azadeh Kermansaravi
 
Mouna Abidi
Mouna AbidiMouna Abidi
Mouna Abidi
 
CSED - Manel Grichi
CSED - Manel GrichiCSED - Manel Grichi
CSED - Manel Grichi
 
Cristiano Politowski
Cristiano PolitowskiCristiano Politowski
Cristiano Politowski
 
Will io t trigger the next software crisis
Will io t trigger the next software crisisWill io t trigger the next software crisis
Will io t trigger the next software crisis
 
MIPA
MIPAMIPA
MIPA
 
Thesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptThesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.ppt
 
Thesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptThesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.ppt
 
Medicine15.ppt
Medicine15.pptMedicine15.ppt
Medicine15.ppt
 
Qrs17b.ppt
Qrs17b.pptQrs17b.ppt
Qrs17b.ppt
 
Icpc11c.ppt
Icpc11c.pptIcpc11c.ppt
Icpc11c.ppt
 
Icsme16.ppt
Icsme16.pptIcsme16.ppt
Icsme16.ppt
 
Msr17a.ppt
Msr17a.pptMsr17a.ppt
Msr17a.ppt
 
Icsoc15.ppt
Icsoc15.pptIcsoc15.ppt
Icsoc15.ppt
 

Kürzlich hochgeladen

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum ComputingGDSC PJATK
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 

Kürzlich hochgeladen (20)

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum Computing
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 

Cpaior13.ppt

  • 1. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Constraint-based Fitness Function for Search-based Software Testing Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions Abdelilah Sakti Yann-Ga¨l Gu´h´neuc e e e Gilles Pesant Department of Computer and Software Engineering ´ Ecole Polytechnique de Montr´al, Qu´bec, Canada e e May 21, 2013 CPAIOR, Yorktown Heights Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
  • 2. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 2 / 14 What is the cost of a programming error ?
  • 3. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 2 / 14 What is the cost of a programming error ? Every year inadequate infrastructure software costs the U.S. economy around 60 billion $.
  • 4. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 2 / 14 What is the cost of a programming error ? Every year inadequate infrastructure software costs the U.S. economy around 60 billion $. Between 1985 and 1987 : the radiotherapy machine Therac-25 sent to patients a X-ray dose 100 times greater than expected. At least five deaths ; Several other patients were severely affected by radiation.
  • 5. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions What is the cost of a programming error ? Every year inadequate infrastructure software costs the U.S. economy around 60 billion $. Between 1985 and 1987 : the radiotherapy machine Therac-25 sent to patients a X-ray dose 100 times greater than expected. At least five deaths ; Several other patients were severely affected by radiation. Only a high quality software can reduce errors cost. Testing is an important technique for validating and checking the correctness of software. Despite that the testing may cost more than 50% of the budget of critical software, it remains an inevitable phase in the life cycle of software. 2 / 14
  • 6. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 3 / 14 Software testing Formal Methods Is a way of modelling and verifying a software by using mathematics or logic techniques ; Aims to prove and argue that a software respect some properties (e.g., an undesired behaviour will never occur). Some complex programmes are difficult or impossible to model.
  • 7. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions Software testing Formal Methods Is a way of modelling and verifying a software by using mathematics or logic techniques ; Aims to prove and argue that a software respect some properties (e.g., an undesired behaviour will never occur). Some complex programmes are difficult or impossible to model. Informal or Pragmatic Techniques Performed by executing a real implementation on a small subset of all possible inputs data that satisfies some criteria (e.g., all statements, all branches...) ; Aims to show the presence of errors. 3 / 14
  • 8. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions Software testing Formal Methods Is a way of modelling and verifying a software by using mathematics or logic techniques ; Aims to prove and argue that a software respect some properties (e.g., an undesired behaviour will never occur). Some complex programmes are difficult or impossible to model. Informal or Pragmatic Techniques Performed by executing a real implementation on a small subset of all possible inputs data that satisfies some criteria (e.g., all statements, all branches...) ; Aims to show the presence of errors. One main challenge is automating inputs data generation. 3 / 14
  • 9. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 4 / 14 Search Based Software Testing (SBST)
  • 10. Constraint-based Fitness Function for Search-based Software Testing Search Based Software Testing (SBST) Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Fitness Evaluation Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions The Fitness Evaluation is the core of SBST. 4 / 14
  • 11. Constraint-based Fitness Function for Search-based Software Testing Search Based Software Testing (SBST) Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction What is the cost of a programming error ? Software testing Search Based Software Testing (SBST) Fitness Evaluation Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions The Fitness Evaluation is the core of SBST. An evolution in the fitness evaluation may bring a significant enhancement to SBST 4 / 14
  • 12. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness Evaluation Conclusions 5 / 14 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target }
  • 13. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } tinyy == z tinyy > 0 tinyx == 10 Evaluation Conclusions 5 / 14 tinyTarget tinyEND
  • 14. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z tinyy > 0 tinyx == 10 Evaluation Conclusions 5 / 14 tinyTarget tinyEND
  • 15. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation Conclusions 5 / 14 tinyTarget tinyEND
  • 16. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 3 fSE
  • 17. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 3+ 90 91 fSE
  • 18. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 3+ 90 91 fSE 90 91
  • 19. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 3+ 90 91 fSE 90 91 + 31 32
  • 20. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 3+ 90 91 90 91 + 31 32 fSE +0
  • 21. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 i2 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 90 3+ 91 = 3.9890 2 + 21 =2.9545 22 fSE 31 + 32 +0= 1.9577 + 21 + 20 =1.9068 22 21 90 91 0
  • 22. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Motivating Example tinyx, y, z 1 2 Introduction 3 Motivating Example 5 Fitness Function Based on Branch Hardness 4 6 int sample(int x,int y,int z){ if(y==z) if(y>0) if(x==10) ...//Target } i1 : (10,−30,60) i2 : (30, −20, −20) tinyy == z i1 tinyy > 0 i2 Critical Branch of i1 tinyx == 10 Evaluation tinyTarget Conclusions tinyEND Test candidate i1 i2 5 / 14 fAL 90 3+ 91 = 3.9890 2 + 21 =2.9545 22 fSE 31 + 32 +0= 1.9577 + 21 + 20 =1.9068 22 21 90 91 0 Both fitness functions favour i2 over i1 because either they do not consider non-executed branches or they ignore some branches’ future.
  • 23. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 6 / 14 Fitness Function Based on Branch Hardness Combines SBST with static analysis of the non-executed branches. Prioritizes branches according to how hard it is to satisfy them.
  • 24. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 6 / 14 Fitness Function Based on Branch Hardness Combines SBST with static analysis of the non-executed branches. Prioritizes branches according to how hard it is to satisfy them. Defines the difficulty to satisfy a constraint in terms of its arity and its projection tightness (the ratio of the approximate number of solutions to the size of its search space).
  • 25. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 6 / 14 Fitness Function Based on Branch Hardness Combines SBST with static analysis of the non-executed branches. Prioritizes branches according to how hard it is to satisfy them. Defines the difficulty to satisfy a constraint in terms of its arity and its projection tightness (the ratio of the approximate number of solutions to the size of its search space). 1. The lower the arity of the constraint, the less freedom we have to choose some of its variables in order to evolve the test candidate.
  • 26. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 6 / 14 Fitness Function Based on Branch Hardness Combines SBST with static analysis of the non-executed branches. Prioritizes branches according to how hard it is to satisfy them. Defines the difficulty to satisfy a constraint in terms of its arity and its projection tightness (the ratio of the approximate number of solutions to the size of its search space). 1. The lower the arity of the constraint, the less freedom we have to choose some of its variables in order to evolve the test candidate. 2. A projection tightness close to 0 will indicate high constrainedness and hardness to satisfy a constraint.
  • 27. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 7 / 14 Branch-Hardness Metrics Difficulty Coefficient (DC) DC is a possible representation of the hardness of a branch ; Each constraint has its own DC that is determined according to its arity and tightness ; DC(c) = B 2 · 1 arityc + B · (1 − tightness) + 1 .
  • 28. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Branch-Hardness Metrics Difficulty Coefficient (DC) DC is a possible representation of the hardness of a branch ; Each constraint has its own DC that is determined according to its arity and tightness ; DC(c) = B 2 · 1 arityc + B · (1 − tightness) + 1 . Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions Difficulty Level (DL) DL is based on DC ranking (r) ; DL is a representation of a relative hardness level of a constraint in a set of constraints ; DL(c, C) = 7 / 14 |C|, 2r−1 · (|C| + 1), if r = 0 . if r > 0
  • 29. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 8 / 14 Branch-Hardness Fitness Functions DC Fitness Function (fDC ) DC is used as a penalty coefficient for breaking a constraint ; The target of this fitness function is determining a standard-branch-distance. fDC (i, C) = c∈C DC(c) · η(i, c) .
  • 30. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Branch-Hardness Fitness Functions DC Fitness Function (fDC ) DC is used as a penalty coefficient for breaking a constraint ; The target of this fitness function is determining a standard-branch-distance. fDC (i, C) = c∈C DC(c) · η(i, c) . Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions DL Fitness Function (fDL ) DL is used as a constant penalty for breaking a constraint in a set of constraints to satisfy ; fDL (i, C) = where (i, c) = 8 / 14 c∈C (i, c) + η(i, c), 0, if η(i, c) = 0 . DL(c, C), if η(i, c) = 0
  • 31. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions 9 / 14 Applying fDC on our motivating example Assume that all domains are equal to [−99, 100]. DC for each branch 1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ; 2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ; 3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95.
  • 32. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Applying fDC on our motivating example Assume that all domains are equal to [−99, 100]. DC for each branch 1. DC(”y == z”) = 102 · 0.5 + 10 · 0.995 + 1 = 60.95 ; 2. DC(”y > 0”) = 102 · 1 + 10 · 0.5 + 1 = 106 ; 3. DC(”x == 10”) = 102 · 1 + 10 · 0.995 + 1 = 110.95. Overview Branch-Hardness Metrics Branch-Hardness Fitness Functions Applying fDC on our motivating example Evaluation Conclusions (fDC ) for each test candidate 90 fDC (i1 , C) = 60.95· 91 +106· 31 +110.95· 0 = 162.9677 ; 32 1 fDC (i2 , C) = 60.95· 0 +106· 21 +110.95· 20 = 206.8485. 1 22 21 Contrary to fAL and fSE this fitness function makes the adequate choice by choosing the test candidate i1 instead of i2 . 9 / 14
  • 33. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Evaluation Subjects and configuration EA Results SA Results Conclusions 10 / 14 Evaluation Subjects and configuration analysed meta-heuristics and fitness functions Two widely used meta-heuristic algorithms are analysed : Simulated Annealing (SA) and Evolutionary Algorithm (EA). fAL and fSE from the literature ; fSEL a natural combination of fAL and fSE ; Our fitness functions fDC and fDL .
  • 34. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Evaluation Subjects and configuration EA Results SA Results Conclusions Evaluation Subjects and configuration analysed meta-heuristics and fitness functions Two widely used meta-heuristic algorithms are analysed : Simulated Annealing (SA) and Evolutionary Algorithm (EA). fAL and fSE from the literature ; fSEL a natural combination of fAL and fSE ; Our fitness functions fDC and fDL . Subjects 440 synthetic test targets that were randomly generated. 20 executions for every combination of fitness function and meta-heuristic algorithm. If test data was not found after 25000 (respectively 100000) fitness evaluations for EA (respectively SA), the search was terminated. 10 / 14
  • 35. Constraint-based Fitness Function for Search-based Software Testing Evaluation EA Results Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant EA: Comparing all fitnesses on 440 test targets Approach Level Symbolic Enhanced Symbolic Enhanced with levels Difficulty Coefficient Difficulty Level Introduction Motivating Example Fitness Function Based on Branch Hardness 80 60 Evaluation Subjects and configuration EA Results 40 SA Results Conclusions 20 0 0 5000 10000 15000 Evaluations 11 / 14 20000 25000
  • 36. Constraint-based Fitness Function for Search-based Software Testing Evaluation SA Results Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant SA: Comparing all fitnesses on 440 test targets Approach Level Symbolic Enhanced Symbolic Enhanced with levels Difficulty Coefficient Difficulty Level Introduction 80 Fitness Function Based on Branch Hardness Evaluation Subjects and configuration EA Results SA Results Conclusions Branch Coverage % Motivating Example 60 40 20 0 0e+00 2e+04 4e+04 6e+04 Evaluations 12 / 14 8e+04 1e+05
  • 37. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 13 / 14 Conclusions Two new metrics to measure the difficulty to satisfy a constraint in the context of test case generation for software testing are defined ; Two new fitness functions for SBST are defined ; Our new fitness functions are significantly more effective and efficient than with the largely used fitness functions from the literature. The obtained results are promising but more experiments must be performed.
  • 38. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions Conclusions Two new metrics to measure the difficulty to satisfy a constraint in the context of test case generation for software testing are defined ; Two new fitness functions for SBST are defined ; Our new fitness functions are significantly more effective and efficient than with the largely used fitness functions from the literature. The obtained results are promising but more experiments must be performed. Future work Performing more experiments on real world programs ; Extending our approach by defining new ways to compute projection tightness for constraints involving other data types than integer. 13 / 14
  • 39. Constraint-based Fitness Function for Search-based Software Testing Abdelilah Sakti, Yann-Ga¨l e Gu´h´neuc, Gilles e e Pesant Introduction Motivating Example Fitness Function Based on Branch Hardness Evaluation Conclusions 14 / 14 Thank you Questions ?