Software testing, anti-patterns, code smells, costs, efforts

1. Introduction
Empirical Study
Results
Conclusion
A Study on the Relation Between Antipatterns and
the Cost of Class Unit Testing
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e
University of Sannio, Benevento, Italy
March 7th, 2013
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
1 / 26

2. Introduction
Empirical Study
Results
Conclusion
Outline
Motivations
Goal
Empirical Study
Results
Other Findings
Conclusion
Ongoing Project
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
2 / 26

3. Introduction
Empirical Study
Results
Conclusion
Definition
Exemple of Antipatterns
Motivations
Goal of the Study
Definition
What are Antipatterns?
Identify poor solutions to recurring design problems[Brown et al.]
Some causes
Time-market pressure
Insufficient knowledge and or experience
Misapplication of some design patterns
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
3 / 26

4. Introduction
Empirical Study
Results
Conclusion
Definition
Exemple of Antipatterns
Motivations
Goal of the Study
Exemple of Antipatterns
Blob (God Class) : Large and complex class that centralises
the behaviour of a portion of a system
Lazy Class : Class with few responsabilities
Spaguetti Code : Class declaring long methods with no
parameters and using global variables
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
4 / 26

5. Introduction
Empirical Study
Results
Conclusion
Definition
Exemple of Antipatterns
Motivations
Goal of the Study
Motivations
Impact of Antipatterns...
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
5 / 26

6. Introduction
Empirical Study
Results
Conclusion
Definition
Exemple of Antipatterns
Motivations
Goal of the Study
Motivations
What about Testability and Testing?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
6 / 26

7. Introduction
Empirical Study
Results
Conclusion
Definition
Exemple of Antipatterns
Motivations
Goal of the Study
Goal of the Study
Study the impact of antipatterns on class testability
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
7 / 26

8. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Class Testability Measure
What is Testability?
Degree to which a system or component facilitates the
establishment of test criteria and performance of tests to
determine whether those criteria have been met [IEEE
Glossary]
Testing effort [Bache et al.]
How to Measure Testability?
Metric-Based approaches [Bruntink et al.]
number of test cases required to satisfy a given coverage
criterion [Bache et al.]
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
8 / 26

9. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
What is it?
Unit testing technique for object oriented programs [Bashir et
al.]
Based on data slices
Core idea: The correctness of a class is equivalent to the
correctness of all its slices
Focus on the test of methods interactions
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
9 / 26

10. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
10 / 26

11. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
11 / 26

12. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Why MaDUM?
Why MaDUM?
Specific to OO programs
Not require specific design documentation
Possibility to compute the number of required test cases
Upper-bound for testing cost
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
12 / 26

13. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Objects
4 systems: Ant, ArgoUML, Checkstyle & JFreechart
Systems belong to different application domains, have
bug-fixing data available and have been used in previous
studies [Abbes et al.], [Bruntink et al.]
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
13 / 26

14. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Subjects
13 antipatterns
Studied in previous studies [Khomh et al.], [Abbes et al.],
Detection tool: Decor [Moha et al.]
Name (Abbr)
Antipattern classes
No Antipattern (None)
Ant
452
297
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
ArgoUML
901
376
CheckStyle
161
99
JFreeChart
245
233
March 7th, 2013
14 / 26

15. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ1: Assess AP class impact on testing effort?
How large is the MaDUM test suite for classes participating in APs
compared to that of other classes?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
15 / 26

16. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ2: Which APs are the most expensive
How does the size of the MaDUM test suite vary among classes
participating in different kinds of APs?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
16 / 26

17. Introduction
Empirical Study
Results
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ3: Cost-effectiveness of APs testing
What is the potential cost-benefit achieved when focusing testing
on APs, as opposed to other classes?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
17 / 26

18. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
Results
Results...
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
18 / 26

19. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
RQ1: How large is the MaDUM test suite for classes
participating in APs compared to that of other classes?
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
0
# of Test Cases (<=100)
20
40
60
80
q
Classes
System
AP
NAP
Ant
AP
NAP
ArgoUML
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
AP
NAP
Checkstyle
AP
NAP
JFreeChart
March 7th, 2013
19 / 26

20. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
RQ1: How large is the MaDUM test suite for classes
participating in APs compared to that of other classes?
System
Ant
ArgoUML
CheckStyle
JFreeChart
Mean TCs AP
18
10
9
26
Mean TCs NAP
9
3
6
13
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
p-Value
< 0.01
< 0.01
= 0.01
< 0.01
Cliff’s d
0.23 (Small)
0.35 (Medium)
NA
0.22 (Small)
March 7th, 2013
20 / 26

21. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
100
RQ2: How does the size of the MaDUM test suite vary
among classes participating in different kinds of APs?
80
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
60
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
20
40
q
q
q
q
q
q
q
q
q
0
# of Test Cases (<=100)
q
AS BCSBA B CDSBP CC
LzC
LM
LPL
MC
RPB
SC
SG
SAK NONE
Kind of Antipattern
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
21 / 26

22. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
100
RQ2: How does the size of the MaDUM test suite vary
among classes participating in different kinds of APs?
80
60
q
q
40
q
q
20
# of Test Cases (<=100)
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
0
q
AS BCSBA B CDSBP CC
LzC
LM
LPL
MC
RPB
SC
SG
SAK NONE
Kind of Antipattern
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
22 / 26

23. Introduction
Empirical Study
Results
Conclusion
RQ1
RQ2
RQ3
80 100
60
40
20
0
Defects in tested classes (cum.)
RQ3: What is the potential cost-benefit achieved when
focusing testing on APs, as opposed to other classes?
0
1000
2000
3000
4000
5000
6000
7000
# of T
est Case (cum.)
It is cost-effective to analyze/test APs classes with a higher priority
than other classes
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
23 / 26

24. Introduction
Empirical Study
Results
Conclusion
Other Findings: Refactoring for Reducing Testing Cost
Effect of traditional refactoring on testing cost?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
24 / 26

25. Introduction
Empirical Study
Results
Conclusion
Other Findings: Refactoring for Reducing Testing Cost
Before
refactoring
Class (system)
Type TRS
TokenFilter (Ant)
CDSBP
5
PropPanel (ArgoUML)
Blob
5
BooleanExpressionComplexityCheck
LPL
6
(Checkstyle)
AxisState (JFreeChart)
NAP
5
DynamicTimeSeriesCollection Blob
4
(JFreeChart)
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
After
refactoring
TCs
TRS
263
2
271
3
TCs
27
43
732
5
132
248
208
1
2
11
122
March 7th, 2013
25 / 26

26. Introduction
Empirical Study
Results
Conclusion
Conclusion
Findings
Most of APs classes are expensive for unit testing
APs classes with excess of responsibilities require more testing
effort than other
Prioritize the testing of APs classes can be more cost-effective
Specific refactoring can help to reduce testing cost
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
26 / 26

27. Introduction
Empirical Study
Results
Conclusion
Conclusion
Future Work
Study more systems
Investigate the symptoms that make APs classes more
expensive
Study the impact of APs on testing effort from other aspects
Investigate specific refactoring for testability
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
27 / 26

28. Introduction
Empirical Study
Results
Conclusion
Thank you very much for your attention!
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 7th, 2013
28 / 26