Several studies have experimented with data mining algorithms to predict the fix-time of reported bugs. Unfortunately, the fix-times as reported in typical open-source cases are heavily skewed with a significant amount of reports registering fix-times less than a few minutes. Consequently, we propose to include an additional filtering step to improve the quality of the underlying data in order to gain better results. Using a small-scale replication of a previously published bug fix-time prediction experiment, we show that the additional filtering of reported bugs indeed improves the outcome of the results.

1. Proceedings of the 16th European Conference on Software Maintenance and
Reengineering
Filtering Bug Reports for Fix-Time
Analysis
Ahmed Lamkanfi, Serge Demeyer
Antwerp Systems and Software Modelling
Ansymo 1 /13

2. Bug Report Fix-Time Prediction
Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs
Panjer et al. Bougie et al. Giger et al.
2 /13

3. Bug Report Fix-Time Prediction
Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs
Panjer et al. Bougie et al. Giger et al.
2 /13

4. Bug Report Fix-Time Prediction
Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs
Panjer et al. Bougie et al. Giger et al.
2 /13

5. History of all reported bugs
Bug Database Uncover facts about history
Make predictions about future
3 /13

6. History of all reported bugs
Bug Database Uncover facts about history
Make predictions about future
Fix-time of a bug?
✓ Time between opening and resolving a bug.
3 /13

7. Cases under Study: Eclipse and Mozilla
Project Nr. of Bugs Period
Platform 76.456 Oct. 2001 - Oct. 2007
PDE 11.117 Oct. 2001 - Oct. 2007
JDT 41.691 Oct. 2001 - Oct. 2007
CDT 11.468 Oct. 2001 - Oct. 2007
GEF 1.587 Oct. 2001 - Oct. 2007
Core 143.542 Mar. 1997 - Jul. 2008
Bugzilla 19.135 Mar. 2003 - Jul. 2008
Firefox 79.272 Jul. 1999 - Jul. 2008
Thunderbird 23.408 Jan. 2000 - Jul. 2008
SeaMonkey 85.143 Nov. 1995 - Jul. 2008
4 /13

8. 3000
1000
300
100
30
Fix−Time (logarithmic scale)
10
3
1
0.3
0.1
0.03
0.01
0.003
0.001
0.0003
0.0001
Platform PDE JDT CDT GEF
Projects
5 /13

9. 3000
1000
300
100
30
Fix−Time (logarithmic scale)
10
3
1
0.3
0.1
0.03
0.01
0.003
0.001
0.0003
0.0001
Core Bugzilla Firefox Thunderbird Seamonkey
Projects
6 /13

10. Summary of the Box-Plots
Project Smallest Fix-time
Platform 10 seconds
PDE 12 seconds
JDT 10 seconds
CDT 9 seconds
GEF 8 seconds
Core 11 seconds
Bugzilla 3 seconds
Firefox 13 seconds
Thunderbird 18 seconds
SeaMonkey 14 seconds
7 /13

11. Ask a
developer!
8 /13

12. Ask a
developer!
➡“the developer has already the
necessary code changes ready to fix a
bug, then files a bug to make sure it's
getting tracked in the system”
8 /13

13. Filtering out unreliable reports?
✓ How does this impact the accuracy?
9 /13

14. Filtering out unreliable reports?
✓ How does this impact the accuracy?
Small experiment
✓ Based on experiment from “Predicting the Fix
Time of Bugs” from Giger et al. (2010)
9 /13

15. Train from the history of bug reports
✓ Fields are extracted from the reports
day opened, month opened, platform, reporter,
➡
severity,...
✓ Naïve Bayes classifiers learns the
characteristics from the reports
✓ 10-fold cross validation
10/13

16. Train from the history of bug reports
✓ Fields are extracted from the reports
day opened, month opened, platform, reporter,
➡
severity,...
✓ Naïve Bayes classifiers learns the
characteristics from the reports
✓ 10-fold cross validation
Bugs are grouped in two sets
✓ Fast: fixtime ≤ median
✓ Slow: fixtime > median
10/13

17. Evaluation:
✓ Receiver Operating Characteristic(ROC) curve
✓ Area Under Curve(AUC): 0.5 is random
prediction; 1.0 perfect classification
11/13

18. Evaluation:
✓ Receiver Operating Characteristic(ROC) curve
✓ Area Under Curve(AUC): 0.5 is random
prediction; 1.0 perfect classification
Two-fold experiment
✓ With and without the filtering of bug reports
✓ Threshold for filtering set to 1/2 of the first
quartile
11/13

19. Accuracy Results
Project AUC Before AUC After
Platform 0.692 0.700
PDE 0.641 0.661
JDT 0.646 0.649
CDT 0.693 0.708
GEF 0.663 0.732
Core 0.663 0.686
Bugzilla 0.722 0.733
Firefox 0.623 0.653
Thunderbird 0.657 0.645
SeaMonkey 0.698 0.706
12/13

20. Conclusions
✓ More investigation needed when dealing
with real-world data
✓ Some bugs are fixed conspicuously fast!
✓ More preprocessing/filtering may lead to
improved results
13/13