Presented at Dagstuhl Seminar 07491, "Mining Programs and Processes", 2007.

1. Quality of Bug Reports
in Open Source
Nicolas Bettenburg Adrian Schröter Rahul Premraj
Saarland University Saarland University Saarland University
Sascha Just Cathrin Weiss Tom Zimmermann
Saarland University University of Zürich University of Calgary

2. Developers

3. Developers
175222
I wand to create a new plugin in Eclipse using CDT. Shall it
possible. I had made a R&D in eclipse documentation. I had
get an idea about create a plugin using Java. But i wand to
create a new plugin ( user defined plugin ) using CDT. After
that I wand to impliment it in my programe. If it possible?.
Any one can help me please...

4. Developers
175222
I20030205
Run the following example. Double click on a tree item and
I wand to create a new plugin in Eclipse using CDT. Shall it
notice that it does not expand.
possible. I had made a R&D in eclipse documentation. I had
get an idea about create a plugin using Java. But i wand to
Comment out the Selection listener and now double click
create a new plugin ( user defined plugin ) using CDT. After
on any tree item and notice that it expands.
that I wand to impliment it in my programe. If it possible?.
Any one can help me please...
public static void main(String[] args) {
Display display = new Display();
...

5. Researchers

6. Researchers

7. What makes a
good bug report?

8. Ask Google

9. Ask Google

10. Ask developers
Experts

11. Survey
Five minutes

12. Participants
809 contacts 156 responses

13. Information
Question 1
Which of the following items have you previously used when fixing bugs?
(select as many items as you wish)
Question 2
Which three items helped you the most?
(select at most three items)

14. Results
Helped most
Used when fixing a bug

15. Results steps to reproduce
Helped most
stack traces
screenshots
observed behavior
test cases/examples
expected behavior
version
hardware severity
Used when fixing a bug

16. Problems
Question 3
Which of the following problems have you encountered when fixing bugs?
(select as many items as you wish)
Question 4
Which three problems caused you most delay in fixing bugs?
(select at most three items)

17. Results
Most delay
Encountered when fixing a bug

18. Results incomplete information
Most delay
erroneous steps to reproduce
wrong observed behaviour
wrong expected behaviour
wrong product information
bad grammar
errors in test cases
duplicates
wrong system information
viruses/spam
Encountered when fixing a bug

19. bug bugzilla case checkbox
behavior biggest
based
comments common component crucial delay
clearly
determine developer error etc expected filed fixing
information issue missing
helpful include incomplete
mozilla number people problem product project
question reports reproduce reproduction
scope sometimes status steps system test testcase
understand used user usually version wrong

20. bug bugzilla case checkbox
behavior biggest
based
comments common component crucial delay
clearly
determine developer error etc expected filed fixing
information issue missing
helpful include incomplete
mozilla number people problem product project
question reports reproduce reproduction
scope sometimes status steps system test testcase
understand used user usually version wrong

21. Good vs. poor reports
Create a new plugin using CDT.
I wand to create a new plugin in Eclipse using CDT. Shall it
possible. I had made a R&D in eclipse documentation. I had get an
idea about create a plugin using Java. But i wand to create a new
plugin ( user defined plugin ) using CDT. After that I wand to
impliment it in my programe. If it possible?. Any one can help me
please...

22. Good vs. poor reports
Create a new plugin using CDT.
I wand to create a new plugin in Eclipse using CDT. Shall it
possible. I had made a R&D in eclipse documentation. I had get an
idea about create a plugin using Java. But i wand to create a new
plugin ( user defined plugin ) using CDT. After that I wand to
impliment it in my programe. If it possible?. Any one can help me
please...
Poor Average Good

23. Good vs. poor reports
+++ APACHE: 229 +++ ECLIPSE: 397 +++ MOZILLA: 560 +++
Create a new plugin using CDT.
I wand to create a new plugin in Eclipse using CDT. Shall it
possible. I had made a R&D in eclipse documentation. I had get an
idea about create a plugin using Java. But i wand to create a new
plugin ( user defined plugin ) using CDT. After that I wand to
impliment it in my programe. If it possible?. Any one can help me
please...
Poor Average Good

24. How can we
measure quality?

25. Measure quality
Report

26. Measure quality
Quality Score
Report CUEzilla

27. Measure quality
Quality Score
Report CUEzilla
Set of Features
Keywords Stack Trace Code Examples
Readability Repro Steps Screenshots

28. Leave-one out validation
Training – build a model
Testing

29. Results
Predicted by CUEzilla
0 0 1 0 0
0 2 0 0 0
Rated by
developers 4 11 29 17 4
0 1 6 12 5
0 1 2 4 1

30. Results
Predicted by CUEzilla
0 0 1 0 0
0 2 0 0 0
Rated by
developers 4 11 29 17 4
0 1 6 12 5
0 1 2 4 1
44% agreement

31. Results
Predicted by CUEzilla
0 0 1 0 0
0 2 0 0 0
Rated by
developers 4 11 29 17 4
0 1 6 12 5
0 1 2 4 1
44% agreement 87% close by one

32. Recommendations
Add a stack trace
to improve your
bug report by 20%.

33. Recommendations
Add a stack trace
to improve your
bug report by 20%.

34. Recommendations
Why should
I care?
Add a stack trace
to improve your
bug report by 20%.

35. Creating incentive

36. Creating incentive
Bug reports with
stack traces get
fixed sooner.

37. Mining facts
Keywords
Readability
Stack Trace
Repro Steps
Code Examples
Screenshots
Set of Features

38. Mining facts
Keywords
Readability
Stack Trace
Time to fix
Repro Steps
Code Examples
Screenshots
Set of Features Fixed or not

39. Mining facts
Keywords
Readability
Stack Trace
Correlate Time to fix
Repro Steps
Code Examples
Screenshots
Set of Features Fixed or not

40. Did you know?
Bug reports containing stack traces
get fixed sooner.
(APACHE/ECLIPSE/MOZILLA)
Bug reports that are easier to read
have lower lifetimes.
(APACHE/ECLIPSE/MOZILLA)
Including code samples in your bug
report increases the chances of it
getting fixed. (MOZILLA)

41. Future work

42. Reporter survey
Question 1
What information have you previously provided in bug reports?
(select as many items as you wish)
Question 2
What of the above information was most difficult to provide?
(select at most three items)
Question 3
In your opinion, what information is most relevant to developers?
(select at most three items)
310 responses

43. Comments
Developers 48
Reporters 127
Comments 175

44. Better tool support

45. Better tool support
Add a stack trace
to improve your
bug report by 20%.
Bug reports with
stack traces get
fixed sooner.

46. bug cases cdt code create cuezilla defined delay
behaviour
developers duplicates eclipse
documentation
fixing
encountered examples helped impliment
expected
information items java mozilla observed please
plugin poor possible problems programe quality
report results
question researchers saarland
stack
select test
screenshots steps system
shall
thanks trace university user viruses wand wrong