Developers' effort, time, exploration, strategies, Mylyn

1. How Developers
Spend their Effort
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Towards Understanding How Developers
Spend their Effort during
Maintenance Activities
e
e e
Z´phyrin Soh, Foutse Khomh, Yann-Ga¨l Gu´h´neuc,
e
Giuliano Antoniol
Department of Computer and Software Engineering
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e
October 16, 2013
Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab

2. How Developers
Spend their Effort
Outline
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Introduction
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Effort vs. Complexity
Factors Affecting Effort
Conclusion
Conclusion
2 / 15

3. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Context and Example (1/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
3 / 15
83
Eclipse bug #1880
Patch #74156
File: 2
LOC : 26
+ 18 LOC
- 8 LOC

4. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Context and Example (1/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
3 / 15
84
Eclipse bug #1348
#94002
Patch
File: 2
LOC : 20
+ 19 LOC
- 1 LOC

5. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Context and Example (1/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Complexity of the Changes
Which change is more complex?
83
Eclipse bug #1880
Patch #74156
File: 2
LOC : 26
+ 18 LOC
- 8 LOC
3 / 15
vs.
84
Eclipse bug #1348
Patch #94002
File: 2
LOC : 20
+ 19 LOC
- 1 LOC

6. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Context and Example (2/2)
Introduction
Context and Example
Data
Effort Needed to Provide a Patch
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Spend a certain effort to:
Explore the program
Find relevant program entities
Understand entities and make changes
4 / 15

7. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Context and Example (2/2)
Introduction
Context and Example
Data
Effort Needed to Provide a Patch
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Spend a certain effort to:
Explore the program
Find relevant program entities
Understand entities and make changes
1. How to estimate the effort spend to provide a patch?
2. Does a complex patch need more effort?
4 / 15

8. How Developers
Spend their Effort
Introduction
Z´phyrin Soh et al.
e
Data
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
5 / 15
Need detailed information
Developers’ programming activities
Interactions histories
⇒ developers’ effort
Changes made to address the tasks
Patches
⇒ source code before and after
changes

9. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Research Question
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
6 / 15
Does the complexity of the implementation of a
task reflect developers effort?

10. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Metrics
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
7 / 15
Developers’ effort
Time Spend: Total duration spent on all files and their
contents
Cyclomatic complexity: Cyclomatic complexity of the
exploration graph

11. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Metrics
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Developers’ effort
Time Spend: Total duration spent on all files and their
contents
Cyclomatic complexity: Cyclomatic complexity of the
exploration graph
Complexity of the changes
Entropy: How much the changes are scattered between
files [1]
Change distance: How much difference between the
source code before the changes and source code after.
[1] A. E. Hassan, Predicting faults using the complexity of code changes, ICSE
2009
7 / 15

12. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (1/2)
Introduction
Context and Example
Data
How do we match interactions and patches?
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
8 / 15
2,408 Interactions histories
?
3,395 Patches

13. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (1/2)
Introduction
Context and Example
Data
How do we match interactions and patches?
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
2,408 Interactions histories
?
3,395 Patches
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Assumption: An interaction is matched to a patch (i.e., the
patch is the result of the corresponding interaction) if and
only if both are attached to the same bug report, by the
same developer at the same date (date/hour/minutes).
8 / 15

14. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (2/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Unbalanced matchings
Developers modify files without interacting with them:
Research Question
Metrics
Matching
Results
Changes not requiring much effort, e.g., propagation of
refactoring
Factors Affecting
Effort
Interactions are not collected when performing the task
Additional Files
Bug Severity
Developers’
Experience
Conclusion
9 / 15

15. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (2/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Unbalanced matchings
Developers modify files without interacting with them:
Research Question
Metrics
Matching
Results
Changes not requiring much effort, e.g., propagation of
refactoring
Factors Affecting
Effort
Interactions are not collected when performing the task
Additional Files
Bug Severity
Developers’
Experience
Conclusion
9 / 15

16. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (2/2)
Introduction
Unbalanced matchings
Context and Example
Data
Effort vs.
Complexity
Developers modify files without interacting with them:
Research Question
Metrics
Matching
Results
Changes not requiring much effort, e.g., propagation of
refactoring
Factors Affecting
Effort
Interactions are not collected when performing the task
Additional Files
Bug Severity
Developers’
Experience
Conclusion
F1
F3 F5
F2 F4 F6
F7
9 / 15
F1
F2
F3
F8

17. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (2/2)
Introduction
Unbalanced matchings
Context and Example
Data
Effort vs.
Complexity
Developers modify files without interacting with them:
Research Question
Metrics
Matching
Results
Changes not requiring much effort, e.g., propagation of
refactoring
Factors Affecting
Effort
Interactions are not collected when performing the task
Additional Files
Bug Severity
Developers’
Experience
Conclusion
F3 F5
F1
F2 F4 F6
F7
F1
F3
9 / 15
F2
F4
F1
F3
F2
F5
F8
F7
F6
F8
F9

18. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Matching (2/2)
Introduction
Unbalanced matchings
Context and Example
Data
Effort vs.
Complexity
Developers modify files without interacting with them:
Research Question
Metrics
Matching
Results
Changes not requiring much effort, e.g., propagation of
refactoring
Factors Affecting
Effort
Interactions are not collected when performing the task
Additional Files
Bug Severity
Developers’
Experience
Conclusion
F3 F5
F1
F2 F4 F6
F7
F1
F3
9 / 15
F2
F4
F1
F3
F2
F5
F8
F7
F6
F8
F9

19. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Results
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
10 / 15
Effort vs. complexity of the changes
1028 matchings and 217 unbalanced matchings

20. How Developers
Spend their Effort
Effort vs. Complexity
Z´phyrin Soh et al.
e
Results
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Effort vs. complexity of the changes
1028 matchings and 217 unbalanced matchings
Developers do not necessary spend more effort on tasks
requiring more complex changes
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Time (sec.)
Time (sec.)
Cyclomatic Complexity
10 / 15
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Entropy
Change distance
Entropy
Change distance

21. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Additional Files
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
11 / 15
Additional Files
Exploring files that should not be modified

22. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Additional Files
Introduction
Additional Files
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Exploring files that should not be modified
Significantly relevant files vs. additional (useful and
accidental) files
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
F1
F2
F4
F7
F3
F5
F6
F9
62%
11 / 15
F8
38%

23. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Additional Files
Introduction
Additional Files
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Exploring files that should not be modified
Significantly relevant files vs. additional (useful and
accidental) files
Effort vs. number of additional files: 0.63 (time) and
0.82 (cyclomatic complexity)
Conclusion
F1
F2
F4
F7
F3
F5
F6
F9
62%
11 / 15
F8
38%

24. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs
[2] Lamkanfi et al., Predicting the severity of a reported bug, MSR 2010
[3] Panjer. Predicting eclipse bug lifetimes, MSR 2007
12 / 15

25. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
12 / 15
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs

26. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Bug Severity
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Bug severity
Bug severity indicates how much a bug can affect the
performance and stability of the system [2]
The resolution time of severe bugs is greater than the
resolution time of less severe bugs [3]
Developers may spent more effort when fixing severe
bugs wrt. less severe bugs
15
663
132
218
12 / 15

27. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (1/2)
Introduction
Developers’ Experience
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
13 / 15
NB
NF and NLOC (Overall and relevant)

28. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (1/2)
Introduction
Developers’ Experience
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
13 / 15
NB
NF and NLOC (Overall and relevant)
Task
T1
NB
F1
F3 F5
F2 F4 F6
F7
NF
NLOC
0
0 (0) 0 (0)
F1 (2 LOC)
F2 (5 LOC)

29. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (1/2)
Introduction
Developers’ Experience
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
NB
NF and NLOC (Overall and relevant)
Task
T1
Conclusion
T2 (Case 1)
13 / 15
NB
F1
F3 F5
F2 F4 F6
F7
F3
F4
F5
F5
F7
NLOC
0
0 (0) 0 (0)
1
2 (0) 7 (0)
F1 (2 LOC)
F2 (5 LOC)
F5
F6
NF
F3
F4

30. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (1/2)
Introduction
Developers’ Experience
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
NB
NF and NLOC (Overall and relevant)
Task
T1
Conclusion
T2 (Case 1)
T2 (Case 2)
13 / 15
NB
F1
F3 F5
F2 F4 F6
F7
F3
F4
F3
F4
F5
F5
F7
F5
F6
F7
0
0 (0) 0 (0)
F3
1
2 (0) 7 (0)
1
2 (1) 7 (5)
F4
F2
F6
NLOC
F1 (2 LOC)
F2 (5 LOC)
F5
F2
NF
F3
F4

31. How Developers
Spend their Effort
Factors Affecting Effort
Z´phyrin Soh et al.
e
Developers’ Experience (2/2)
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
Developers’ Experience
Developers experience does not reduce their effort
When a program evolves, developers may increasingly
perform tasks on parts of the program on which they
have no previous experience
Consistent result with [4] (#commits) for Mylyn and
PDE project
⇒ NB and NF can assess developers’ experience
[4] Robbes et al., Using developer interaction data to compare expertise
metrics, MSR 2013
14 / 15

32. How Developers
Spend their Effort
Conclusion
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
15 / 15
Time (sec.)
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Entropy
Change distance
Entropy
Change distance

33. How Developers
Spend their Effort
Conclusion
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
Conclusion
15 / 15
F1
Time (sec.)
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Change distance
Entropy
Change distance
F2
F4
Entropy
F7
F3
F5
F6
F8
F9
62%
38%

34. How Developers
Spend their Effort
Conclusion
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
F1
Time (sec.)
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Change distance
Entropy
F7
F3
F5
F6
15
663
132
218
F8
F9
62%
Change distance
Conclusion
15 / 15
F2
F4
Entropy
38%

35. How Developers
Spend their Effort
Conclusion
Z´phyrin Soh et al.
e
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
F1
Time (sec.)
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Change distance
Entropy
F2
F3
F4
Entropy
F7
F5
F6
Conclusion
F8
F9
62%
Change distance
38%
Task
NB
15
663
132
218
T1
T2 (Case 1)
15 / 15
F1
F3 F5
F2 F4 F6
F7
F3
F4
F5
F5
F7
F2 (5 LOC)
F5
F6
NF
NLOC
0
0 (0) 0 (0)
1
2 (0) 7 (0)
F1 (2 LOC)
F3
F4

36. How Developers
Spend their Effort
Conclusion
Z´phyrin Soh et al.
e
Thanks for your attention!
Introduction
Context and Example
Data
Effort vs.
Complexity
Research Question
Metrics
Matching
Results
Factors Affecting
Effort
Additional Files
Bug Severity
Developers’
Experience
F1
Time (sec.)
Time (sec.)
Cyclomatic Complexity
Cyclomatic Complexity
0.16
0.27
0.31
0.33
Change distance
Entropy
F2
F3
F4
Entropy
F7
F5
F6
Conclusion
F8
F9
62%
Change distance
38%
Task
NB
15
663
132
218
T1
T2 (Case 1)
15 / 15
F1
F3 F5
F2 F4 F6
F7
F3
F4
F5
F5
F7
F2 (5 LOC)
F5
F6
NF
NLOC
0
0 (0) 0 (0)
1
2 (0) 7 (0)
F1 (2 LOC)
F3
F4