Maintenance, developers studies, empirical studies, exploration strategies, Mylyn

1. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
On the Effect of Exploration Strategies on
Maintenance Tasks
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Z´phyrin Soh, Yann-Ga¨l Gu´h´neuc, Giuliano Antoniol
e
e
e e
Bram Adams, Foutse Khomh
Results
RQ1
RQ2
RQ3
Department of Computer and Software Engineering
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e
e
RQ4
Conclusion and
Future Work
March 21, 2013
Conclusion
Threats to Validity and
Future Work
Pattern Trace Identification, Detection, and Enhancement in Java
SOftware Cost-effective Change and Evolution Research Lab

2. Exploration
Strategies
Outline
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Introduction
Goal
Problem
Benefits
Empirical Study
Research Questions
Goal
Data
Metrics
Strategy Identification
Results
Empirical Study
RQ1
RQ2
RQ3
Results
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
2 / 30
Conclusion and Future Work

3. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example (1/3)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
3 / 30
Developers always explore the program when performing
a maintenance task i.e., move from a program entity to
another.

4. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example (1/3)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
3 / 30
Developers always explore the program when performing
a maintenance task i.e., move from a program entity to
another.
Types of program entities (e.g., Java): project, package,
file, class, attribute, method, resource (XML file, etc.)

5. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example (1/3)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
3 / 30
Developers always explore the program when performing
a maintenance task i.e., move from a program entity to
another.
Types of program entities (e.g., Java): project, package,
file, class, attribute, method, resource (XML file, etc.)
Consider an exploration graph where:

6. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example (1/3)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
3 / 30
Developers always explore the program when performing
a maintenance task i.e., move from a program entity to
another.
Types of program entities (e.g., Java): project, package,
file, class, attribute, method, resource (XML file, etc.)
Consider an exploration graph where:
Node: a program entity

7. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example (1/3)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
3 / 30
Developers always explore the program when performing
a maintenance task i.e., move from a program entity to
another.
Types of program entities (e.g., Java): project, package,
file, class, attribute, method, resource (XML file, etc.)
Consider an exploration graph where:
Node: a program entity
Edge: moving from a program entity to another.

8. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
4 / 30
Introduction
Motivating Example (2/3)

9. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
4 / 30
Introduction
Motivating Example (2/3)

10. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
4 / 30
Introduction
Motivating Example (2/3)

11. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
4 / 30
We focus on
Introduction the program entity of
type Example Containment
Motivating class ⇒ (2/3)
principle

12. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
4 / 30
Introduction
Motivating Example (2/3)

13. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
5 / 30
Introduction
Motivating Example (3/3)

14. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
5 / 30
Introduction
Motivating Example (3/3)

15. Exploration
Strategies
Introduction
Z´phyrin Soh et al.
e
Motivating Example (3/3)graph
A
represents a program exploration
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
(a) Graph 1
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Is there any difference between exploration graphs?
How you find the difference?
Why there is difference?
How much is the difference?
5 / 30
(b) Graph 2

16. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Context (1/2)
Introduction
Motivating Example
Context
Goal
Program Exploration
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
Two main concepts:
Interaction event (IE): Each developer’s action on an
entity (through IDE) e.g., select a file Foo.java through
package explorer.
RQ1
Interaction history (IH): a sequence of interaction
events.
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
6 / 30
Many tools used to gather IH: Mylyn, SmartGroups,
CodingTracker.

17. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Context (2/2)
Introduction
Motivating Example
Context
Objectives of Program Exploration
Developers look at the relevant program entities
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Developers perform on average 19.31 navigation actions
between two modifications [1]
⇒ They spend time finding where and how to perform
a change on a program.
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
[1] D. R¨thlisberger, SmartGroups: Focusing on Task-Relevant Source
o
Artifacts in IDEs, ICPC 2011
7 / 30

18. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Context (2/2)
Introduction
Motivating Example
Context
Objectives of Program Exploration
Developers look at the relevant program entities
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Developers perform on average 19.31 navigation actions
between two modifications [1]
⇒ They spend time finding where and how to perform
a change on a program.
Results
RQ1
Research on Program Exploration
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Reduce the developers’ search effort (when searching
the relevant entities to perform a given task).
Improve the developers’ productivity.
[1] D. R¨thlisberger, SmartGroups: Focusing on Task-Relevant Source
o
Artifacts in IDEs, ICPC 2011
7 / 30

19. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem (1/1)
Main Question
How can we help developers based on the way they explore a
program?
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
8 / 30

20. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem (1/1)
Main Question
How can we help developers based on the way they explore a
program?
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
First Step
Investigate if developers have different exploration
strategies
Exploration strategies (ES): the way (how) the
developers navigate through the program entities
RQ3
Study some cause/effect relationships around ES
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
8 / 30

21. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem (1/1)
Main Question
How can we help developers based on the way they explore a
program?
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
First Step
Investigate if developers have different exploration
strategies
Exploration strategies (ES): the way (how) the
developers navigate through the program entities
RQ3
Study some cause/effect relationships around ES
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
8 / 30

22. Exploration
Strategies
Z´phyrin Soh et al.
e
Goal
Benefits (1/1)
Introduction
Motivating Example
Usefulness of the Exploration Strategies
Context
Goal
Exploration strategies can help to:
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
9 / 30
Evaluate a developer’s performance: a specific strategy
can tend to be more efficient.

23. Exploration
Strategies
Z´phyrin Soh et al.
e
Goal
Benefits (1/1)
Introduction
Motivating Example
Usefulness of the Exploration Strategies
Context
Goal
Exploration strategies can help to:
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
9 / 30
Evaluate a developer’s performance: a specific strategy
can tend to be more efficient.
Improve our knowledge on developers’ comprehension or
programming process: a top-down or bottom-up
comprehension can be related to a specific strategy

24. Exploration
Strategies
Z´phyrin Soh et al.
e
Goal
Benefits (1/1)
Introduction
Motivating Example
Usefulness of the Exploration Strategies
Context
Goal
Exploration strategies can help to:
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
9 / 30
Evaluate a developer’s performance: a specific strategy
can tend to be more efficient.
Improve our knowledge on developers’ comprehension or
programming process: a top-down or bottom-up
comprehension can be related to a specific strategy
Characterize developers’ expertise: the experienced
developers can tend to follow a specific strategy that
can be used to help inexperienced ones

25. Exploration
Strategies
Z´phyrin Soh et al.
e
Goal
Benefits (1/1)
Introduction
Motivating Example
Usefulness of the Exploration Strategies
Context
Goal
Exploration strategies can help to:
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
9 / 30
Evaluate a developer’s performance: a specific strategy
can tend to be more efficient.
Improve our knowledge on developers’ comprehension or
programming process: a top-down or bottom-up
comprehension can be related to a specific strategy
Characterize developers’ expertise: the experienced
developers can tend to follow a specific strategy that
can be used to help inexperienced ones
Develop new exploration features for IDE: a strategy
can highlight the need of a specific feature.

26. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Research Questions (1/1)
Introduction
Motivating Example
Context
Questions
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
10 / 30
Do developers follow specific exploration strategies
when performing maintenance tasks?

27. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Research Questions (1/1)
Introduction
Motivating Example
Context
Questions
Goal
Problem
Benefits
Empirical Study
Do developers follow specific exploration strategies
when performing maintenance tasks?
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
10 / 30
Do specific exploration strategies affect the maintenance time?

28. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Research Questions (1/1)
Introduction
Motivating Example
Context
Questions
Goal
Problem
Benefits
Empirical Study
Do developers follow specific exploration strategies
when performing maintenance tasks?
Research Questions
Data
Metrics
Strategy Identification
Results
Do specific exploration strategies affect the maintenance time?
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
10 / 30
Do specific exploration strategies affect the exploration effort?

29. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Research Questions (1/1)
Introduction
Motivating Example
Context
Questions
Goal
Problem
Benefits
Empirical Study
Do developers follow specific exploration strategies
when performing maintenance tasks?
Research Questions
Data
Metrics
Strategy Identification
Results
Do specific exploration strategies affect the maintenance time?
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Do specific exploration strategies affect the exploration effort?
Conclusion
Threats to Validity and
Future Work
10 / 30
Does extensive work results to a specific exploration strategy?

30. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Data (1/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
11 / 30
Data Collection
Four open-source projects: ECF, Mylyn, PDE, Eclipse
Platform
We focus on IHs build by Mylyn
Mylyn is an industrial tool provided as an Eclipse’s
plug-in (also CodingTracker)
Mylyn’s IHs are available: all contributions to Mylyn
must be made using Mylyn

31. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Data (1/3)
Introduction
Motivating Example
Context
Example: Mylyn’s IHs as attachment (XML file) to a bug
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
11 / 30

32. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
11 / 30
Empirical Study
Data (1/3)
Mylyn’s Interaction History

33. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
11 / 30
Empirical Study
Data (1/3)
Mylyn’s Interaction History

34. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
11 / 30
Empirical Study
Data (1/3)
Mylyn’s Interaction History

35. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

36. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

37. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

38. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

39. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

40. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
12 / 30
Empirical Study
Data (2/3)
Parsing Interaction Histories

41. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Data (3/3)
Introduction
Motivating Example
Context
Parsing StructureHandle
Identify the type and parts of program entity
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
13 / 30

42. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Data (3/3)
Introduction
Motivating Example
Context
Parsing StructureHandle
Identify the type and parts of program entity
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Parsing ⇒ A graph representation of an interaction
history
Remember: We focus on the program entity of type
class
13 / 30

43. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
14 / 30
Empirical Study
Metrics (1/3)
NumEvent(Classes) =
e∈Classes
NumEvent(e)

44. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
15 / 30
Empirical Study
Metrics (2/3)
NumRevisits(c) = number of times a class c is revisited

45. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Metrics (3/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
16 / 30
NumEdit: total number of edit events on all classes and
their contents
NumEdit(e) = event e NumEdit(event)
NumEdit(Classes) = e∈Classes NumEdit(e)

46. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Metrics (3/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
16 / 30
NumEdit: total number of edit events on all classes and
their contents
NumEdit(e) = event e NumEdit(event)
NumEdit(Classes) = e∈Classes NumEdit(e)
Exploration Effort: edit ratio which is related to the
identification of the relevant entities
NumEdit(Classes)
EditRatio(Classes) = NumEvent(Classes)

47. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Metrics (3/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
NumEdit: total number of edit events on all classes and
their contents
NumEdit(e) = event e NumEdit(event)
NumEdit(Classes) = e∈Classes NumEdit(e)
Exploration Effort: edit ratio which is related to the
identification of the relevant entities
NumEdit(Classes)
EditRatio(Classes) = NumEvent(Classes)
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
16 / 30
Class Level Duration (CLD): total duration spent on all
classes and their contents
Duration(event) = EndDate(event) − StartDate(event)
Duration(e) = event e Duration(event)
CLD = Duration(Classes) = e∈Classes Duration(e)

48. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Metrics (3/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
NumEdit: total number of edit events on all classes and
their contents
NumEdit(e) = event e NumEdit(event)
NumEdit(Classes) = e∈Classes NumEdit(e)
Exploration Effort: edit ratio which is related to the
identification of the relevant entities
NumEdit(Classes)
EditRatio(Classes) = NumEvent(Classes)
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Class Level Duration (CLD): total duration spent on all
classes and their contents
Duration(event) = EndDate(event) − StartDate(event)
Duration(e) = event e Duration(event)
CLD = Duration(Classes) = e∈Classes Duration(e)
Overall Duration (OD): total duration spent on an IH
OD = event∈IH Duration(event)
16 / 30

49. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Strategy Identification (1/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
Gini Inequality Index
Used in econometrics to measure the inequality of
income among a population
Gini ∈ [0, 1]: 0 (perfect equality), 1 (maximal
inequality)
We measure how much classes are (un)equally revisited
NumRevisits = income of a class
⇒ Apply Gini [2] on NumRevisits
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Gini(IH) =
1
2n2 µ
n
n
i=1 j=1
| NumRevisit(eni ) − NumRevisit(enj ) |
Conclusion
Threats to Validity and
Future Work
[2] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?,
Technical report, Department of Economics, Dalhouse University, 2004
17 / 30

50. Exploration
Strategies
Z´phyrin Soh et al.
e
Empirical Study
Strategy Identification (2/3)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Problem 1: Gini Threshold
How to know if classes are (un)equally revisited?
⇒ Define a Gini Threshold
Data
Metrics
Strategy Identification
Results
RQ1
If Gini(IH) < Threshold
⇒ Classes are almost equally revisited
⇒ EES: Extended Exploration Strategy
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
18 / 30
If Gini(IH) ≥ Threshold
⇒ Some classes are more revisited than others (core vs.
periphery)
⇒ CES: Centralized Exploration Strategy

51. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Empirical Study
Strategy Identification (3/3)
Problem 2: Cut-Off Point
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
19 / 30
For CES, core vs. periphery classes
⇒ Define a COP
Ordered the classes by NumRevisits
{(C 1, 9), (C 2, 2), (C 3, 4), (C 4, 16), (C 5, 11)}
⇒ (C 4, 16), (C 5, 11), (C 1, 9), (C 3, 4), (C 2, 2)

52. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Empirical Study
Strategy Identification (3/3)
Problem 2: Cut-Off Point
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
19 / 30
For CES, core vs. periphery classes
⇒ Define a COP
Ordered the classes by NumRevisits
{(C 1, 9), (C 2, 2), (C 3, 4), (C 4, 16), (C 5, 11)}
⇒ (C 4, 16), (C 5, 11), (C 1, 9), (C 3, 4), (C 2, 2)
COP: a maximum difference between NumRevisit of
two consecutive classes
⇒ COP = 5: Between C4-C5, and C1-C3

53. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Empirical Study
Strategy Identification (3/3)
Problem 2: Cut-Off Point
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
19 / 30
For CES, core vs. periphery classes
⇒ Define a COP
Ordered the classes by NumRevisits
{(C 1, 9), (C 2, 2), (C 3, 4), (C 4, 16), (C 5, 11)}
⇒ (C 4, 16), (C 5, 11), (C 1, 9), (C 3, 4), (C 2, 2)
COP: a maximum difference between NumRevisit of
two consecutive classes
⇒ COP = 5: Between C4-C5, and C1-C3
More than one COP: the one that provides more core
classes
⇒ Between C1-C3

54. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Empirical Study
Strategy Identification (3/3)
Problem 2: Cut-Off Point
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
For CES, core vs. periphery classes
⇒ Define a COP
Ordered the classes by NumRevisits
{(C 1, 9), (C 2, 2), (C 3, 4), (C 4, 16), (C 5, 11)}
⇒ (C 4, 16), (C 5, 11), (C 1, 9), (C 3, 4), (C 2, 2)
COP: a maximum difference between NumRevisit of
two consecutive classes
⇒ COP = 5: Between C4-C5, and C1-C3
More than one COP: the one that provides more core
classes
⇒ Between C1-C3
Core classes = {C 4, C 5, C 1}
Peripheral classes = {C 3, C 2}
19 / 30

55. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (1/4)
Introduction
Motivating Example
Context
Problem 1: Manual Validation
Build an Oracle
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
20 / 30
Random set of heterogenous interaction history
Sample size: proportionally distributed among projects
95% ± 10 confidence level
Use Graphviz to generate the exploration graphs
Subjects: Three PhD students with Java Experience

56. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (1/4)
Introduction
Motivating Example
Context
Problem 1: Manual Validation
Build an Oracle
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
Random set of heterogenous interaction history
Sample size: proportionally distributed among projects
95% ± 10 confidence level
Use Graphviz to generate the exploration graphs
Subjects: Three PhD students with Java Experience
RQ2
RQ3
Validation Process
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Explain the meaning of the graphs
Subjects’ labels of a graph: C, E or D (Doubt)
Final label: CES (or EES) if at less two C (or E)
Undecided if completely different labels or two D
20 / 30

57. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (2/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
21 / 30

58. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (2/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
Compromise Threshold
RQ2
RQ3
F-Measure: maximize both precision and recall
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
21 / 30

59. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (3/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
22 / 30
Observation 1: Developers follow mostly the EES when
performing a maintenance task.

60. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (3/4)
Introduction
Motivating Example
Context
Goal
Observation 1: Developers follow mostly the EES when
performing a maintenance task.
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Methodical developers do not reinvestigate methods as
frequently as opportunistic developers [3]
CES ⇒ opportunistic developers
EES ⇒ methodical developers
22 / 30
[3] M. P. Robillard et al., How effective developers investigate source code: An
exploratory study, IEEE TSE, 2004

61. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (4/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
23 / 30
Confounding Factor
Developers navigate through relationships between
classes and key binding ⇒ The architecture can affect
the ES

62. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (4/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
23 / 30
Confounding Factor
Developers navigate through relationships between
classes and key binding ⇒ The architecture can affect
the ES
Classes (and relations) involved in an IH = part of the
system used to perform a task

63. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (4/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Confounding Factor
Developers navigate through relationships between
classes and key binding ⇒ The architecture can affect
the ES
Classes (and relations) involved in an IH = part of the
system used to perform a task
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
23 / 30
If IHs A and B have almost the same part of a system
⇒ same ES
A and B different parts ⇒ Different ES

64. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (4/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Confounding Factor
Developers navigate through relationships between
classes and key binding ⇒ The architecture can affect
the ES
Classes (and relations) involved in an IH = part of the
system used to perform a task
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
23 / 30
If IHs A and B have almost the same part of a system
⇒ same ES
A and B different parts ⇒ Different ES
Number of common classes (NCC) between A and B to
capture their common part

65. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ1 (4/4)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Confounding Factor
Developers navigate through relationships between
classes and key binding ⇒ The architecture can affect
the ES
Classes (and relations) involved in an IH = part of the
system used to perform a task
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
23 / 30
If IHs A and B have almost the same part of a system
⇒ same ES
A and B different parts ⇒ Different ES
Number of common classes (NCC) between A and B to
capture their common part
Compute NCC for each pair of IH
No difference between the NCC of the IHs with the
same ES and the NCC of the IHs with different ES
⇒ The architecture does not affect the ES

66. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ2 (1/1)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
24 / 30
Observation 2: The EES is on average 69.39% less time
consuming than the CES.

67. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ2 (1/1)
Introduction
Motivating Example
Context
Goal
Observation 2: The EES is on average 69.39% less time
consuming than the CES.
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Methodical developers performs tasks two times faster
than opportunistic ones [3]
CES ⇒ read source code in details [3]
EES ⇒ look at explicit program entity(ies) [3]
24 / 30
[3] M. P. Robillard et al., How effective developers investigate source code: An
exploratory study, IEEE TSE, 2004

68. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ3 (1/1)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
25 / 30
Observation 3: An EES requires more exploration effort
than a CES.

69. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ3 (1/1)
Introduction
Motivating Example
Context
Goal
Observation 3: An EES requires more exploration effort
than a CES.
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
Methodical developers are successful developers
compared to opportunistic developers [3]
CES ⇒ code modifications in one place [3]
EES surprising: IH edits vs. source code repository edits
25 / 30
[3] M. P. Robillard et al., How effective developers investigate source code: An
exploratory study, IEEE TSE, 2004

70. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ4 (1/2)
Introduction
Motivating Example
Context
Goal
Problem
Expand the Work on Many Days
Open-source project: developers are volunteers
⇒ they work on their spare time.
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
26 / 30
Number of working days spent on change request

71. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ4 (1/2)
Introduction
Motivating Example
Context
Goal
Problem
Expand the Work on Many Days
Open-source project: developers are volunteers
⇒ they work on their spare time.
Benefits
Empirical Study
Number of working days spent on change request
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
26 / 30
Unbalanced proportion of IH: the more days developers
work on a change task, the more time they spend on
program entities

72. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ4 (2/2)
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
27 / 30
Observation 4: Developers usually adopt a CES when
performing maintenance tasks spanning on more than
three days.

73. Exploration
Strategies
Results
Z´phyrin Soh et al.
e
RQ4 (2/2)
Introduction
Motivating Example
Context
Goal
Problem
Observation 4: Developers usually adopt a CES when
performing maintenance tasks spanning on more than
three days.
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
27 / 30
Developers refresh their mind by (re)understanding the
entities that they explored before
Mylyn task context feature sometimes push developers
to (re)explore entities already explored in previous
working sessions

74. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
28 / 30
Conclusion and Future Work
Conclusion (1/1)

75. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
28 / 30
Conclusion and Future Work
Conclusion (1/1)

76. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
28 / 30
Conclusion and Future Work
Conclusion (1/1)

77. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
28 / 30
Conclusion and Future Work
Conclusion (1/1)

78. Exploration
Strategies
Conclusion and Future Work
Z´phyrin Soh et al.
e
Threats to Validity and Future Work (1/1)
Introduction
Motivating Example
Context
Threats to Validity and Future Work
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
29 / 30
Construct validity: An IH can be a part of developers’
work

79. Exploration
Strategies
Conclusion and Future Work
Z´phyrin Soh et al.
e
Threats to Validity and Future Work (1/1)
Introduction
Motivating Example
Context
Threats to Validity and Future Work
Goal
Problem
Benefits
Empirical Study
Construct validity: An IH can be a part of developers’
work
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
29 / 30
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data

80. Exploration
Strategies
Conclusion and Future Work
Z´phyrin Soh et al.
e
Threats to Validity and Future Work (1/1)
Introduction
Motivating Example
Context
Threats to Validity and Future Work
Goal
Problem
Benefits
Empirical Study
Construct validity: An IH can be a part of developers’
work
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data
Internal validity: We use only Mylyn’s IH
⇒ Use the IH from other tools
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
29 / 30

81. Exploration
Strategies
Conclusion and Future Work
Z´phyrin Soh et al.
e
Threats to Validity and Future Work (1/1)
Introduction
Motivating Example
Context
Threats to Validity and Future Work
Goal
Problem
Benefits
Empirical Study
Construct validity: An IH can be a part of developers’
work
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
Conclusion validity: The time recorded can be
different to the “real” time spent
⇒ Perform an experiment to collect a data
Internal validity: We use only Mylyn’s IH
⇒ Use the IH from other tools
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
29 / 30
External validity: Our subject projets are Eclipse-based
projects
⇒ Use other systems

82. Exploration
Strategies
Z´phyrin Soh et al.
e
Introduction
Motivating Example
Context
Goal
Problem
Benefits
Empirical Study
Research Questions
Data
Metrics
Strategy Identification
Results
RQ1
RQ2
RQ3
RQ4
Conclusion and
Future Work
Conclusion
Threats to Validity and
Future Work
30 / 30
Thanks for your attention!