Traceability recovery is a key software maintenance activity in which software engineers extract the relationships among software artifacts. Information Retrieval (IR) has been widely accepted as a method for automated traceability recovery based on the textual similarity among the software artifacts. However, a notorious difficulty for IR-based methods is that artifacts may be related even if they are not textually similar. A growing body of work addresses this challenge by combining IR-based methods with structural information from source code. Unfortunately, the accuracy of such methods is highly dependent on the IR methods. If IR methods perform poorly, the combined approaches may perform even worse. In this paper, we propose to use the feedback provided by software engineers when classifying candidate links to regulate the effect of using structural information. Specifically, our approach only considers structural information when the traceability links from the IR methods are verified by developers and classified as correct links. An empirical evaluation conducted on three systems suggests that our approach outperforms both a pure IR-based method and a simple approach for combining textual and structural information.

1. When and How Using
Structural Information to
Improve IR-based Traceability
Recovery
Annibale Panichella1, Collin McMillan2, Evan Moritz 3, Davide Palmieri 4,
Rocco Oliveto 4, Denys Poshyvaniyk 3, Andrea De Lucia1
1 Software Engineering Lab , University of Salerno, Italy
2 University of Notre Dame, Notre Dame, USA
3 The College of William and Mary, Williamsburg, USA
2 University of Molise, Pesche (IS), Italiy

3. Imapct Analysis
Requirements tracing
Program Comprehension

4. Traceability information is still not
commonplace in software projects!

5. Information Retrieval

6. IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java

7. IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java

8. IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
Similarity
42%

9. IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
True Link

10. IR-based Traceability Recovery
Source_1 Target_2 95.4%
Source_3 Target_4 92.1%
Source_1 Target_1 85.6%
Source_2 Target_2 83.2%
Source_3 Target_3 81.2%
Source_1 Target_3 79.0%
Source_3 Target_2 77.5%
Source_2 Target_4 64.3%
Source_2 Target_3 53.2%
Target
Artifacts
IR Engine
Source
Artifacts
Candidate Links

11. Source_1 Target_2 95.4%
Source_3 Target_4 92.1%
Source_1 Target_1 85.6%
Source_2 Target_2 83.2%
Source_3 Target_3 81.2%
Source_1 Target_3 79.0%
Source_3 Target_2 77.5%
Source_2 Target_4 64.3%
Source_2 Target_3 53.2%
IR-based Traceability Recovery
Target
Artifacts
IR Engine
Source
Artifacts
Candidate Links

12. Structural Information and Traceability
Combining Textual and
Structural Analysis of Software
Artifacts for Traceability Link
Recovery - Collin McMillan, Denys
Poshyvanyk, Meghan Revelle
TEFSE 2009

13. /* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Structural Information and Traceability
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
linked

14. public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
linked
linked
Structural Information and Traceability

15. /* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
linked
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
Structural Information and Traceability

16. /* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
linked
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
Structural Information and Traceability
Structural dependecy

17. /* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
Structural Information and Traceability
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
linked
Transitivity

18. /* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
Structural Information and Traceability
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
linked
Transitivity

19. Open Issues
1) The choice of the bonus value is cricual
• Different systems require different bonus
• Different IR methods require different bonus

20. Open Issues
1) The choice of the bonus value is cricual
• Different systems require different bonus
• Different IR methods require different bonus
2) When applying the bonus?
. The transitivity property does not always hold

21. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUIDoctorData {
private jFrame window;
private jButton insert;
...
public GUIDoctorData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
Not
linked

22. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
Not
linked

23. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
public class Authorization{
public void setAuthorization(Doctor
pDoctor, Laboratory pLab){
...
}
...
}
Autorization.java
Not linked
Not
linked

24. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
LaboratoryAuthorization.java
Not linked
Not
linked

25. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
LaboratoryAuthorization.java
Not linked
Not
linked

26. Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
Laboratory.java
Not linked
linked
Structural information is not
always useful

27. Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_2 Class_3 53.2%
Code
Classes
IR Engine
Source
Artifacts
Candidate Links
Step 1: traditional IR process

28. Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ Bonus
Step 2: applying bonus to all the links

29. Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Step 2: applying bonus to all the links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ Bonus
+ Bonus

30. Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Step 2: applying bonus to all the links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ 2Bonus
+ Bonus
+ Bonus

31. Approach 1: Optimistic Combination (O-CSTI)
End?
YES
NO
Apply the bonus to all the links
Tradidional IR process
The user judges the i-th link
Re-order the list

32. Approach 2: User Driven Combination (U-CSTI)

33. Approach 2: User Driven Combination (U-CSTI)

34. Approach 2: User Driven Combination (U-CSTI)
Tradidional IR process
The user judges the i-th link
if correct
Apply bonus No bonus
Re-order the list
End?
YES
NO
YES
NO

35. Adaptive bonus
Sim(Source1, Class1) = Sim(Source1, Class1) + δ * Sim(Source1, Class1)

36. Adaptive bonus
Sim(Source1, Class1) = Sim(Source1, Class1) + δ * Sim(Source1, Class1)
yvariabilitlistranked
2
)min()max(



SimSim

Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_2 Class_3 53.2%
Candidate Links
max(Sim)
min(Sim)
variability

37. Implementation
We provide the experiments and datasets for download at
http://www.cs.wm.edu/semeru/data/csmr13/
TraceLab Components
1) Adaptive Bonus
2) Optimistic Comibination
(O-CSTI)
3) User Driven Combination
(U-CSTI)
4) Different IR Methods:
- Vector Space Model
- Jensen-Shannon

38. Implementation
We provide the experiments and datasets for download at
http://www.cs.wm.edu/semeru/data/csmr13/
TraceLab Components
1) Adaptive Bonus
2) Optimistic Comibination
(O-CSTI)
3) User Driven Combination
(U-CSTI)
4) Different IR Methods:
- Vector Space Model
- Jensen-Shannon
IR
method

39. Empirical Evaluation

40. Context
We compared three IR-based processes:
1) IR process alone
2) O-CSTI (optimistic combination)
3) U-CSTI (user driven combination)

41. Results
Vector Space Model

42. Results
Jensen-Shannon Divergence

43. Results
Tracing Use Cases onto Code Classes on SMOS
Vector Space Model

44. Results
Tracing Use Cases onto Code Classes on SMOS
Vector Space Model Jensen-Shannon Divergence

45. Optimality of the Adaptive Bonus
Adaptive Bonus vs Fixed Bonus
- We used different fixed bonus values

46. Optimality of the Adaptive Bonus
EasyClinic UC-CC with VSM EasyClinic TC-CC with JS
Adaptive Bonus vs Fixed Bonus
- We used different fixed bonus values

47. Conclusions