Paper: An Evaluation of the Strategies of Sorting, Filtering, and Grouping API Methods for Code Completion Authors: Daqing Hou and Dave Pletcher Session: Research Track Session 8 -Program Comprehension

1. Evaluating Strategies of Sorting,
Filtering, and Grouping API Methods
for Code Completion
Daqing Hou, David M. Pletcher
Clarkson University, Potsdam NY
1
Wednesday, September 28, 2011 1

2. Evaluating Strategies of Sorting,
Filtering, and Grouping API Methods
for Code Completion
2
Wednesday, September 28, 2011 2

3. Information Overloading
• Large APIs
ex: JDK 1.6, 3,777 classes and interfaces
• Learning barriers [Ko et al. 2004]
- design
- selection
- coordination & use
- information & understand
3
Wednesday, September 28, 2011 3

4. Evaluating Strategies of Sorting,
Filtering, and Grouping API Methods
for Code Completion
4
Wednesday, September 28, 2011 4

5. Code Completion as Antidote
current
• In Eclipse, completion proposals sorted
alphabetically or by-relevance.
5
Wednesday, September 28, 2011 5

6. Evaluating Strategies of Sorting,
Filtering, and Grouping API Methods
for Code Completion
BCC: Better Code Completion
6
Wednesday, September 28, 2011 6

7. Code Completion as Antidote
new - 2 ways of sorting
• type-based sorting
7
Wednesday, September 28, 2011 7

8. Code Completion as Antidote
new - 2 ways of sorting
• use count-based sorting
8
Wednesday, September 28, 2011 8

9. Code Completion as Antidote
new - filtering
1. public, non-API methods
ex: javax.swing.JComponent.updateUI()
2. Visible only for limited receiver types
ex: JPanel.add(), but not JButton.add()
3. Visible in subclass only
ex: java.awt.Component.paint()
9
Wednesday, September 28, 2011 9

10. Code Completion as Antidote
new - grouping
• Display logically related methods together
ex: add()/getComponent()/remove() for
a Container
• Filtering and grouping rules for each API
need to be specified only once.
10
Wednesday, September 28, 2011 10

11. Evaluating Strategies of Sorting,
Filtering, and Grouping API Methods
for Code Completion
11
Wednesday, September 28, 2011 11

12. Evaluation Methodology
• Compare 14 configurations with Eclipse’s by-
relevance as baseline in terms of rank reduction.
- rank reduction (7)
- primer (2)
- structuring (5)
• 9 open-source projects
• Metric
- rank for an API method
- project-wide rank
12
Wednesday, September 28, 2011 12

13. Evaluation Result 1: Rank Reduction
7 configurations ordered by rank reduction!
Best: regular dynamic ranking!
13
Wednesday, September 28, 2011 13

14. Evaluation Result 2: Primer
7/9 dynamic primers worse than regular dynamic ranking!
adjusted primers mixed results!
14
Wednesday, September 28, 2011 14

15. Evaluation Result 3: Structuring
• Two mechanisms: grouping and static primers.
• Regular dynamic ranking+grouping is the best
configuration for grouping.
• 8/9 static primers are better than type+filtering.
• Static primers worse than regular dynamic ranking
+grouping, but can still be useful.
15
Wednesday, September 28, 2011 15

16. Design Recommendations
• Regular dynamic ranking and adjusted primers for
rank reduction.
• Regular dynamic ranking+grouping for additional
benefit of API learning.
• Static primers worse than regular dynamic ranking
+grouping, but can still be useful.
• Provide multiple options for developers choose
from.
16
Wednesday, September 28, 2011 16

17. Conclusion and future work
• Implemented Code Completion strategies for
moving forward needed API methods, hiding non-
API public methods, and grouping API methods
according to their semantic relatedness.
• Evaluated 14 configurations of these strategies on
9 projects that use AWT/Swing.
• Gave design recommendations.
• Future work includes user studies for features such as
grouping, and ways to make primers work.
• Contact me for BCC source code.
17
Wednesday, September 28, 2011 17