Talk given at the 2011 International Conference on Program Comprehension in Kingston, ON, Canada.

1. A Lightweight Approach
to Uncover Technical
Information in
Unstructured Data
Nicolas Bettenburg, Bram Adams, Ahmed E. Hassan
Queen’s University, Kingston, ON
Michel Smidt
University of Bremen, Germany
1

2. 2

3. The code after "if callback.isAcceleratorInUse
(SWT.ALT | character ) )" inside Eclipse's
MenuManager.java removes the mnemonic, but it seems
like Eclipse should be checking isAcceleratorInUse.
2

4. “Our developers have a severe problem.”
3

5. “Our developers have a severe problem.”
NLP
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
3

6. “Our developers have a severe problem.”
NLP
pronoun, possessive
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
3

7. “Our developers have a severe problem.”
NLP
pronoun, possessive
[ Our_PRP$
noun, common, plural
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
3

8. “Our developers have a severe problem.”
NLP
pronoun, possessive
[ Our_PRP$
noun, common, plural
developers_NNS ])
<: have_VBP :> verb, present tense
([ a_DT determiner
severe_JJ adjective, ordinal
problem_NN ])._. noun, common, singular
3

9. Structured Text
4

10. NLP?
can’t deal with
the source code parts
PARSERS?
can’t deal with
the natural language parts
5

11. Past Solutions
infoZilla
Bettenburg et al. - MSR’08
• Based on heuristics
• Stack Traces
• Code snippets at block level
• Patches
6

12. Past Solutions
Miler
Bacchelli et al. - ICSE’10,
ICPC’10
• Based on heuristics
• Classifies lines as source code
• Classifies documents as containing code
• Finds class names
7

13. Past Solutions
infoZilla Miler
Bettenburg et al. - MSR’08 Bacchelli et al. - ICPC’10
• only specific kinds of technical information!
• new heuristics to extend (complex, error prone)
• for some kind of technical information, infeasible
8

14. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
9

15. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
9

16. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
Challenge!
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
9

17. Spelling and
Grammar Checkers
... are really good at finding “what’s not right“ in
natural language text!
10

18. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
11

19. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
12

20. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
Actual Spelling Mistakes!
13

21. Add Heuristics
14

22. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
14

23. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
H2: Programming Language Keywords
printf, fork, fi, ...
14

24. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
H2: Programming Language Keywords
printf, fork, fi, ...
H3: Special Characters
tree();
14

25. Evaluation
Manually annotated 20 complete Bug Reports and
Discussions from ECLIPSE.
Manually annotated 20 complete Email Discussions from
POSTGRESQL developers Mailing List.
15

26. Annotation GUI
2
1
3
16

27. Precision / Recall
T PS i
Precision(Si ) = T PSi +F PSi
T PS i
Recall(Si ) = T PSi +F NSi
TP = We annotated and tool annotated
FP = Tool annotated, we did not
FN = We annotated, tool did not
17

28. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
18

29. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
18

30. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
Hunspell used by OpenOffice and
Mozilla Suite
18

31. Comparison
Line-wise classification of source code
1 Launch the plugin as part of Eclipse IDE 3. Press Alt+H to
2 bring down the Help menu (to go along with our example in #1)
3
4 BUG: Notice "Software Updates" is missing its mnemonic.
5
6 public void update(String property) {
7 IContributionItem items[] = getItems();
8 for (int i = 0; i < items.length; i++) {
9 items[i].update(property);
10 }
11 }
12
13 Any status on this bug?
19

32. Comparison
Line-wise classification of source code
1 Launch the plugin as part of Eclipse IDE 3. Press Alt+H to
2 bring down the Help menu (to go along with our example in #1)
3
4 BUG: Notice "Software Updates" is missing its mnemonic.
5
6 public void update(String property) {
7 IContributionItem items[] = getItems();
8 for (int i = 0; i < items.length; i++) {
9 items[i].update(property);
10 }
11 }
12
13 Any status on this bug?
20

33. Comparison
Line-wise classification of source code
Precision Recall
Our approach 89.27% 86.46%
State-of-the-Art 66.13% 69.37%
21

34. Comparison
Line-wise classification of source code
Precision Recall
Our approach 89.27% 86.46%
State-of-the-Art 66.13% 69.37%
21

35. Summary
22

36. Summary
22

37. Summary
22

38. Summary
22

39. Summary
22

40. masayuki
reed
cjcypoi02 dietrich
steve.england corevette
steffen.wilberg
davemgarrett
mmortal03 timeless mano
fittysix
matspal
longsonr
zurtex
matti edilee
mconnor
cwwmozilla beltzner
dveditz
adelfino zeniko
kliu
alice0775
sziadeh mark.finkle robert.bugzilla
philringnalda
sgautherie.bz kev
faaborg
johnath
martijn.martijn
jmjeffery jo.hermans nrthomas gavin.sharp polidobj
m-wada
jbecerra jdarmochwal
john.p.baker jruderman mak77
ria.klaassen
VYV03354 cbook bomfog
dao
elmar.ludwig sdaugherty
vseerror
nightstalkerz l10n highmind63 twalker
mh+mozilla
klaas1988
ehsan stephen.donner
me.at.work
phiw
hskupin
ctalbert
tchung tomer
marcia timwi rotis
uliss
sylvain.pasche
bugzilla
marco.zehe cl-bugs-new2
tonglebeak
deletesoftware
abillings info
anselm.meyer
eddy_nigg
matt
RainerStroebel
samuel.sidler+old alex
hasham8888
aarobertxtr
manujsabarwal johnjbarton
myles7897
paulc
shaver
smichaud
mozilla
zhangchunlin dtownsend
jdaggett
kbrosnan
bzbarsky
sdwilsh
23

41. Internet Explorer reed
masayuki
dietrich
cjcypoi02
steve.england corevette
steffen.wilberg
davemgarrett
mmortal03 timeless mano
fittysix
matspal
longsonr
zurtex
matti edilee
mconnor
cwwmozilla beltzner
dveditz
adelfino zeniko
kliu
alice0775
sziadeh mark.finkle robert.bugzilla
philringnalda
sgautherie.bz kev
faaborg
johnath
martijn.martijn
nrthomas gavin.sharp
XML Parser
jmjeffery jo.hermans polidobj
m-wada
jbecerra jdarmochwal
john.p.baker jruderman mak77
ria.klaassen
VYV03354 cbook bomfog
dao
elmar.ludwig sdaugherty
vseerror
nightstalkerz l10n highmind63 twalker
mh+mozilla
klaas1988
ehsan stephen.donner
me.at.work
phiw
hskupin
ctalbert
tchung tomer
marcia timwi rotis
uliss
sylvain.pasche
bugzilla
JavaScript
marco.zehe cl-bugs-new2
UI
tonglebeak
deletesoftware
abillings info
Engine
anselm.meyer
eddy_nigg
matt
RainerStroebel
samuel.sidler+old alex
hasham8888
aarobertxtr
manujsabarwal johnjbarton
myles7897
paulc
shaver
smichaud
mozilla
zhangchunlin dtownsend
jdaggett
kbrosnan
bzbarsky
sdwilsh
23

42. 24