An Empirical Investigation into the Role of API-Level Refactoring during Software Evolution, Miryung Kim, Dongxiang Cai, Sunghun Kim, ICSE' 11: Proceedings of the 2011 ACM and IEEE 33rd International Conference on Software Engineering
2. It is believed that refactoring improves
software quality and maintainability.
The role of refactoring has not been
systematically investigated using fine‐
grained revision history at the method level.
3. We studied the relationships between API‐level
refactorings, bug fixes, the time taken to fix
bugs, and release cycles.
• The number of bug‐fixes increases after API‐level
refactorings.
• A large number of refactoring revisions include
bug fixes or are related to later fixes.
• Bug fix time tends to be shorter after than before.
• API‐level refactorings occur more frequently
before major releases than after.
5. Refactoring improves software quality and
maintainability [Opdyke, Griswold, Mens, etc.]
A lack of refactoring incurs technical debt.
[Cunningham, Lehman]
Refactor mercilessly [Beck, eXtreme
Programming]
Productivity improves after re‐architecting a
system [Carriere et al.]
Defect density decreases after refactoring
[Ratzinger et al.]
6. The number of bug reports increases after
refactorings [Weißgerber & Diehl]
Incomplete or inconsistent refactorings cause
bugs [Görg and Weißgeber]
Code churns are correlated with defect
density [Nagappan & Ball]
9. Q1: Are there more bug fixes after API‐level
refactorings?
Q2: Are API‐level refactorings correlated to
bug fixes?
Q3: Do API‐level refactorings reduce the time
taken to fix bugs?
Q4: Are there relatively fewer API‐level
refactorings before release dates?
10. API‐level
Refactoring Fix Revision
[ICSE 2007,
Reconstruction S. Kim et al.]
[ICSE 2007, M. Kim et al.]
Identify Refactoring Identify
Revisions Fix Revisions
Introducing
Change Distilling
Change [TSE 2007,
[MSR 2005, B. Fluri et al.]
Sliwerski et al.]
Identify Bug Life Time Map line‐level changes
to methods
12. API‐level
Refactoring Rename, move, and signature changes
Reconstruction at or above the method header level
[ICSE 2007, M. Kim et al.]
Foo.mA(int)
Foo.mA()
Foo.mB(int)
Foo.mB()
Foo.mC()
Foo.mC()
Bar.mA(long)
Boo.mA(long)
Baz.mA(int)
Boo.mB(long)
Baz.mB(int)
15. Fix Revision Use heuristics to identify bug fix revisions
[ICSE 2007,
e.g., [bug] [id #], [fixed]
S. Kim et al.]
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix
16. Introducing
Change Use SVN blame (diff) to track down
[MSR 2005, the changes that led to fixes.
Sliwerski et al.]
Bug‐introducing changes
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix
18. Change Distilling Map line‐level location to a method level location
[TSE 2007, Filter out meaningless changes.
B. Fluri et al.]
r1 r2 r3 r4 r5 r6 r7 r8
Ref Fix Ref/Fix Fix
19. Eclipse JDT jEdit Columba
core
Type IDE Text editor Email client
Period June 5th, 2001 to Sep 2, 2001 to July 9th, 2006 to
May 16th 2007 Apr, 2009 Jan 31st 2010
# revisions 15000 11102 421
# of API‐level 6755 3557 424
refactorings
# of bug fix 3752 1073 150
revisions
# of refactoring 1089 423 36
revisions
20. GT source # Prec. recall
API 14 API‐level refactoring detection 14 0.93 0.93
Refactoring [M. Kim et al. ]
revision
Inspection of change logs only 2 0.50 0.07
Inspection of bug reports only 5 0.75 0.21
Fix 62 Fix‐revision detection [S. Kim 49 0.96 0.76
Revision et al.]
Inspection of change logs only 55 0.92 0.81
Inspection of bug reports only 49 0.94 0.74
Automated techniques vs. Manual Inspection
26. Study Method 1: We measure a fix rate (# fix
revisions / K) within K sliding window after
each refactoring revision. K revision windows
Ref Ref Fix Ref Fix Fix Ref Fix
Study Method 2: We measure a fix rate at
the method level
M.foo()=>M.bar()
Ref Fix Fix
28. Results: fix rates at the method level also
increase after refactorings
Fix rate at a method level
0.40% 1.00% 8.00%
0.30% 0.80% 6.00%
0.60%
0.20% 4.00%
0.40%
0.10% 0.20% 2.00%
0.00% 0.00% 0.00%
‐30 ‐20 ‐10 0 10 20 30 ‐30 ‐20 ‐10 0 10 20 30 ‐30 ‐20 ‐10 0 10 20 30
Refactoring timing
JDT jEdit Columba
There is a short term increase in the number of bug fixes
after refactorings
29. 20 revision windows
Ref Fix Fix
In Eclipse, 208 refactoring revisions are followed by at least
one bug fix to the same method location within 20 revisions.
30. 20 revision windows
Ref Fix Fix
We randomly sampled 50 revision pairs and manually
inspected them.
31. R F
Types Frequency
1. To fix several related bugs, a developer first refactors 6%
code.
2. Incorrect refactorings cause a bug, inducing a later fix. 6%
3. After a developer performs refactorings, she discovers a 2%
new bug and fixes it.
4 Refactoring with a bug fix is incomplete. A developer 50%
applies a supplementary fix later.
5. A bug fix happened after a refactoring in the same 36%
location, but they are not related.
34. Probability of fixes and refactorings at the revision level
Project P(F) P(R) P(R|F) P(R|not F) P(F|R) P(F|not R)
Eclipse 25% 7.3% 12% 5.7% 41.5% 23.7%
JDT
JEdit 9.7% 3.8% 11.5% 3.0% 29.1% 8.9%
Columba 35.7% 8.6% 10.7% 7.4% 44.4% 34.8%
Refactorings and bug fixes often occur together in the same
revisions.
35. Results:
# of Ref‐revisions that were # of Non‐ref revisions that were
related to fix within (0,K] related to fix within (0,K]
2245
(14.0%)
495 (32.0%)
1053 13847 (86.0%)
(68.0%)
K=20
Refactoring revisions are followed by related bug fixes more
often than non‐refactoring revisions.
41. Results:
average fix
K=100
time (days)
25 120 250 230.2
19.7 98.6
20 100 200
80 149.6
15 150
60 42.6
10 7.3 100
5.6 40
4.4 21.5
12.2 38.4 48.4
5 20 50
0 0 0
BB AA XB BA BB AA XB BA BB AA XB BA
JDT jEdit Columba
When it comes to fixing bugs introduced near the time of
refactorings, the average fix time tends to decrease after
refactorings
43. Results
30% Eclipse JDT
25%
refactoring rate
20%
15%
10%
5%
0%
K=‐20 K=20 K=‐30 K=30 K=‐40 K=40
window size (K)
There are more refactorings prior to major version releases.
44. Study intra‐method refactorings and
complex refactorings (e.g., use RefFinder
[Prete et al.])
Account for the size of edits and the number
of refactorings
Phase & activity level
Micro‐commit practices
45. API‐level refactoring and bug fixes are
correlated temporally and spatially.
API‐level refactoring serves the role of both
facilitating bug fixes and introducing bugs.
API‐level refactoring occurs as a part of
behavior correcting edits
Refactoring engines must support safe application of
refactorings and semantic changes together.
The result calls for an in‐depth study on the cost and
benefit of refactorings.