We revisit the performance of template-based APR to build comprehensive knowledge about the effectiveness of fix patterns, and to highlight the importance of complementary steps such as fault localization or donor code retrieval. To that end, we first investigate the literature to collect, summarize and label recurrently-used fix patterns. Based on the investigation, we build TBar, a straightforward APR tool that systematically attempts to apply these fix patterns to program bugs. We thoroughly evaluate TBar on the Defects4J benchmark. In particular, we assess the actual qualitative and quantitative diversity of fix patterns, as well as their effectiveness in yielding plausible or correct patches. Eventually, we find that, assuming a perfect fault localization, TBar correctly/plausibly fixes 74/101 bugs. Replicating a standard and practical pipeline of APR assessment, we demonstrate that TBar correctly fixes 43 bugs from Defects4J, an unprecedented performance in the literature (including all approaches, i.e., template-based, stochastic mutation-based or synthesis-based APR).

1. TBar: Revisiting Template-based
Automated Program Repair
Kui LIU, Anil Koyuncu, Dongsun Kim and Tegawendé F. Bissyandé
SnT, University of Luxembourg, Luxembourg
2019-07-17, Wednesday @ISSTA 2019, Beijing, China

2. 1
> Fix Pattern
Fix pattern: e.g., PAR, kim2013ICSE [23].
Fix template: e.g., SOFix, liu2018SANER [40].
Abstract modification: e.g., SimFix, jiang2018ISSTA [18].
Mutation operator: e.g., CapGen, wen2018ICSE [69].
Modification rule: e.g., ssFix, xin2017ASE [74].
Code transform: e.g., Genesis, long2017FSE [41].
Program transformation schemas: e.g., SketchFix, hua2018ICSE[17].
“A fix pattern is a pair of a code context representing a buggy code entity
and a set of related change operations, which can be applied to a given
buggy code entity to generate fixing code.” ---- Liu et al. TSE 2018 [35].
FP: <Code Context, {Change Operations}>

3. 2
> Limited Characterization of Fix Patterns
APR Tool Venue Number of Fix Patterns
PAR [23] ICSE-2013 10
jMutRepair [40] ISSTA-2016 2
HDRepair [29] SANER-2016 11
NPEfix [13] SANER-2017 9
ssFix [74] ASE-2017 34
Genesis [41] FSE-2017 108
S3 [25] ASE-2017 4
ELIXIR [62] ASE-2017 8
SketchFix [17] ICSE-2018 6
SOFix [40] SANER-2018 12
SimFix [18] ISSTA-2018 16
CapGen [69] ICSE-2018 30
AVATAR [39] SANER-2019 13
1.Diversity of fix
patterns.
2.Bug fix performance
of each fix pattern.
3.Fault localization
sensitiveness of fix
patterns.

4. 3
> Study Methodology
1. SLR: Systematically review the APR literature related to fix patterns.
2. Taxonomy: enumerate and summarize fix patterns.
3. A Baseline Repair Tool: implement a comprehensive pattern based APR tool.
4. Empirical Study on Performance and Sensitiveness.

5. 4
SLR and Taxonomy

6. 5
> Fix Pattern Inference in the Literature
1. Manual Summarization.
2. Automated Mining.
3. Pre-definition.
4. Statistics of Actions.
1) Pan et al. EMSE-2009, 27 fix patterns, 5 Java projects.
2) Kim et al. ICSE-2013, 10 fix patterns, 62,656 patches.
1) Long et al. FSE-2017, 108 fix patterns for 3 kinds of defects.
2) Liu and Zhong, SANER-2018, 12 fix patterns, Stack Overflow.
1) Saha et al. ASE-2017, 11 fix patterns, on the top of PAR.
2) Hua et al. ICSE-2018, 6 fix patterns.
1) Wen et al. ICSE-2018, top-30 frequent code change actions.
2) Jiang et al. ISSTA-2018, top-16 frequent code change actions.

7. 6
> Fixing a Bug with a Fix Pattern
--- a/source/org/jfree/chart/renderer/category/AbstractCategoryItemRenderer.java
+++ b/source/org/jfree/chart/renderer/category/AbstractCategoryItemRenderer.java
@ -1796,4 +1796,4 @
CategoryDataset dataset = this.plot.getDataset(index);
- if (dataset != null) {
+ if (dataset == null) {
return result;
}
A Defects4J Bug Chart-1.
Action Type: Update,
Granularity: Expression.
Spread: Single statement.
Fix Pattern: Mutate Operator.

8. 7
> Fix Pattern Taxonomy
FP1. Insert Cast Checker.
FP2. Insert Null Pointer Checker.
FP3. Insert Range Checker.
FP4. Insert Missed Statement.
FP5. Mutate Class Instance Creation.
FP6. Mutate Conditional Expression.
FP7. Mutate Data Type.
FP8. Mutate Integer Division Operation.
FP9. Mutate Literal Expression.
FP10. Mutate Method Invocation Expression.
FP11. Mutate Operator.
FP12. Mutate Return Statement.
FP13. Mutate Variable.
FP14. Move Statement.
FP15. Remove Statement.
15 Fix Pattern Categories  35 fix patterns.

9. 8
> Change Properties of Fix Patterns
Action Type # fix patterns
Update 17
Delete 4
Insert 13
Move 1
Granularity # fix patterns
Expression 21
Statement 17
Method 1
Spread # fix patterns
Single-
statement
30
Multi-
statement
7

10. 9
TBar: a Baseline APR System

11. 10
> TBar: Template-Based automated program repair
Overall Workflow of TBar.
GZoltar and Ochiai

12. 11
> Fix Pattern Selection
Selected
fix pattern
A ranked
list of
suspicious
statements
Fix pattern
data base
Code AST
…
// Buggy code of Closure-13 in Defects4J dataset.
if(options.dependencyOptions.needsManagement() &&
!options.skipAllPasses &&
options.closurePass) {
// AST.
IfStatement
---InfixExpression
------MethodInvocation
--------- ……
------Operator
------PrefixExpression
--------- ……
------Operator
------QualifiedName
--------- ……
FP6. Mutate Conditional Expression.
FP6.1: - ...condExp1...
+ ...condExp2...
FP6.2: - ...condExp1 Op condExp2...
+ ...condExp1...
FP6.3: - ...condExp1...
+ ...condExp1 Op condExp2...
Breadth First Search

13. 12
> Patch Generation
// Buggy code of Closure-13 in Defects4J dataset.
if(options.dependencyOptions.needsManagement() &&
!options.skipAllPasses &&
options.closurePass) {
// Patch Candidate I.
- if(options.dependencyOptions.needsManagement() &&
- !options.skipAllPasses &&
+ if(!options.skipAllPasses &&
options.closurePass) {
// Patch Candidate II.
if (options.dependencyOptions.needsManagement() &&
- !options.skipAllPasses &&
options.closurePass) {
// Patch Candidate III.
if (options.dependencyOptions.needsManagement() &&
- !options.skipAllPasses &&
- options.closurePass) {
+ !options.skipAllPasses) {
FP6.2: - ...condExp1 Op condExp2...
+ ...condExp1...
Patch CandidatesSelected
fix pattern
Mutate
suspicious code
FP6. Mutate Conditional Expression.
FP6.1: - ...condExp1...
+ ...condExp2...
FP6.2: - ...condExp1 Op condExp2...
+ ...condExp1...
FP6.3: - ...condExp1...
+ ...condExp1 Op condExp2...

14. 13
> Assessment Benchmark
Project Chart
(C)
Closure
(Cl)
Lang
(L)
Math
(M)
Mockito
(Mc)
Time
(T)
Total
# bugs 26 133 65 106 38 27 395
# test cases 2,205 7,927 2,245 3,602 1,457 4,130 21,566
Benchmark Defects4J dataset information

15. 14
Repair Suitability of Fix Patterns

16. 15
> Experiment #1： TBar with Perfect Fault Localization
Research Questions :
RQ1. How many bugs from Defects4J can be correctly fixed by
fix patterns from our taxonomy?
RQ2. Can each Defects4J bug be fixed by different fix patterns?
RQ3. What are the properties of fix patterns that are successfully
used to fix bugs?

17. 16
> RQ1: Number of Fixed Bugs
Project Chart
(C)
Closure
(Cl)
Lang
(L)
Math
(M)
Mockito
(Mc)
Time
(T)
Total
# fully
fixed bugs
12/13 20/26 13/18 23/25 3/3 3/6 74/101
# partially
fixed bugs
2/4 3/6 1/4 0/4 0/0 1/1 7/20
Number of bugs fixed by fix patterns in TBar with perfect fault localization

18. 17
> Limitation and Discussion
1. Insufficient fix patterns.
2. Less effective search of fix ingredients.
public String generateToolTipFragment(String toolTipText) {
- return " title="" + toolTipText
+ return " title="" + ImageMapUtilities.htmlEscape(toolTipText)
+ "" alt=""";
}
Matchable fix pattern: FP9.2.
Donor code searching
in “local” file

19. 18
> RQ2: Number of Fix Patterns vs. Fixing Bugs
28
9
2
1
38
10
5
3
10
4
4
3
1
1
1
1
1
2
3
4
5
>5#FIX
PATTERNS
P #1 #2 #3 #4
It is important to select appropriate fix patterns for different bugs.

20. 19
> RQ3: Fix Pattern Properties
85
49
24
4
43
32
18
2
Update Insert Delete Move
#fixedbugs
122
38
2
54
39
2
Expression Statement Method
140
22
71
24
Single Multiple
Plausibly BUT Incorrectly fixed bugs. Correctly fixed bugs.
a. Change Actions b. Change Granularity c. Changed Spread
Repair performance is different among fix patterns properties.

21. 20
Repair Performance Comparison

22. 21
> Experiment #2: TBar with Normal Fault Localization
Research Questions:
RQ4. What performance can be achieved by TBar in a standard
and practical repair scenario?
RQ5. To what extent are the fix pattern based repair tool sensitive
to noise in fault localization (i.e., spotting bug positions)?

23. 22
> RQ4: Performance Comparison with the State-of-the-Art
TBar outperforms all recent state-of-the-art APR tools
that were evaluated on the Defects4J dataset.

24. 23
> RQ5: Fault Localization Sensitiveness
Fault localization noise has a significant impact on
the performance of TBar.
Plausibly but incorrectly fixed bugs
24 1514
1
Fully fixed Partially fixed
#fixed
bugs
incorrect position
correct position

25. 24
> DISCUSSION
1. Fix pattern diversity.
2. Fix pattern selection.
3. Donor code searching.
4. Patch prioritization.
5. Fault Localization Sensitiveness.

26. 25
> SUMMARY
10
> TBar: Template-Based automated program repair
Overall Workflow of TBar.
GZoltar and Ochiai
23
> Standard Practical Performance
https://github.com/SerVal-DTF/TBar