Presentation given at FOSDEM on an open-source library to generate, analyze and refactor Java code

1. JavaParser
A tool to generate, analyze, and refactor Java code
Federico Tomassetti founder of

2. Goal
JavaParser is evolving in a suite of libraries to support:
• Code generation
• Code analysis
• Code refactoring
It has a growing user-base but we are not sure on which usages
to focus:
Can you help us figure out
who could we serve better?

3. What JavaParser does?
JavaParser… parse Java code into a Java AST
package foo.bar;
class A {
int field;
}
CompilationUnit
PackageDecl ClassDecl
FieldDecl
PrimitiveType

4. What JavaParser does?
JavaParser unparse an AST into code
package foo.bar;
class A {
int field;
}
CompilationUnit
PackageDecl ClassDecl
FieldDecl
PrimitiveType

5. Why do you need symbol resolving?
int foo;
public void aMethod(int foo) {
foo = 1;
}
public void anotherMethod() {
foo = 1;
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the assignment of a thing named ”foo”, no idea what that thing is

6. public void print1(String foo) {
System.out.print(foo);
}
public void print2(int foo) {
System.out.print(foo);
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the call of a method named “print”, no idea which signature that
has
Why do you need symbol resolving?

7. class A { }
public void creator1() {
new A();
}
public void creator2() {
class A { }
new A();
}
To JP these two statements looks the same: they produce the same
AST nodes.
It is the instantiation of a class named “A”, no idea where it is defined
Why do you need symbol resolving?

8. What JavaSymbolSolver does?
JavaSymbolSolver resolves symbols in the JavaParser AST
package foo.bar;
class C {
D field;
}
package foo.bar;
class D {
}
CompilationUnit
PackageDecl ClassDecl
FieldDecl
ReferenceType

9. Relationship JP & JSS
Certain methods in the AST requires additional intelligence.
myCallExpression.calculateResolvedType();
myCallExpression.getName();
JSS not enabled JSS enabled
JSS not enabled JSS enabled

10. Configuring JavaSymbolSolver
A typical usage:
CombinedTypeSolver typeSolver = new CombinedTypeSolver(
new ReflectionTypeSolver(),
new JavaParserTypeSolver(new File("src/main/java")),
new JavaParserTypeSolver(new File("src/test/java")),
new JarTypeSolver("libs/guava.jar"),
new JarTypeSolver("libs/log4j.jar"));

11. Supporting code generation
We want to support people who has to generate code:
• Because they want to generate boilerplate code from
some source (e.g., a database schema)
• They want to build a transpiler targeting Java

12. JavaParser to generate code
CompilationUnit cu = new CompilationUnit();
cu.setPackageDeclaration("jpexample.model");
ClassOrInterfaceDeclaration book =
cu.addClass("Book");
book.addField("String", "title");
book.addField("Person", "author");

13. JavaParser to generate code
book.addConstructor(Modifier.PUBLIC)
.addParameter("String", "title")
.addParameter("Person", "author")
.setBody(new BlockStmt()
.addStatement(new ExpressionStmt(new AssignExpr(
new FieldAccessExpr(
new ThisExpr(), "title"),
new NameExpr("title"),
AssignExpr.Operator.ASSIGN)))
.addStatement(new ExpressionStmt(new AssignExpr(
new FieldAccessExpr(
new ThisExpr(), "author"),
new NameExpr("author"),
AssignExpr.Operator.ASSIGN))));

14. JavaParser to generate code
book.addMethod("getTitle", Modifier.PUBLIC).setBody(
new BlockStmt().addStatement(
new ReturnStmt(new NameExpr("title"))));
book.addMethod("getAuthor", Modifier.PUBLIC).setBody(
new BlockStmt().addStatement(
new ReturnStmt(new NameExpr("author"))));
System.out.println(cu.toString());

15. JavaParser to generate code
package jpexample.model;
public class Book {
String title;
Person author;
public Book(String title, Person author) {
this.title = title;
this.author = author;
}
public void getTitle() {
return title;
}
public void getAuthor() {
return author;
}
}

16. Supporting code analysis
We want to support people who have to analyse code:
• Because they want some metrics on code
• Because they want to ensure some quality standards are
met
• Because they want to write quick, one-off queries about
their codebase

17. JavaParser to run queries
Question: How many methods take more than 3 parameters?
long n = getNodes(allCus, MethodDeclaration.class)
.stream()
.filter(m -> m.getParameters().size() > 3)
.count();
System.out.println("N of methods with >3 params: "
+ n);
Answer: 11

18. JavaParser to run queries
Question: What are the three top classes with most methods?
Answer: CoreMatchers: 35 methods
BaseDescription: 13 methods
IsEqual: 9 methods
getNodes(allCus, ClassOrInterfaceDeclaration.class)
.stream()
.filter(c -> !c.isInterface())
.sorted(Comparator.comparingInt(o ->
-1 * o.getMethods().size()))
.limit(3)
.forEach(c ->
System.out.println(c.getNameAsString()
+ ": " + c.getMethods().size()
+ " methods"));

19. JavaParser to run queries
Question: What is the class with most ancestors?
Answer: org.hamcrest.core.StringContains: org.hamcrest.core.SubstringMatcher,
org.hamcrest.TypeSafeMatcher, org.hamcrest.BaseMatcher, org.hamcrest.Matcher,
org.hamcrest.SelfDescribing, java.lang.Object
ResolvedReferenceTypeDeclaration c = getNodes(allCus,
ClassOrInterfaceDeclaration.class)
.stream()
.filter(c -> !c.isInterface())
.map(c -> c.resolve())
.sorted(Comparator.comparingInt(o ->
-1 * o.getAllAncestors().size()))
.findFirst().get();
List<String> ancestorNames = c.getAllAncestors()
.stream()
.map(a -> a.getQualifiedName())
.collect(Collectors.toList());
System.out.println(c.getQualifiedName() + ": " +
String.join(", ", ancestorNames));
JSS at work here

20. Supporting code refactoring
We want to support people who has to refactor code:
• Because they have to modernize legacy code
• Because they want to update some dependencies and
the API of the libraries used changed
• Because they want to change some usage patterns

21. JavaParser for automated
refactoring
getNodes(allCus, MethodCallExpr.class)
.stream()
.filter(m -> m.resolveInvokedMethod()
.getQualifiedSignature()
.equals("foo.MyClass.oldMethod(java.lang.String,
int)"))
.forEach(m -> m.replace(replaceCallsToOldMethod(m)));
A new version of a library comes up and a deprecated method named
oldMethod is replaced by newMethod.
The new method takes 3 parameters: the first one as oldMethod but
inverted and the third one is a boolean, which we want to be always
true

22. JavaParser for automated
refactoring
public MethodCallExpr replaceCallsToOldMethod(
MethodCallExpr methodCall) {
MethodCallExpr newMethodCall = new MethodCallExpr(
methodCall.getScope().get(), "newMethod");
newMethodCall.addArgument(methodCall.getArgument(1));
newMethodCall.addArgument(methodCall.getArgument(0));
newMethodCall.addArgument(new BooleanLiteralExpr(true));
return newMethodCall;
}

23. Some other features

24. Comments attribution
void foo() {
// comment1
int a =
1 + 2; // comment2
}
// comment1
int a =
1 + 2;
CompilationUnit cu = JavaParser.parse(code);
ExpressionStmt expressionStmt =
cu.findFirst(ExpressionStmt.class).get() ;
System.out.println("Comment on the expression statement: "
+ expressionStmt.getComment().get().getContent());

25. Lexical preservation
Pretty printing is good for new code but for existing code users
want lexical preservation.
It is tricky because you need to know how to modify code
preserving the style and making the code correct.
Original code Change Final code
int a, b, c; remove b, adapt commas int a, c;
void foo() {
int a;
}
add new statement, indent
it as statement a previous
line
void foo() {
int a;
a = 0;
}

26. Things we are considering

27. JavaParser to identify patterns
We are working on the Matcher library to reduce the complexity, it is in the early
stages
This gives you a list of pairs name-type for all the properties in your bean.

28. Java templates
We are discussing the idea of Java templates.
We would like to have Java code with placeholders for variable parts
and be able to validate those templates.
class A extends ${BASE_CLASS:Id} {
private ${FIELD_TYPE:Type} myField;
}
class A extends ${BASE_CLASS:Id} {
${E:Expression} myField;
}
OK
KO

29. Making Java extensible
Researchers (but also a certain company…) would be interested in
writing extensions for Java based on JavaParser.
How can you add a new statement or expression to Java today? You
have to fork the parser. We have one user who forked JavaParser just
to change the grammar file.
We are looking for a better way.
Any suggestions?

30. JavaParser: Visited
Book on JavaParser and JavaSymbolSolver,
from the core committers.
Avalaible for 0+ $
Currently 1.000+ readers
https://leanpub.com/javaparservisited
Federico Tomassetti
See you on GitHub at
javaparser/javaparser