How to Write Language “Compiler”

1. How to Write Language “Compiler”
Philip Zhong
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1

2. • Language Compilers
• JAVACC
• SQL Parser
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

3. • ANTLR
• YACC
• JAVACC
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3

4. • Another Tool for Language Recognition
• Java/C++/C/C#/Python/Ruby/object C
• BSD
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4

5. • Yet Another Compiler Compile
• C++/C for Unix
• BSD
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5

6. • Java Compiler Compile
• Java
• BSD
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6

7. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7

8. "n" newline
* zero or more copies of the preceding expression
+ one or more copies of the preceding expression
? zero or one copy of the preceding expression
| or
[] optional
˜[] matches any single character that is not in the empty set
() must appear
EOF end of line
"a"-"z" any letter, from a to z
"0" - "9" any numeric
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8

9. • Options
• Program header
• Tokens
• Production
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

10. options {
JDK_VERSION = "1.6";
IGNORE_CASE=true ;
JAVA_UNICODE_ESCAPE = true;
UNICODE_INPUT=true;
DEBUG_PARSER=false ;
STATIC = false;
}
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10

11. PARSER_BEGIN(SqlParser)
package com.webex.wddl.engine.parser.sql;
public class SqlParser implements Parser
{
final public void setStatement(String sqlStatement) {
InputStream stream = new ByteArrayInputStream(sqlStatement.getBytes());
...
public SqlParser()
{
}
}
PARSER_END(SqlParser)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11

12. • TOKEN: The regular expressions in this regular expression
production describe tokens in the grammar.
• SPECIAL_TOKEN: The regular expressions in this regular
expression production describe special tokens.
• SKIP: Matches to regular expressions in this regular expression
production are simply skipped (ignored) by the token manager.
• MORE: Sometimes it is useful to gradually build up a token to be
passed on to the parser. Matches to this kind of regular
expression are stored in a buffer until the next TOKEN or
SPECIAL_TOKEN match.
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12

13. TOKEN:
{
<X_AND:"AND">
| <X_FROM:"FROM">
| <X_IN:"IN">
| <X_LIKE:"LIKE">
| <X_SELECT:"SELECT">
| <X_WHERE:"WHERE">
...
}
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13

14. SPECIAL_TOKEN:
{
<LINE_COMMENT: "--"(~["r","n"])*>
| <MULTI_LINE_COMMENT: "/*" (~["*"])* "*" ("*" | (~["*","/"] (~["*"])*
"*"))* "/">
}
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14

15. SKIP:
{
""
| "t"
| "r"
| "n"
}
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15

16. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16

17. Statement parse(String SQL):
...
{
...
(
statement = insert()
|
statement = merge()
...
|
statement = select()
)(<EOF>|";")
...
}
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17

18. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18

19. • Define tokens
• Define parser tree classes
• Write parser logic
• Create parser classes
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19

20. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20

21. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21

22. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22

23. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23

24. Thank you.
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24