1. PEG.js
- Javascript Parser Generator -
Hidetomo Suzuki

2. 2013 / 5 / 30
Scope
Page 2
This MTG Target
Understand What’s PEG.js
Know How to Use PEG.js
Make New DSL and Parser

3. 2013 / 5 / 30
Agenda
Page 3
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

4. 2013 / 5 / 30
Agenda
Page 4
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

5. 2013 / 5 / 30 Page 5
1. How to make parser
1. Lexical Analysis(字句解析)
2. Syntactic Parsing(構文解析)
3. Semantic Analysis(意味解析)
4. Intermidiate Code Generation(中間コード生成)
5. Code Optimization(コード最適化)
6. Code Generation(コード生成）
General Step of Processing with Compiler
Parser

6. 2013 / 5 / 30 Page 6
1. How to make parser
1. Lexical Analysis(字句解析)
2. Syntactic Parsing(構文解析)
3. Semantic Analysis(意味解析)
Analysis for Make Parser
Make strings(minimum unit of string has semantic) from characters
Make tree structure from strings which result of Lexical Analysis
Type Check
※ A string of Lexical Analysis is called “Token”.

7. 2013 / 5 / 30 Page 7
1. How to make parser
1. Lexical Analysis
position_x = position_x + 2.0 * time
 Identifier : position_x
 Substitution Symbol : =
 Identifier : position_x
 Addition Symbol : +
 Number : 2.0
 Multiple Symbol : *
 Identifier : time

8. 2013 / 5 / 30 Page 8
1. How to make parser
2. Syntactic Parsing
 Identifier : position_x Substitution Symbol : =
 Identifier : position_x Addition Symbol : +
 Number : 2.0 Multiple Symbol : *
 Identifier : time
Substitution Symbol
Identifier
position_x
Expression
ExpressionExpression
Expression Expression
Identifier
position_x
Number
2.0
Identifier
time
=
+
*

9. 2013 / 5 / 30 Page 9
1. How to make parser
3. Semantic Analysis
Substitution Symbol
Identifier
position_x
Expression
ExpressionExpression
Expression Expression
Identifier
position_x
Number Identifier
=
+
 Correct Case : Real Number * Integer Number
 Wrong Case : Real Number * Function Pointer
2.0 time
*

10. 2013 / 5 / 30
Agenda
Page 10
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

11. 2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
 Parser Generator for JavaScript
 Use PEG
(Parsing Expression Grammar)
 Easy to try with the web page
Outline of PEG.js
http://pegjs.majda.cz/

12. 2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
PEG(Parsing Expression Grammar) is one of grammar for artificial language.
Focus point is how input will be analyzed.
What‘s PEG
Ex) Simple Expression Grammar which Has Four Arithmetic Operations
expression <- addexp
addexp <- multiexp (“+” multiexp / “-” multiexp)*
multiexp <- number (“*” number / “/” number)*
number <- [0-9]+
Accept : 2*3+6/2-5*3 -> multiexp + multiexp – multiexp -> ・・・
Decline : (1+1)*3+6/2-(7-2)*3 -> ?*3+6/2-?*3 -> can’t recognize…

13. 2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
• Nothing Confusion
• Different : “a b / a”, “a / a b”
• Easy to use compared with CFG
• Don’t need to implement
scanner (Lexical Analyzer)
• Don’t Express Left Recursive
PEG vs CFG(Context Free Grammar)
• Confusion Existing
• Same : “a b | a”, “a | a b”
• Need to implement scanner
• Can Express Left Recursive
PEG CFG

14. 2013 / 5 / 30 Page 14
2. What‘s PEG.js?
1. Make Grammar with PEG
2. Generate Parser with “pegjs” Command
3. Use the Parser Loaded as a Library
Step of Make Parser with PEG.js
vim filename.pegjs
Pegjs filename.pegjs parser.js
<script src=“./parser.js” type=“text/javascript”></script>
<script type=“text/javascript”>
parser.parse(“something”);

15. 2013 / 5 / 30 Page 15
2. What‘s PEG.js?
Install PEG.js
# Install Pythonbrew
$ curl -kL http://xrl.us/pythonbrewinstall | bash
$ vim ~/.bash_profile
[[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc
$ source ~/.bashrc
$ pythonbrew install 2.7.2
$ pythonbrew switch 2.7.2
# Install Node
$ git clone git://github.com/creationix/nvm.git ~/.nvm
$ nvm install v0.8.7
$ source ~/.nvm/nvm.sh
$ nvm use v0.8.7
$ vim ~/.npmrc
$ registry = http://registry.npmjs.org/

16. 2013 / 5 / 30 Page 16
2. What‘s PEG.js?
1. Make Grammar with PEG
2. Generate Parser with “pegjs” Command
3. Use the Parser Loaded as a Library
Let’s use PEG.js a little bit
$ vim ffirst.pegjs
start = addexp
addexp = integer ("+" integer / "-" integer)*
integer = [0-9]+
$ ./node_modules/pegjs/bin/pegjs first.pegjs parser.js
$ vim parser.js
1st line: module.export = … -> var parser = …
$ vim index.html
<script type=“text/javascript” src=“./parser.js”></script>
<script type=“text/javascript”>
alert(parser.parse(“1+2+3-5”));
</script>

17. 2013 / 5 / 30
Agenda
Page 17
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

18. 2013 / 5 / 30 Page 18
3.Try to used to be PEG
PEG.js Online Version (http://pegjs.majda.cz/online)

19. 2013 / 5 / 30 Page 19
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : integer)
Grammar
Test Input
Test Input Result
start = integer
integer = digits:[0-9]+ { return parseInt(digits.join(“”), 10); }
123456789
123456789

20. 2013 / 5 / 30 Page 20
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : add two integer)
Grammar
Test Input
Test Input Result
start = target1:integer "+" target2:integer { return target1+target2; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
81+19
100

21. 2013 / 5 / 30 Page 21
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : add and minus two integer)
Grammar
Test Input
Test Input Result
start
= target1:integer "+" target2:integer { return target1+target2; }
/ target1:integer "-" target2:integer { return target1-target2; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
100-19
81

22. 2013 / 5 / 30 Page 22
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : add and minus integers)
Grammar
Test Input
Test Input Result
start = expression
expression
= ope1:integer "+" ope2:expression { return ope1+ope2; }
/ ope1:integer "-" ope2:expression { return ope1-ope2; }
/ ope:integer { return ope; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
1+2+3+4+5-6+7-8+9-10
9

23. 2013 / 5 / 30 Page 23
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : four arithmetic)
Grammar
Test Input
Test Input Result
start = expression
expression
= ope1:integer "+" ope2:expression { return ope1+ope2; }
/ ope1:integer "-" ope2:expression { return ope1-ope2; }
/ ope1:integer "*" ope2:expression { return ope1*ope2; }
/ ope1:integer "/" ope2:expression { return ope1/ope2; }
/ ope:integer { return ope; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
2*3/2
3

24. 2013 / 5 / 30 Page 24
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : four arithmetic with priority)
Grammar
Test Input
Test Input Result
start = expression
expression = ope1:multiple "+" ope2:expression { return ope1+ope2; }
/ ope1:multiple "-" ope2:expression { return ope1-ope2; }
/ ope:multiple { return ope; }
multiple = ope1:integer "*" ope2:multiple { return ope1*ope2; }
/ ope1:integer "/" ope2:multiple { return ope1/ope2; }
/ ope:integer { return ope; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
2*3/2+8/4
5

25. 2013 / 5 / 30 Page 25
3. Try to used to be PEG
Very Simple PEG.js Grammar(calc : four arithmetic with priority)
Grammar
Test Input
Test Input Result
start = expression
expression = ope1:multiple "+" ope2:expression { return ope1+ope2; }
/ ope1:multiple "-" ope2:expression { return ope1-ope2; }
/ ope:multiple { return ope; }
multiple = ope1:bracket "*" ope2:multiple { return ope1*ope2; }
/ ope1:bracket "/" ope2:multiple { return ope1/ope2; }
/ ope:bracket { return ope; }
bracket = "(" exp:expression ")" {return exp; }
/ ope:integer {return ope; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
(7+3)*3/2+8/(6-2)
17

26. 2013 / 5 / 30
Agenda
Page 26
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

27. 2013 / 5 / 30 Page 27
4. Let‘s make DSL
Config File Format
Parser Result
Web Page Image
Target
<LogicalExpression?>Y---[function param]
N---[function param]
if (isLogicalExpression) function(param);
else function(param);
Ex)
<Hydea?>Y---[show ‘hydea’]
N---[show ‘hydeb’]
Ex)
if (isHydea) show(‘hydea’);
else show(‘hydeb’);
URL: ex.com/?p=hydea URL: ex.com/?p=hydeb

28. 2013 / 5 / 30 Page 28
4. Let‘s make DSL
Just write over specification
Make Grammar
start = "<" logic "?>Y---[" function " " param
"]nN---[" function " " param "]"
logic = [a-zA-Z]+
function = [a-zA-Z]+
param = "'" [a-zA-Z]+ "'"

29. 2013 / 5 / 30 Page 29
4. Let‘s make DSL
Refactoring a little bit
Make Grammar
start = "<" identifier "?>Y---[" identifier " " param
"]nN---[" identifier " " param "]"
param = "'" identifier "'"
identifier = [a-zA-Z]+

30. 2013 / 5 / 30 Page 30
4. Let‘s make DSL
Finish to make param and identifier part
Make Grammar
start = "<" identifier "?>Y---[" identifier " " param
"]nN---[" identifier " " param "]"
param
= "'" id:identifier "'"
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+
{return chars.join(""); }

31. 2013 / 5 / 30 Page 31
4. Let‘s make DSL
Organize Function Part
Make Grammar
start = "<" identifier "?>Y---[" function
"]nN---[" function "]"
function
= id:identifier " " param:param
{return id + "(" + param + ")"; }
param
= "'" id:identifier "'"
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+
{return chars.join(""); }

32. 2013 / 5 / 30 Page 32
4. Let‘s make DSL
Finish to make
Make Grammar
start
= "<" id:identifier "?>Y---[" f1:function
"]nN---[" f2:function "]"
{return "if (is" + id + ") " + f1 + ";nelse " + f2 + ";"; }
function
= id:identifier " " param:param
{return id + "(" + param + ")"; }
param
= "'" id:identifier "'"
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+
{return chars.join(""); }

33. 2013 / 5 / 30
Agenda
Page 33
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

34. 2013 / 5 / 30 Page 34
5. Application
Show My Demo(Demo’s grammar)
start = tree
tree = stat:statement? { return stat; }
statement = stat:if_statement { return stat.toString(); }
/ stat:proc_statement { return stat.toString(); }
if_statement = fac:if_factor 'Y' edge branch1:proc_statement 'n'
'N' 'n' edge branch2:statement
{ return branch2.toString().match(/if/) ?
"if (" + fac + ")nt" + branch1 + 'nelse ' + branch2
: 'if (' + fac + ')nt' + branch1 + 'nelsent' + branch2;}
proc_statement = fac1:proc_factor [-|]+ fac2:proc_statement { return fac1 + '.then(' + fac2.slice(0,fac2.length-1) + ');'; }
/ factor:proc_factor { return factor + ';'; }
proc_factor = '[' procexp:expression ']' { return procexp; }
if_factor = '<' ifexp:expression '>' { return ifexp; }
expression = elem:element ' ' cdr:arguments { return elem.toString() + '(' +cdr + ')'; }
/ elem:element { return elem.toString(); }
arguments = elem:element ',' arg:arguments { return elem.toString() + ',' + arg; }
/ elem:element { return elem.toString(); }
element
= characters:[a-zA-Z0-9-_."]+ { return characters.join('') }
edge
= symbols:[-|n]+ { return symbols.join('') }

35. 2013 / 5 / 30 Page 35
5. Application
Recommendation
Conf1 Conf2 Conf3 ・・・
Recommendation
Engine
Choose a config file

36. 2013 / 5 / 30
Reference
Page 36
Famous Book for Compiler

37. 2013 / 5 / 30
Thank you
Page 37
Thank you for your listening