1. Convention-Based Syntactic Descriptions Ray Toal Derek Smith Loyola Marymount University CSIE 2009, Los Angeles 2009-04-02

3. Goals and Objectives

4. Motivation and Challenges

5. Approach

6. Conventions

7. Summary

8. Future Work

11. ... with some regex notation and custom extensions

14. Idea: Try to adapt convention over configuration to reduce markup requirements!

16. Microsyntax Example For a little C-like language: LETTER -> <L> DIGIT -> <Nd> CHAR -> [ˆ<Cc>&quot; | 'apos; [ˆ<Cc>] ID -> LETTER (LETTER | DIGIT | '_')* KEYWORD -> 'var' | 'fun' | 'read' | 'write' | 'while' | 'do' | 'end' NUMLIT -> DIGIT+ ('.' DIGIT+)? ([Ee] [+-]? DIGIT+)? STRLIT -> '&quot;' CHAR* '&quot;' SKIP -> <Zs> | #09 | #0A | #0D | '//' [ˆ#0A#0D]* [#0A#0D]

18. Rules must be non-recursive, and RHS can only use symbols from previous rules

21. Token set inferred

24. Category: <L>, <Nd>, <C>, <Zl>

25. String: 'while', ';'

26. One-of: [aeiou], [0-9A-Fa-f], [<L><Nd>_]

29. e 1 | e 2 or

30. e? optional

31. e* zero or more

32. e+ one or more

33. e^n exactly n e's

34. (…) grouping

35. Macrosyntax Example PROGRAM => BLOCK BLOCK => (DEC ';')* (STMT ';')+ DEC => 'var' ID ('=' EXP)? | 'fun' ID '(' IDLIST? ')' '=' EXP STMT => ID '=' EXP | 'read' IDLIST | 'write' EXPLIST | 'while' EXP 'do' BLOCK 'end' IDLIST => ID (',' ID)* EXPLIST => EXP (',' EXP)* EXP => TERM ([+-] TERM)* TERM => FACTOR ([*/] FACTOR)* FACTOR => NUMLIT | STRLIT | ID | CALL | '(' EXP ')' CALL => ID '(' EXPLIST? ')'

37. Means that SKIP* can appear before and after any token

38. Maximal munch assumed for tokenization

39. No delimiters needed between rules

40. First rule is the start symbol

41. Recursion is fine

42. Tokens can be introduced here, too!

43. Abstract Syntax var y; fun half(x) = x / 2; while x - (5 * x) do write half(10.4), x + 2; read x; end; (Program (Block (Var y) (Fun half x (/ (Ref x) (Numlit 2))) (While (- (Ref x) (* (Numlit 5) (Ref x))) (Block (Write (Call half (Numlit 10.4)) (+ (Ref x)(Numlit 2))) (Read x)))))

46. Last expression encountered is the “value”

47. Some rules don't even need AST expressions

48. Can have variables which are implicitly list valued, with special syntax for reassignment

49. Abstract Syntax PROGRAM => b:BLOCK {Program b} BLOCK => (d:DEC ';')* (s:STMT ';')+ {Block d s} DEC => 'var' i:ID ('=' e:EXP)? {Var i e} | 'fun' i:ID '(' p:IDLIST? ')' '=' e:EXP {Fun i p e} IDLIST => i:ID (',' i:ID)* The value of IDLIST is not an AST node; it's just a list since the last thing evaluated was stored in i Value of d is a list of values from each DEC Value of PROGRAM is an AST node with root 'Program; We're purposely using the same variable twice

50. Abstract Syntax EXP => t1:TERM (o=[+-] t2=TERM t1={o t1 t2})* TERM => f1:FACTOR (o=[*/] f2=FACTOR f1={o f1 f2})* FACTOR => n:NUMLIT {Numlit n} | s:STRLIT {Strlit s} | i:ID {Ref i} | c:CALL | '(' e:EXP ')' Each time we iterate through the ([*/] FACTOR)* syntax element, the vaues of the variables o and f1 are reassigned. Here o refers to the Variable because it Is lowercase It's okay that some of the alternatives produce AST nodes and some do not

52. Maximal munch assumed

53. SKIP is just another rule

54. Token set inferred

55. => implies SKIP* separators

56. AST variables lowercase; nodes capitalized

57. Value of abstract syntax object is value of last object encountered in L-R parse

59. C, JSON, other specifications on web (http://xlg.cs.lmu.edu/ssd/)

60. Tool development ongoing

62. Added custom features to EBNF

63. Defined conventions to simplify the notation

64. Provided examples of use

68. Ultimate convention: AST nodes automatically generated according to syntax category