SlideShare ist ein Scribd-Unternehmen logo
1 von 67
Of Rats And
                                Dragons
                               Achieving Parsing Sanity



                                     Sean Cribbs

                                   Web Consultant
                                Ruby and Erlang Hacker

Saturday, September 12, 2009
Quick Review
Saturday, September 12, 2009
context-free
                                grammars


Saturday, September 12, 2009
G = (V, Σ, R, S)



Saturday, September 12, 2009
V→w



Saturday, September 12, 2009
non-deterministic
                        pushdown
                         automata


Saturday, September 12, 2009
stack machine
                           w/ backtracking


Saturday, September 12, 2009
massage it



Saturday, September 12, 2009
ε productions



Saturday, September 12, 2009
produce nothing
                         never produced


Saturday, September 12, 2009
cycles



Saturday, September 12, 2009
ambiguities
                               “dangling else”


Saturday, September 12, 2009
if A then if B then C else D

                if A then if B then C else D

                if A then if B then C else D




Saturday, September 12, 2009
parsing expression
                     grammars


Saturday, September 12, 2009
top-down parsing
                       language (70’s)


Saturday, September 12, 2009
direct
                      representation of
                      parsing functions


Saturday, September 12, 2009
Brian Ford 2002



Saturday, September 12, 2009
focused on
                               recognizing


Saturday, September 12, 2009
computer
                               languages


Saturday, September 12, 2009
V←e



Saturday, September 12, 2009
e1 e2



Saturday, September 12, 2009
e1 / e2



Saturday, September 12, 2009
e+



Saturday, September 12, 2009
e*



Saturday, September 12, 2009
&e



Saturday, September 12, 2009
!e



Saturday, September 12, 2009
e?



Saturday, September 12, 2009
“string”
                                   .


Saturday, September 12, 2009
PEG > regexps



Saturday, September 12, 2009
combined
                               lex+parse


Saturday, September 12, 2009
no ambiguity



Saturday, September 12, 2009
choice is ordered



Saturday, September 12, 2009
dangling else
                                 obviated


Saturday, September 12, 2009
greedy repetition



Saturday, September 12, 2009
unlimited
                                 lookahead
                               with predicates


Saturday, September 12, 2009
no left-recursion!
                               (use *,+)




Saturday, September 12, 2009
Parsing
                               Techniques


Saturday, September 12, 2009
Tabular
                               test every rule


Saturday, September 12, 2009
Recursive-descent
                    call & consume


Saturday, September 12, 2009
Predictive
                               yacc/yecc


Saturday, September 12, 2009
Packrat
                               RD with memo


Saturday, September 12, 2009
sacrifice memory
                           for speed


Saturday, September 12, 2009
supports PEGs and
                   some CFGs


Saturday, September 12, 2009
Treetop
                                Pappy
                               neotoma


Saturday, September 12, 2009
neotoma
                 Behind the CodeTM




Saturday, September 12, 2009
can:has(cukes) ->
                          false.

Saturday, September 12, 2009
Cucumber uses
                                  Treetop


Saturday, September 12, 2009
PEG → leex/yecc
                              FAIL


Saturday, September 12, 2009
parsec → eParSec



Saturday, September 12, 2009
HOF protocol



Saturday, September 12, 2009
% Implements "?" PEG operator
                      optional(P) ->
                        fun(Input, Index) ->
                          case P(Input, Index) of
                            {fail, _} -> {[], Input, Index};
                            {_,_,_} = Success -> Success
                              % {Parsed, RemainingInput, NewIndex}
                          end
                        end.




Saturday, September 12, 2009
% PEG
                optional_space <- space?;


                % Erlang
                optional_space(Input,Index) ->
                  optional(fun space/2)(Input, Index).




Saturday, September 12, 2009
Yay! RD!
                               make it memo


Saturday, September 12, 2009
ets
                               Erlang Term
                                 Storage


Saturday, September 12, 2009
{key, value}




Saturday, September 12, 2009
key = Index



Saturday, September 12, 2009
value = dict



Saturday, September 12, 2009
% Memoization wrapper
     p(Inp, StartIndex, Name, ParseFun, TransformFun) ->
       % Grab the memo table from ets
       Memo = get_memo(StartIndex),
       % See if the current reduction is memoized
       case dict:find(Name, Memo) of
         % If it is, return the result
         {ok, Result} -> Result;
         % If not, attempt to parse
         _ ->
            case ParseFun(Inp, StartIndex) of
              % If it fails, memoize the failure
              {fail,_} = Failure ->
                memoize(StartIndex, dict:store(Name, Failure, Memo)),
                Failure;
              % If it passes, transform and memoize the result.
              {Result, InpRem, NewIndex} ->
                Transformed = TransformFun(Result, StartIndex),
                memoize(StartIndex, dict:store(Name, {Transformed,
     InpRem, NewIndex}, Memo)),
                {Transformed, InpRem, NewIndex}
            end
       end.


Saturday, September 12, 2009
parse_transform



Saturday, September 12, 2009
alternative(Input, Index) ->
                        peg:p(Input, Index, alternative, fun(I,P) ->
                            peg:choose([fun sequence/2, fun primary/2])(I,P)
                             end).

                      rule(alternative) ->
                        peg:choose([fun sequence/2, fun primary/2]);




Saturday, September 12, 2009
rules <- space? declaration_sequence space?;
                declaration_sequence <- head:declaration tail:(space declaration)*;
                declaration <- nonterminal space '<-' space parsing_expression space? ';';
                parsing_expression <- choice / sequence / primary;
                choice <- head:alternative tail:(space '/' space alternative)+;
                alternative <- sequence / primary;
                primary <- prefix atomic / atomic suffix / atomic;
                sequence <- head:labeled_sequence_primary tail:(space labeled_sequence_primary)+;
                labeled_sequence_primary <- label? primary;
                label <- alpha_char alphanumeric_char* ':';
                suffix <- repetition_suffix / optional_suffix;
                optional_suffix <- '?';
                repetition_suffix <- '+' / '*';
                prefix <- '&' / '!';
                atomic <- terminal / nonterminal / parenthesized_expression;
                parenthesized_expression <- '(' space? parsing_expression space? ')';
                nonterminal <- alpha_char alphanumeric_char*;
                terminal <- quoted_string / character_class / anything_symbol;
                quoted_string <- single_quoted_string / double_quoted_string;
                double_quoted_string <- '"' string:(!'"' ("" / '"' / .))* '"';
                single_quoted_string <- "'" string:(!"'" ("" / "'" / .))* "'";
                character_class <- '[' characters:(!']' ('' . / !'' .))+ ']
                anything_symbol <- '.';
                alpha_char <- [a-z_];
                alphanumeric_char <- alpha_char / [0-9];
                space <- (white / comment_to_eol)+;
                comment_to_eol <- '%' (!"n" .)*;
                white <- [ tnr];




Saturday, September 12, 2009
self-hosting



Saturday, September 12, 2009
Future directions



Saturday, September 12, 2009
self-contained
                                   parsers


Saturday, September 12, 2009
inline code in PEG



Saturday, September 12, 2009
Reia
                               retem
                               sedate


Saturday, September 12, 2009
questions?



Saturday, September 12, 2009

Weitere ähnliche Inhalte

Andere mochten auch

French Revolution Comic
French Revolution ComicFrench Revolution Comic
French Revolution Comic
drloewen
 
2015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V22015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V2
Marina Micik
 
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test ActivityTechniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
IJTET Journal
 

Andere mochten auch (16)

Ftp
FtpFtp
Ftp
 
Hover Ige
Hover IgeHover Ige
Hover Ige
 
Income statement
Income statementIncome statement
Income statement
 
One direction
One directionOne direction
One direction
 
Utfordringer ved interhospital intensivtransport i Norge
Utfordringer ved interhospital intensivtransport i NorgeUtfordringer ved interhospital intensivtransport i Norge
Utfordringer ved interhospital intensivtransport i Norge
 
French Revolution Comic
French Revolution ComicFrench Revolution Comic
French Revolution Comic
 
Microsoft Learning Experiences Skills and Employability
Microsoft Learning Experiences Skills and Employability Microsoft Learning Experiences Skills and Employability
Microsoft Learning Experiences Skills and Employability
 
Smart Student Tracking System for Schools and Parents from CodeLand
Smart Student Tracking System for Schools and Parents from CodeLandSmart Student Tracking System for Schools and Parents from CodeLand
Smart Student Tracking System for Schools and Parents from CodeLand
 
DSM Based low oversampling using SDR transmitter
DSM Based low oversampling using SDR transmitterDSM Based low oversampling using SDR transmitter
DSM Based low oversampling using SDR transmitter
 
Computadora (tecnologia)
Computadora (tecnologia)Computadora (tecnologia)
Computadora (tecnologia)
 
Practica 20 mayo 2016
Practica 20 mayo 2016Practica 20 mayo 2016
Practica 20 mayo 2016
 
2015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V22015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V2
 
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
 
Colegio El Rosario 2016 -Partido seleccion y maestros
Colegio El Rosario 2016 -Partido seleccion y maestrosColegio El Rosario 2016 -Partido seleccion y maestros
Colegio El Rosario 2016 -Partido seleccion y maestros
 
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test ActivityTechniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
 
Project management overview
Project management overviewProject management overview
Project management overview
 

Ähnlich wie Of Rats And Dragons

DelveUI Slides
DelveUI SlidesDelveUI Slides
DelveUI Slides
jkosoy
 
Spring in-the-cloud
Spring in-the-cloudSpring in-the-cloud
Spring in-the-cloud
Joshua Long
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
Caue Guerra
 
Kostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-EntwicklungKostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-Entwicklung
xrb
 

Ähnlich wie Of Rats And Dragons (20)

DelveUI Slides
DelveUI SlidesDelveUI Slides
DelveUI Slides
 
Code Stinkers Anonymous
Code Stinkers AnonymousCode Stinkers Anonymous
Code Stinkers Anonymous
 
Google in Education (just a peek)
Google in Education (just a peek)Google in Education (just a peek)
Google in Education (just a peek)
 
Building A Framework On Rack
Building A Framework On RackBuilding A Framework On Rack
Building A Framework On Rack
 
Semcomp de São Carlos
Semcomp de São CarlosSemcomp de São Carlos
Semcomp de São Carlos
 
Software livre e padrões abertos no desenvolvimento Web
Software livre e padrões abertos no desenvolvimento WebSoftware livre e padrões abertos no desenvolvimento Web
Software livre e padrões abertos no desenvolvimento Web
 
Its all about collaboration
Its all about collaborationIts all about collaboration
Its all about collaboration
 
Localizing iOS Apps
Localizing iOS AppsLocalizing iOS Apps
Localizing iOS Apps
 
100% JS
100% JS100% JS
100% JS
 
Latinoware Rails 2009
Latinoware Rails 2009Latinoware Rails 2009
Latinoware Rails 2009
 
Expand to or work in China and Germany
Expand to or work in China and GermanyExpand to or work in China and Germany
Expand to or work in China and Germany
 
Zeno rocha - HTML5 APIs para Mobile
Zeno rocha - HTML5 APIs para MobileZeno rocha - HTML5 APIs para Mobile
Zeno rocha - HTML5 APIs para Mobile
 
Mobile Approaches - LinguÁgil 2012
Mobile Approaches - LinguÁgil 2012Mobile Approaches - LinguÁgil 2012
Mobile Approaches - LinguÁgil 2012
 
Spring in-the-cloud
Spring in-the-cloudSpring in-the-cloud
Spring in-the-cloud
 
06 Data
06 Data06 Data
06 Data
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
 
The Beauty of Bootstrapping - Doing it Anyway
The Beauty of Bootstrapping - Doing it AnywayThe Beauty of Bootstrapping - Doing it Anyway
The Beauty of Bootstrapping - Doing it Anyway
 
Kostentreiber bei der iOS Entwicklung
Kostentreiber bei der iOS EntwicklungKostentreiber bei der iOS Entwicklung
Kostentreiber bei der iOS Entwicklung
 
Kostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-EntwicklungKostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-Entwicklung
 
Oxente on Rails 2009
Oxente on Rails 2009Oxente on Rails 2009
Oxente on Rails 2009
 

Mehr von Sean Cribbs

Riak (Øredev nosql day)
Riak (Øredev nosql day)Riak (Øredev nosql day)
Riak (Øredev nosql day)
Sean Cribbs
 
Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)
Sean Cribbs
 
Content Management That Won't Rot Your Brain
Content Management That Won't Rot Your BrainContent Management That Won't Rot Your Brain
Content Management That Won't Rot Your Brain
Sean Cribbs
 

Mehr von Sean Cribbs (19)

Eventually Consistent Data Structures (from strangeloop12)
Eventually Consistent Data Structures (from strangeloop12)Eventually Consistent Data Structures (from strangeloop12)
Eventually Consistent Data Structures (from strangeloop12)
 
Eventually-Consistent Data Structures
Eventually-Consistent Data StructuresEventually-Consistent Data Structures
Eventually-Consistent Data Structures
 
A Case of Accidental Concurrency
A Case of Accidental ConcurrencyA Case of Accidental Concurrency
A Case of Accidental Concurrency
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
 
Riak with node.js
Riak with node.jsRiak with node.js
Riak with node.js
 
Schema Design for Riak (Take 2)
Schema Design for Riak (Take 2)Schema Design for Riak (Take 2)
Schema Design for Riak (Take 2)
 
Riak (Øredev nosql day)
Riak (Øredev nosql day)Riak (Øredev nosql day)
Riak (Øredev nosql day)
 
Riak Tutorial (Øredev)
Riak Tutorial (Øredev)Riak Tutorial (Øredev)
Riak Tutorial (Øredev)
 
The Radiant Ethic
The Radiant EthicThe Radiant Ethic
The Radiant Ethic
 
Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)
 
Riak with Rails
Riak with RailsRiak with Rails
Riak with Rails
 
Schema Design for Riak
Schema Design for RiakSchema Design for Riak
Schema Design for Riak
 
Introduction to Riak - Red Dirt Ruby Conf Training
Introduction to Riak - Red Dirt Ruby Conf TrainingIntroduction to Riak - Red Dirt Ruby Conf Training
Introduction to Riak - Red Dirt Ruby Conf Training
 
Introducing Riak and Ripple
Introducing Riak and RippleIntroducing Riak and Ripple
Introducing Riak and Ripple
 
Round PEG, Round Hole - Parsing Functionally
Round PEG, Round Hole - Parsing FunctionallyRound PEG, Round Hole - Parsing Functionally
Round PEG, Round Hole - Parsing Functionally
 
Story Driven Development With Cucumber
Story Driven Development With CucumberStory Driven Development With Cucumber
Story Driven Development With Cucumber
 
Achieving Parsing Sanity In Erlang
Achieving Parsing Sanity In ErlangAchieving Parsing Sanity In Erlang
Achieving Parsing Sanity In Erlang
 
Erlang/OTP for Rubyists
Erlang/OTP for RubyistsErlang/OTP for Rubyists
Erlang/OTP for Rubyists
 
Content Management That Won't Rot Your Brain
Content Management That Won't Rot Your BrainContent Management That Won't Rot Your Brain
Content Management That Won't Rot Your Brain
 

Kürzlich hochgeladen

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 

Of Rats And Dragons