Lecture2 B

CMSC 723 / LING 645: Intro to Computational Linguistics September 8, 2004: Monz Regular Expressions and Finite State Automata (J&M 2) Prof. Bonnie J. Dorr Dr. Christof Monz TA: Adam Lee

Regular Expressions and Finite State Automata ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Regular Expressions ,[object Object],[object Object],[object Object]

Regular Expressions ,[object Object],[object Object]

Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],* + Stephen Cole Kleene

Regular Expressions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Disjunction, Grouping, Precedence ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Perl Commands While ( $line=<STDIN> ){ if ( $line =~ /the/ ){ print “MATCH: $line”; } }

Writing correct expressions ,[object Object],/the/ / [tT] he/ / [tT] he / / [â-zA-Z] [tT] he [â-zA-Z] / / ( ^ | [â-zA-Z] ) [tT] he [â-zA-Z] /

A more complex example ,[object Object],/ $ [0-9]+/ / $ [0-9]+ [0-9][0-9] / / $ [0-9]+ ( [0-9][0-9] )? / / $ [0-9][0-9]?[0-9]? ( [0-9][0-9] )? / / [0-9]+ * ([MG]Hz|[Mm]egahertz| [Gg]igahertz) / / [0-9]+ * (Mb|[Mm]egabytes ? ) / / [0-9]([0-9]+) * (Gb|[Gg]igabytes ? ) /

Advanced operators should be _

Substitutions and Memory ,[object Object],s/colour/color/ s/colour/color/g s/([Cc]olour)/$1olor/ /the (.*)er they were, the $1er they will be/ /the (.*)er they (.*), the $1er they $2/ Substitute as many times as possible! Case insensitive matching s/colour/color/i ,[object Object]

Eliza [Weizenbaum, 1966] ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Eliza-style regular expressions ,[object Object],[object Object],[object Object],[object Object],Step 1: replace first person with second person references s/ I(’m| am) /YOU ARE/g s/ my /YOUR/g S/ mine /YOURS/g Step 2: use additional regular expressions to generate replies Step 3: use scores to rank possible transformations

Finite-state Automata ,[object Object],[object Object],[object Object]

Finite-state Automata (Machines) /^baa+!$/ q 0 q 1 q 2 q 3 q 4 b a a ! a state transition final state baa! baaa! baaaa! baaaaa! ...

Input Tape REJECT a b a ! b q 0 0 1 2 3 4 b a a ! a

Input Tape ACCEPT b a a a q 0 q 1 q 2 q 3 q 3 q 4 ! 0 1 2 3 4 b a a ! a

Finite-state Automata ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

State-transition Tables 4: 3 2 1 0 State Ø 3 Ø 4 3 Ø Ø Ø Ø Ø 2 Ø Ø Ø 1 ! a b Input

D-RECOGNIZE function D-RECOGNIZE ( tape , machine ) returns accept or reject index  Beginning of tape current-state  Initial state of machine loop if End of input has been reached then if current-state is an accept state then return accept else return reject elsif transition-table [current-state, tape[index]] is empty then return reject else current-state  transition-table [current-state, tape[index]] index  index + 1 end

Adding a failing state q 0 q 1 q 2 q 3 q 4 b a a ! a q F a ! b ! b ! b b a !

Adding an “all else” arc q 0 q 1 q 2 q 3 q 4 b a a ! a q F = = = =

Languages and Automata ,[object Object],[object Object],[object Object],[object Object]

Languages and Automata ,[object Object],[object Object]

Using NFSAs to accept strings ,[object Object],[object Object],[object Object]

Using NFSAs Input Ø 4 Ø Ø Ø ! 4: 3 2 1 0 State Ø 2,3 Ø Ø Ø Ø Ø Ø Ø Ø 2 Ø Ø Ø 1  a b

Readings for next time ,[object Object]

Lecture2 B

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (17)

Ähnlich wie Lecture2 B

Ähnlich wie Lecture2 B (20)

Lecture2 B