Regular expressions provide a concise way to match patterns in text. They work by converting the regex into a state machine that can efficiently search a string to find matches. Important regex syntax includes quantifiers like *, +, ?, character classes like [a-z], and anchors like ^ and $. Regular expression engines turn the regex pattern into a program that can search strings. Thompson's NFA construction algorithm is commonly used to build the state machine from a regex for efficient matching.
2. Several important Facts
1. Everything in computing was discovered in
one form or another in the 70-80’s and was
probably thought about during the 60’s.
2. The easiest way to become a great computer
engineer in the 80’s was to work for Bell Labs
and have a beard.
4. What are regular expressions?
From Wikipedia:
In computing, a regular expression provides a
concise and flexible means to "match" (specify
and recognize) strings of text, such as particular
characters, words, or patterns of characters.
Common abbreviations for "regular expression"
include regex and regexp.
5. Why do we need regular expressions
(in programming)
Many reasons but most of them are in their base
finding strings in text .
Preferably without reading it
^(?("")(""[^""]+?""@)|(([0-9a-z]((.(?!.))|[-
!#$%&'*+/=?^`{}|~w])*)(?<=[0-9a-
z])@))(?([)([(d{1,3}.){3}d{1,3}])|(([0-9a-z][-
w]*[0-9a-z]*.)+[a-z0-9]{2,17}))$
^(?=.*[^a-zA-Z])(?=.*[a-z])(?=.*[A-Z])S{8,}$
6. Regular Expressions Syntax
meta characters
Grouping
. – match any other character
[ ] – grouping, match single character that is inside the group
[^ ] – grouping, match single character that is not inside the group
( ) – sub expression, in Perl can be recalled later from special variables
Quantifier
{m,n} –specifies that the character/sub expression before need to be matched
at least m times and no more than n times
* - derived from Kleene star in formal logic, matches 0 or more amount of the
character before it.
? –matches zero or one of the preceding elements
+ - derived from Kleene cross in formal logic, matches 1 or more of the
character before it.
Location
^ - Marking start of line
$ - Marking end of line
7. Regular Expressions Syntax
Character groups
[:alpha:] - Any alphabetical character - [A-Za-z]
[:alnum:] - Any alphanumeric character - [A-Za-z0-9]
[:ascii:] - Any character in the ASCII character set.[:blank:] - A GNU
extension, equal to a space or a horizontal tab ("t")
[:cntrl:] - Any control character
[:digit:] - Any decimal digit - [0-9], equivalent to "d“
[:graph:] - Any printable character, excluding a space
[:lower:] - Any lowercase character - [a-z]
[:print:] - Any printable character, including a space
[:punct:] - Any graphical character excluding "word" characters
[:space:] - Any whitespace character. "s" plus the vertical tab ("cK")
[:upper:] - Any uppercase character - [A-Z]
[:word:] - A Perl extension - [A-Za-z0-9_], equivalent to "w“
[:xdigit:] - Any hexadecimal digit - [0-9a-fA-F].
8. What is a regular expression engine
A regular expression engine is a program that takes
a set of constraints specified in a mini-
language, and then applies those constraints to a
target string, and determines whether or not the
string satisfies the constraints.
In less grandiose terms, the first part of the job is to
turn a pattern into something the computer can
efficiently use to find the matching point in the
string, and the second part is performing the search
itself.
11. How the Perl Regex engine works
• Unlike the army only two steps
– Compilation
• Parsing (Size, Construction)
• Peep-hole optimization and analysis
– Execution
• Start position and no-match optimizations
• Program execution
16. Thompson NFA method
• In 1968 Thompson wrote an article on how to
convert a regular expression to still unnamed
automata (NFA)
• The article included code to explain the point
17. Thompson NFA method
1. Check the regex and inject . For concat action
a(b|c)*d
2. Convert to reverse polish notation
abc|*.d.
18. Thompson NFA method cont.
Check single character
OR
char
exp
exp
Kleene star
exp
21. Some examples of Regex
• ([^s]+(.(?i)(jpg|png|gif|bmp))$)
– Match file with specific extentions
• ^(https?://)?([da-z.-]+).([a-z.]{2,6})([/w
.-]*)*/?$
– Match URL
• /^#?([a-f0-9]{6}|[a-f0-9]{3})$/
– Match a hex value
• [ -~]
– An interesting one.
Hinweis der Redaktion
Where they worked on MULTICS which later became unix
From XKCD
The first is for email from MSDN the second is for a password of 8 characters with atleast one lowercase upper case and other symbol
This is from the POSIX specification
Again those are the POSIX groups which are not implemented everywhere the examples are for Perl regex
Camel for Perl, Japaneas for Onigurama (Devil Chariot), Henry Spencers picture, PCRE logo, google for RE2 and a gnu for the GNU REGEX
In compiler theory, peephole optimization is a kind of optimization performed over a very small set of instructions in a segment of generated code. The set is called a "peephole" or a "window". It works by recognising sets of instructions that can be replaced by shorter or faster set of instructions.The execution is including other algorithms like Bayer-Moore and other ways to shorten execution timeFunctions – compilationReg() for parsingReg_branch() Reg_piece()Reg_atom()Reg_tail()Study_chunk() for optimizationFunctions – ExecutionRe_intuit_start() – for starting locationsReg_try()Regmatch()
Not the department of foreign affairs
Accepts the language of all numbers ends with 10 or 01
NFA’s transition table for the regex (l|e)*n?(i|e)el*
Thompson on the left and ritchy on the rightThompson invented B helped inventing CAlso created worked on QED and ed, and invented regular expressionsHelped develop utf8 and the go language
The original article included algol code for the IBM mainframe and was implanted in QED editor
Reverse polish notation is used to remove ambiguity and make it easier to work with stack
Natural language processing is becoming common due to computer speed and massive amounts of data, using statistical tools.UNICODE VS ASCII is a problem for developer due to characters that might be cought as the same while being different
Notice it doesn’t use the xdigit which is not supported in all engines