Handwritten Text Recognition for manuscripts and early printed texts
Lille2010markp
1. Mark
Proctor
Project Lead
●
The SkyNet funding bill is passed.
●
The system goes online on August 4th, 1997.
●
Human decisions are removed from strategic defense.
●
SkyNet begins to learn at a geometric rate.
●
It becomes self-aware at 2:14am Eastern time, August
29th
●
In a panic, they try to pull the plug.
●
And, Skynet fights back
5. Classes
C a s h f lo w
A cco u n t
D a te d a te
lo n g a c c o u n t N o
d o u b le a m o u n t
d o u b le b a la n c e
in t t y p e
lo n g a c c o u n t N o
A c c o u n t in g P e r io d
D a te s ta r t
D a te e n d
5
6. Creating Views with
Triggers
date amount type accountNo AccountingPeriod
12-Jan-07 100 CREDIT 1 start end
2-Feb-07 200 DEBIT 1 01-Jan-07 31-Mar-07
18-May-07 50 CREDIT 1
9-Mar-07 75 CREDIT 1 Account
accountNo balance
1 0
increase balance for AccountPeriod Credits decrease balance for AccountPeriod Debits
select * from Account acc, select * from Account acc,
Cashflow cf, AccountPeriod ap Cashflow cf, AccountPeriod ap
where acc.accountNo == cf.accountNo and where acc.accountNo == cf.accountNo and
cf.type == CREDIT cf.type == DEBIT
cf.date >= ap.start and cf.date >= ap.start and
cf.date <= ap.end cf.date <= ap.end
trigger : acc.balance += cf.amount trigger : acc.balance -= cf.amount
CashFlow CashFlow
date amount type
date amount type
12-Jan-07 100 CREDIT
2-Feb-07 200 DEBIT
9-Mar-07 75 CREDIT
Account
accountNo balance
1 -25
6
7. What is a Rule
salience <int>
Quotes on Rule names are
agenda-group <string>
optional if the rule name has
no-loop <boolean>
no spaces.
auto-focus <boolean>
duration <long>
....
• rule “<name>”
<attribute> <value>
when
<LHS>
then
<RHS> RHS can be any valid java.
Or MVEL. Other languages
end could be added.
7
8. Imperative vs Declarative
Methods that must specific passing of
be called directly instances
• public void helloMark(Person person) {
Rules can never if ( person.getName().equals( “mark” ) {
be called directly System.out.println( “Hello Mark” );
} Specific instances
} cannot be passed.
• rule “Hello Mark” LHS
when
Person( name == “mark” )
then
System.out.println( “Hello Mark” );
end
RHS
8
9. What is a Pattern
P a tte rn
O b je c t T y p e F ie ld C o n s t r a in t
F ie ld N a m e R e s t r ic t io n
E v a lu a t o r V a lu e
S h o w e r( te m p e r a tu re = = “ h o t” )
9
10. Bringing it Together
select * from Account acc,
Cashflow cf, AccountPeriod ap
where acc.accountNo == cf.accountNo and
cf.type == CREDIT
cf.date >= ap.start and
cf.date <= ap.end Pattern
trigger : acc.balance += cf.amount
Pattern Binding
field Binding
rule “increase balance for AccountPeriod Credits”
when Variable Restriction
ap : AccountPeriod()
acc : Account( $accountNo : accountNo )
CashFlow( type == CREDIT,
accountNo == $accountNo,
Literal Restriction date >= ap.start && <= ap.end,
$ammount : ammount )
then
acc.balance += $amount; Multri Restriction - Variable
end Restriction
field Binding
Consequence (RHS)
10
11. Rules as a “view”
date amount type accountNo AccountingPeriod
12-Jan-07 100 CREDIT 1 start end
2-Feb-07 200 DEBIT 1 01-Jan-07 31-Mar-07
18-May-07 50 CREDIT 1
9-Mar-07 75 CREDIT 1 Account
accountNo balance
1 0
rule “increase balance for AccountPeriod rule “decrease balance for AccountPeriod
Credits” Debits”
when when
ap : AccountPeriod() ap : AccountPeriod()
acc : Account( $accountNo : accountNo ) acc : Account( $accountNo : accountNo )
CashFlow( type == CREDIT, CashFlow( type == DEBIT,
accountNo == $accountNo, accountNo == $accountNo,
date >= ap.start && <= ap.end, date >= ap.start && <= ap.end,
$ammount : ammount ) $ammount : ammount )
then then
acc.balance += $amount; acc.balance -= $amount;
end CashFlow end CashFlow
date amount type date amount type
12-Jan-07 100 CREDIT 2-Feb-07 200 DEBIT
9-Mar-07 75 CREDIT
Account
accountNo balance
1 -25
11
12. Patterns in more details
CashFlow( type == “credit” )
$ap : AccountPeriod()
CashFlow( date >= $ap.start )
$ap : AccountPeriod()
CashFlow( date >= $ap.start && <= $ap.end )
$ap : AccountPeriod()
CashFlow( type == “credit”,
date >= $ap.start && <= $ap.end )
12
13. More Pattern Examples
Person( $age : age )
Person( age == ( $age + 1 ) )
Person( $age : age )
Person( eval( age == $age + 1 ) )
Person( $age1 : age )
Person( $age2 : age )
eval( $age2 == $age1 + 1 )
13
15. What is a Production Rule
System
Codification of
the business Repository of
knowledge inserted Java
instances
Inference
Engine
Production Pattern Working
Memory Matcher Memory
(rules) (facts)
Agenda
insert
Rules can change update
on the fly retract
15
16. Production Rule System
Approximated by SQL and
Views
T a b le s A cco unt A c c o u n t in g P e r io d C a s h f lo w
O b je c t T y p e s A cco unt A c c o u n t in g P e r io d C a s h f lo w
V ie w s v ie w 1 v ie w 2
V ie w
m a in v ie w R u le s r u le 1 r u le 2
agenda
agenda
16
17. Conflict Resolution with
Salience
Salience
rule “Print blance for AccountPeriod”
salience -50
when
ap : AccountPeriod()
acc : Account( )
then
System.out.println( acc.accountNo + “ : “ acc.balance );
end
Agenda
1 increase balance
2 decrease balance arbitrary
3 increase balance
4 print balance
17
18. RuleFlow
rule “increase balance for AccountPeriod Credits”
ruleflow-group “calculation”
when
ap : AccountPeriod()
acc : Account( $accountNo : accountNo )
CashFlow( type == CREDIT,
accountNo == $accountNo,
date >= ap.start && <= ap.end,
$ammount : ammount )
then
acc.balance += $amount;
end
ruleflow-group
rule “Print blance for AccountPeriod”
ruleflow-group “report”
when
ap : AccountPeriod()
acc : Account( )
then
System.out.println( acc.accountNo + “ : “ acc.balance );
end
18
19. Two Phase System
Determine
possible rules to
fire
Agenda Evaluation
Working Memory Action
Rule
insert modify
Found Select
Fire Rule
Rule to Fire
retract
No Rule
Found
exit
19
23. From CE for Expressions
Using 'from' to reason over the nested list
rule “Find all the pets for a given owner”
when
$owner : Person( name == “mark” )
Pet( name == “rover” ) from $owner.pets
23
24. From CE for Expressions
'from' can work on any expression, not just a nested
field on a bound variable.
rule “Find People for given zip code”
when
$zipCode : ZipCode()
Person( ) from $hbn.getNamedQuery(“Find People”)
.setParameters( [ “zipCode” : $zipCode ] )
.list()
Hibernate session
24
28. Accumulate CE
rule "accumulate"
when
$sum : Number( intValue > 100 )
from accumulate( Bus( color == "red", $t :
takings )
init( sum = 0 ),
action( sum += $t ),
result( sum ) )
then
print "sum is “ + $sum;
end
28
29. Accumulate CE
rule "accumulate"
when
$sum : Number( intValue > 100 )
from accumulate( Bus( color == "red", $t :
takings ) sum( $t ) )
then
print "sum is “ + $sum;
end
29
30. Accumulate CE
Patterns and CE's can be chained with 'from'
rule "collect"
when
$zipCode : ZipCode()
$sum : Number( intValue > 100 )
from accumulate( Bus( color == "red", $t : takings )
from $hbn.getNamedQuery(“Find
Buses”)
.setParameters( [ “zipCode” :
$zipCode ] )
.list(),
sum( $t ) )
then 30
32. Timers
rule “name”
timer 1m30s
when
$l : Light( status == “on” )
then
SendEmail( “turn the light off” )
rule “name”
timer (int: 0 1m30)
when
$l : Light( status == “on” )
then
SendEmail( “turn the light off” )
32
33. Timers
Field Name Mandatory? Allowed Values Allowed Special Characters
Seconds YES 0-59 ,-*/
Minutes YES 0-59 ,-*/
Hours YES 0-23 ,-*/
Day of month YES 1-31 ,-*?/LW
Month YES 1-12 or JAN-DEC ,-*/
Day of week YES 1-7 or SUN-SAT ,-*?/L#
Year NO empty, 1970-2099 ,-*/
rule “name”
timer ( cron: 0 0/15 * * * * )
when
$l : Light( status == “on” )
then
sendEmail( “turn the light off” )
33
34. Calendars
rule "weekdays are high priority"
calendars "weekday"
timer (int:0 1h)
when
Alarm()
then
send( "priority high - we have an alarm” );
end
rule "weekend are low priority"
calendars "weekend"
timer (int:0 4h)
when
Alarm()
then
send( "priority low - we have an alarm” );
end
34
36. TMS and Inference
rule "Issue Child Bus Pass" Couples the logic
when
$p : Person( age < 16 )
then
insert(new ChildBusPass( $p ) );
end
rule "Issue Adult Bus Pass"
What happens when the Child
when stops being 16?
$p : Person( age >= 16 )
then
insert(new AdultBusPass( $p ) );
end
36
37. TMS and Inference
Bad
● Monolithic
● Leaky
● Brittle integrity - manual maintenance
37
38. TMS and Inference
A rule “logically” inserts an object
When the rule is no longer true, the object is
retracted.
when de-couples the logic
$p : Person( age < 16 )
then
logicalInsert( new IsChild( $p ) )
end
Maintains the truth by
when automatically retracting
$p : Person( age >= 16 )
then
logicalInsert( new IsAdult( $p ) )
end
38
39. TMS and Inference
rule "Issue Child Bus Pass"
when
$p : Person( )
IsChild( person =$p )
then
logicalInsert(new ChildBusPass( $p ) );
end
rule "Issue Adult Bus Pass" The truth maintenance cascades
when
$p : Person( age >= 16 )
IsAdult( person =$p )
then
logicalInsert(new AdultBusPass( $p ) );
end
39
40. TMS and Inference
rule "Issue Child Bus Pass"
when
$p : Person( )
not( ChildBusPass( person == $p ) )
then
requestChildBusPass( $p );
The truth maintenance cascades
End
40
41. TMS and Inference
Good
● De-couple knowledge responsibilities
● Encapsulate knowledge
● Provide semantic abstractions for those encapsulation
● Integrity robustness – truth maintenance
41
50. Scalability
Rule engines do not scale for CEP. They have a single point of insertion
and are single threaded, CEP has concurrent streams of events.
session.insert( event ) ; $c : Custumer( type == “VIP” )
BuyOrderEvent( customer == $c )
Single Point of
entry
Patterns, evaluate
facts sequentially in a
single thread.
50
51. Scalability
So lets allow multiple
named entry points for
those streams
EntryPoint entryPoint = session.getEntryPoint( “Home Broker Stream” );
entryPoint.insert( event ) ;
So now we can insert When not specified uses
different streams the “default” entry-point
concurrently
$c : Custumer( type == “VIP )
BuyOrderEvent( customer == $c ) from entry-point “Home Broker Stream”
Patterns can now optional
specify their entry-point.
51
52. Automatic Life-Cycle
Management
All Fact life-cycles must be managed by the user, so
retractions are manual.
declare StockTick Just use the declare statement to
@role( event ) declare a type as an event and it
end will be retracted when it is no longer
needed
declare StockTick
@role( event )
@timestamp( timestampAttr )
companySymbol : String
stockPrice : double The declare statement can also
timestampAttr : long specify an internal model, that
end external objects/xml/csv map on to.
We support Smooks and JAXB
52
53. Operators
Rule engines do not have rich enough set of temporal comparison
operators BackAckEvent must occur
between 1s and 10s 'after'
$c : Custumer( type == “VIP ) BuyOrderEvent
$oe : BuyOrderEvent( customer == $c )
from entry-point “Home Broker Stream”
BuyAckEvent( relatedEvent == $oe.id, this after[1s, 10s] $oe )
from entry-point “Stock Trader Stream”
The Full set of Operators are supported
● coincides ● overlaps ● starts
● before ● overlappedby ● startedby
● after ● during ● finishes
● meets ● includes ● finishedby
● metby
53
55. Operators
$c : Custumer( type == “VIP )
$oe : BuyOrderEvent( customer == $c )
from entry-point “Home Broker Stream”
not BuyAckEvent( relatedEvent == $oe.id, this after[1s, 10s] $oe )
from entry-point “Stock Trader Stream”
Existing Drools 'not'
Conditional Elements can
be used to detect non-
occurrence of events
55
56. Sliding time windows
Rule engines react to events happening now, there is no
temporal understanding of changes over time.
StockTicker( symbol == “RHAT” ) over window:time( 5s ) 5s
StockTicker( symbol == “RHAT” ) over window:length( 1000 )
1000 tickers
That isn't much without the ability to deal with aggregations, rules
engines suck.
56
57. Aggregations
Rule Engines do not deal with aggregations
Over 5 seconds
$n : Number( intValue > 100 )
from accumulate( $s : StockTicker( symbol == “RHAT” ) over window:time( 5s ),
average( $s.price ) )
Aggregate ticker price
The pattern 'Number' for RHAT over last 5
reasons 'from' the seconds
accumulate result
$n : accumulate( $s : StockTicker( symbol == “RHAT” ) over window:time( 5s ),
average( $s.price ) > 100 )
We can use some sugar to reduce
verbosity
57
69. Example
Business decisions are externalized using a decision
service
rule
rule Decision1
when Decision1
rule Decision1
// conditions
when when
then // conditions
//
// actions
then
end conditions
// actions
end then
//
actions
end 69
70. Example
What if there is a lot of business logic like this?
rule Decision1 rule Decision1
rule when Decision1
rule when Decision1
rule rule rule
rule Decision1 // conditions // conditionsrule Decision1 rule Decision1
when Decision1
when Decision1 rule rule when Decision1
rule
rule Decision1 then // conditions rule rule conditions Decision1 when Decision1
rule Decision1 Decision1
then // when Decision1
rule rule Decision1
// conditionswhen when
rule Decision1
when // conditions // conditions
when when then // conditionswhen Decision1 when when
Decision1
// actions rule actions
// Decision1
then // conditions when when
then // conditions// conditions end // conditionsthen // conditionsthen // conditions
// end when when
// actions
then
// actions
then when when
// actions
then
then // conditions then // conditions// actions
then
// //
// actions
then
end end
conditions// actions //
// actions end //
// actions conditions conditions
// actions then // actions end
then // actions end // actions
end then end end actions end end actions end
conditions
// conditions
// then end then
// end then end then // //
actions // // actions actions
end actions actions end end
end end
70
73. Self monitoring and
adaptive
declare ProcessStartedEvent
@role( event )
end
rule "Number of process instances above threshold" when
Number( nbProcesses : intValue > 1000 )
from accumulate(
e: ProcessStartedEvent(
processInstance.processId ==
"com.sample.order.OrderProcess" )
over window:size(1h),
count(e) )
then
System.err.println( "WARNING: Nb of order processes in
the last hour > 1000: " +
nbProcesses );
73
89. Questions?
• Dave Bowman: All right, HAL; I'll
go in through the emergency
airlock.
• HAL: Without your space helmet,
Dave, you're going to find that
rather difficult.
• Dave Bowman: HAL, I won't argue
with you anymore! Open the
doors!
• HAL: Dave, this conversation can
serve no purpose anymore.
Goodbye.
Joshua: Greetings, Professor Falken.
Stephen Falken: Hello, Joshua.
Joshua: A strange game. The only
winning move is not to play. How
about a nice game of chess?
89
