Event Processing and Stream Reasoning with ETALIS

INFORMATIK
FZI FORSCHUNGSZENTRUM
Event Processing and Stream Reasoning with
ETALIS
Darko Anicic
AIFB Graduiertenkolloquium, KIT
AIFB Graduate Colloquium, KIT

Event Processing & Stream Reasoning

MOTIVATION

23.11.2011 © FZI Forschungszentrum Informatik 2

Real Time Information Processing

Business
Amount of Time to
pressure to
generated react
detect
digitalised
relevant RT
information
situations
Ensure high Ensure Leave more
throughput timeliness time for
computing and low- appropriate
latency reactions

Event Processing


Shifting Event Processing Toward More Intelligent
Event Processing

iEP
EP


Shifting Reasoning Toward Stream Reasoning

Stream
Reasoning
Reasoning


Scenario: Finding a Path Between Two Cities

Deductive reasoning
can solve the
problem


Scenario: Traffic Monitoring

Event Processing
solves the problem


Scenario: Finding a Path in Real Time

Stream Reasoning
solves the problem


Preliminaries & Related Work

INTRODUCTION


Event Processing
An event is defined as an occurrence within a particular system or
domain. It is something that has happened, or is contemplated as having
happened in that domain [Etzion and Niblett, EPIA‟10].

Event Processing is computing that performs operations on events.
Common event processing operations include
reading, creating, transforming, and deleting events [Etzion and
Niblett, EPIA‟10].
 Event-driven interactions vs. request-response interactions
 Asynchronous interactions vs. synchronous interactions
 Information Push vs. Information Pull
 Events as a means to declare changes
 the principle of decoupling

With respect to
Logic Programming...


Overview of ETALIS Approach

ETALIS Foundation

ETALIS Execution EPN in
Language Model ETALIS

ETALIS Extensions

Retraction in Out-of-Order
EP-SPARQL
EP EP

Practical Considerations

Implementation Evaluation ETALIS in Use


Related Work

 Active Databases • XChangeEQ [Bry al.‟89]
HiPAC [McCarthy et al.‟07]
• Introducing time into RDF
• Ode [Gehani et al.„92] rules
Homogenous reaction
[Gutierrez et al.‟07]
• SAMOS [Gatziu et al.„92]
[Paschke et al.‟07]
• SPARQL-ST [Perry et al.‟07]
 Event Processing Systems • Snoop [Chakravarthy et
Statelog [Lausen et al.‟96]
TelegraphCQ
• Temporal SPARQL [Tappolet
•
• al.„94]Maintenance System
[Chandrasekaran et al.‟03]
ECA rules with process
Truth
et al.‟09]
• Amit [Adi [Behrends et al.‟96]
Sentinel [Chakravarthy„97]
algebras et al.‟04]
[Doyle‟78 and „79]
 Logic-Based Approaches • stSPARQL [Koubarakis et
•
• Cayuga [Carney et al.‟07]
SnoopIB[Demers et al.‟02]
Borealis [Adaikkalavan et
Incremental Reasoning on
• al.‟10] stream processing with
Event [BargaRich
•
• al.„06] and et al.‟07]
Streams
CEDR
Replay-based revision al.‟10]
• C-SPARQL [Barbieri et [Li et
out-of-order data arrival
 Retraction in EP • Background sliding windows
Semantics ofKnowledge
[Maskey etKnowledge Bases
• Streaming al.‟02]
al.‟07] et al.‟09]
• [Barbieri et streaming [Barga
Consistent al.‟10]
[Krämer
• [Walavalkarstreaming [Barga
Consistent etet al.‟10]
al.‟08]
• Prova [Kozlenkov et al.‟06]
ZStream [Mei
et al.‟02] SPARQL [Bolles
• Streaming
et al.‟02] et
 Out-of-Order EP • SASE [Wu
• et al.‟08] Pattern Query
Sequence
al.‟06, Gyllstrom et
• Incrementalover Out-of-on
Processing Reasoning
al.‟08, Agrawal et al.‟08]
 Semantic-Based Approaches Streams and Rich
• Order Event Streams [Liu et
Esper, Coral8, StreamSQL,
Background Knowledge
al.‟09]
CCL etc.
• [Barbieri et al.‟10]
Speculative out-of-order
events [Brito al.‟09]


Research Questions

 Can we devise a uniform formalism to formally express both,
complex event patterns and background knowledge for EP and SR?

 How to effectively use logic inference to derive complex events in a
timely fashion (in an event-driven fashion)?

 By realising EP with concepts from LP, can we detect more real time
situations that are otherwise undetectable with sole EP?

 Would an LP approach for EP be extensible enough for specific
requirements found in EP?

 Do we need to compromise on performance, to get in return
detections based on events patterns and background knowledge?



ETALIS Foundation


ETALIS Extensions

EP-SPARQL
EP EP




Design Principles and Requirements (I)

 Formal declarative semantics
 Patterns describe what situations need to be detected, and do not
specify possible ways of detecting them, the order of execution etc.
 Point-based vs. interval-based temporal semantics
 Interval-based semantics enables richer semantics
 possible inconsistencies encountered with point-based
events, e.g., e1 before (e2 before e3)
 Seamless integration of events with queries
 Databases are often used in enriching events with additional data
 Support for query processing (including recursive queries)
 Seamless integration of events with domain knowledge
 To enable reasoning about events and knowledge
 Derivation of implicit information in order to propose
recommendations, or to accomplish event
classification, clustering, filtering


Role of Logic in Event Processing

 Declarative semantics to ground well defined behaviour of event-
based systems
 On-the-fly adaptive: everything is data (patterns can be as easy
changed and adapted as data)
 Justifications: why did an event occur? Why didn‟t it occur?
 Reasoning about events (over time, space, context, their relations
and constraints):
 Contradicting complex events/situations;
 Detection of not yet fulfilled complex patterns (e.g., 80% fulfilled event);
 Event retraction (revision) and out-of-order events.
 Detection of complex events, states, situations of interest, and
further controlling reactive behaviour (actions/reactions) triggered
by detected events;
 Pattern rule management: consistency checking, minimal set of
pattern rules, correctness of pattern rules etc.

Design Principles and Requirements (II)

 Event-driven incremental reasoning
 Events derived in timely fashion and in the asynchronous push mode
 Expressivity
 Support for various event processing agents
 Support for various other features for event-driven applications
 Set at a time vs. event at a time processing
 Computation is performed whenever a relevant event occurs
 Simplicity and ease-of-use
 Rules: data extraction, event composition (event hierarchies), temporal
and causal relations, aggregations, non-monotonic features
 Extensibility
 Capability to support extensions


ETALIS Language for Events - Syntax

A predicate name with t is a term of q is a nonnegative
arity n, t(i) denotes terms type boolean rational number

BIN: SEQ, AND, PAR, OR, EQUALS, MEETS, STARTS, or FINISHES

Event rule is defined as a formula of the following shape:

where p is an event pattern containing all variables occurring in


ETALIS: Interval-based Semantics


ETALIS Language for Events - Semantics


ELE: Complexity Properties

 Complexity Properties depend on the conditions put on the formalism‟s
signature
 EXPTIME-complete, without further restrictions [Dantsin et al. (2001)]
 The formalism is decidable and tractable if both C, and the arity of functions
and predicates, is bounded

 Function free Horn logic is hard for PTIME
 Function symbol “materialisation” can be done in polynomial time
 There are polynomially many static ground atoms
 There are polynomially many event ground atoms to be possible entailed

Dantsin, E., Eiter, T., Gottlob, G., & Voronkov, A. (2001). Complexity and expressive power of logic programming.
ACM Computing Surveys, 33(3), 374–425.

ETALIS: Operational Semantics (SEQ)

1. Complex event pattern
a SEQ b SEQ c → ce1

((a SEQ b) SEQ c) → ce1 2. Decoupling

a SEQ b → ie 3. Binarization
ie SEQ c → ce1
4. Event-driven backward
chaining (EDBC) rules


Order of Rule Execution: SEQ & AND
a ie
c SEQ b → a OP1 OP2
b SEQ a → ie

c b
c SEQ b → a
b AND a → ie


Properties of EDBC Rules

 A simple model: two-input intermediate goals (events)
 Goals are automatically asserted by rules as relevant events occur
 Goals are persisted over a period of time “waiting” to support
detection of a more complex goal
 Goals are unique
 Only goals useful w.r.t the given patterns are computed
 Rules are executed backwards, but they exhibit a forward chaining
behaviour
 During rule evaluation, no backtracking occurs


An Example


Examples of Iterative and Aggregative Patterns

 The k-fold sequential execution of an event a:

 A length-based window of size n:

 A sum over an unbound event stream until a threshold value is met:


Event Processing Network

23.11.2011 Source: O. Etzion undFZI Niblett. Event Processing in Action
© P. Forschungszentrum Informatik 27

Event Processing Agents
Event
Processing
Agent

Pattern
Filter Transformation
detection

Translate Aggregate Split Compose

Enrich

Project

23.11.2011 Source: O. Etzion und P. Niblett. Event Processing in Action 28

Event Filtering in ETALIS Language



ETALIS Foundation


ETALIS Extensions

EP-SPARQL
EP EP




Event Retractions in ETALIS

 Events are often assumed to be immutable and therefore always
correct
 In some situations however revisions are required:
 an event was reported by mistake, but did not happen in reality
 an event was triggered and later revoked due to a transaction failure
 Events used for transaction monitoring and auditing may be
retracted when a transaction fails.
 As recognised in [Ryvkina et al. ICDE‟06], event stream sources
may issue revision tuples that amend previously issued events
 ETALIS takes revision tuples into account and produce correct
revision outputs.


ETALIS: Operational Semantics (rSEQ)

Standard EDBC rules
for SEQ

Additional EDBC
rules for retraction
Rules to save ie1 and
enable its retraction

Processing Out-of-Order Events

 Events are often assumed to be totally ordered
 Delays caused by network latencies, sensor and machine failures
 Delayed events are known as out-of-order events
ce1 ← stock(Agent1, “GO”, Pr1,Vol1) SEQ
stock(Agent2, “GO”, Pr2,Vol2) WHERE Pr1*1.20<Pr2.
ce2 ← stock(Agent1, “MS”, Pr1,Vol1) SEQ
stock(Agent2, “MS”, Pr2,Vol2) WHERE Pr1>1.20*Pr2.

 Missing complex events due to out-of-order stream: stock(agent1,
“GO”, 100,10) SEQ stock(agent2, “GO”, 125,10);
 False positives complex event ce2 due to an out-of-order event.


ETALIS: Operational Semantics (outSEQ)

Additional EDBC
rules for out-of-order


EP-SPARQL: Toward Real-Time Semantic Web

Rapidly changing data Static or slowly evolving
represented as events background knowledge
handles

handles
Event Processing Semantic Web technologies
(EP) including

EP SPARQL EP-SPARQL

• Temporal relatedness
• Semantic relatedness
• Stream reasoning


EP-SPARQL - Syntax

 Extends SPARQL to enable event-based processing by taking into
account temporal situatedness of triple assertions.
 Syntactical and semantic downward-compatibility to plain SPARQL.

 Operators:
FILTER, AND, UNION, OPTIONAL, SEQ, EQUALS, OPTIONALSEQ, a
nd EQUALSOPTIONAL
 getDURATION() yields a literal of type xsd:duration giving the time
interval associated to the graph pattern
 getSTARTTIME() and getENDTIME() retrieve the time stamps of type
xsd:dateTime of the start and end of the interval;


EP-SPARQL - Semantics


EP-SPARQL Example: Traffic Monitoring



ETALIS Foundation


ETALIS Extensions

EP-SPARQL
EP EP




ETALIS: System Diagram

Parser

Compiler

Execution Auxiliary
components

EP-SPARQL: System Diagram

Parser
RDFS Parser
and Compiler

Compiler

Execution

ETALIS Interfaces


Performance Evaluation - Settings

 Intel Core Quad CPU Q9400 2,66GHz, 8GB of RAM running
Windows Vista x64
 SWI Prolog version 5.6.64
 YAP Prolog version 5.1.3
 Esper 3.3.0
 All tested engines ran in a single dedicated CPU core
 Output generated from all tests is validated
 Data sets:
 Stream generator creates time series data with probabilistic values
 Streams with stock data from Google Finance and Yahoo Finance
 sensor readings from the National Data Buoy Center (NDBC)
 subclass relations computed with the Ethan Plants ontology
 to explore routes in Milan we use the Milan ontology
 GeoNames ontologies to identify important geographic locations
affected by weather observations detected in our use case


Common Operators I
Test patterns:

Esper 3.3.0 P-SWI P-YAP Esper 3.3.0 P - SWI P - Yap
Throughput (1000 x Events/Sec)

35 30
30 25
25
20
20
15
15
10
10
5 5
0 0
25 50 75 100 25 50 75 100
Event stream size x 1000 Event stream size x 1000


Common Operators II
Test patterns:

Esper 3.3.0 P-SWI P-Yap Esper 3.3.0 P-SWI P-Yap

50 10
45 9
40 8
35 7
30 6
25 5
20 4
15 3
10 2
5 1
0 0
25 50 75 100 2.5 5 7.5 10
Event stream size x 1000 Event stream size x 1000


Iterative Event Patterns

Test patterns:

Supply chain
Check between from
paths the path 100
and 5000
the beginning or from
the last event


Extensions I
Test patterns:

Retraction in EP with
ETALIS
Revision Flag off Revision Flag on
35
Out of order In order

30 50.0
25
40.0
Out-of-order
20 EP with
30.0
15
ETALIS
20.0
10
10.0
5

0 0.0
SEQ AND PAR OR 0% 10% 20% 33%
Operator Percentage of out-of-order


Scenario: Stream Reasoning Evaluation

 A Goods Delivery system in the city of Milan
 An agent delivers goods to a certain location
 While visiting a location, the system “listens”
to traffic events related to the next location
 Inaccessible routes are recomputed on-the-fly

1 Visitor 10 Visitors 1 Visitor 10 Visitors
1400 1600

Consumed Memory in kB
Consumed time in ms

1200 1400
1000 1200
1000
800
800
600
600
400 400
200 200
0 0
5 10 15 20 5 10 15 20
Number of locations Number of locations


ETALIS in Use

• Event Processing in Action, by
The Fast Flower Opher Etzion and Peter Niblett
Delivery in EPIA
• An EPN implemented in ETALIS

The Drug • Collaborative work on drug design
Discovery in
SYNERGY • ETALIS: extending SOA with EDA

On the Live • MesoWest sensor network
Measurements of
Environmental • Analysing sensor data over
time and geographical space
Phenomena


Conclusions and Outlook

SUMMARY



ETALIS Foundation RR’10, RuleML’11(I), REDS’10, AAIJ’11


ETALIS Extensions RuleML’11(II), PADL’11, WWW’11

EP-SPARQL
EP EP

Practical Considerations SWJ’11



Research Questions

 Can we devise a uniform formalism to formally express
both, complex event patterns and background knowledge for EP
and SR?
 ETALIS Language for Events
 How to effectively use logic inference to derive complex events in a
timely fashion (in an event-driven fashion)?
 EDBC rules
 By realising EP with concepts from LP, can we detect more real time
situations that are otherwise undetectable with sole EP?
 The approach pays off when background knowledge is evolving
 Would an LP approach for EP be extensible enough for specific
requirements found in EP?
 Retraction in EP, out-of-order EP, EP-SPARQL
 Do we need to compromise on performance, to get in return
detections based on events patterns and background knowledge?
 It depends how big and complex background knowledge is

Outlook: Event-Driven Dynamic Processes


Thank you for your attention!

References:

Anicic et al. Real-Time Complex Event Recognition and Reasoning – Anicic et al. EP-SPARQL: A Unified Language for Event Processing
A Logic Programming Approach, the Applied Artificial Intelligence and Stream Reasoning, WWW 2011.
journal, Special Issue on Event Recognition. 2011.
Anicic et al. A Rule-Based Language for Complex Event Processing
Anicic et al. Stream Reasoning and Complex Event Processing in and Reasoning, RR 2010
ETALIS, Semantic Web Journal, Special Issue: Semantic Web Tools
and Systems. 2011. Fodor, Anicic, Rudolph. Results on Out-of-Order Event
Processing, PADL 2011.
Anicic et al. Retractable Complex Event Processing and Stream
Reasoning, RuleML 2011. Xu, N.Stojanovic, Lj.Stojanovic, Anicic, Studer. An approach for more
efficient energy consumption based on real-time situational
Anicic et al. ETALIS: Rule-Based Reasoning in Event awareness, ESWC 2011.
Processing, Chapter in Reasoning in Event-based Distributed
Systems, 2010. Springer. N.Stojanovic, Milenovic, Xu, Lj.Stojanovic, Anicic, Studer. An
intelligent event-driven approach for efficient energy consumption in
Anicic et al. A Declarative Framework for Matching Iterative and commercial buildings: smart office use case, DEBS 2011.
Aggregative Patterns against Event Streams, RuleML 2011.
Apostolou, Stojanovic, Anicic. Responsive Knowledge Management
Anicic et al. Event-driven Approach for Logic-based Complex Event for Public Administration: an Event-Driven Approach, IEEE Intelligent
Processing, The IEEE International Conference on Computational Systems, Special Issue: Transforming E-government & E-
Science and Engineering 2009. participation. 2009.


Event Processing and Stream Reasoning with ETALIS

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Event Processing and Stream Reasoning with ETALIS

Ähnlich wie Event Processing and Stream Reasoning with ETALIS (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Event Processing and Stream Reasoning with ETALIS