A Middleware Platform to Federate ComplexEvent Processing

1. A Middleware Platform to Federate
Complex Event Processing
Fawaz Paraïso, Gabriel Hermosillo, Romain Rouvoy, Philippe Merle,
Lionel Seinturier
The Sixteenth IEEE International EDOC Conference (2012)
University of Lille & Inria lille-Nord Europe (France)
1

2. Agenda
• Motivation
• Challenges
• Contribution
• Validation
• Conclusion & Perspectives
2

3. Motivation
• What do we mean by event?
– A piece of data that represents something
happened in the real world
• Event-driven behaviour in daily life
– Computer
– Systems
–…
3

4. Motivation
• Events are everywhere
Produce events
4

5. Motivation
• Events are useless if they are not filtered and
correlated
Processing
Events
5

6. Motivation
• What is Complex Event Processing (CEP)?
– Real time processing
– Intelligent business applications
• What applications can benefit from CEP?
– Real-time supply chain management
– Algorithm trading
– Monitoring (transaction, network, …)
– Credit card fraud detection
6

7. Motivation
• The need for real-time processing of
information is relevant for many systems
– Business activity monitoring
– Fraud detection
– Nuclear crisis management
7

8. National
Motivation Radiation
Survey
Network
Weather
Forecast
Experts
Police Firemen
Population
Media
Nuclear Central
Emergency
Army Medical
Service
Operation
Decision
Local
authority
8

9. Agenda
• Motivation
• Challenges
• Contribution
• Validation
• Conclusion & Perspectives
9

10. Challenges
• Challenge 1: Communication heterogeneity
10

11. Challenges
• Challenge 2: Heterogeneous CEP Engines
StreamCruncher
Esper Etalis
CEP
ruleCore Server
11

12. Challenges
• Challenge 3: Scalability
Performance
&
Scalability
12

13. Challenges
• Challenge 4: Adaptability
13

14. Agenda
• Motivation
• Challenges
• Contribution
• Validation
• Conclusion & Perspectives
14

15. Contribution
• A Middleware Platform to Federate Complex
Event Processing
– Federate distributed CEP Engines
– Supports multiple communication services
REST, JMS, WS-Notification
– The DiCEPE Platform is an SCA-based solution
– Implemented in SCA using FraSCAti
Reflective component model
Runtime adaptative system
15

16. Contribution
• Distributed Platform Architecture
DiCEPE
DiCEPE
DiCEPE
16

17. Contribution
• Platform Architecture
*
Listener
Binding
Rest
*
Binding
Engine Statement
JMS
Context
DiCEPE
Legend
Composite Component
Service Reference
Property Wire 17

18. Contribution
• Platform Architecture
– Communication heterogeneity
Orchestrate heterogenenous services
Different bindings
REST, WS, JMS, JNA, UPnP, RPC ,RMI, JGroups, etc.
– Reconfiguration capability
Dynamic reconfigurable runtime architecture
18

19. Contribution
• Platform architecture
– Facililate the integration of CEP engine
Compose an heterogenous piece of software to build a
new service
Supports various
Implementation technologies (Java, BPEL, C, C++, Python, …)
Interface definiton language (WSDL, Java)
19

20. Contribution
• The Platform adresses the challenges of :
– Communication heterogeneity
– Heterogeneous CEP
– Scalability
– Adaptability
20

21. Agenda
• Motivation
• Challenges
• Contribution
• Validation
• Conclusion & Perspectives
21

22. Validation
• DiCEPE for nuclear crisis management
Available here: http://dicepe-broker.soceda.cloudbees.net
22

23. Validation
• The SCA validates the challenge:
– Communication heterogeneity
– Heterogeneous CEP
– Scalability
– Adaptability
23

24. Validation
• Integration with the Esper and Etalis engine
Input
1
Events EventExecutionWorker
1 2
EsperServiceProvider
Configuration
3 EPL 4 4
Event object statement 2
Statements PrologEngineWrapper
Listeners 5
3
Etalis
Output EtalisEventListener
Prolog
Events
Overview of Esper Engine Architecture EtalisWrapper
5
3
4 DiCEPE
4 Architecture
1 2 3
1 2 24

25. Validation
• The integration of Esper and Etalis CEP
engine validates the challenge :
– Communication heterogeneity
– Heterogeneous CEP
– Scalability
– Adaptability
25

26. Validation
• DiCEPE Cost Analysis
Implementation Avg. Exec. Time SCA overhead
Esper 27 sec -
DiCEPE (Esper+ FraSCAti) 30 sec 11%
26

27. Validation
• DiCEPE Scalability
Firemen Events Failures Avg. Sessions Avg. response
10,000 500,000 0 89 0.113 ms
15,000 750,000 0 135 0.142 ms
+ 50% + 50% - + 51% + 26% 27

28. Validation
• The scalability analysis validates the challenge
– Communication heterogeneity
– Heterogeneous CEP
– Scalability
– Adaptability
28

29. Validation
• Dynamic reconfiguration
29

30. Validation
• The FraSCAti validates the challenge:
– Communication heterogeneity
– Heterogeneous CEP
– Scalability
– Adaptability
30

31. Agenda
• Motivation
• Challenges
• Contribution
• Validation
• Conclusion & Perspectives
31

32. Conclusion & Perspectives
• DiCEPE offers interoperability between CEP engines via
federation
• Flexible component architecture
– Successful integration and validation of CEP engines
– Multiple communication protocols
• Real scalability
• Integrate a Domain Specific Language(DSL) to express rules
• Deployment of DiCEPE on heterogeneous cloud
environments
• Error handling capabilities for distributed environments
32

33. Thank you
Questions?
@email: fawaz.paraiso@inria.fr
33