Tous les projets stratégiques ont besoin de gestion et de surveillance, et les questions qui se posent souvent: 1. Comment puis-je visualiser la santé de ma plate-forme? 2. Comment puis-je analyser les données de mes applications? 3. Comment puis-je visualiser la performance de l'entreprise et la part des objectifs atteints? Dans cette présentation nous allons voir comment nous pouvons répondre à ces questions et donner des tableaux de bord spécifiques fondés sur un ensemble de produits open source qui permettent de centraliser et d'analyser des données de votre SI en temps réel.

1. L’analyse en temps réel
de Big Data
 Le monitoring de flux
par l’exemple
#DevoxxMA #BigData @a_bouchama
Abdellatif BOUCHAMA

2. Abdellatif BOUCHAMA
@a_bouchama
• Middleware architect, and
passionate about the new
technology: Big Data,IoT and
contributor in the Open source
• Co-founder of BusHorn.com
@a_bouchama#DevoxxMA #BigData

3. Agenda
Big Data in 2015:Trends & Statistics
The emergence of real time
Use Case: Flow Activity Monitoring
Demo
@a_bouchama#DevoxxMA #BigData

4. What is Big Data?
Big Data
Velocity
30KB/s --> 30GB/s
· Batch
· Real-Time
· Streaming
Volume
1-2 Terabytes  ∞
Variety
· Structured
· Semi-Structured
· UnStructured

5. Big Data Expectations:
Things That Excite Executives About Big Data
It’s the “next oil.” from the Ginni Rometty, CEO of IBM
The potential to revolutionize industries, and change business models (e.g.,
Uber and Airbnb), with what we learn from the data.
The ability to provide real-time graphing solutions for data relationships.
Real-time operational and business data allows people to make decisions
faster thereby saving significant money.
The opportunities it gives us to help clients solve real business problems.
https://dzone.com/articles/14-things-that-excite-executives-about-big-data?oid=big_data

6. Big Data view by Gartner:
@a_bouchama#DevoxxMA #BigData

7. Big Data in 2015:
75% of Companies Are Investing or Planning to Invest in Big Data in the NextTwoYears
Goals for Big Data initiatives:
1. Enhancing the customer experience
2.Streamlining existing processes
3.Achieving more targeted marketing and reducing costs
Last year, 37% of big data projects were initiated by the CIO, while 25% were initiated by business unit
heads. In 2015, the roles are nearly tied, at 32% and 31 %, respectively.
http://www.gartner.com/newsroom/id/3130817@a_bouchama#DevoxxMA #BigData

8. The emergence of Real-Time
Low
Pure Batch
Operationalizing
Near RealTime
Or
Interactive
Analytics
High
RealTime
Analytics
• Right timeReal time
• Smart dataBig Data
« It is the right time for Real Time to use Big Data as a Smart Data »

9. Use case: Flow Activity
Monitoring
@a_bouchama#DevoxxMA #BigData

10. ESB
ESB
ESB

11. Objectives
Monitor all data flows which goes through the
different layers (transport, mediation, ..etc).
Analyze and centralize data
Perform the correlation between the functional
and technical data
Provide a centralized view in real time.
Deals
Integrate all data flows, analyze them and
correlate them quickly.
Manage a very high data rate, near millions of
events
Manage a number of messages per second
@a_bouchama#DevoxxMA #BigData

12. Constraints & Requirements
Constraints
• Non intrusive system
• No modification on business flows.
• We can plug it and unplug it easily.
Requirements
• System cost should be mastered and
adaptable
• System automatized
• Measurement
@a_bouchama#DevoxxMA #BigData

13. Architecture
CH
ESB
US
ESB
FR
ESB
ElasticSearch

14. Why Elasticsearch stack?
Open source
Easy to deploy
Distributed
Linear scalability
REST Interface
Kibana & Logstash to complete the Landscape
@a_bouchama#DevoxxMA #BigData

15. Data flow and constraints
Collect
JMX
Store & transport
Transform & Access
Modelize & Analyze
Visualize & Predict
JMS

16. DemoTime!
Be prepared for it to fail, because demos always do
@a_bouchama#DevoxxMA #BigData

17. What’s Next ?
• Enrichment analysis, after adding business
information in the logs
• Integration of backend and frontend
applications
• Audit
@a_bouchama#DevoxxMA #BigData

18. THANKS FOR LISTENING
Any Questions?
#DevoxxMA #BigData @a_bouchama