Abundant data is all around. The most important aspect is how you as an organization can access the data, process it, and present information to the relevant authorities on time. To gain competitive advantage the means of accessing, processing and presenting the data should be optimal, highly available and scalable. In this talk, we will discuss different deployment patterns that can provide you with a suitable solution that lets you analyze relevant data in batch, real-time or interactively and predict future states. We will discuss how you can leverage and deploy WSO2 Data Analytics Server, WSO2 IoT Server, WSO2 Enterprise Service Bus and other WSO2 products in order to make better decisions for your organization’s success.

1. Patterns for Deploying Analytics in the
Real World
Sriskandarajah Suhothayan
Associate Director / Architect,
WSO2

2. What’s Analytics ?

3. Problems to think about
• Can it handle my load ?
• How costly it is ?
• Agility ?
• Adaptability ?
• Can it analyse 3rd
party systems ?
• etc ...

4. Where to start ?

5. Where to start ?
• Think Big !

6. Where to start ?
• Think Big !
But...
• Start simple !
• Eat Your Own Dog Food
• Analyse what you already have

7. Step 1 :
Find Data Inside Your Organisation...

8. Collect Data Internally
• Don’t worry about
– Data formats
– Data sources
– Platforms
– Protocols
Start with WSO2 DAS
it has a unified data capturing framework !

9. Deployment for Data Collection

10. Step 2 :
Understand how things have been ...

11. Deployment for Interactive & Batch Analytics …
• Enable Searchability
– Full text search
– Drill down search
• See what has happened
– Summarise the Data
– Understand patterns and behaviors

12. Deployment for Interactive & Batch Analytics …
• Enable Searchability
– Full text search
– Drill down search
• See what has happened
– Summarise the data
– Understand patterns and behaviors
• Simple Deployment
– 2 Nodes
– Use RDBMS to store the data

13. Deployment for Interactive & Batch Analytics
2 Node
Deployment

14. Step 3 :
Keep informed ...

15. Deployment for Realtime Analytics
• Keep informed
– Dashboard
– Alerts
– Feedback loops

16. Deployment for Realtime Analytics
• Keep informed
– Dashboard
– Alerts
– Feedback loops
• High Availability
– Zero downtime
– Zero data loss

17. Realtime High Availability Deployment
Minimum 2 nodes
Max throughput == 1 Node throughput

18. Deployment for Communicating & Alerting
Legacy & Internal
Services

19. Step 4 :
Think ahead ...

20. Deployment for Predictive Analytics
1 Node of WSO2 ML 1 Node of WSO2 ML

21. Minimum High Availability Deployment
All you need a
2 Node
Deployment

22. Step 5 :
Expanding as a Connected Business …

23. Deployment for Exposing Data to 3rd Parties
HTTP
Utilize API Analytics !

24. Analyse Business with API Analytics
• APIs involved
• Who invokes the APIs
• Extract business information from
– Payloads
– Resources URIs
Monetize APIs !

25. Step 6 :
Scale with your Data ...

26. Scaling Analytics Deployment… The Changes !
• Realtime
– Supported by Apache Storm
• For High Memory Requirement or CPU Intensive Processing
– No query changes
• Batch
– Move from RDBMS to HBase/Cassandra
• WSO2 DAS have a Data Abstraction Layer
• Independent of underlying Data Store
Seamless migration :)

27. Realtime Scalable Deployment ...
Event Processing
offloaded to
Siddhi Running on
Apache Storm
Seamlessly :)

28. Realtime Scalable Deployment
Apache Storm Cluster + N CEP nodes

29. Deployment for Scalable Data Analytics
Minimum 8 Nodes
Deployment
(+ Storm if needed)

30. Step 7 :
Sense the world around you ...

31. Deployment of Analytics at the Edge
Analytics on the Edge
with WSO2 Siddhi
Push

32. Analytics Life Cycle
Predefined analytics
• Bundled as CApps
• Allows migration and continues integration
Dev → Test → Preprod → Prod
Analytics on Production Environment
• Interactive Analytics
• Personalizing Dashboards
• Customised Alerts

33. Deployment Management
WSO2 Servers are already puppetized !
Less configuration hazard for Devops
https://github.com/wso2/puppet-modules

34. Summary
• Start small and scale as you grow
• Minimum HA Deployment
– 2 Nodes
• Fully Distributed Deployment
– 8+ Nodes
– Scale based on need, horizontally and vertically
• Analyser, Indexer, Receiver,
Realtime (With Apache Storm), Dashboard
• Use puppet to manage deployment

35. Start analysing data today
for a better tomorrow !

36. Thank You!
#WSO2ConEU
Share your feedback this session
wso2con.com/app