Smarter Analytics and Big Data
Building The Next Generation Analytical insights
Joel Waterman, Regional Director of Business Analytics for the Middle East and Africa, discusses how IBM is making significant investments in smarter analytics and big data through acquisitions, technical expertise, and research. IBM's big data platform moves analytics closer to data through technologies like Hadoop, stream computing, and data warehousing. The platform is designed for analytic application development and integration using accelerators, user interfaces, and IBM's ecosystem of business partners.
IBM Software Day 2013. Smarter analytics and big data. building the next generation of analytical insights
1. Smarter Analytics and Big Data
Building The Next Generation
Analytical insights
Joel Waterman
Regional Director, Business Analytics Middle East & Africa
2.
WATCH A MOVIE
DOWNLOAD AN
APP
READ A PAPER
5. Why Business Analytics Matter
Eight out of ten CEOs expect complexity to Enterprises that apply advanced analytics
increase significantly in the next five years; have 33% more revenue growth and 12X
insight and intelligence is ranked a top 3 more profit growth.
priority.
- CEO study 2010 – CFO study 2010
Financial outperformers are 64% more likely CIOs rank analytics as the #1 factor
to use analytics to evaluate talent supply and contributing to an organization’s
demand on an ongoing basis. competitiveness.
– CHRO study 2010 – CIO study 2009
Top-performing enterprises use business analytics 5 times more than lower performers.
– 2010 joint study, MIT and IBM Institute for Business Value
6. Why Business Analytics Matter
The Need for Analytics is Pervasive Across Business and Industry
The healthcare industry spends $250 - $300 billion on healthcare fraud,
per year. In the US alone this is a $650 million per day problem.1
One rogue trader at a leading global financial services firm created
$2 billion worth of losses, almost bankrupting the company.
$93 billion in total sales is missed each year because retailers don’t
have the right products in stock to meet customer demand.
5 billion global subscribers in the telco industry are demanding unique
and personalized offerings that match their individual lifestyles. 2
Source: 1. Harvard, Harvard Business Review, April 2010.
2. IBM Institute for Business Value, The Global CFO Study, 2010.
7. IBM is Making Significant Investments in Smarter Analytics
Investment $ 16B for 30 acquisitions since 2005
8 Analytic Solutions Centers worldwide
Technical Expertise More than 10,000 technical professionals
9,000 consultants delivering IBM analytics solutions
Power7: workload optimized analytic processing
Technology Smart Analytics cloud
Comprehensive, heterogeneous Big Data platform for the Enterprise
World’s largest math department in private industry
FOAK breakthrough innovations including IBM Watson
Research
Number 1 in patent ranking for 19 years and more than 500 analytics-related
patents / year for last two years
Business Partners More than 27,000 Business Partner certifications
8. Imagine the Possibilities of Analyzing All Available Data
Faster, More Comprehensive, Less Expensive
Real-time Traffic Fraud & risk Understand and act on
Flow Optimization detection customer sentiment
Accurate and timely Predict and act on Low-latency network
threat detection intent to purchase analysis
9.
10. Cisco turns to IBM big data
for intelligent infrastructure
management
• Optimize building energy consumption with
centralized monitoring
• Automate preventive and corrective maintenance
Capabilities Utilized:
• Streaming Analytics
• Hadoop System
• Business Intelligence
Applications:
• Log Analytics
• Energy Bill Forecasting
• Energy consumption optimization
• Detection of anomalous usage
• Presence-aware energy mgt.
• Policy enforcement
11. Harnessing the Largest Predictive Focus Group in the World
Purpose
• Understand public sentiment towards an event:
movie trailers
• Deeply understand the potential customer profile:
gender, occupation, intent to watch
• Alter marketing launch plans based on insight
Background
• 1.1 Billion Tweets analyzed
• 5.7 Million blogs/forum posts
• 3.5 million messages
• Also: Facebook, Google+, Tumblr, Flickr
12. Conclusion – Actionable Insight
Adjust the marketing launch plan before
execution begins
• Creative – adjust messaging
• Trailers – alter scenes shown
• Budget – re-direct, increase, or decrease
• Execution – theatre placement, advertisement
placement
14. IBM Big Data Strategy: Move the Analytics Closer to the Data
Analytic Applications
New analytic applications drive the BI /Reporting Exploration /
Visualization
Functional
App
Industry
App
Predictive
Analytics
Content
Analytics
requirements for a big data platform
IBM Big Data Platform
• Visualize all available data for ad-hoc Visualization Application Systems
analysis & Discovery Development Management
• Development environment for building
new analytic applications Accelerators
• And a whole lot good solid IT stuff...
Hadoop Stream Data
System Computing Warehouse
Information Integration & Governance
15. Big Data Platform - Hadoop System
• Manages a wide variety and huge volume of data
• Augments open source Hadoop with enterprise
capabilities
• Performance Optimization
• Development tooling
• Enterprise integration
• Analytic Accelerators
• Application and industry accelerators
• Visualization
• Security
17. IBM’s Hadoop System provides unique business value
• Integration with enterprise systems
• Industry & application accelerators
• Templates for quick starting app dev
• Visualization tools to help business users
to explore big data
18. Big Data Platform - Stream Computing
Built to analyze data in motion
• Multiple concurrent input streams
• Massive scalability
Process and analyze a variety of data
• Structured, unstructured content,
video, audio
• Advanced analytic operators
19. Asian telco reduces
billing costs and improves
customer satisfaction
Capabilities:
Stream Computing
Analytic Accelerators
Real-time mediation and analysis of
6B CDRs per day
Data processing time reduced from
12 hrs to 1 sec
Hardware cost reduced to 1/8th
Proactively address issues
(e.g. dropped calls) impacting customer
satisfaction.
20. Stream Computing provides unique business value
• Real-time answers = Better outcomes for time sensitive
applications (e.g. fraud detection, network management)
• Solution when data is too large or expensive to store
• Analyze data as it comes to you
• Keep data of interest for deeper analysis
21. Big Data Platform - Data Warehousing
Workload optimized systems
• Deep analytics appliance
• Configurable operational analytics appliance
• Data warehousing software
Capabilities
• Massive parallel processing engine
• High performance (OLAP)
• Mixed operational and analytic workloads
22. Deep Analytics Appliance – Revolutionizing Analytics
Purpose-built analytics appliance
Dedicated High
Performance
Disk Storage
Speed: 10-100x faster than traditional systems
Simplicity: Minimal administration and tuning
Scalability: Peta-scale user data capacity
Smart: High-performance advanced analytics
Blades With
Custom FPGA
Accelerators
23. Pacific Northwest Smart Grid
Demonstration Project
Capabilities:
Stream Computing – real-time
control system
Deep Analytics Appliance – analyze
massive data sets
Demonstrates scalability from 100 to
500K homes while retaining 10 years’
historical data
60k metered customers in 5 states
Accommodates ad hoc analysis of price
fluctuation, energy consumption profiles,
risk, fraud detection, grid health, etc.
24. Data Warehousing provides unique business value
• Consolidate, manage and reconcile data
for enterprise business intelligence
• Establish trust, quality and governance
where necessary
• Financial data
• Credit card data
• Healthcare
• Combine deep and operational analytics
• Maintain history for trending and
historical reporting
Image: David Castillo Dominici
25. Big Data Platform - Accelerators
Analytic accelerators
• Analytics, operators, rule sets
Industry and Horizontal Application
Accelerators
• Analytics
• Models
• Visualization / user interfaces
• Adapters
26. Analytic Accelerators Designed for Variety
Text
(listen, verb), Simple & Acoustic
(radio, noun) Advanced Text
Mining in Advanced
Microseconds Mathematical Models
Predictive ∑R( s , a )
population
t t
Statistics
GeoSpatial Image & Video
27. Accelerators Improve Time to Value
Telecommunications Retail Customer
CDR streaming analytics Intelligence
Deep Network Analytics Customer Behavior and Lifetime
Value Analysis
Finance Social Media Analytics
Streaming options trading Sentiment Analytics, Intent to
purchase
Insurance and banking DW
models
Public transportation Data mining
Real-time monitoring and Streaming statistical analysis
routing optimization
Over 100 sample User Defined Toolkits Standard Toolkits Industry Data Models
applications Banking, Insurance, Telco,
Healthcare, Retail
28.
29. Telecommunications CDR Analytic Accelerator
Analyze Call Detail Records in real time
Streaming Analytic Accelerators
• Dropped call analysis
• Who are VIP customers with service issues –
proactive alerts
• Analytic Operators – CDR de-duplication, dropped
call detection, termination reason, customer
importance
• Visualization – real-time KPI dashboard
Data Warehouse Appliance
• Integrated network, devices, customer, and
services model
• Telecom model, KPIs, and KQIs Image: nokhoog_buchachon
30. Big Data Platform – Information Integration and Governance
Integrate any type of data to the big data
platform
• Structured
• Unstructured
• Streaming
Govern big data
• Secure sensitive data
• Lifecycle management to control data growth
• Master data to establish single version of the
truth
32. Marketing Services Leader
integrates big data for
Leaders integrate and govern massive Big Data with InfoSphere
customer intelligence
Capabilities Utilized:
Information Integration – data
quality, ETL
Deep Analytics Appliance
Complex customer data integration for
54M records/hour
Processing
5B simultaneous records
32 Manages
32 2 petabytes of data,
33. Big Data Platform - User Interfaces
Business Users
•Visualization of a large volume and wide variety of data
Developers
• Similarity in tooling and languages
• Mature open source tools with enterprise
capabilities
• Integration among environments
Administrators
• Consoles to aid in systems management
34. Visualization - Spreadsheet-style user interface
• Ad-hoc analytics for LOB user
• Analyze a variety of data - unstructured
and structured
• Browser-based
• Spreadsheet metaphor for exploring/
visualizing data
Gather Extract Explore Iterate
Crawl – gather statistically Document-level info Analyze, annotate, filter Iterate through any prior
Adapter–gather Cleanse, normalize Visualize results step
dynamically
35. Big Data Platform - Analytic Applications
Big Data Platform is designed for analytic
application development and integration
BI/Reporting – Cognos BI, Attivio
Predictive Analytics – SPSS, G2, SAS
Exploration/Visualization – BigSheets, Datameer
Instrumentation Analytics – Brocade, IBM GBS Content
Analytics – IBM Content Analytics
Functional Applications – Algorithmics, Cognos Consumer
Insights, Clickfox, i2, IBM GBS
Industry Applications – TerraEchos, Cisco, IBM GBS
36. IBM’s big data business partner ecosystem
100thCC&G Partners
Big Data
Business Partner
Signed
37. IBM’s unique strengths in Big Data
Big Data in Real-Time
Fit for purpose analytics
Enterprise Class
Integration