2. Streaming Data Processing*
Over the next few years we'll see the adoption of
scalable frameworks and platforms for handling
streaming, or near real-time, analysis and processing. In
the same way that Hadoop has been borne out of
large-scale web applications, these platforms will be
driven by the needs of large-scale location-aware
mobile, social and sensor use. –
Edd Dumbill O’REILLY
7. Gartner on Analytics, Big Data and Internet of Things
Forecasts of over 50 billion intelligent devices by 2015 and 275
Exabytes per day of data being sent across the Internet by 2020 are
indicators of looming challenges
• Gartner's Analytics Key Initiative focus is on two analytical styles:
predictive and real-time.
• Information of extreme size and need for rapid processing.
• Analytics help uncover root causes, support decisions, and make
predictions in real time.
8.
9.
10.
11.
12.
13.
14. The value of timely analytics, on demand if necessary…
15. The value of timely analytics, on demand if necessary…
16. Velocity Pipeline
2.0
3.0
TRADITIONAL
ANALYTICS
• Primarily descriptive
analytics & reporting
• Internally sourced,
relatively small,
structured data
• “Back Room” teams
of business analysts
BIG DATA
• Complex, large,
unstructured data sources
• New analytical and
computational capabilities
• Data Scientists emerge
RAPID INSIGHTS
PROVIDING
BUSINESS IMPACT
• Analytics integral to running the
business; considered strategic
competitive asset. CEP thinking…
• Rapid and agile insight delivery by
analytical models at point of decision
making, in data streams…
• Able to author and manage data
pipelines and predictive models
31. Evolving from Reactive to Proactive
Notify
Use preferred channels to notify
and insure receipt of information
Plan
Assess risk / options &
decide on course of action
Validate
Validate performance, document actions,
retrain models if warranted
Predict
Provide early warning of degradation with
priority assessment and fault indication
Act
Collaborate to secure resources
& execute work
Diagnose
Facilitate Collaboration between
analyst, subject matter experts &
service personnel
32. Provides analysis on driving style/behavior and predictive model for vehicle maintenance
Application: Driver Style Impact
Sources Inputs Analytic Components Outputs
Onboard Measurements • Speed
• Acceleration
• Deceleration
• Gear Changes
• Brake Pressure
• Steering
• Brake pressure
• Accelerometer
• Gear shift position
• Fuel
• Miles Driven
• Fuel Used
• CO2 Emissions
Aggregate Driver Behavior • Driving Techniques
• Age/gender /locality
/experience of Driver
• Speeding violations data
• Acceleration/deceleration
• Brake pressure
• Accelerometer
• Steering
• Eco Index
• Driving Behavior rating
• Projected savings if driving
behavior improves
Environment Conditions/Factors • Emission data
• Emission standard
• CO2 emissions
• CO ,HC, NOx emission
• Impact on environment
Vehicle study data • Driver behaviour
• Vehicle study data on driver
behaviour’s impact on vehicle
• Brake
• Accelerometer
• Gear
• Fuel
• Impact on vehicle, and
maintenance (i.e. you will have
to change brakes earlier than
scheduled due to poor driving)
- - DRAFT - -
33. Application: Preventive Alerts
Sources Inputs Analytic Components Outputs
OBU • Tire pressure
• Battery voltage
• Brake Pressure
• Vehicle acceleration
• Oil & other fluid levels
• Suspension
• Steering
• Air conditioning
• Engine data
• Lights & exhaust
• Sensor data
• Tire
• Battery
• Brake Disc
• Accelerometer
• Brake Fluid
• Oil & other fluid levels
• Suspension
• Steering
• Air conditioning
• Engine
• Lights & exhaust
• Sensors
• Component Name (Battery, Radio,
Tire Pressure, etc.)
• Status (Red, Yellow, Green)
• Problem Description
Historical aggregate vehicle data • Age of components( battery, tire,
etc)
• Age at which each component was
changed /service history
• Condition of the component
• Battery
• Tire
• Brake disc
• Component Name (Battery, Radio,
Tire Pressure, etc.)
• Status (Red, Yellow, Green)
• Display(will require replacement in 2
months)
Vehicle research data • Tire pressure effect on fuel
consumption and tire life
• Tire pressure • Display(2% increase in fuel
consumption due to under inflated
tire)
Environmental factors • Emission data
• Emission standard
• CO2 emissions
• CO ,HC, NOx emission
• Display(warning)
- - DRAFT - -
35. o A regression model was built to analyze
relationship between telematics data and
diagnostic outcome.
o More specifically, the model predicts the
probability that a vehicle component will
fail to function properly.
o Precise error detection: The model makes
use of several predictor variables (i.e.
telematic signals, non telematic data) that
may be either numerical or categorical.
o The model also determines which factors
(vehicle speed, brake velocity etc.) influence
more the probability of component failure.
Failure Estimation Model
Probability
Medium-Probability of
component failure
Low-Probability of
component failure
High-Probability
component failure
Signal Data
Application: Preventive Alerts
Real Time Diagnostic Model
39. Monitor, act on,
& log real-time
data.
Analyze and
model logged
data.
Mind the gap between investigate and operational analytics…
40.
41.
42. Event Stream both stored
and processed
1
Analysis produces models
2
Model can be installed directly to the event
processing service for operational analytics
3
Produce real time alerts
and actions based on
predictive models.
4
Event Stream
Analysis
Model
Event Processing
Engine
Alerts & Action
43. Velocity Pipeline enables data to flow across
an enterpriseinfrastructure and Internet spanning the
devices where valuable data is gatheredfrom employees and
customers, to the back-end systems where that data can be
translatedinto insightsand action
45. • Beecham Research: M2M World of Connected Things Sector Map
• ABI Research: Internet of Things, M2M , Wireless Sensor Networks
• Forrester:
• Internet of Things Reports
• Search for Internet of Things, M2M, Sensor
• Frost & Sullivan: Internet of Things, M2M, Sensor
• Gartner:
• Internet of Things and Internet of Things Blog Posts
• M2M and M2M Blog Posts
• Sensor and Sensor Blog Posts
• IDC: Search for Internet of Things, M2M
“Key elements of the IoT which are being embedded in a variety of mobile devices include embedded
sensors, image recognition technologies and NFC payment”. – Gartner (link)
“As billions of devices connect via the
Internet, exchanging information and
taking autonomous actions based on
continuous input, we will face a
paradigm change that will transform
our personal lives and revolutionize
business. These radical transformations
will pose unprecedented data privacy
and security challenges to security and
risk (S&R) professionals”. – Forrester (link)
46. • BusinessWeek: Internet of Things, M2M
• CIO.com: Internet of Things, M2M
• Computer World: Internet of Things, M2M
• Forbes: Internet of Things, M2M
• InformationWeek: Internet of Things, M2M
• InfoWorld: Internet of Things, M2M
• Venture Beat: Internet of Things, M2M
• Wall Street Journal: Internet of Things, M2M
• Wired.com Internet of Things, M2M
• ZDNet: Tapping M2M: The Internet of Things, M2M
Tapping M2M: The Internet of Things, by ZDNet
47. • Accenture: Toasters, refrigerators and Internet of Things
• Ericsson Labs: Internet of Things
• Harbor Research: Website
• HP: Implementing “The Internet of Things” in Your Business and
• IBM: The Internet of Things (video) and Internet of Things (IBM Academy of Technology paper)
• Information Builders: Internet of Things
• Infosys: Internet of Things: Endless Opportunities
• Intel: Simplifying the Internet of Things
• Microsoft: Building the Internet of Things
• Oracle: M2M Solutions: The Move to Value Creation and the Internet of Things
• SAP: The Ubiquitous Internet of Things: Managing Cities the Smart Way and Making The Internet
Of Things A Reality With Mobile Management
• Siemens: The Next Network
48. • Google Blog Search: Internet of Things / M2M / Sensors
• Google+ Communities Search: Internet of Things, M2M
• LinkedIn Group Search: Internet of Things, M2M and
Group: Sensor Networks
• Pinterest Search: Internet of Things, Machine to Machine
• Twitter hashtag searches: #IoT / #M2M / #sensors
• Tumblr Search: Internet of Things
• YouTube: Internet of Things Playlists and Internet of Things
Channel
• YouTube: M2M Playlists and Machine to Machine Channel
• Wikipedia: Internet of Things, M2M
Internet of Things Playlists on YouTube