Presentation of Patric Uebele at Munick Wearable Data Hack

1. © 2015 MapR Technologies, confidential
®
®
Patric Uebele
15-06-21

2. © 2015 MapR Technologies, confidential
®
© 2015 MapR Technologies, confidential
®
Business Intelligence and the IoT

3. © 2015 MapR Technologies, confidential
®
The value of BI
businessvalue
reporting
what happened?
OLAP
why did it happen?
real-time analytics
what and why is it happening, now?
predictive analytics
what might happen?

4. © 2015 MapR Technologies, confidential
®
What is the Internet of Things
Wearables
Connected Homes
Industrial Internet
Connected Cities
Connected Cars

5. © 2015 MapR Technologies, confidential
®
IoT—a superset of the Internet
thin fat
stationary
mobile
Devices and their deployment

6. © 2015 MapR Technologies, confidential
®
© 2015 MapR Technologies, confidential
®
IoT use case categories and apps

7. © 2015 MapR Technologies, confidential
®
Personal loT
wearables
smart phones
clothes
Industrial loT
smart factories
agriculture
retail
manufacture
Group loT
vehicles
smart houses
tourism
education
Community loT
smart cities
smart roads
parks

8. © 2015 MapR Technologies, confidential
®
Internet of Things is Here Now and Growing
The New Essential Infrastructure

9. © 2015 MapR Technologies, confidential
®
Operational Efficiency
Waste
Management
• 20K trucks, 30M GPS pings a day
• Integrate 17 different systems that have
operational significance related to driver, vehicle,
and route performance management
• Over 1.6M labor hours saved

10. © 2015 MapR Technologies, confidential
®
Self Driving
Cars Sensors Collaborate on the Road
Google’s Self-driving car generates 13 million laser
measurements and calculates 20 driving decisions
EACH second

11. © 2015 MapR Technologies, confidential
®
© 2015 MapR Technologies
®
Connected Car Use Case I – Driver
Profile and Automobile Status

12. © 2015 MapR Technologies, confidential
®
APPROACH
WHAT WE WANTED AND HOW WE STARTED
§ Open source project
§ Raspberry Pi based python script
§ Pre programed OBD Mapping
§ Bluetooth OBD-II connection integration
First approach
Solution: OBDPi
§ Access Motor data through OBD-II connection
§ Log key data points locally
Goal
Gain insights about driver profile and automobile status through telematics

13. © 2015 MapR Technologies, confidential
®
OBT-II ACCESSING THE STATUS OF VARIOUS VEHICLE SUB-SYSTEMS
§ Required to be within 2 feet of steering wheel
§ Required to be built into all cars sold in California since
1991
§ Supplies Vehicle and sensor information to technicians
§ Supply diagnostic trouble codes (DTC)
§ OBD-II replaced the OBD standard in 1996
§ 16 Pin Connector
On Board Diagnostics-II

14. © 2015 MapR Technologies, confidential
®
RASPERRY PI
CREDIT CARD-SIZED SINGLE-BOARD COMPUTER
§ Multi-functional microcontroller development board
§ Size of a credit card
§ Linux based OS Capabilities (Raspbian-Debian variant)
§ Initially created to promote computer science in schools
§ Also can be used as powerful development board
§ Features include:
– CPU: 700 MHz ARM Microcontoller
– GPU: Boradcom VideoCore IV @ 250 Mhz
– 512 MB Memory (SDRAM)
– 4 USB ports
– 46 GPIOS (Model B+)
RAM
I/O CPU/GPU USB hub
Ethernet
2x USB

15. © 2015 MapR Technologies, confidential
®
PY-OBD
JOINING RASPERRY PI WITH OB-II
PY-
OBD
Rasperry
Pi
OBD-II
§ Py-OBD is a project from GitHub User PeterH
§ Uses Data from OBD-II Port to display current Driving data
§ Project is GNU General Public Licensed and intended to be shared, modified and
improved

16. © 2015 MapR Technologies, confidential
®
SYSTEM OVERVIEW
RASPBERRY PI DATA ACQUISITION
Car
OBD-II Dongle
Raspberry Pi B +
WiFi adapter for peripherals
GPS

17. © 2015 MapR Technologies, confidential
®
HOW TO GAIN GREATER FUNCTIONALLITY
1
2
3
4
Stream Data into MapR Enterprise Database Platform using Fluentd
Data Storage and Manipulation with MapR Cluster
Data Aggregation with open source Apache Drill beta
Stream visualization with Grafana (real time time series with openTSDB on top of
MapR-DB)
4
5 QlikView data summary visualization and Adhoc with Apache Drill SQL based view

18. © 2015 MapR Technologies, confidential
®
SYSTEM OVERVIEW
STREAM PROTOCOL
Stream ProtocolData Acquisition Data Store & Analytics Data Visualization
Raspberry PI
OBD2
fluentd fluentd
Grafana
QlikView
real-time dashboard
flat-files
OLAP
Ad-Hoc
Realtime

19. © 2015 MapR Technologies, confidential
®
FLUENTD
TREASURE DATA BASED STREAM PROTOCOL
§ Unified logging layer
§ Fluentd allows you to unify data collection and consumption for
a better use and understanding of data.
§ Plugin library allows for a variety of applications and platform
use (more than flume)
§ Easier to Configure (rpm and start/stop scripts | ruby code for
quick patches without the need for compilation)
§ Simple encryption plugins allow for the secure streaming of
data
§ Only requires 40 MB of memory and can process 13,000
events/second/core
Rasberry PI
IN
Rasberry PI
Out
MapR Cluster
OpenTSDB
on MapR-DB
MapR-FS
tail_in plugin out_forward_plugin or
out_forward_secure_plugin
in_forward_plugin or
in_forward_secure_plugin
out_file_plugin
opentsdb_plugin
out_copy

20. © 2015 MapR Technologies, confidential
®
Connecting Qlikview for Adhoc Queries using Apache Drill
Node Node Node Node
MapR-Cluster
Mapr-FS Mapr-FS Mapr-FS Mapr-FS
Drillbit Drillbit Drillbit Drillbit
Drill Windows
ODBC Driver
CSV Files
from Fluentd
Drill Explorer
Create View with
Datatypes
Access View
from Qlikview
ODBC
NFS
Data Analyst Windows Qlikview Server

21. © 2015 MapR Technologies, confidential
®
PRACTICE
DATA VISUALIZED IN OUR DASHBOARD
First incoming data
QlikView Dashboard

22. © 2015 MapR Technologies, confidential
®
Connecting Grafana to Opentsdb
Node Node Node Node
MapR-Cluster
Mapr-DB Mapr-DB Mapr-DB Mapr-DB
OpenTSDB OpenTSDB OpenTSDB OpenTSDB
Grafana
Fluentd
Rest
Opentsdb
plugin

23. © 2015 MapR Technologies, confidential
®
PRACTICE
Realtime View using Grafana
Realtime stats using velocity,
tachometer, load

24. © 2015 MapR Technologies, confidential
®
© 2015 MapR Technologies
®
Connected Car Use Case II – Including
Google Nest

25. © 2015 MapR Technologies, confidential
®
APPROACH
WHAT WE WANTED
§ High Arrival Data with Millions of Data Points per Second
§ Analyzing Car Location Data in Realtime
§ Start Heating and Cooling based on distance measuring between the car to home and to work
First approach
Solution: MapR-DB / OpenTSDB
§ Access Motor data through OBD-II connection
§ Log key data points locally
Goal
Optimize Heating and Cooling Costs in house by maschine to maschine communication

26. © 2015 MapR Technologies, confidential
®
What is Google Nest?
Nest Learning Thermostat learns your schedule, programs itself
Nest could be controlled remotely
Nest Thermostat can lower your heating and cooling bills up to 20%

27. © 2015 MapR Technologies, confidential
®

28. © 2015 MapR Technologies, confidential
®

29. © 2015 MapR Technologies, confidential
®
MapR Distribution including Apache Hadoop
MapR Data Platform
(Random Read/Write)
Data HubEnterprise Grade Operational
MapR-FS
(POSIX)
MapR-DB
(High-Performance NoSQL)
NFS HDFS API HBase API JSON API
MapRControlSystem
(ManagementandMonitoring) Security
YARN
Pig
Cascading
Spark
Batch
Spark
Streaming
Storm
Streaming
HBase
Solr
NoSQL &
Search
Juju
Provisioning
&
Coordination
Savannah
Mahout
MLLib
ML, Graph
GraphX
MapReduce
v1 + v2
APACHE HADOOP AND OSS ECOSYSTEM
EXECUTION ENGINES DATA GOVERNANCE AND OPERATIONS
Workflow
& Data
GovernanceTez
Hive
Impala
Spark SQL
Drill
SQL
Sentry Oozie ZooKeeperSqoop
Knox* WhirrFalcon*Flume
Data
Integration
& Access
HttpFS
Hue

30. © 2015 MapR Technologies, confidential
®
MapR Distribution for Hadoop
Theme Requirements Features Product
Enterprise Grade
• Uptime service levels
• Site to site DR & Business
Continuity
• Backup/recovery
• Security
• High velocity data ingress
• HW/SW HA
• Mirroring & Promotable
Mirrors
• Snapshots
• Authorization, Kerberos
• 2X-5X performance
MapR
Enterprise Edition
Data Hub
• Hadoop
• Traditional applications
• Data of record
• Batch and interactive
• HDFS
• POSIX
• Strong consistency
• MapReduce and SQL
MapR
Enterprise Edition
Operational
• Real time
• NoSQL
• Operational analytics
• HBase
• Update in place
• Concurrent read/write
• Timeseries
MapR
Enterprise Database Edition
MapR Patent Pending – “Table Format for Map Reduce”
“Map Reduce Ready Distributed File System”
Enterprise Grade
Operational
Data Hub

31. ®
© 2015 MapR Technologies 31
Global Real-time Synchronization
Multi-master (aka, active/active) replication
Active Read/Write
End Users
• Faster data access – minimize network
latency on global data with local clusters
• Reduced risk of data loss – real-time,
bi-directional replication for synchronized
data across active clusters
• Application failover – upon any cluster
failure, applications continue via
redirection to another cluster
®
© 2015 MapR Technologies 31

32. © 2015 MapR Technologies, confidential
®
MapR Data Protection Advantages for IoT
Advanced Business Continuity with Industry’s only mirroring &
point-in-time consistent snapshots

33. © 2015 MapR Technologies, confidential
®
MapR Security Advantages for IoT
Strongest authentication, most flexible authorization, most powerful
auditing

34. © 2015 MapR Technologies, confidential
®
Q&A
@mapr maprtech
puebele@mapr.com
MapR
maprtech
mapr-technologies
®
© 2015 MapR Technologies 34