Designing and Implementing your IOT Solutions with Open Source

1. © 2016 MapR Technologies 1© 2016 MapR Technologies
Designing & Implementing your IOT Solutions with Open Source

2. © 2016 MapR Technologies 2
Agenda
• Use Case
• Demo
• Layers
– Sensor
– Controller
– Protocol
– Messaging
– Computation
– Persistence
– Dashboard

3. © 2016 MapR Technologies 3© 2015 MapR Technologies
Use Case – Temperature Monitoring

4. © 2016 MapR Technologies 4
Use Case – Temperature monitoring dashboard

5. © 2016 MapR Technologies 5
Do I need big data for IOT ?
Sensor reads temperature once every second
• Number of readings per year
3600*24*365 = 31,536,000 (31 million)
• Every reading is 100 bytes
31,536,000 * 100 = 3,153,600,000 (3.1 GB)
• Every store has 50 aisel(4 sensors) and 6000+ stores
200 * 6000 = 1,200,000 (1.2 million sensors)
3 * 200 * 6000 = 3,600,000 (3.6 PetaBytes)

6. © 2016 MapR Technologies 6© 2015 MapR Technologies
Demo

7. © 2016 MapR Technologies 7
Demo – Data Flow

8. © 2016 MapR Technologies 8
Demo – Data Flow

9. © 2016 MapR Technologies 9
IOT Big Data Reference Architecture

10. © 2016 MapR Technologies 10© 2015 MapR Technologies
Sensors/Actuators

11. © 2016 MapR Technologies 11
Why do we need Sensor/Actuator Layer ?
Sensor/actuator will be responsible for
1. Interacting with physical world
1. Sensor – Measures or detects
physical property
1. Thermistor
2. Temperature Oscillators
2. Actuator – Move or control mechanism
or system
1. LED
2. Servo Motor

12. © 2016 MapR Technologies 12
Temperature Sensors
DS18B20 DHT22

13. © 2016 MapR Technologies 13
Sensor Pin Layout
DS18B20 DHT22

14. © 2016 MapR Technologies 14
Actuator Pin Layout

15. © 2016 MapR Technologies 15
Home Automation

16. © 2016 MapR Technologies 16
IOT Home Automation
You can build home automation
1. Temp Monitoring – For AC
2. Auto turning of lights on and off
3. Sprinkler System
4. Primitive home security
5. Gas detection

17. © 2016 MapR Technologies 17© 2015 MapR Technologies
Device

18. © 2016 MapR Technologies 18
Why do we need Device Layer ?
The controller layer is responsible for
1. Reading data from sensor
2. Transform sensor data into
message
3. Communicate with IOT Gateway
using suitable protocol
4. Listening to message from IOT Gateway
5. Activating with actuators based on message from Gateway

19. © 2016 MapR Technologies 19
Arduino
Arduino Uno
NodeMCU V2 –ESP8266

20. © 2016 MapR Technologies 20
Arduino Application

21. © 2016 MapR Technologies 21
Install Sensor Library

22. © 2016 MapR Technologies 22
Temperature sensor – Sensor Library

23. © 2016 MapR Technologies 23
Temperature sensor – Start Sensor

24. © 2016 MapR Technologies 24
Arduino – Read temperature

25. © 2016 MapR Technologies 25
Upload code to Arduino

26. © 2016 MapR Technologies 26
Raspberry Pi

27. © 2016 MapR Technologies 27
Raspberry pi - Logic for reading temperature

28. © 2016 MapR Technologies 28
Raspberry Pi – Development Life Cycle

29. © 2016 MapR Technologies 29
IOT Device comparison
Arduino Raspberry Pi
Architecture Micro controller Micro Processor
Operating System No Yes
Power Can go to sleep Continuous 5V
Network No Yes
Sensor Analog + digital Digital
Development
Environment
Arduino IDE – C code Java, Python,… etc

30. © 2016 MapR Technologies 30© 2015 MapR Technologies
Protocols

31. © 2016 MapR Technologies 31
Why do we need Protocol Layer ?
The protocol layer is responsible for
1. Communication between IOT Device and
IT infrastructure
2. The IOT Devices operate in constrained
environment
1. Limited processing power
2. Limited amount of memory (Less than 100K)
3. Network usage
1. Less Reliable network
2. Optimized usage of network
4. Operate on battery
1. Sleep mode

32. © 2016 MapR Technologies 32
MQTT – Message Queue Telemetry Transport
MQTT is publish/subscribe messaging protocol designed for lightweight
M2M communication.
• Open: MQTT is OASIS standard created by IBM
• Lightweight Client: Client libraries less than 30 KB
• Optimized Network usage: Small packet size 2 bytes (header)
– TCP Based. MQTT-SN is UDP based
– Easy to parse packets
• Reliable: Three QoS patterns to avoid packet loss on client
disconnect
– At-most-once – does not survive failures, but is never duplicated
– At-least-once – survives connection loss and other failures, but can be
duplicated
– Exactly-once – survives connection loss and other failures, and is never
duplicated

33. © 2016 MapR Technologies 33
MQTT vs. HTTPS
1. Performance is 80 times better
2. Battery usage 10 times less
http://stephendnicholas.com/archives/1217

34. © 2016 MapR Technologies 34
MQTT
Device 1
Device 2
Device 3
MQTT
Broker
Subscriber 1
Subscriber 2
/store/fremont/aisel1
/store/fremont/aisel2
/store/sanjose/aisel1
/store/fremont/#
/store/#

35. © 2016 MapR Technologies 35
MQTT – Server & Broker
Subscriber 1
Subscriber 2
/store/fremont/aisel1
/store/fremont/aisel2
/store/sanjose/aisel1
/store/fremont/#
/store/#

36. © 2016 MapR Technologies 36
MQTT – Include MQTT client library

37. © 2016 MapR Technologies 37
MQTT – Setup MQTT Connection

38. © 2016 MapR Technologies 38
Arduino – Send Receive Message

39. © 2016 MapR Technologies 39
Arduino – Memory usage

40. © 2016 MapR Technologies 40
CoAP – Constrained Application Protocol
The CoAP is request/response protocol similar to HTTP + REST
• Open: CoAP is IETF standard - RFC 7252 IETF
• Lightweight Client: Client libraries less than 30 KB
• Optimized Network usage
– Based on UDP (6LowPAN), SMS in addition to TCP
– Compact 4 bytes header, compressed
– CoAP uses familiar HTTP stuff like Method (Get, Put, Post, Put, Delete), status
codes, URIs, content types/mapping
– Can be used to carry different type of data. It uses content type to identify type
of data being carried
– HTTP Proxy can be used to map COAP to HTTP
• Reliable: The HTTP proxy takes care of reliability

41. © 2016 MapR Technologies 41
Arduino – Read Temp. + Publish MQTT Message
http://slideplayer.com/slide/9365664/

42. © 2016 MapR Technologies 42
COAP
Device1
Device 2
Device 3
HTTP
Proxy
Subscriber 1
Subscriber 2
/store/fremont/aisel1
/store/fremont/aisel2
/store/sanjose/aisel1 coap://192.168.1.2:5683:/store/sanjose/aisel1
http://192.168.1.2:5683:/store/fremont/store1

43. © 2016 MapR Technologies 43
Arduino – Read Temp. + Publish COAP Message

44. © 2016 MapR Technologies 44© 2015 MapR Technologies
Messaging

45. © 2016 MapR Technologies 45
Why do we need Messaging Layer ?
The Messaging layer is responsible for
1. Highly available MQTT
1. Scale MQTT layer horizontally
2. Provide high availability
2. Decouple IOT layer from the
computation layer
3. Allow multiple consumers to process the IOT data to meet
different business requirements

46. © 2016 MapR Technologies 46
MQTT load balancing – No Kafka
Device
Device
Device
Device
MQTT
MQTT
MQTT
Spark
Spark
Spark

47. © 2016 MapR Technologies 47
MQTT load balancing - Kafka
Device
Device
Device
Device
MQTT
MQTT
MQTT
Spark
Spark
Spark
Kafka

48. © 2016 MapR Technologies 48
MQTT Kafka Bridge
MQTT Kafka
Bridge 1
Bridge 2

49. © 2016 MapR Technologies 49© 2015 MapR Technologies
Computation

50. © 2016 MapR Technologies 50
Why do we need computation layer ?
The computation layer is responsible for
1. Process huge amount of data in linear
scalable manner
1. Read streaming data
2. Transform/Enrich/Analyze data
3. Store the incoming data into persistence storage
2. Analyze IOT data
1. Business Rules
2. Predictive model

51. © 2016 MapR Technologies 51
Spark Streaming

52. © 2016 MapR Technologies 52
Business rules

53. © 2016 MapR Technologies 53© 2015 MapR Technologies
Storage

54. © 2016 MapR Technologies 54
Why do we need Storage Layer ?
The storage layer is responsible for
1. Providing persistent store for IOT
data
2. Store data in time series format
1. Efficient storage of data
2. Ingest data at high speed
3. Retrieve/Query data

55. © 2016 MapR Technologies 55
Time Series Database
• Data Point
(time, temperature)
• Metrics
sensor.temp.1s
• Tags
sensorId=2, location=bed1
• Metrics + Tags = Time Series

56. © 2016 MapR Technologies 56
InfluxDB

57. © 2016 MapR Technologies 57© 2015 MapR Technologies
Dashboard

58. © 2016 MapR Technologies 58
Why do we need Dashboard Layer ?
The dashboard layer is responsible for
1. Interact with TSDB
2. Visualize TSDB data in easy to
understand format

59. © 2016 MapR Technologies 59
Configure InfluxDB connection

60. © 2016 MapR Technologies 60
Temperature Variation Graph

61. © 2016 MapR Technologies 61
Temperature Variation Graph

62. © 2016 MapR Technologies 62
Resources
• Streaming Processing for Real-Time Analytics and Dashboard
• MapR Streams
• MapR DB
• Eclipse IOT
• COAP Tutorial
• GitHub For source code

63. © 2016 MapR Technologies 63© 2015 MapR Technologies
Questions ?
Sunil Patil, MapR Technologies
sunilpatil@mapr.com

64. © 2016 MapR Technologies 64
Arduino - Logic for reading temperature