5º #MeetUp 2016 de la ARQconf con Juan Caracoche, nuestro CTO para Latam, explicando qué es realmente #IoT y cómo funciona.

1. ©2014 GlobalLogic Inc. All Rights Reserved.
Juan Caracoche
juan.caracoche@globallogic.com
@jcaracoche

2. 2 ©2015 GlobalLogic Inc. All Rights Reserved.
IoT: What is it really and
how does it work?

3. 3 ©2015 GlobalLogic Inc. All Rights Reserved.
Similar quote attributed in various forms to
Nobel prize-winning physicists Albert Einstein
and Richard Feynman
“If you can’t explain something
to an intelligent twelve year
old, you probably don’t
understand it yourself.”

4. 4 ©2015 GlobalLogic Inc. All Rights Reserved.
Lego Mindstorm Robot courtesy of TM & © 2015 Marvel & Subs.

5. 5 ©2015 GlobalLogic Inc. All Rights Reserved.
+ “Brain”
+ Sensors
+ Actuators

6. 6 ©2015 GlobalLogic Inc. All Rights Reserved.
“Brain”
Cloud

7. 7 ©2015 GlobalLogic Inc. All Rights Reserved.
+ Sensors “Brain”
Cloud

8. 8 ©2015 GlobalLogic Inc. All Rights Reserved.
+ Sensors “Brain”
Cloud
+ Actuators

9. 9
IoT Architecture Paradigm
©2015 GlobalLogic Inc. All Rights Reserved.
Immediate Actions
(Alerts, Actuators,
Notifications)
End User and
Management Clients
Complex Actions
(Workflow)
Resources Platform Applications
External
Federated Data
Context
Historical
Data
Deep Insights
Fast Decision Making
Observations
and Requests
Connected Devices
and Systems
Action
Triggers
ActuatorsSensors
Sensor
s
Brain Actuators

10. 10
IoT Architecture Paradigm
©2015 GlobalLogic Inc. All Rights Reserved.
Immediate Actions
(Alerts, Actuators,
Notifications)
End User and
Management Clients
Observations
and Requests
Sensors
External
Federated Data
ConnectedDevices
and Systems
Action
Triggers
Complex Actions
(Workflow)
Actuators
Resources Platform Applications
Context
Historical
Data
Deep Insights
Fast Decision Making

11. 11
The Internet of Things
©2015 GlobalLogic Inc. All Rights Reserved.
A set of technologies that connects data from
a variety of sensors with the ability to make
contextually aware decisions and act on them
in near real-time

12. 12
IoT Architecture Paradigm: Sensor Technologies
©2015 GlobalLogic Inc. All Rights Reserved.
Resources
Sensors
Sensor
s
• Alljoyn
• MQTT
• HomeNET (Apple)
• Thread (Google)
• EMIT (Panasonic)
• Electric Imp
• CCNx (PARC)
• OIC / IIC (Intel)
Sensors

13. 13
Speed-layer Technologies:
• Spark Streaming
• Storm
• Samza
• IBM Streams
• Esper
• Sidhi CEP
• AWS Kinesis (Cloud based)
IoT Architecture Paradigm: Analytics Technologies
©2015 GlobalLogic Inc. All Rights Reserved.
External
Federated Data
Context
Historical
Data
Deep Insights
Fast Decision Making
Brain
Batch-layer Technologies:
• Hadoop Map-Reduce (Cloudera, Hortonworks,
MapR )
• HPCC
• Spark
• Hive
• Spark SQL
• Pig
• Analytic Databases (MarkLogic, Vertica, Infobright)
Fast Decision Making Deep Insights

14. 14
• Impala
• VoltDB
• ElephantDB
• SploutSQL
• Hive on Spark
• Hbase+Phoenix
• Couchbase
• MongoDB
IoT Architecture Paradigm: Analytics Technologies
©2015 GlobalLogic Inc. All Rights Reserved.
External
Federated Data
Context
Historical
Data
Deep Insights
Fast Decision Making
Brain
• Oracle Data Federation
• Redhat Data Virtualization
• Mule ESB + Custom code
• EspressoLogic
View Layer
Data Federation

15. 15
IoT Architecture Paradigm: Applications Technologies
©2015 GlobalLogic Inc. All Rights Reserved.
Immediate Actions
(Alerts, Actuators,
Notifications)
End User and
Management Clients
Complex Actions
(Workflow)
Applications
Connected Devices
and Systems
Actuators
Actuators
Technologies:
• MBaaS (Mobile Backend as a service
• mPaaS / aPaaS systems
• Workflow and Rules systems
• Pub / Sub
• Mobile app frameworks
• JS / Browser apps
• Sensors / “M2M” Technologies listed
Actuators and
Applications

16. 16 Internet of Things
Architectural Challenges
And Design Considerations

17. 17 CONFIDENTIAL
Architectural Challenges
Internet of Things
• Scalability
- Billions of devices and ZBs of data
• Interoperability
- Heterogeneous systems and distributed
resources
• Extensibility
- New things and new opportunities
• Resiliency
- Network partition and system errors
• Performance
- Near real-time processing and high
processing
• Security
- Privacy and Deceit
• Maintenance
- Resource and Platform

18. 18 CONFIDENTIAL
Design Considerations
Internet of Things
• Identity Management
- Identity of things and identity of people
• Context awareness
- Location and identity
• Data generation mode
- Event driven and time driven
• Resource capability
- Processing power and memory size
• Forward and backward capability

19. 19 CONFIDENTIAL
Reference Architecture and Platform
Internet of Things
External Federated Data
z
z z
M2M, Devices, and
Intelligent Gateways
Fast Decision Making
Historical Information
Notifications
Deep Insights Complex Actions
Platform Functions
External Federated Data
End User and
Management Clients

20. ©2015 GlobalLogic Inc. CONFIDENTIAL
Embedded systems in IoT

21. CONFIDENTIAL21 Internet of Things
• Secure communication capabilities
• Interoperability means talking to each other
• Sensing/interacting embedded technology
IoT = network of
physical objects

22. 22 Internet of Things
Real world interaction is
provided by sensors and
actuators
• Light, pressure, gyro,
movement, temperature, etc.
sensors
• MEMS, motors, valves,
relays, etc.
• Machine vision, human
interaction and other systems
Communication
capabilities are
provided by protocols
CoAP
Well-defined and
secure interoperability
means:
• AAA, provisioning and network
management
• Inter-system capabilities
negotiations and configuration
• Cloud/edge/fog optimal work
distribution
• Security, security and also
security

23. 23 Internet of Things
• SSL/TLS based security
(certificates)
• Flexible transport plugins –
AllJoyn or MQTT or REST or
…
• Transport-independent initial
provisioning mechanism
• Capabilities negotiation
between all endpoints based
on GUIDs
• “Edge” and “Fog” computing
concepts support
Security Transport
AllJoyn MQTT REST …
Provisioning
Capabilities negotiation
IFTTT rules & computing
Data flow
Vision of embedded systems in IoT

24. 24 CONFIDENTIAL
Definitions
Internet of Things
• Sensor/Actuator
- Pressure sensors, camera, stepper motor, etc.
• Controller
-Collect, process and possibly store data,
implement some custom IFTTT or other logic
-Can have a set of Sensors or Actuators
connected to it
-Can talk to other Controllers or M2M Gateway
• M2M Gateway
- Provides Internet access to one or some
set of Controllers
- Can be smart enough to implement some
custom IFTTT or other logic

25. 25
Device
Gateway PlatformController M2M Gateway
Request
certificate signing
M2M transport connection
Sign GW
certificate
with Private
Key
Create private key
by serial
Create
platform certificate
Return
signed certificateCreate private key
by serial
Request certificate signing
Sign CTL
certificate
with Private
Key
Return signed certificate
Controller-Gateway certificate exchange
(Transport authentication mechanism)
Device registration process
Gateway and Controller are authorized by the Platform, securedconnectionis established
Provisioning

26. 26
Device Gateway Platform
Controller M2M Gateway
Authentication
Check
certificate
authorize
data processing
actions
Connection → announce DeviceInfo + certificate
Device Authorized,setResourceID
getData events
setData events
Message loop
Announce capabilities
Update Device Info
AAA

27. ©2015 GlobalLogic Inc. CONFIDENTIAL
Platform Core

28. 28 CONFIDENTIAL
Reference Architecture and Platform
Internet of Things
External Federated Data
z
z z
M2M, Devices, and
Intelligent Gateways
Fast Decision Making
Historical Information
Notifications
Deep Insights Complex Actions
Platform Functions
External Federated Data
End User and
Management Clients

29. 29 CONFIDENTIAL
Platform Core | Cloud
Internet of Things
Heterogeneous
Data
Large
Volume
Near Real-Time
Processing
Security Notifications

30. 30
Heterogeneous Data Solutions
Internet of Things
• Transform any format of data to ONE format
• Apply basic rules
• Cache & Aggregate data
• Platform Gateways

31. 31
Large Volume Solutions
Internet of Things
• Fast Hundreds of MBs of RW per second,
serialize/deserialize
• Scalable Elastically and transparently
expanded without downtime
• Durable persisted and replicated to prevent
data loss
• Distributed fault-tolerance guarantees

32. 32
Near Real-Time Processing
Internet of Things
• Low Latency persisted and replicated to prevent
data loss
• Scale High throughput and scale-out by adding
instances
• Dynamic Change the rules dynamically using API
interface
• Complex detecting relationships between events

33. 33
Security Solutions
Internet of Things
• Privacy Multi tenancy and data sharing
• Security Policies apply security policies on
the data before storing
• In Memory Process non-persistable data in
memory
• Transport Secure data transfer to the DMZ

34. ©2015 GlobalLogic Inc. CONFIDENTIAL
IoT Analytics

35. 35 CONFIDENTIAL
Reference Architecture and Platform
Internet of Things
External Federated Data
z
z z
M2M, Devices, and
Intelligent Gateways
Fast Decision Making
Historical Information
Notifications
Deep Insights Complex Actions
Platform Functions
External Federated Data
End User and
Management Clients

36. 36 Internet of Things
The Internet of Things Analytics(IoTA) , the measurement and
transformation into business intelligence of the Internet of Things
Data Paradigms
Volume
application and
transactional
data
Velocity Machine Variety
Social Data

37. 37 CONFIDENTIAL
IoT Analytics Challenges
Internet of Things
• Data Integration
– Aggregation and integration of the collected data streams in a manner that makes them suitable for
analysis.
– Key challenge in dealing with more diverse and disparate data sources before meaningful analysis is
viable.
• Data Storage
– Implementation and management of the data store for the analytics process.
– Key issue to address in this context is how to store time-series sensor data, which can increase
dramatically in volume compared to e.g., transactional data readings
• Different components of IOT analytics

38. 38 CONFIDENTIAL
IoT Analytics Challenges (Cont’d)
Internet of Things
• Core Analytics
– Processing of the data by an analytics engine and the subsequent delivery of insights.
• Data Presentation
– Further presentation of the delivered analytical insights to the end-user.
– In IoT analytics, the geography of data is a particularly important presentation element, given
that the location of physical things matters more to analysis than, say, the location of an
ecommerce transaction.
• Different components of IOT analytics

39. 39 CONFIDENTIAL
Enabling IOT Analytics using Lambda Architecture
Internet of Things
Components of Lambda Architecture
new data
batch layer serving layer
speed layer
Query
Query
master data set
batch view
batch view
real time view
real time view

40. 40
Technology Stack
Internet of Things
Component Technology
Choices
Rationale
Batch View Generator (Batch
Layer)
Hive on Spark ,
Impala
•Queries, especially those involving multiple reducer stages, runs faster.
• Can anytime move to using only Hive by just a JDBC connection parameters change.
• Easy of use than Map Reduce and other processing framework.
• Support SQL by default.
Real Time View Generator (Speed
Layer)
Volt DB , Spark
Streaming ,Storm
•Processes data fast by keepingit in memory and runs on commodity hardware.
•Data can be ingested directly from Kafka queues so would be on low latency as thereis
no intermediate layer betweendataingestion.
Consolidate View Generator
(Serving Layer)
Hbase + Phoenix ,
Couchbase
•Is easy to use and provides real time read and write.
•Good for time series based evaluation.
• Phoenix is used to provide SQL support over hbase
Analytics Engine(Statistical
programming language)
R , Mahout •Gives wide variety of statistical and graphical algorithm.
•It has linear and non linear modeling, time series analysis, classification,clustering etc.
Real time Machine Learning
(Machine Learning Library)
SparkML •Distributed machine learning library.
• Has support for common algorithms.
Data Ingestion Layer (Messaging
System)
Kafka , RabbitMQ •High Throughput , data canbe pulledat consumer's own pace.
•Can replay any messages or set of messages given the necessary selection criteria.

41. 41
IOT Analytics Industry Trends
Internet of Things
Informatica Vibe Data Stream for
Machine Data
ParStream IOT Analytic Framework
Source- http://www.informatica.com/in/products/big-data/vibe-data-stream/, https://www.parstream.com/

42. 42
IOT Analytics Industry Trends
Internet of Things
Source- http://voltdb.com/

43. ©2015 GlobalLogic Inc. CONFIDENTIAL
Applications in IoT

44. 44 CONFIDENTIAL
GlobalLogic Reference Architecture and Platform
Internet of Things
External Federated Data
z
z z
M2M, Devices, and
Intelligent Gateways
Fast Decision Making
Historical Information
Notifications
Deep Insights Complex Actions
Platform Functions
External Federated Data
End User and
Management Clients

45. 45 CONFIDENTIAL
“There’s an app for that”
– Apple’s advertising tagline, circa 2011
With the brain layer
disappearing and becoming
part of the background, how
do we build apps for IoT?

46. 46 CONFIDENTIAL
What are the Challenges?
Internet of Things
The Past The Present The Near Future The Future
Concept from Jeffery S. Engelhardt Courtesy Sony Courtesy Nokia / Issam Trabelsi
Courtesy bushlemon.deviantart.com
A rapidly changing ecosystem

47. 47 CONFIDENTIAL
What are the Challenges?
Internet of Things
Very high fluctuation in usage

48. 48 CONFIDENTIAL
What are the Challenges?
Internet of Things
Ability to handle all types of use cases
Control home appliances
(Connected Home)
Gather meteorological data from
sensors for weather forecasting
Communicate to an aero
engine manufacturer that
this engine needs servicing

49. 49 CONFIDENTIAL
What are the Challenges?
Internet of Things
A completely new UX paradigm
How do we do UX for
agriculture?
Will apps on TVs
require a new UX?
How will appliances work in
a vehicle?

50. 50
What are the Challenges? The Future is an Unknown
Internet of Things
2020
2010
2000
1990
1980
1970
1960
1950
1944
1 10 1000s 100s of 1000s 1 Million 100s of Millions 1 Billion 8 Billion 50 Billion
One
Ten
Thousands
Hundreds of Thousands
One Million
Hundreds of Millions
Billion
8 Billion
50 Billion

51. 51 CONFIDENTIAL
What are the Challenges?
Internet of Things
Being comfortable with legacy

52. 52 CONFIDENTIAL
What are the Challenges?
Internet of Things
Lack of a default standard
What will become the default standard is just a guess for now

53. 53 CONFIDENTIAL
Summary
Internet of Things
With challenges of
this kind, it is best to
build IoT applications
on a platform
So: how do we do it,
and what would an
application platform
look like?

54. 54 CONFIDENTIAL
Plenty of Platforms Already Exist
Internet of Things
Xively There are plenty more
being developed . . .
And all of them do
ingestion and analytics
ThingWorx
Mnubo
Bug Labs

55. 55
App Platform Feature Set
Internet of Things
Challenges Solutions
• A rapidly changing ecosystem
• Extreme difference in consumption based on success
• Ability to handle all types of use cases
• A completely new UX paradigm
• The future is an unknown
• Being comfortable with legacy
• A lack of standardization
• Use of Auto-update like features (similar to Google’s Chrome
• Have a plug n play approach to building applications
• Use of cloud and cloud patterns to handle varying levels of
consumption and to make this scalable and fault tolerant and
highly available
• Have the ability to Integrate with all existing systems via an ESB
• Ability to incorporate new devices displays in a seamless
manner by separating out the rendering out of the view from the
data and the view
• Build once, Reuse everywhere, Continuously update
• Follow the Rapid Application Delivery (RAD model)
With the right Platform as your partner and with the right vendor, go to market would be reduced by months

56. 56 CONFIDENTIAL
Proposed Architecture of the Application Platform
Internet of Things
Views
AppEngineSDK
AppRenderer
WebApps
(MV*)
Native
App
Custom
AppDesktops /
Laptops / Tablets
Smart Phones
Devices &
Gateways
Business Logic
AppPlatformAPI
Worker
VM
(Task A)
Worker
VM
(Task B)
Worker
VM
(Task N)
Orchestration
Manager
Integration ESB
Integration layer
(3rd Party Adapters)
Auto Scale
Manager
PlatformAccessLayer
App DB
Platform Tier
Data
Services
Streaming
Service
Platform
Config
Services
Analytics
Services
Platform Management
…

57. 57 CONFIDENTIAL
Proposed Architecture of the Application Platform
Internet of Things
Views
AppEngineSDK
AppRenderer
WebApps
(MV*)
Native
App
Custom
AppDesktops /
Laptops / Tablets
Smart Phones
Devices &
Gateways
Business Logic
AppPlatformAPI
Worker
VM
(Task A)
Worker
VM
(Task B)
Worker
VM
(Task N)
Orchestration
Manager
Integration ESB
Integration layer
(3rd Party Adapters)
Auto Scale
Manager
PlatformAccessLayer
App DB
Platform Tier
Data
Services
Streaming
Service
Platform
Config
Services
Analytics
Services
Platform Management
…
Custom built once
for each stack
Custom built once
for each stack
Vert.x / Node.js / Web
API
Polyglot DB
Polyglot
Programming
Custom / Drools /
WWF
Mule / Neuron /
Fuse / etc.
Vert.x / Node.js / Web
API

58. 58 CONFIDENTIAL
Reference Architecture and Platform
Internet of Things
External Federated Data
z
z z
M2M, Devices, and
Intelligent Gateways
Fast Decision Making
Historical Information
Notifications
Deep Insights Complex Actions
Platform Functions
External Federated Data
End User and
Management Clients

59. ©2014 GlobalLogic Inc. All Rights Reserved.