3. Sensor Network
• Composed of
• Microcontroller units (MCU)
• Various sensors
• Temperature
• Humidity
• Photocells
• Web API
• Control MCUs
• Message Broker
• Route messages
• Manage communication channels
• Manage ques
4. The Ideal MCU
• Size
• Small form factor
• Power
• Low Power Consumption
• Battery Based
• Sleep Modes
• GPIO
• Digital
• Analog
• Communication
• Wifi 802.11 b/g/n
• Bluetooth
• Cellular 2G/3G
• Software
• Developed support
• Functionality Libraries
• Language Support
• C
• Javascript
5. Our Pick: ESP8266-12e
● Size
○ 24mm x16mm x 3mm
● Power
○ 3v3
○ Sleep mode
● Full IP Stack
○ WIFI 802.11 b/g/n
○ Can act as client or host
● Low Cost
○ $1.50
● Software
○ Several firmata available
○ Huge online community
https://acrobotic.com/media/wysiwyg/products/esp8266_esp12e_horizontal-01.png
6. Web API
• Web API
• Supervises MCUs
• MCUs initialization
• Connect to API Web Server
• Over Websockets
• Enter REPL
• Read Evaluate Print Loop (REPL)
• Inject Code directly into device
• Keep Status of Devices
• Modes
• Sleep
• On
• Off
7. Message Queue Telemetry Transport
(MQTT)
• MQTT: Lightweight Comm. Protocol
• Over TCP/IP
• Topic
• Data stream subject/identifier
• Publisher
• Transmits data on select topic
• Active connection
• Subscriber
• Listens for data on select topic
• Passive connection
• MQTT Broker
• Routes traffic based on subscriptions
8. Message Broker
• Intermediary
• Between sender and receiver
• Manages
• Delivery
• Routing
• Message Queue
• Protocol Conversion
• Message translation
Topic Subscriber Publisher
Temperature Server Temp. Sensor
Feedback Server Thermostat
mn … m4 m3 m2 m1 m0
m0 m1 m2 m3 m4 … mn
Temp. Sensor
Thermostat
Message Broker
Topic: Feedback
Topic: Temperature Server Side
Calculations
Message Queues
9. RabbitMQ
• Cross Language Support
• Java, Python, JavaScript, Ruby, & .NET
• Cross Protocol Support
• MQTT, AMQP, HTTP, & STOMP
• Asynchronous Messaging
• Many subs
• Many pubs
• QoS
• Data persists in queues until read by subscriber
• Low Latency
• Critical for real time apps.
10. Alternative Input Devices
• ROS Robots
• Drones
• Rovers
• Multi Input, Multi Output Systems
• Sensor + Actuator Pairs
12. What is Kura?
● OSGI based framework for IOT Gateways
● Runs in a JVM
● Built-in MQTT cloud services
● Browser based GUI
13. Key Features-Network and Cloud Services
● Extensive MQTT configuration options
● Helps implement more complex interaction flows beyond publish/subscribe
● Remote management of M2M applications
14. Key Features-Configurable Services
● 3 ways to add service packages:
○ Eclipse Marketplace
○ URL
○ Uploaded files.
● Can be configured during runtime through the web GUI
● Can access device hardware: GPIO, GPS, etc
15. Key Features-Kura Wires
● Add new processes to Kura in a block based visual representation
● Runs automatically once changes are applied; no need to compile
● Implementations for MQTT, building databases, data filtering, and more
● Additional assets can be added to Kura Wires by adding packages
17. Prototype Project with Temperature Sensor
Processing Tool:
● Handle streams of big data in real-time
● Low Latency
● Robust
● Simple / Flexible Implementation
23. SQL & NoSQL Comparison
SQL
● Vertically Scalable
● Predetermined data structure
● ACID (Atomicity, Consistency, Isolation
and Durability)
● Uses the Universal SQL (Structured
Query Language) language which
provides a powerful tool to manipulate
and define data
● Allows for complex queries
● Examples:
○ MySQL, Sqlite, and Postgres
NoSQL
● Horizontally Scalable
● Unstructured data storage
● CAP theorem ( Consistency, Availability
and Partition tolerance )
● Typically NoSQL relies on a collection of
documents and the syntax varies per the
database
● No standard interface for complex
queries
● Examples:
○ MongoDB, Redis, and Hbase
24. MySQL
● Open source relational database management system
(RDBMS)
● Provides a password system that is very flexible and
secure
● MySQL supports large databases, up to 50 million rows
or more in a table
● MySQL is also used in the industry as well (Facebook,
Twitter, Flickr and YouTube)
25. Where does
MySQL fit?
● Data Storage portion of the
architecture
● Apache Storm writes data
to MySQL
● Spark is then able to query
the database and provide a
visualization tool
27. Design Challenges (Kyle)
● Normalizing the database was challenging
because I never experienced high level data
modeling before. It took me a couple of office
hour sessions with my professor for me to
grasp the concept.
● Creating the connection between Apache
Storm was incredibly frustrating. It was a
gruelling debugging process that me and a
fellow engineer were stuck on for about a day
or two.
● The time spent to build up my knowledge of all
the technologies integrated into the framework.
31. Cloud Batch Processing
• Four components to Apache Spark:
• Spark SQL
• Introduces the concept of the “Resilient
Distributed Dataset”.
• Enables reading/writing and querying a
database using SQL.
• Spark MLib
• Machine Learning Algorithms
• Spark Streaming
• Real-time streaming using the process
of micro-batching.
• Spark GraphX
• Extends RDD’s for graphs and graph-
parallel computation.
33. Data Visualization
• Web-based book for data analysis
and visualization.
• Frames the question: “What is really
happening behind the hood?”.
• Versatile
• Interpreters for any language.
• Easy to share web-based notebook
• Able to set permissions per person.
• Integration with Apache Spark
Web Browser
Web
Server
Local
Interpreters
Zeppelin Daemon
Remote
Interpreters
Spark Master Node
Spark Worker
Node
Spark Worker
Node
34. Apache Spark Apache Zeppelin
✔ Python rather than Scala.
✔Plenty of Examples with a big community.
✔Troubleshooting was not that bad.
✔Basically Spark, but visualized.
✔Creating interpreters is useful.
✔Fun to explore and use.
🗙I used Python, but I’m new to Python
also.
🗙Understanding the program is one
thing, using it is another.
🗙New to Open-Source and Linux OS.
🗙 I learned Zeppelin, before Spark
🗙 Not as big of a community as Spark.
🗙 Troubleshooting was more difficult.
🗙More to Zeppelin than just Spark
interpreter..
36. Parking on campus
• Time-to-park is an issue on UTSA campus
• UTSA claims to have enough parking to meet typical demand
• Too many people in the same parking lot / Random distribution of
newly opened parking spots
• Solution: Accurate and reliable system to track (and predict)
parking patterns, providing real-time information to drivers to
assist in parking decisions