Deep Learning on The Edge, a hands-on a approach to running deep learning work loads on the edge for IoT as well as in Apache NiFi and in Hadoop 3.1 YARN as dockerized work loads.

1. 1 © Hortonworks Inc. 2011–2018. All rights reserved.
A Hands-on deep dive on using Apache NiFi with MiniFi, Apache MXNet
and other Deep Learning Libraries on the Edge
Timothy Spann, Senior Solutions Engineer
Hortonworks @PaaSDev
Deep Learning On The Edge

2. 2 © Hortonworks Inc. 2011–2018. All rights reserved.
Disclaimer
• This document may contain product features and technology directions that are under
development, may be under development in the future or may ultimately not be
developed.
• Technical feasibility, market demand, user feedback, and the Apache Software
Foundation community development process can all effect timing and final delivery.
• This document’s description of these features and technology directions does not
represent a contractual commitment, promise or obligation from Hortonworks to deliver
these features in any generally available product.
• Product features and technology directions are subject to change, and must not be
included in contracts, purchase orders, or sales agreements of any kind.
• Since this document contains an outline of general product development plans,
customers should not rely upon it when making a purchase decision.

3. 3 © Hortonworks Inc. 2011–2018. All rights reserved.
IoT
Multiple devices, protocols, frameworks, languages, data types, sensors and networks
Protocols
• MQTT
• HTTPS / SSL (REST/JSON)
• OPC UA
• CoAP
• AMQP
• TCP/IP
• UDP
• JSON
• XML
• CSV
• Raw Text
• Images (JPEG, PNG)
• Raw Data Streams
Data Types Sensors
• Cameras
• Temperature/Humidity
• IR
• Proximity
• Motion Sensors
• GPS
Protocols
• NVidia Jetson TX1
• Raspberry Pi
• Arduino
• TS-7800 V2
• ESP8266
• DragonBoard 410c
• BeagleBone Black
• Sino:bit

4. 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
http://mxnet.incubator.apache.org/
• Cloud ready
• Experienced team (XGBoost)
• AWS, Microsoft, NVIDIA, Baidu, Intel backing
• Apache Incubator Project
• Run distributed on YARN
• In my early tests, faster than TensorFlow.
• Runs on Raspberry PI, NVidia Jetson TX1
and other constrained devices
• Great documentation
• Gluon
• Great Python Interaction
• Model Server Available
• ONNX Support
• Now in Version 1.2.1!
• Great Model Zoo
https://mxnet.incubator.apache.org/how_to/cloud.html
https://github.com/apache/incubator-mxnet/tree/1.2.1/example

5. 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache MXNet 1.2.1 Pre-Built Models
• CaffeNet
• SqueezeNet v1.1
• Inception v3
• Single Shot Detection (SSD)
• VGG19
• ResidualNet 152
• LSTM
http://mxnet.incubator.apache.org/model_zoo/index.html
https://github.com/dmlc/mxnet-model-gallery
https://github.com/onnx/models
https://mxnet.incubator.apache.org/api/python/gluon/model_zoo.html

6. 6 © Hortonworks Inc. 2011–2018. All rights reserved.
Using Apache MXNet on Devices with Sensors and Intel Movidius
https://community.hortonworks.com/articles/176932/apache-deep-learning-101-using-apache-mxnet-on-
the.html

7. 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
• TensorFlow (C++, Python, Java) via ExecuteStreamCommand
• TensorFlow NiFi Java Custom Processor
• TensorFlow Running on Edge Devices (MiniFi C++ Built-in)
• TensorFlow Running on Edge Devices (MiniFi Java)
• TensorFlow Running in Spark
• TensorFlow Running Through Livy to PySpark
• TensorFlow Running on HDP 3.0 in Dockerized Containers
Apache NiFi Integration with TensorFlow Options

8. 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
TensorFlow Python ExecuteStreamCommand NiFi
https://community.hortonworks.com/articles/58265/analyzing-images-in-hdf-20-using-tensorflow.html

9. 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Run TensorFlow on YARN 3.0
https://community.hortonworks.com/articles/83872/data-lake-30-containerization-erasure-coding-gpu-p.html

10. 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Why TensorFlow? Also Apache MXNet, PyTorch and DL4J.
• Google
• Multiple platform
support
• Hadoop integration
• Spark integration
• Keras
• Large Community
• Python and Java APIs
• GPU Support
• Mobile Support
• Inception v3
• Clustering
• Fully functional demos
• Open Source
• Apache Licensed
• Large Model Library
• Buzz
• Extensive Documentation
• Raspberry Pi Support

11. 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
TensorFlow Java Processor in NiFi
https://community.hortonworks.com/content/kbentry/116803/building-a-custom-processor-in-apache-nifi-12-for.html
https://github.com/tspannhw/nifi-tensorflow-processor

12. 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
TensorFlow Running on Edge Nodes (MiniFi)

13. 13 © Hortonworks Inc. 2011–2018. All rights reserved.
Sensors via Python API
{"tempf": 75.14, "temp": 35.08,
"pitch": 1.0,
"diskfree": "1211.8 MB", "yaw": 55.0,
"cputemp": 52.08,
"ts": "2017-06-16 17:39:08",
"humidity": 41.5, "pressure": 0.0,
"memory": 23.0, "y": 0.0, "x": -1.0, "z": 1.0,
"roll": 1.0}
https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html
https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html
https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html

14. 14 © Hortonworks Inc. 2011–2018. All rights reserved.
MQTT
https://community.hortonworks.com/articles/167199/sending-messages-and-displaying-them-on-an-oled-sc.html
https://community.hortonworks.com/articles/183474/iot-using-minifi-java-agent-to-send-mqtt-messages.html
https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
https://community.hortonworks.com/articles/72420/ingesting-remote-sensor-feeds-into-apache-phoenix.html

15. 15 © Hortonworks Inc. 2011–2018. All rights reserved.
Run everywhere
Orchestration
Queueing
Simple Event Processing
Deep Learning Framework
Enabling Record Processing
Schema Management
Open Source Components
Edge Ingest
Data Buffering
Device Integration

16. 16 © Hortonworks Inc. 2011–2018. All rights reserved.
Open Source Image Analytical Components
Streaming Analytics
Manager
Image Ingest
Distributed queue
Buffering
Process decoupling
Routing and Pre-Processing
Orchestration
Queueing
Simple Event Processing
Image Capture
Image Processing

17. 17 © Hortonworks Inc. 2011–2018. All rights reserved.
Streaming Analytics
Manager
Part of MiniFi C++ Agent
Detect metadata and data
Extract metadata and data
Content Analysis
Deep Learning Framework
Open Source Image Analytical Components
Enabling Record Processing
Schema Management

18. 18 © Hortonworks Inc. 2011–2018. All rights reserved.
Edge Intelligence with MiNiFi
à Guaranteed delivery
à Data buffering
‒ Backpressure
‒ Pressure release
à Prioritized queuing
à Flow specific QoS
‒ Latency vs. throughput
‒ Loss tolerance
à Data provenance
à Recovery / recording a rolling log
of fine-grained history
à Designed for extension
Different from Apache NiFi
à Design and Deploy
à Warm re-deploys
Key Features

19. 19 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Java Agent C++ Agent
Multi Cloud HDP Clusters
HBase Druid
YARN
HDFS
Hive
Weather
Location
Sensors

20. 20 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Apache NiFi: The Ecosystem
Enter the MiNiFi Command & Control
– Provide tooling to map the UX of
interactive command and control in
NiFi to the design and deploy
approach of MiNiFi
But more instances complicate my operational management!
https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control

21. 21 © Hortonworks Inc. 2011–2018. All rights reserved.
What do we want to do?
• Python collects camera images and sensor data
• MiniFi executes Apache MXNet and/or Tensorflow at the
edge
• Run Inception to classify objects in image
• Apache NiFi stores sensor readings, gps, images, metadata
and enriched data in Hadoop
• One or more Heterogenous Devices in distributed locations
• One or more Heterogenous Data Storage Platforms

22. 22 © Hortonworks Inc. 2011–2018. All rights reserved.
Overview
• Build Physical IoT Devices (Raspberry Pi, Movidius Neural Stick, Webcam)
• Choose MiniFi C++ or MiniFi Java Agent
• For MiniFi Java Agent, Install JDK 8
• Design Flow For MiniFi
• Install config.yml on Device
• Connect to Sensors and Equipment via Python, Go, C++ or Java
• Add Shell Script if needed
• Start MiniFi Java Agent as a Service
• MiniFi Calls Apache NiFi IoT Gateway via S2S or MQTT
• Build Data Stream Real-Time in Apache NiFi as Data Arrives
• Complete Simple Event Processing
• Store Data at Scale in HBase, Druid, HDFS, S3 or Hive LLAP
• Add additional processing with Apache Spark, Apache Storm or Apache
Beam via Apache Kafka

23. 23 © Hortonworks Inc. 2011–2018. All rights reserved.
• https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_installing-
hdf/content/ch_install-ambari.html
• docker pull hortonworks/nifi
• Apache NiFi Configuration for IoT
• https://community.hortonworks.com/articles/67756/ingesting-log-data-using-minifi-
nifi.html
• nifi.remote.input.host and nifi.remote.input.socket.port
Installation of Apache NiFi or HDF 3.x
Pre Step 1

24. 24 © Hortonworks Inc. 2011–2018. All rights reserved.
• Download MiniFi (https://nifi.apache.org/minifi/download.html) You can choose Java or C++. For
your first usage, I recommend the Java edition unless your device is too small. You can also install on a
RHEL or Debian Linux machine or OSX.
• Download MiniFi Toolkit (https://nifi.apache.org/minifi/minifi-toolkit.html)
• https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.5.0
• https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.5.0
• https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_release-
notes/content/ch_hdf_relnotes.html#centos7
• https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon-
data-in.html
• https://community.hortonworks.com/content/kbentry/107379/minifi-for-image-capture-and-
ingestion-from-raspbe.html
MiniFi Java Agent Installation (Latest version 0.5)
Pre Step 2

25. 25 © Hortonworks Inc. 2011–2018. All rights reserved.
• You need to do some extra work here
compiling for your platform
• This is better for smaller devices.
MiniFi C++ Agent Installation
Pre Step 3 (Optional)
https://community.hortonworks.com/articles/167193/building-and-running-minifi-cpp-in-orangepi-zero.html
https://nifi.apache.org/minifi/getting-started.html

26. 26 © Hortonworks Inc. 2011–2018. All rights reserved.
• https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processo
r=CPU
• Install build tools and build from scratch
• Walk through install: https://community.hortonworks.com/articles/176932/apache-deep-learning-
101-using-apache-mxnet-on-the.html
Install Apache MXNet
Pre Step 4

27. 27 © Hortonworks Inc. 2011–2018. All rights reserved.
• Soon Using MiniFi Offline Designer
• Design a flow in Apache NiFi UI
MiniFi Flow Design
Step 1
https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html
https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html

28. 28 © Hortonworks Inc. 2011–2018. All rights reserved.
• Save to NiFi Registry
• Export Template
MiniFi Version
Step 2

29. 29 © Hortonworks Inc. 2011–2018. All rights reserved.
MiniFi Toolkit
• config.sh transform yourtemplate.xml config.yml
MiniFi Convert XML Template to Config YAML
Step 3
https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html
https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html
cd minifi-toolkit-0.5.0
bin/config.sh transform /somedir/16749-rainbowforminifi.xml config.yml
Java home: /Library/Java/Home
MiNiFi Toolkit home: /somedir/minifi-toolkit-0.5.0
No validation errors found in converted configuration.

30. 30 © Hortonworks Inc. 2011–2018. All rights reserved.
• Copy config.yml to Device’s MiniFi conf directory (SCP or C2 Server)
• Restart MiniFi Agent
• Check logs
• Check status
• ./bin/minifi.sh flowStatus
systemdiagnostics:heap,processorstats,contentrepositoryusage,flowfilerepositoryusage,
garbagecollection
MiniFi Deployment
Step 4
https://nifi.apache.org/minifi/minifi-toolkit.html
https://nifi.apache.org/minifi/download.html
https://community.hortonworks.com/articles/167193/building
-and-running-minifi-cpp-in-orangepi-zero.html

31. 31 © Hortonworks Inc. 2011–2018. All rights reserved.
MiniFi Deployment – C2 Server
Step 4b – Cutting Edge Option
https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi-
ce1d6ed565bc
• Add PullHttpChangeIngestor to MiniFi’s Local conf/bootstrap.conf
https://github.com/apache/nifi-minifi/tree/master/minifi-c2
https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control

32. 32 © Hortonworks Inc. 2011–2018. All rights reserved.
• Make sure remote host is set
• If changing settings, restart
• Add a Input Port to the Workspace
• Green Means Go!
Apache NiFi Server Receive
Step 5

33. 33 © Hortonworks Inc. 2011–2018. All rights reserved.
• Store in HDFS Securely
• Build or Insert into Hive Tables
• Store in Hbase or Phoenix
• Query, Report, Analyze
• Analyze data with Spark ML
Data Stored
Final

34. 34 © Hortonworks Inc. 2011–2018. All rights reserved.
References
v https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html
v https://community.hortonworks.com/articles/80339/iot-capturing-photos-and-analyzing-the-image-with.html
v https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
v https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
v https://community.hortonworks.com/articles/155475/powering-apache-minifi-flows-with-a-movidius-neura.html
v https://community.hortonworks.com/articles/155606/iot-ingesting-gps-data-from-odroid-xu4-devices-wit.html
v https://community.hortonworks.com/articles/155604/iot-ingesting-camera-data-from-nanopi-duo-devices.html
v https://community.hortonworks.com/articles/198855/executing-tensorflow-classifications-from-apache-n.html

35. 35 © Hortonworks Inc. 2011–2018. All rights reserved.
Contact
https://community.hortonworks.com/users/9304/tspann.html
https://dzone.com/users/297029/bunkertor.html
https://www.meetup.com/futureofdata-princeton/
https://twitter.com/PaaSDev
@PaasDev
https://github.com/tspannhw/StrataNYC2018

36. 36 © Hortonworks Inc. 2011–2018. All rights reserved.
Hortonworks Community Connection
Read access for everyone, join to participate and be recognized
• Full Q&A Platform (like StackOverflow)
• Knowledge Base Articles
• Code Samples and Repositories

37. 37 © Hortonworks Inc. 2011–2018. All rights reserved.
Community Engagement
Participate now at: community.hortonworks.com© Hortonworks Inc. 2011 – 2015. All Rights Reserved
4,000+
Registered Users
10,000+
Answers
15,000+
Technical Assets
One Website!