Steve Griset, alwaysAI http://aisf19.xnextcon.com

1. Unleash Computer
Vision at the Edge
AI NextCon 2019
Steve Griset
CTO alwaysAI

2. Catalysts Driving Computer
Vision on the Edge
• New Approaches to Computer Vision
• Optimized Model Architectures
• Hardware Innovation & Proliferation
• Improving Software Tools

3. New Approaches to
Computer Vision
• What is Computer Vision (CV)?
• CV is finding features in images to help discriminate objects and/or classes of objects
• Traditional Approaches
• Algorithms and methods for detecting features in image
• Colors
• Edges
• Corners
• Human engineered (Crafted not Learned)
• Accuracy and the reliability directly depend on the features extracted and on the
methods used.

4. Feature Extraction
Algorithms
• Feature Extraction
1. SURF (Speeded-Up
Robust Features)
2. BRIEF (Binary Robust
Independent Elementary
Features)
3. SIFT (Scale-Invariant
Feature Transform)
• Demo:
A. Canny Edge Detector to
count coins

5. Machine Learning
• Its hard for traditional computer algorithms to cope when
1. Number of classes to detect increases
2. Image clarity goes down
• The Machine Learning approach to Computer Vision
A. Introduces the concept of machine based end-to-end learning
B. No longer a need for defining the features and do feature
engineering
C. Initially to much computing power was need for this approach to be
practical in edge computing

6. Machine Learning
“If you want to teach a [deep] neural
network to recognize a cat, for
instance, you don’t tell it to look for
whiskers, ears, fur, and eyes. You
simply show it thousands and
thousands of photos of cats, and
eventually it works things out. If it
keeps misclassifying foxes as cats,
you don’t rewrite the code. You just
keep coaching it.”
Wired | 2016
Input
Feature
Extractor
Features Output
Traditional Flow
Input Machine Learning Output
Machine Learning Flow

7. Types of Machine Learning
1.Unsupervised Learning (self-taught learning)
• There are no labels associated with the input data and thus we cannot correct our
model if it makes an incorrect prediction
• The ML Algorithm is trying to understand the structure of a dataset (example k-
means clustering)
2. Supervised Learning
• Given a set of training data, a model (or “classifier”) is created through a training
process where predictions are made on the input data and labels, then corrected
when the predictions are wrong.
3. Semi-Supervised Learning
• Some training data has labels and other data elements do not (hybrid)
• The end result is an accurate classifier (but normally not as accurate as a
supervised classifier) with less effort

8. Image Inferencing
Image Classification
• When performing traditional image classification
the goal is to predict a set of labels to
“characterize” the contents of an input image
Object Detection
• Object detection builds on image classification,
but this time allows for the localize of each
object in an image. The image is now
characterized by:
1. Bounding box (x, y)-coordinates for each
object
2. An associated class label for each bounding
box
Semantic Segmentation
• Semantic segmentation algorithms require the
association of every pixel in an input image with
a class label (including a class label for the
background). Semantic segmentation algorithms
are capable of labeling every object in an image
they cannot differentiate between two objects of
the same class.
Instance Segmentation
• Instance segmentation algorithms compute a
pixel-wise mask for every object in the image,
even if the objects are of the same class label

9. Optimized Model
Architectures
ML Constrained Devices
• Processing and Memory Constraints are important
• Model Size Matters!
AlexNet Changed Everything
• Eight Layer CNN
• First fast GPU-implementation of a CNN to win an
image recognition contest
• Model Size 207 MB
• Top-1 Accuracy ~55%
Two Edge Optimized Model Architectures
• GoogleLeNet
• 22 Layer CNN
• Model Size 22MB
• Top-1% 66% - 67%
• ShuffleNet
• 50 Layer CNN
• Model Size 7.4MB
• Top-1% 67% - 68%
Image Classification Demo
• GoogleLeNet
• ShuffleNet

10. Hardware Innovation &
Proliferation
The camera has become a general purpose sensor
1. Continuous Quality Improvement
2. Camera cost are falling
Embedded IoT microprocessor make up that vast majority of the earth’s computing power
and are getting more powerful with each ARM release.
A. According to ARM’s media fact sheet as of July 2017 over 100 billion arm based processor had
been shipped (Only ~30 billion of those are used in cell phones)
B. Raspberry Pi 4 SBC Specifications
1. SoC: Broadcom BCM2711B0 quad-core A72 (ARMv8-A) 64-bit @ 1.5GHz.
2. RAM: 1GB, 2GB, or 4GB LPDDR4 SDRAM
C. Inforce 6640 SBC Specifications
SoC Qualcomm Snapdragon 820 quad-core 64-bit @ 2.2GHz
RAM: 4 GB of RAM, 32 GB of internal storage

11. Hardware Innovation &
Proliferation
AI Accelerators (Hardware and Software)
1. Off Loading Inferences from the CPU for example to GPU (Hardware)
Jetson Nano 128 core GPU
2. Pruning
Reducing Neurons and Synapses from Machine Learning Network
3. Quantization (TPU)
• Convert input values from a large set to output values in a smaller set
• Reduces power consumption while decreasing inference time
Demo Object Detection
A. Using the CPU for Inferencing
B. Using Intel’s Myriad Accelerator

12. Software Tools
Rich Open Source Platforms - OpenCV & Dlib
A. Provides a large base of computer vision algorithms &
utilities
Transitioning from C++ to Python Based Tooling
1. Increasing developer population
2. Reducing development and testing times
Model Frameworks - Caffe, Torch, Tensorflow & others
Diverse choice of model architects and inference engines

13. Software Tools
Model Tooling - Keras, Scikit-learn & others
Model training and customization
Software Accelerators - XNOR and OpenVino
Fast CPU based Inference Engines with specialized machine learning
models
Application Development Platforms - alwaysAI
1. Containerized platform that is model framework & accelerator
agnostic
2. API’s Speeds up application development and deployment