Majority of IoT solutions use data analysis at the Cloud level, collecting a huge amount of raw data from many thousands of peripherals. What if I told you that you can move from raw data collection to knowledge aggregation by implementing Artificial Intelligence into IoT systems?
During the talk, I will show the benefits of introducing AI at the earliest possible stages, applying the concept of moving from Cloud computing to Fog computing. The basic principle of constructing AIoT systems is the use of the node logic, where a node of the system has to process the provided information in a form of abstract concepts, but not in a form of raw information.
Further, the experience of one device learning and the history of its life cycle can be applied to new models, automatically programming their production cycles for the most efficient use. Actually, IoT solutions should apply AI components at each level of data transfer. Following this approach, the whole system becomes self-optimizing.
Also, during the talk, I will present related case studies and demonstrate a working stand.
5. Confidential
5
Why Now
Intel Neural Network Stick
• Price $109
• Size 75x27x14 mm
• Weight 18 gram
• Based on Myriad X VPU
• 16 video CPU
• MobileNet v2 20 fps
• TensorFlow, Cafee
• Current up to 40 mA
NVIDIA Jetson Nano
• Price $100
• Size 95x76x30mm
• Weight 136 gram
• 64bit quad-core A57
• 128-core Nvidia GPU
• MobileNet v2 12 fps
• Current up to 1220 mA
Google TPU Accelerator
• Price $75
• Size 65x30x8 mm
• Weight 12 gram
• Google Edge TPU
• MobileNet v2 40+ fps
• TensorFlow Lite
• Current up to 20 mA
Google Coral Dev Board
• Price $150
• Size 85x56x20mm
• Weight 78 gram
• NXP i.MX 8M SOC
• Google Edge TPU
• MobileNet v2 100+ fps
• TensorFlow Lite
• Current up to 960 mA
6. Confidential
6
Platforms Benchmark
Resume:
• Absolute winner is Coral Dev Board.
• Intel NN Stick is effective at face
recognition related projects.
• Coral USB Accelerator is the cheapest
solution with a good performance.
• NVIDIA Jetson Nano is not
recommended for use.
7. Confidential
7
IoT and AI Synergy
Collect
StoreExecute
Analyse Process
IoT
Collect
StoreExecute
Analyse Process
AI
8. 8
Confidential
8
Edge IoT System Concept
Cloud part consists of:
● different but similar components
● data ETL core
● different presentation layers.
But, we may have similar sensors or
Edge Layer:
AI Embedded in the Edge
Physical
Edge
Local
Network
Gateway
Wide
Network
Security
Middleware
ETL
Presentation
Notification Configuration
BackEndEdge
Ad Hoc /
Mesh
M2MM2P
Big Data,
Analytic
12. 12
Confidential
12
Edge IoT System Concept
Cloud part consists of
● different but similar components
● data ETL core
● different presentation layers.
But, we may have similar sensors or
Edge Layer:
AI Embedded in the Edge
Physical
Edge
Local
Network
Gateway
Wide
Network
Security
Middleware
ETL
Presentation
Notification Configuration
BackEndEdge
Ad Hoc /
Mesh
M2MM2P
Big Data,
Analytic
13. 13
Confidential
13
Edge AIoT System Concept
Changes from IoT model
● Universal AI Edge sensor
● ML models DB
● IFTTT State machine (Configurable)
● Additional parameters in LvM2M
modes
AI Edge
Local
Network
Gateway
Wide
Network
Security
Middleware
ETL
Presentation
Notification
Configuration
BackEndEdge
Ad Hoc /
Mesh
M2MM2P
FSM
AI Edge
ML Models
IFTTT
14. 14
Confidential
14
Fog Computing AI-Based Architecture Flow
...
...
...
Abstract data
New ML model
Raw data request
and send
Model tuning
● Exchanging knowledge in place of data
● Uploading abstract
● Downloading knowledge
● Self improvement
● Raw data upload by a request from the
upper layer
15. 15
Confidential
15
ML Models Optimization for Edge AI
Frameworks:
● TensorFlow
● TensorFlow Lite
● Caffe
Optimizations:
● Quantization
● Tensor Fusion
MobileNet SSD
SqueezeNet
Inception
Models:
● MobileNet SSD v1 & v2
● SqueezeNet 1.0, SqueezeNet 1.1
● Inception v1 & v2 & v3 3 & v4
Libraries:
● OpenCV
● OpenVINO
QuantizationTensorRT Optimisation
next input
3x3 CBR 3x3 CBR 1x1 CBR
1x1 CBR max pool
next input
input
tensor
Expansion
layer
Depthwise
layer
filter
the
data
Projection
layer
uncompress
the data
compress
the data
outpt
tensor
Min
Input (float)
Min Max
Quantize
QuantizedRelu
Dequantize
Output (float)
Ma
x
Eight
Bit
Min Ma
x
Eight
Bit
maxpool/2
conv1
fire2
fire3
fire4
fire5
fire6
fire7
fire8
fire9
fire10
softmax
maxpool/2
globalavgpool
maxpool/2
96
128
128
256
256
384
384
512
512
1000
16. 16
Confidential
16
IFTTT Final State Machine
● Configurable FSM
● Generic data concept
● Visual simulation
● Visual monitoring
States list
Objects list
Events list
Action list
18. 18
Confidential
18
Edge AIoT Accelerator Module
Battery
AI Device
Display
Camera
● A fully standalone video processing
device
● Does not require cloud or mobile device
connection for operation presenting
● Contains replaceable parts
● Single hardware for any IoT solution
demonstration
● Contains port for extended sensors and
Ethernet with PoE
● ML models can be downloaded or
updated from the cloud to support a
specific IoT solution demonstration
19. 19
Confidential
19
AIoT Accelerator Cloud Part
Reaction
Objects
AI Camera
Vision
AB SM
AI camera classifies objects
● Object attributes:
○ ID, probability, location, and state
● AB SM events:
○ Object ID change
○ Time of the day
○ Calendar
○ Public events
● AB SM reactions:
○ Change of state
○ Giving out notifications
○ Control of accelerators
○ Change Object attributes
20. 20
Confidential
20
AIoT Accelerator Block Diagram
Object
Action
Location
Detection Event generation
FSM updateActions
Actions
Internal
● Update Edge ML nodel
● Start record stream
● Take an image shoot
● Assign ID to object/location
● Change object attribute
External
● Make a call
● Control switch/attenuator
● Send a sound message
● ETL action
24. Confidential
24
IFTTT STM Design
● Templates from different solution
● Custom Customization
● Visual simulation
● Visual history and action logic
● Ability for a third party device integration
● API for the external applications
