An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordination
a presentation for ISCIT2018, Bangkok, Sep 2018.
a1809talk iscit-edge-ai-dev-env-181002a.pdf
3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background
Research Purpose
The aim of this research is to develop device-edge-cloud coordination in
edge AI chip-based intelligent IoT systems.
Background
Cloud-based AI has advantages of flexibility, capacity and throughput,
but it has disadvantages of high communication costs and long delays.
Edge represents a node which is connected to devices and a cloud. It
has low latency direct connection to devices. It also has a connection
to a cloud, but does not have the full capabilities of the cloud,
It is promising to utilize an edge AI development environment where a
framework of edge-cloud AI coordination is deployed.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 3 / 22
4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Related Studies
Research on edge intelligent computing consists of three areas:
a) feasibility and framework of edge computing: fine-grained edge offloading
architecture [Cozzolino17], auction scheme in edge resource allocation
[Prasad17], conditions for edge computing feasibility [Kalim17]
b) AI in edge computing: latency-aware edge computing in video
analytics[Yi17], distributed processing of a Deep CNN in smart camera
[Castillo17], feasibility of AI and ML at edge computing [Sharma18]
c) intelligent systems in edge-cloud coordination: healthcare optimization in
wearable-edge-cloud coordination[Strässle17] docker overhead in Mobile
Edge Computing [Avino17]
The past research did not cover the special purpose AI-engine-chip-based
device-edge-cloud coordination from the software perspective.
The originality of this paper lies in its identification of software-based
approach of AI-engine-chip-based device-edge-cloud coordination.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 4 / 22
5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Research Method
Identifying Edge AI requirements,
Designing a Prototype Implementation of Edge-Cloud Coordinated AI,
Implementing a Prototype using FPGA (Field-Programmable Gate Array)
Discussing Lessons Learned in Prototype Implementation
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 5 / 22
6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Challenges of AI-engine-based AIoT systems
Aspect Issues
AI-Engine Induction performance, Power-consumption-efficiency,
Low-latency, Privacy of data, Cost
Cloud Learning efficiency, Performance of learning models
Edge-Cloud coordi-
nation
Alignment of edge processing and cloud processing,
Controllability of device execution using knowledge in
the cloud.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 6 / 22
7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Requirements of Edge AI Computing
Item Description
Low latency Intelligent device control requires low latency. It is dif-
ficult at cloud-based AI systems.
Low communication
costs
Massive IoT systems need low communication costs in
order to prevent cost explosion. The communication
costs to the cloud can be a prohibitive factor.
Privacy and security In-edge AI computing does not expose data to be pro-
cessed. It is difficult at cloud-based AI systems.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 7 / 22
8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Advantages of edge-cloud coordination
Item Description
Utilization of Data
Analytics
The edge-cloud coordination enables utilization of
stored learned models at the cloud.
Coordination of dis-
tributed edges
The edge-cloud coordination enables real-time context
utilization with aggregated data from distributed edges
in a real-time manner.
Task migration The edge-cloud coordination enables task migration
and resource optimization among multiple edges.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 8 / 22
9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Use cases of Edge-Cloud Coordinated AI
Item Description
Data sharing When necessary, data at the edge need sharing with the
cloud.
Cloud-based control When a context is recognized by a cloud, a cloud-based
control needs coordination at the edge side so that an
intelligent cloud-based control is facilitated. For exam-
ple, learned models can be switched according to the
request from the cloud.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 9 / 22
10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A prototype environment
Item Description
Simulating environ-
ment
Xilinks ep706-based DMP evaluation SDK kit.
CPU FPGA embedded
OS Xilinks Linux OS
Simulated core DMP IP Core
Languages C++ to control device logics, Python to interact with
a cloud.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 10 / 22
14. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Demo features implemented on a prototype environment
Item Description
Video recording Video recording is performed with a camera connected
to the device.
Object detection Objects are detected by the pre-set learned models. The
number of detected objects are counted and compared
with the pre-set threshold.
Image sharing Images with rectangle-framed detected objects are up-
loaded.
Remote setup of
learned models
The learned models can be switched by a command
from a cloud using a switch-and-reset command.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 14 / 22
15. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Model Switch Performance
This paper does not cover detailed performance analysis of the current
implementation.
For model switch time, It takes approximately 15 second to switch learned
models In the current implementation. The major time consumption lies in
a) model load and execution and b) waiting time for graceful exit for model
loading. Without the latter, the execution of new learned model is unstable.
Also, the acknowledgement of model switch command takes 3 seconds at
maximum.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 15 / 22
16. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Advantages of the Proposed Approach
The implementation of a prototype in FPGA-simulated IP core verifies
points (see Result Table)
The demo demonstrates the feasibility of edge-cloud coordination
using ASIC-based AI engines as edge computing.
There are lessons learned in the implementation on needs of
software-based support for hardware-involved integration (see Lessons
Table).
Possible mechanisms to enhance stability of edge AI computing are
needed (see Possible mechanisms table).
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 16 / 22
17. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Results of the implementation
Item Description
Edge-closed process-
ing
Low latency, privacy, and communication-less process-
ing are implemented.
High-level learned
model management
It is feasible to switch multiple learned models depend-
ing contexts.
On-demand infor-
mation sharing
When a context requires full image transfer, needed
images can be uploaded to the cloud.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 17 / 22
18. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Lessons learned in the implementation
Item Description
Timing to ASIC It is critical to align some hardware timing. When the
timing is not aligned, a switch of learned models does
not work and the system is frozen without any outputs.
Timing coordination
between an edge and
a cloud
In an object detection use case, time alignment between
an edge and a cloud is subtle. The slight time differ-
ence gives a small discrepancy between the detected
rectangles and the transferred image.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 18 / 22
19. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Possible mechanisms to enhance stability of edge AI computing
Item Description
Debug Logging, tracing, and probing functions are provided.
Self-diagnosis Self diagnosis of monitoring process is performed in or-
der to identify induction process faults without human
intervention..
Adaptive computing Fault-tolerant processing when some parts of camera-
AI process chain do not work properly.
Remote setup of
learned models
The learned models can be switched by a command
from a cloud using a switch-and-reset command.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 19 / 22
20. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Limitations
This implementation is a proof of concept implementation. Camera works
and detected data can be shared, but performance, scalability, and reliability
are out of the scope of this paper.
Function is based on conceptual verification. Command protocol is not
implemented and interaction with the cloud is performed by the state
variable control at the cloud. Images are not encoded at the device and
shared by a binary array format. In-depth synchronization of detected
objects and shared images remains for future studies.
This research is a prototype implementation based on a simulated
environment constructed on ASIC. The real world deployment will be based
on specialized battery-efficient Edge AI processor core. The system
performance for such specialized AI hardware remains future studies.
Theoretically, model conversions from Caffe should work, but it does not
perform reliably on the implementation in this paper.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 20 / 22
21. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Conclusion
Requirements of low latency, privacy, and limited bandwidth drive the shift
from cloud computing and edge computing.
In this paper, the design of an experimental prototype for device-edge-cloud
coordination for object detection use cases is presented. The feasibility of
device-edge-cloud coordination is verified by an implementation using FPGA.
It shows the effectiveness of edge-based real-time object detection with
high-level cloud-base manipulation of multiple learned models stored in an
edge node. This experimental implementation can lead to the full
ASIC-based implementation when the IP core is ASIC-fabricated.
At the same time, the experiment exposes challenges in hardware-software
coordination. The author discusses design of software-based compensation
towards these challenges.
The design derived from this implementation is a stepping stone for full
ASIC-based AI-engine solutions in the device-edge-cloud coordination.
Toshihiko Yamakami (ACCESS Confidential)An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordinatio2018/09 21 / 22
