This is a hack material to help IoT partners/customers to know how to build a ML Edge module with Azure ML Service and deploy it to the Edge device.

2. Intro to Azure Machine Learning Inferencing
Manage and deploy AI models

3. Building your own AI models for Transforming Data into Intelligence
Prepare Data Build & Train Deploy

4. Azure AI Services
Azure Infrastructure
Tools

5. the team workspace - logical

6. Prepare
Data
Register and
Manage Model
Train & Test
Model
Build
Image
…
Build model
(your favorite IDE)
Deploy
Service
Monitor
Model
Prepare Experiment Deploy

7. 1/16/2019 9

8. My Computer
Data Store
Azure ML
Workspace
Compute Target
Docker Image

9. 1/16/2019 11

10. http://portal.azure.com

12. http://notebooks.azure.com/
Import sample notebooks

14. Model Management

15. Docker
Containers
Azure Kubernetes Service (AKS)
Azure Batch
Azure IoT EdgeAzure
Machine Learning
Step 3: Deploy
Any other container host…

16. Deploying models at scale
1/15/2019 18

17. The AI lifecycle
1/15/2019 19

18. The Azure ML Deployment Pipeline

19. Understanding the Edge: Heavy Edge vs Light Edge
Cloud: Azure Heavy Edge Light Edge
Description
An Azure host that
spans from CPU to GPU
and FPGA VMs
A server with slots to insert CPUs, GPUs, and FPGAs or a X64 or ARM system that needs to be
plugged in to work
A Sensor with a SOC (ARM CPU, NNA, MCU) and memory that
can operate on batteries
Example
DSVM / ACI / AKS /
Batch AI
- DataBox Edge
- HPE
- Azure Stack
- DataBox Edge - Industrial PC
-Video Gateway
-DVR
-Mobile Phones
-VAIDK
-Mobile Phones
-IP Cameras
-Azure Sphere
- Appliances
What runs
model
CPU,GPU or FPGA CPU,GPU or FPGA CPU, GPU x64 CPU Multi-ARM CPU
Hw accelerated
NNA
CPU/GPU MCU

23. Model Management
Edge Integration

24. Why Intelligent Edge?
High-speed data processing,
analytics and shorter response
times are more essential than ever.
Intelligent Cloud
• Business agility and scalability: unlimited computing
power available on demand.
Intelligent Edge
• Can handle priority-one tasks locally
even without cloud connection.
• Can handle generated data that is too
large to pull rapidly from the cloud.
• Enables real-time processing through
intelligence in or near to local devices.
• Flexibility to accommodate data privacy related
requirements.

25. The Azure ML
Deployment Pipeline

26. Challenges of Running AI on the Edge
• Reduced Compute Power
• No common HW abstraction for
NN
• Driver version fragmentation
• Need familiarity with every
platform

27. The components of a ML application
Vision
AI dev
kit
Vision
AI dev
kit

28. The components of a ML application

29. Vision AI Developer Kit
Hardware Specification

30. Vision AI Development kit – System Architecture

32. https://.portal.azure.com
https://docs.microsoft.com/zh-tw/azure/iot-edge/quickstart-linux
https://docs.microsoft.com/zh-tw/azure/iot-edge/tutorial-deploy-
machine-learning