Dog Breed Classification using the PyTorch Estimator on Azure Machine Learning service

1. Dog Breed Classification
HEATHER SPETALNICK
PROGRAM MANAGER, MICROSOFT
@microheather
With PyTorch on Azure Machine Learning

2. What will we cover?
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

3. What will we cover?
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

4. What will we cover?
Deep learning and Transfer learning
PyTorch
How to do Deep Learning on Azure Machine
Learning service
Dog breed image classification workshop
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

5. Deep learning overview
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

6. ARTIFICIAL INTELLIGENCE
MACHINE LEARNING
DEEP LEARNING
Deep neural networks revolutionize AI
The idea of “intelligent machines”
Development of algorithms to learn from data
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

7. Deep learning applications
autonomous
vehicles
language
translation
predictive
analytics
forecasting
speech
recognition
And many more…
Image
recognition
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

8. What is deep learning?
"BLACK BOX"
Cat (0.96)
Dog (0.03)
Other (0.01)
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

9. Neural networks
low-level features mid-level features high-level features
Source
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

10. Deep learning vs. traditional ML
manual feature extraction
ML
DL
classification algorithm
Cat
Dog
Other
feature learning + classification
Cat
Dog
Other
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

11. Convolutional neural network (CNN)
penultimate
layer
RGB channels
of input image
Convolution layer Pooling layer
Fully connected layer
Cat
Dog
Other
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

12. Lifecycle 1. define problem
2. acquire + process data
3. design model architecture
4. train model5. test/evaluate
a. initialize
b. feed in minibatch of data
c. calculate lossd. optimize: minimize loss
e. update weights
y =Wx + b
loss = |desired – actual outcome|δ
6. deploy

13. Characteristics of Deep Learning
Massive amounts of training data
Computationally expensive
Excels with raw, unstructured data
Automatic feature extraction
GPU
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

14. Transfer Learning
 Popular method in deep learning that allows you to train Deep
Neural Networks with comparatively little data
 The reuse of a pre-trained model on a new, but related,
problem
 The weights and parameters of a network that has al been trained on
a large dataset by somebody else and ‘fine-tune’ the data with your
own dataset
 Mostly used in Computer Vision and Natural Language
Processing tasks
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

15. Transfer Learning
 We don’t need to redefine low level features like curves and edges
 ConvNet as fixed feature extractor:
 Take a Convolutional Network that is pretrained on ImageNet
 Remove the last fully-connected layer and replace it with your own
classifier
 Freeze the weights of all the other layers and train the network normally
 ResNet Model
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

16. Quick break…
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

17. Popular frameworks
Use your favorite deep learning frameworks
TensorFlow PyTorch Scikit-Learn
MXNet Chainer Keras
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

18. PyTorch
 Python library for deep learning written in Python and C++
 Popular in research and development
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

19. PyTorch
 NumPy ndarray-like Tensors for GPU-accelerated computation
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather
import torch
a = torch.ones(3)
print(a)
Out:
tensor([1., 1., 1.])
a = a.to(“cuda”)
print(a)
Out:
tensor([1., 1., 1.], device=‘cuda:0’)

20. PyTorch
 Dynamic computational graph: define-by-run
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

21. PyTorch
 Dynamic computational graph: define-by-run
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather
import torch
x = torch.tensor(4)
y = torch.tensor(6)
z = x + y
print(z)
Out:
tensor(10)
import tensorflow as tf
x = tf.constant(4)
y = tf.constant(6)
z = x + y
print(z)
Out:
Tensor(‘add:0’, shape=(), dtype=int32)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print (sess.run(z))
Out:
10

22. How do we classify dogs?
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

23. Stanford Dog Dataset
 http://vision.stanford.edu/aditya86/ImageNetDogs/
Images of 120 breeds of dogs from around
the world
~150 images per class
Total images: 20,580
Built using ImageNet for fine-grained Image
Categorization
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

24. Azure AI
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather

25. Azure Bot Service
Azure Cognitive Services
Azure Cognitive SearchAzure Databricks
Azure Machine Learning
Knowledge miningAI apps & agents Machine learning
Azure AI
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather

26. Azure Bot Service
Azure Cognitive Services
Azure Cognitive SearchAzure Databricks
Azure Machine Learning
Knowledge miningAI apps & agents Machine learning
Azure AI
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather

27. Sophisticated pretrained models
To simplify solution development
Azure
Databricks
Machine Learning
VMs
Popular frameworks
To build advanced deep learning solutions TensorFlow KerasPytorch Onnx
Azure
Machine Learning
LanguageSpeech
…
SearchVision
On-premises Cloud Edge
Productive services
To empower data science and development teams
Powerful infrastructure
To accelerate deep learning
Flexible deployment
To deploy and manage models on intelligent cloud and edge
Machine Learning on Azure

28. Scale out with Azure
Machine Learning
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather

29. HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather
Local machine
Scale up on VMs
Scale out with Spark
Scale out with Azure Batch AI
Machine Learning Service
SDK for Python
Machine Learning CLI
VS Code Tools for AI
Notebooks
Open source Python IDEs
Azure Machine Learning service
Build and train models anywhere

30. HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather
Docker
On-prem deployment
Azure Container Instance
Azure IoT edge
IoT accelerated targets
Project Brainwave (FPGAs)
Machine Learning Service
Azure Managed Kubernetes Service
Azure Machine Learning service
Deploy models anywhere

31. Prepare
Data
Register and
Manage Model
Train & Test
Model
Build
Image
…
Build model
(your favorite IDE)
Deploy
Service
Monitor
Model
Prepare Experiment Deploy
Data Scientist’s “inner loop”
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

32. • Leverage service side capture of run
metrics, output logs and models
• Use leaderboards, side by side run
comparison and model selection
• Manage training jobs locally, scaled-up or
scaled-out
• Run distributed TensorFlow, PyTorch or MPI
training jobs
• Conduct a hyperparameter search on
traditional ML or DNN
• Use your favorite IDEs, editors, notebooks,
and frameworks
U S E T H E M O S T P O P U L A R I N N O V A T I O N S
U S E A N Y T O O L
U S E A N Y F R A M E W O R K O R L I B R A R Y
Experimentation

33. • Do distributed training at cloud scale using a
framework of choice
• Leverage system managed Azure Machine
Learning Compute or bring your own
compute (VM, Databricks/HDI cluster)
• Manage resources across subscription and
share them within a workspace
• Autoscale resources to only pay while
running a job
• Use the latest NDv2 series VMs with the
NVIDIA V100 GPUs
Training Infrastructure

34. • Manage and track model versions with a central
model registry
• Deploy models to Azure Kubernetes service with
automatic scaling capability
• Container-based hosting for improved time to
market
• Deploy models to the cloud, on-premises, to IoT
edge and to FPGAs
• Management and monitoring of deployments
through Azure
AppInsights
• Enable DevOps with full CI/CD integration with
VSTS
Model management
and deployment

35. Aka.ms/pycon-canada-workshop
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

36. Azure Machine Learning Overview
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

37. My Computer
Experiment
Docker Image
Data Store
Compute Target
Azure ML
Workspace

38. What compute to use?
HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather

39. HEATHER SHAPIRO | TECHNICAL EVANGELIST, MICROSOFT
@microheather
Powerful infrastructureAccelerate deep learning
CPUs
Optimized for flexibility Optimized for performance
GPUs FPGAs
Deep learning
N Series
Specialized hardware
accelerated deep learning
Project Brainwave
Support for image classification and recognition scenarios
ResNet 50, ResNet 152, VGG-16, SSD-VGG, DenseNet-121
FPGA NEW UPDATES:
General purpose machine
learning
D, F, L, M, H Series

40. Pre-Configured environments in
the cloud for
Data Science & AI Modeling,
Development & Deployment.
Samples to get started
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

41. DSVM – Windows Server 2016
DSVM – Linux – Ubuntu
Deep Learning Virtual Machine
Geo AI Data Science VM

42. Why Data Science VMs?

43. Azure BatchAI

44. Purpose Graphics
VM Family NV v1
GPU NVIDIA M60
Sizes 1, 2 or 4 GPU
Interconnect PCIe (dual root)
2nd Network
VM CPU Haswell
VM RAM 56-224 GB
Local SSD ~380-1500 GB
Storage Std Storage
Driver Quadro/Grid PC
Azure Family of GPUs
Compute Compute Compute
NC v1 NC v2 NC v3
NVIDIA K80 NVIDIA P100 NVIDIA V100
1, 2 or 4 GPU 1, 2 or 4 GPU 1, 2 or 4 GPU
PCIe (dual root) PCIe (dual root) PCIe (dual root)
FDR InfiniBand FDR InfiniBand FDR InfiniBand
Haswell Broadwell Broadwell
56-224 GB 112-448 GB 112-448 GB
~380-1500 GB ~700-3000 GB ~700-3000 GB
Std Storage Prem Storage Prem Storage
Tesla Tesla Tesla
Deep Learning
ND v1
NVIDIA P40
1, 2 or 4 GPU
PCIe (dual root)
FDR InfiniBand
Broadwell
112-448 GB
~700-3000 GB
Prem Storage
Tesla

45. New GPU Families
Purpose Graphics
VM Family NV v2
GPU NVIDIA M60
Sizes 1, 2 or 4 GPU
Interconnect PCIe (dual root)
2nd Network
VM CPU Broadwell
VM RAM 112-448 GB
Local SSD ~700-3000 GB
Storage Prem Storage
Driver Quadro/Grid PC
Deep Learning
ND v2
NVIDIA V100
8 GPU
NVLink
Skylake
672 GB
~1300 GB
Prem Storage

46. What did we cover?
Deep learning and Transfer learning
PyTorch
Azure Machine Learning service
Dog breed image classification workshop
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

47. RESOURCES
 Workshop: Aka.ms/pycon-canada-workshop
 Azure Machine Learning service Docs: Aka.ms/azureml
 Azure Machine Learning Github Notebooks:
https://github.com/Azure/MachineLearningNotebooks/tree/ma
ster/
 PyTorch
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

48. CONTACT ME
 Email: heather.spetalnick@microsoft.com
 Twitter @microheather
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather

49. THANKS!
HEATHER SPETALNICK | PROGRAM MANAGER, MICROSOFT
@microheather