Introduction to Machine Learning, Deep Learning and MXNet

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Osemeke Isibor,
Solutions Architect.
iosemeke@amazon.com
31st October 2017
Introduction to Machine Learning,
Deep Learning and MXNet

Agenda
• Introduction
• Machine Learning
• Deep Learning
• MXNet
• Conclusion

Introduction
Artificial Intelligence
Machine Learning
Deep Learning

Timeline Of Intelligent Machines
1950 1952 1957 1979 1986 1997 2011 2012 2014 2016
The Learning
Machine (Alan Turing)
Machine Playing Checker
(Author Samuel)
Perceptron
(Frank Rosenblatt)
Stanford Cart
Backpropagation
(D. Rumelhart, G. Hinton, R. Williams)
Deep Blue Beats
Kasparov
Watson Wins Jeopardy
DeepMind Wins GoGoogle NN recognizing
cat in Youtube
Facebook DeepFace,
Amazon Echo

What is Machine Learning?

Marketing Offer On A New Product

Option 1- Build A Rule Engine
Age Gender Purchase
Date
Items
30 M 3/1/2017 Toy
40 M 1/3/2017 Books
…. …… ….. …..
Input Output
Age Gender Purchase
Date
Items
30 M 3/1/2017 Toy
…. …… ….. …..
Rule 1: 15 <age< 30
Rule 2: Bought Toy=Y, Last
Purchase<30 days
Rule 3: Gender = ‘M’, Bought
Toy =‘Y’
Rule 4: ……..
Rule 5: ……..
Human
Programmer

Problem with Hand
Designed Rules
Adaptability
Scalability
Closed Loop

Option 2 - Learn The Business Rules From Data
Age Gender Purchase
Date
Items
30 M 3/20/2017 Toy *
40 M 1/3/2017 Books
…. …… ….. …..
Learning
Algorithm
Model
Output
Historical Purchase Data
(Training Data)
Prediction
Age Gender Items
35 F
39 M Toy
Input - New Unseen Data

We Call This Approach Machine Learning

Why Use Machine Learning?
• Use ML when you can’t code it
• Complex tasks where deterministic solution don’t suffice
• E.g. Recognizing speech/images
• Use ML when you can’t scale it
• Replace repetitive tasks needing human like expertise
• E.g Recommendations, spam, fraud detection, machine translation.
• Use ML when you have to adapt/personalize
• E.g. Recommendation and personalization
• Use ML when you can’t track it
• E.g. Automated driving, fraud detection.

Types Of Machine Learning

Supervised Learning
It is a cat.
No, it’s a
Labrador.

Supervised Learning – How Machine Learn
Human intervention and validation required
e.g. Photo classification and tagging
Input
Label
Machine
Learning
Algorithm
Labrador
Prediction
Cat
Training Data
?
Label
Labrador
Adjust Model

Unsupervised Learning
No human intervention required
(e.g. Customer segmentation)
Input
Machine
Learning
Algorithm
Prediction

Model Training

Model Training – Split training data
All Labeled Dataset
Training Data
70% 30%

Model Training – Training w/ training data
All Labeled Dataset
Training Data
70% 30%
Training
Trial
Model

Model Training – Split the test data
All Labeled Dataset
Training Data
70% 30%
Training
Trial
Model
Test
Data

Model Training – Model evaluation
All Labeled Dataset
Training Data
70% 30%
Training
Test
Data
Evaluation
Result
Trial
Model

Model Training - Performance Measurement
All Labeled Dataset
Training Data
70% 30%
Training
Test
Data
Evaluation
Result
Trial
Model
Accuracy

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Learning

What is Deep Learning?
• Deep Learning is a subfield of machine learning
concerned with algorithms inspired by the structure
and function of the brain called artificial neural
networks.
• Data is passed through multiple non-linear
transformations to generate a prediction
• Objective: Learn the parameters of the transformations
that minimize a cost function

Performance
Data
Performance
Traditional Machine
Learning Algorithms
Deep Learning Algorithms

Sample Deep Learning Use Cases
ASR/NLU Language Translation Self Driving Cars
Playing Go Financial Risk Medical Diagnosis

Algorithms
Data
Programming
Models
GPUs &
Acceleration
The Advent of Deep Learning
image understanding
natural language
processing
speech recognition
autonomy

Artificial Neuron/Perceptron
Input:
Vector of training data x
Output:
Linear function of input
Nonlinearity:
Transform output into desired
range of value
Training
Learn the weights and bias b
by minimize loss
f(x) = 𝜎 (⟨w, x⟩ + b)
X0
X1
X2
Xn
…
w0
w1
w2
wn
OutputInputs
⟨w, x⟩ 𝜎
Neuron
b

Human Brain Neuron
Inputs Output

Neural Network
X0
Xn
Neuron 0
Neuron n
Neuron 0
Neuron n
Output
Neuron
Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer
……
……
……
w1
0
w1
1
w1
2
w1
3

Neural Network – Forward Propagation
X0
Xn
Neuron 0
Neuron n
Neuron 0
Neuron n
Output
Neuron
……
……
……
Input 5
w1
0
w1
1
w1
2
w1
3

Neural Network – Backpropagation
X0
Xn
Neuron 0
Neuron n
Neuron 0
Neuron n
Output
Neuron
……
……
……
5
Label
4
?
Input
Error/Loss
Error/LossError/Loss
Error/Loss Error/Loss
w1
0
w1
1
w1
2
w1
3

Neural Network – Backpropagation
X0
Xn
Neuron 0
Neuron n
Neuron 0
Neuron n
Output
Neuron
……
……
……
5
Label
4
?
Error/Loss
Input
Error/LossError/Loss
Error/Loss Error/Loss
W1’
0
W1’
1
W1’
2
W1’
3
Update
weights

Classification
Computer Vision – Deep Learning Approach
Raw Image Pixels
Edge Detection
Object Parts Detection
(Combination of edges)
Object Model Detection
Object Prediction
Feature
Extraction

你
好
吗
“How”
“Are”
“You”
Encoder Decoder
0.643
0.875
0.345
.
.
Input
One Word At a
Time
Model ModelEncoded Vector
Output
One Word At a
Time
Memory of previous word
influences next result
Memory of previous word
influences next result
Language Translation – Deep Learning Approach

Demo – http://amzn.to/takeselfie

• Flexible - Supports both imperative and symbolic programming
• Portable - Runs on CPUs or GPUs, on clusters, servers, desktops, or mobile phones
• Multiple Languages - C++, Python, R, Scala, Julia, Matlab, Javascript, and Perl
• Distributed on Cloud - Supports distributed training on multiple CPU/GPU machines
• Performance Optimized - Optimized C++ backend engine parallelizes both I/O and
computation
• Broad Model Support - CNN, RNN/LSTM

MXNet Architecture
BLAS Dep Engine Comm
ND
Array
Symbolic Expr
Binder
KV
Store
User Facing
Modules
System
Modules
…CPU GPU Android iOS Hardware &
OS
C++ Python R Julia… Language
Interface

MXNet API Components
• NDArray – Provides imperative tensor operations
• Symbol – Provides neural network graph and auto-
differentiation
• RNN Cell – Tools for building RNN symbolic graph
• Module – Provides interface for performing
computation with Symbol
• Data Loading – Provides iterators for reading data
• Metric - Evaluation metric to evaluate performance of
trained model

Model training flow in MXNet
Data
Loading
Input Data
(text, image,
sound, …)
Data Iterator
(NDArrayIter,
CSVIter, …)
Symbol,
RNN Cell
Module
Data Loading and Processing
Network Graph &
Error Function
Learned Model
(Network Graph,
Parameters)
Model Training
Optimizer
(sgd, adam, …)
Context
(cpu,gpu)
Metric
(acc, mse, … )
Device
Selection
Optimizer
Selection
Metric
Selection

Distributed Deep Learning – Data Parallelism
Data Shards
Model
Replica
Parameter Server W ’ = W - 𝛼∆W
W ∆W W ∆W ∆WW
Data Shards Data Shards
Model
Replica
Model
Replica

Distributed Deep Learning – Model Parallelism

Conclusion: Get Started
https://aws.amazon.com/amazon-ai/what-is-ai/

Osemeke Isibor,
Solutions Architect
iosemeke@amazon.com

Introduction to Machine Learning, Deep Learning and MXNet

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Introduction to Machine Learning, Deep Learning and MXNet

Ähnlich wie Introduction to Machine Learning, Deep Learning and MXNet (20)

Mehr von Amazon Web Services

Mehr von Amazon Web Services (20)

Introduction to Machine Learning, Deep Learning and MXNet