The slides majorly covers 1.Introduction to CNN. 2. Visual Representation of Perceptron being trained on a dataset. 3. Key Concepts used in CNN modeling.

1. Convolution Neural Network
By-Amit Kushwaha
•The basic learning entity in the network - Perceptron
•Important Aspects of Deep Learning
Presentation Covers

2. Machine Learning Engineer
@
Who am I

3. a Biological Inspiration
Artificial Neural Net ( ANN)

4. The most basic form of neural network which can also learn
Perceptron
The elementary entity of a neural network.

5. Perceptron
A Basic Linear discriminant Classifier

6. How Perceptron Work
0.3
-0.5
0.15
Activation
Function
ϴ
4
2
-2
0
ϴ= (4*0.3 + 2*(-0.5) + (-2*0.15)

7. Activation Functions

8. Training a Perceptron
Let’s train a Perceptron to mimic this pattern
x y z
0 0 1
1 1 1
1 0 1
0 1 1
Training Set
out
0
1
1
0
T(out)

9. Training a Perceptron
Perceptron Model
W1
w4
Activation
Function
ϴ OutPut
W2
W3
x
y
z
1

10. Wi = Wi + ∆Wi
∆W = -η * ( target output - perceptron output)*X
η is learning rate for perceptron
Training Rules :
W1
w4
Activation
Function
ϴ OutPut
W2
W3
x
y
z
1
Training a Perceptron

11. Let's learn wi such that it minimizes the squared error
On simplifying
Training a Perceptron

12. W1
ϴ Out
Assign random values to
Weight Vector([W1,W2,W3,W4])
x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
Training Set
out
0
1
1
0
0
1
1
0
T(out)
W2
W3
W4
x
y
z
bias
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

13. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
0
1
1
0
Training Set Weights P(out) ∆Weights
out
0
0
1
1
0
1
1
0
T(out)
0
ϴ=0 0
0
0
0
0
0
1
1
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
out
0
1
1
0
0
1
1
0

14. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
0
1
1
0
Training Set Weights P(out) ∆Weights
out
0
0
1
1
0
1
1
0
T(out)
{
{
epoch 1
epoch 2
0
ϴ=0 0
0
0
0
1
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
out
0
1
1
0
0
1
1
0

15. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
0
1
1
0
Training Set Weights P(out) ∆WeightsT(out)
{
{
epoch 1
epoch 2
0
ϴ=0 0
0
0
0
1
0
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
out
0
1
1
0
0
1
1
0

16. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
0
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
{
{
epoch 1
epoch 2
1
ϴ=1 1
0
1
1
0
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

17. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
0
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
1
ϴ=2 1
0
1
1
0
0
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

18. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
1
ϴ=1 1
0
0
0
1
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

19. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
1
ϴ=1 1
0
0
0
1
0
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

20. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
1 1 1 1
0 0 0 0
0 -1 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
0 0 0 0
0 0 0 0
1 1 1 1
1 1 1 1
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
1
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
1
ϴ=0 0
0
0
0
0
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{
{
epoch 1
epoch 2
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron

21. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
0 0 0 0
1 0 1 1
0 0 0 0
0 0 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
1 0 0 0
0 0 0 0
0 0 0 0
1 0 1 1
1 0 1 1
1 0 0 0
1 0 0 0
1 0 0 0
out
0
0
0
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
{epoch3
1
ϴ=0 0
0
0
0
0
0
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{epoch4

22. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
0 0 0 0
1 0 1 1
0 0 0 0
0 0 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
1 0 0 0
1 0 0 0
0 0 0 0
1 0 1 1
1 0 1 1
1 0 0 0
1 0 0 0
1 0 0 0
out
0
1
0
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
{epoch3
1
ϴ=1 0
0
0
0
1
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{epoch4

23. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
1 0 0 0
1 0 0 0
1 0 0 0
1 0 1 1
1 0 1 1
1 0 0 0
1 0 0 0
1 0 0 0
out
0
1
1
1
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
{epoch3
0
ϴ=-2 0
-1
-1
-1
1
0
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{epoch4

24. x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 -1 -1
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
1 0 1 1
1 0 0 0
1 0 0 0
1 0 0 0
out
0
1
1
0
1
1
1
0
Training Set Weights P(out) ∆Weights
out
0
1
1
0
0
1
1
0
T(out)
{epoch3
1
ϴ=1 1
0
0
0
0
1
1
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{epoch4

25. ∆w1 ∆w2 ∆w3 ∆w4
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
w1 w2 w3 w4
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
out
0
1
1
0
Training Set Weights P(out) ∆WeightsT(out)
1
ϴ Out
0
0
0
x
y
z
1
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
{epoch3
Wi = Wi + ∆Wi
∆Wi = -η * [P(out) -T(Out )] * xi
η is learning rate for perceptron
x y z bias
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
0 0 1 1
1 1 1 1
1 0 1 1
0 1 1 1
out
0
1
1
0
0
1
1
0
In epoch3, ‘loss’ is 0. So, we can assume
that Perceptron has learnt the pattern{epoch4

26. We trained our Perceptron Model

27. On a Linear Separable Distribution
We trained our Perceptron Model

28. It cannot generalize on data distributed like this
Limited Functionality of Linear Hyperplane

29. Neural Networks
Its like Perceptrons are together in a defined fashion

30. Two Layered Neural Network
These array of perceptrons form the Neural Nets
h1
h2
Input 1
Input 2
Output
Activation

31. Convolution Neural Network
• CNNs are Neural Networks that are sparsely (not fully)
connected on the input layer.
• This makes each neuron in the following layer responsible
for only part of the input, which is valuable for recognizing
parts of images, for example.

32. Convolutional Neural Network
•Convolutional Neural Networks are very similar to ordinary Neural
Networks
•The architecture of a CNN is designed to take advantage of the 2D
structure of an input image

33. What is an Image ?
• Image is a matrix of pixel values
• Believe me Image is nothing more than this.

34. CNN aka Deep Neural Network
1.Convolution
2.Activation Functions (ReLU)
3.Pooling or Sub Sampling
4.Classification (Fully Connected Layer)

35. Image
Filter
1. What is Convolution

36. This is Convolution
Yeah ! That’s it.

37. 2. Activation Functions (Relu)
Relu
In CNN “Relu" Activation is preferred over other Activation Functions because of it’s
inherent simplicity

38. This enables models to attain non-linearity in the
decision boundary.
•Greatly accelerate the convergence of sgd. It is argued that this is
due to its linear, non-saturating form.
•Avoids expensive computations
Relu advantages

39. 3. Max-Pooling

40. Pooling in Action
Spatial Pooling (also called subsampling or downsampling) reduces the
dimensionality of each feature map but retains the most important information.
Spatial Pooling can be of different types: Max, Average, Sum etc.

41. The purpose of the Fully Connected layer is to use
high-level features for classifying the input image into
various classes based on the training dataset.
4. Fully Connected Layer

42. Connecting dots…
1.Initialize the layers with random values.
2.The network takes a training image as input, goes through the forward propagation step (convolution, ReLU and pooling
operations along with forward propagation in the Fully Connected layer) and finds the output probabilities for each class.
3.Lets say the output probabilities for the boat image above are [0.2, 0.4, 0.1, 0.3], while the target probabilities are [0, 0,
1, 0].
4.Since weights are randomly assigned for the first training example, output probabilities are also random.
5.Calculate the total error at the output layer (summation over all 4 classes.
1.Total Error = ∑ ½ (target probability – output probability) ²
6.Use Back-propagation to calculate the gradients of the error with respect to all weights in the network and use gradient
descent to update all filter values / weights and parameter values to minimize the output error
•Repeat steps 2-6 with all images in the training set.

43. Give me the code !
Find the code at
https://github.com/yardstick17/ConnectingDots
Perceptron Training Snapshot CNN training Snapshot

44. Thanks

45. References

46. @yardstick17
Find me on web