1. Pranjut Gogoi
Lead Consultant
Knoldus Software LLP
Artificial Neural Network
..first interaction

4. ● Dendrites: A short branched extension of a nerve cell, along which impulses
received from other cells at synapses are transmitted to the cell body.
● Axon: An axon is a long, slender projection of a nerve cell, or neuron, that
typically conducts electrical impulses away from the neuron's cell body.
● Synapse: A junction between two nerve cells, consisting of a minute gap
across which impulses pass by diffusion of a neurotransmitter.
● Cell body or Soma: The soma or "cell body" is the bulbous, non-process
portion of a neuron or other brain cell type, containing the cell nucleus.
● Nucleus: The nucleus is a membrane-enclosed organelle found in eukaryotic
cells.

5. Brain facts
●
Having density of neurons around approx 10 ^ 11 neurons
●
Each neuron is connected to other approx 10 ^ 4 neurons
●
Switching time of biological neuron is approx 10 ^ -3
seconds
●
And it just take around 10 ^ -1 seconds to identify someone
known.
●
Switching time of computer is 10 ^ -10 seconds
●
Not efficient enough to take complex decisions like human.
●
Brain’s neural net is embarrassingly parallel.

7. Artificial Neural network
●
ANN works on real-valued, discrete-valued and vector
valued.
●
An artificial neuron is a mathematical function conceived as
a model of biological neurons
●
The artificial neuron receives one or more inputs and sums
them to produce an output.
●
The sums of each node are weighted, and the sum is
passed through a non-linear function known as an
activation function or transfer function.
●
The activation functions usually have a sigmoid shape, but
they may also take the form of other non-linear
functions, piecewise linear functions, or step functions.

11. Neural net with Sigmoid function
● Weights for first layer of first feature = W11, W12, W13
● Weights for first layer of second feature = W21, W22, W23
● Input 1st
Neuron, Input 2nd
Neuron, Input 3rd
Neuron = (X1W11 + X2W21),
(X1W12 + X2W22), (X1W13 + X2W23)
e.g. For input (3, 5), Z21 = 3W11 + 5W21, Z22 = 3W12 + 5W22, Z3 = 3W13 +
5W23
● def sigmoid(z) = 1/(1 + e ^ -z)
● Z1 = sigmoid(Z21), Z2 = sigmoid(Z22), Z3 = sigmoid(Z23)
● Weights for output layer = W31, W32, W33
● Weighted input for output layer, Z = Z1W31 + Z2W32 + Z3W33
● Y = sigmoid(z) ---→ your predicted output

12. See what Tom Mitchell has to say
"A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance
at tasks in T, as measured by P, improves with experience E“ –
T. Michell (1997)

13. Backpropagation
● Inspired from biological neural net
● The backward propagation of errors
● Use Gradient descent for weight update
● Its Supervised learning

14. Gradient descent
● What is gradient??
●
Ans: An increase or decrease in the magnitude of a property observed in
passing from one point or moment to another
●
Or
●
In mathematics, the gradient is a multi-variable generalization of the
derivative.
● Error = - Y
● Squared error function E(w) = 1/2
● Gradient
● Weight update: where

15. Gradient descent contd….
● Derivative of error or if we find the gradient it will be ….

16. Gradient descent contd….
● Derivative of error or if we find the gradient it will be ….

17. Stochastic Gradient descent
●
The Local minima problem
●
Gradient descent training rule computes
weight updates after summing over all
the training examples.
●
Stochastic gradient descent
approximates the gradient descent
search by updating weights
incrementally, following the calculation
of the error for each individual example.

18. Back Propagation Algorithm

19. Thank You