1. Artificial Neural Networks
Presented By:
Ezhilan A K
Prakash K
PG Scholars – Industrial
Engineering, PSG College of
Technology.

2. HISTORY
1943 − The concept of neural network started with the work of physiologist, Warren
McCulloch, and mathematician, Walter Pitts, when in 1943 they modelled a simple
neural network using electrical circuits in order to describe how neurons in the
brain might work.
1949 − Donald Hebb’s book, The Organization of Behaviour, put forth the fact that
repeated activation of one neuron by another increases its strength each time they
are used.
1958 − A learning method for McCulloch and Pitts neuron model named Perceptron
was invented by Rosenblatt.
1960 − Bernard Widrow and Marcian Hoff developed models called "ADALINE" and
“MADALINE.”
1961 − Rosenblatt made an unsuccessful attempt but proposed the
“backpropagation” scheme for multilayer networks.
1969 − Multilayer perceptron (MLP) was invented by Minsky and Papert.

3. INTRODUCTION
• Neural networks (NN) are parallel computing (many calculations or
the execution of processes are carried out concurrently)
• Which is basically an attempt to make a computer model of the
brain.
• It works like a Neuron of a Human Brain.
• The main objective is to develop a system to perform various
computational tasks faster than the traditional systems.
• These tasks include pattern recognition and classification,
approximation, optimization, and data clustering.

4. BIOLOGICAL NEURAL NETWORK
• The Basic unit of Nervous system is Neuron (Nerve cell).
• Neuron is a special biological cell that processes information.
• Human Brain contains huge number of neurons, approximately 1011
with numerous interconnections.

5. NEURON-SCHEMATIC DIAGRAM

6. NEURON-ANATOMY
• Dendrites − They are tree-like branches, responsible for receiving
the information from other neurons it is connected to or in other
sense, we can say that they are like the ears of neuron.
• Soma − It is the cell body of the neuron and is responsible for
processing of information, they have received from dendrites.
• Axon − It is like a cable through which neurons send the information.
• Synapses − It is the connection between the axon and other neuron
dendrites.

7. ARTIFICIAL NEURAL NETWORK
• Artificial Neural Network (ANN) is an efficient computing system
whose central theme is borrowed from the analogy of biological
neural networks.
• ANN acquires a large collection of units that are interconnected in
some pattern to allow communication between the units.
• These units, also referred to as Nodes or Neurons.

8. ANN Vs BNN
Biological Neural Network Artificial Neural Network
Soma Node
Dendrite Input
Synapse Weight or Interconnections
Axon Output

9. Comparison Between ANN and BNN
CRITERIA BNN ANN
Processing
Massively parallel, slow
but superior than ANN
Massively parallel, fast but
inferior than BNN
Size
1011 neurons and 1015
interconnections
102 to 104 nodes mainly
depends on the type of
application in a network
designer
Learning
They can tolerate
ambiguity
Very precise, structured and
formatted data is required to
tolerate ambiguity
Fault
tolerance
Performance degrades with
even partial damage
It is capable of robust
performance, hence has the
potential
to be fault tolerant
Storage
capacity
Stores the information in
the synapse
Stores the information in
continuous memory locations

10. Start
Initialize Input
Define and format
Input data
Divide data into 3 sets
(Training, Testing, Validation)
Create Network Layers Train the Network
Make Total Error
= 0
Last
pattern
trained
Apply the Pattern and
Train
Total Error
<
Final Target
Error
Get Error for each
output Neuron and
add to Total Error
Simulate Network End
YES
NO
YES
NO
F
L
O
W
C
H
A
R
T

11. ARTIFICIAL NEURON-REPRESENTATION
X1
W1
X2
X3
Xn
W2
W3
Wn
x1, x2,……,xn – Input to the
artificial neuron
w1,w2,….,wn – Weights
attached to the input links
∑ - Summation unit
– Thresholding Function
y - Output
Summation Unit
Thresholding Unit

12. Contd…
• x1, x2, x3, .... xn are the n inputs to the artificial neuron w1, w2,
…. wn, are the weights attached to the input links.
• In general we know that biological neuron receives all inputs
through dendrites, sums them and produce an output if sum is
greater than threshold value.
• The input signals are passed on to the cell body(SOMA),
through the SYNAPSE which may accelerate or retard an
arriving signal.

13. Contd…
• Effective synapse which transmits a stronger signal will have a
correspondingly larger weight while a weak synapse will have
smaller weights. Thus weights are multiplicative factors of the
inputs to account for the strength of the synapse.
• Total Input (I) = w1x1 + w2x2 +….…+ wnxn
= 𝑖=1
𝑛
𝑤𝑖 𝑥𝑖

14. Contd…
• To generate final output (y), the sum is passed on to a non-linear
filter ∅ called Activation function, or Transfer function or squash
function which releases the output.
y= ∅ (I)
• Activation functions used in ANN
• Thresholding function (Binary)
• Signum function(Bi-Polar)
• Sigmoidal function ( Both Binary and Bi-Polar)
• In Thresholding function, the sum is compared with a threshold value
𝜃, If the value of I is greater than 𝜃, then the output is 1 else it is 0.
• In Signum function, the sum is compared with a threshold value 𝜃, If
the value of I is greater than 𝜃, then the output is 1 , if I is lesser or
equivalent to 𝜃 it is -1.

15. Contd…
Step(x) = 1 if x ≥ t, else 0 threshold=t
Sign(x) = +1 if x ≥ 0, else –1
Sigmoid(x) = 1/(1+e-Øx)

16. Human EYE Perceptron

17. Early Neural Network Architecture
1. Rosenblatt’s Perceptron
The perceptron is a computational model of the retina of the eye
and hence, is named ‘perceptron’. The network comprises three units,
the
1. Sensory unit (S)
2. Association unit (A)
3. Response Unit (R)

18. Rosenblatt’s Perceptron
Output
Photo detectors
Sensory Unit (S)
Output
Association unit (A) Response Unit (R)
Fixed
weights Adjustable
weights

19. Contd…
• The Sensory Unit (S) receives an input and provides an electric signal
as output ( ‘0’ or ‘1’ – Binary form).
• The Sensory Unit is randomly connected to the Association Unit (A).
• The Association Unit predicts the output of ‘S’ Unit.
• The ‘R’ Unit comprises Perceptron, which receives the results of
predicates in binary form.

20. 2. ADALINE Network
• The Adaptive Linear Neural Element Network formed by
Bernard Widrow of Stanford University.
• This Network uses Supervised Learning.
• There is only one output neuron and the output values are
bipolar(-1 or +1)
• Input can be Binary or Bipolar .
• If the weighted sum of inputs is greater than or equal to 0, then
the output is 1 otherwise -1.

21. X1
W1
X2
X3
Xn
W2
W3
Wn
x1, x2,……,xn – Input to the
artificial neuron
w1,w2,….,wn – Weights
attached to the input links
∑ - Summation unit
– Thresholding Function
y - Output
A simple ADALINE network
Summation Unit
Thresholding Unit

22. 3. MADALINE Network
• MADALINE is created by combining a number of ADALINES.
• This network consist of many layers.
• This Network uses Supervised Learning
Adaline
Adaline
Adaline

23. Limitation of Perceptron
• Perceptron cannot handle, in particular, task which are linearly
inseparable.
• Set of points in two dimensional spaces are linearly separable if
the sets can be separated by straight line.
• So we go for Back propagation training.

24. Linearly Separable Vs Linearly Inseparable

25. Modern Neural Network Architectures
The modern neural network architecture are as follows
1. Single Layer Feedforward Network
2. Multilayer Feedforward Network
3. Recurrent Networks

26. Single Layer Feedforward Network
• Single Layer Feedforward Network consist of two layers, namely
input and the output layer.
• The input layer neurons receives the input signals and the output
layer neurons receives the output signals
• The synaptic links will carry the weights from input to output, but
not vice versa.
• These networks are acyclic in nature.

27. Contd…
Input Layer
Output Layer

28. Multilayer Feedforward Network
• Multilayer Feedforward Network contains multiple layers.
• This also contains input and the output layers, in addition to that I
have intermediate layers called Hidden layers.
• The hidden layer performs computations before directing the input
to the output.
• One hidden layer is sufficient for the large majority of problems.
• The optimal size of the hidden layer is usually between the size of
the input and size of the output layers.

29. Contd…

30. Recurrent Network
• Recurrent Network differs from other architecture in the sense that it
contains at least one feedback loop.
• In this the output of a neuron is feedback to the input neuron or the
hidden neuron.

31. ARTIFICIAL NEURAL NETWORK LEARNING
METHODS
(TRAINING METHODS)

32. ANN Algorithm Learning
Methods
Supervised Learning
Unsupervised
Learning
Reinforced Learning
Error Correction
Gradient Descent
Stochastic
Least Mean Square Back Propagation Hebbian Competitive
Learning Methods

33. • Every input pattern that is used to train the network is associated
with an output pattern.
• A teacher is available to indicate whether a system is performing
correctly, or to indicate the amount of error in system performance.
• Comparison is made with computed output and the correct expected
output, to determine the error.
1. Supervised learning

34. 2.Unsupervised Learning
• This is Learning by doing.
• The target output is not presented to the network. It is as if there is
no teacher to present the desired patterns.
• The system learns of its own by discovering and adapting to structural
features in the input patterns.

35. 3.Reinforced learning
• A teacher is available.
• He/she Does not present the expected answer.
• But indicates if the computed output is correct or incorrect.
• This indication provided helps the network in its learning process.

36. Training Cycle Vs Error

37. Training – Validation – Test

38. A Simple Neural Network

39. Backpropagation
• The backpropagation algorithm (Rumelhart and McClelland, 1986) is
used in layered feed-forward Artificial Neural Networks.
• Back propagation is a multi-layer feed forward, supervised learning
network based on gradient descent learning rule.
• We provide the algorithm with examples of the inputs and outputs
we want the network to compute, and then the error (difference
between actual and expected results) is calculated.
• The idea of the backpropagation algorithm is to reduce this error,
until the Artificial Neural Network learns the training data.

40. Backpropagation Preparation
• Training Set
A collection of input-output patterns that are used to train the
network
• Testing Set
A collection of input-output patterns that are used to assess network
performance
• Learning Rate-η
A scalar parameter, analogous to step size in numerical integration,
used to set the rate of adjustments

41. Backpropagation Algorithm
• Randomly choose the initial weights
• The activation function of the artificial neurons in ANNs implementing
the backpropagation algorithm is a weighted sum (the sum of the
inputs xi multiplied by their respective weights wji).
• The most common output function is the sigmoidal and is calculated.
• Since the error is the difference between the actual and the desired
output, the error depends on the weights, and we need to adjust the
weights in order to minimize the error.
• The backpropagation algorithm now calculates how the error
depends on the output, inputs, and weights.

42. Contd…
• We need to calculate how much the error depends on the
output.
• Calculate the adjusted weight.
• If we want to adjust vik, the weights (let’s call them vik ) of a
previous layer, we need first to calculate how the error depends
not on the weight, but in the input from the previous layer.
• Proceed till we get minimum error.
• Once we get the minimum error, end the iteration.

43. Characteristics of Neural Network
• The Neural Network exhibit mapping capabilities, that is, they can
map inputs patterns to their associated output patterns.
• The Neural Network learn by examples.
• NN Architecture can be trained with known examples of a problem
before they are tested for their inference capability on unknown
instances of the problem.
• They can identify new objects previously untrained.
• They can predict new outcomes from past trends.
• Neural Networks are Robust – Fault tolerant .

44. MATLAB Input Data Matrix

45. MATLAB Output Data Matrix

46. Input and Output Data Importing
S
t
e
p
1

47. S
t
e
p
2
Dividing the Data for Training,
Validation, Testing

48. S
t
e
p
3
Defining Hidden Neuron

49. Training the Network
S
t
e
p
4

50. Making Target Error = 0
S
t
e
p
4

52. Operating Characteristics

53. Advantages of Artificial Neural
Network
1. A neural network can perform tasks that a linear program can
not.
2. When an element of the neural network fails, it can continue
without any problem by their parallel nature.
3. A neural network learns and does not need to be
reprogrammed.
4. It can be implemented in any application.
5. It can be implemented without any problem.

54. Disadvantages of Artificial Neural Network
1. The neural network needs training to operate
2. The architecture of a neural network is different from the
architecture of microprocessors therefore needs to be
emulated.
3. Requires high processing time for large neural networks.

55. Applications
• Signal processing: suppress line noise, with adaptive echo canceling,
blind source separation.
• Siemens successfully uses neural networks for process automation in
basic industries, e.g., in rolling mill control more than 100 neural networks
do their job, 24 hours a day.
• Robotics - navigation, vision recognition.
• Pattern recognition, i.e. recognizing handwritten characters, e.g. the
current version of Apple's Newton uses a neural net.
• Medicine, storing medical records based on case information.
• Speech production: reading text aloud (NETtalk).
• Vision: face recognition , edge detection, visual search engines.
• Financial Applications: Time series analysis, Stock market prediction.
• Data Compression: speech signal, image, e.g. faces.

56. A Glimpse on Neural Network
• NN is interconnected network that resembles human brain.
• The most important characteristic of NN is its ability to learn.
When presented with training set (form of supervised learning)
where input and output values are known.
• NN model could be created to help with classifying new data.
• Results that are achieved by using NN are encouraging,
especially in some fields like pattern recognition.

57. References:
• Neural Networks, Fuzzy Systems, and Evolutionary Algorithms
Synthesis and Application by S.Rajasekaran,
G.A.Vijayalakshmi Pai.
• Introduction to Neural Networks – Open Course Ware,
Massachusetts Institute of Technology.
• Back Propagation Neural Network - Dr K Vijayarekha
• Mahesh B. Parappagoudar, D.K. Pratihar , G.L. Datta , Forward
and reverse mappings in green sand mould system using
neural networks. Applied Soft Computing 8 (2008) 239–260.
• Marek Vrabea, Ildiko Mankovaa, Jozef Benoa, Jaroslav
Tuharskýa, Surface roughness prediction using artificial neural
networks when drilling Udimet 720. Procedia Engineering 48
( 2012 ) 693 – 700.

58. Thank You