1. Introduction to Neural
Networks
Presented by:
Hafiz Syed Adnan Ahmed

2. Introduction
• Artificial Neural Network is based on the biological nervous
system as Brain
• It is composed of interconnected computing units called
neurons
• ANN like human, learn by examples

3. Why Artificial Neural Networks?
There are two basic reasons why we are interested in
building artificial neural networks (ANNs):
• Technical viewpoint: Some problems such as
character recognition or the prediction of future
states of a system require massively parallel and
adaptive processing.
• Biological viewpoint: ANNs can be used to
replicate and simulate components of the human
(or animal) brain, thereby giving us insight into
natural information processing.
3

4. Science: Model how biological neural
systems, like human brain, work?
• How do we see?
• How is information stored in/retrieved
from memory?
• How do you learn to not to touch fire?
• How do your eyes adapt to the amount
of light in the environment?
• Related fields: Neuroscience,
Computational Neuroscience,
Psychology, Psychophysiology, Cognitive
Science, Medicine, Math, Physics.
4

5. Real Neural Learning
• Synapses change size and strength with
experience.
• Hebbian learning: When two connected neurons
are firing at the same time, the strength of the
synapse between them increases.
• “Neurons that fire together, wire together.”
5

6. Biological Neurons
• Human brain = tens of thousands
of neurons
• Each neuron is connected to
thousands other neurons
• A neuron is made of:
• The soma: body of the neuron
• Dendrites: filaments that provide
input to the neuron
• The axon: sends an output signal
• Synapses: connection with other
neurons – releases certain
quantities of chemicals called
neurotransmitters to other
neurons
6

7. Modeling of Brain Functions
7

8. Modelling a Neuron
in i
j
W j , ia j
•
•
•
•
•
aj
wj,I
inI
aI
g
:Activation value of unit j
:Weight on the link from unit j to unit i
:Weighted sum of inputs to unit i
:Activation value of unit i
:Activation function

9. What is an artificial neuron ?
• Definition : Non linear, parameterized function with
restricted output range
y
n 1
y
f w0
wi xi
i 1
w0
x1
x2
x3

10. Simple Neuron
X1
Inputs
X2
Output
Xn
b

11. An Artificial Neuron
synapses
neuron i
x1
x2
Wi,1
Wi,2
…
…
xi
Wi,n
xn
n
net input signal
net i (t )
wi , j (t ) x j (t )
j 1
output
x i (t )
f i ( net i ( t ))

12. Activation functions
20
18
16
Linear
14
12
10
y
8
6
x
4
2
0
0
2
4
6
8
10
12
14
16
18
20
2
1.5
Logistic1
1
0.5
0
y
-0.5
1
-1
exp(
x)
-1.5
-2
-10
-8
-6
-4
-2
0
2
4
6
8
10
2
Hyperbolic tangent
1.5
1
0.5
y
-1
-1.5
-2
-10
-8
-6
-4
-2
0
2
4
6
8
10
exp( x )
exp(
x)
exp( x )
0
-0.5
exp(
x)

13. How do NNs and ANNs work?
• Information is transmitted as a series of
electric impulses, so-called spikes.
• The frequency and phase of these spikes
encodes the information.
• In biological systems, one neuron can be
connected to as many as 10,000 other
neurons.
• Usually, a neuron receives its information
from other neurons in a confined area
13

14. Navigation of a car
• Done by Pomerlau. The network takes inputs from a 34X36 video image
and a 7X36 range finder. Output units represent “drive straight”, “turn
left” or “turn right”. After training about 40 times on 1200 road
images, the car drove around CMU campus at 5 km/h (using a small
workstation on the car). This was almost twice the speed of any other
non-NN algorithm at the time.
14

15. Automated driving at 70 mph on a
public highway
Camera
image
30 outputs
for steering
4 hidden
units
30x32 pixels
as inputs
30x32 weights
into one out of
four hidden
unit

16. Computers vs. Neural Networks
“Standard” Computers
Neural Networks
one CPU
highly parallel
processing
fast processing units
units
slow processing
reliable units
unreliable units
static infrastructure
infrastructure
dynamic
16

17. Neural Network
Input Layer
Hidden 1
Hidden 2
Output Layer

18. Network Layers
The common type of ANN consists of three layers
of neurons: a layer of input neurons connected to
the layer of hidden neuron which is connected to
a layer of output neurons.

19. Architecture of ANN
• Feed-Forward networks
Allow the signals to travel one way from input to
output
• Feed-Back Networks
The signals travel as loops in the network, the
output is connected to the input of the network

21. Comparison of Brains and Traditional
Computers
• 200 billion neurons, 32
trillion synapses
• Element size: 10-6 m
• Energy use: 25W
• Processing speed: 100 Hz
• Parallel, Distributed
• Fault Tolerant
• Learns: Yes
• Intelligent/Conscious:
Usually
• 1 billion bytes RAM but
trillions of bytes on disk
• Element size: 10-9 m
• Energy watt: 30-90W (CPU)
• Processing speed: 109 Hz
• Serial, Centralized
• Generally not Fault Tolerant
• Learns: Some
• Intelligent/Conscious:
Generally No

22. Neural Networks (Applications)
• Face recognition
• Time series prediction
• Process identification
• Process control
• Optical character recognition
• Adaptative filtering
• Etc…

23. And Finally….
“If the brain were so simple
that we could understand it
then we’d be so simple that
we couldn’t”

24. Introduction is End of Neural Networks