Dissertation character recognition - Report

DISSERTATION
ON

CHARACTER RECOGNITION USING
NEURAL NETWORK
IN PARTIAL FULFILLMENT OF
SURVEY ON RESEARCHER’S TOPIC
AS A PART OF CURRICULUM IN
MASTER’S DEGREE IN COMPUTER APPLICATION (M.C.A)
SEMESTER – v
Gujarat University (2010-2011)

Submitted By
Sachinkumar M. Bharadva
Dhara Solanki

Internal Guide
Mr. Sandeep R. Vasant

A.E.S Institute Of Computer Studies
(AESICS)
B++ ACCREDITATION BY NAAC OF UGC
THE AHMEDABAD EDUCATION SOCIETY
(Affiliated to Gujarat University, M.C.A Programme)
H.L. College Campus, P.B. 4206, Navarangpura,
AHMEDABAD - 380009

CHARACTER RECOGNITION USING NEURAL NETWORK

INDEX
___________________________________________________
SR.NO._______
1.

2.

3.

4.

_CONTENTS_____

__PAGE NO.

Introduction to Neural Networks
1.1 What is a Neural Network?
1.2 Historical background
1.2.1 History 1940’s to 1970’s
1.2.2 History 1980’s to the Present
1.3 Why use Neural Networks?
1.4 Neural Networks versus Conventional Computers

12
13

Human and Artificial Neurones - Investigating the Similarities
2.1 The Human Brain, Neural Network and Computers
2.2 The Artificial Neural Network and Artificial Intelligence
2.3 Neural Network and Neuro Science

15
17
20

Architecture
3.1 Architecture of Neural Networks
3.2 Feed-forward (associative) networks
3.3 Feedback (autoassociative) networks
3.4 Network Layers
3.5 Acyclic Network
3.6 Modular Neural Network
3.7 Perceptrons

2

6
6
10

22
24

24
25
26
26
27

Applications of Neural Networks
4.1 Neural Networks in Practice
30
4.2 Neural Networks in Medicine
31
4.2.1 Modelling and Diagnosing the Cardiovascular System
4.2.2 Electronic Noses - Detection and Reconstruction
4.2.3 Instant Physician - a commercial neural net
- diagnostic program
4.3 Neural Networks in Business
33
4.4 Marketing
33
4.5 Credit Evaluation
34
4.6 Signal Processing
34
4.7 Speech Recognition
34
4.8 Intelligent Control
35
4.9 Financial Forecasting
35
4.10Condition Monitoring
36
4.11 Neuro Forecasting
36
4.12 Pattern Analysis[B35]
36
4.13 Classification
37

AES Institute Of Computer Studies


INDEX
___________________________________________________
SR.NO._______
5.

_CONTENTS_____

6.

Introduction to Character Recognition
5.1 What is Character Recognition?
5.2 History for Character Recognition
5.3 Steps for the Character Recognition
How to Recognize Character

7.

__PAGE NO.

Matlab
7.1 What is Matlab?
7.2 History and Present
7.3 Neural Network Toolbox

8.

39
40
41
42

47
48
49

Literature Survey
8.1 Kevin Larson Paper and Application
8.2 Ketil Hunn Paper and Application
8.3 Current Market Scenario
8.4 Current Research

54
59
62
64

Proposed Work
9.1 Work Introduction
9.2 Application
9.3 Accuracy and Result
9.4 Recognize Digit and Success/Failure Phase

66
66
71
72

10.

Enhancements

75

11.

Conclusion

78

12.

Sources and Bibliography

80

9.



ACKNOWLEDGEMENT

I sincerely
successful

thank to

completion

of

all
my

the persons whoever played a vital role in the
Dissertation

under

titled

CHARACTER

RECOGNITION USING NEURAL NETWORK. We are grateful to the whole staff
of the AES Institute of Computer Studies under whose guidance I completed my
successful Dissertation, who devoted their precious time. In spite of, their in busy
schedules they always came forward to guide us in our work whenever needed.

We are also thankful to Mr. Bipin Mehta for his constant support. We accept
this opportunity to acknowledgement all other individuals for their direct or indirect
contribution to our work.

We are also very thankful to Prof. Sandeep Vasant and to our staff members
without whom it would have been impossible for us to carry out the Dissertation
work. Again with a word of thanks to all we represent this Dissertation.



INTRODUCTION TO NEURAL
NETWORK


1


1.1 What is a Neural Network?
An artificial neural network is an information processing system that has been
developed as a generalization of the mathematical model of human cognition (faculty
of knowing). A neural network is a network of interconnected neurons, inspired from
the studies of the biological nervous system. Neural network functions in a way
similar to the human brain. The function of a neural network is to produce an output
pattern when presented with an input pattern.

An Artificial Neural Network (ANN) is an information processing paradigm
that is inspired by the way biological nervous systems, such as the brain, process
information. The key element of this paradigm is the novel structure of the
information processing system. It is composed of a large number of highly
interconnected processing elements (neurones) working in unison to solve specific
problems. ANNs, like people, learn by example. An ANN is configured for a specific
application, such as pattern recognition or data classification, through a learning
process. Learning in biological systems involves adjustments to the synaptic
connections that exist between the neurones. This is true of ANNs as well.
A neural network is the study of networks consisting of nodes connected by
adaptable weights, which store experimental knowledge from ktask examples
through a process of learning.

The human brain provides proof of the existence of massive neural networks
that can succeed at those cognitive, perceptual, and control tasks in which humans are
successful. The brain is capable of computationally demanding perceptual acts (e.g.
recognition of faces, speech) and control activities (e.g. body movements and body
functions). The advantage of the brain is its effective use of massive parallelism, the
highly parallel computing structure, and the imprecise information processing
capability.


2


The human brain is a collection of more than 10billion interconnected
neurons. Each neuron is a cell (Figure1) that uses biochemical reactions to receive,
process, and transmit information.

Treelike networks of nerve fibers called dendrites are connected to the cell
body or soma, where the cell nucleusis located. Extending from the cell body is a
single long fiber called the axon, which eventually branches into strands and
substrands, a dare connected to other neurons through synaptic terminals or synapses.
The transmission of signals from one neural onto another at synapsesisa complex
chemical process in which specific transmitter substances are released from the
sending end of the junction. The effect is to raise or lower the electrical potential in
side the body of the receiving cell. If the potential reaches a threshold, a pulse is sent
down the axon and the cell is ‘fired’.


3


Traditionally, the term neural network had been used to refer to a network or
circuit of biological neurons; the modern usage of the term often refers to artificial
neural networks, which are composed of artificial neurons or nodes. Thus the term has
two distinct usages:
Biological neural networks are made up of real biological neurons that are
connected or functionally related in the peripheral nervous system or the central
nervous system. In the field of neuroscience, they are often identified as groups of
neurons that perform a specific physiological function in laboratory analysis.
Artificial neural networks are made up of interconnecting artificial neurons
(programming constructs that mimic the properties of biological neurons). Artificial
neural networks may either be used to gain an understanding of biological neural
networks, or for solving artificial intelligence problems without necessarily creating a
model of a real biological system. The real, biological nervous system is highly
complex and includes some features that may seem superfluous based on an
understanding of artificial networks.

In general a biological neural network is composed of a group or groups of
chemically connected or functionally associated neurons. A single neuron may be
connected to many other neurons and the total number of neurons and connections in
a network may be extensive. Connections, called synapses, are usually formed from


4


axons to dendrites, though dendrodendritic microcircuits and other connections are
possible. Apart from the electrical signaling, there are other forms of signaling that
arise from neurotransmitter diffusion, which have an effect on electrical signaling. As
such, neural networks are extremely complex.
Artificial intelligence and cognitive modeling try to simulate some properties
of neural networks. While similar in their techniques, the former has the aim of
solving particular tasks, while the latter aims to build mathematical models of
biological neural systems.
In the artificial intelligence field, artificial neural networks have been applied
successfully to speech recognition, image analysis and adaptive control, in order to
construct software agents (in computer and video games) or autonomous robots. Most
of the currently employed artificial neural networks for artificial intelligence are
based on statistical estimation, optimization and control theory.
The cognitive modeling field involves the physical or mathematical modeling
of the behavior of neural systems; ranging from the individual neural level (e.g.
modeling the spike response curves of neurons to a stimulus), through the neural
cluster level (e.g. modeling the release and effects of dopamine in the basal ganglia)
to the complete organism (e.g. behavioral modeling of the organism's response to
stimuli). Artificial intelligence, cognitive modeling, and neural networks are
information processing paradigms inspired by the way biological neural systems
process data.


5


1.2 Historical Background
The concept of neural networks started in the late 1800s as an effort to
describe how the human mind performed.
Neural network simulations appear to be a recent development. However, this
field was established before the advent of computers, and has survived at least one
major setback and several eras.
The Austrian School of economics theory of spontaneous order was explained
by Murray Rothbard to have been first realised by Zhuangzi (Chuang Tzu) who said,
"Good order results spontaneously when things are let alone."
Many important advances have been boosted by the use of inexpensive computer
emulations. Following an initial period of enthusiasm, the field survived a period of
frustration and disrepute. During this period when funding and professional support
was minimal, important advances were made by relatively few researchers. These
pioneers were able to develop convincing technology which surpassed the limitations
identified by Minsky and Papert.

1.2.1 History 1940’s to 1970’s
1940 – Donald Hebb
In the late 1940s Donald Hebb made one of the first hypotheses of learning
with a mechanism of neural plasticity called Hebbian learning. Hebbian learning is
considered to be a 'typical' unsupervised learning rule and it and later variants were
early models for long term potentiation. These ideas started being applied to
computational models in 1948 with Turing's B-type machines and the perceptron.
In 1943 - Warren McCulloch
The first artificial neuron was produced in 1943 by the neurophysiologist
Warren McCulloch and the logician Walter Pits. But the technology available at that
time did not allow them to do too much.


6


Walter Pitts wrote a paper on how neurons might work. In order to describe
how neurons in the brain might work, they modeled a simple neural network using
electrical circuits.
1949 - Donald Hebb
In 1949, Donald Hebb wrote The Organization of Behavior, a work which
pointed out the fact that neural pathways are strengthened each time they are used, a
concept fundamentally essential to the ways in which humans learn. If two nerves fire
at the same time, he argued, the connection between them is enhanced.
1950 - Nathanial Rochester
In 1950’s when computers became more advanced, It was finally possible to
simulate a hypothetical neural network. So Nathanial Rochester from the IBM
research laboratories trying to implement on it but unfortunately for him, the first
attempt to do so failed.
1959 - Bernard Widrow and Marcian Hoff
In 1959, Bernard Widrow and Marcian Hoff of Stanford developed models
called "ADALINE" and "MADALINE." In a typical display of Stanford's love for
acronymns, the names come from their use of Multiple ADAptive LINear Elements.
ADALINE was developed to recognize binary patterns so that if it was reading
streaming bits from a phone line, it could predict the next bit. MADALINE was the
first neural network applied to a real world problem, using an adaptive filter that
eliminates echoes on phone lines. While the system is as ancient as air traffic control
systems, like air traffic control systems, it is still in commercial use.
1962 - Widrow & Hoff
In 1962, Widrow & Hoff developed a learning procedure that examines the
value before the weight adjusts it (i.e. 0 or 1) according to the rule: Weight Change =
(Pre-Weight line value) * (Error / (Number of Inputs)). It is based on the idea that
while one active perceptron may have a big error, one can adjust the weight values to


7


distribute it across the network, or at least to adjacent perceptrons. Applying this rule
still results in an error if the line before the weight is 0, although this will eventually
correct itself. If the error is conserved so that all of it is distributed to all of the
weights than the error is eliminated.
1969 - Minsky and Papert
Minsky and Papert, published a book (in 1969) in which they summed up a
general feeling of frustration (against neural networks) among researchers, and was
thus accepted by most without further analysis. Currently, the neural network field
enjoys a resurgence of interest and a corresponding increase in funding.
About 15 years after the publication of McCulloch and Pitt's pioneer paper, a
new approach to the area of neural network research was introduced. In 1958 Frank
Rosenblatt, a neuro-biologist at Cornell University began working on the Perceptron.
The perceptron was the first "practical" artificial neural network. It was built using the
somewhat primitive and "ancient" hardware of that time. The perceptron is based on
research done on a fly's eye. The processing which tells a fly to flee when danger is
near is done in the eye. One major downfall of the perceptron was that it had limited
capabilities and this was proven by Marvin Minsky and Seymour Papert's book of
1969 entitled, "Perceptrons".
1972 - Kohonen and Anderson
In 1972, Kohonen and Anderson developed a similar network independently,
They both used matrix mathematics to describe their ideas but did not realize that
what they were doing was creating as array of analog ADALINE circuits. The
neurons are supposed to activate a set o outputs instead of just one.
The later success of the neural network, traditional con Neumann architecture
took over the computing scene, and neural research was let behind, Neumann himself
suggested the imitation of neural functions by using telegraph relays or vacuum tubes.


8


In the same time paper was written that suggested of some neural networks led
to an exaggeration of the potential of neural networks, especially practical technology
at the time.
The idea of a computer which programs itself is very appealing. If microsoft’s
windows 2000 could reprogram itself, it might be able to repair the thousands of bugs
that the programming staff made. Such ideas were appealing but very difficult to
implement
After that first multilayered network was developed in 1975, an
unsupervised network.


9


1.2.2 History 1980’s to the Present
After 1975 when first multilayered network was developed the scientist
interest renewed and trying to making something new in the neural network field.
After 1980’s they gave new way to the neural network
1982 - John Hopfield
In 1982, John Hopfield of Caltech presented a paper to the National Academy
of Sciences. His approach was to create more useful machine by using bidirectional
lines. Previously, the connections between neurons was only one way.
Reilly and Cooper
In the same year 1982, Reilly and Cooper used a “Hybrid network” with
multiple layers, each layer using a different problem-solving strategy.
There was a joint US-Japan conference on Cooperative/Competitive Neural
Networks. Japan announced a new Fifth Generation effort on neural networks, and US
papers generated worry that the US could be left behind in the field. (Fifth generation
computing involves artificial intelligence. First generation used switches and wires,
second generation used the transister, third state used solid-state technology like
integrated circuits and higher level programming languages, and the fourth generation
is code generators.) As a result, there was more funding and thus more research in the
field.
1986 - Based on Widrow-Hoff Rule
In this year multiple layered neural networks flash in news, the problem occur
was how to extend the Widrow-Hoff rule to multiple layers. Three independent
groups of researchers,
One of which included David Rumelhart, a former member of Stanford’s
psychology department, came up with similar ideas which are now called back
propagation networks because it distributes pattern recognition errors throughout the


10


network. Hybrid networks used just two layers, these back-propagation network use
many. The result is that back-propagation networks are “slow learners,” needing
possibly thousands of iterations to learn.
After that neural networks are used in several applications, some o which we
will describe in below. The fundamental idea behind the nature o neural networks is
that if it works in nature, it must be able to work in computers. The future of neural
networks, through, lies in the development of hardware. Much like the advance chessplaying machines like Deep Blue, fast, efficient neural networks depend on hardware
being specified for its eventual use.


11


1.3 Why use Neural Networks?
Neural networks, with their remarkable ability to derive meaning from
complicated or imprecise data, can be used to extract patterns and detect trends that
are too complex to be noticed by either humans or other computer techniques. A
trained neural network can be thought of as an "expert" in the category of information
it has been given to analyse. This expert can then be used to provide projections given
new situations of interest and answer "what if" questions.
Other advantages include:
1. Adaptive learning:
An ability to learn how to do tasks based on the data given for training
or initial experience.
2. Self-Organization:
An ANN can create its own organization or representation of the
information it receives during learning time.
3. Real Time Operation:
ANN computations may be carried out in parallel, and special
hardware devices are being designed and manufactured which take
advantage of this capability.
4. Fault Tolerance via Redundant Information Coding:
Partial destruction of a network leads to the corresponding degradation
of performance. However, some network capabilities may be retained
even with major network damage.


12


1.4 Neural Networks versus Conventional Computer
Neural networks take a different approach to problem solving than that of
conventional computers. Conventional computers use an algorithmic approach i.e. the
computer follows a set of instructions in order to solve a problem. Unless the specific
steps that the computer needs to follow are known the computer cannot solve the
problem. That restricts the problem solving capability of conventional computers to
problems that we already understand and know how to solve. But computers would be
so much more useful if they could do things that we don't exactly know how to do.
Neural networks process information in a similar way the human brain does. The
network is composed of a large number of highly interconnected processing elements
(neurones) working in parallel to solve a specific problem. Neural networks learn by
example. They cannot be programmed to perform a specific task. The examples must
be selected carefully otherwise useful time is wasted or even worse the network might
be functioning incorrectly. The disadvantage is that because the network finds out
how to solve the problem by itself, its operation can be unpredictable.
On the other hand, conventional computers use a cognitive approach to problem
solving; the way the problem is to solved must be known and stated in small
unambiguous instructions. These instructions are then converted to a high level
language program and then into machine code that the computer can understand.
These machines are totally predictable; if anything goes wrong is due to a software or
hardware fault.
Neural networks and conventional algorithmic computers are not in competition but
complement each other. There are tasks are more suited to an algorithmic approach
like arithmetic operations and tasks that are more suited to neural networks. Even
more, a large number of tasks, require systems that use a combination of the two
approaches (normally a conventional computer is used to supervise the neural
network) in order to perform at maximum efficiency.
Neural networks do not perform miracles. But if used sensibly they can
produce some amazing results.


13


HUMAN AND ARTIFICIAL
NEURONS


14


2.1 The Human Brain, Neural Networks and Computers

A function approximator like an ANN can be viewed as a black box and when
it comes to FANN, this is more or less all you will need to know. However, basic
knowledge of how the human brain operates is needed to understand how ANNs
work.
The human brain is a highly complicated system which is capable to solve
very complex problems. The brain consists of many different elements, but one of its
most important building blocks is the neuron, of which it contains approximately 1011.
These neurons are connected by around 1015 connections, creating a huge neural
network. Neurons send impulses to each other through the connections and these
impulses make the brain work. The neural network also receives impulses from the
fiber nerve senses and sends out impulses to muscles to achieve motion or speech.
The individual neuron can be seen as an input-output machine which waits for
impulses from the surrounding neurons and, when it has received enough impulses, it
sends out an impulse to other neurons.
Neural networks, as used in artificial intelligence, have traditionally been
viewed as simplified models of neural processing in the brain, even though the
relation between this model and brain biological architecture is debated.


15


A subject of current research in theoretical neuroscience is the question
surrounding the degree of complexity and the properties that individual neural
elements should have to reproduce something resembling animal intelligence.
Historically, computers evolved from the von Neumann architecture, which is
based on sequential processing and execution of explicit instructions. On the other
hand, the origins of neural networks are based on efforts to model information
processing in biological systems, which may rely largely on parallel processing as
well as implicit instructions based on recognition of patterns of 'sensory' input from
external sources. In other words, at its very heart a neural network is a complex
statistical processor (as opposed to being tasked to sequentially process and execute).
Neural coding is concerned with how sensory and other information is
represented in the brain by neurons. The main goal of studying neural coding is to
characterize the relationship between the stimulus and the individual or ensemble
neuronal responses and the relationship among electrical activity of the neurons in the
ensemble. It is thought that neurons can encode both digital and analog information.


16


2.2 The Artificial Neural Networks and Artificial
Intelligence
Artificial neurons are similar to their biological counter parts. They have input
connections which are summed together to determine the strength of their output,
which is the result of the sum being fed into an activation function. Though many
activation functions exist, the most common is the sigmoid activation function, which
outputs a number between 0 (for low input values) and 1 (for high input values). The
resultant of this function is then passed as the input to other neurons through more
connections, each of which are weighted. These weights determine the behavior of the
network.
A neural network (NN), in the case of artificial neurons called artificial neural
network (ANN) or simulated neural network (SNN), is an interconnected group of
natural or artificial neurons that uses a mathematical or computational model for
information processing based on a connectionistic approach to computation. In most
cases an ANN is an adaptive system that changes its structure based on external or
internal information that flows through the network.
In more practical terms neural networks are non-linear statistical data
modeling or decision making tools. They can be used to model complex relationships
between inputs and outputs or to find patterns in data.
However, the paradigm of neural networks - i.e., implicit, not explicit ,
learning is stressed - seems more to correspond to some kind of natural intelligence
than to the traditional Artificial Intelligence, which would stress, instead, rule-based
learning.
(1) Background
An artificial neural network involves a network of simple processing elements
(artificial neurons) which can exhibit complex global behavior, determined by the
connections between the processing elements and element parameters. Artificial
neurons were first proposed in 1943 by Warren McCulloch, a neurophysiologist, and


17


Walter Pitts, an MIT logician. One classical type of artificial neural network is the
recurrent Hopfield net.
In a neural network model simple nodes, which can be called variously "neurons",
"neurodes", "Processing Elements" (PE) or "units", are connected together to form a
network of nodes — hence the term "neural network". While a neural network does
not have to be adaptive per se, its practical use comes with algorithms designed to
alter the strength (weights) of the connections in the network to produce a desired
signal flow.
In modern software implementations of artificial neural networks the approach
inspired by biology has more or less been abandoned for a more practical approach
based on statistics and signal processing. In some of these systems, neural networks,
or parts of neural networks (such as artificial neurons), are used as components in
larger systems that combine both adaptive and non-adaptive elements.
The concept of a neural network appears to have first been proposed by Alan
Turing in his 1948 paper "Intelligent Machinery".
(2) Applications of natural and of artificial neural networks
The utility of artificial neural network models lies in the fact that they can be
used to infer a function from observations and also to use it. This is particularly useful
in applications where the complexity of the data or task makes the design of such a
function by hand impractical.
The tasks to which artificial neural networks are applied tend to fall within the
following broad categories:
Function approximation, or regression analysis, including time series
prediction and modeling.
Classification, including pattern and sequence recognition, novelty
detection and sequential decision making.
Data processing, including filtering, clustering, blind signal separation
and compression.


18


Application areas of ANNs include system identification and control (vehicle
control, process control), game-playing and decision making (backgammon, chess,
racing), pattern recognition (radar systems, face identification, object recognition,
etc.), sequence recognition (gesture, speech, handwritten text recognition), medical
diagnosis, financial applications, data mining (or knowledge discovery in databases,
"KDD"), visualization and e-mail spam filtering.
Moreover, some brain diseases, e.g. Alzheimer, are apparently, and essentially,
diseases of the brain's natural NN by damaging necessary prerequisites for the
functioning of the mutual interconnections between neurons.


19


2.3 Neural Networks and Neuro Science
Theoretical and computational neuroscience is the field concerned with the
theoretical analysis and computational modeling of biological neural systems. Since
neural systems are intimately related to cognitive processes and behavior, the field is
closely related to cognitive and behavioral modeling.
The aim of the field is to create models of biological neural systems in order to
understand how biological systems work. To gain this understanding, neuroscientists
strive to make a link between observed biological processes (data), biologically
plausible mechanisms for neural processing and learning (biological neural network
models) and theory (statistical learning theory and information theory).

Classification of Neural Networks

Artificial neural networks can be classified on the basis of:

1. Pattern of connection between neurons, (architecture of the network)
2. Activation function applied to the neurons
3. Methods of determining weights on the connection


20


ARCHITECTURE


21


3.1 Architecture of Neural Network
The neurons are assumed to be arranged in layers, and the neurons in the same
layer behave in the same manner. All the neurons in a layer usually have the same
activation function. Within each layer, the neurons are either fully interconnected or
not connected at all. The arrangement of neurons into layers and the connection
pattern within and between layers is known as network architecture.

Input layer:
The neurons in this layer receive the external input signals and perform
no computation, but simply transfer the input signals to the neurons on
another layer.

Output layer:
The neurons in this layer receive signals from neurons either in the
input layer or in the hidden layer.

Hidden layer:
The layer of neurons that are connected in between the input layer and
the output layer is known as hidden layer.

Neural nets are often classified as single layer networks or multilayer
networks. The number of layers in a net can be classified as the number of layers of
weighted interconnection links between various layers.


22


Fully Connected N
Networks
An artificial neural network architecture in which every node is connected to
every node, and these connections may be either excitatory (positive weights)
inhibitory (negative weights), or irrelevant (almost zero weights).

This is the most general neural net architecture imaginable, and every other
architecture can be seen to be its special case, obtained by setting some weights to
zeroes. In a fully connected asymmet network, the connection from one node to
asymmetric
,
another may carry a different weight than the connection from the second node to the
first.

This architecture is seldom used deposit its generality and conceptual
simplicity, due to the large numbers of parameters. In a network there are n2 weights.
It is difficult to devise fast learning schemes that can produces fully connected
networks that generalize well. It is practically never the case that every node has
the
direct influence on every other node.


23


3.2 Feed Forward Network
Feed-forward ANNs (figure 1) allow signals to travel one way only; from
input to output. There is no feedback (loops) i.e. the output of any layer does not
affect that same layer. Feed-forward ANNs tend to be straight forward networks that
associate inputs with outputs. They are extensively used in pattern recognition. This
type of organisation is also referred to as bottom-up or top-down.

Figure 1.
This is a subclass of acyclic networks in which a connection is allowed from a
node in layer i only to node in layer i+1.

These networks, generally with no more than four layers, are among the most
common neural nets in use. So much so that some users identify the phrase “neural
networks” to mean only feedforward networks. Conceptually, nodes in successively
higher layers abstract successively higher level features from preceding layers.

3.3 Feedback Network
Feedback networks (figure 1) can have signals travelling in both directions by
introducing loops in the network. Feedback networks are very powerful and can get
extremely complicated. Feedback networks are dynamic; their 'state' is changing
continuously until they reach an equilibrium point. They remain at the equilibrium
point until the input changes and a new equilibrium needs to be found. Feedback
architectures are also referred to as interactive or recurrent, although the latter term is
often used to denote feedback connections in single-layer organisations.


24


3.4 Network Layers
The commonest type of artificial neural network consists of three groups, or
layers, of units: a layer of "input" units is connected to a layer of "hidden" units,
which is connected to a layer of "output" units. (See below Figure)
The activity of the input units represents the raw information that is fed into
the network.

The activity of each hidden unit is determined by the activities of the input
units and the weights on the connections between the input and the hidden
units.

The behavior of the output units depends on the activity of the hidden units
and the weights between the hidden and output units.
This simple type of network is interesting because the hidden units are free to
construct their own representations of the input. The weights between the input and
hidden units determine when each hidden unit is active, and so by modifying these
weights, a hidden unit can choose what it represents.

Figure : An example of a simple feedforward network
We also distinguish single-layer and multi-layer architectures. The singlelayer organization, in which all units are connected to one another, constitutes the
most general case and is of more potential computational power than hierarchically


25


structured multi-layer organizations. In multi-layer networks, units are often
numbered by layer, instead of following a global numbering.
These are networks in which nodes are partitioned into subsets called layers,
with no connections that lead from layer j to layer k if j > k.

We adopt the convention that a single input arrives at and is distributed to
other nodes by each node of the “input layer” or “layer 0”; no other computation
occurs at nodes in layer 0, and there are no intra-layer connections among nodes in
nodes layer. Connections with arbitrary weights, may exits from any node in layer i to
any node in layer j for j >= i; intra-layer connections may exits.

3.5 Acyclic Network
There is a subclass of layered networks in which there are no intra-layered
connections. In other words, a connection may exits between any node in layer i and
any nude in layer j for i < j, but a connection is not allowed for i = j.

The computation processes in acyclic networks are much simpler than those in
networks with exhaustive, cyclic or intra-layer connections. Networks that are not
acyclic are referred to as recurrent networks.

3.6 Modular Neural Network
Many problems are best solved using neural networks whose architecture
consist of several modules, with spare interconnections between modules. Modularity
allows the neural network developer to solve smaller tasks separately using small
modules and then combine these modules in a logical manner.


26


3.7 Perceptron
The perceptron is a type of artificial neural network invented in 1958 at the
Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest
kind of feedforward neural network: a linear classifier.

The perceptron is essentially a linear classifier for classifying data specified by
parameters and an output function f = w'x + b. Its parameters are adapted with an adhoc rule similar to stochastic steepest gradient descent. Because the inner product is a
linear operator in the input space, the perceptron can only perfectly classify a set of
data for which different classes are linearly separable in the input space, while it often
fails completely for non-separable data. While the development of the algorithm
initially generated some enthusiasm, partly because of its apparent relation to
biological mechanisms, the later discovery of this inadequacy caused such models to
be abandoned until the introduction of non-linear models into the field.
The cognitron (1975) designed by Kunihiko Fukushima was an early multilayered
neural network with a training algorithm. The actual structure of the network and the
methods used to set the interconnection weights change from one neural strategy to
another, each with its advantages and disadvantages. Networks can propagate
information in one direction only, or they can bounce back and forth until selfactivation at a node occurs and the network settles on a final state. The ability for bidirectional flow of inputs between neurons/nodes was produced with the Hopfield's
network (1982), and specialization of these node layers for specific purposes was
introduced through the first hybrid network.
The parallel distributed processing of the mid-1980s became popular under the
name connectionism.
The rediscovery of the backpropagation algorithm was probably the main
reason behind the repopularisation of neural networks after the publication of
"Learning Internal Representations by Error Propagation" in 1986 (Though
backpropagation itself dates from 1969).


27


The original network utilized multiple layers of weight-sum units of the type f
= g(w'x + b), where g was a sigmoid function or logistic function such as used in
logistic regression. Training was done by a form of stochastic Gradient descent. The
employment of the chain rule of differentiation in deriving the appropriate parameter
updates results in an algorithm that seems to 'backpropagate errors', hence the
nomenclature. However it is essentially a form of gradient descent. Determining the
optimal parameters in a model of this type is not trivial, and steepest gradient descent
methods cannot be relied upon to give the solution without a good starting point. In
recent times, networks with the same architecture as the backpropagation network are
referred to as Multi-Layer Perceptrons. This name does not impose any limitations on
the type of algorithm used for learning.
The backpropagation network generated much enthusiasm at the time and
there was much controversy about whether such learning could be implemented in the
brain or not, partly because a mechanism for reverse signalling was not obvious at the
time, but most importantly because there was no plausible source for the 'teaching' or
'target' signal.


28


APPLICATION


29


APPLICATIONS OF ARTIFICIAL NEURAL NETWORK

There have been many impressive demonstrations of artificial neural
networks. A few areas where neural networks are currently applied are mentioned
below.

4.1 Neural Networks in Practice
Given this description of neural networks and how they work, what real world
applications are they suited for? Neural networks have broad applicability to real
world business problems. In fact, they have already been successfully applied in many
industries.
Since neural networks are best at identifying patterns or trends in data, they are well
suited for prediction or forecasting needs including:
sales forecasting

industrial process control

customer research

data validation

risk management

target marketing
But to give you some more specific examples :
ANN are also used in the following specific paradigms: recognition of
speakers in communications; diagnosis of hepatitis; recovery of telecommunications
from faulty software; interpretation of multimeaning Chinese words; undersea mine


30


detection; texture analysis; three-dimensional object recognition; hand-written word
recognition; and facial recognition.

4.2. Neural Networks in Medicine
Artificial Neural Networks (ANN) are currently a 'hot' research area in
medicine and it is believed that they will receive extensive application to biomedical
systems in the next few years. At the moment, the research is mostly on modelling
parts of the human body and recognizing diseases from various scans (e.g.
cardiograms, CAT scans, ultrasonic scans, etc.).
Neural networks are ideal in recognizing diseases using scans since there is no need to
provide a specific algorithm on how to identify the disease. Neural networks learn by
example so the details of how to recognize the disease are not needed. What is needed
is a set of examples that are representative of all the variations of the disease. The
quantity of examples is not as important as the 'quantity'. The examples need to be
selected very carefully if the system is to perform reliably and efficiently.

4.2.1 Modelling and Diagnosing the Cardiovascular System
Neural Networks are used experimentally to model the human cardiovascular
system. Diagnosis can be achieved by building a model of the cardiovascular system
of an individual and comparing it with the real time physiological measurements
taken from the patient. If this routine is carried out regularly, potential harmful
medical conditions can be detected at an early stage and thus make the process of
combating the disease much easier.
A model of an individual's cardiovascular system must mimic the relationship
among physiological variables (i.e., heart rate, systolic and diastolic blood pressures,
and breathing rate) at different physical activity levels. If a model is adapted to an
individual, then it becomes a model of the physical condition of that individual. The
simulator will have to be able to adapt to the features of any individual without the
supervision of an expert. This calls for a neural network.


31


Another reason that justifies the use of ANN technology, is the ability of
ANNs to provide sensor fusion which is the combining of values from several
different sensors. Sensor fusion enables the ANNs to learn complex relationships
among the individual sensor values, which would otherwise be lost if the values were
individually analysed. In medical modelling and diagnosis, this implies that even
though each sensor in a set may be sensitive only to a specific physiological variable,
ANNs are capable of detecting complex medical conditions by fusing the data from
the individual biomedical sensors.

4.2.2 Electronic Noses
ANNs are used experimentally to implement electronic noses. Electronic
noses have several potential applications in telemedicine. Telemedicine is the practice
of medicine over long distances via a communication link. The electronic nose would
identify odours in the remote surgical environment.
These identified odours would then be electronically transmitted to another
site where an door generation system would recreate them. Because the sense of smell
can be an important sense to the surgeon, telesmell would enhance telepresent
surgery.

4.2.3 Instant Physician
An application developed in the mid-1980s called the "instant physician"
trained an auto associative memory neural network to store a large number of medical
records, each of which includes information on symptoms, diagnosis, and treatment
for a particular case. After training, the net can be presented with input consisting of a
set of symptoms; it will then find the full stored pattern that represents the "best"
diagnosis and treatment.


32


4.3 Neural Networks in Business
Business is a diverted field with several general areas of specialization such as
accounting or financial analysis. Almost any neural network application would fit into
one business area or financial analysis.
There is some potential for using neural networks for business purposes,
including resource allocation and scheduling. There is also a strong potential for using
neural networks for database mining, that is, searching for patterns implicit within the
explicitly stored information in databases. Most of the funded work in this area is
classified as proprietary. Thus, it is not possible to report on the full extent of the
work going on. Most work is applying neural networks, such as the Hopfield-Tank
network for optimization and scheduling.

4.4 Marketing
There is a marketing application which has been integrated with a neural
network system. The Airline Marketing Tactician (a trademark abbreviated as AMT)
is a computer system made of various intelligent technologies including expert
systems. A feedforward neural network is integrated with the AMT and was trained
using back-propagation to assist the marketing control of airline seat allocations. The
adaptive neural approach was amenable to rule expression.
Additionally, the application's environment changed rapidly and constantly,
which required a continuously adaptive solution. The system is used to monitor and
recommend booking advice for each departure. Such information has a direct impact
on the profitability of an airline and can provide a technological advantage for users
of the system. While it is significant that neural networks have been applied to this
problem, it is also important to see that this intelligent technology can be integrated
with expert systems and other approaches to make a functional system. Neural
networks were used to discover the influence of undefined interactions by the various
variables. While these interactions were not defined, they were used by the neural
system to develop useful conclusions. It is also noteworthy to see that neural networks
can influence the bottom line.


33


4.5 Credit Evaluation
The HNC Company, founded by Robert Hecht-Nielsen, has developed several
neural network applications. One of them is the Credit Scoring system which
increases the profitability of the existing model up to 27%. The HNC neural systems
were also applied to mortgage screening. A neural network automated mortgage
insurance underwriting system was developed by the Nestor Company. This system
was trained with 5048 applications of which 2597 were certified. The data related to
property and borrower qualifications. In a conservative mode the system agreed on
the underwriters on 97% of the cases. In the liberal model the system agreed 84% of
the cases. This is system run on an Apollo DN3000 and used 250K memory while
processing a case file in approximately 1 sec.

4.6 Signal Processing
In digital communication systems, distorted signals cause inter-signal
interference. One of the first commercial applications of neural networks was to
suppress noise cancellation and it was implemented by widrow using ADALINE. The
ADALINE is trained to remove the noise from the telephone line signal.

4.7 Speech Recognition
In recent years, speech recognition has received enormous attention. It
involves three modules namely, the front end which samples the speech signals and
extracts the data the word processor which is used for finding the probability of words
of words in the vocabulary that match the features of spoken words and the sentence
processor which determines if the recognized word makes sense in the sentence.
Neural networks remove the noise in the speech.


34


4.8 Intelligent Control
A problem of concern in industrial motor control is the ability to predict
system failure. Motor failure depends on specific motor parameters, transient currents
and motor positioning. Neural networks are employed to learn the motor-current
variants as well as installation characteristics. 80% to 90% accurate prediction rate
has been reported. Many of the neural networks applications merge in Robotics. The
simplest tasks performed are arm movement, object recognition and object
manipulation. Neural networks have been used in many vehicular applications
including trains and automatic gear transmission in cars.
Neural networks had been used in steel rolling machines to control the strip
thickness within very tight tolerances.
Function Approximation

Many computational models can be described as functions mapping the input
vectors to numerical outputs. Neural networks are employed to construct the function
that generates approximately the same output fro a given input vector based on the
available training data.

4.9 Financial Forecasting
One of the most sought after applications of neural networks have been in
financial application. The idea that if one dealer can predict market values or assess
risk better than his competitors even by a very small amount, leads to an expectation
of huge financial benefits. Much of the work in this area is based on the so called
‘time series prediction’.

In contrast with the time series prediction, neural networks have also been
used in financial modeling which aims at testing theories about the dynamics of
market.


35


4.10 Condition Monitoring
The ability of neural nets to receive input from a large number of sensors and
to learn to assess the significance of these is a major advantage in condition
monitoring. The sensitivity to novelty is almost an emergent property of most neural
systems. This has obvious applications in the detection of departures from normality.
Time dependent behavior of nets can also be used in predicating the occurrence of
malfunction ahead of time. Applications under this range from health monitoring of
turbo machinery to gear boxes, jets and internal combustion engines.
Process Monitoring and Control

Neurocontrol principles can be applied to various areas of the process control
industry. Neural techniques come into their own when the link between the
measurement and operation parameter selection cannot be obtained analysis. These
techniques can be applied in processes such as obtained by analysis. These techniques
can be applied in processes such as chemical and biochemical, pharmaceutical, water
purification, food and beverage production, power distribution and oil and gas
industries.

4.11 Neuro Forecasting
A major application of neural nets is the prediction of prospective long term
foreign exchange rates. Neural networks provide greater flexibility in this regard
compared to the traditional methods.

4.12 Pattern Analysis [B35]
This application works in various areas such as online industrial plant
monitoring, smart sensor systems and industrial image analysis as may be required in
inspection and control tasks. Other classical problems of pattern analysis such as real
time audio bandwidth systems, images and data management can also be tackled with
the help of neural networks.


36


4.13 Classification
A neural network can discover the distinguishing features needed to perform a
classification task. Classification is the assignment of each object to a specific class,
which is an important aspect in image classification. Neural networks have been used
successfully in a large number of classification tasks which includes:

a. Recognition of printed or handwritten characters.
b. Classification of SONAR and RADAR signals.


37


INTRODUCTION TO
CHARACTER RECOGNITION


38


5.1 What is Character Recognition?
Character recognition is the mechanical or electronic translation of images of
handwritten or printed text into machine-editable text.

It is widely used to convert a books and documents into electronic files, to
computerize a record-keeping system in an office, or to publish the text on a website.
It is often useful to have a machine perform pattern recognition. In particular,
machines that can read symbols are very cost effective. A machine that reads banking
checks can process many more checks than a human being in the same time. This kind
of application saves time and money, and eliminates the requirement that a human
perform such a repetitive task.
Many scientist and major company implement on character recognition from
1945 to till the date with use of neural network. And most of them are very successful
with 99% accuracy.


39


5.2 History for Character Recognition
1915 : U.S. Patent on handwriting recognition user interface with a stylus.

1957 : Stylator tablet: Tom Dimond demonstrateselectronic tablet with pen for
Computer input and handwriting recognition.

1961 : RAND Tablet inveted : better known than earlier Stylator system.

1962 : Computer recognition of connected/script handwriting.

1969 : GRAIL system: handwriting recognition with electronic ink display, gesture
Commands.

1973 : Application CAD/CAM computer system using the Ledeen recognizer for
Handwriting recognition.

1980 : Retail handwriting-recognition systems: Pencept and CIC both offer PC
Computers for the consumer market using a tablet and handwriting recognition
Instead of a keyboard and mouse.
Cadre System markets Inforite point-of-sale terminal using handwriting
Recognition and a small electronic tablet and pen.

1989 : Portable handwriting recognition computer : GRIDPAD from Grid System.

1997 : First handwritten address interpretation system(HWAI) deployed by United
State Postal Service.

2007 : First automatic writer recognition system : CEDAR-FOX.


40


5.3 Steps for The Character Recognition
There are different steps which should be follow for recognize the character
which are given below.

1. The input neurons receive the pixel data from the image.

2. Each input neuron calculates its output.

3. This output is fed directly to the neurons in the hidden layer.

4. The neurons in the hidden layer calculate their output.

5. This output is fed to the neurons in the output layer.

6. The neurons in the output layer calculate their output.

7. The output of the output layer is our result. We interpret it the way we want.

Practically how its work and which functions we can use in the character
Recognition using MATLAB, all the details are given in next chapter. MATLAB is a
mathematical software which cointain Artificial Neural Network toolbox and
mathematical functions as well as neural network functions like back propagation
network, logsig, tansig, linear methods, and also many image processing tools. We
will also discuss about MATLAB in next chapter.


41


HOW TO RECOGNIZE
CHARACTER


42


HOW TO RECOGNIZE CHARACTER

A network is to be designed and trained to recognize the 26 letters of the
alphabet. An imaging system that digitizes each letter centered in the system's field of
vision is available. The result is that each letter is represented as a 5 by 7 grid of
Boolean values.
For example, here is the letter A.

However, the imaging system is not perfect, and the letters can suffer from noise.


43


Perfect classification of ideal input vectors is required, and reasonably
accurate classification of noisy vectors.
The twenty-six 35-element input vectors are defined in the function prprob as
a matrix of input vectors called alphabet. The target vectors are also defined in this
file with a variable called targets. Each target vector is a 26-element vector with a 1 in
the position of the letter it represents, and 0's everywhere else. For example, the letter
A is to be represented by a 1 in the first element (as A is the first letter of the
alphabet), and 0's in elements two through twenty-six.

When Insert in network
The network receives the 35 Boolean values as a 35-element input vector. It is
then required to identify the letter by responding with a 26-element output vector. The
26 elements of the output vector each represent a letter. To operate correctly, the
network should respond with a 1 in the position of the letter being presented to the
network. All other values in the output vector should be 0.
In addition, the network should be able to handle noise. In practice, the network does
not receive a perfect Boolean vector as input. Specifically, the network should make
as few mistakes as possible when classifying vectors with noise of mean 0 and
standard deviation of 0.2 or less.

The neural network needs 35 inputs and 26 neurons in its output layer to
identify the letters. The network is a two-layer log-sigmoid/log-sigmoid network. The


44


log-sigmoid transfer function was picked because its output range (0 to 1) is perfect
for learning to output Boolean values.
The hidden (first) layer has 10 neurons. This number was picked by
guesswork and experience. If the network has trouble learning, then neurons can be
added to this layer.
The network is trained to output a 1 in the correct position of the output vector
and to fill the rest of the output vector with 0's. However, noisy input vectors can
result in the network's not creating perfect 1's and 0's. After the network is trained the
output is passed through the competitive transfer function compet. This makes sure
that the output corresponding to the letter most like the noisy input vector takes on a
value of 1, and all others have a value of 0. The result of this postprocessing is the
output that is actually used.
After applying training with noise and without noise we can get the following
result as a character recognition.


45


MATLAB


46


7.1 WHAT IS MATLAB ?
MATLAB (for matrix laboratory) is a numerical computing environment and
fourth-generation programming language. Developed by MathWorks, MATLAB
allows matrix manipulations, plotting of functions and data, implementation of
algorithms, creation of user interfaces, and interfacing with programs written in other
languages, including C, C++, and Fortran.It provides many readymade functionality
of solving difficult area like processing and recognition etc in a simple way. It contain
different formula and mathematical functions and many valid tool box for easily
maintain and overcome from difficult processing with a simplest way. And also its
much faster and successful result in many area.
There are different types of versions available in market with new upgraded
tools and functionality. The full name of Matlab - the language of technical
computing. We can also get a 3d module like figure and mapping of two or more than
two things in a different way with different tools.
The nnet Toolbox extends the basic capabilities of Matlab by providing a
number of functions for character recognize from images. It also recognize character
from image either it is good or bad condition. Matlab also provide Toolbox for image
processing. This toolbox provide number of input output functions and control for
image processing. E.e image blur – deblur , image scaling, image binding, resizing
pixels and arrange pixel for making a good quality picture.
Although MATLAB is intended primarily for numerical computing, an
optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic
computing capabilities. An additional package, Simulink, adds graphical multidomain simulation and Model-Based Design for dynamic and embedded systems.

In 2004, MATLAB had around one million users across industry and
academia. MATLAB users come from various backgrounds of engineering, science,
and economics. MATLAB is widely used in academic and research institutions as
well as industrial enterprises.


47


7.2 History AND Present
MATLAB was created in the late 1970s by Cleve Moler, the chairman of the
computer science department at the University of New Mexico.[3] He designed it to
give his students access to LINPACK and EISPACK without having to learn Fortran.
It soon spread to other universities and found a strong audience within the applied
mathematics community. Jack Little, an engineer, was exposed to it during a visit
Moler made to Stanford University in 1983. Recognizing its commercial potential, he
joined with Moler and Steve Bangert. They rewrote MATLAB in C and founded
MathWorks in 1984 to continue its development. These rewritten libraries were
known as JACKPAC.[4] In 2000, MATLAB was rewritten to use a newer set of
libraries for matrix manipulation.

There are diferent types of version available each new version contain much
powerful and successful toolbox and methods which are very useful in natural science
and new implementation.

MATLAB release its first version in 1984.when they launch its 3.5 version it
higher then 1.0 so it required 386 processor and math coprocessor.and it ran on MSDOS. After that they realese last version R2007a and R2007b for wondows 2000 and
PowerPC Mac.

MATLAB release 7.8 for 32-bit and 64-bit window–7, 64-bit Mac and Solaris
SPARC. And finally last version for intel 32-bit Mac in year 2010.


48


7.3 NEURAL NETWORK TOOLBOX
Neural networks are composed of simple elements operating in parallel. These
elements are inspired by biological nervous systems. As in nature, the connections
between elements largely determine the network function. You can train a neural
network to perform a particular function by adjusting the values of the connections
(weights) between elements. Typically, neural networks are adjusted, or trained, so
that a particular input leads to a specific target output. The next figure illustrates such
a situation. There, the network is adjusted, based on a comparison of the output and
the target, until the network output matches the target. Typically, many such
input/target pairs are needed to train a network.

Neural networks have been trained to perform complex functions in various
fields, including pattern recognition, identification, classification, speech, vision, and
control systems. Neural networks can also be trained to solve problems that are
difficult for conventional computers or human beings. The toolbox emphasizes the
use of neural network paradigms that build up to—or are themselves used in—
engineering, financial, and other practical applications. The next sections explain how


49


to use three graphical tools for training neural networks to solve problems in function
fitting, pattern recognition, and clustering.
We can solve following application by using Neural Network Toolbox
(Nnet).
Aerospace

High-performance aircraft autopilot, flight path
simulation, aircraft control systems, autopilot
enhancements, aircraft component simulation,
and aircraft component fault detection

Automotive

Automobile automatic guidance system,
and warranty activity analysis

Banking

Check and other document reading and
credit application evalution

Defence

Weapon steering, target tracking, object
discrimination, facial recognition, new
kinds of sensors, sonar, radar and image
signal processing including data compression,
feature extraction and noise suppression,
and signal/image identification

Electronics

Code sequence prediction, integrated
circuit chip layout, process control,
chip failure analysys, machine vision,
voice synthesis, and nonlinear modeling

Entertainment

Animation, special effects, and market forecasting

Financial

Real estate appraisal, loan advising, mortgage
screening, corporate bond rating, credit-line
use analysis, credit card activity tracking,
portfolio trading program, corporate financial
analysis, and currency price prediction


50


Industrial

Prediction of industrial processes, such
as the output gases of furnances, replacing
complex and costly equipment used for
this purpose in the past

Insurance

Policy application evaluation and product optimization

Manufacturing

Manufacturing process control, product
design and analysis, process and machine
diagnosis, real-time particle identification,
visual quality inspection systems, beer testing,
welding quality analysis, paper quality
prediction, computer-chip quality analysis,
analysis of grinding operations, chemical
product design analysis, project bidding,
planning and management, and dynamic
modeling of chemical process system

Medical

Breast cancer cell analysis, EEG and ECG
analysis, prosthesis design, optimization
of transplant times, hospital expense reduction,
hospital quality improvement, and
emergency-room test advisement

Oil and gas

Exploration

Robotics

Trajectory control, forklift robot, manipulator
controllers, and vision systems

Speech

Speech recognition, speech compression,
vowel classification, and text-to-speech synthesis

Securities

Market analysis, automatic bond rating,
and stock trading advisory systems

Telecommunications Image and data compression, automated


51


information services, real-time translation of
spoken language, and customer payment
processing systems
Transportation

Truck brake diagnosis systems, vehicle
scheduling, and routing systems


52


LITERATURE SURVEY


53


8.1Kevin Larson paper and Application
Kevin Larson from microsoft corporation,he research on Advance Reading
Technology in july 2004. According to him, Evidence from the last 20 years of work in
cognitive psychology indicate that we use the letters within a word to recognize a word. Many
typographers and other text enthusiasts I’ve met insist that words are recognized by the outline made
around the word shape. Some have used the term bouma as a synonym for word shape, though He was
unfamiliar with the term. The term bouma appears in Paul Saenger’s 1997 book Space Between Words:
The Origins of Silent Reading. There he learned to his chagrin that we recognize words from their
word shape and that “Modern psychologists call this image the ‘Bouma shape.’”

In his paper he did dozen of experiments and all come from peer reviewed journal where the
experiments are well specified so that anyone could reproduce the experiment and expect to achieve the
same result. The paper was originally presented as a talk at the ATypI conference in Vancouver in
September ,2003.

The main goal of his paper was to review the history of why psychologists moved from
a word shape model of word recognition to a letter recognition model, and to help others to come to the
same conclusion. Paper covered many topics in relatively few pages. Along the way he presents
experiments and models that he couldn’t hope to cover completely without boring the reader.

STRATEGY
In his paper he described three major categories of word recognition models that are as below.

Word shape model

Serial model

Parallel model for letter recognition.

Model - 1: Word Shape
The word recognition model that says words are recognized as complete units is the
oldest model in the psychological literature, and is likely much older than the
psychological literature. The general idea is that we see words as a complete patterns


54


rather than the sum of letter parts. Some claim that the information used to recognize
a word is the pattern of ascending, descending, and neutral characters. Another
formulation is to use the envelope created by the outline of the word. The word
patterns are recognizable to us as an image because we have seen each of the patterns
many times before. James Cattell (1886) was the first psychologist to propose this as a
model of word recognition. Cattell is recognized as an influential founder of the field
of psycholinguistics, which includes the scientific study of reading.

Figure 1: Word shape recognition using the pattern of ascending, descending, and
neutral characters.

Characters

Figure 2: Word shape recognition using the envelope around the word
Cattell’s study was sloppy by modern standards, but the same effect was
replicated in 1969 by Reicher. He presented strings of letters – half the time real
words, half the time not – for brief periods.
The subjects were asked if one of two letters were contained in the string,
for example D or K. Reicher found that subjects were more accurate at
recognizing D when it was in the context of WORD than when in the context of
ORWD. This supports the word shape model because the word allows the subject to
quickly recognize the familiar shape. Once the shape has been recognized, then the


55


subject can deduce the presence of the correct letter long after the stimulus
presentation.
The second key piece of experimental data to support the word shape model is
that lowercase text is read faster than uppercase text. Woodworth (1938) was the first
to report this finding in his influential textbook Experimental Psychology. This
finding has been confirmed more recently by Smith (1969) and Fisher (1975).
Participants were asked to read comparable passages of text, half completely
in uppercase text and half presented in standard lowercase text. In each study,
participants read reliably faster with the lowercase text by a 5-10% speed difference.
This supports the word shape model because lowercase text enables unique patterns of
ascending, descending, and neutral characters. When text is presented in all
uppercase, all letters have the same text size and thus are more difficult and slower to
read.
Model - 2: Serial Letter Recognition
The shortest lived model of word recognition is that words are read letter-byletter serially from left to right.
Gough (1972) proposed this model because it was easy to understand, and far
more testable than the word shape model of reading. In essence, recognizing a word in
the mental lexicon was analogous to looking up a word in a dictionary. You start off
by finding the first letter, than the second, and so on until you recognize the word.
This model is consistent with Sperling’s (1963) finding that letters can be
recognized at a rate of 10-20ms per letter. Sperling showed participants strings of
random letters for brief periods of time, asking if a particular letter was contained in
the string. He found that if participants were given 10ms per letter, they could
successfully complete the task.
For example, if the target letter was in the fourth position and the string was
presented for 30ms, the participant couldn’t complete the task successfully, but if
string was presented for 40ms, they could complete the task successfully. Gough


56


noted that a rate of 10ms per letter would be consistent with a typical reading rate of
300 wpm.
The serial letter recognition model is also able to successfully predict that
shorter words are recognized faster than longer words. It is a very robust finding that
word recognition takes more time with longer words. It takes more time to recognize
a 5-letter word than a 4-letter word, and 6-letter words take more time to recognize
than 5-letter words. The serial letter recognition model predicts that this should
happen, while a word shape model does not make this prediction. In fact, the word
shape model should expect longer words with more unique patterns to be easier to
recognize than shorter words.
The serial letter recognition model fails because it cannot explain the Word
Superiority Effect.
Model - 3: Parallel Letter Recognition
The model that most psychologists currently accept as most accurate is the
parallel letter recognition model. This model says that the letters within a word are
recognized simultaneously, and the letter information is used to recognize the words.


57


Above Figure shows a generic activation based parallel letter recognition
model. In this example, the reader is seeing the word work. Each of the stimulus
letters are processed simultaneously.
The first step of processing is recognizing the features of the individual letters,
such as horizontal lines, diagonal lines, and curves. The details of this level are not
critical for our purposes. These features are then sent to the letter detector level, where
each of the letters in the stimulus word are recognized simultaneously. The letter level
then sends activation to the word detector level.
The W in the first letter detector position sends activation to all the words that
have a W in the first position (WORD and WORK). The O in the second letter detector
position sends activation to all the words that have an O in the second position
(FORK, WORD, and WORK). While FORK and WORD have activation from three of
the four letters, WORK has the most activation because it has all four letters activated,
and is thus the recognized word.


58


8.2 Ketil Hunn Paper and Application
Ketil Hunn created Character Recognition application using back propagation
in neural network. He used Parallel distributed Processing in University of Pittsburgh,
Solving Problem

Reducing the resolution of each character to 15x9 pixels was necessary to
decrease the use of memory and learning time.

By letting the user draw characters directly in the program we eliminate the
problem with noise described earlier. Noise, if desirable, can be added by
drawing arbitrary patterns around the drawn character. Letting the user draw
characters directly in the program also simplifies demonstrations since a
demonstration can be created on the fly without having to create input files
etc.

Each character will be drawn in a separate box eliminating the problem of
separating characters.

Above figure shows a snapshot of his application. In this we just have to enter
character. Here we can see that four character ‘H’,’E’,’L’ and ‘O’. He Train network
and learn the network via backpropagation algorithm. And in this how many iteration
fired at hidden layer and pattern graph are given below.


59


Learning Character
New characters can be added to the database by drawing the character in the
box , selecting which character it will map onto and then pressing Add character.
When done entering all new characters, pressing Learn characters will make the
program learn the characters in the database using backprop. The program does this
by first initializing all weights and biases with random values between -1.0 and 1.0.
Then for each pattern it chooses randomly, it then computes the hidden and response
activities as described in the previous chapter before it backpropagates the error in the
network. The error in each response unit is calculated using douti=(Ti-ri)*ri(1-ri)*dt
and

then

the

error

in

the

hidden

units

are

calculated

using

dhidi=Sum((kdoutk*vki*hi(1-hi)*dt). Finally the program updates the weights and
biases; dci=douti*dt, dbi=dhidi*dt, dvij=douti*hj*dt and dwij=dhidi*sj*dt. The
procedure of choosing a random pattern, performing feedforward and then backprop
continues until the total error of the network is less than the error limit or until the
user interrupts the learning. The total error of the network is calculated as the sum
square error between the target and the response value, e=(T-r)². The error shown
during the learning procedure is the greatest error of all the patterns.


60


Result :
There were several problems to take into consideration when performing
backprop on a neural network, such as the number of hidden units to use, setting the
learning rate etc.
The results were surprisingly good, almost 90% accuracy! Normally it will take a real
character recognition program years to get to this level of accuracy.

Suggetion
He also suggest the following things in his paper for more accuracy and more
clear result.

Character scaling

Since a neural network only receives data in
form of input stimuli it cannot recognize different
sizes of a character without having to learn all
possible sizes. By letting the program scale characters this problem would be
reduced to a minimum.
Character centering
Another problem appears when the user draws
characters with a different alignment than the one
learned, i.e. drawing a left aligned I when all I’s
that have been learned are centered. This can easily be solved by letting the
program center all characters before they are stored in the database and before
interpreting them.


61


8.3 Current Market Scenario
There are many huge Software company that have implement on such a
software and also available on the web(on Line). Among them some application are
costly and some are much cheaper. We can have brief idea from below chart.


62



63


8.4 Current Research
While initially research had been concerned mostly with the electrical
characteristics of neurons, a particularly important part of the investigation in recent
years has been the exploration of the role of neuromodulators such as dopamine,
acetylcholine, and serotonin on behaviour and learning.
Biophysical models, such as BCM theory, have been important in
understanding mechanisms for synaptic plasticity, and have had applications in both
computer science and neuroscience. Research is ongoing in understanding the
computational algorithms used in the brain, with some recent biological evidence for
radial basis networks and neural backpropagation as mechanisms for processing data.
Computational devices have been created in CMOS for both biophysical
simulation and neuromorphic computing. More recent efforts show promise for
creating nanodevices for very large scale principal components analyses and
convolution. If successful, these efforts could usher in a new era of neural computing
that is a step beyond digital computing, because it depends on learning rather than
programming and because it is fundamentally analog rather than digital even though
the first instantiations may in fact be with CMOS digital devices.


64


Proposed Work


65


9.1 Work Introduction
In research work me and my coleague try to understand how its application
work. In this work we create a character recognition application using
MATLAB.more detailed review and understanding about this application are as given
below.

9.2 Application
Here is first screen of the Application.in which we can see the main image
screen and other processing buttons with processed image view.


66


Open Image and Select character


67


Image Processing

In this stage we can see the original image in selected character part. From that
image we conver image in to gray scale and build the image in binary.
we can see these image in Process On Image.

After converting it in to binary and build it we find the blank space area from
top side then bottom,right and at last left and cropping image and convert image into
5 * 7 single vector Image.


68


PLOTTING IMAGE IN CHARVEC

After cropping binary image we mapping pixels and pust into 35 element array
in the network and create a character vector. We can see that charvec in the below
stage.

Here is the charvec with some noisy environment for more clear appearance
we have to train network much higher rate. From this charvec it recognize the
character and finally show it in to recognize frame.


69


Recognize Character


70


9.3 Accuracy and Results
Actually we have create this application for character recognition but
unforunately its work for digit only.it doesn’t able to recognize Alphabet due to some
network problem.

But suprisingly its accuracy is around 85- 90 % of recognize the character.
Due to some reason like more noisy image or dull image or may be the scale of
image. Here some snapshot of success and failure of the recognition of the image.


71


9.4 Recognize Digit – 5 Success Phase


72


9.4 Recognize Digit– 3 Failure Phase


73


9.4 Recognize Digit– 5 Failure Phase


74


ENHANCEMENTS


75


Enhancements
Recent advances and future applications of NNs
Integration of fuzzy logic into neural networks
•

Fuzzy logic is a type of logic that recognizes more than simple true and false
values, hence better simulating the real world. For example, the statement
today is sunny might be 100% true if there are no clouds, 80% true if there are
a few clouds, 50% true if it's hazy, and 0% true if rains all day. Hence, it takes
into account concepts like -usually, somewhat, and sometimes.

•

Fuzzy logic and neural networks have been integrated for uses as diverse as
automotive engineering, applicant screening for jobs, the control of a crane,
and the monitoring of glaucoma.

Pulsed neural networks
•

"Most practical applications of artificial neural networks are based on a
computational model involving the propagation of continuous variables from
one processing unit to the next. In recent years, data from neurobiological
experiments have made it increasingly clear that biological neural networks,
which communicate through pulses, use the timing of the pulses to
transmit information and perform computation. This realization has
stimulated significant research on pulsed neural networks, including
theoretical analyses and model development, neurobiological modeling, and
hardware implementation."

Hardware specialized for neural networks
Some networks have been hardcoded into chips or analog devices ? this
technology will become more useful as the networks we use become more
complex.
The primary benefit of directly encoding neural networks onto chips or
specialized analog devices is SPEED!


76


NN hardware currently runs in a few niche areas, such as those areas where
very high performance is required (e.g. high energy physics) and in embedded
applications of simple, hardwired networks (e.g. voice recognition).
Many NNs today use less than 100 neurons and only need occasional training.
In these situations, software simulation is usually found sufficient
When NN algorithms develop to the point where useful things can be done
with 1000's of neurons and 10000's of synapses, high performance NN
hardware will become essential for practical operation.
Improvement of existing technologies
•

All current NN technologies will most likely be vastly improved upon in the
future. Everything from handwriting and speech recognition to stock market
prediction will become more sophisticated as researchers develop better
training methods and network architectures.
NNs might, in the future, allow:
robots that can see, feel, and predict the world around them
improved stock prediction
common usage of self-driving cars
composition of music
handwritten documents to be automatically transformed into formatted
word processing documents
trends found in the human genome to aid in the understanding of the
data compiled by the Human Genome Project
self-diagnosis of medical problems using neural networks and much
more!


77


CONCLUSION


78


Conclusion
The computing world has a lot to gain from neural networks. Their ability to
learn by example makes them very flexible and powerful. Furthermore there is no
need to devise an algorithm in order to perform a specific task; i.e. there is no need to
understand the internal mechanisms of that task. They are also very well suited for
real time systems because of their fast responseand computational times which are
due to their parallel architecture.
Neural networks also contribute to other areas of research such as neurology and
psychology. They are regularly used to model parts of living organisms and to
investigate the internal mechanisms of the brain.
Perhaps the most exciting aspect of neural networks is the possibility that some day
'consious' networks might be produced. There is a number of scientists arguing that
conciousness is a 'mechanical' property and that 'consious' neural networks are a
realistic possibility.
Finally, I would like to state that even though neural networks have a huge potential
we will only get the best of them when they are integrated with computing, AI, fuzzy
logic and related subjects.


79


SOURCES AND
BIBILIOGRAPHY


80


REFERENCES
[1]. S.N.SIVANANADAM and M.PAULRAJ = Introduction to ARTIFICIAL
NEURAL NETWORKS

[2]. CHILUKURI K.MOHAN and SANJAY RANKA = ELEMENTS OF
Artificial

Neural Network

[3]. Ajith Abraham Oklahoma State University, Stillwater, OK, USA = Artificial Neural
Networks.
[4]. Kevin Larson from Microsoft Corporation, = Advance Reading Technology
– July 2004
[5]. Ketil Hunn Paper and Application using Parallel Distributed processing in
University of Pittsburgh.

BIBILOGRAPHY
www.google.com
www.wikipedia.org
www.wikipedia.org/wiki/Neural_network
www.library.thinkquest.org/C007395/tqweb/history.html

http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
http://www-cs-faculty.stanford.edu
http://www.emsl.pnl.gov:2080/docs/cie/neural/neural.homepage.html
http://en.wikipedia.org/wiki/Character_recognition
http://www.microsoft.com/typography/ctfonts/wordrecognition.aspx
http://home.eunet.no/~khunn/papers/2039.html


81


http://en.wikipedia.org/wiki/List_of_optical_character_recognition_software


82

Dissertation character recognition - Report

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (19)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Dissertation character recognition - Report

Ähnlich wie Dissertation character recognition - Report (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Dissertation character recognition - Report