SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Downloaden Sie, um offline zu lesen
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 1
Dept. Of ECE Mangalam College of Engineering
1. INTRODUCTION
There are a number of technologies thriving for human computer interaction or
human machine interaction such as speech recognition, vision-based gesture recognition and
tablet types of devices. These devices facilitate the intercommunication between humans and
machine more freely and without the use of traditional keyboards or mouse or remote
controls. Gesture or trajectory recognition based on inertial devices is an original, direct
human-computer interacting way, because the motion of limbs is driven by the force from
muscles and the inertial devices could detect the outcome of the force, say acceleration and
angular velocity, directly, so people could even use them without any distraction all the time.
A significant advantage of inertial sensors for general motion sensing is that they
can be operated without any external reference and limitation in working conditions.
However, motion trajectory recognition is relatively complicated because different users have
different speeds and styles to generate various motion trajectories. Thus, many researchers
have tried to narrow down the problem domain for increasing the accuracy of handwriting
recognition systems [1].
The term gesture recognition has been used to refer more narrowly to non-text-input
handwriting symbols. Gesture recognition enables humans to communicate with the machine
(HMI) and interact naturally without any mechanical devices. Using the concept of
gesture recognition, it is possible to point a finger at the computer screen so that the cursor
will move accordingly. This could potentially make conventional input devices such as
mouse, keyboards and even touch-screens redundant. Just as speech recognition can
transcribe speech to text, certain types of gesture recognition software can transcribe the
symbols represented through sign language into text. By using proper sensors (eg.
accelerometers) worn on the body of a patient and by reading the values from those sensors,
robots can assist in patient rehabilitation. The best example can be stroke rehabilitation.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 2
Dept. Of ECE Mangalam College of Engineering
2. SYSTEM ANALYSIS
2.1 EXISTING SYSTEM
In the present system, a trajectory recognition algorithm was used for digit and
gesture recognition applications.this paper developed a pen-type portable device and a
trajectory recognition algorithm. The pen-type portable device consists of a triaxial
accelerometer, a microcontroller, and an RF wireless transmission module. The acceleration
signals measured from the tri-axial accelerometer are transmitted to a computer via the
wireless module. Users can utilize this digital pen to write digits and make hand gestures at
normal speed. The measured acceleration signals of these motions can be recognized by the
trajectory recognition algorithm.
The recognition procedure is composed of acceleration acquisition, signal pre-
processing, feature generation, feature selection, and feature extraction. The acceleration
signals of hand motions are measured by the pen-type portable device. The signal pre-
processing procedure consists of calibration, a moving average filter, a high-pass filter, and
normalization. First, the accelerations are calibrated to remove drift errors and offsets from
the raw signals. These two filters are applied to remove high frequency noise and
gravitational acceleration from the raw data, respectively. The features of the pre-processed
acceleration signals of each axis include mean, correlation among axes, inter-quartile range
(IQR), mean absolute deviation (MAD), root mean square (RMS), VAR, standard deviation
(STD), and energy. Before classifying the hand motion trajectories, we perform the
procedures of feature selection and extraction methods.
In general, feature selection aims at selecting a subset of size m from an original set
of d features (d > m). Therefore, the criterion of kernel-based class separability (KBCS) with
best individual N (BIN) is to select significant features from the original features (i.e., to pick
up some important features from d) and that of linear discriminate analysis (LDA) is to
reduce the dimension of the feature space with a better recognition performance (i.e., to
reduce the size of m). The objective of the feature selection and feature extraction methods is
not only to ease the burden of computational load but also to increase the accuracy of
classification. The reduced features are used as the inputs of classifiers. In this project, we
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 3
Dept. Of ECE Mangalam College of Engineering
adopted a probabilistic neural network (PNN) as the classifier for handwritten digit and hand
gesture recognition. The contributions of this paper include the following:
1) the development of a portable digital pen with a trajectory recognition algorithm,
i.e., with the digital pen, users can deliver diverse commands by hand motions to control
electronics devices anywhere without space limitations, and
2) an effective trajectory recognition algorithm, i.e., the proposed algorithm can
efficiently select significant features from the time and frequency domains of acceleration
signals and project the feature space into a smaller feature dimension for motion recognition
with high recognition accuracy.
Microcontroller Wireless Transeiver
Fig. 2.1 Schematic diagram of existing system
The proposed trajectory recognition algorithm can be summarized in the following
steps.
Step 1) Acquire the raw acceleration signals from the pen type accelerometer module.
Step 2) Filter out the high-frequency noise of the raw accelerations by the moving
average filter and then remove the gravity from the filtered accelerations by a high-pass filter.
Finally, normalize each segmented motion interval into equal sizes via interpolation.
Step 3) Generate the time and frequency domain features from the pre-processed
acceleration of each axis including meanx,y,z, STDx,y,z, VARx,y,z, IQRx,y,z, corrxy,yz,xz,
MADx,y,z, rmsx,y,z, and energyx,y,z.
Steps 4) By KBCS (kernel-based class separability) method select significant features.
Step 5) Reduce the dimension of the selected features by LDA (Linear Discriminant
Analysis).
Tri-axial
Accelerometer
USB
Driver
MicrocontrollerPC
A/D
Converter
Processor RF
Tx
RF
Rx
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 4
Dept. Of ECE Mangalam College of Engineering
2.2 PROPOSED SYSTEM
There are a number of technologies thriving for human computer interaction or
human machine interaction such as speech recognition, vision-based gesture recognition and
tablet types of devices. These devices facilitate the intercommunication between humans and
machine more freely and without the use of traditional keyboards or mouse or remote
controls.
The proposed system is an triaxial accelerometer based digital pen. The system
operates with microcontroller as its heart. AVR ATmega32 is the microcontroller used in this
project. Microcontroller is being programmed in embedded C language and is being
simulated in AVR Studio 5. The input device to this microcontroller is a three axis
accelerometer. The accelerometer is used to measure the acceleration signals with respect to
x, y and z axis. The system is capable of identifying the digits written on air, display it on
LCD .The digital pen can also control the operation of devices. A fan and light can be turned
on and off with the aid of gestures made by the digital pen.
The proposed system can be used as a system for human machine interaction. The
propsoed system finds its application in many fields. A few are listed below:
• Sign language recognition: Just as speech recognition can transcribe speech to text,
certain types of gesture recognition software can transcribe the symbols represented
through sign language into text.
• For socially assistive robotics: By using proper sensors (accelerometers and gyros)
worn on the body of a patient and by reading the values from those sensors, robots
can assist in patient rehabilitation. The best example can be stroke rehabilitation.
• Directional indication through pointing: Pointing has a very specific purpose in
robotics. The use of gesture recognition to determine where a person is pointing is
useful for identifying the context of statements or instructions. This application is of
particular interest in the field of robotics.
• Alternative computer interfaces: Foregoing the traditional keyboard and mouse setup
to interact with a computer, strong gesture recognition could allow users to
accomplish frequent or common tasks using hand gestures to a camera.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 5
Dept. Of ECE Mangalam College of Engineering
• Immersive game technology: Gestures can be used to control interactions within
video games to try and make the game player's experience more interactive or
immersive.
• Virtual controllers: For systems where the act of finding or acquiring a physical
controller could require too much time, gestures can be used as an alternative control
mechanism. Controlling secondary devices in a car, or controlling a television set are
examples of such usage.
• Affective computing: In affective computing, gesture recognition is used in the
process of identifying emotional expression through computer systems.
• Remote control: Through the use of gesture recognition, "remote control with the
wave of a hand" of various devices is possible. The signal must not only indicate the
desired response, but also which device to be controlled.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 6
Dept. Of ECE Mangalam College of Engineering
3. PROJECT DESCRIPTION
This project presents a digital pen based system to find out the trajectory of the
written digits and gestures. As a HMI application to this device, various gestures are used to
turn on and off other devices.
x-axis y-axis z-axis
Fig.3.1 Block diagram
Microcontroller is the heart of this system. AVR ATmega32 is the microcontroller
used in this project. Microcontroller is being programmed in embedded C language and is
being simulated in AVR Studio 5. The input device to this microcontroller is a three axis
accelerometer. The accelerometer is used to measure the acceleration signals with respect to
x, y and z axis. The acceleration values are displayed on the LCD Display.
The working of the system is as follows. The entire working can be said to have two
parts - Digit recognition part and the device control part. The digital pen can identify the
Accelerometer
Module
Light
LCD Display
Power
Supply
Microcontroller
FanPC
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 7
Dept. Of ECE Mangalam College of Engineering
digits from zero to seven and display it on a PC. The trajectories of the digits are shown in
Table 3.1.
The device control part aims at giving a further application to this pen. So it aimed
of a human machine interaction one. The digital pen can be used to control other machines
like fan, light etc. Four gestures where used – up, down, right and left. Table 3.2 shows the
various gestures used. These gestures can turn on and off a light and fan respectively. In
future further more devices can also be incorporated. This can thus find application in
assisting disabled or physically challenged people to communicate easily with the
surroundings. This device thus provides a tool for an intelligent environment.
Table 3.1 Trajectory of digits
Table 3.2 Gestures
Up Down Right Left
The programming involved of various stages:
1. Display unit interface. LCD is being interfaced to PORTD of microcontroller. So
PORTD is an output port.
2. Interface with input module - the accelerometer and display its x-axis, y-axis and z-
axis reading on the LCD display. Accelerometer is attached to PORTA and thus
PORTA is set as an input port. PORTA acts as an ADC. The data from accelerometer
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 8
Dept. Of ECE Mangalam College of Engineering
is analog in nature. To display it on an LCD screen it needs to be converted into
digital format.
3. Trial and error method of finding the axis readings corresponding to various digits.
4. Accelerometer is subjected to gravity. It gave reading when the accelerometer is
displaced. This caused error in finding digits. So we added a switch. The switch is
connected to PORTD and thus PORTD is set as an input port. The device thus carries
out digit recognition only when the switch is pressed.
5. The control of the devices light and fan is carried out without the incorporation of the
switch. These devices are interfaced to PORTB.
6. The digits that are identified are displayed on a PC. The microcontroller has CMOS
logic. A MAX232 module is attached in between the PC and the microcontroller for
the RS232 serial communication with the digital pen module.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 9
Dept. Of ECE Mangalam College of Engineering
4. SYSTEM SPECIFICATIONS
4.1 HARDWARE REQUIREMENTS
The hardware used includes a microcontroller, accelerometer module, power supply,
display unit and output devices (fan, light).
4.1.1 MICROCONTROLLER
Microcontroller can be termed as a single on chip computer which includes number
of peripherals like RAM, EEPROM, Timers etc., required to perform some predefined task.
There are number of popular families of microcontrollers which are used in different
applications as per their capability and feasibility to perform the desired task, most common
of these are 8051, AVR and PIC microcontrollers. Atmel®AVR® 8-bit Microcontroller is
used in this project.
AVR was developed in the year 1996 by Atmel Corporation. The architecture of
AVR was developed by Alf-Egil Bogen and Vegard Wollan. AVR derives its name from its
developers and stands for Alf-Egil Bogen Vegard Wollan RISC microcontroller, also known
as Advanced Virtual RISC. The AVR is a modified Harvard architecture 8-bit RISC single
chip microcontroller. The AVR was one of the first microcontroller families to use on-chip
flash memory for program storage, as opposed to one-time programmable ROM, EPROM, or
EEPROM used by other microcontrollers at the time. AVR microcontrollers are available in
three categories:
1. TinyAVR – Less memory, small size, suitable only for simpler applications
2. MegaAVR – These are the most popular ones having good amount of memory (up to
256 KB), higher number of inbuilt peripherals and suitable for moderate to complex
applications.
3. XmegaAVR – Used commercially for complex applications, which require large
program memory and high speed.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 10
Dept. Of ECE Mangalam College of Engineering
Atmel®AVR®ATmega32 is a low-power CMOS 8-bit microcontroller belonging to
the family of Reduced Instruction Set Computer (RISC). In RISC architecture the instruction
set of the computer are not only fewer in number but also simpler and faster in operation. By
executing powerful instructions in a single clock cycle, the ATmega32 achieves throughputs
approaching 1 MIPS per MHz allowing the system designer to optimize power consumption
versus processing speed.
It has a rich instruction set with 32 general purpose working registers. All the 32
registers are directly connected to the Arithmetic Logic Unit (ALU), allowing two
independent registers to be accessed in one single instruction executed in one clock cycle.
The resulting architecture is more code efficient while achieving throughputs up to ten times
faster than conventional CISC microcontrollers.
The ATmega32 provides the following features: 32Kbytes of In-System
Programmable Flash Program memory with Read-While-Write capabilities, 1024bytes
EEPROM, 2Kbyte SRAM, 32 general purpose I/O lines, 32 general purpose working
registers, a JTAG interface for Boundary scan, On-chip Debugging support and
programming, three flexible Timer/Counters with compare modes, Internal and External
Interrupts, a serial programmable USART, a byte oriented Two-wire Serial Interface, an 8-
channel, 10-bit ADC with optional differential input stage with programmable gain (TQFP
package only), a programmable Watchdog Timer with Internal Oscillator, an SPI serial port,
and six software selectable power saving modes. The Idle mode stops the CPU while
allowing the USART, Two-wire interface, A/D Converter, SRAM, Timer or Counters, SPI
port, and interrupt system to continue functioning.
The Power-down mode saves the register contents but freezes the Oscillator,
disabling all other chip functions until the next External Interrupt or Hardware Reset. In
Power-save mode, the Asynchronous Timer continues to run, allowing the user to maintain a
timer base while the rest of the device is sleeping. The ADC Noise Reduction mode stops the
CPU and all I/O modules except Asynchronous Timer and ADC, to minimize switching noise
during ADC conversions. In Standby mode, the crystal/resonator Oscillator is running while
the rest of the device is sleeping. This allows very fast start-up combined with low-power
consumption. In Extended Standby mode, both the main Oscillator and the Asynchronous
Timer continue to run.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 11
Dept. Of ECE Mangalam College of Engineering
The device is manufactured using Atmel‟s high density non-volatile memory
technology. The on-chip ISP Flash allows the program memory to be reprogrammed in-
system through an SPI serial interface, by a conventional non-volatile memory programmer,
or by an On-chip Boot program running on the AVR core. The boot program can use any
interface to download the application program in the Application Flash memory. Software in
the Boot Flash section will continue to run while the Application Flash section is updated,
providing true Read-While-Write operation. By combining an 8-bit RISC CPU with In-
System Self-Programmable Flash on a monolithic chip, the Atmel ATmega32 is a powerful
microcontroller that provides a highly-flexible and cost-effective solution to many embedded
control applications.
The Atmel AVR ATmega32 is supported with a full suite of program and system
development tools including: C compilers, macro assemblers, program debugger/simulators,
in-circuit emulators, and evaluation kits. It has four ports- PORTA, PORTB, PORTC and
PORTD. PORTA serves as the analog inputs to the A/D Converter. It also serves as an 8-bit
bi-directional I/O port, if the A/D Converter is not used. Port pins can provide internal pull-
up resistors. When pins PA0 to PA7 are used as inputs and are externally pulled low, they
will source current if the internal pull-up resistors are activated. The PORTA pins are tri-
stated when a reset condition becomes active, even if the clock is not running. PORTB is an
8-bit bi-directional I/O port with internal pull-up resistors. The PORTB output buffers have
symmetrical drive characteristics with both high sink and source capability. As inputs,
PORTB pins that are externally pulled low will source current if the pull-up resistors are
activated. PORTC is an 8-bit bi-directional I/O port with internal pull-up resistors.
The PORTC output buffers have symmetrical drive characteristics with both high
sink and source capability. As inputs, PORTC pins that are externally pulled low will source
current if the pull-up resistors are activated. The PORTC pins are tri-stated when a reset
condition becomes active, even if the clock is not running. If the JTAG interface is enabled,
the pull-up resistors on pins PC5 (TDI), PC3 (TMS) and PC2 (TCK) will be activated even if
a reset occurs. PORTD is an 8-bit bi-directional I/O port with internal pull-up resistors
(selected for each bit). The PORTD output buffers have symmetrical drive characteristics
with both high sink and source capability. As inputs, PORTD pins that are externally pulled
low will source current if the pull-up resistors are activated. The PORTD pins are tri-stated
when a reset condition becomes active, even if the clock is not running.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 12
Dept. Of ECE Mangalam College of Engineering
It provides several different interrupt sources. These interrupts and the separate reset
vector each have a separate program vector in the program memory space. All interrupts are
assigned individual enable bits which must be written logic one together with the Global
Interrupt Enable bit in the Status Register in order to enable the interrupt. The External
Interrupts are triggered by the INT0, INT1, and INT2 pins. Sleep modes enable the
application to shut down unused modules in the MCU, thereby saving power. The AVR
provides various sleep modes allowing the user to tailor the power consumption to the
application‟s requirements. Reliability Qualification results show that the projected data
retention failure rate is much less than 1 PPM over 20 years at 85°C or 100 years at 25°C.
4.1.2 ACCELEROMETER MODULE
The accelerometer is a device that can measure the force of acceleration, whether
caused by gravity or by movement. An accelerometer can therefore measure the speed of
movement of an object it is attached to. Accelerometers are one of the most frequently
utilized physical sensors for detecting and measuring motion. Accelerometers have found
applications ranging from measurement and control to inertial navigation. MEMS
implementations of accelerometers have found a large commercial market in automotive
airbag deployment systems. The basic configuration of an accelerometer is the same for all of
these applications and is shown in Figure4.1. Three items are the basic components of an
accelerometer:
• Inertial mass
• Suspension
• Sensing element
The suspension supporting the inertial mass will deflect under acceleration due to
D‟Alembert‟s principle. A sensing element will transduce the deflection of the suspension to
an electrical signal. The transduction can be accomplished by a number of means, but the
most common utilize piezoresistive, piezoelectric, or capacitance means [3].
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 13
Dept. Of ECE Mangalam College of Engineering
Fig.4.1 Accelerometer components [3]
ADXL335 accelerometer module is being used in this paper. It is a small, thin, low
power, complete 3-axis accelerometer with signal conditioned voltage outputs. The product
measures acceleration with a minimum full-scale range of ±3 g. It can measure the static
acceleration of gravity in tilt-sensing applications, as well as dynamic acceleration resulting
from motion, shock, or vibration. The user selects the bandwidth of the accelerometer using
the CX, CY, and CZ capacitors at the XOUT, YOUT, and ZOUT pins. Bandwidths can be
selected to suit the application, with a range of 0.5 Hz to 1600 Hz for the X and Y axes, and a
range of 0.5 Hz to 550 Hz for the Z axis. The ADXL335 is available in a small, low profile, 4
mm × 4 mm × 1.45 mm, 16-lead, plastic lead frame chip scale package.
It contains a polysilicon surface micro-machined sensor and signal conditioning
circuitry to implement open-loop acceleration measurement architecture. The output signals
are analog voltages that are proportional to acceleration. The accelerometer can measure the
static acceleration of gravity in tilt-sensing applications as well as dynamic acceleration
resulting from motion, shock, or vibration. The sensor is a polysilicon surface-
micromachined structure built on top of a silicon wafer. Polysilicon springs suspend the
structure over the surface of the wafer and provide a resistance against acceleration forces.
Deflection of the structure is measured using a differential capacitor that consists of
independent fixed plates and plates attached to the moving mass. The fixed plates are driven
by 180° out-of-phase square waves. Acceleration deflects the moving mass and unbalances
the differential capacitor resulting in a sensor output whose amplitude is proportional to
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 14
Dept. Of ECE Mangalam College of Engineering
acceleration. Phase-sensitive demodulation techniques are then used to determine the
magnitude and direction of the acceleration.
Rather than using additional temperature compensation circuitry, innovative design
techniques ensure that high performance is built in to the ADXL335. As a result, there is no
quantization error or non-monotonic behavior, and temperature hysteresis is very low
(typically less than 3 mg over the −25°C to +70°C temperature range). The ADXL335 uses a
single structure for sensing the X, Y, and Z axes. As a result, the three axes‟ sense directions
are highly orthogonal and have little cross-axis sensitivity. Mechanical misalignment of the
sensor die to the package is the chief source of cross-axis sensitivity.
Capacitance measurement is one of the most versatile methods of position
measurement. The parallel-plate capacitor can vary either with vertical motion of a movable
plate, modifying the gap, or by transverse motion of one plate relative to another, modifying
the effective area of the capacitor. Figure 4.2shows the arrangement of a parallel plate
capacitor.
Fig. 4.2 Parallel plate capacitor [2]
Motion of the moveable component in the indicated direction increases one
capacitance and decreases the other. A variant of the parallel-plate differential capacitor
would have the middle and lower plate fixed and only the upper plate moveable. In this
configuration, motion of the upper plate modifies one capacitor while the other remains
constant.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 15
Dept. Of ECE Mangalam College of Engineering
4.1.3 POWER SUPPLY AND VOLTAGE REGULATOR
A power supply is a device that supplies electric power to an electrical load. 230V,
50Hz ac mains supply is made to step down to 12 V by means of a step down transformer.
The 12 V ac is made to dc with the aid of a bridge rectifier. Bridge rectifier makes use of four
diodes in a bridge arrangement to achieve full-wave rectification. W10 bridge rectifier used
here has high case dielectric strength, high surge current capability, high reliability, low
reverse current, low forward voltage drop and ideal for printed circuit board. It also has
Reliable low cost construction utilizing moulded plastic technique.
Fig 4.3 Voltage Regulator IC
A regulated power supply is one that controls the output voltage or current to a
specific value. The controlled value is held nearly constant despite variations in either load
current or the voltage supplied by the power supply's energy source. 7805 is a voltage
regulator integrated circuit. It is a member of 78xx series of fixed linear voltage regulator
ICs. The voltage source in a circuit may have fluctuations and would not give the fixed
voltage output. The voltage regulator IC maintains the output voltage at a constant value. The
xx in 78xx indicates the fixed output voltage it is designed to provide. 7805 provides +5V
regulated power supply. Capacitors of suitable values can be connected at input and output
pins depending upon the respective voltage levels. ST 7805 is the voltage regulator used. It is
a 3-terminal 1 A positive voltage regulator with Thermal Overload Short Circuit and Output
Transistor Safe Operating Area Protection. Figure 4.3shows a voltage regulator IC.
4.1.4 DISPLAY UNIT
An LCD is a small low cost display. A 16 x 2 LCD display is very basic module and
is very commonly used in various devices and circuits. These modules are preferred over
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 16
Dept. Of ECE Mangalam College of Engineering
seven segments and other multi segment LEDs. LCDs are economical, easily programmable
and have no limitation of displaying special & even custom characters. It is easy to interface
with a micro-controller. A 16 x 2 LCD means it can display 16 characters per line and there
are 2 such lines. In this LCD each character is displayed in 5x7 pixel matrix.
Table 4.1LCD Commands
Command Operation
0x01 LCD Clear
0x38 Select 5 x 7 matrix
0x80 to move cursor to first line first position
0x81 to move cursor to first line second position
0x8F to move cursor to first line last position
0xC0 display on cursor on
0x0E display on cursor off
0x0C to move cursor to second line first position
0x06 shift cursor to right
0x1C shift entire display to right by one position
0x18 shift entire display to left by one position
This LCD has two registers, namely, Command and Data. The command register
stores the command instructions given to the LCD. A command is an instruction given to
LCD to do a predefined task like initializing it, clearing its screen, setting the cursor position,
controlling display etc. The data register stores the data to be displayed on the LCD. The data
is the ASCII value of the character to be displayed on the LCD. The display is capable of
displaying 224 different characters and symbols. It is also facilitated with backlight Led.
Table 4.1shows some commonly used LCD commands.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 17
Dept. Of ECE Mangalam College of Engineering
4.1.5 MAX232
The MAX232 is an IC, first created by Maxim Integrated Products, that converts
signals from an RS-232 serial port to signals suitable for use in TTL compatible digital logic
circuits. The MAX232 is a dual driver/receiver and typically converts the RX, TX, CTS and
RTS signals. The drivers provide RS-232 voltage level outputs (approx. ± 7.5 V) from a
single + 5 V supply via on-chip charge pumps and external capacitors. This makes it useful
for implementing RS-232 in devices that otherwise do not need any voltages outside the 0 V
to + 5 V range, as power supply design does not need to be made more complicated just for
driving the RS-232 in this case. The receivers reduce RS-232 inputs (which may be as high as
± 25 V), to standard 5 V TTL levels. These receivers have a typical threshold of 1.3 V, and a
typical hysteresis of 0.5 V.
When a MAX232 IC receives a TTL level to convert, it changes a TTL Logic 0 to
between +3 and +15 V, and changes TTL Logic 1 to between -3 to -15 V, and vice versa for
converting from RS232 to TTL. This can be confusing on realizing that the RS232 Data
Transmission voltages at a certain logic state are opposite from the RS232 Control Line
voltages at the same logic state. Table 4.2 shows the voltage levels.
Table 4.2 Voltage Levels
RS232 Line Type & Logic Level RS232
Voltage
TTL Voltage to/from
MAX232
Data Transmission (Rx/Tx)
Logic 0
+3 V to +15 V
Data Transmission (Rx/Tx)
Logic 1
-3 V to -15 V
Control Signals
(RTS/CTS/DTR/DSR)
Logic 0 -3 V to -15V
Control Signals
(RTS/CTS/DTR/DSR)
Logic 1 +3 V to+15V
4.1.6 OTHER DEVICES
The other output devices interfaced to this are a light and a fan.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 18
Dept. Of ECE Mangalam College of Engineering
4.2 SOFTWARE REQUIREMENTS
4.2.1 AVR STUDIO 5
The software used is AVR Studio 5. It is a Windows based Integrated Development
Environment (IDE) from the Atmel Technology incorporated AVR microcontroller families.
AVR Studio 5 supports all AVR microcontrollers. The various features of this software are
• Capability to work in C programming Environment.
• Creates source code using the built in editor.
• Compile and link source code using various tools.
• A compiler, linker and library come with AVR Studio.
AVR Studio 5 includes a compiler, assembler and a simulator, and interfaces
seamlessly with in-system debuggers and programmers to make code development easier. It
is a full software development environment with an editor, simulator, programmer, etc. The
AVR Software Framework is a collection of production-ready source code, written and
optimized by experts and tested in hundreds of production designs. It comes with its own
integrated C compiler the AVR GNU C Compiler (GCC). As such you do not need a third
party C compiler.
It provides a single environment to develop programs for both the 8-bits and 32-bits
AVR series of microcontrollers. Using these peripheral drivers, communication stacks and
application-specific libraries is the quick and effortless way to complete a project. The AVR
Studio 5 editor simplifies code editing and facilitates coding more efficiently. Type a few
letters of a symbol, and AVR Studio 5 will display a list of suggestions. Figure 4.4 to 4.9
shows the various stages of programming in AVR Studio 5.
Steps to program a code in AVR Studio 5 is as follows:
1. Click open AVR Studio 5.
2. Select New Project
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 19
Dept. Of ECE Mangalam College of Engineering
Fig.4.4Opening window AVR Studio 5
3. Select C Executable Project
Fig.4.5Project Select
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 20
Dept. Of ECE Mangalam College of Engineering
4. Name the project (here named as DIGITALPEN) and click ok.
Fig4.6Name Project
5. Select the device as ATmega32 and click ok
Fig4.7Device selection
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 21
Dept. Of ECE Mangalam College of Engineering
6. Enter the code. The programming is done in Embedded C language.
Fig.4.8 Programming window
7. From tool bar select Build and from its drop down window select build solution
Fig.4.9Build Solution
8. The output window shows errors and warnings in case of wrong entries, missing
variables or symbols. On successful completion it displays build succeeded.
9. This code needs to be loaded to the microcontroller for its operation.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 22
Dept. Of ECE Mangalam College of Engineering
4.2.2 KHAZAMA AVR PROGRAMMER
The tool used to program the ATmega32 AVR microcontroller is the Khazama AVR
Programmer. Programming using this tool is very simple, fast and reliable. Khazama can be
used to burn the code into all microcontrollers in AVR family. The steps followed in
programming microcontroller are depicted in Figure 4.10 to Figure 4.12.
1. Connect the microcontroller to the PC.
2. Click open Khazama AVR programmer
Fig.4.10Opening Window
3. Select the AVR from the dropdown window. Here the microcontroller used is
ATMEGA32.
Fig.4.11AVR Selection
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 23
Dept. Of ECE Mangalam College of Engineering
4. Click command and read chip signature from the drop down window.
Fig.4.12Reading Chip Signature
5. Select icon and Load flash to Hex buffer. Click Auto Program. The hex code will
be burned on to the AVR microcontroller and the window shows a successfully
completed message when the burning is complete.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 24
Dept. Of ECE Mangalam College of Engineering
5. SIMULATION RESULTS
The programming language used in this project is Embedded C. The code was
written and simulated in AVR Studio 5. Figure 5.1 shows the window showing the successful
compilation of the code.
Fig. 5.1 Simulation Result
The code was loaded to the microcontroller. The device successfully recognized
digits zero to seven. The digits were displayed on the LCD. The four gestures used in this
project turned on and off the light and the fan.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 25
Dept. Of ECE Mangalam College of Engineering
6. CONCLUSION
This project has presented an accelerometer based framework that can that can be
utilized for acceleration-based handwriting and gesture recognition. The proposed system is
utilized for digit and gesture recognition and thus it is an effective tool for HCI applications.
Users can use the pen to write digits or make hand gestures as shown in Table 3.1 and 3.2,
and the accelerations of hand motions are measured by the accelerometer. The recognized
digits are displayed on a computer for further applications. So, by changing the position of
accelerometer users can show the numerical characters on the LCD.
Gesture recognition is useful for processing information from humans which is not
conveyed through speech or type. As well, there are various types of gestures which can be
identified by computers.The system finds its application in following fields. It acts as a
communication tool between deaf, dumb and the blind. It has other human machine
applications in the field of game control, titlt recognition, tv remote, mobile text scrolling,
mobile application selection, presentation pointer etc.
Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 26
Dept. Of ECE Mangalam College of Engineering
7. REFERENCES
[1] J.C.Wang, and F.C Chuang, „‟An Accelerometer-Based Digital Pen with a Trajectory
Recognition Algorithm for Handwritten Digit and Gesture Recognition‟‟, IEEE
Trans. on industrial electronics, vol. 59, no. 7, July 2012
[2] Stephen D Senturia, “Microsystem Design”, Kluwer Academic Publishers, New
York.
[3] L. L. Faulkner, James J. Allen, „‟Micro Electro mechanical System Design‟‟, Taylor
& Francis Group, LLC,2005

Weitere ähnliche Inhalte

Was ist angesagt?

Human Activity Recognition in Android
Human Activity Recognition in AndroidHuman Activity Recognition in Android
Human Activity Recognition in Android
Surbhi Jain
 
Reading System for the Blind PPT
Reading System for the Blind PPTReading System for the Blind PPT
Reading System for the Blind PPT
Binayak Ghosh
 
Medical Handsfree System - Project Paper
Medical Handsfree System - Project PaperMedical Handsfree System - Project Paper
Medical Handsfree System - Project Paper
Guy Peleg
 

Was ist angesagt? (17)

IRJET- Enhanced Look Based Media Player with Hand Gesture Recognition
IRJET-  	  Enhanced Look Based Media Player with Hand Gesture RecognitionIRJET-  	  Enhanced Look Based Media Player with Hand Gesture Recognition
IRJET- Enhanced Look Based Media Player with Hand Gesture Recognition
 
Hand gesture recognition using ultrasonic sensor and atmega128 microcontroller
Hand gesture recognition using ultrasonic sensor and atmega128 microcontrollerHand gesture recognition using ultrasonic sensor and atmega128 microcontroller
Hand gesture recognition using ultrasonic sensor and atmega128 microcontroller
 
B010320711
B010320711B010320711
B010320711
 
Human Activity Recognition in Android
Human Activity Recognition in AndroidHuman Activity Recognition in Android
Human Activity Recognition in Android
 
Literature on safety module for vehicular driver assistance
Literature on safety module for vehicular driver assistance Literature on safety module for vehicular driver assistance
Literature on safety module for vehicular driver assistance
 
Complex Weld Seam Detection Using Computer Vision Linked In
Complex Weld Seam Detection Using Computer Vision Linked InComplex Weld Seam Detection Using Computer Vision Linked In
Complex Weld Seam Detection Using Computer Vision Linked In
 
Project-FINAL
Project-FINALProject-FINAL
Project-FINAL
 
IRJET- Bemythirdeye- A Smart Electronic Blind Stick with Goggles
IRJET- Bemythirdeye- A Smart Electronic Blind Stick with GogglesIRJET- Bemythirdeye- A Smart Electronic Blind Stick with Goggles
IRJET- Bemythirdeye- A Smart Electronic Blind Stick with Goggles
 
An Approach for Object and Scene Detection for Blind Peoples Using Vocal Vision.
An Approach for Object and Scene Detection for Blind Peoples Using Vocal Vision.An Approach for Object and Scene Detection for Blind Peoples Using Vocal Vision.
An Approach for Object and Scene Detection for Blind Peoples Using Vocal Vision.
 
IRJET - Real Time Muscle Fatigue Monitoring using IoT Cloud Computing
IRJET - Real Time Muscle Fatigue Monitoring using IoT Cloud ComputingIRJET - Real Time Muscle Fatigue Monitoring using IoT Cloud Computing
IRJET - Real Time Muscle Fatigue Monitoring using IoT Cloud Computing
 
311 314
311 314311 314
311 314
 
Reading System for the Blind PPT
Reading System for the Blind PPTReading System for the Blind PPT
Reading System for the Blind PPT
 
30 ball
30 ball30 ball
30 ball
 
Medical Handsfree System - Project Paper
Medical Handsfree System - Project PaperMedical Handsfree System - Project Paper
Medical Handsfree System - Project Paper
 
IRJET= Air Writing: Gesture Recognition using Ultrasound Sensors and Grid-Eye...
IRJET= Air Writing: Gesture Recognition using Ultrasound Sensors and Grid-Eye...IRJET= Air Writing: Gesture Recognition using Ultrasound Sensors and Grid-Eye...
IRJET= Air Writing: Gesture Recognition using Ultrasound Sensors and Grid-Eye...
 
Wastlund What You See Is Where You Go Testing A Gaze Driven Power Wheelchair ...
Wastlund What You See Is Where You Go Testing A Gaze Driven Power Wheelchair ...Wastlund What You See Is Where You Go Testing A Gaze Driven Power Wheelchair ...
Wastlund What You See Is Where You Go Testing A Gaze Driven Power Wheelchair ...
 
Motion capture process and systems
Motion capture process and systemsMotion capture process and systems
Motion capture process and systems
 

Ähnlich wie Gesture final report new

Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
IJERA Editor
 
Ijarcet vol-2-issue-3-938-941
Ijarcet vol-2-issue-3-938-941Ijarcet vol-2-issue-3-938-941
Ijarcet vol-2-issue-3-938-941
Editor IJARCET
 
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETERDESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
IJCSEA Journal
 
Gesture control algorithm for personal computers
Gesture control algorithm for personal computersGesture control algorithm for personal computers
Gesture control algorithm for personal computers
eSAT Journals
 

Ähnlich wie Gesture final report new (20)

E010122431
E010122431E010122431
E010122431
 
Digital Pen for Handwritten Digit and Gesture Recognition Using Trajectory Re...
Digital Pen for Handwritten Digit and Gesture Recognition Using Trajectory Re...Digital Pen for Handwritten Digit and Gesture Recognition Using Trajectory Re...
Digital Pen for Handwritten Digit and Gesture Recognition Using Trajectory Re...
 
Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
Hand Motion Gestures For Mobile Communication Based On Inertial Sensors For O...
 
Analysis of Inertial Sensor Data Using Trajectory Recognition Algorithm
Analysis of Inertial Sensor Data Using Trajectory Recognition AlgorithmAnalysis of Inertial Sensor Data Using Trajectory Recognition Algorithm
Analysis of Inertial Sensor Data Using Trajectory Recognition Algorithm
 
Media Control Using Hand Gesture Moments
Media Control Using Hand Gesture MomentsMedia Control Using Hand Gesture Moments
Media Control Using Hand Gesture Moments
 
1886 1892
1886 18921886 1892
1886 1892
 
1886 1892
1886 18921886 1892
1886 1892
 
A Digital Pen with a Trajectory Recognition Algorithm
A Digital Pen with a Trajectory Recognition AlgorithmA Digital Pen with a Trajectory Recognition Algorithm
A Digital Pen with a Trajectory Recognition Algorithm
 
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardDevelopment of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
 
Ijarcet vol-2-issue-3-938-941
Ijarcet vol-2-issue-3-938-941Ijarcet vol-2-issue-3-938-941
Ijarcet vol-2-issue-3-938-941
 
The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)
 
Aacellerometer
AacellerometerAacellerometer
Aacellerometer
 
GESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGYGESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGY
 
Gestures Based Sign Interpretation System using Hand Glove
Gestures Based Sign Interpretation System using Hand GloveGestures Based Sign Interpretation System using Hand Glove
Gestures Based Sign Interpretation System using Hand Glove
 
IRJET - A Smart Assistant for Aiding Dumb People
IRJET - A Smart Assistant for Aiding Dumb PeopleIRJET - A Smart Assistant for Aiding Dumb People
IRJET - A Smart Assistant for Aiding Dumb People
 
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETERDESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
DESIGN AND DEVELOPMENT OF WRIST-TILT BASED PC CURSOR CONTROL USING ACCELEROMETER
 
Real Time Vision Hand Gesture Recognition Based Media Control via LAN & Wirel...
Real Time Vision Hand Gesture Recognition Based Media Control via LAN & Wirel...Real Time Vision Hand Gesture Recognition Based Media Control via LAN & Wirel...
Real Time Vision Hand Gesture Recognition Based Media Control via LAN & Wirel...
 
Gesture control algorithm for personal computers
Gesture control algorithm for personal computersGesture control algorithm for personal computers
Gesture control algorithm for personal computers
 
Gesture control algorithm for personal computers
Gesture control algorithm for personal computersGesture control algorithm for personal computers
Gesture control algorithm for personal computers
 
Controlling Computer using Hand Gestures
Controlling Computer using Hand GesturesControlling Computer using Hand Gestures
Controlling Computer using Hand Gestures
 

Gesture final report new

  • 1. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 1 Dept. Of ECE Mangalam College of Engineering 1. INTRODUCTION There are a number of technologies thriving for human computer interaction or human machine interaction such as speech recognition, vision-based gesture recognition and tablet types of devices. These devices facilitate the intercommunication between humans and machine more freely and without the use of traditional keyboards or mouse or remote controls. Gesture or trajectory recognition based on inertial devices is an original, direct human-computer interacting way, because the motion of limbs is driven by the force from muscles and the inertial devices could detect the outcome of the force, say acceleration and angular velocity, directly, so people could even use them without any distraction all the time. A significant advantage of inertial sensors for general motion sensing is that they can be operated without any external reference and limitation in working conditions. However, motion trajectory recognition is relatively complicated because different users have different speeds and styles to generate various motion trajectories. Thus, many researchers have tried to narrow down the problem domain for increasing the accuracy of handwriting recognition systems [1]. The term gesture recognition has been used to refer more narrowly to non-text-input handwriting symbols. Gesture recognition enables humans to communicate with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly. This could potentially make conventional input devices such as mouse, keyboards and even touch-screens redundant. Just as speech recognition can transcribe speech to text, certain types of gesture recognition software can transcribe the symbols represented through sign language into text. By using proper sensors (eg. accelerometers) worn on the body of a patient and by reading the values from those sensors, robots can assist in patient rehabilitation. The best example can be stroke rehabilitation.
  • 2. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 2 Dept. Of ECE Mangalam College of Engineering 2. SYSTEM ANALYSIS 2.1 EXISTING SYSTEM In the present system, a trajectory recognition algorithm was used for digit and gesture recognition applications.this paper developed a pen-type portable device and a trajectory recognition algorithm. The pen-type portable device consists of a triaxial accelerometer, a microcontroller, and an RF wireless transmission module. The acceleration signals measured from the tri-axial accelerometer are transmitted to a computer via the wireless module. Users can utilize this digital pen to write digits and make hand gestures at normal speed. The measured acceleration signals of these motions can be recognized by the trajectory recognition algorithm. The recognition procedure is composed of acceleration acquisition, signal pre- processing, feature generation, feature selection, and feature extraction. The acceleration signals of hand motions are measured by the pen-type portable device. The signal pre- processing procedure consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. These two filters are applied to remove high frequency noise and gravitational acceleration from the raw data, respectively. The features of the pre-processed acceleration signals of each axis include mean, correlation among axes, inter-quartile range (IQR), mean absolute deviation (MAD), root mean square (RMS), VAR, standard deviation (STD), and energy. Before classifying the hand motion trajectories, we perform the procedures of feature selection and extraction methods. In general, feature selection aims at selecting a subset of size m from an original set of d features (d > m). Therefore, the criterion of kernel-based class separability (KBCS) with best individual N (BIN) is to select significant features from the original features (i.e., to pick up some important features from d) and that of linear discriminate analysis (LDA) is to reduce the dimension of the feature space with a better recognition performance (i.e., to reduce the size of m). The objective of the feature selection and feature extraction methods is not only to ease the burden of computational load but also to increase the accuracy of classification. The reduced features are used as the inputs of classifiers. In this project, we
  • 3. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 3 Dept. Of ECE Mangalam College of Engineering adopted a probabilistic neural network (PNN) as the classifier for handwritten digit and hand gesture recognition. The contributions of this paper include the following: 1) the development of a portable digital pen with a trajectory recognition algorithm, i.e., with the digital pen, users can deliver diverse commands by hand motions to control electronics devices anywhere without space limitations, and 2) an effective trajectory recognition algorithm, i.e., the proposed algorithm can efficiently select significant features from the time and frequency domains of acceleration signals and project the feature space into a smaller feature dimension for motion recognition with high recognition accuracy. Microcontroller Wireless Transeiver Fig. 2.1 Schematic diagram of existing system The proposed trajectory recognition algorithm can be summarized in the following steps. Step 1) Acquire the raw acceleration signals from the pen type accelerometer module. Step 2) Filter out the high-frequency noise of the raw accelerations by the moving average filter and then remove the gravity from the filtered accelerations by a high-pass filter. Finally, normalize each segmented motion interval into equal sizes via interpolation. Step 3) Generate the time and frequency domain features from the pre-processed acceleration of each axis including meanx,y,z, STDx,y,z, VARx,y,z, IQRx,y,z, corrxy,yz,xz, MADx,y,z, rmsx,y,z, and energyx,y,z. Steps 4) By KBCS (kernel-based class separability) method select significant features. Step 5) Reduce the dimension of the selected features by LDA (Linear Discriminant Analysis). Tri-axial Accelerometer USB Driver MicrocontrollerPC A/D Converter Processor RF Tx RF Rx
  • 4. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 4 Dept. Of ECE Mangalam College of Engineering 2.2 PROPOSED SYSTEM There are a number of technologies thriving for human computer interaction or human machine interaction such as speech recognition, vision-based gesture recognition and tablet types of devices. These devices facilitate the intercommunication between humans and machine more freely and without the use of traditional keyboards or mouse or remote controls. The proposed system is an triaxial accelerometer based digital pen. The system operates with microcontroller as its heart. AVR ATmega32 is the microcontroller used in this project. Microcontroller is being programmed in embedded C language and is being simulated in AVR Studio 5. The input device to this microcontroller is a three axis accelerometer. The accelerometer is used to measure the acceleration signals with respect to x, y and z axis. The system is capable of identifying the digits written on air, display it on LCD .The digital pen can also control the operation of devices. A fan and light can be turned on and off with the aid of gestures made by the digital pen. The proposed system can be used as a system for human machine interaction. The propsoed system finds its application in many fields. A few are listed below: • Sign language recognition: Just as speech recognition can transcribe speech to text, certain types of gesture recognition software can transcribe the symbols represented through sign language into text. • For socially assistive robotics: By using proper sensors (accelerometers and gyros) worn on the body of a patient and by reading the values from those sensors, robots can assist in patient rehabilitation. The best example can be stroke rehabilitation. • Directional indication through pointing: Pointing has a very specific purpose in robotics. The use of gesture recognition to determine where a person is pointing is useful for identifying the context of statements or instructions. This application is of particular interest in the field of robotics. • Alternative computer interfaces: Foregoing the traditional keyboard and mouse setup to interact with a computer, strong gesture recognition could allow users to accomplish frequent or common tasks using hand gestures to a camera.
  • 5. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 5 Dept. Of ECE Mangalam College of Engineering • Immersive game technology: Gestures can be used to control interactions within video games to try and make the game player's experience more interactive or immersive. • Virtual controllers: For systems where the act of finding or acquiring a physical controller could require too much time, gestures can be used as an alternative control mechanism. Controlling secondary devices in a car, or controlling a television set are examples of such usage. • Affective computing: In affective computing, gesture recognition is used in the process of identifying emotional expression through computer systems. • Remote control: Through the use of gesture recognition, "remote control with the wave of a hand" of various devices is possible. The signal must not only indicate the desired response, but also which device to be controlled.
  • 6. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 6 Dept. Of ECE Mangalam College of Engineering 3. PROJECT DESCRIPTION This project presents a digital pen based system to find out the trajectory of the written digits and gestures. As a HMI application to this device, various gestures are used to turn on and off other devices. x-axis y-axis z-axis Fig.3.1 Block diagram Microcontroller is the heart of this system. AVR ATmega32 is the microcontroller used in this project. Microcontroller is being programmed in embedded C language and is being simulated in AVR Studio 5. The input device to this microcontroller is a three axis accelerometer. The accelerometer is used to measure the acceleration signals with respect to x, y and z axis. The acceleration values are displayed on the LCD Display. The working of the system is as follows. The entire working can be said to have two parts - Digit recognition part and the device control part. The digital pen can identify the Accelerometer Module Light LCD Display Power Supply Microcontroller FanPC
  • 7. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 7 Dept. Of ECE Mangalam College of Engineering digits from zero to seven and display it on a PC. The trajectories of the digits are shown in Table 3.1. The device control part aims at giving a further application to this pen. So it aimed of a human machine interaction one. The digital pen can be used to control other machines like fan, light etc. Four gestures where used – up, down, right and left. Table 3.2 shows the various gestures used. These gestures can turn on and off a light and fan respectively. In future further more devices can also be incorporated. This can thus find application in assisting disabled or physically challenged people to communicate easily with the surroundings. This device thus provides a tool for an intelligent environment. Table 3.1 Trajectory of digits Table 3.2 Gestures Up Down Right Left The programming involved of various stages: 1. Display unit interface. LCD is being interfaced to PORTD of microcontroller. So PORTD is an output port. 2. Interface with input module - the accelerometer and display its x-axis, y-axis and z- axis reading on the LCD display. Accelerometer is attached to PORTA and thus PORTA is set as an input port. PORTA acts as an ADC. The data from accelerometer
  • 8. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 8 Dept. Of ECE Mangalam College of Engineering is analog in nature. To display it on an LCD screen it needs to be converted into digital format. 3. Trial and error method of finding the axis readings corresponding to various digits. 4. Accelerometer is subjected to gravity. It gave reading when the accelerometer is displaced. This caused error in finding digits. So we added a switch. The switch is connected to PORTD and thus PORTD is set as an input port. The device thus carries out digit recognition only when the switch is pressed. 5. The control of the devices light and fan is carried out without the incorporation of the switch. These devices are interfaced to PORTB. 6. The digits that are identified are displayed on a PC. The microcontroller has CMOS logic. A MAX232 module is attached in between the PC and the microcontroller for the RS232 serial communication with the digital pen module.
  • 9. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 9 Dept. Of ECE Mangalam College of Engineering 4. SYSTEM SPECIFICATIONS 4.1 HARDWARE REQUIREMENTS The hardware used includes a microcontroller, accelerometer module, power supply, display unit and output devices (fan, light). 4.1.1 MICROCONTROLLER Microcontroller can be termed as a single on chip computer which includes number of peripherals like RAM, EEPROM, Timers etc., required to perform some predefined task. There are number of popular families of microcontrollers which are used in different applications as per their capability and feasibility to perform the desired task, most common of these are 8051, AVR and PIC microcontrollers. Atmel®AVR® 8-bit Microcontroller is used in this project. AVR was developed in the year 1996 by Atmel Corporation. The architecture of AVR was developed by Alf-Egil Bogen and Vegard Wollan. AVR derives its name from its developers and stands for Alf-Egil Bogen Vegard Wollan RISC microcontroller, also known as Advanced Virtual RISC. The AVR is a modified Harvard architecture 8-bit RISC single chip microcontroller. The AVR was one of the first microcontroller families to use on-chip flash memory for program storage, as opposed to one-time programmable ROM, EPROM, or EEPROM used by other microcontrollers at the time. AVR microcontrollers are available in three categories: 1. TinyAVR – Less memory, small size, suitable only for simpler applications 2. MegaAVR – These are the most popular ones having good amount of memory (up to 256 KB), higher number of inbuilt peripherals and suitable for moderate to complex applications. 3. XmegaAVR – Used commercially for complex applications, which require large program memory and high speed.
  • 10. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 10 Dept. Of ECE Mangalam College of Engineering Atmel®AVR®ATmega32 is a low-power CMOS 8-bit microcontroller belonging to the family of Reduced Instruction Set Computer (RISC). In RISC architecture the instruction set of the computer are not only fewer in number but also simpler and faster in operation. By executing powerful instructions in a single clock cycle, the ATmega32 achieves throughputs approaching 1 MIPS per MHz allowing the system designer to optimize power consumption versus processing speed. It has a rich instruction set with 32 general purpose working registers. All the 32 registers are directly connected to the Arithmetic Logic Unit (ALU), allowing two independent registers to be accessed in one single instruction executed in one clock cycle. The resulting architecture is more code efficient while achieving throughputs up to ten times faster than conventional CISC microcontrollers. The ATmega32 provides the following features: 32Kbytes of In-System Programmable Flash Program memory with Read-While-Write capabilities, 1024bytes EEPROM, 2Kbyte SRAM, 32 general purpose I/O lines, 32 general purpose working registers, a JTAG interface for Boundary scan, On-chip Debugging support and programming, three flexible Timer/Counters with compare modes, Internal and External Interrupts, a serial programmable USART, a byte oriented Two-wire Serial Interface, an 8- channel, 10-bit ADC with optional differential input stage with programmable gain (TQFP package only), a programmable Watchdog Timer with Internal Oscillator, an SPI serial port, and six software selectable power saving modes. The Idle mode stops the CPU while allowing the USART, Two-wire interface, A/D Converter, SRAM, Timer or Counters, SPI port, and interrupt system to continue functioning. The Power-down mode saves the register contents but freezes the Oscillator, disabling all other chip functions until the next External Interrupt or Hardware Reset. In Power-save mode, the Asynchronous Timer continues to run, allowing the user to maintain a timer base while the rest of the device is sleeping. The ADC Noise Reduction mode stops the CPU and all I/O modules except Asynchronous Timer and ADC, to minimize switching noise during ADC conversions. In Standby mode, the crystal/resonator Oscillator is running while the rest of the device is sleeping. This allows very fast start-up combined with low-power consumption. In Extended Standby mode, both the main Oscillator and the Asynchronous Timer continue to run.
  • 11. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 11 Dept. Of ECE Mangalam College of Engineering The device is manufactured using Atmel‟s high density non-volatile memory technology. The on-chip ISP Flash allows the program memory to be reprogrammed in- system through an SPI serial interface, by a conventional non-volatile memory programmer, or by an On-chip Boot program running on the AVR core. The boot program can use any interface to download the application program in the Application Flash memory. Software in the Boot Flash section will continue to run while the Application Flash section is updated, providing true Read-While-Write operation. By combining an 8-bit RISC CPU with In- System Self-Programmable Flash on a monolithic chip, the Atmel ATmega32 is a powerful microcontroller that provides a highly-flexible and cost-effective solution to many embedded control applications. The Atmel AVR ATmega32 is supported with a full suite of program and system development tools including: C compilers, macro assemblers, program debugger/simulators, in-circuit emulators, and evaluation kits. It has four ports- PORTA, PORTB, PORTC and PORTD. PORTA serves as the analog inputs to the A/D Converter. It also serves as an 8-bit bi-directional I/O port, if the A/D Converter is not used. Port pins can provide internal pull- up resistors. When pins PA0 to PA7 are used as inputs and are externally pulled low, they will source current if the internal pull-up resistors are activated. The PORTA pins are tri- stated when a reset condition becomes active, even if the clock is not running. PORTB is an 8-bit bi-directional I/O port with internal pull-up resistors. The PORTB output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, PORTB pins that are externally pulled low will source current if the pull-up resistors are activated. PORTC is an 8-bit bi-directional I/O port with internal pull-up resistors. The PORTC output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, PORTC pins that are externally pulled low will source current if the pull-up resistors are activated. The PORTC pins are tri-stated when a reset condition becomes active, even if the clock is not running. If the JTAG interface is enabled, the pull-up resistors on pins PC5 (TDI), PC3 (TMS) and PC2 (TCK) will be activated even if a reset occurs. PORTD is an 8-bit bi-directional I/O port with internal pull-up resistors (selected for each bit). The PORTD output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, PORTD pins that are externally pulled low will source current if the pull-up resistors are activated. The PORTD pins are tri-stated when a reset condition becomes active, even if the clock is not running.
  • 12. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 12 Dept. Of ECE Mangalam College of Engineering It provides several different interrupt sources. These interrupts and the separate reset vector each have a separate program vector in the program memory space. All interrupts are assigned individual enable bits which must be written logic one together with the Global Interrupt Enable bit in the Status Register in order to enable the interrupt. The External Interrupts are triggered by the INT0, INT1, and INT2 pins. Sleep modes enable the application to shut down unused modules in the MCU, thereby saving power. The AVR provides various sleep modes allowing the user to tailor the power consumption to the application‟s requirements. Reliability Qualification results show that the projected data retention failure rate is much less than 1 PPM over 20 years at 85°C or 100 years at 25°C. 4.1.2 ACCELEROMETER MODULE The accelerometer is a device that can measure the force of acceleration, whether caused by gravity or by movement. An accelerometer can therefore measure the speed of movement of an object it is attached to. Accelerometers are one of the most frequently utilized physical sensors for detecting and measuring motion. Accelerometers have found applications ranging from measurement and control to inertial navigation. MEMS implementations of accelerometers have found a large commercial market in automotive airbag deployment systems. The basic configuration of an accelerometer is the same for all of these applications and is shown in Figure4.1. Three items are the basic components of an accelerometer: • Inertial mass • Suspension • Sensing element The suspension supporting the inertial mass will deflect under acceleration due to D‟Alembert‟s principle. A sensing element will transduce the deflection of the suspension to an electrical signal. The transduction can be accomplished by a number of means, but the most common utilize piezoresistive, piezoelectric, or capacitance means [3].
  • 13. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 13 Dept. Of ECE Mangalam College of Engineering Fig.4.1 Accelerometer components [3] ADXL335 accelerometer module is being used in this paper. It is a small, thin, low power, complete 3-axis accelerometer with signal conditioned voltage outputs. The product measures acceleration with a minimum full-scale range of ±3 g. It can measure the static acceleration of gravity in tilt-sensing applications, as well as dynamic acceleration resulting from motion, shock, or vibration. The user selects the bandwidth of the accelerometer using the CX, CY, and CZ capacitors at the XOUT, YOUT, and ZOUT pins. Bandwidths can be selected to suit the application, with a range of 0.5 Hz to 1600 Hz for the X and Y axes, and a range of 0.5 Hz to 550 Hz for the Z axis. The ADXL335 is available in a small, low profile, 4 mm × 4 mm × 1.45 mm, 16-lead, plastic lead frame chip scale package. It contains a polysilicon surface micro-machined sensor and signal conditioning circuitry to implement open-loop acceleration measurement architecture. The output signals are analog voltages that are proportional to acceleration. The accelerometer can measure the static acceleration of gravity in tilt-sensing applications as well as dynamic acceleration resulting from motion, shock, or vibration. The sensor is a polysilicon surface- micromachined structure built on top of a silicon wafer. Polysilicon springs suspend the structure over the surface of the wafer and provide a resistance against acceleration forces. Deflection of the structure is measured using a differential capacitor that consists of independent fixed plates and plates attached to the moving mass. The fixed plates are driven by 180° out-of-phase square waves. Acceleration deflects the moving mass and unbalances the differential capacitor resulting in a sensor output whose amplitude is proportional to
  • 14. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 14 Dept. Of ECE Mangalam College of Engineering acceleration. Phase-sensitive demodulation techniques are then used to determine the magnitude and direction of the acceleration. Rather than using additional temperature compensation circuitry, innovative design techniques ensure that high performance is built in to the ADXL335. As a result, there is no quantization error or non-monotonic behavior, and temperature hysteresis is very low (typically less than 3 mg over the −25°C to +70°C temperature range). The ADXL335 uses a single structure for sensing the X, Y, and Z axes. As a result, the three axes‟ sense directions are highly orthogonal and have little cross-axis sensitivity. Mechanical misalignment of the sensor die to the package is the chief source of cross-axis sensitivity. Capacitance measurement is one of the most versatile methods of position measurement. The parallel-plate capacitor can vary either with vertical motion of a movable plate, modifying the gap, or by transverse motion of one plate relative to another, modifying the effective area of the capacitor. Figure 4.2shows the arrangement of a parallel plate capacitor. Fig. 4.2 Parallel plate capacitor [2] Motion of the moveable component in the indicated direction increases one capacitance and decreases the other. A variant of the parallel-plate differential capacitor would have the middle and lower plate fixed and only the upper plate moveable. In this configuration, motion of the upper plate modifies one capacitor while the other remains constant.
  • 15. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 15 Dept. Of ECE Mangalam College of Engineering 4.1.3 POWER SUPPLY AND VOLTAGE REGULATOR A power supply is a device that supplies electric power to an electrical load. 230V, 50Hz ac mains supply is made to step down to 12 V by means of a step down transformer. The 12 V ac is made to dc with the aid of a bridge rectifier. Bridge rectifier makes use of four diodes in a bridge arrangement to achieve full-wave rectification. W10 bridge rectifier used here has high case dielectric strength, high surge current capability, high reliability, low reverse current, low forward voltage drop and ideal for printed circuit board. It also has Reliable low cost construction utilizing moulded plastic technique. Fig 4.3 Voltage Regulator IC A regulated power supply is one that controls the output voltage or current to a specific value. The controlled value is held nearly constant despite variations in either load current or the voltage supplied by the power supply's energy source. 7805 is a voltage regulator integrated circuit. It is a member of 78xx series of fixed linear voltage regulator ICs. The voltage source in a circuit may have fluctuations and would not give the fixed voltage output. The voltage regulator IC maintains the output voltage at a constant value. The xx in 78xx indicates the fixed output voltage it is designed to provide. 7805 provides +5V regulated power supply. Capacitors of suitable values can be connected at input and output pins depending upon the respective voltage levels. ST 7805 is the voltage regulator used. It is a 3-terminal 1 A positive voltage regulator with Thermal Overload Short Circuit and Output Transistor Safe Operating Area Protection. Figure 4.3shows a voltage regulator IC. 4.1.4 DISPLAY UNIT An LCD is a small low cost display. A 16 x 2 LCD display is very basic module and is very commonly used in various devices and circuits. These modules are preferred over
  • 16. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 16 Dept. Of ECE Mangalam College of Engineering seven segments and other multi segment LEDs. LCDs are economical, easily programmable and have no limitation of displaying special & even custom characters. It is easy to interface with a micro-controller. A 16 x 2 LCD means it can display 16 characters per line and there are 2 such lines. In this LCD each character is displayed in 5x7 pixel matrix. Table 4.1LCD Commands Command Operation 0x01 LCD Clear 0x38 Select 5 x 7 matrix 0x80 to move cursor to first line first position 0x81 to move cursor to first line second position 0x8F to move cursor to first line last position 0xC0 display on cursor on 0x0E display on cursor off 0x0C to move cursor to second line first position 0x06 shift cursor to right 0x1C shift entire display to right by one position 0x18 shift entire display to left by one position This LCD has two registers, namely, Command and Data. The command register stores the command instructions given to the LCD. A command is an instruction given to LCD to do a predefined task like initializing it, clearing its screen, setting the cursor position, controlling display etc. The data register stores the data to be displayed on the LCD. The data is the ASCII value of the character to be displayed on the LCD. The display is capable of displaying 224 different characters and symbols. It is also facilitated with backlight Led. Table 4.1shows some commonly used LCD commands.
  • 17. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 17 Dept. Of ECE Mangalam College of Engineering 4.1.5 MAX232 The MAX232 is an IC, first created by Maxim Integrated Products, that converts signals from an RS-232 serial port to signals suitable for use in TTL compatible digital logic circuits. The MAX232 is a dual driver/receiver and typically converts the RX, TX, CTS and RTS signals. The drivers provide RS-232 voltage level outputs (approx. ± 7.5 V) from a single + 5 V supply via on-chip charge pumps and external capacitors. This makes it useful for implementing RS-232 in devices that otherwise do not need any voltages outside the 0 V to + 5 V range, as power supply design does not need to be made more complicated just for driving the RS-232 in this case. The receivers reduce RS-232 inputs (which may be as high as ± 25 V), to standard 5 V TTL levels. These receivers have a typical threshold of 1.3 V, and a typical hysteresis of 0.5 V. When a MAX232 IC receives a TTL level to convert, it changes a TTL Logic 0 to between +3 and +15 V, and changes TTL Logic 1 to between -3 to -15 V, and vice versa for converting from RS232 to TTL. This can be confusing on realizing that the RS232 Data Transmission voltages at a certain logic state are opposite from the RS232 Control Line voltages at the same logic state. Table 4.2 shows the voltage levels. Table 4.2 Voltage Levels RS232 Line Type & Logic Level RS232 Voltage TTL Voltage to/from MAX232 Data Transmission (Rx/Tx) Logic 0 +3 V to +15 V Data Transmission (Rx/Tx) Logic 1 -3 V to -15 V Control Signals (RTS/CTS/DTR/DSR) Logic 0 -3 V to -15V Control Signals (RTS/CTS/DTR/DSR) Logic 1 +3 V to+15V 4.1.6 OTHER DEVICES The other output devices interfaced to this are a light and a fan.
  • 18. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 18 Dept. Of ECE Mangalam College of Engineering 4.2 SOFTWARE REQUIREMENTS 4.2.1 AVR STUDIO 5 The software used is AVR Studio 5. It is a Windows based Integrated Development Environment (IDE) from the Atmel Technology incorporated AVR microcontroller families. AVR Studio 5 supports all AVR microcontrollers. The various features of this software are • Capability to work in C programming Environment. • Creates source code using the built in editor. • Compile and link source code using various tools. • A compiler, linker and library come with AVR Studio. AVR Studio 5 includes a compiler, assembler and a simulator, and interfaces seamlessly with in-system debuggers and programmers to make code development easier. It is a full software development environment with an editor, simulator, programmer, etc. The AVR Software Framework is a collection of production-ready source code, written and optimized by experts and tested in hundreds of production designs. It comes with its own integrated C compiler the AVR GNU C Compiler (GCC). As such you do not need a third party C compiler. It provides a single environment to develop programs for both the 8-bits and 32-bits AVR series of microcontrollers. Using these peripheral drivers, communication stacks and application-specific libraries is the quick and effortless way to complete a project. The AVR Studio 5 editor simplifies code editing and facilitates coding more efficiently. Type a few letters of a symbol, and AVR Studio 5 will display a list of suggestions. Figure 4.4 to 4.9 shows the various stages of programming in AVR Studio 5. Steps to program a code in AVR Studio 5 is as follows: 1. Click open AVR Studio 5. 2. Select New Project
  • 19. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 19 Dept. Of ECE Mangalam College of Engineering Fig.4.4Opening window AVR Studio 5 3. Select C Executable Project Fig.4.5Project Select
  • 20. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 20 Dept. Of ECE Mangalam College of Engineering 4. Name the project (here named as DIGITALPEN) and click ok. Fig4.6Name Project 5. Select the device as ATmega32 and click ok Fig4.7Device selection
  • 21. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 21 Dept. Of ECE Mangalam College of Engineering 6. Enter the code. The programming is done in Embedded C language. Fig.4.8 Programming window 7. From tool bar select Build and from its drop down window select build solution Fig.4.9Build Solution 8. The output window shows errors and warnings in case of wrong entries, missing variables or symbols. On successful completion it displays build succeeded. 9. This code needs to be loaded to the microcontroller for its operation.
  • 22. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 22 Dept. Of ECE Mangalam College of Engineering 4.2.2 KHAZAMA AVR PROGRAMMER The tool used to program the ATmega32 AVR microcontroller is the Khazama AVR Programmer. Programming using this tool is very simple, fast and reliable. Khazama can be used to burn the code into all microcontrollers in AVR family. The steps followed in programming microcontroller are depicted in Figure 4.10 to Figure 4.12. 1. Connect the microcontroller to the PC. 2. Click open Khazama AVR programmer Fig.4.10Opening Window 3. Select the AVR from the dropdown window. Here the microcontroller used is ATMEGA32. Fig.4.11AVR Selection
  • 23. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 23 Dept. Of ECE Mangalam College of Engineering 4. Click command and read chip signature from the drop down window. Fig.4.12Reading Chip Signature 5. Select icon and Load flash to Hex buffer. Click Auto Program. The hex code will be burned on to the AVR microcontroller and the window shows a successfully completed message when the burning is complete.
  • 24. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 24 Dept. Of ECE Mangalam College of Engineering 5. SIMULATION RESULTS The programming language used in this project is Embedded C. The code was written and simulated in AVR Studio 5. Figure 5.1 shows the window showing the successful compilation of the code. Fig. 5.1 Simulation Result The code was loaded to the microcontroller. The device successfully recognized digits zero to seven. The digits were displayed on the LCD. The four gestures used in this project turned on and off the light and the fan.
  • 25. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 25 Dept. Of ECE Mangalam College of Engineering 6. CONCLUSION This project has presented an accelerometer based framework that can that can be utilized for acceleration-based handwriting and gesture recognition. The proposed system is utilized for digit and gesture recognition and thus it is an effective tool for HCI applications. Users can use the pen to write digits or make hand gestures as shown in Table 3.1 and 3.2, and the accelerations of hand motions are measured by the accelerometer. The recognized digits are displayed on a computer for further applications. So, by changing the position of accelerometer users can show the numerical characters on the LCD. Gesture recognition is useful for processing information from humans which is not conveyed through speech or type. As well, there are various types of gestures which can be identified by computers.The system finds its application in following fields. It acts as a communication tool between deaf, dumb and the blind. It has other human machine applications in the field of game control, titlt recognition, tv remote, mobile text scrolling, mobile application selection, presentation pointer etc.
  • 26. Most Probable Longest Common Subsequence for Recognition of Gesture Character Input 26 Dept. Of ECE Mangalam College of Engineering 7. REFERENCES [1] J.C.Wang, and F.C Chuang, „‟An Accelerometer-Based Digital Pen with a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition‟‟, IEEE Trans. on industrial electronics, vol. 59, no. 7, July 2012 [2] Stephen D Senturia, “Microsystem Design”, Kluwer Academic Publishers, New York. [3] L. L. Faulkner, James J. Allen, „‟Micro Electro mechanical System Design‟‟, Taylor & Francis Group, LLC,2005