This document summarizes a study on facial expression recognition using Local Binary Pattern (LBP) as the feature extraction technique. The study used the Japanese Female Facial Expressions (JAFFE) database to test LBP and Support Vector Machine (SVM) classification on seven basic facial expressions. LBP was shown to extract important facial features and achieved an average recognition accuracy of 89%, significantly outperforming without LBP at 67%. Angry, happy, and surprise expressions achieved over 90% accuracy, demonstrating the effectiveness of the LBP-SVM approach for facial expression recognition.
A study of techniques for facial detection and expression classificationIJCSES Journal
Automatic recognition of facial expressions is an important component for human-machine interfaces. It
has lot of attraction in research area since 1990's.Although humans recognize face without effort or
delay, recognition by a machine is still a challenge. Some of its challenges are highly dynamic in their
orientation, lightening, scale, facial expression and occlusion. Applications are in the fields like user
authentication, person identification, video surveillance, information security, data privacy etc. The
various approaches for facial recognition are categorized into two namely holistic based facial
recognition and feature based facial recognition. Holistic based treat the image data as one entity without
isolating different region in the face where as feature based methods identify certain points on the face
such as eyes, nose and mouth etc. In this paper, facial expression recognition is analyzed with various
methods of facial detection,facial feature extraction and classification.
Facial expression identification by using features of salient facial landmarkseSAT Journals
Abstract
Facial expression recognition/identification (FER) systems plays vital role in the field of biometrics. Localizing the facial components accurately is a challenging task in image analysis and computer vision. Accurate detection of face and facial components gives effective performance with classification of expressions. This paper proposes feature based facial recognition system using JAFFE and CK databases. 18 facial landmarks were located using Haar cascade classifier. The distances between 12 points were extracted as features. These features were classified using SVM and K-NN classifier and comparison based on accuracy and execution time is done. The proposed algorithm gives better performance.
Facial expression identification by using features of salient facial landmarkseSAT Journals
Abstract
Facial expression recognition/identification (FER) systems plays vital role in the field of biometrics. Localizing the facial components accurately is a challenging task in image analysis and computer vision. Accurate detection of face and facial components gives effective performance with classification of expressions. This paper proposes feature based facial recognition system using JAFFE and CK databases. 18 facial landmarks were located using Haar cascade classifier. The distances between 12 points were extracted as features. These features were classified using SVM and K-NN classifier and comparison based on accuracy and execution time is done. The proposed algorithm gives better performance.
An Accurate Facial Component Detection Using Gabor FilterjournalBEEI
Face detection is a critical task to be resolved in a variety of applications. Since faces include various expressions it becomes a difficult task to detect the exact output. Face detection not only play a main role in personal identification but also in various fields which includes but not limited to image processing, pattern recognition, graphics and other application areas. The proposed system performs the face detection and facial components using Gabor filter. The results show accurate detection of facial components
Facial Expression Recognition System Based on SVM and HOG TechniquesCSCJournals
Facial expression is one of the most commonly used nonverbal means by humans to transmit internal emotional states and, therefore, it plays a fundamental role in interpersonal interactions. Although there is a wide range of possible facial expressions, psychologists have identified six fundamental ones (happiness, sadness, surprise, anger, fear and disgust) that are universally recognized. Automatic facial expression recognition (FER) is a topic of growing interest mainly due to the rapid spread of assistive technology applications, as human–robot interaction, where a robust emotional awareness is a key point to best accomplish the assistive task. The proposed work aims to design a robust facial expression recognition system (FER). FER system can be divided into three modules, namely facial registration, feature extraction and classification. The objective of this work is the recognition of facial expressions based the Histogram of Oriented Gradients (HOG) and support vector machine (SVM) algorithm.
A study of techniques for facial detection and expression classificationIJCSES Journal
Automatic recognition of facial expressions is an important component for human-machine interfaces. It
has lot of attraction in research area since 1990's.Although humans recognize face without effort or
delay, recognition by a machine is still a challenge. Some of its challenges are highly dynamic in their
orientation, lightening, scale, facial expression and occlusion. Applications are in the fields like user
authentication, person identification, video surveillance, information security, data privacy etc. The
various approaches for facial recognition are categorized into two namely holistic based facial
recognition and feature based facial recognition. Holistic based treat the image data as one entity without
isolating different region in the face where as feature based methods identify certain points on the face
such as eyes, nose and mouth etc. In this paper, facial expression recognition is analyzed with various
methods of facial detection,facial feature extraction and classification.
Facial expression identification by using features of salient facial landmarkseSAT Journals
Abstract
Facial expression recognition/identification (FER) systems plays vital role in the field of biometrics. Localizing the facial components accurately is a challenging task in image analysis and computer vision. Accurate detection of face and facial components gives effective performance with classification of expressions. This paper proposes feature based facial recognition system using JAFFE and CK databases. 18 facial landmarks were located using Haar cascade classifier. The distances between 12 points were extracted as features. These features were classified using SVM and K-NN classifier and comparison based on accuracy and execution time is done. The proposed algorithm gives better performance.
Facial expression identification by using features of salient facial landmarkseSAT Journals
Abstract
Facial expression recognition/identification (FER) systems plays vital role in the field of biometrics. Localizing the facial components accurately is a challenging task in image analysis and computer vision. Accurate detection of face and facial components gives effective performance with classification of expressions. This paper proposes feature based facial recognition system using JAFFE and CK databases. 18 facial landmarks were located using Haar cascade classifier. The distances between 12 points were extracted as features. These features were classified using SVM and K-NN classifier and comparison based on accuracy and execution time is done. The proposed algorithm gives better performance.
An Accurate Facial Component Detection Using Gabor FilterjournalBEEI
Face detection is a critical task to be resolved in a variety of applications. Since faces include various expressions it becomes a difficult task to detect the exact output. Face detection not only play a main role in personal identification but also in various fields which includes but not limited to image processing, pattern recognition, graphics and other application areas. The proposed system performs the face detection and facial components using Gabor filter. The results show accurate detection of facial components
Facial Expression Recognition System Based on SVM and HOG TechniquesCSCJournals
Facial expression is one of the most commonly used nonverbal means by humans to transmit internal emotional states and, therefore, it plays a fundamental role in interpersonal interactions. Although there is a wide range of possible facial expressions, psychologists have identified six fundamental ones (happiness, sadness, surprise, anger, fear and disgust) that are universally recognized. Automatic facial expression recognition (FER) is a topic of growing interest mainly due to the rapid spread of assistive technology applications, as human–robot interaction, where a robust emotional awareness is a key point to best accomplish the assistive task. The proposed work aims to design a robust facial expression recognition system (FER). FER system can be divided into three modules, namely facial registration, feature extraction and classification. The objective of this work is the recognition of facial expressions based the Histogram of Oriented Gradients (HOG) and support vector machine (SVM) algorithm.
Multi Local Feature Selection Using Genetic Algorithm For Face IdentificationCSCJournals
Face recognition is a biometric authentication method that has become more significant and relevant in recent years. It is becoming a more mature technology that has been employed in many large scale systems such as Visa Information System, surveillance access control and multimedia search engine. Generally, there are three categories of approaches for recognition, namely global facial feature, local facial feature and hybrid feature. Although the global facial-based feature approach is the most researched area, this approach is still plagued with many difficulties and drawbacks due to factors such as face orientation, illumination, and the presence of foreign objects. This paper presents an improved offline face recognition algorithm based on a multi-local feature selection approach for grayscale images. The approach taken in this work consists of five stages, namely face detection, facial feature (eyes, nose and mouth) extraction, moment generation, facial feature classification and face identification. Subsequently, these stages were applied to 3065 images from three distinct facial databases, namely ORL, Yale and AR. The experimental results obtained have shown that recognition rates of more than 89% have been achieved as compared to other global-based features and local facial-based feature approaches. The results also revealed that the technique is robust and invariant to translation, orientation, and scaling.
Facial Expression Recognition Based on Facial Motion Patternsijeei-iaes
Facial expression is one of the most powerful and direct mediums embedded in human beings to communicate with other individuals’ feelings and abilities. In recent years, many surveys have been carried on facial expression analysis. With developments in machine vision and artificial intelligence, facial expression recognition is considered a key technique of the developments in computer interaction of mankind and is applied in the natural interaction between human and computer, machine vision and psycho- medical therapy. In this paper, we have developed a new method to recognize facial expressions based on discovering differences of facial expressions, and consequently appointed a unique pattern to each single expression.by analyzing the image by means of a neighboring window on it, this recognition system is locally estimated. The features are extracted as binary local features; and according to changes in points of windows, facial points get a directional motion per each facial expression. Using pointy motion of all facial expressions and stablishing a ranking system, we delete additional motion points that decrease and increase, respectively, the ranking size and strenghth. Classification is provided according to the nearest neighbor. In the conclusion of the paper, the results obtained from the experiments on tatal data of Cohn-Kanade demonstrate that our proposed algorithm, compared to previous methods (hierarchical algorithm combined with several features and morphological methods as well as geometrical algorithms), has a better performance and higher reliability.
Facial Expression Recognition Using Local Binary Pattern and Support Vector M...AM Publications
Facial expression analysis is a remarkable and demanding problem, and impacts significant applications in various fields like human-computer interaction and data-driven animation. Developing an efficient facial representation from the original face images is a crucial step for achieving facial expression recognition. Facial representation based on statistical local features, Local Binary Patterns (LBP) is practically assessed. Several machine learning techniques were thoroughly observed on various databases. LBP features- which are effectual and competent for facial expression recognition are generally used by researchers Cohn Kanade is the database for present work and the programming language used is MATLAB. Firstly, face area is divided in small regions, by which histograms, Local Binary Patterns (LBP) are extracted and then concatenated into single feature vector. This feature vector outlines a well-organized representation of face and is helpful in determining the resemblance among images.
Facial Expression Recognition Using Local Binary Pattern and Support Vector M...AM Publications
Facial expression analysis is a remarkable and demanding problem, and impacts significant applications in various fields like human-computer interaction and data-driven animation. Developing an efficient facial representation from the original face images is a crucial step for achieving facial expression recognition. Facial representation based on statistical local features, Local Binary Patterns (LBP) is practically assessed. Several machine learning techniques were thoroughly observed on various databases. LBP features- which are effectual and competent for facial expression recognition are generally used by researchers Cohn Kanade is the database for present work and the programming language used is MATLAB. Firstly, face area is divided in small regions, by which histograms, Local Binary Patterns (LBP) are extracted and then concatenated into single feature vector. This feature vector outlines a well-organized representation of face and is helpful in determining the resemblance among images.
Pose and Illumination in Face Recognition Using Enhanced Gabor LBP & PCA IJMER
This paper presents the face recognition based on Enhanced GABOR LBP and PCA. Some
of the challenges in face recognition are occlusion, pose and illumination .In this paper, we are more
focused on varying pose and illumination. We divided this algorithm into five stages. First stage finds
the fiducial points on face using Gabor filter bank as this filter is well known for illumination
compensation. Second stage applies the morphological techniques for reduce useless fiducial points.
Third stage applies the LBP on reduced fiducial points with neighborhood pixel for improving the pose
variation. Forth stage uses PCA to detect the best variance points which are necessary to characterize
the training images. The last recognition stage includes finding the Euclidean norm of the feature
weight vectors with the test weight vector. In this project, we used 20 images of 20 different persons
from ORL database for training. For testing, we used images with varying illumination, pose and
occluded images of the same training
Fiducial Point Location Algorithm for Automatic Facial Expression Recognitionijtsrd
We present an algorithm for the automatic recognition of facial features for color images of either frontal or rotated human faces. The algorithm first identifies the sub images containing each feature, afterwards, it processes them separately to extract the characteristic fiducial points. Then Calculate the Euclidean distances between the center of gravity coordinate and the annotated fiducial points coordinates of the face image. A system that performs these operations accurately and in real time would form a big step in achieving a human like interaction between man and machine. This paper surveys the past work in solving these problems. The features are looked for in down sampled images, the fiducial points are identified in the high resolution ones. Experiments indicate that our proposed method can obtain good classification accuracy. D. Malathi | A. Mathangopi | Dr. D. Rajinigirinath ""Fiducial Point Location Algorithm for Automatic Facial Expression Recognition"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-3 , April 2019, URL: https://www.ijtsrd.com/papers/ijtsrd21754.pdf
Paper URL: https://www.ijtsrd.com/computer-science/data-miining/21754/fiducial-point-location-algorithm-for-automatic-facial-expression-recognition/d-malathi
PRE-PROCESSING TECHNIQUES FOR FACIAL EMOTION RECOGNITION SYSTEMIAEME Publication
Humans share a universal and fundamental set of emotions which are exhibited through consistent facial expressions. Recognition of human emotions from the imaging templates is useful in a wide variety of human-computer interaction and intelligent systems applications. However, the automatic recognition of facial expressions using image template matching techniques suffer from the natural variability with facial features and recording conditions. In spite of the progress achieved in facial emotion recognition in recent years, the effective and computationally simple feature extraction and classification technique for emotion recognition is still an open problem. Image pre-processing and normalization is significant part of face recognition systems. Changes in lighting conditions produces dramatically decrease of recognition performance. In this paper, the image pre-processing techniques like K-Nearest Neighbor, Cultural Algorithm and Genetic Algorithm are used to remove the noise in the facial image for enhancing the emotion recognition. The performance of the preprocessing techniques are evaluated with various performance metrics.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
IJERA (International journal of Engineering Research and Applications) is International online, ... peer reviewed journal. For more detail or submit your article, please visit www.ijera.com
FACIAL EXPRESSION RECOGNITION USING DIGITALISED FACIAL FEATURES BASED ON ACTI...csandit
Facial Expression Recognition is a hot topic in recent years. As artificial intelligent technology is growing rapidly, to communicate with machines, facial expression recognition is essential.The recent feature extraction methods for facial expression recognition are similar to face
recognition, and those caused heavy load for calculation. In this paper, Digitalized Facial Features based on Active Shape Model method is used to reduce the computational complexity
and extract the most useful information from the facial image. The result shows by using this
method the computational complexity is dramatically reduced, and very good performance was obtained compared with other extraction methods.
Multi Local Feature Selection Using Genetic Algorithm For Face IdentificationCSCJournals
Face recognition is a biometric authentication method that has become more significant and relevant in recent years. It is becoming a more mature technology that has been employed in many large scale systems such as Visa Information System, surveillance access control and multimedia search engine. Generally, there are three categories of approaches for recognition, namely global facial feature, local facial feature and hybrid feature. Although the global facial-based feature approach is the most researched area, this approach is still plagued with many difficulties and drawbacks due to factors such as face orientation, illumination, and the presence of foreign objects. This paper presents an improved offline face recognition algorithm based on a multi-local feature selection approach for grayscale images. The approach taken in this work consists of five stages, namely face detection, facial feature (eyes, nose and mouth) extraction, moment generation, facial feature classification and face identification. Subsequently, these stages were applied to 3065 images from three distinct facial databases, namely ORL, Yale and AR. The experimental results obtained have shown that recognition rates of more than 89% have been achieved as compared to other global-based features and local facial-based feature approaches. The results also revealed that the technique is robust and invariant to translation, orientation, and scaling.
Facial Expression Recognition Based on Facial Motion Patternsijeei-iaes
Facial expression is one of the most powerful and direct mediums embedded in human beings to communicate with other individuals’ feelings and abilities. In recent years, many surveys have been carried on facial expression analysis. With developments in machine vision and artificial intelligence, facial expression recognition is considered a key technique of the developments in computer interaction of mankind and is applied in the natural interaction between human and computer, machine vision and psycho- medical therapy. In this paper, we have developed a new method to recognize facial expressions based on discovering differences of facial expressions, and consequently appointed a unique pattern to each single expression.by analyzing the image by means of a neighboring window on it, this recognition system is locally estimated. The features are extracted as binary local features; and according to changes in points of windows, facial points get a directional motion per each facial expression. Using pointy motion of all facial expressions and stablishing a ranking system, we delete additional motion points that decrease and increase, respectively, the ranking size and strenghth. Classification is provided according to the nearest neighbor. In the conclusion of the paper, the results obtained from the experiments on tatal data of Cohn-Kanade demonstrate that our proposed algorithm, compared to previous methods (hierarchical algorithm combined with several features and morphological methods as well as geometrical algorithms), has a better performance and higher reliability.
Facial Expression Recognition Using Local Binary Pattern and Support Vector M...AM Publications
Facial expression analysis is a remarkable and demanding problem, and impacts significant applications in various fields like human-computer interaction and data-driven animation. Developing an efficient facial representation from the original face images is a crucial step for achieving facial expression recognition. Facial representation based on statistical local features, Local Binary Patterns (LBP) is practically assessed. Several machine learning techniques were thoroughly observed on various databases. LBP features- which are effectual and competent for facial expression recognition are generally used by researchers Cohn Kanade is the database for present work and the programming language used is MATLAB. Firstly, face area is divided in small regions, by which histograms, Local Binary Patterns (LBP) are extracted and then concatenated into single feature vector. This feature vector outlines a well-organized representation of face and is helpful in determining the resemblance among images.
Facial Expression Recognition Using Local Binary Pattern and Support Vector M...AM Publications
Facial expression analysis is a remarkable and demanding problem, and impacts significant applications in various fields like human-computer interaction and data-driven animation. Developing an efficient facial representation from the original face images is a crucial step for achieving facial expression recognition. Facial representation based on statistical local features, Local Binary Patterns (LBP) is practically assessed. Several machine learning techniques were thoroughly observed on various databases. LBP features- which are effectual and competent for facial expression recognition are generally used by researchers Cohn Kanade is the database for present work and the programming language used is MATLAB. Firstly, face area is divided in small regions, by which histograms, Local Binary Patterns (LBP) are extracted and then concatenated into single feature vector. This feature vector outlines a well-organized representation of face and is helpful in determining the resemblance among images.
Pose and Illumination in Face Recognition Using Enhanced Gabor LBP & PCA IJMER
This paper presents the face recognition based on Enhanced GABOR LBP and PCA. Some
of the challenges in face recognition are occlusion, pose and illumination .In this paper, we are more
focused on varying pose and illumination. We divided this algorithm into five stages. First stage finds
the fiducial points on face using Gabor filter bank as this filter is well known for illumination
compensation. Second stage applies the morphological techniques for reduce useless fiducial points.
Third stage applies the LBP on reduced fiducial points with neighborhood pixel for improving the pose
variation. Forth stage uses PCA to detect the best variance points which are necessary to characterize
the training images. The last recognition stage includes finding the Euclidean norm of the feature
weight vectors with the test weight vector. In this project, we used 20 images of 20 different persons
from ORL database for training. For testing, we used images with varying illumination, pose and
occluded images of the same training
Fiducial Point Location Algorithm for Automatic Facial Expression Recognitionijtsrd
We present an algorithm for the automatic recognition of facial features for color images of either frontal or rotated human faces. The algorithm first identifies the sub images containing each feature, afterwards, it processes them separately to extract the characteristic fiducial points. Then Calculate the Euclidean distances between the center of gravity coordinate and the annotated fiducial points coordinates of the face image. A system that performs these operations accurately and in real time would form a big step in achieving a human like interaction between man and machine. This paper surveys the past work in solving these problems. The features are looked for in down sampled images, the fiducial points are identified in the high resolution ones. Experiments indicate that our proposed method can obtain good classification accuracy. D. Malathi | A. Mathangopi | Dr. D. Rajinigirinath ""Fiducial Point Location Algorithm for Automatic Facial Expression Recognition"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-3 , April 2019, URL: https://www.ijtsrd.com/papers/ijtsrd21754.pdf
Paper URL: https://www.ijtsrd.com/computer-science/data-miining/21754/fiducial-point-location-algorithm-for-automatic-facial-expression-recognition/d-malathi
PRE-PROCESSING TECHNIQUES FOR FACIAL EMOTION RECOGNITION SYSTEMIAEME Publication
Humans share a universal and fundamental set of emotions which are exhibited through consistent facial expressions. Recognition of human emotions from the imaging templates is useful in a wide variety of human-computer interaction and intelligent systems applications. However, the automatic recognition of facial expressions using image template matching techniques suffer from the natural variability with facial features and recording conditions. In spite of the progress achieved in facial emotion recognition in recent years, the effective and computationally simple feature extraction and classification technique for emotion recognition is still an open problem. Image pre-processing and normalization is significant part of face recognition systems. Changes in lighting conditions produces dramatically decrease of recognition performance. In this paper, the image pre-processing techniques like K-Nearest Neighbor, Cultural Algorithm and Genetic Algorithm are used to remove the noise in the facial image for enhancing the emotion recognition. The performance of the preprocessing techniques are evaluated with various performance metrics.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
IJERA (International journal of Engineering Research and Applications) is International online, ... peer reviewed journal. For more detail or submit your article, please visit www.ijera.com
FACIAL EXPRESSION RECOGNITION USING DIGITALISED FACIAL FEATURES BASED ON ACTI...csandit
Facial Expression Recognition is a hot topic in recent years. As artificial intelligent technology is growing rapidly, to communicate with machines, facial expression recognition is essential.The recent feature extraction methods for facial expression recognition are similar to face
recognition, and those caused heavy load for calculation. In this paper, Digitalized Facial Features based on Active Shape Model method is used to reduce the computational complexity
and extract the most useful information from the facial image. The result shows by using this
method the computational complexity is dramatically reduced, and very good performance was obtained compared with other extraction methods.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Dr. Vinod Kumar Kanvaria
Exploiting Artificial Intelligence for Empowering Researchers and Faculty,
International FDP on Fundamentals of Research in Social Sciences
at Integral University, Lucknow, 06.06.2024
By Dr. Vinod Kumar Kanvaria
Thinking of getting a dog? Be aware that breeds like Pit Bulls, Rottweilers, and German Shepherds can be loyal and dangerous. Proper training and socialization are crucial to preventing aggressive behaviors. Ensure safety by understanding their needs and always supervising interactions. Stay safe, and enjoy your furry friends!
Executive Directors Chat Leveraging AI for Diversity, Equity, and InclusionTechSoup
Let’s explore the intersection of technology and equity in the final session of our DEI series. Discover how AI tools, like ChatGPT, can be used to support and enhance your nonprofit's DEI initiatives. Participants will gain insights into practical AI applications and get tips for leveraging technology to advance their DEI goals.
This presentation includes basic of PCOS their pathology and treatment and also Ayurveda correlation of PCOS and Ayurvedic line of treatment mentioned in classics.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
A Study On Facial Expression Recognition Using Local Binary Pattern
1. Vol.7 (2017) No. 5
ISSN: 2088-5334
A Study on Facial Expression Recognition Using Local Binary Pattern
Shahreen Kasim#
, Rohayanti Hassan*
, Nur Hadiana Zaini*
, Asraful Syifaa’ Ahmad*
, Azizul Azhar Ramli#
,
Rd Rohmat Saedudin+
#
Soft Computing and Data Mining Centre, Faculty of Computer Science and Information Technology,
Universiti Tun Hussein Onn, Johor, Malaysia
*
Software Engineering Research Group, Faculty of Computing, Universiti Teknologi Malaysia, 81310 Skudai, Johor, Malaysia
E-mail: rohayanti@utm.my
+
School of Industrial Engineering, Telkom University, 40257 Bandung, West Java, Indonesia
Abstract— How to get the proper combination of feature extraction and classification is still crucial in facial expression recognition,
and it has been addressed conducted over two decades. Hence, if inadequate features are used, even the best classifier could fail to
achieve the accurate recognition. Therefore, Local Binary Pattern (LBP) is used as a feature extraction technique for facial
expressions recognition where it is evaluated based on statistical local features. LBP is proven successful technique by the recent study
due to its speed and discrimination performance aside of robust to low-resolution images. For the classification, Support Vector
Machine is chosen, and the algorithm is implemented in MATLAB and tested on JAFFE (Japanese Female Facial Expressions)
database in order to achieve the objectives and the goal of this research which is to obtain high accuracy in facial expressions and
identify the seven basic facial expressions. The performance of feature extraction and classification is evaluated based on the
recognition accuracy. The observation on results obtained in facial expressions recognition rate indicated the effectiveness of the
proposed algorithm based on SVM-LBP features.
Keywords— facial expression recognition; feature extraction; local binary pattern; support vector machine; JAFFE
I. INTRODUCTION
Biometric is not foreign things in the technology of
computer science which has been adapted for various
applications. Basically, biometric is an authentication of
behavioural characteristics and physical of individuals as a
form of access control or identification. The well-known
biometric authentication such iris recognition, fingerprint,
DNA, palm print, hand geometry, face and voice recognition
has become widespread in world applications, yet there are
still have challenges to overcome. As compared to
fingerprints and iris, face recognition has diverse advantages
as it non-contact process. A numerous study has been
conducted over two decades to address the problem of face
recognition, especially in facial expression. Even though a
lot of approaches had been discussed, expression recognition
is difficult task to achieve the optimal pre-processing,
feature extraction or selection and classification under a
certain condition.
Nowadays, a study on the combination of face
representation and classification is crucial in facial
expression recognition. Moreover, the best classifier could
fail to obtain accurate recognition if inadequate features are
used. Numerous studies have been made on recognizing
facial expression with a high accuracy yet remains difficult
due to subtlety, complexity, and variability of facial
expressions. Furthermore, low-resolution images in real-
world environments make real-life expression recognition
much more difficult [1].
It is necessary to extract important facial features for
classifying facial expressions into variance categories which
contribute in identifying proper and expression. Facial basic
expressions, for example, are sad, happy, disgust, fear,
surprise, angry and neutral. Those particular facial
expressions of emotions are termed as universal emotion by
Ekman [2] hence over a decade; other researchers have used
a similar method in their research [3], [4], [5]. Local Binary
Pattern (LBP) is proposed in this paper which originally for
texture analysis and recently have been introduced to
represent faces in facial images analysis. Zhang [5] has
proposed an improved approach for facial expression
analysis where LBP histogram of different block sizes of a
face image is used as feature vectors and then various facial
expressions were classified using Principal Component
Analysis (PCA). Lajevardi et al. [6] have used LBP in their
1621
2. study that focused on facial expressions recognition issues in
variance resolutions with six basic facial expressions.
Feature extraction technique is a key significance to the
whole classification process [17], [18]. The process of
feature extraction yields a prohibitively large number of
features, and subsequently, a smaller subset of features needs
to be selected according to some optimality criteria. The
prominent characteristics on the faces must be extracted in
order to recognize and identify human faces features like
eyes, nose, and mouth by applying the geometry distribution
and the shape of the face. There are two features used for
extraction which is global and local features. Global features
are often used as coarse representation and encoding the
characteristics of the whole face whereas local features are
often used as finer representation and encoding detailed
variations within local areas in the face [7].
LBP has been found to be a powerful feature for texture
classification which is a type of visual descriptor used for
classification in computer vision. LBP is known as a method
that combining multiple operators for multiresolution
analysis. Also, joint distributions in LBP with orthogonal
measures is a very powerful tool for rotation invariant
texture analysis. As proposed by Ahonen [12], LBP is
presented as a novel and efficient facial image representation
based on texture features where the face image is divided
into several regions from which the LBP feature
distributions are extracted and concatenated into an
enhanced feature vector to be used as a face descriptor.
Furthermore, a lot of research on facial expression
recognition had been experimented using LBP technique as
LBP showing delightful results [9], [1].
Besides that, Rouhi et al. [8] have tested and analysed
several other feature extraction methods namely Gabor Filter,
and Elastic Bunch Graph Matching (EBGM). Rouhi et al. [8]
found that 15-Gabor filter with fuzzy filter leads to a high
rate of the feature extraction in the face recognition. Apart
from that, multiple Gabor filter achieved high recognition
rate and robust to facial expression variations [11] whereby
Gabor filter able to reduce the space complexity of the
system and reveal both transient and intransient facial
features. However, Ou et al. [10] had applied Gabor filter
and claimed that the Gabor filter is only showing an average
result.
Meanwhile, EBGM is an extension to elastic graph
matching for object classes with a common structure, such as
faces in an identical pose. Tiwari [13] found that EBGM
performed better than PCA which obtained highest accuracy
rate of face recognition, but somehow it required long
computational time. In other works, Senaratne et al. [14] has
hybridized the EBGM with Particle Swarm Optimization
and found it was more efficiency in face recognition.
This study has been motivated to analyze the seven basic
facial expressions of emotion which are angry, sad, happy,
fear, disgust, surprise and neutral by using Local Binary
Pattern (LBP) as the feature extraction technique to obtain
high accuracy recognition.
The remainder of this paper is organized as follows. In
Section II, the material and methods used are discussed in
detail. Section III presents the analysis and the discussion of
the experimental results. Finally, Section IV summarizes the
future work and conclusion.
II. MATERIAL AND METHOD
Based on Fig. 1, there are several stages involved in facial
expression recognition which are data acquisition and
preparation, image pre-processing, feature extraction
technique and as well as the feature classification technique.
In this study, Japanese Female Facial Expressions database
(JAFFE) was used as a dataset. This database can be
downloaded from http://www.kasrl.org/jaffe_info.html.
(a)
(b)
Fig. 1 The process of facial expression recognition with example; (a)
flowchart of facial expression recognition, (b) example of facial expression
recognition process
1622
3. A. Data Acquisition and Preparation
The JAFFE database contains 213 images of seven facial
expressions: sad, angry, happy, fear, disgust, surprise and
neutral. Ten models are posed three or four times of seven
basic expressions. The sample of JAFFE database is shown
in Fig. 2. There were seven facial expressions were selected,
and 150 facial expression images sample were used in this
study. The size of all pure facial images cut out from JAFFE
database was normalized to 256 × 256 pixels.
a) Angry
b) Happy
c) Neutral
d) Surprise
e) Sad
f) Disgust
g) Fear
Fig. 2 Seven facial expression of JAFFE database; (a) Angry; (b) Happy; (c)
Neutral; (d) Surprise; (e) Sad; (f) Disgust; (g) Fear
B. Image Processing
The image pre-processing procedure is very vital in the
facial expressions recognition task. The aim of this phase is
to retrieve sequences of images which have normalized
intensity that was uniform in size and shape and depict only
the face region. Thus, the effects of illumination and lighting
should be eliminated in this stage as well as the noise and the
normalization from the variation of pixel position [12]. In
the pre-processing, the image will be cropped to eliminate
the unnecessary part in the image and focused only on the
interested region only as illustrated in Fig. 3.
Fig. 3 The example of image processing
C. Feature Extraction Technique Using LBP
An LBP description of a pixel is developed by
thresholding the values of the matrix size 3 × 3 and result as
in binary number. Based on the operator, LBP code is
labeled for each pixel of an image. According to Feng et al.
[19], we used texture descriptor as illustrated in Fig. 4, with
256-bin histogram of the labels, contains the density of each
label.
Fig. 4 The basic LBP operator
LBP method is implemented on images of the face in
order to extract features which can be used to get a measure
of the similarity between these images. Fig. 5 presents the
process of LBP. Firstly, the face images were divided into
several blocks. After that, the LBP histogram was calculated
for each block. Then, the block LBP histograms were
concatenated into a single vector. Histograms were then
being compared between the images by measuring the
distance similarity between the histograms. Moreover, every
bin in histograms contains the number of its appearance in
the region. Lastly, the feature vector was constructed by
concatenating the regional histograms to one big histogram
after every pixel is calculated [12].
D. Support Vector Machine (SVM)
SVM was then implemented as a classifier to distinguish
the facial expression. SVM is a maximal margin hyperplane
classification method which is hyperplane is drawn between
the training vectors that maximizing the distance between
1623
4. the different classes [15]. The experiment was carried out
using grid-search on the hyper-parameters in the 10-fold
cross-validation. K-fold cross-validation has been applied
using all expressions’ images of a person as the validation
data, and the remaining images as the training data. In order
to evaluate the performance of algorithm objectively, k-fold
was done for ten times. Finally, the accuracy of recognition
was calculated in order to determine the performance of LBP.
Fig. 5 The process of LBP
III. RESULTS AND DISCUSSION
Table 1 demonstrates the analysis of facial expressions
using the normal histogram and LBP histogram. The pixel’s
intensity range in fear and angry emotion were very close
because of the mouth region area was similar. The histogram
equalization of happy emotion shows a gradual pattern in
which the pixel’s intensity has no peak pixel compared to the
surprise emotion. The mouth region was detected in a high
dark part area that caused the peak pixel on the histogram.
The Sad emotion of pixel’s intensity showed less dark spot
on the mouth region hence no peak pixel on the histogram
whereas disgust image showed there was three peaks pixel in
every subregion and neutral images showed very low pixel
in the histogram. The pixel’s intensity emotion was different
for every image.
TABLE I
ANALYSIS OF FACIAL EXPRESSIONS IN HISTOGRAM AND LBP HISTOGRAM
a. Analysis of angry facial expression
b. Analysis of disgust facial expression
c. Analysis of fear facial expression
d. analysis of happy facial expression
1624
5. e. Analysis of sad facial expression
f. Analysis on surprise facial expression
g. Analysis of neutral facial expression
Table 2 presents the performance and evaluation on the
accuracy of recognition which using LBP operator and
without using LBP. We can see the recognition accuracy of
seven facial expressions using LBP features achieved the
average accuracy with 89% outperformed the average
accuracy of without using LBP with 67%. This show on how
important the feature extraction technique in achieving better
accuracy of facial expression recognition process. Angry
expressions succeed 100% in recognition of facial
expressions accuracy whereas happy and surprise facial
expressions using LBP achieved 95% accuracy. Sad and
neutral expressions reported 90% accuracy, while fear
expression reported 80% accuracy.
Some of the images experimented in JAFFE database
look alike even though expressions stated is different could
possibly affect the results achieved. The result was proven
that by using LBP as a feature extraction technique gave a
betterment result in recognition accuracy [6]. Furthermore, a
facial expression with using LBP can help to reduce the
computation complexity which this scenario was shown in
Table 1. The LBP histogram demonstrated a shorter pattern
compared to the basic histogram. This led to a better
detection ratio and eventually improved the recognition
accuracy
TABLE II
ACCURACY OF SEVEN BASIC EXPRESSIONS WITH LBP FEATURES AND
WITHOUT LBP FEATURES
IV.CONCLUSIONS
This study had combined local binary pattern as a feature
extraction technique with support vector machine in order to
improve the accuracy of facial expression recognition. Seven
facial expressions from JAFFE database have been used as a
case study. The overall result showed that the average
accuracy when using local binary pattern was 22% better
from recognition process without using local binary pattern.
However, the proposed combination have a limitation on
generalization to other datasets. This limitation can be
addressed in our future work. In future development, this
research could be enhanced by referring to various other
works available such as [21]-[29].
ACKNOWLEDGMENT
This research was funded by Research Contact Grant, Vot
No: 4C907. Also many thanks to collaborative sponsor,
GATES Scholars Foundation of GATES IT Solution Sdn.
Bhd. and to RMC, UTM.
REFERENCES
[1] C. Shan, S. Gong, and P. W. McOwan, “Facial expression
recognition based on Local Binary Patterns: A comprehensive study,”
Image Vis. Comput., vol. 27, no. 6, pp. 803–816, 2009.
[2] P. L. B.-E. Ekman, “Facial expression and emotion,” Am. Psychol.,
vol. 48, no. 4, pp. 384–92, 1993.
[3] C. Chibelushi and F. Bourel, “Facial expression recognition: A brief
tutorial overview,” On-Line Compend. Comput. Vis., 2003.
[4] C. Chan, J. Kittler, and K. Messer, “Multi-scale local binary pattern
histograms for face recognition,” Adv. biometrics, 2007.
[5] H. Zhang, “Image Preprocessing Methods in Face Recognition,” in
2011 Symposium on Photonics and Optoelectronics (SOPO), 2011,
pp. 1–4.
[6] S. Lajevardi and Z. Hussain, “Feature extraction for facial expression
recognition based on hybrid face regions,” Adv. Electr. Comput.,
2009.
[7] Y. Su, S. Shan, X. Chen, and W. Gao, “Hierarchical ensemble of
global and local classifiers for face recognition,” IEEE Trans. Image,
2009.
[8] R. Rouhi, M. Amiri, and B. Irannejad, “A review on feature
extraction techniques in face recognition,” Signal Image Process.,
2012.
[9] A. Hussain, M. S. Khan, M. Nazir, and M. A. Iqbal, “Survey of
various feature extraction and classification techniques for facial
1625
6. expression recognition,” Proc. 11th WSEAS Int. Conf. Electron.
Hardware, Wirel. Opt. Commun. Proc. 11th WSEAS Int. Conf. Signal
Process. Robot. Autom. Proc. 4th WSEA, pp. 138–142, 2012.
[10] J. O. J. Ou, X.-B. B. X.-B. Bai, Y. P. Y. Pei, L. M. L. Ma, and W. L.
W. Liu, “Automatic Facial Expression Recognition Using Gabor
Filter and Expression Analysis,” Second Int. Conf. Comput. Model.
Simulation, 2010. ICCMS ’10, vol. 2, pp. 215–218, 2010.
[11] Q. Alshebani and P. Premaratne, “A hybrid feature extraction
technique for face recognition,” Proc. Comput. …, 2014.
[12] T. Ahonen, A. Hadid, and M. Pietikäinen, “Face recognition with
local binary patterns,” Comput. vision-eccv 2004, 2004.
[13] M. Tiwari, “Gabor Based Face Recognition Using EBGM and
PCA,” ) Int. J. Comput. Sci. …, 2012.
[14] R. Senaratne, S. Halgamuge, and A. Hsu, “Face recognition by
extending elastic bunch graph matching with particle swarm
optimization.,” J. Multimed., 2009.
[15] M. Dumas, “Emotional expression recognition using support vector
machines,” Proc. Int. Conf., 2001.
[17] S. Yuhandri, E. P. Madenda and W. Karmilasari, “Object Feature
Extraction of Songket Image Using Chain Code Algorithm,”
International Journal on Advanced Science, Engineering and
Information Technology, vol 7, no. 1, pp. 235-241, 2017.
[18] I. E. W. Rachmawan, Y. Kiyoki and S. Sasaki, "An Automatic
Feature Extraction Method of Satellite Multispectral Images for
Interpreting Deforestation Effects in Soil Degradation"
International Journal on Advanced Science, Engineering and
Information Technology, vol 7, no 2, pp. 676-687, 2017.
[19] X. Feng, M. Pietikainen, and A. Hadid, "Facial expression
recognition with local binary patterns and linear programming",
Pattern Recognition And Image Analysis C/C of Raspoznavaniye
Obrazov I Analiz Izobrazhenii, vol 15, no 2, pp. 546, 2005.
[20] S. Singh, A. Kaur, and A. Taqdir, “A Face Recognition Technique
using Local Binary Pattern Method”, International Journal of
Advanced Research in Computer and Communication Engineering,
vol 4, no 3, 2015.
[21] U. K. Hassan, N. M. Nawi, and S. Kasim, “Classify a protein domain
using sigmoid support vector machine,” in Information Science and
Applications (ICISA), 2014 International Conference, IEEE, May.
2014. pp. 1-4.
[22] U. K. Hassan, N. M. Nawi, and S. Kasim, A. A, Ramli, M. F. M.,
Fudzee, and M. A. Salamat,”Classify a Protein Domain Using SVM
Sigmoid Kernel”. In Recent Advances on Soft Computing and Data
Mining, pp. 143-151, Springer, Cham.
[23] S., Ismail, R. M., Othman, S., Kasim, R., Hassan, H., Asmuni, and J.,
Taliba, “Pairwise protein substring alignment with latent semantic
analysis and support vector machines to detect remote protein
homology”, International Journal of Bio-Science and Bio-
Technology, Volume 3, Issue 3, 2011, Pages 17-34.
[24] F. M., Abdullah, R. M., Othman, S., Kasim, R., Hashim, R., Hassan,
H., Asmuni, and J., Taliba, “An Optimal Mesh Algorithm for Remote
Protein Homology Detection”, Communications in Computer and
Information Science, Volume 151 CCIS, Issue PART 2, 2011, Pages
471-497.
[25] S., Ismail, R. M., Othman, S., Kasim, R., Hassan, H., Asmuni, and J.,
Taliba, “Pairwise protein substring alignment with latent semantic
analysis and support vector machines to detect remote protein
homology” Communications in Computer and Information
Science,Volume 151 CCIS, Issue PART 2, 2011, Pages 526-546.
[26] F. M., Abdullah, R. M., Othman, S., Kasim, R., Hashim, R., Hassan,
H., Asmuni, and J., Taliba, “An Optimal Mesh Algorithm for Remote
Protein Homology Detection”, International Journal of Bio-Science
and Bio-Technology, Volume 3, Issue 2, June 2011, Pages 13-38.
[27] S., Kasim, M. F. M., Fudzee, S., Deris, S., and R. M., Othman, “Gene
Function Prediction Using Improved Fuzzy c-Means Algorithm”,
In Information Science and Applications (ICISA), 2014 International
Conference on (pp. 1-4). IEEE.
[28] S., Kasim, S., Deris, S., and R. M., Othman, “Multi-stage filtering for
improving confidence level and determining dominant clusters in
clustering algorithms of gene expression data”, Computers in biology
and medicine, 43(9), 1120-1133, 2013.
[29] S., Kasim, S., Deris, S., and R. M., Othman, “A new computational
framework for gene expression clustering”, In International
Conference on Advanced Data Mining and Applications (pp. 603-
610). Springer, Berlin, Heidelberg, 2010.
1626