Eigenface based recognition of emotion variant faces
AngeloStekardisPoster edit
1. Title of Research Project
Researcher’s Name
Department(s), University of Kentucky, Lexington, KY
1. Sullivan, Vahid Kazemi Josephine. "One Millisecond Face Alignment with an Ensemble of Regression Trees." (n.d.): n. pag. Cv-foundation.org.
Web. Mar.-Apr. 2015.
2. Alabort-i-medina, Joan, Epameinondas Antonakos, and James Booth. "Menpo: A Comprehensive Platform for Parametric Image Alignment and
Visual Deformable Models." (2013): n. pag. Menpo.org. Menpo. Web. Mar.-Apr. 2015.
From the accuracy in which surprise was determined in this project, it can be concluded that Menpo is a
promising option for further research involving emotion detection. Between the versatility of its
implementations and ease of use, some examples of future uses of Menpo include:
• Aiding in emotion detection for children that have difficulties identifying emotions.
• Determining if someone is lying based off of facial cues.
• Scanning faces in a crowd to determine the average feeling towards a speech, sporting event, etc.
The simple foundations for these methods that I studied elucidate the true potential for computer vision’s
impact on modern society.
One example of the facial regions that Menpo has the ability to analyze
Technology that utilizes cameras in order to perform complex analysis on
individuals, groups, and environments has grown to be an area of major interest
over the past few years. It has become evident that computer vision and its
applications have potential to make significant impacts in consumer, security and
military technology. One of the most interesting applications of these technologies
relate to how imaging data from human faces is processed. Depending on the
programming language, there exists different software platforms that allow users to
analyze image data at a higher level. For Python, Menpo is one of the platforms that
specializes in facial point localization. In this work, I studied the ways in which tools
such as Menpo may be used in order to solve simple problems related to facial
feature recognition.
A Demonstration of Landmarks Being Plotted
Example of Program Output
Menpo is a set of Python libraries used for manipulating data that is typically used in
computer vision. Some of the benefits of Menpo are as follows:
• Menpo is one of few high level, easy to use, open source software distributions that can be
used to perform facial recognition.
• Menpo can be used to analyze rigid alignment, not rigid alignment and visuals of deformable
models (human faces, for example).
• Menpo brings many of the modern image processing algorithms together into one, easy to
use framework.
• Menpo enables researchers to study new areas with greater ease.
The author would like to thank Nathan Jocobs for providing constructive guidance in addition to reviewing his code and poster
The process for image analysis almost always begins with machine learning or training. Machine learning
has its origins in pattern recognition, which became increasingly apparent as I progressed with this
research project. Machine learning deals with algorithms that have the ability to learn from and make
predictions on data. Additionally, the two main algorithm families used in detecting the face are detailed in
the following bullet points:
• Lucas-Kanade: Looks for specific differences between existing image models and the target image.
• Constrained Local Methods: A set of methods that locate specific points on a target image.
Once the landmarks were plotted onto the faces used in the program, some simple calculations were
utilized in order to determine how surprised an individual was. In this project, the 18 landmarks that are
plotted on an individuals mouth were used in order to calculate the level of the surprise of the person in
the image. The logic of the code can be broken down into the following steps:
• Detect and plot the landmarks on the face
• Determine the average distance from the x-axis of the 18 points on the mouth, and define this average
distance as the middle of the mouth.
• Find the average distance of the points on the upper lip from the middle point. Repeat for the bottom
lip.
• Determine the level of surprise based off of the upper and lower lip distances
In reality, emotions such as surprise cannot be measured by data gathered from one area on the face, so
there were some errors in determining if a person in an image is surprised. For example, due to the
separation of the subject’s lips in the above image, the program determined that the person is moderately
surprised.
Detecting Surprise Using Menpo
Presenter(s): Angelo Stekardis
Department: Computer Science
University of Kentucky, Lexington, KY
One of the sample images used in the project. When running the
program, it says that the subject of this image is moderately
surprised.
Overview Detecting Facial Landmarks Method
Method
Conclusion
Acknowledgements
Menpo Basics
References
The graph to the left demonstrates a
comparison between Menpo and two other
publically available image processing
applications (IntraFace and DRMF). Clearly,
Menpo is competitive with other state-of-the-
art applications.
Fig. 1. Menpo Performance Comparison [2]