Graph Application on Face for Personal Authentication and Recognition

1. Graph Application on Face for
Personal Authentication and
Recognition
Authors: D. R. Kisku, A. Rattani, *M. Tistarelli, P. Gupta
*Contact author: tista@uniss.it
Computer Vision Laboratory, DAP,
University of Sassari, Italy.
Massimo Tistarelli; E-Mail: tista@uniss.it 1

2. Why face recognition problem is a
complex problem in computer vision?
Due to variations in illumination
Variations in nearby clutter
Variability in translation, rotation, scale and pose
Due to facial expression
Due to occlusion
Due to lighting conditions
2
Massimo Tistarelli; E-Mail: tista@uniss.it

3. Face recognition algorithms:
Appearance-based method (uses holistic texture
features) [1-3]
Feature-based approach (uses geometrical facial
features) [4]
Model-based approach (uses active appearance models,
shape models, fitting morphable models)
3
Massimo Tistarelli; E-Mail: tista@uniss.it

4. Agenda:
Introduction to SIFT-based face recognition model
Processes in the proposed model
SIFT features extraction
Discussion of the related works
EBGM face model
Proposed face recognition model
Results and discussion
Concluding remarks
Bibliography
4
Massimo Tistarelli; E-Mail: tista@uniss.it

5. Introduction to SIFT-based face
recognition model
Face recognition models are built with graph topology
drawn on Scale Invariant Feature Transform (SIFT)
features [6-7], [10].
SIFT features are extracted from face images, which are
invariant to rotation, scaling and partly illumination. Also
invariant to 3D projective transform (see Figure 1).
Face projections on images, represented by a graph,
can be matched onto new images by maximizing a
similarity function taking into account spatial distortions
and the similarities of the local features.
5
Massimo Tistarelli; E-Mail: tista@uniss.it

6. Contd…
50
100
150
200
50 100 150 200
SIFT features are extracted
Face image
from Face image
Figure 1. Left figure shows the face image and right figure shows the
face image with SIFT features.
6
Massimo Tistarelli; E-Mail: tista@uniss.it

7. Processes in the proposed model:
First the face image is photometrically normalized by
using histogram equalization [11].
The rotation and scale invariant SIFT features are then
extracted from the face image.
2D feature space is divided into sub-regions with
extracted SIFT features.
Finally the graph-based topology is used for matching
the pair of sub-regions of the corresponding two face
images.
7
Massimo Tistarelli; E-Mail: tista@uniss.it

8. SIFT feature extraction:
The scale invariant feature transform, called SIFT
descriptor, was proposed by Lowe [10] and proved to be
invariant to image rotation, scaling, and partly
illumination changes. The basic idea of the SIFT
descriptor is detecting feature points efficiently through a
staged filtering approach that identifies stable points in
the scale-space. This is achieved by the following steps:
Select candidates for feature points by searching peaks
in the scale-space from a difference of Gaussian (DoG)
function.
Localization of feature points by using the measurement
of their stability.
8
Massimo Tistarelli; E-Mail: tista@uniss.it

9. Contd…
Assign orientations based on local image properties.
Calculate the feature descriptors which represent local
shape distortions and illumination changes.
9
Massimo Tistarelli; E-Mail: tista@uniss.it

10. Discussion of related works: The SIFT-
based face recognition models
Gallery image based match constraint
(GIBMC) [11]
Reduced point based match constraint
(RPBMC) [11]
10
Massimo Tistarelli; E-Mail: tista@uniss.it

11. Gallery image based match constraint
(GIBMC) [11]:
An assumption has been made the fiducial points would be
available around identical positions in the face image.
SIFT feature points are extracted from both the probe face
and gallery face images.
Euclidean distance metric is used to eliminate the false pair
matches and obtained a set of correspondence matched
pairs.
Identical number of interest points are not found on both
faces. Due to that many feature points might be discarded
either from the second face or from the first face, or many
repetitions might be available for a single point either on the
second face or on the first face (see Figure 2 and Figure 3).
11
Massimo Tistarelli; E-Mail: tista@uniss.it

12. Gallery image based match constraint
Contd…….
After computing distances between a pair of points, only the
minimum pair distance would be taken.
6.9
X1 Y1
50
7.6
X2
7.2 100
X3 Y9
150
Xi Yi
10.9
X
200
9.7
50 100 150 200 250 300 350 400
First Face Image Second Face Image
Figure 2. The corresponding points of the Figure 3. Feature points and their
First Face Image mapped into the second matches are shown for a pair of
Face Image. faces, computed from Euclidean
distance metric
12
Massimo Tistarelli; E-Mail: tista@uniss.it

13. Reduced point based match constraint
(RPBMC) [11]:
Limitations due to multiple assignments estimated in
gallery image based match constraint is removed and the
technique is furthermore extended by reduced point based
match constraint.
The false matches due to multiple assignments are
eliminated by pairing the points with the minimum
distance.
The false matches due to one way assignments are
eliminated by removing the correspondence links that do
not have any corresponding assignment from the other
face (see Figure 4 and Figure 5).
13
Massimo Tistarelli; E-Mail: tista@uniss.it

14. Reduced point based match constraint…..
Contd…
50
100
7.6 150
X2
200
X3 Y9
7.2 50 100 150 200 250 300 350 400
XN
9.7
50
100
First Face Image Second Face Image
150
200
Figure 4. Elimination of false matches.
50 100 150 200 250 300 350 400
Figure 5. Example of reduced point
based match constraint.
14
Massimo Tistarelli; E-Mail: tista@uniss.it

15. Elastic Bunch Graph Matching (EBGM)
face recognition model:
In EBGM face model [4], a single labeled graph is
matched onto an image.
A labeled graph has a set of jets arranged in a particular
spatial order. A corresponding set of jets can be selected
from the Gabor-wavelet transform of the image.
The image jets initially have the same relative spatial
arrangement as the graph jets, and each image jet
corresponds to one graph jet.
The similarity of the graph with the image is simply the
average jet similarity between image and graph jets.
15
Massimo Tistarelli; E-Mail: tista@uniss.it

16. Proposed face recognition model: Regular
grid based match constraint (RGBMC)
The graph matching technique has been developed with
the concept of matching of the corresponding sub-
graphs for a pair of face images.
First the face image is divided into sub-images, using a
regular grid with overlapping regions.
The matching between a pair of face images is
performed by comparing sub-images and computing
distances between all pairs of corresponding sub-image
graphs in a pair of face images.
Finally, averaging the dissimilarity scores for a pair of
sub-images.
16
Massimo Tistarelli; E-Mail: tista@uniss.it

17. Results and discussion
The proposed graph matching strategy, namely, RGBMC
and the other two techniques, namely, GIBMC and
RPBMC have been evaluated and tested on the IITK
face database and on the BANCA database [11].
EBGM face recognition model [4] has tested on IITK face
database only.
From the receiver operating characteristics (ROC) curve,
it is clearly seen that the regular grid based match
constraint outperform other two SIFT-based methods
along with the EBGM face model while test is performed
on IITK face database (for illustration, see Figure 6 and
the Table 1).
17
Massimo Tistarelli; E-Mail: tista@uniss.it

18. Contd…
When the test is performed on BANCA database with the
three SIFT-based graph matching techniques including
the proposed RGBMC, the RGBMC outperformed other
two methods with G1 group face images.
But, with G2 group face images the proposed RGBMC
shows increased in errors than the RPBMC method.
The overall performance of the proposed model is much
more satisfactory while is compared with other methods
on the two datasets.
18
Massimo Tistarelli; E-Mail: tista@uniss.it

19. Contd…
ROC Curve ROC Curve for G1 Group
100 100
90
98
80
96
False Negative Error
70
94
False Negative Error
60
92 50
90 40
30
88
20 RPBMC
86 RGBMC
10
GIBMC
GIBMC
84
0
RPBMC 0 10 20 30 40 50 60 70 80 90 100
82 False Positive Error
RGBMC
ROC Curve for G2 Group
EBGM
80 100
90
78
0 10 20 30 40 50 60 70 80 90
80
False Positive Error
False Negative Error
70
Figure 6. ROC curves for different 60
techniques are tested on large size 50
IITK face database are shown. 40
30
20 RPBMC
RGBMC
Figure 7. On the right, ROC curves for different 10
GIBMC
SIFT-based techniques are tested on small size 0
0 10 20 30 40 50 60 70 80 90 100
False Positive Error
BANCA face database are shown.
19
Massimo Tistarelli; E-Mail: tista@uniss.it

20. Contd…
Table 1: FRR, FAR and WER are shown for different techniques
computed from IITK face database.
FRR FAR WER WER WER Avg
(%) (%) (R=0.1) (R=1.0) (R=10) WER
(%)
GIBMC 15 10.94 14.63 12.97 11.31 12.97
RPBMC 10.5 6.46 10.13 8.48 6.83 8.48
EBGM 12.07 7.87 11.68 9.97 8.25 9.97
Proposed 7.31 4.65 7.06 5.98 4.89 5.98
RGBMC
20
Massimo Tistarelli; E-Mail: tista@uniss.it

21. Contd…
Table 2: Weighted Error Rates are shown for three SIFT-based
techniques computed from BANCA face database.
GIBMC RPBMC (%) RGBMC (%)
(%)
WER (R = 0.1) on G1 10.24 7.09 4.07
WER (R = 0.1) on G2 6.83 2.24 3.01
WER (R = 1) on G1 10.13 6.66 4.6
WER (R = 1) on G2 6.46 1.92 2.52
WER (R = 10) on G1 10.02 6.24 4.12
WER (R = 10) on G2 6.09 1.61 2.02
21
Massimo Tistarelli; E-Mail: tista@uniss.it

22. Concluding remarks:
The work shows a remarkable increase in the performance
of the system with respect to the previous two works based
on the SIFT features.
The obtained results show the capability of the system in
respect of illumination changes and occlusions occurring in
the database or the query face.
The proposed method is found to be superior while it is
compared with the EBGM technique.
Due to invariant nature of SIFT features it is realized that
even comparison made with the some well known SIFT-
based techniques along with the EBGM model on the
different standard databases, the proposed model drawn on
SIFT features is shown to be robust and efficient technique
in terms of facial expressions, illumination changes, lighting
conditions, occlusions, pose changes, etc.
22
Massimo Tistarelli; E-Mail: tista@uniss.it

23. References:
[1] M. Turk, and A. Pentland, “Eigenfaces for Recognition,” J.
Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.
[2] W. Zhao, R. Chellappa, and P. Phillips, “Subspace Linear
Discriminant Analysis for Face Recognition,” Technical Report
CAR-TR-914, Center for Automation Research, Univ. of
Maryland, College Park, 1999.
[3] T.W. Lee, Independent Component Analysis: Theory and
Applications. Boston: Kluwer Academic Publishers, 1998.
[4] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg,
“Face recognition by elastic bunch graph matching”, IEEE
Trans. Pattern Anal. Mach. Intell. 19 (7), pp. 775–779, 1997.
[5] A.M. Martinez, “Recognizing Imprecisely Localized, Partially
Occluded, and Expression Variant Faces from a Single Sample
per Class,” IEEE Trans. Pattern Analysis andMachine
Intelligence, vol. 24, no. 6, pp. 748-763, June 2002.
23
Massimo Tistarelli; E-Mail: tista@uniss.it

24. Contd…
[6] M. Bicego, A. Lagorio, E. Grosso, and M. Tistarelli, “On the use of
SIFT features for face authentication”, Proc. of IEEE Int Workshop on
Biometrics, in association with CVPR, pp. 35, 2006.
[7] D. G. Lowe , “Object recognition from local scale invariant features,”
International Conference on Computer Vision, Corfu, Greece, pp.
1150–1157, September 1999.
[8] K. Mikolajczyk, C. Schmid: An affine invariant interest point detector.
Proc. of European Conference on Computer Vision, 2002, pp. 128-
142.
[9] T. Tuytelaars, L.V. Gool: Wide baseline stereo matching based on
local affinely invariant regions. Proc. of British Machine Vision
Conference, pp. 412-422, 2000.
[10] D. G. Lowe. Distinctive image features from scale invariant keypoints.
Int. Journal of Computer Vision, 60(2), 2004.
[11] D. R. Kisku, A. Rattani, E. Grosso, and M. Tistarelli, “Face
identification by SIFT-based complete graph topology”, IEEE
workshop on Automatic Identification Advanced Technologies, pp. 63-
68, 2007.
24
Massimo Tistarelli; E-Mail: tista@uniss.it

25. Thank You !
25
Massimo Tistarelli; E-Mail: tista@uniss.it

26. Questions ???
26
Massimo Tistarelli; E-Mail: tista@uniss.it