1. Robust Real-Time Object DetectionAuthor: Paul Viola, Michael J. Jones
International Journal of Computer Vision 57(2), 137–154, 2004
Presented by ERLI – M10115801
8. Integral Image
• Image representation
• Feature
• The integral image at location x,y
contains the sum of the pixels
above and to the left of x,y
Integral Image | AdaBoost | Cascade
Sum of pixel
value in this area
(x,y)
9. Integral Image
• Image representation
• Feature
• The integral image at location x,y
contains the sum of the pixels
above and to the left of x,y
Integral Image | AdaBoost | Cascade
Sum of pixel
value in this area
(x,y)
How to calculate D ?
A B
C D
1 2
3 4
10. Integral Image
• Image representation
• Feature
Integral Image | AdaBoost | Cascade
How to calculate D ?
D = (1+4) – (2+3)
A B
C D
1 2
3 4
1
2
3
4
11. Integral Image Feature
Integral Image | AdaBoost | Cascade
Detection window
A
B
C
D
Detection window
Detection window Detection window
The value of integral image
feature is the difference between
the sum of the pixels within the
two rectangular regions.
S1 S2 |S1-S2|
12. Integral Image Feature
Integral Image | AdaBoost | Cascade
The size and position of feature can be
different.
• The eye region is darker
than the upper-cheeks.
• The nose bridge region is
brighter than the eyes.
13. Integral Image
• Different size and position of integral image give so many possible
features
• How do we obtain the best representing features possible?
• How can we refrain from wasting time on image background? (i.e.
non-object)
Integral Image | AdaBoost | Cascade
Good Feature Not Good Feature
AdaBoost…
15. Weak Classifier
• Consist of just 1 feature
• 1 feature that can separate the image data with error rate less than 0.5
Feature 1
Feature 2
Feature 3
θ 1
θ 2
θ 3
Weak
Classifier 1
Weak
Classifier 2
Weak
Classifier 3
1/10 = 0.1
6/10 = 0.6
3/10 = 0.3
Integral Image | AdaBoost | Cascade
16. Strong Classifier
• Set of features/weak classifier
Integral Image | AdaBoost | Cascade
1 1 1
strong
1
1 ( ) ( )
( ) 2
0 otherwise
n n nh h
h
x x
x
Weak Classifier
How to build the strong classifier?
AdaBoost
17. AdaBoost
Pseudo Code
• T = number of feature we choose
• h(x, f,p,θ)) = A weak classifier
• f = feature
• θ = threshold
• p = polarity indicating the direction of the
inequality (less than or greater than θ)
Integral Image | AdaBoost | Cascade
18. AdaBoost starts with a uniform distribution
of “weights” over training examples.
Select the classifier with the lowest weighted
error (i.e. a “weak” classifier)
Increase the weights on the training
examples that were misclassified.
(Repeat)
At the end, carefully make a linear
combination of the weak classifiers obtained
at all iterations.
AdaBoost Ilustration
1 1 1
strong
1
1 ( ) ( )
( ) 2
0 otherwise
n n nh h
h
x x
x
Slide taken from a presentation by Qing Chen, Discover Lab, University of Ottawa
Integral Image | AdaBoost | Cascade
19. Classifier
• By increasing the number of features per classifier, we:
◦ Increase detection accuracy.
◦ Decrease detection speed.
• Experiments showed that a 200 feature classifier
makes a good face detector:
◦ Takes 0.7 seconds to scan an 384 by 288 pixel image.
• Problem: Not real time! (At most 0.067 seconds
needed).
Integral Image | AdaBoost | Cascade
Cascade of Classifier…
21. Cascade of Classifier
• Classifier is structured as several layer/hierarchy
• Each layer consist of 1 strong classifier (set of weak
classifiers/features)
• Evaluating all input window
• Ignore non-face window
• Continue for processing “suspected” face window
Integral Image | AdaBoost | Cascade
22. Cascade of Classifier
• The number of feature/weak classifier in every layer is based on the
value of false positive rate and detection rate determined by user
• Previous layers have simpler classifier than next layer
Integral Image | AdaBoost | Cascade
Stage 1 Stage 2 Stage 3 …
Further
Processing
Input Image
Window
Ignore/Reject
Input
Ignore/Reject
Input
Ignore/Reject
Input
Ignore/Reject
Input
Y
N
Y
N
Y
N
Y
N
25. Experiment
• Training set
• Face: 4916 labeled frontal faces
•Testing / Real-time test
• Face: MIT + CMU frontal face test set
• Contain: 130 images with 507 labeled frontal faces
27. Result
They use dataset MIT+CMU with 5 images removed
Voting (Using 3 Viola-Jones Classifier with different parameter)
++- : Face
--+ : Non Face
28. Failure Mode
• Informal observation suggests that the face detector can detect
faces that are tilted up to about ±15 degrees in plane and about ± 45
degrees out of plane
• Harsh backlighting in which the faces are very dark while the
background is relatively light sometimes causes failures
• Proposed algorithm will also fail on significantly occluded faces