Region based skin color detection.

1. Region-based Skin Color Detection
Rudra P K Poudel (Presenter), Hammadi Nait-Charif, Jian Jun Zhang
Media School, Bournemouth University, UK
David Liu
Siemens Corporate Research, USA

2. Outline of the talk
1. Introduction- skin color detection
2. Literature Review
3. Current Problems
4. Region-Based Technique
5. Proposed Region-Based Technique
6. Experimental Results
7. Conclusions

3. 1. Introduction
• Task: separate skin and non-skin
regions (not pixels)
• Motivation: invariant of rotation,
scaling and occlusion
• Problems: illumination, ethnicity
background, make-up, hairstyle,
eyeglasses, background color,
shadows, motion illumination, skin
look like colors, etc.
Source: Harry Potter movie

4. 1.1 Applications
• Hand tracking, face detection, pornography detection, person
tracking
• Skin color detection module equally applicable for other color
editing, detection etc applications
• Color is used as primary clue in many image processing and
computer vision applications

5. 2. Literature Review
2.1 Color space
• RGB
• HSV
• YCbCr
• Perceptually uniform color systems (CILLAB, CIELUV,
LAB)
• Normalized RGB
2.2 Skin color classifier
• Nonparametric methods: histogram, Bayes classifier,
self-organizing map
• Parametric methods: single Gaussian, mixture of
Gaussian
• Others: neural network

6. 2. Literature Review
Summary:
• Color space: RGB and HSV are two widely used techniques
• Classification method: Naïve Bayes classifier and mixture of
Gaussian are widely used techniques
• Gaussian model: need few training data, difficulties on parameter
tuning, need less memory space, processing/detection slow
• Bayes theorem: need for larger training data, easy for learning,
need more memory space, processing/detection fast
State-of-the-art method:
• Jones, M. J. and Rehg, J. M. (2002). Statistical color models with
application to skin detection. International Journal of Computer Vision,
46(1):81–96.

7. 3. Current Problem
• Probability accumulation for higher level vision task- as
probability for skin/non-skin vary highly even for adjacent
pixels
Naturally skin is
continuous region

8. 4 Region Based Approach
• Yang and Ahuja (1998) and Kruppa et al. (2002)- search
elliptical regions for face detection
• Sebe Sebe et al.(2004)- 3x3 fixed size patches to train
Bayesian network
• Our approach treat skin as region with varying sizes, which is
purely based on image evidence

9. Proposed Technique/Framework
1. Region extraction- quick shift image
segmentation, also know as “superpixels”
2. Region classification- pixel/region based
3. Smoothing- Conditional Random Field (CRF)
5. Proposed Region-Based technique

10. 5.1 Region Extraction
Region/Superpixel
extraction – quick shift
image segmentation using
RGB color and positional
(XY) coordinate

11. 5.1 Region Extraction
- Region extraction is purely evidence based i.e. based on RGB
color and spatial location (xy-coordinate) of the image
- Regions have different size and shape, which is depend upon
complexity of the image
- No explicit concept of boundary
- Quick shift preserve the boundary of the objects, hence we
could get very accurate object segmentation
- We could set importance on color difference vs spatial distance

12. 5.2 Region Classification
5.2.1 Basic Skin Color Classifer
Naïve Bayes: posterior likelihood * prior∝
However, we could use any suitable/best method for skin classification
)(
)()/(
)/(
cp
spscp
csp =
)(
)()/(
)/(
cp
nspnscp
cnsp =
Θ>
)/(
)/(
cnsp
csp
Θ>
)()/(
)()/(
nspnscp
spscp
1
)/(
)/(
>
nscp
scp Where, c = color, s = skin and ns = non-skin

13. 5.2 Region Classification
• Average the skin probability (s) of all color pixels (c)
belongs to the given superpixel (sp)
• Average the non-skin probability (ns) of all color pixels
(c) belongs to the given superpixel (sp)
∑=
N
i
icsP
N
spsP )|(
1
)|(
∑=
N
i
icnsP
N
spnsP )|(
1
)|(

14. 5.3 Smoothing with CRF
• Conditional Random Field (CRF) optimization
equation
• Color potential
• Edge and boundary potential
∑∑ ∈∈
Φ+Ψ−=−
Ess
jiji
Ss
ii
jii
ssccslSLP
),(
),|,()|());|(log( ωω
))|(log()|( iiii slPsl =ψ
[ ]ji
ji
ji
jiji cc
ss
ssL
sscc ≠








−+
=Φ ,
||||1
),(
),|,(

15. 5.4 Training
First Phase (training histogram):
• Train 2 histograms for skin and non-skin separately
Second phase (training CRF): learning :
si sj
…
px1(s|c)
px1(ns|c)
px1(s|c)
px1(ns|c)
color difference
+
boundary length
∑∑ ∈∈
Φ+Ψ−=−
Ess
jiji
Ss
ii
jii
ssccslSLP
),(
),|,()|());|(log( ωω
ω

16. 6 Experimental Results
• Dataset content 14 thousands images collected freely from
the web (Compaq dataset)
• 4,700 are skin and 9,000 non-skin images
• Approximately 1 billion pixels are manually labeled
• 50% is use for training and 50% for testing
Method True
Positive
False Positive
Jones and Rehg (2002) 90% 14.2%
Our (Superpixel only) 91.44% 13.73%
Our (Superpixel and CRF) 91.17% 13.12%

17. 6 Experimental Results
Our proposed new region-
based technique outperform
current state-of-the–art
technique

18. 6 Experimental Results
Applying CRF is always not good !

19. 6 Experimental Results
However, in aggregate CRF performs better!

20. 7. Conclusions
• Region-based technique performs better than pixel-based
• Region-based technique could easily incorporate texture info
and other type of features to improve the result
• Aggregation of pixels into region help to reduce local
redundancy.
• Region-based technique extracts larger smooth regions,
which is very helpful for higher-level vision task
The message to take home:
It is better/natural to treat skin as regions instead of
individual pixels!

21. Thank you !
Questions ???