1. Laureando: Muhammad Umar Riaz
Matricola: 80929
Texture Classification Methods
For Human Detection
Relatore: Dott. Sergio Benini
Correlatore: Ing. Michele Svanera

2. Scenario
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Human detection:
❖ visual content management
❖ video surveillance
❖ autonomous vehicles, for
pedestrian detection

3. Scenario
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Human detection -> Major limitation:
❖ incapability to perform when the
only available information is
human hair

4. Objectives
hair detection and segmentation in the wild
(with pure texture analysis)
(with no a-priori knowledge of face/head location)
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
System
processing

5. Figaro database
❖ It contains 840 unconstrained view images, subdivided into seven different hair
texture classes
Straight Wavy Curly Kinky Braids Dreadlocks Short-men
Description Ground-truth
http://projects.i-ctm.eu/en/project/figaro
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

6. All images in the database contain the corresponding manually labelled ground truth
Ground truth
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Description Ground-truth

7. Proposed pipeline
Feature
extraction
Patch-based
classification
Pixel-level
segmentation
System
processing
Workflow Feature extraction Patch-based classification Pixel-level classification
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

8. Feature extraction
Workflow Feature extraction Patch-based classification Pixel-level classification
Feature
extraction
Pixel-level
segment.
Feature vector
Patch
class.
Workflow HOG LTP DeCAF
HOG
LTP
DeCAF
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

9. Histogram of Oriented Gradients (HOG)
Workflow Feature extraction Patch-based classification Pixel-level classification
histogram of gradients
Workflow HOG LTP DeCAF
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

10. Local Ternary Patterns (LTP)
Workflow Feature extraction Patch-based classification Pixel-level classification
Workflow HOG LTP DeCAF
En =
8
><
>:
1, Vn  V0 + t
0, |Vn V0| < t
+1, Vn V0 + t
n = 1, 2, .., 8
Vn
V0
t
neighbouring pixel index
intensity value of neighbouring pixel
central pixel intensity
threshold value
n = 1, 2, .., 8
Vn
V0
t
n = 1, 2, .., 8
Vn
V0
t
n = 1, 2, .., 8
Vn
V0
t
histogram of TU numbers
. . . .Neighbourhood
63 28 30
88 40 35
67 40 21
1 -1 0
1 0
1 0 -1
1 0 0
1 0
1 0 0
Thresholded
neighbourhood
0 1 0
0 0
0 0 1
41
1 0 0
8 0
32 0 0
0 2 0
0 0
0 0 128
1 2 4
8 16
32 64 128
Upper pattern
Lower pattern
Pixel weights
Upper texture unit
Lower texture unit
130
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Upper TU
number
Lower TU
number

11. Deep Convolutional Activation Feature (DeCAF)
Workflow Feature extraction Patch-based classification Pixel-level classification
❖ CaffeNet
❖ Input image:
227x227x3
❖ conv1: 55x55x96
❖ conv2: 27x27x256
❖ conv3: 13x13x384
❖ conv4: 13x13x384
❖ conv5: 13x13x256
❖ fc6: 4096
❖ fc7: 4096
❖ Output: 1000
Workflow HOG LTP DeCAF
N.B. Input constraint of 227x227
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

12. Patch-based classification
Workflow Feature extraction Patch-based classification Pixel-level classification
Workflow Results
Random Forest
Learning
Feature
extract.
Training set Training gt
Learnt
Random Forest
Classifier
evaluation
Feature
extract.
Testing set Testing gtTesting results
Feature
extraction
Pixel-level
segment.
Patch
class.
Testing
Training
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

13. Results
Workflow Feature extraction Patch-based classification Pixel-level classification
Workflow Results
Features Parameters Precision(%) Recall(%) Accuracy(%) F1-score(%)
HOG Patch = 35x35, Cell = 35x35, Bins= 12 65.7 87.4 81.4 75.0
LTP Patch = 35x35, LTP_thr = 0.05 75.3 83.2 85.9 79.0
HOG + LTP Patch = 35x35, Cell = 35x35, Bins= 12, LTP_thr = 0.02 74.6 84.4 85.9 79.2
DeCAF Patch = 227x227, Step size = 32, fc7 43.3 98.8 58.3 60.2
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

14. LTP Class.
DeCAF Class.
Ground truth

15. High Precision Pixel-level classificationLTP Class.
DeCAF Class.
Ground truth

16. Pixel-level classification
Workflow Feature extraction Patch-based classification Pixel-level classification
FH-based segmentation Super-pixel-based segmentation
Workflow Results
I
I(P)
...
. . .
Isegm(P)
. . .
. . .
...
...
...
a) multiple FH-based
segmentations b) segment images
c) match step
Isegm(P)
i
Is it the best
match?
Ipatch
Feature extraction Patch classification
yes
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

17. Results
Workflow Feature extraction Patch-based classification Pixel-level classification
FH-based segmentation Super-pixel-based segmentation
Workflow Results
Features Parameters for patch-based classification Precision(%) Recall(%) Accuracy(%) F1-score(%)
LTP Patch = 35x35, LTP_thr = 0.02 80.8 75.4 86.2 77.5
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

18. High Precision
High Recall
LTP Class.
DeCAF Class.
Ground truth
Probability map
+
Overlapping patches (227x227)
Step size 32
“Greedy” classifier

19. Pixel-level classification
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Workflow Feature extraction Patch-based classification Pixel-level classification
Workflow Results
FH-based segmentation Super-pixel-based segmentation
SLIC0 - segmentation
Feature
extraction
Patch class.
Background
removal
Probability map (DeCAF)
227
227
Super-pixel
classif.
with DeCAF
Using “greedy”
nature of the
classifier

20. Results
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Workflow Feature extraction Patch-based classification Pixel-level classification
Workflow Results
Segmentation method Precision(%) Recall(%) Accuracy(%) F1-score(%)
Super-pixel method
(SLIC0)
87.5 73.1 88.1 79.6
FH-based segmentation Super-pixel-based segmentation

21. Overall Results
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Hair patch classification Hair segmentation
Accuracy(%) F1-score(%) Accuracy(%) F1-score(%)
HOG 81.4 75.0
FH-method
84.0 75.4
LTP 85.9 79.0 86.2 77.5
HOG+LTP 85.9 79.2 86.2 75.7
DeCAF 58.3 60.2
Super-pixel
method
88.1 79.6

22. Pixel Class.
(DeCAF)
Ground truth
Patch Class.
(LTP)

23. Conclusion & future developments
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments
Conclusion:
❖ Figaro database , first multi-class image database for
hair detection in the wild
❖ Novel hair detection and segmentation approach,
based on pure texture analysis
Future developments:
❖ use of other texture features
❖ application of cross-validation
❖ vary and increase non-hair data
❖ hair type classification
(1) http://projects.i-ctm.eu/en/project/figaro
1

24. Straight Wavy Curly Kinky Braids Dreads Shaved Prec.
Straight 27 5 1 0 1 1 5 0.79
Wavy 4 32 3 0 0 0 1 0.74
Curly 0 2 34 3 0 1 0 0.77
Kinky 0 0 3 36 1 0 0 0.82
Braids 2 3 3 4 28 1 0 0.90
Dreads 1 0 0 1 1 37 0 0.90
Shaved 0 1 0 1 0 1 37 0.86
Pred.
Label
Conclusion & future developments
Hair type classification
Scenario Objectives Figaro-database Proposed pipeline Results Conclusion & future developments

25. This work has been submitted to IEEE International Conference on Image Processing (ICIP) 2016
Thank you for your attention