This document summarizes an approach to improving spatial codification in semantic segmentation. It proposes using figure-border-ground spatial pooling with object candidates as well as applying a contour-based spatial pyramid, including crown-based and cartesian-based variations. Experiments show these approaches improve accuracy over traditional spatial pooling, especially when using CPMC and MCG object candidates, achieving state-of-the-art performance on PASCAL VOC 2011 and 2012 datasets.
5. Related Work
● The Visual Extent of
an Object [1]
5
[1] Uijlings et al, The VIsual Extent of an Object. IJCV’12
6. 1st Contribution
● Using a Figure-Border-Ground spatial pooling with object
candidates
6
Figure-Ground
spatial pooling
Figure-Border-Ground
spatial pooling
7. Related Work
● Beyond bags of features:
Spatial pyramid matching
for recognizing natural
scene categories [1]
7
[1] Lazebnik et al, Beyond bags of features: Spatial pyramid matching for recognizing natural scenes. CVPR’06
8. Related Work
● Variations of SPM
○ Non-arbitrary division
■ Object-centric pooling [1]
■ Object confidence map
partition [2]
○ SPM over bounding boxes [3]
[4]
[1] Russakovky et al, Object-centric spatial pooling for image classification. ECCV’12
[2] Chen et al, Hierarchical Matching with Side Information for Image Classification. CVPR’12
[3] Arbeláez et at, Semantic segmentation using regions and parts. CVPR’12
[4] Gu et al, Multi-component models for object detection. ECCV’12
8
11. Architecture
● Architecture proposed and released in [1]
[1] Carreira et al, Semantic segmentation with second-order pooling. ECCV’12
Train Test
DataBase
Object
Candidates
Feature
Extraction
Test
Model
Prediction
Evaluation
AAC
Ground
Truth
Train
CPMC SIFT-based
features (O2P)
11
21. Conclusions
● 2 proposals beyond the classic Figure-Ground pooling
○ Figure-Border-Ground spatial pooling
■ Extended to realistic scenario with CPMC object
candidates
○ A novel contour-based spatial pyramid has been introduced
■ Cartesian-based spatial pyramid
■ Crown-based spatial pyramid
● Validation of both proposals also for MCG object candidates
21
22.
23. Related Work
● The Visual
Extent of an
Object (Uijlings
et al, IJCV’12)
23