Provide a summary for "RGB-(D) Scene Labeling: Features and Algorithms" paper, written by X Ren, L Bo, D Fox - Computer Vision and Pattern Recognition 2012 - ieeexplore.ieee.org

1. RGB-(D) Scene Labeling:
Features and Algorithms
Ahmed Taha
May 2014
Supervised by
Dr : Marwan A. Torki

2.  Introduction
 Scene labeling challenges
 Pipeline
 Feature Extraction
 Super-pixel formulation and classification
 Classifying segmentation tree paths
 Classifying super-pixels MRF
 Datasets and results
Agenda

3.  Scene Labeling
 Labeling of each pixel in an image to a certain class
 Scene Labeling can be done
 Indoors
 Label a Sofa in a Bedroom
 Label a door in a living room
 Outdoors
 Label a car in street
 Label building in street
Scene Labeling

4. Scene Labeling

5. Scene Labeling

6.  Indoor scene labeling challenges
 Large variations of scene types
 Lack of distinctive features
 Poor illumination
Scene Labeling

7.  Benefits of using depth feature in scene labeling
 Increased accuracy and robustness
 Body pose estimation
 3D mapping
 Object recognition
 3D modeling and interaction
Scene Labeling

8. Pipeline

9. 1. Extract features using Kernel descriptor (KDES).
2. Aggregate descriptors in dense region into super-
pixels using Efficient match kernels (EMK)
3. Classify super-pixels using Linear support vector
machine (SVM)
4. Label super-pixels by classifying paths of
segmentation tree.
5. Label super-pixels using super-pixel MRF
Pipeline

10.  Kernel Descriptors (KDES), a unified framework that
uses different aspects of similarity (kernel) to derive
patch descriptors.
 Image gradient
 Spin/normal
 Color
 Depth gradient
Features Extraction (Step 1)

11.  Efficient match kernels (EMK) to transform and
aggregate descriptors in a set S (grid locations in the
interior of a superpixel ‘s’).
 Super-pixels are not of the same size.
Super-pixel formation (Step 2)

12.  Linear Support vector machine (SVM)
 Non-probabilistic binary linear classifier.
Classify superpixels (Step 3)

13.  Classifying paths in segmentation tree
Contextual Models (Step 4)

14.  Classifying paths in segmentation tree
Contextual Models (Step 4)

15.  Classifying paths in segmentation tree
Contextual Models

16.  Classifying paths in segmentation tree
 If we accumulate features over paths, the accuracy
continues to increase to the top level
 The initial part of the curves overlap, suggesting there is
little benefit going to superpixels at too fine scales
Contextual Models

17.  Superpixel MRF with gPb
Contextual Models (Step 5)

18.  Superpixel MRF with gPb
 standard MRF formulation. We use Graph Cut to find
the labeling that mini- mizes the energy of a pairwise
MRF
Contextual Models (Step 5)

19. Pipeline

20.  NYU-D dataset
 Improve accuracy from 50% to 76%
 Stanford Background dataset
 Improve accuracy 79% to 82%
Datasets - Results

21. Datasets - Results

22.  Rgb-(d) scene labeling: Features and algorithms
 X Ren, L Bo, D Fox - Computer Vision and Pattern
Recognition 2012 - ieeexplore.ieee.org
 Context by region ancestry
 JJ Lim, P Arbeláez, C Gu, J Malik - Computer Vision, 2009
IEEE 2009 - ieeexplore.ieee.org
References