1. Improved Context-AwareImproved Context-Aware
Saliency Detection UsingSaliency Detection Using
Local Search WindowLocal Search Window
Shanshan Wand and Amr Abdel-Dayem
Department of Mathematics and Computer Science
Laurentian University
Ramsey Lake Road, Sudbury, Canada
aabdeldayem@cs.laurentian.ca
2. ContentsContents
• Introduction
• Saliency Detection Approaches
• Context-aware Saliency (CASal)
• Improvement of CASal
• Results
• Conclusion
2International Conference on Computer Science and Engineering ICGST2011
3. IntroductionIntroduction
• Problem definition
Find an appropriate saliency map for content-aware
image retargeting problem.
oWhat is content-aware image retargeting (CAIR) problem?
oWhat is saliency map?
3International Conference on Computer Science and Engineering ICGST2011
4. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
4International Conference on Computer Science and Engineering ICGST2011
5. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
5International Conference on Computer Science and Engineering ICGST2011
6. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
6International Conference on Computer Science and Engineering ICGST2011
7. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
7International Conference on Computer Science and Engineering ICGST2011
8. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
8International Conference on Computer Science and Engineering ICGST2011
9. IntroductionIntroduction
• Content-aware image retargeting (CAIR)
o Given an image I with m×n and a new size m’×n’, produce
a new image J with size m’×n’ , that will be a good
representative of the image I.
o The main objectives of retargeting methods:
•Preserve the important content
•Limit visual artifacts in the retargeted images
•Preserve internal structures of the original images
o CAIR is a kind of techniques, which retarget images considering
important content.
9International Conference on Computer Science and Engineering ICGST2011
10. Saliency MapSaliency Map
• Requirements of importance map in CAIR
o Detect important objects and their surrounding contexts.
o More important in the boundary of objects and internal
structures, less important in the inner region of objects
10International Conference on Computer Science and Engineering ICGST2011
11. Saliency MapSaliency Map
• Saliency map
o Definition
Saliency map is the output of saliency detection, which presents the
sensory information before further processing.
o How to determine saliency for an image region?
The difference between it and its surrounding regions, based on various
features .
e.g. color, orientation, motion, high-level features (texts, faces), etc.
o Applications
•Content-aware image retargeting (CAIR)
•Content-based information retrieval (CBIR)
•Image browsing on mobile devices
•Object recognition
•Automatic cropping
•Image summarization
•…
11International Conference on Computer Science and Engineering ICGST2011
12. Saliency Detection ApproachesSaliency Detection Approaches
o Low-level features
• Intensity, color contrast, directions, etc.
o High-Level features can be added
• Edges, textures, text, faces,.....etc.
• Common approaches
o To identify the human vision attracted fixation points
Itti’s method
Graph-Based Visual Saliency (GBVS)
o To detect a dominant object
CRFSal
o To detect dominant objects with their essential contexts
Context-aware saliency detection (CASal)
12International Conference on Computer Science and Engineering ICGST2011
13. Saliency Detection ApproachesSaliency Detection Approaches
original
Itti’s method GBVS
CRFSal CASal 13International Conference on Computer Science and Engineering ICGST2011
14. Context-aware Saliency (CASal)Context-aware Saliency (CASal)
• Four Principles of Human Visual Attention
o P1: Local low-level features
Color contrast
Saliency in the regions that have distinctive colors should be high
o P2: Global features
Suppress frequently occurring features
o P3: Visual organization rules
Visual forms consist of one or several centres of gravity
o P4: High-level factors
faces
14International Conference on Computer Science and Engineering ICGST2011
15. Context-aware Saliency (CASal)Context-aware Saliency (CASal)
• Based On:
o Dissimilarity Measure between Patch Pairs
o Multiple-scale Saliency
o Foci of Attention
15International Conference on Computer Science and Engineering ICGST2011
16. Context-aware Saliency (CASal)Context-aware Saliency (CASal)
• Discussions
o High accurate saliency map
o Computationally too expensive
Around 38 minutes (256 × 256)
16International Conference on Computer Science and Engineering ICGST2011
17. Improvement of CASalImprovement of CASal
• Using single scale instead of multi scales
• Using multi scales with patch size 7×7
• Adding local search window
17International Conference on Computer Science and Engineering ICGST2011
18. Improvement of CASalImprovement of CASal
• Using single scale instead of multi scales
Image scale 1/4r
Patch size 3×3
Image scale 1/2r
Patch size 5×5
Image scale r
Patch size 7×7
Original image
18International Conference on Computer Science and Engineering ICGST2011
19. Improvement of CASalImprovement of CASal
• Using multi scales with patch size 7×7
Multi scales
Patch size 7×7
Original CASal
Original image
19International Conference on Computer Science and Engineering ICGST2011
20. Improvement of CASalImprovement of CASal
• Adding local search window
Shrink the neighbouring region for searching the K most similar patches
Original image
Original CASal CASal with local window 57×57
CASal with local window 85×85 CASal with local window 113×113
20International Conference on Computer Science and Engineering ICGST2011
21. ResultsResults
• 86 images from the benchmark for image retargeting
• Three sizes for local windows are tested: 57×57 85×85 113×113
• Using local window with 85×85 produces almost the same saliency maps as
the original CASal, but spends 21.7% of the time of the original CASal.
Window size Average time (seconds)
57×57 267.82
85×85 498.15
113×113 824.18
Full image (original CASal [9]) 2294.67
Average running time using different search window sizes
21International Conference on Computer Science and Engineering ICGST2011
26. ConclusionConclusion
• Using the local search window can reduce the
computational cost of the CASal algorithm.
• The proposed solution produces comparable saliency
map to the original algorithm.
• Finding simpler and more efficient similarity measure
can further significantly reduce the algorithm’s running
time in the future.
26International Conference on Computer Science and Engineering ICGST2011
