1. An Approach to Re-Rank Retrieved
Images - Image Search Results with
Visual Similarity
Presented by Guided by
Veningston .K Mr. M. Newlin Rajkumar
M.E student, Dept of CSE, Lecturer, Dept of CSE,
Anna University – Coimbatore, Anna University – Coimbatore,
India. India.

2. Objective
 To address a ranking problem in web image
retrieval
 System to re-rank images returned by image
search engine
 Re-ranking images by incorporating,
visual aspects
visual similarity
04/17/12 Department of Computer Science and Engineering 2

3. Introduction
 Image Retrieval System
 Comparative study on text & image based search
 Interest point extraction - visual content of images
 Re-ranking the results of text based systems
using visual information
 Finding the largest set of most similar images
 Rearranging images based on the similarity
04/17/12 Department of Computer Science and Engineering 3

4. Why image Re-ranking?
 To maximize relevancy of image results
 To achieve diversity of image results
04/17/12 Department of Computer Science and Engineering 4

5. Proposed Scheme
 Goal
 Retrieve image results that are relevant
 Finding common features among images
 Overview
 Interest points on the images are extracted
 Similarity of each pair of images are computed
 Generate graph model
 Apply page ranking
04/17/12 Department of Computer Science and Engineering 5

6. Finding common features among
images
Similarity measurement to handle potential rotation,
scale and perspective transformations.
04/17/12 Department of Computer Science and Engineering 6

7. Scale Invariant Feature Transform
 Image matching and features to similarity
04/17/12 Department of Computer Science and Engineering 7

8. Image similarity
 Given two images u and v,
 Corresponding descriptor vector, Du = (d1u,
d2u, ...dmu ) and Dv = (d1v, d2v, ...dnv ),
 Define the similarity between two images
simply as the number of interest points
shared between two images divided by their
average number of interest points.
04/17/12 Department of Computer Science and Engineering 8

9. Graph model
 Given the visual similarities of the images to
be ranked.
 Treat images as web documents
 Treat similarities as visual hyperlinks
 Estimate the probability of images being visited by
a user // using page rank
 Images with more estimated visits are ranked
higher
04/17/12 Department of Computer Science and Engineering 9

10. Similarity to Centrality
 Centrality – importance of images
 Given a graph with vertices and a set of
weighted edges, define and measure the
“importance” of each of the vertices
 Vertices = images
 Egde weights = similarity
 A vertex closer to an important vertex should
rank higher than others
04/17/12 Department of Computer Science and Engineering 10

11. Similarity to Centrality
 Ranking scores correspond to probability of
arriving in each vertex by traversing through
the graph
Matrix constructed from the
weights of the edges in the
Adjacency matrix for graph
unweighted graph Decision to take a
particular path defined by
weighted edges
04/17/12 Department of Computer Science and Engineering 11

12. Page Ranking (PR)
A B
 Determines the order of E
search results
 Method of measuring a C D
page’s importance
 Results are based on Search result Priority
this priority order B 3
A 2
C 1
D 1
E 1
04/17/12 Department of Computer Science and Engineering 12

13. Overall scheme
 Computationally infeasible to compute
similarities for all images indexed by search
engine
 Pre-cluster web images based on metadata
 Define the similarity of images
 Given a query, extract top - N results
returned, create graph of visual similarity on
the N images
 Compute image rank only on this subset
04/17/12 Department of Computer Science and Engineering 13

14. Performance metrics
 Precision and recall
number of relevant images in the returned images
 Recall = ----------------------------------------------------------------------
total number of relevant images in the database
number of relevant images in the returned images
 Precision = ---------------------------------------------------------------------
total number of returned images
 Result analysis
 Screen shots
04/17/12 Department of Computer Science and Engineering 14

15. Merits
 Minimizing irrelevant images
 Selecting small set of images
 Computational cost
04/17/12 Department of Computer Science and Engineering 15

16. Conclusion
 Ability to reduce the number of irrelevant
images
 A tiny set of important images can be
selected from a very large set of candidates
04/17/12 Department of Computer Science and Engineering 16

17. Future work
 Extensions of this technique to a query driven
feature selection.
04/17/12 Department of Computer Science and Engineering 17

18. References
 [1] Yushi Jing, Shumeet Baluja. VisualRank: Applying PageRank to
Large-Scale Image Search. IEEE Transaction on Pattern Analysis and
Machine Intelligence, Vol 30, No. 11:1887–1890, 2008.
 [2] R. Datta, D. Joshi, J. Li, and J. Wang. Image Retrieval: Ideas,
Influences, and Trends of the New Age. ACM Computing Surveys,
vol. 40, no. 2, 2008.
 [3] Yushi Jing, Shumeet Baluja. PageRank for Product Image Search.
WWW 2008, April 21–25, 2008, Beijing, China. ACM International conf.
2008.
 [4] K. Mikolajczyk and C. Schmid. A performance evaluation of local
descriptors. IEEE Transaction on Pattern Analysis and Machine
Intelligence, 27(10):1615–1630, 2005.
 [5] S. Belongie, J. Malik, and J. Puzicha. Shape matching and object
recognition using shape contexts. IEEE Transactions on Pattern
Analysis and Machine Intelligence (PAMI), 24(24):509–522, 2002.
04/17/12 Department of Computer Science and Engineering 18

19. Thank you for your
Attention
04/17/12 Department of Computer Science and Engineering 19