The document proposes a new method called Sparse Isotropic Hashing (SIH) to learn compact binary codes for image retrieval. SIH imposes additional constraints of sparsity and isotropic variance on the hash functions to make the learning problem better posed. It formulates SIH as an optimization problem that balances orthogonality, isotropic variance and sparsity, and develops an algorithm to solve it. Experiments on a landmark dataset show SIH achieves comparable retrieval accuracy to the state-of-the-art method while learning hash codes 20 times faster.

1. Sparse Isotropic Hashing
Ikuro Sato, Mitsuru Ambai, Koichiro Suzuki
Denso IT Laboratory, Inc.
{isato, manbai, ksuzuki}@d-itlab.co.jp
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 1/28
Presented at MIRU 2013, Japan.
Peer reviewed paper available at http://www.am.sanken.osaka-u.ac.jp/CVA/

2. • Introduction
• Proposed method
• Experiment
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 2/36

3. Practical issues of large-scale image retrieval
• ex) descriptor-matching approach
millions of sums-of-product / query
?
slow
query
image
query image
DB: ~108 images
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 3/28

4. Potential solution: descriptor binarization
computational time of similarity
real 512
bit
256
bit
128
bit
64
bit
32
bit
binary codes1
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 4/28

5. Binarization by hash functions
1. supervised
– uses known point-to-point correspondences
• ex) Ambai et al, 2012.
2. unsupervised
– intends to preserve similarities among the original real
vectors
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 5/28

6. Popular hash function
ex)
Random Proj. (Goemans et al, 1995)
Very Sparse Rand. Proj. (Li et al, 2006)
Sequential Proj. (Wang et al, 2010)
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved.
Iterative Quantization (Gong et al, 2011)
Isotropic Hashing (Kong et al, 2012)
this work
state-of-the-art
6/28

7. Most related work: Isotropic Hashing (Kong et al, 2012)
1. orthonormality
2. isotropic variance
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 7/28

8. Most related work: Isotropic Hashing (Kong et al, 2012)
1. orthonormality
2. isotropic variance
Robust to noise from spherically
symmetric distribution.
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 8/28

9. Learning of Isotropic Hashing
• Lift and Projection (LP) algorithm
isotropic orthogonal
Gradient Flow
algorithm omitted.
intersection
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 9/28
1) PCA:

10. Under-constrained problem
It’s more natural to impose additional conditions
to make the problem over-constrained.
our motivation
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 10/28

11. Our contribution
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 11/28

12. • Introduction
• Proposed method
• Experiment
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 12/36

13. Problem setup
1. rotational matrix
2. isotropic variance
3. sparsity
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 13/28

14. Condition-1: Special orthogonal group
-1
1
0
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 14/28

15. Condition-2: Cost function for isotropic variance
Exact solutions exist according to the Schur-Horn Theorem (AJM1954).
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 15/28

16. 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 16/28

17. Our optimization problem
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 17/28

18. Algorithm
Sparse Isotropic Hashing (SIH)
• Repeat until convergence.
endfor
notations
simplified
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 18/28

19. Illustration of the optimization process
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 19/28

20. • Introduction
• Proposed method
• Experiment
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 20/36

21. Dataset
etc.
* M. Ambai and I. Sato, “Fast binary coding of local descriptors based on supervised learning” (MIRU2012).
descriptor
query set
(u=1)
training set
(u=2, 3, 4)
test set
(u=5, 6)
CARD (Ambai et al, 2011)
without binarization
12896 50053 25238
# local descriptors
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 21/28

22. Evaluation criterion
• Mean Average Precision (mAP)
– expected value of area under Precision-Recall curve
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved.
precision
recall
1.0
Average
Precision
22/28

23. Methods compared
All methods use
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 23/28
state-of-the-art

24. mAP for CARD
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 24/28

25. mAP for CARD
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 25/28

26. mAP for CARD
almost
on top
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 26/28

27. mAP for CARD
10% drop in mAP,
20x faster coding
env.: VS2010, C program
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 27/28

28. Conclusion
Isotropic Hashing (Kong et al, 2012):
highly under-constrained
8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 28/28