KDD2013: Palette Power: Enabling Visual Search through Colors

1. Palette Power: Enabling Visual Search through Colors
Aug 14, 2013
eBay Research Labs
http://labs.ebay.com

2. Changing Landscape of Search
2

3. Visual Search for Fashion
eBay
Inventory
Given an item image,
find similar eBay inventory
Query Image
Similar Items
3

4. Item Similarity
Frequency
Color Distributions
Dots Floral Checks
Patterns & Textures
Styles
4

5. Approach Overview
Large Image Data Speed Requirements
Take Advantage of Context
Our Approach – The Power of Color Distributions
Color Spaces
𝑑 = 𝑓(𝑖1, 𝑖2)
Distance Functions
5

6. 6
Challenges (1/3)
Low contrast between background and foreground

7. 7
Challenges (2/3)
Background Clutter

8. 8
Challenges (3/3)
Lighting Variation

9. 9
Insights From Data
 Object localization using spatial priors
 Choosing the right color space

10. Why Object Localization?
10
 Cluttered background degrades performance.
 State-of-the-art segmentation too expensive.
 Need a fast and reliable solution!
Spatial Prior to the rescue!

11. Understanding Spatial Prior
11

12. 12
Choosing Best Color Space

13. 13
Handling Color Confusion

14. 14
Generating Color Histogram

15. Faster Lookup via k-center
15
 Scaling via backend clustering/indexing.
 Potential for semantic/intent diversification
- e.g. query t-shirt image where you like style but not colors
 Achieves 60x speedup close to 70% overlap!
Median speed-up Median %-overlap

16. 16
Architecture

17. 17
Experiment I – Fashion Dataset
 Categories: Women’s Dresses, Tops & Blouses, Coats & Jackets,
Skirts, Sweaters and T-Shirts
 Data Sets: 1600 Queries & 1 Million Inventory images, 15 users for
30 days

18. 18
Results - Solid Queries

19. Results - Pattern Queries
19

20. Experiment II – Generic ecommerce Dataset
Categories: Toys, Sports, Camera
Data Sets: Query & Inventory sets for each category
Ground Truth: ~15 per query
20

21. Example Inventory Images
21
Toys
Sports
Camera

22. 22
MAP Performance

23. 23
Experiment III – INRIA Holidays Dataset
 Categories: Personal Holidays Photos
 Data Sets: 500 Queries (1 per group) & 1491 Inventory Images
 Ground Truth: Human Annotations

24. 24
MAP Performance

25. 25
Computational Costs
Feature Extraction Time 10 ms
Retrieval Time 80 ms
Feature Vector Size 196 Bytes
Memory Required 190 MB
Machine Stats: 24 GB RAM, 2.53GHz
Index Size: 1M+

26. 26
Summary
 Color a fundamental cue
 Spatial Prior can eliminate need for expensive
background removal
 Future work to focus on efficient descriptors

27. 27
Questions?