This document discusses using machine learning techniques to find similar items across large, diverse datasets. It describes using learned similarity metrics for efficient searches of images, time series, and tables. Graph-based techniques like personalized page rank are used to generate richer similarities between items and allow clustering. The goal is to develop scalable similarity search to enable applications like geo-tagging photos and finding matching products.

1. Searching for Similar Items in Diverse
Universes
Large-scale machine learning at Google
Corinna Cortes
Google Research
corinna@google.com
1

2. pageMachine Learning at Google 2014
Browse for Fashion
2

3. pageMachine Learning at Google 2014
Browse forVideos
3

4. pageMachine Learning at Google 2014
Outline
Metric setting
• Using similarities (efficiency)
• Learning similarities (quality)
Graph-based setting
• Generating similarities
4

5. pageMachine Learning at Google 2014
Image Browsing
Image browsing relies on some measure of similarity.
5
⎛
⎜
⎜
⎜
⎝
x11
x12
...
x1N
⎞
⎟
⎟
⎟
⎠
⎛
⎜
⎜
⎜
⎝
x21
x22
...
x2N
⎞
⎟
⎟
⎟
⎠
Sim(x1, x2) =
⎛
⎜
⎜
⎜
⎝
x11
x12
...
x1N
⎞
⎟
⎟
⎟
⎠
·
⎛
⎜
⎜
⎜
⎝
x21
x22
...
x2N
⎞
⎟
⎟
⎟
⎠

6. pageMachine Learning at Google 2014
Clustering of Images
Compute all the pair-wise distances between related
images and form clusters:
6

7. pageMachine Learning at Google 2014
Similar Images - version 0.0
7
Demo

8. pageMachine Learning at Google 2014
Helsinki, some cluster
8

9. pageMachine Learning at Google 2014
“Machine Learning” comes up empty
9

10. pageMachine Learning at Google 2014
The Web is Huge
Image Swirl was precomputed and restricted to top
20K queries.
People search on billions of different queries.
Cannot precompute billions of distances
we need to do something smarter.
10

11. pageMachine Learning at Google 2014
Approximate Nearest Neighbors
Preprocessing:
• Represent images by short vectors, kernel-PCA;
• Grow tree top-down based on ‘random’
projections. Spill a bit for robustness;
• Save the node ID(s) with each image.
At query time:
• propagate down tree to find node ID;
• retrieve other image with same ID;
• rank according to similarity with the short vector
and other meta data.
11

12. pageMachine Learning at Google 2014
Similar Images,Version 1.0
Live Search, Similar Images
Inspect the single clusters
• cluster 700
• cluster 700 + machine learning
• cluster 1000
• cluster 400
12

13. pageMachine Learning at Google 2014
Cluster 700
13

14. pageMachine Learning at Google 2014
Cluster 700 + Machine Learning
14

15. pageMachine Learning at Google 2014
Cluster 1000
15

16. pageMachine Learning at Google 2014
Cluster 1000 + Machine Learning
16

17. pageMachine Learning at Google 2014
Cluster 400
17

18. pageMachine Learning at Google 2014
Cluster 400 + Machine Learning
18

19. pageMachine Learning at Google 2014
Finding Similar Time Series
19

20. pageMachine Learning at Google 2014
Flu Trends
20
http://www.google.org/flutrends/us/#US

21. pageMachine Learning at Google 2014
Flu Trends
21

22. pageMachine Learning at Google 2014
Split time series into N/k chunks:
Represent each chunk with a set of cluster centers
(256) using k-means. Save the coordinates of the
centers, (ID, coordinates).
Save each series as a set of closest IDs, hashcode.
Asymmetric Hashing, Indexing
22

23. pageMachine Learning at Google 2014
For given input u, divide it into its N/k chunks, uj:
• Compute the N/k * 256 distances to all centers.
• Compute the distances to all hash codes:
MN/k additions needed.
The “Asymmetric” in “Asymmetric Hashing” refers
to the fact that we hash the database vectors but
not the search vector.
Asymmetric Hashing, Searching
23
d2
(u, vi
) =
N/k
j=1
d2
(uj, c(vi
j))

24. pageMachine Learning at Google 2014
Google Correlate
http://www.google.com/trends/correlate/
24

25. pageMachine Learning at Google 2014
Outline
Metric setting
• Using similarities (efficiency)
• Learning similarities (quality)
Graph-based setting
• Generating similarities
25

26. pageMachine Learning at Google 2014Machine Learning at Google 2014
Table Search
26
http://webdatacommons.org/webtables/
March 5, 2014: 11 billion HTML tables reduced to 147 million quasi-relational Web tables.

27. Standard Learning with Kernels
The user is burdened with choosing an appropriate kernel.
User
Algorithm
m points
(x, y)
27

28. Learning Kernels
Demands less commitment from the user: instead of a
specific kernel, only requires the definition of a family of
kernels.
User
Algorithm
m points
28

29. pageMachine Learning at Google 2014pageMachine Learning at Google
Two stages:
Outperforms uniform baseline and previous algorithms.
Centered alignment is key: different from notion used by
(Cristiannini et al., 2001).
2014
Centered Alignment-Based LK
K h
29
[C. Cortes, M. Mohri, and A. Rostamizadeh:Two-stage learning kernel methods, ICML 2010]

30. pageMachine Learning at Google 2014Machine Learning at Google 2014
Centered Alignment
Centered kernels:
Centered alignment:
Choose kernel to maximize alignment with the
target kernel:
where expectation is over pairs of points.
KY (xi, xj) = yiyj.

31. pageMachine Learning at Google 2014pageMachine Learning at Google 2014
Alignment-Based Kernel Learning
Theoretical Results:
• Concentration bound.
• Existence of good predictors.
Alignment algorithms:
• Simple, highly scalable algorithm
• Quadratic Program based algorithm.
31

32. pageMachine Learning at Google 2014Machine Learning at Google 2014
Table Search
http://research.google.com/tables
• Machine Learning Conferences
• Marathons USA
Research Tool in Docs
32

33. pageMachine Learning at Google 2014
Outline
Metric setting
• Using similarities (efficiency)
• Learning similarities (quality)
Graph-based setting
• Generating similarities
33

34. pageMachine Learning at Google 2014
Graph-based setting
Personalized Page Rank
• PPR used to densify the graph
• Single-Linkage Clustering used to find clusters in
the graph
34

35. pageMachine Learning at Google 2014
Example
35

36. pageMachine Learning at Google 2014
PPR Clustering
Personalized PageRank (PPR) of :
• Probability of visiting in a random walk starting
at :
• With probability , go to a neighbor
uniformly at random.
• With probability , go back to ;
Resulting graph: single hops enriched with multiple
hops, resulting in a denser graph.Threshold graph at
appropriate level.
36

37. pageMachine Learning at Google 2014
2 Pass Map-Reduce
• Each push operation takes a single vertex u, moves
an fraction of the probability from r(u) onto
p(u), and then spreads the remaining (1 )
fraction within r, as if a single step of the lazy
random walk were applied only to the vertex u.
• Initialization: p = ~0 and r = indicator function at u
Algorithm
37

38. pageMachine Learning at Google 2014
“Single-Linkage” Clustering
Examples
Can be parallelized;
Can be repeated hierarchically.
38

39. pageMachine Learning at Google 2014
Demo, Phileas, Landmark
The goal of this project is to develop a system to
automatically recognize touristic landmarks and
popular location in images and videos. Applications
include automatic geo-tagging and annotation of
photos or videos. Current clients of the service
include Personal Photos Search, and Google Goggles,
Map Explorer
39

40. pageMachine Learning at Google 2014pageMachine Learning at Google 2014
Summary
Challenging very large-scale problems:
• efficient and effective similarity measures
algorithms.
Still more to do:
• learning similarities for graph based algorithms;
• similarity measures vs discrepancy measures:
• match time series on spikes
• match shoes with shoes, highlighting differences.
40