As the fastest-growing type of content on the Internet, consumer produced videos are a wealth of information about the world that's essentially untapped. We present ICSI's research on the large-scale video search methods using an application that reveals the geo-location of a consumer- produced video based on its content. Gerald Friedland, University of California, Berkeley.
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
Looking for a Needle in Video Haystack #appsummit14
1. Finding the Needle
in the Video Haystack
Dr. Gerald Friedland
Director Audio and Multimedia Research
International Computer Science Institute Berkeley, CA
friedland@icsi.berkeley.edu
4. Multimedia in the Internet
is Growing
3
• YouTube alone claims 48 72 100 hours
video uploads every minute.
5. Multimedia in the Internet
is Growing
3
• YouTube alone claims 48 72 100 hours
video uploads every minute.
• Youku (Chinese YouTube) claims 80k
video uploads per day
6. Multimedia in the Internet
is Growing
3
• YouTube alone claims 48 72 100 hours
video uploads every minute.
• Youku (Chinese YouTube) claims 80k
video uploads per day
• Flickr, Instagram, Liveleak, Vimeo...
14. The Opportunity
• Consumer-Produced Multimedia allows
empirical studies at never-before-seen
scale:
– sociology,
– medicine,
– economics,
– …
• Problem: Videos need to be searchable
beyond keywords.
5
15. The Opportunity
• Consumer-Produced Multimedia allows
empirical studies at never-before-seen
scale:
– sociology,
– medicine,
– economics,
– …
• Problem: Videos need to be searchable
beyond keywords.•
5
16. Our Approach
6
Ball sound
Male voice (near)
Child’s voice (distant)
Child’s whoop (distant)
Room tone
Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
17. Our Approach
Multimodal exploitation of video content,
including audio and temporal information.6
Ball sound
Male voice (near)
Child’s voice (distant)
Child’s whoop (distant)
Room tone
Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
18. Location Estimation
7
J. Choi, G. Friedland, V. Ekambaram, K. Ramchandran: "Multimodal Location
Estimation of Consumer Media: Dealing with Sparse Training Data," in
Proceedings of IEEE ICME 2012, Melbourne, Australia, July 2012.
19. Bayesian graphical
framework
8
{berkeley,
sathergate,
campanile}
{berkeley,
haas}
{campanile} {campanile,
haas}
Node:
Geoloca7on
of
the
image
Edge:
Correlated
loca7ons
(e.g.
common
tag)
Edge
Poten,al:
Strength
of
an
edge,
(e.g.
posterior
distribu7on
of
loca7ons
given
common
tags)
p(xi, xj|{tk
i } {tk
j })
p(xj|{tk
j })p(xi|{tk
i })