Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context
•
0 gefällt mir•821 views
This document summarizes a research paper that extends the PhotoTOC algorithm to cluster photos of social events using richer contextual information. The approach first sorts photos temporally and splits them into mini-clusters, then sequentially merges mini-clusters based on weighted similarities across time, location, tags, and user information. Experiments on the ReSEED dataset show the approach can group related photos while handling diversity in users, locations, and qualities, though it may still incorrectly split or merge some clusters. Future work is needed to optimize parameters, improve the fusion approach, and make criteria event-dependent.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (17)
Ähnlich wie Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context
Ähnlich wie Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context (20)
Mehr von Universitat Politècnica de Catalunya
Mehr von Universitat Politècnica de Catalunya (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context
- 1. Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context Daniel Manchón-Vizuete Irene Gris-Sarabia Xavier Giró-i-Nieto ICMR 2014 Workshop on Social Events in Web Multimedia, Glasgow (Scotland/UK), 01/04/2014
- 2. Outline Motivation Related work Approach Results Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 3. Motivation
- 4. Motivation
- 5. Motivation Credit: National Institute on Aging and FDA.
- 6. Motivation
- 7. Motivation Credit: Alan Smeaton and Cathal Gurrin (Dublin City University, 2013)
- 8. Outline Motivation Related work Approach Results Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 9. Related work PhotoTOC [Platt et al, PACRIM 2003]
- 10. Related work [Becker, Naaman & Gravano, ACM WSDM 2010] Rich context User-provided annotations Automatically generated information Title Description Creation time Geo-location
- 11. Outline Motivation Related work Approach Results Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 12. Approach (a) Temporal sorting by each user independently Hi, I’m John. Hi, I’m Emily.
- 13. Approach (b) Temporal-based oversegmentation in mini-clusters PhotoTOC [Platt et al, PacRim 2003]
- 14. Approach (b) Temporal-based oversegmentation in mini-clusters
- 15. Approach (c) Sequential merging of mini-clusters ? t avg(·) avg(·) avg(·)avg(·)
- 16. Approach (c) Sequential merging of mini-clusters Weighted modalities ● creation (or upload) time ● geolocation ● textual labels ● same user
- 17. Approach (c) Sequential merging of mini-clusters Geolocation (d=haversine)Time stamp (d=L1) Text labels (d=Jaccard) Same user (d=boolean)
- 18. Approach (c) Sequential merging of mini-clusters
- 19. Approach (c) Sequential merging of mini-clusters Mean and std. deviation learned on pairs of photos within the same training event.
- 20. Approach (c) Sequential merging of mini-clusters phi function
- 21. Approach (c) Sequential merging of mini-clusters decision threhold
- 22. Approach (c) Sequential merging of mini-clusters
- 23. Outline Motivation Related work Approach Experiments Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 24. Dataset ReSEED - Social Event Detection Dataset Timo Reuter, Symeon Papadopoulos, Vasilios Mezaris & Philipp Cimiano (ACM MM Sys 2014) [Paper: http://dx.doi.org/10.1145/2557642.2563674] [Dataset: http://greententacle.techfak.uni-bielefeld.de/reseed/]
- 25. Results
- 26. Results
- 27. Results Diversity of cameras (qualities)
- 29. Results Visual redundancy (not exploited)
- 32. Outline Motivation Related work Approach Experiments Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 33. Future work ● Median-based normalisation for robustness against outliers. ● Better optimisation of parameters K and d. ● Improve fusion approach (SVM ?). ● Event-dependent merging criteria. ● Efficient use of visual features (higher computation).
- 34. Outline Motivation Related work Approach Experiments Future work Conclusions Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto
- 35. Conclusions ● All considered context (geo, user, tags) can help. ● Watch out for outliers. ● Divide and conquer… sequentially. Thank you SEWM !