Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

CBMI 2013 Presentation: User Intentions in Multimedia

1.026 Aufrufe

Veröffentlicht am

Veröffentlicht in: Technologie, Unterhaltung & Humor

CBMI 2013 Presentation: User Intentions in Multimedia

  1. 1. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  2. 2. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  3. 3. Query By Example• User has particular information need• Need reflected by example image• Query is expressed visually
  4. 4. We all know that ...• Some features work better than others• Features have different characteristics• Some features work out well for somedomains, while others don’t
  5. 5. PHOG & Flowers
  6. 6. ColorLayout & Sunsets
  7. 7. EdgeHistogram & Portraits
  8. 8. JCD & Portraits
  9. 9. ColorLayout & Landscapes
  10. 10. PHOG & Birds on the Water
  11. 11. Which one is right?• How to determine the right feature?• What are the necessary characteristics?• How do I define visual similaritywithin the domain?• What is visual similarityfor the user?
  12. 12. Why is there a differentranking?
  13. 13. Users in Context
  14. 14. Definition: Context“Context is any information that can be usedto characterize the situation of an entity.An entity is a person, place, or object thatis considered relevant to the interactionbetween a user and an application, includingthe user and applications themselves.”Ref. G. Abowd et al., “Towards a better understandingof context and context-awareness. In Handheld andUbiquitous Computing, vol. 1707 LNCS, 1999.
  15. 15. Definition: Intentionnoun(1) thing intended; an aim or plan(2) Medicine the healing process of a wound(3) (intentions) Logic conceptions formedby directing the mind towards an object
  16. 16. Context vs. Intention?Context is anyinformation that canbe used tocharacterize thesituation of anentity. An entity is aperson, […]noun(1) thing intended; anaim or plan[…]
  17. 17. A User’s Intention is• part of a user’s context• of manageable size (verb & frame)• related to the information need in searchExamples• I want to download a new background for my mobile.• I want to share the first laugh of my daughter.• I want to see what a Lancia Lyra looks like.
  18. 18. User Intentions in the WebUnderlying goals of web searches• Informational– to learn / know something• Navigational– to go to a specific place (on the web)• Transactional– to go somewhere to ultimately buy sth.Ref. Broder: A Taxonomy of Web Search. SIGIR Forum,2002
  19. 19. User Intentions in the WebRef. Rose & Levinson: Understanding user goals in websearch, WWW 2004Broder Survey Broder LogAnalysisR&L St udy 1 R&L St udy 2 R&L St udy 324,5 2014,7 11,7 13,539 48 60,961,3 61,5363024,3 27 25Navigat ionalInf ormat ionalTransact ional
  20. 20. User Intentions nMultimedia• Search• Production• Sharing• Archiving• Image• Video• Audio• Multiple modalities
  21. 21. Hand-picked ExamplesRight now there is no all-in-one publicationon user intentions in multimedia …
  22. 22. User Intentions inImage SearchRef. Lux, Kofler & Marques: A classification scheme foruser intentions in image search, CHI 2010beautiful sunset for thebackground of my mobileold vs. newStarbucks logomy friend’s newcar on flickr.comLatest “explore”photos
  23. 23. Do queries help with thesearch intention?User information need vs. query formulation invideo search.• How to support users with video indexing andsearch methods?• Search goal failure is (partially) predictable– Based on keywords and– Based on natural languageRef. Kofler, Larson & Hanjalic: To Seek, Perchance toFail: Expressions of User Needs in Internet VideoSearch, ECIR 2011
  24. 24. Asking for the “Why?”behind the “What?”Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012Hear others singing …Learn to sing …
  25. 25. Asking for the “Why?”behind the “What?”• Information– news, commercial, advertisement, documentary, science,commentary, education, learning, …• Experience– tutorial, how-to, advise, help, training• Affect– books, podcast, music, comedy, series, art, movie, action,gaming, film, episode, entertainment, …Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012
  26. 26. Helping with clever Uis?Ref. Lagger, Lux & Marques: An Adaptive Video RetrievalSystem Based On Recent Studies On User IntentionsWhile Watching Videos Online. ACM CIE, online.
  27. 27. Who are the Users in aVideo Search System?• Study on users of– YouTube– BBC iPlayer– Uitzending GemistRef. Kemman, Kleppe& Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  28. 28. Who are the Users in a VideoSearch System?Ref. Kemman, Kleppe & Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  29. 29. Why do People make Videos?• Study on four main goals:– Affection, Function, Sharing & Preservation.Ref. Lux & Huber: Why did you record this video?WIAMIS 2012Sharing Af f ect ion Funct ion- 0,59 - 0,7 8 - 0,36- 0,05 - 0,26 - 0,360,39 0,84 0,55- 0,50 - 0,930,25 - 0,070,46 0,21- 0,43- 0,210,47Preservat ionSharingAf f ect ion
  30. 30. Finding User Intentions &Goals is a hard task ….
  31. 31. Demand Media – TheAnswer Factory• Demand mined from search queries• Requests for content put on auction• Contractors create content• Crowd does quality controlsee i.e. eHow.comThe Answer Factory: Demand Media and the Fast,Disposable, and Profitable as Hell Media Model,http://www.wired.com/magazine/2009/10/ff_demandmedia/
  32. 32. Human Computation
  33. 33. Human Computation
  34. 34. Human Computation• Is crowd-sourcing of any help here?– cp. ACM MM & work of Kofler, Larson &Hanjalic
  35. 35. Crowd-Sourcing• It’s hard to judge intentions of others– That makes it error prone“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  36. 36. turkersRecall situation 2 2 0 1 0 2Preserve good feeling 2 -2 1 0 0 1Publish online 2 0 0 0 1 2Show to family & friends 2 1 2 1 1 0Support task of mine 0 -2 1 1 1 -2Preserve bad feeling -2 0 -2 0 -2 -2“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  37. 37. Crowd-Sourcing• Turkers disagreed with original publishers.• But pretests had better inter-rateragreements.Ref. Lux, Taschwer & Marques: A Closer Look atPhotographers’ Intentions: a Test Dataset, CrowdMM WSat ACM MM 2012Int ent ions Ot hert urkers 0,147 0,232pret est s 0,57 1 0,510
  38. 38. Human Computation• How about motivating people, i.e. withfun & rewarding experience?
  39. 39. Games with aadditional Purpose
  40. 40. Games with aadditional Purpose• Tag a Tune• Popvideo• Matchin• Flip It• Verbosity
  41. 41. Games with aadditional Purpose• How to go beyond annotation?– classical applications are focused on annotation• How to infer user intentions?– proves to be hard to “guess” intentions of others– even “own” intentions may not be explicit• How to leverage user intentions?– finding which intentions can be leveraged and which goalscan be supported
  42. 42. Leveraging EducationalNeeds …
  43. 43. Where did we go?• CBIR & QBE• User Intentions– Search– Production– Sharing• Games with additionalPurpose
  44. 44. What is left?• Lots of loose ends & open grounds forresearch …… let me propose four different PhDtheses …
  45. 45. Open PhD Theses I• General Model for User Intentions &Goals in Multimedia.– Is there a unified model?– What are the class cardinalities?– How to map production, archiving, search andsharing intentions?
  46. 46. Open PhD Theses II• GWAP, HC & UIs for determining & inferring &utilizing User Intentions & Goals– Which UI elements, game mechanics and HC mechanicshelp in this scenario?– What are appropriate design patterns and scenarios?– What is an appropriate research methodology andhow to (easily) evaluate?
  47. 47. Open PhD Theses III• Bringing Context to the Query inMultimedia Information Systems.– How to utilize Intentions & Goals withinsearch and indexing methodology?– Building MMIS around a model for userintentions.
  48. 48. Open PhD Theses IV• Adaptable Applications– How to adapt an application to users’intentions?– Which elements & process to display, etc.?
  49. 49. Thanks for listening …• Mathias Lux• Klagenfurt University, AT• mlux@itec.uni-klu.ac.at