Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Crowdsourcing Visual Ads

55 Aufrufe

Veröffentlicht am

Images have the power to convey messages in striking and memorable ways. Although constructing visual messages is currently too hard for computers or novice users, by combining the intelligence of people and computers we can create compelling visual messages computationally. In this talk, we present VisiBlends, a flexible workflow for creating visual blends that follows the design process with steps involving brainstorming, synthesis, and iteration. An evaluation of the workflow shows that (1) decentralized groups of people can generate blends in independent microtasks, (2) co-located groups can collaboratively make visual blends for their own messages, and (3) VisiBlends improves novices’ ability to make visual blends.

We discuss how to decompose other complex tasks so that people and computers can collaborate in generating novel, useful and creative solutions to problems.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Crowdsourcing Visual Ads

  1. 1. Crowdsourcing Visual Ads Lydia Chilton Columbia CS Department, DSI Computational Social Science Seminar 25 January 2019
  2. 2. Ads are interesting because they convey a message visually
  3. 3. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors How can we systemically create visual metaphors?
  4. 4. Visual Metaphors are a creative design problem. There is no simple recipe. Design process: • Brainstorm, • Prototype many ideas, • Select the best ones • Iterate to improve designs Ideate Evaluate Prototype
  5. 5. Outline: Crowdsourcing Visual Ads • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  6. 6. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors
  7. 7. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching Design Pattern: Single Shape Mapping
  8. 8. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching A creative problem is now a search problem: For two concepts, search for symbols that meet this constraint
  9. 9. Starbucks + Summer
  10. 10. Prototype blend
  11. 11. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
  12. 12. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable Output: A visual blend
  13. 13. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable
  14. 14. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  15. 15. A) Brainstorm for summer B) Find and annotate images for summer C) Matching algorithm Starbucks + summer D) Automatic blends + human evaluation Starbucks + summer matches = [] for a in summer_symbols: for b in starbucks_symbols: a_ratio = a.height / a.width b_ratio = b.height / b.width ratios = sort(a_ratio, b_ratio) if (a.shape == b.shape) and (a.coverage != coverage) and (ratios[0] >= 0.5*ratios[1]): matches.push([a, b]) return matches VisiBlends: A web interface to collaboratively make blends
  16. 16. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  17. 17. Decompose the pipeline Five people Five people One person One person One person One person One person Computer: matching and blending algorithms
  18. 18. Rules for finding images are complex
  19. 19. Our first attempt failed. People weren’t finding the right images.
  20. 20. To fix it, we trained people on the pipeline steps backwards. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline
  21. 21. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy
  22. 22. When we taught the process backwards, users were successful
  23. 23. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  24. 24. Study 2: Joe’s Coffee + Night “Joe’s Coffee: Open Late” Ad
  25. 25. Study 2: Hand-washing + smart “Wash your hands. It’s the smart move.” PSA
  26. 26. Study 2: Women + CS “Panel Discussion: Women in Computer Science” Ad:
  27. 27. Study 2: Football + Dangerous “Football linked to lasting brain damage.” News
  28. 28. Study 2: Philosophy + Christmas “Join the Philosophy Dept’s Holiday Celebration” Ad:
  29. 29. Groups can collaboratively make blends for their own messages What goes wrong? (why do we need iteration?)
  30. 30. Football + Dangerous
  31. 31. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  32. 32. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  33. 33. NYC + Healthy
  34. 34. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  35. 35. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  36. 36. Lego + Healthy
  37. 37. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  38. 38. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  39. 39. Orange + Healthy
  40. 40. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers Part of object Shape covers All of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  41. 41. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Evaluate Blend Exercise equipment
  42. 42. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Exercise equipment
  43. 43. When do we need to iterate? No matches Improve object fit Objects are not identifiable + = + = Within same search space, meet other constraints. Find versions of an object with different color, style, aspect ratio Focus on meeting a specific constraint: Find symbols with a different shape Search in a new subspace Find symbols with a different shape
  44. 44. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  45. 45. Controlled Study • 13 undergraduates • Each made 6 blends • Control-first condition • 3 blends without VisiBlends • 3 blends with VisiBlends • System-first condition • 3 blends with VisiBlends • 3 blends without VisiBlends • How many successful blends could they make?
  46. 46. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made
  47. 47. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made 0.56 5.56 (t(18)=4.88, p<0.001) 0.675.67 (t(21)=5.84, p<0.001)
  48. 48. Why is this task so hard? There are a lot of constraints. Novices without the system focused on meeting one constraint at the expense of others. Usually they, found symbols, and then forced them together.
  49. 49. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  50. 50. CinematographyNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
  51. 51. Material Science Kristin Persson, Lawrence Berkeley National Laboratory
  52. 52. Summary
  53. 53. Advertisements can convey messages visually.
  54. 54. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching by using an abstract design pattern to turn it into a search problem. We can decompose design problems
  55. 55. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy But…
  56. 56. Knowledge of the pipeline is needed to motivate the rules, and fill in gaps. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline1 2 3
  57. 57. Groups can collaboratively make blends for their own messages But…
  58. 58. Iteration is necessary to: Fix small problems like aesthetic refinement Fix bigger problems by refining the search for different shapes Fix failure cases with unforeseen problems by re-directing the search for different symbols
  59. 59. 10x improvement in novices’ ability to make blends
  60. 60. Materials ScienceNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
  61. 61. Future Work Adding computational assistance to: • Brainstorming • Finding symbols • Annotation • Evaluation Improving the blend quality
  62. 62. Crowdsourcing Visual Ads Lydia Chilton chilton@cs.columbia.edu Columbia CS + DSI We can use design patterns to turn design problems into search problems. Iteration is necessary to meet all search constraints Groups can collaborate on visual blends 10x improvement to novices ability 1. Two objects are integrated into one object Visual Blending Structure: Whole-to-Part Shape Matching

×