Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Bridging AI and UX
Methods at Etsy
_________
IA Summit
March 25, 2017
Gio
Staff Engineer
Jenny
Taxonomist
Jill
Staff UX
Researcher
Etsy is a global marketplace where people around the world connect,
both online and offline, to make, sell and buy unique ...
By the Numbers
1.7M
active sellers
AS OF DECEMBER 31, 2016
28M
active buyers
AS OF DECEMBER 31, 2016
$2.84B
annual GMS
IN ...
SELLER
Pursues craft,
grows business
ETSY
Invests in the
platform and delivers
a global base of
buyers
ETSY
Facilitates th...
SELLER
Pursues craft,
grows business
ETSY
Invests in the
platform and delivers
a global base of
buyers
ETSY
Facilitates th...
“e-Search”
A case study
8
Frame the
problem
“E-Search” - A case study
WINTER 2015
Iterative
evaluation
Concept
testing
SPRING 2014 SUMMER 2014
“E-Search” - A Case Study
Parallel streams of effort
UX RESEARCH
UX Research
13
Etsy.com
What are people
shopping for on Etsy?
People shop for ideas on Etsy
17
GIFTS LIFE EVENTS
SEASONAL
CELEBRATIONS
STYLE/EMOTION
POP CULTURE/
TRENDS
NOT “known item...
People shop for ideas on Etsy
18
GIFTS LIFE EVENTS STYLE/EMOTION
NOT “known item” searches
Often, using broad, thematic se...
Framing the problem
19
Existing “browse”pages
were low performers, i.e.
not converting.
PROBLEM HOW MIGHT WE…
Make finding on Etsy
easier with a ...
Customers are doing
broad searches and
getting “overwhelmed”
with search results.
PROBLEM HOW MIGHT WE…
Turn broad search
...
UX research as the new kid in town
FEB
2014
MARCH
2014 MARCH
2015
JUNE
2015
JULY
2014
AUGUST
2014
● Etsy was founded in 20...
Etsy almost solely
relied on AI and
experimentation to frame
and solve problems.
PROBLEM HOW MIGHT WE…
Use this new resear...
Jewelry
8M items
Necklaces
3M items
GOAL:
Observe “jewelry” search
behavior on- and off- Etsy
to introduce product teams
to shoppers’ expectations
and needs
T...
Goals
1 2 3
UX Insights
Searching on Etsy is conceptually different
Goals
1 2 3
Highly visual
UX Insights
Searching on Etsy is conceptually different
“I’m not on track”
“I’m looking for some...
Goals
1 2 3
Highly visual Use search to browse
UX Insights
Searching on Etsy is conceptually different
“I’m not on track”
...
Goals
1 2 3
Highly visual Use search to browse
UX Insights
Searching on Etsy is conceptually different
“I’m not on track”
...
Guiding Principles
Visual
Exploratory
Surprise
JOMO
E-Search
SINGLE CATEGORY: JEWELRY > NECKLACE CROSS-CATEGORY: MID-CENTURY MODERN, MOTHER’S DAY
Concept testing (2x)
Goals
1 2 3
UX Insights
Until items feel more similar than different, the more categorization the better
Goals
1 2 3
Less FOMO
UX Insights
"It lets me see how it's
organized and go on.
It guides you."
"It's nice the categories ...
Goals
1 2 3
Less FOMO
UX Insights
"It lets me see how it's
organized and go on.
It guides you."
"It's nice the categories ...
Goals
1 2 3
Less FOMO
UX Insights
"It lets me see how it's
organized and go on.
It guides you."
"It's nice the categories ...
UX Vision
Find more, search less
Ye Olde Librarian Knowledge
Berrypicking
“the query is satisfied...by
a series of selections of individual
references and bits of
information at each ...
UX Vision
Question:
Can we build that?
Jousting Windmills
43
Jewelry
“silver necklace”
AI Jewelry!
Query Classification
Query Classification
Document Classification
Jewelry > Necklaces > Charm Necklaces
Document Classification
49
Disaster
50
Goals
1 2 3
Data is Noisy
Disaster
AI
Data is Noisy
AI
Data is Noisy
54
“turtle costume”
55
“turtle costume”
56
“hose”
57
“hose”
58
“chair”
59
“chair”
Goals
1 2 3
Data is Noisy
Disaster
Expensive False
Positives
“sunglasses”
“shoes”
Goals
1 2 3
Data is Noisy
Disaster
Expensive False
Positives
Sellers Last
Sellers last
64
Jousting Windmills
UX RESEARCH
AI
Reboot
GOOD DATA USER AGENCY
Reboot
TAXONOMY
Reboot
UX RESEARCH
AI
UX+AI+TAXO
Reboot
69
A taxonomy to serve
both user groups
70
TAXONOMY
MERCH MARKETING POLICY
ENG MAKERS
LEGAL DATAUX
Hat?
● Handmade > Clothing > Dresses
● Handmade > Women > Clothing >
Dresses
● Handmade>Weddings>Dresses
● Vintage > Clothing >...
title
Knitting Technique
Glass Material
Geekery Subculture
Vintage Age
Children Who
Holidays When
Old Categories Aspect
Seller Control is Good UX
Not One. Two Taxonomies
Seller Control
Metadata is Emotional
Sellers
Buyers
Communication is Product
• Transparent
• Art not SEO
• Translatable
• Flexible
• Evolves
• Buyer Friendly
• Powers Structured Data
85
The Power of ...
AI +
Inventory
Data
Taxonomy
AI + Search
Epilogue
87
Takeaways
89
Data is queen.
90
TAKEAWAYS
AI is not enough.
91
TAKEAWAYS
Holistic solutions require
multi-disciplinary thinking.
92
TAKEAWAYS
Respect and trust your users.
93
TAKEAWAYS
Thanks!
Gio
giokincade@gmail.com
@giokincade
Jenny
jbenevento@etsy.com
@jennybento
Jill
jfruchter@etsy.com
@jillfruchter
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
 A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy
Nächste SlideShare
Wird geladen in …5
×

A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy

1.960 Aufrufe

Veröffentlicht am

video available here: https://blueprintdigital.com/ia-summit-2017/jenny-benevento-giovanni-fernandez-kincade-jill-fruchter/

This was a talk given at IA Summit 2017 in Vancouver, BC by Jenny Benevento, Gio Fernandez-Kincade, and Jill Fruchter.

Etsy is a marketplace where people around the world connect, both online and offline, to make, sell and buy unique goods. Etsy is also a tech company that invests in the craft of coding and data-driven product development as a strategic priority. Etsy has employed AI and machine learning to tackle personalization, recommendations, image understanding, item similarity, search relevance, spelling correction, and many other tasks. We’ll talk through several examples of how Etsy leverages data, where it’s excelled, and where this hammer hasn’t quite hit the nail on the head.

We will be asking ourselves hard questions, recognizing the limitations of decisions driven purely by big data:
- Who are we satisfying? Our customers or our mathematical models?
- Are those models even an accurate reflection of the outcomes we want?
- In a dual marketplace, where complex changes depend on interactions between both sides of the market, can one metric or measure of success tell the full story?
- How do we consider the impact our models are having on our users?
- Are we even addressing real human needs and motivations in the first place?
- How do we inform and enrich AI with expert created & applied taxonomy & metadata?

Veröffentlicht in: Daten & Analysen
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Hello! High Quality And Affordable Essays For You. Starting at $4.99 per page - Check our website! https://vk.cc/82gJD2
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

A Taxonomist, a Software Engineer, and a UX Researcher Walk Into a Bar: Bridging AI and User Experience Methods at Etsy

  1. 1. Bridging AI and UX Methods at Etsy _________ IA Summit March 25, 2017
  2. 2. Gio Staff Engineer Jenny Taxonomist Jill Staff UX Researcher
  3. 3. Etsy is a global marketplace where people around the world connect, both online and offline, to make, sell and buy unique goods. 3
  4. 4. By the Numbers 1.7M active sellers AS OF DECEMBER 31, 2016 28M active buyers AS OF DECEMBER 31, 2016 $2.84B annual GMS IN 2016 45+M items for sale AS OF DECEMBER 31, 2016 Photo by Kirsty-Lyn Jameson
  5. 5. SELLER Pursues craft, grows business ETSY Invests in the platform and delivers a global base of buyers ETSY Facilitates the transaction BUYER Finds unique goods that are hard to find elsewhere ETSY EMPOWERMENT LOOP ETSY EMPOWERMENT LOOP 6
  6. 6. SELLER Pursues craft, grows business ETSY Invests in the platform and delivers a global base of buyers ETSY Facilitates the transaction BUYER Finds unique goods that are hard to find elsewhere ETSY EMPOWERMENT LOOP ETSY EMPOWERMENT LOOP 7 SEARCH
  7. 7. “e-Search” A case study 8
  8. 8. Frame the problem “E-Search” - A case study WINTER 2015 Iterative evaluation Concept testing SPRING 2014 SUMMER 2014
  9. 9. “E-Search” - A Case Study
  10. 10. Parallel streams of effort UX RESEARCH
  11. 11. UX Research 13
  12. 12. Etsy.com What are people shopping for on Etsy?
  13. 13. People shop for ideas on Etsy 17 GIFTS LIFE EVENTS SEASONAL CELEBRATIONS STYLE/EMOTION POP CULTURE/ TRENDS NOT “known item” searches
  14. 14. People shop for ideas on Etsy 18 GIFTS LIFE EVENTS STYLE/EMOTION NOT “known item” searches Often, using broad, thematic search terms SEASONAL CELEBRATIONS POP CULTURE/ TRENDS
  15. 15. Framing the problem 19
  16. 16. Existing “browse”pages were low performers, i.e. not converting. PROBLEM HOW MIGHT WE… Make finding on Etsy easier with a combination of search & browse improvements? 20
  17. 17. Customers are doing broad searches and getting “overwhelmed” with search results. PROBLEM HOW MIGHT WE… Turn broad search queries into satisfying experiences of finding and discovery? 21
  18. 18. UX research as the new kid in town FEB 2014 MARCH 2014 MARCH 2015 JUNE 2015 JULY 2014 AUGUST 2014 ● Etsy was founded in 2005 ● No UX Research until 2014 (Thick data) ● Since 2010, data-driven design and evaluation (Big data) RESEARCH BRAND DESIGN PRODUCT DESIGN
  19. 19. Etsy almost solely relied on AI and experimentation to frame and solve problems. PROBLEM HOW MIGHT WE… Use this new research capacity to help frame the problem and build a better product? 23
  20. 20. Jewelry 8M items
  21. 21. Necklaces 3M items
  22. 22. GOAL: Observe “jewelry” search behavior on- and off- Etsy to introduce product teams to shoppers’ expectations and needs TEST STIMULI: - The Internet - Self-guided searches Contextual observation (2x)
  23. 23. Goals 1 2 3 UX Insights Searching on Etsy is conceptually different
  24. 24. Goals 1 2 3 Highly visual UX Insights Searching on Etsy is conceptually different “I’m not on track” “I’m looking for something to catch my eye”
  25. 25. Goals 1 2 3 Highly visual Use search to browse UX Insights Searching on Etsy is conceptually different “I’m not on track” “I’m looking for something to catch my eye” “6PM is like running into the drugstore. On Etsy, I’m always finding surprises.” “It’s like watching Reality TV. I like the process.”
  26. 26. Goals 1 2 3 Highly visual Use search to browse UX Insights Searching on Etsy is conceptually different “I’m not on track” “I’m looking for something to catch my eye” “6PM is like running into the drugstore. On Etsy, I’m always finding surprises.” “It’s like watching Reality TV. I like the process.” FOMO “Even if it were $10,000. I want to know about it.” “Patience on Etsy is rewarded.”
  27. 27. Guiding Principles Visual Exploratory Surprise JOMO
  28. 28. E-Search SINGLE CATEGORY: JEWELRY > NECKLACE CROSS-CATEGORY: MID-CENTURY MODERN, MOTHER’S DAY Concept testing (2x)
  29. 29. Goals 1 2 3 UX Insights Until items feel more similar than different, the more categorization the better
  30. 30. Goals 1 2 3 Less FOMO UX Insights "It lets me see how it's organized and go on. It guides you." "It's nice the categories are right here. It prompts me to narrow down an extremely broad search." Until items feel more similar than different, the more categorization the better
  31. 31. Goals 1 2 3 Less FOMO UX Insights "It lets me see how it's organized and go on. It guides you." "It's nice the categories are right here. It prompts me to narrow down an extremely broad search." Until items feel more similar than different, the more categorization the better More intuitive “I’m making more progress with minimal effort.”
  32. 32. Goals 1 2 3 Less FOMO UX Insights "It lets me see how it's organized and go on. It guides you." "It's nice the categories are right here. It prompts me to narrow down an extremely broad search." Until items feel more similar than different, the more categorization the better More intuitive “I’m making more progress with minimal effort.” More “Etsy” “Expands for me what is on Etsy because I haven't thought of that being there.”
  33. 33. UX Vision Find more, search less
  34. 34. Ye Olde Librarian Knowledge
  35. 35. Berrypicking “the query is satisfied...by a series of selections of individual references and bits of information at each stage of the ever-modifying search...retrieval of this sort is here called berrypicking.” Source: Marcia Bates (1989). "The Design of Browsing and Berrypicking Techniques for the Online Search Interface."
  36. 36. UX Vision Question: Can we build that?
  37. 37. Jousting Windmills 43
  38. 38. Jewelry
  39. 39. “silver necklace” AI Jewelry! Query Classification
  40. 40. Query Classification
  41. 41. Document Classification
  42. 42. Jewelry > Necklaces > Charm Necklaces Document Classification
  43. 43. 49
  44. 44. Disaster 50
  45. 45. Goals 1 2 3 Data is Noisy Disaster
  46. 46. AI Data is Noisy
  47. 47. AI Data is Noisy
  48. 48. 54 “turtle costume”
  49. 49. 55 “turtle costume”
  50. 50. 56 “hose”
  51. 51. 57 “hose”
  52. 52. 58 “chair”
  53. 53. 59 “chair”
  54. 54. Goals 1 2 3 Data is Noisy Disaster Expensive False Positives
  55. 55. “sunglasses”
  56. 56. “shoes”
  57. 57. Goals 1 2 3 Data is Noisy Disaster Expensive False Positives Sellers Last
  58. 58. Sellers last 64
  59. 59. Jousting Windmills UX RESEARCH AI
  60. 60. Reboot GOOD DATA USER AGENCY
  61. 61. Reboot TAXONOMY
  62. 62. Reboot UX RESEARCH AI UX+AI+TAXO
  63. 63. Reboot 69
  64. 64. A taxonomy to serve both user groups 70
  65. 65. TAXONOMY MERCH MARKETING POLICY ENG MAKERS LEGAL DATAUX
  66. 66. Hat?
  67. 67. ● Handmade > Clothing > Dresses ● Handmade > Women > Clothing > Dresses ● Handmade>Weddings>Dresses ● Vintage > Clothing > Dresses ● Vintage > Women > Clothing > Dresses ● Weddings > Dresses ● Women > Clothing > Dresses
  68. 68. title Knitting Technique Glass Material Geekery Subculture Vintage Age Children Who Holidays When Old Categories Aspect
  69. 69. Seller Control is Good UX
  70. 70. Not One. Two Taxonomies
  71. 71. Seller Control
  72. 72. Metadata is Emotional
  73. 73. Sellers Buyers
  74. 74. Communication is Product
  75. 75. • Transparent • Art not SEO • Translatable • Flexible • Evolves • Buyer Friendly • Powers Structured Data 85 The Power of Two Taxonomies
  76. 76. AI + Inventory Data Taxonomy AI + Search
  77. 77. Epilogue 87
  78. 78. Takeaways 89
  79. 79. Data is queen. 90 TAKEAWAYS
  80. 80. AI is not enough. 91 TAKEAWAYS
  81. 81. Holistic solutions require multi-disciplinary thinking. 92 TAKEAWAYS
  82. 82. Respect and trust your users. 93 TAKEAWAYS
  83. 83. Thanks!
  84. 84. Gio giokincade@gmail.com @giokincade Jenny jbenevento@etsy.com @jennybento Jill jfruchter@etsy.com @jillfruchter

×