Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Know Your Gatekeepers: How to Sell Taxonomy as a Product (IAC 2019)

221 Aufrufe

Veröffentlicht am

IA projects often need approval from gatekeepers who don’t fully understand your complex project. In this talk, a product manager & a taxonomist will present a case study of the tools they used to convince designers, engineers, and analysts to buy into a complex data & IA project. Learn how a team of 4 people with no engineers or designers made a business case to launch sitewide filters on Etsy.com.

They’ll discuss how they identified gatekeepers, learned about the dependencies between different systems, struggled to maintain visibility with a team of remote employees, how they put themselves on the roadmaps of gatekeepers already overloaded with goals of their own, learned how to gain access to analytics to inform design and measure success, and successfully ran a winning experiment to help gatekeepers understand that their complex project was worth saying yes to.

This talk was given at Information Architecture Conference in Orlando on March 16, 2019.

Veröffentlicht in: Internet
  • Als Erste(r) kommentieren

Know Your Gatekeepers: How to Sell Taxonomy as a Product (IAC 2019)

  1. 1. 1 Liz Graefe & Matt Nicole March 16, 2019 Know Your Gatekeepers How to Sell Taxonomy as a Product Agenda 1 - Intros 2 - What were we trying to do? 3 - Finding our gatekeepers 4 - Appeasing our gatekeepers 5 - Where are we now? 6 - What would we do different?
  2. 2. 2 MattLiz
  3. 3. Etsy is a marketplace Etsy is home to a universe of special, extraordinary items, from unique handcrafted pieces to vintage treasures. Our mission is to Keep Commerce Human. We believe that our sellers are the experts about their wares, and we empower them with control over their content. Translation: Anyone can list anything* and they can call it whatever they want. 3 *Almost. Contrary to that Facebook meme, you can’t actually list your children.
  4. 4. Anything & Everything 4
  5. 5. 5 What were we trying to do? ● Add filters to Etsy ● Just like every other ecommerce website
  6. 6. Filters ● Filters are built on Attributes ● Attributes are built on Categories ● Categories & Attributes are populated by Listings ● Listings are created by Sellers
  7. 7. 7 Our First Gatekeepers: Etsy Sellers
  8. 8. 8 How We Taxonomy at Etsy We don’t enforce categorization. We provide a hierarchy for the seller to classify their products, but classification can happen at almost any level, and accuracy is not enforced. Until recently, it was even possible to classify a product at the top level. We don’t require attributes. We provide category-specific structured attributes, but we rarely require sellers to fill them out. We allow sellers to vary by anything. While we do offer structured variation attributes, sellers have the ability to build their own variation attribute using their own terms.
  9. 9. Family Llama T-shirt by Good4theGoose If our structured metadata isn’t meeting a seller’s needs, in many cases they have the power to create their own attributes and values. Sellers will find a way.
  10. 10. 10 We can’t turn on filters if sellers don’t fill out attributes. But sellers won’t fill out attributes if they don’t see what we’re using them for. Chicken, meet Egg. Any plan to launch filters was going to need communication to sellers so they would understand the intent behind our actions.
  11. 11. 11 Change Communication It all begins with data. In order for any taxonomy change to have a chance at success, the data has to be there to support it. 1. Products have to be classified into categories. 2. Categories need attributes that meet the specifications of the products and the needs of the users 3. Attributes have to be filled out correctly. If the data powering your taxonomy comes from other people, make sure they know how to use it correctly. If something is changing, make sure they’re prepared. Changes need to be communicated.
  12. 12. 12 Our team ● Product Manager ● Dedicated Designer ● Fully Staffed Engineering Team ● Dedicated Analyst ● Taxonomist
  13. 13. 13 Our Second Gatekeeper: Staffing
  14. 14. 14 Our team ● Product Manager ● The role of designer ● The role of Engineering Team ● The role of Analyst ● TaxonomistActual
  15. 15. Nerd Sniping xkcd.com/356
  16. 16. Enumerate the Roles I am not an astronaut.
  17. 17. 17 Be the product manager ● There are Different kinds ○ 6 types ○ 3 different types ○ 7 other kinds ● Product management is about building the right things for the right people. ○ Lower risk, effort, friction ● Many myths
  18. 18. 18 Where are we going? ● Maps ● Navigating
  19. 19. Maps Define the frontier
  20. 20. Maps Find the dragons
  21. 21. Maps Slay the first dragon first.
  22. 22. Maps The perfect is the enemy of the good.
  23. 23. Maps Tell people where you can’t see the trees through the forest.
  24. 24. 24 Our Third Gatekeeper: Engineering Complexity
  25. 25. Navigating The map is not the terrain
  26. 26. Navigating Find the critical path things.
  27. 27. 27 How are we going to get there? ● Who is “we”? ● We work together
  28. 28. “We” Gatekeepers are people too.
  29. 29. Better “We” Spreadsheet to the promised land
  30. 30. 30 Our Fourth Gatekeeper: Organizational Complexity
  31. 31. “We” Humans are messy "Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand." Norm Kerth, Project Retrospectives: A Handbook for Team Review
  32. 32. Scarcity What is value? I’m in your cart.
  33. 33. 33 Selling your product manager ● Talk about the Product Triangle ○ Users (personas) ○ Business (canvas) ○ Technology (lots of links here) ● Talk in impact ○ “We can move this metric” vs ○ “I want to build this thing” ○ Some starter metrics ● Know what a good one looks like ○ SVPG ● Create specific user stories ○ As a taxonomist (or other persona) ○ I’d like to be able to (some capability) ○ In order to (make some impact)
  34. 34. Shared “We” GMS Gross Merchandise Sales
  35. 35. 35 Culture of Experiments ● Does anyone want this? ● Will this contribute to our shared goals? KNOW YOUR GATEKEEPERS
  36. 36. 36 Our Fifth Gatekeeper: Reality
  37. 37. Listen to the users. “I come to Etsy when I have time to kill.” “I will literally click through 250 pages of results in order to find what I’m looking for.”
  38. 38. “I want shoes in the most popular size.” Listen to the users.
  39. 39. No one has ever said this. “I want shoes in the most popular size.”
  40. 40. 40 Issues with Past Experiments Traffic In order to achieve significance in a timely fashion, we created filters that were broad and applied to all categories. And when that didn’t work, we tried adding filters to top-level categories.
  41. 41. Holidays & Occasions
  42. 42. 42 Issues with Past Experiments Traffic In order to achieve significance in a timely fashion, we created filters that were broad and applied to all categories. And when that didn’t work, we tried adding filters to top-level categories. Ordering Values displayed in order of highest inventory. Filters displayed in the order that they were created in. We had no control over how our data was being displayed. Listing Adoption In order for filters to succeed, the values need to meet a user’s needs, but so does the assortment that those values return. If we build a new attribute, it can take months for sellers to update existing listings or create new ones that fill them out.
  43. 43. 43 If our overall goal was just “proving that filters are a win” or just “getting filters defined everywhere regardless of what they are,” then we’d created a vanity metric that would lead us to develop products that aren’t useful. Chasing a “win” was only ever going to lead us to failure.
  44. 44. 44 An Experiment that Worked The Right Filters Everywhere At this point we’d been creating attributes for a few years. Some of our highest-trafficked verticals had category-specific filters with good coverage at all levels of the taxonomy. Filters vs. No Filters This time we didn’t try to test the experience on a single vertical. We went full A/B. Our users would either see all of our filters, or they wouldn’t see any. Old Filters vs. New Filters The experiment closely followed the release of a redesign of the Clothing vertical that included a lot of brand new attributes. We took the opportunity to include these filters, knowing that they might not have the best adoption.
  45. 45. 45 Analytics ● What’s going on in there? ● Now we know.
  46. 46. 46 Our Sixth Gatekeeper: Entropy
  47. 47. In Need of Analytics ● How many of our products use the value April Fools’? ● How many active listings are in Unisex T-shirts? ● How many people use the filters on Wall Art? ● What are the top search queries for Slime? ● How many people browse vs. search? In the past, we had analytics from experiments, which weren’t that helpful. If we wanted other insights we had to ask an analyst and wait until they had time to send us a one- day snapshot of the data we needed.
  48. 48. We Built Ourselves Some Tools
  49. 49. 49 Where are we now? Questions we still can’t answer: ● When users click on filters, which values are they choosing most often? ● Is there any impact to filter engagement or seller attribute adoption if we reorder our attributes? ● How do our listings change over time? ● How often are our listings recategorized? Problems we still face: ● The code that classifies search queries to our taxonomy still doesn’t see all categories ● We don’t have the ability to migrate listings, which means we can’t delete obsolete categories. ● Browse sessions are often logged as search sessions, giving us an incomplete view of how our Buyers are looking for products.
  50. 50. 50 Things we wished we’d done sooner. ● Securing access to Analytics ● Manually ordering attributes and values ● Being able to QA changes on the site before they go live. ● Being able to QA changes in the tools before launching them to the site.
  51. 51. 51 We’re done. Thanks. Liz Graefe - @sistermischief Matt NIcole - @d44pan