Presentation for Arena. Why is data a designer's best friend?

1. Why is data a designer's best friend?

2. Introduction
My name is Andriy, and I'm a product designer. I was
previously a product designer at Grooveshark, a UI/UX
designer at 6Wunderkinder, and UX designer at Kayak.
Originally from Ukraine, I have also lived and worked in
Germany and the United States.
Twitter: @askyruler Dribbble: @skyruler
Recently, I joined Mixpanel. We help businesses
understand how their customers and users behave in
their products, like apps and websites.

3. How design is all about questions
Case study: Building Lexicon using data
What is product analytics (and why it matters to design)
… and why is data a designer's best friend?
What we’ll cover today:

4. What is product analytics?

5. “How many users that arrive from an ad end up
booking a hotel room in Costa Rica?”
“How often do my most loyal users in the United
States come back?”
“Which feature should I invest in further to drive up
customer conversion?”
“Which A/B test got more people to convert to a
paying customer in our marketplace?”
It’s all about questions

6. “Which feature should I invest in further
to drive up customer conversion?”
It’s all about questions

7. “How many users that arrive from an ad
end up searching for and booking a
hotel room in Costa Rica?”
It’s all about questions

8. “Which A/B test got more people to
convert to a paying customer in our
marketplace?”
It’s all about questions

9. “How often do my most loyal users in
the United States come back?”
It’s all about questions

10. How can you start asking questions?

11. Create a tracking plan before writing any code
Use qualitative research until you get at least 100 users
You need data

12. Case study: Building Lexicon

13. Not trusting or understanding their data
“Natural causes”
Price
Why are our customers churning?

14. Primarily, customers needed two things:
• A shared language around event names, for both
new and old teammates
• Verification that the events entered into the
system were the right events
After looking at both qualitative and quantitative data on
churn, we found out that people sometimes didn’t trust
their data because they didn’t know what everything
meant.
Digging into the trust problem

15. “After the previous round when our internal champion
left the company, the current team has had troubles
locating the original spec and figuring out what events
tied to what actions on the app itself.”
FORTUNE 100 RETAIL COMPANY
Customer feedback

16. What if users did not have to work too hard to
understand what each event meant?
What if users had guidance for which events they
could trust?
What if users had a single place for describing their data
so everyone is on the same page?
How might we build trust?

17. What if users did not have to work too hard to
understand what each event meant?
What if users had guidance for which events they
could trust?
What if users had a single place for describing
their data so everyone is on the same page?
Create a “data definitions” page that allows admins to
share more information about each event (what action
triggers it, where it is sent from, etc)
Update the Mixpanel UI to display data definitions to
users in the context of an actual report
Give admins the ability to “verify” events so all users
will know that they can trust those verified events
Goals Solutions
How we can solve this?

18. Show username and date for verified events
For admins For general users
A “data definitions” page
Admin-“verified” events
Display data definitions in reports
Designing “trust” on both sides

19. Answer: Lexicon
Lexicon

20. Admin wireframes

21. Admin exploration

22. Event name Description VerifiedModified
Add EventsALL VERIFIEDUNVERIFIED
BlockedPage - Add Exception User clicked "Take me back" on the warning screen JJ
Cancel Save150 characters left
AUC protection got temporarily disabled because classification … 2 secondsAUC - Performance - Enabled EL
AUC - Performance - Disabled AUC protection re-enabled after requests taking less than 1 second AS
AUC - Whitelist Error After loading the whitelist or applying a differential update … the hash supplied SD
SearchLexicon
Microsite Project Account Help Center Live Chat Mason Yarnell
Funnels
Sementation
Retention
Formulas
Live View
A/B Testing
Explore
Insight
Notifications
Surveys
Revenue
PEOPLE
ENGAGEMENT
Applications
Admin MVP

23. Dropdown exploration

24. Dropdown prototype

25. Dropdown prototype v2.0

26. RECENTLY VIEWED
Notification sent
$signup
Viewed report
All events
Your Top Events
[Admin internal] Revenue analytics viewed
[Applications] Application link clicked
[Autotrack] iOS10 report
[AT] Read more tweets
[AT] # of events
[Autotrack] iOS 10 report
Segmentation query
ALL EVENTS
Properties
base/DNT.coffee fired in the DNT working code
when a single tracker gets toggled on/off
[AT] Read more tweets
Add definitions to your events with Lexicon
This event was verified by Jonathan L. on 6/13/2017
Reporting UI MVP

27. Reporting UI MVP

28. We launched Lexicon with a limited number of customers
to validate the idea and get data back on how well our
features solved their problems.
We watched, through product analytics, to learn how these
customers used Lexicon in their daily workflows.
Testing our hypotheses

29. 202 projects initially adopted Lexicon
Lexicon usage follows power law distribution:
~25% of users contribute ~80% of definitions
865 event definitions added
Initial beta results

30. “AWESOME”
“We like it!”
“Excited to add qualitative data to [our]
project… Lexicon help[s] with product usability,
product navigation, and communication
between teams.”
Early qualitative feedback

31. Listen to the customers
Use product analytics to define
and track user behavior
Iterate and test
Using data to build better products

32. Next up: building trust with ML features

33. Why is data a designer's best friend?
It helps designers to understand user behavior at scale
It helps designers make better decisions
It helps designers to know who your users are

34. Thank you!