Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

How to use Big Data to drive product strategy and adoption

248 Aufrufe

Veröffentlicht am

Today, billions of activities and interactions happen online. The level of interactions with online applications are getting more complex as well. Within the UX, we have the opportunity to understand collective behavior and various experiences through big data.

Specifically, large scale and strategical directions to products can be determined and evaluated through big data behavioral analysis. In this talk, I will go through various types of research objectives, appropriate methodologies and explain how we can use quantitative methodologies to solve UX and user behavior problems and drive product strategy. In this presentation, I will go through a couple of example case studies and topics where behavioral data can help us better understand users and inform strategic product development.

Presented by Saide Bakhshi

Veröffentlicht in: Präsentationen & Vorträge
  • Als Erste(r) kommentieren

How to use Big Data to drive product strategy and adoption

  1. 1. How to use big data to drive UX strategy? 1 Saeideh Bakhshi Quantitative UX Researcher, Facebook bakhshi@fb.com UXPA 2018
  2. 2. At Georgia Tech College of Computing Started PhD in CS 2009 2013 Focused on behaviors that drove users' engagement with photos Started working at Y! Labs As a Quantitative UX Researcher Started working at Facebook 2016 2012 Got interested in HCI and started working on user engagement Changed thesis topic 2014 And started to work full time at Yahoo Labs Graduated with PhD
  3. 3. Agenda 3 1 How does big data fit into UX? 2 Which big data methods are useful for UX? 3 What are some examples of using big data methods? 4 How is this different from what data scientists/analysts do?
  4. 4. How does big data fit into UX? 1 4
  5. 5. 5 In 1995,Victor Minichiello described how qualitative and quantitative research differ in concept and methodology.
  6. 6. Concept Qualitative Concerned with understanding human behavior from end user's perspective Assumes dynamic and negotiable reality Used to gain an understanding of reasons, opinions, and motivations Less generalizable Quantitative Concerned with understanding behavior from researcher's perspective Assumes a fixed and measurable reality Uses measurable data to formulate facts and uncover patterns More generalizable Minichiello, 1995 Method Qualitative Data is collected through participant observation or interviews Data is presented through the language of the end user More in-depth information on a few cases primary inductive process used to formulate theory or hypotheses Quantitative Data is collected through measuring things Data is presented through the language of statistical analysis Less in-depth but more breadth of information across a large number of cases Primary deductive process used to test pre-specified concepts, constructs, and hypotheses that make There's been a lot of progress made in quantitative research methods since 1995
  7. 7. Advantages of using big data 7 1Actions may tell a different story than words. 2We can find new patterns that were previously hidden. 3 Technology use cases are getting more complex and harder to find patterns. 4A lot of this data is already available.
  8. 8. Big data methods can be useful at various stages of user research StrategicFoundational Predictive and Hypothesis driven Strategic research can specially benefit from big data methods where we need to test hypotheses about behaviors or predict trends. Exploratory & Generative Variety of big data methods (e.g. unsupervised methods) can help us explore behaviors and attitudes associated with product. Characterize use and measure success After a product is launched, big data methods can help us characterize behaviors, measure success and connect the two. Evaluative 8
  9. 9. 9 Theorize/Predict what's next: We can use past behavior to predict future behavior and strategize products around that. Characterize behaviors and estimate success Use unsupervised methods to find patterns and trends Conceptual model around behavior: Based on findings, we can come up with theories on how/why it is happening Test hypotheses Analysis: We can analyze large scale behavioral data to find patterns around the product or topic we are interested in How can behavioral data help us with strategy?
  10. 10. What are some useful machine learning methods we can use with large data? 2 10
  11. 11. 11 Problem type Big data methods Types of data Exploratory exploratory data analysis, unsupervised methods (clustering, assoication rules), text analysis behavioral or user generated data or self reported data Deep dive in an existing behavior supervised methods (regression, classification) behavioral + attitudinal Hypothesis validation Hypothesis testing, supervised methods (regression, classification) behavioral + attidunal Trends forecasting Timeserie analysis, Forecasting, prediction behavioral
  12. 12. Data Preparation, exploration and reduction: Collection, Cleaning and preparing data, bivariate analysis, visualization, dimensionality reduction Prediction: Linear regression, K-nearest neighbors, Regression trees, Neural networks, Ensembles Time-series analysis: Regression based, Smoothing methods Theorize/Predict what's next: Sometimes, we can use past behavior to predict future behavior and strategize around that Classification: K-nearest neighbor, Naive-bayes, Classification trees, Logistic regression, Neural networks, Discriminant analysis, Ensembles What goes together: Association rules, collaborative filtering Segmentation: Clustering Model evaluation and selection: accuracy and performance evaluations Descriptive analysis: bivariate and multivariate analysis, Data visualization Insights: Gather insights How to use machine learning in UX to drive strategy?
  13. 13. Two Examples of using big data methods 3 13
  14. 14. Are the social signals of a review indicative of the polarity of the text? How do users use various voting options on Yelp reviews? Are the social signals of a review indicative of how much the rating differs from the mean rating for that business? 14
  15. 15. 15 We obtained data that was shared by Yelp. 11,537 businesses 43,873 reviewers 229,907 reviews It contained, data about businesses in Phoenix,AZ area. Yelp data
  16. 16. We created the response variable to indicate the difference between review's rating and the average rating Business's avg rating Review's rating Funny, cool and useful votes Different set of variables we considered 16
  17. 17. 17
  18. 18. Take-aways 18 1 Funny votes imply lower ratings and higher negativity in the tone. 2 Useful votes imply lower ratings and lower positivity in the tone. 3 Cool votes imply higher ratings and higher positivity in the tones. 4 Community signals carry implied meaning beyond their specific labels. We used a big data model to better understand how our users use the features that are available to them on the site.
  19. 19. Connection: What is the structure of social connection on Pinterest? For example, are women or men more connected? Comparison: How does behavior on Pinterest compare to behavior on other social network sites? Letting Twitter be our point of reference, do Pinterest and Twitter users systematically differ in what they talk about? What is Pinterest and what do people do there? Activity: What drives user activity on Pinterest: what about a pin—and its pinner—“grabs the attention” of other users? For example, what role does gender play? Do pins from women receive more, less, or equal attention than those from men? 19
  20. 20. Data collection process 20 steps taken in this work to collect and prepare data for modeling
  21. 21. Pins 21 Pin is a unit of analysis for content on Pinterest • We collected data around pins: • repins: how many times this pin was repinned • likes: how many other users liked this pin • comments: textual comments included by the pin’s pinner • text: the pin’s description and comments by others
  22. 22. Users on Pinterest are called pinners.They can create boards and pin images to their boards. Pinners 22 Aenean eu leo quam Vehicula Etiam sed diam eget risus varius blandit sit amet non magna. • We collected data around pinners: • followers: how many pinners follow this pinner • follows: how many users this pinner follows • boards: how many boards this pinner has created • pins: how many pins this pinner has created
  23. 23. we also collected additional data features that could help us understand Pinterest activities better. Additional data 23 • location: a free-text description of a pinner’s location, via either Pinterest or Facebook; if they specified it in both places, we used the Pinterest location • gender: a pinner’s self-reported gender via their Facebook profile page • tweet text: via their linked Twitter handle, the text of all tweets written by this pinner in the last year
  24. 24. Take-aways 24 1Being American or British earns repins. 2Being female means fewer followers. 3Pinterest’s verbs: use, look, want, need. 4Twitter: lol, watching now.
  25. 25. Geographic distribution of users we used in the analysis 25
  26. 26. We run a negative binomial regression with number of followers as a dependent variable. Understanding follows 26 Main findings: Gender: being female is associated with lower number of followers Country: Being American or British earns repins.While we were unable to analyze every country in our dataset because of sparsity, we do find that pinners from the United States and the United Kingdom attract more repins than the rest of the world.
  27. 27. We plot the distributions of followers for each gender: Gender follow up analysis 27 while the median male follower count is lower than the female median follower count (67 vs. 86), the mean male follower count is substantially higher than the mean female follower count (1,063 vs.270).
  28. 28. What differentiates posts on Pinterest from the posts on Twitter? 28 Pinterest’s verbs: use, look, want, need.
  29. 29. What differentiates posts on Pinterest from the posts on Twitter? 29
  30. 30. What differentiates posts on Pinterest from the posts on Twitter? 30 Twitter: lol, watching now.
  31. 31. How is this different from Data Science/ Analytics? 4 31
  32. 32. Basic unit is interactionBasic unit is task Basic unit is action 32 Qualitative methods DS big data methodsUX big data methods Large dataBasic unit is task Large data Data from replication of real world usage Data from real world usage Data from real world usage Human centric Human centric Business centric Answers why Answers what, sometimes why Answers what
  33. 33. Footercontent20pts.Loremipsumdolorsitamet,erosverteremsalutatusnamte.Viscutamquamconclusionemque. With Large scale behavioral analysis we can be both human centric and business centric. We can connect people's needs and behaviors to business metrics and propose new metrics using behavior. Small scale Large scale Business centric Human centric Qualitative UX methods Data Science Methods Large-Scale Behavioral Methods Survey Methods Large scale machine learning methods can help us report business metrics based on actual behavior
  34. 34. Accuracy Explainability UX models are often intended to be descriptive Business models often aim to be predictive Explainability vs accuracy trade-off
  35. 35. 1 2 Make sure to learn and understand how data is collected and what are the limitations of it. 3 4 Having access to big data and being able to analyze that data will not do you a bit of good if you are unable to translate those efforts into successful actions. Keep it simple: that any modeling exercise has inherent risk.Although advanced statistical methods indisputably make for better models, complex models are often not practical. Big data can't answer all types of research questions Start with a hypothesis Complexity is not always better Understand your data Turn insights into actions It's easy to get lost with big data. Often projects are exploratory so there is always a need to approach these projects with established hypotheses. Things to remember 35
  36. 36. Be ready to triangulate with qualitative methods 36 Algorithms are not as good as Humans in telling the full story
  37. 37. How to use big data to drive UX strategy? 37 Saeideh Bakhshi Quantitative UX Researcher, Facebook bakhshi@fb.com UXPA 2018