Main takeaways:
- De-mystify the concept of a data-driven product
- Understand the benefits of treating your product like a data product
- Understand how to start making changes to your current product research and development process
5. T O N I G H T ’ S S P E A K E R
Keisha Johnson
6. data-driven product [ dey-tuh-driv-uhn-prod-uhkt ]
noun
any product that leverages data insights to inform decisions
in order to deliver user value
Johnson, Keisha. “Data-Driven Product.” Unofficial Made Up Definitions, 6 Mar. 2019
7. How is this different from a “regular” product?
16. Step 3:
Identify the data you’ll need to validate or
invalidate your hypothesis and the success
metrics you are working toward
17. Step 4:
Evaluate your results
Article: The time I tried to ruin Halo 2
- What is the data, not the user, telling you?
- What assumptions have you brought into the
process that could be impacting your evaluation?
- What are alternate explanations for your results?
18. Step 5:
Allow your findings to inform your next steps
- Track your results over time to ensure you are still
meeting your desired metrics
20. - We have data analysts and data scientists embedded on
our teams
- Every PM is empowered to be their own data professional
- PMs regularly sift through hundreds, even thousands, of
data points in search of information
- We want to understand the “why” behind the numbers
- We try to define honest and clear metrics
- Everyone respects the process
In our work,
If you’re interested to connect with other Product Managers, aspiring PMs, or those within tech, join our Slack community of over 40,000 professionals. It’s a great place to network and to find interesting content. We host a weekly AMA through our Slack channel on Tuesdays from 11:15am - 12pm PST. We have also recently launched the Job Portal where you can find the latest Product Management opportunities! As members of the Product School community, we'd like to provide you with these resources at your disposal.
Product School’s Product Management Certificate Path comprises of 3 part-time courses for professionals with strong technical or business background who want to further explore Product Management at software-based companies.
During Product Management Training you will first learn Product Management fundamentals to understand the software product lifecycle and what it takes to successfully transition into a product management role.
You’ll then be trained to retrieve data, understand its value and make impactful decisions with SQL, data visualization and Tableau. Learn to understand your users to deliver exceptional UX/UI design and develop a robust digital marketing plan. During the Full Stack Product Management Training, you will deep dive into the technical knowledge to enhance your ability to work with agile teams.
Finally, Product Leadership Training will elevate your product knowledge to become an effective Product Leader. You'll do an in-depth analysis on how to implement best PM practices on a strategic level to significantly impact your company’s portfolio and revenue. Learn the soft skills to manage product teams and manage stakeholders to deliver performing products.
As well as individual courses we provide corporate training across the world! If you’d like to upskill your product team this is the best option for you. We have trained employees from multiple companies such as Deloitte, Salesforce, JP Morgan, Bank of America amongst many other companies across all industries.
I fully recognize I broke rule no. 1 of creating definitions
Amazon is a really great example of a data driven product
Intuit is another
It’s really not, if you’re doing product right. Having a data-driven product IS the way you build products
This is about how you gather information, what you gather, and how you use it
Mixed methods research
You might not actually care, but here is why you should
Mixed methods research is the process of combining multiple data gathering tactics, both quant and qual, in order to evaluate a problem or opportunity
In doing so, you can create a more robust picture of the thing you are trying to solve for
Example: which tier of business plan? Respondents may provide favorable feedback
Example: which tier of business plan works best? Heat maps show little time spent viewing and very low hit rates. Conversely, recently launched a new help page and getting lots of traffic, only to realize people are struggling with your app
One without the other might give you the wrong story
There are any number of things you can capture, in a plethora of ways, for any number of reasons.
Be a good steward of the data that you capture and use - just because you can doesn’t mean that you should
Ex. PS will not share learner data with enterprises
Include them right from the beginning
They will look at the problem differently and help define success outcomes
Can help with setting up data gathering, dashboard tracking, interpreting the data, etc.
I don’t have someone at my disposal
Be a critical thinker
There is a place for algos and machine generated models in all of this - to get started, you don’t need all of that.
What are the metrics you’re looking to evaluate? This might be usage, CTR, page hot spots, user performance over time, a combination of metrics that either correlate or do not
For example, if you created a new search feature for your product, you could test it with users to understand how they are finding it, but also
Exit rates
Queries per user
View time on search results page
Goal might be to decrease search exit rate by 5% - cannot evaluate this by user feedback, but also need user feedback to help add color to why rate has gone up or down
This is an example of mixed methods research and analysis
How does what the user told you explain or differ from the data?
Time of day/seasonality of use
Article: sometimes the data is wrong and you have to go with your gut
Torture the data and it will confess anything
At the end of the day, you, the human, makes the final call. Use all the data you’ve gathered to make the most informed decision
Be sure to track results over time to ensure you are still meeting your metrics
Watch for abnormalities, whether good or bad
We’ve taken steps to ensure that we make data more of a priority
Data practitioners help guide the framing of the hypothesis and identifying appropriate metrics
Because data practitioners can be supporting multiple teams at once, PMs take on some of the ownership of doing some of their own data analysis
Don’t just take the metrics at face value - dig deeper into the why. What is the cause or relationship?
Honest: what is realistic? Clear: exactly why number are we moving and why
No one questions you if you do, but might get questions if you don’t
At the end of the day, a lot of this comes down to process. Setting your success metrics up front will help drive the next steps
Feel free to speak with me and I can point you in the right direction (explain where to apply). Or you can visit www.productschool.com
Have a good night!