In this talk, Ria introduced the audience to the heart, mind and soul of Product Management: Customer Obsession, Metrics, and Product Sense. She discussed a broad understanding of top research methods, product management frameworks and metrics used by Product Managers at Facebook and Microsoft.

1. How to Build Winning Products by Microsoft Sr.
Product Manager
www.productschool.com

2. FREE INVITE
Join 23,000+ Product Managers on

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Blockchain and Cryptocurrencies
Learn how to trade cryptocurrencies and build products using the blockchain

7. TONIGHT’S SPEAKER
Ria Sankar

8. Building Winning Products
Decoding Customer Obsession, Product Sense, and
Execution
Ria Sankar
Product Management @Microsoft

9. What does a Product Manager do?
Source: https://www.onedesk.com/product-manager-meme/

10. “Mini-CEO” responsible for overseeing and leading
teams through product planning, feature definition and
execution
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2

11. Today’s Takeaways: Top tools for product management
Landscape of top research methods + closer look at 5
For each: When to use, Expectations of PM, Variations
Landscape of top product management frameworks + closer look at 5
For each: Framework, case studies
Time permitting: Picking the right metric to measure success of your
feature
1
3
2
Closing & Q&A

12. User Research
Plan
DefineExecute
User Research: Identify underlying
user needs
1

14. dictionary.com wordcloud generated by

15. Categories of User Research
By research source
• Primary
• Secondary
• Third party
By type of insights
• Qualitative
• Quantitative
• Behavioral
By context
• Contextual
• Scripted

16. Landscape of research methods by type of insights
What people do
What people say
Why & How to
improve
How many & how
much
Behavioral
Qualitative
N=10 N=Thousands+
Quantitative
Reference: Christian Rohrer 2014
N=Hundreds
Optimize /
improve existing
product
Build a new
product

17. What people do
What people say
Why & How to
improve
How many & how
much
Behavioral
Qualitative
Quantitative
N=10
N=Thousands+
N=Hundreds
Ethnographies
SurveysEye-tracking
Observations
Diary Studies
Task History
Path & Click Analysis
Panel studies
Cohort Analysis
Focus groups
Scripted Remote Studies
Concept-value tests
Message testing
Landscape of research methods by sample size
Legend
Reported Behavior
Actual Behavior
Optimize existing product →← Build new product

18. Top research methods used @Microsoft
1. Qualitative: Focus Groups
1. Qualitative: Remote studies
1. Quantitative: Surveys
1. Hybrid Qual/Quant: Message testing
1. Behavioral: Cohort Analysis

19. • When to use:
• Deep qualitative feedback on complicated experiences
• Directional feedback in early product development
•PM provides:
• Research questions
• Sample experiences – mockups, prototypes, competitors
• Participant screening (i.e. women <34 that use Prime Now app on iPhone
weekly)
• Optional: Pre-work
•Variations:
• Moderated remote studies, diary studies & customer journals
1. Focus groups

20. • When to use:
• Validating results of focus groups
• Comparing design options
• Prioritize features during design, before
development begins
•PM provides:
• List of multiple-choice Q&A
• Some open-ended for remaining qual feedback
• Sample experiences
• Screening segmentation or questions
•Variations:
• Concept value tests
2. Surveys

21. • When to use:
• For more detailed feedback than surveys
• Budget or resource constraints
• Message testing, Usability study of prototypes
•PM provides:
• Research questions
• Sample experiences
• Participant screening
•Variations:
• Diary studies
• Customer journals or Ethnographies
3. Remote Studies

22. • When to use:
• When planning go-to-market activities
• Prioritize among message alternatives to land the best story with
consumers
•PM provides:
• List of canvases: Banner Ads, headers, in-product alerts
• List of channels: social, email, blogs, influencers
• List of messages for each type of canvas & channel
• GTM-ready experiences
• Participant screening
•Variations:
• Conversion rate optimization using Persado or competitors
4. Message testing

23. • When to use:
• Deeply understand user behavior of an existing or newly
launched product
•PM provides:
• Work closely with Data Scientist to define the problem
statement, research questions, and cohort type (gender, location,
start date)
• Research questions
• Metrics to measure: Engagement, Revenue, Returning Usage
• User screening (i.e. users who visited Prime Day deals during
crash window)
•Variations:
• Path or clickstream analysis, Task histories
• Longitudinal studies, A/B Tests
5. Cohort Analysis

24. Takeaways / Things to remember
Pros Cons When to use
Qualitative research –
moderator driven
focus groups
Great for brainstorming, discovering
latent needs
Experienced moderators distill key
insights
Expensive Early product envisioning
Qualitative research –
remote studies
Good for unearthing pain points,
concept value testing
Cheaper than moderator-led focus
groups
Operational aspects of group chat
tool can be challenging
Best for 1:1 observations
Early product envisioning, Design, best
for message testing
Quantitative research
- surveys
Quantitative validation of known
pain points and concepts
Best for feature prioritization
Does not represent actual user
behavior
Post qualitative research for product
development
Behavioral analyses Quantify actual user behavior (how
much, how often)
Needs investments in
instrumentation and data collection
Post launch

25. Frameworks
Plan
DefineExecute
Frameworks: Structure and
frame problem spaces
2

26. Businessdictionary.com

27. Why Frameworks?
• Find your niche
• Build consensus with popular
frameworks
Customer
CompetitiveBusiness
Niche

28. Landscape of Product Management Frameworks
Customer
• AARM or REAN or AIDA
• RFM
• HEART
• Jobs Theory
• Fogg Behavior Model
• Lean UX
Business & Decision making
• 4 Ps
• SWOT
• Pricing: 5 Cs
• Kano Model
• BCG Matrix
Competitive
• Porter’s 5 forces
• Blue Ocean Strategy
• McKinsey 7 degrees of
Freedom - Growth

29. Why would you hire a milkshake?
https://hbswk.hbs.edu/item/clay-christensens-milkshake-marketing
Image: McDonalds.com

30. Jobs Theory
Seeing tasks from a customer lens vs. product lens
#1
Functional
“Help me wake up
with the best coffee
at consistent
quality”
Social
“Give me a place to
connect with my
friends”
Emotional
“Help me treat
myself
at the end of a long
day”

31. Jobs Theory: Framework
Deeply understand the customer
• What’s their Situation “When”: skills, wealth, time, demographic, spending habits?
• What’s their Motivations and Expected outcomes
1
Identify the high-priority job-to-be-done
• How does the customer do the job today?
• What barriers prevent the customer from doing it (well)?
2
Frame your value proposition
• Relative to competing solutions, where does our solution meet the bar?
• Where to delight?
• Where can our solution be below the bar?
3 Identify related emotional, social and functional jobs
4

32. Framing the value prop: Jobs Theory guides you to ask the right
question and frame the topmost user need
Framing: WHEN <…> I WANT <…> SO I CAN <…>
https://design.org/wp-content/uploads/2016/11/Jobs-To-Be-Done-Framework.jpg
Jobs
Main Job
Functional
Emotional
Social
Emotional
Related
Jobs

33. Case Study: Immediate Clinic
• Time to appointment
• Time to care
• Cost of treatment
• Effective treatment
• Provider expertise
From: Yelp.com
WHEN my daughter is hurt, I WANT immediate care SO I CAN relax knowing that she is pain-free

34. Jobs Theory: Summary & Takeaways
• An innovative approach to solving customer problems
• Framing the customer’s problem in 3 dimensions:
• Situation “When..”
• Motivation “I want to..”
• Expected Outcome “…so I can …”
• Identify where your product needs to compete vs. delight vs. compromise
• Great framework as you prepare for focus groups

35. Blue Ocean Strategy
Systematic approach to finding uncontested markets or creating new demand
From: cirquedusoleil.com
#2
2 types of blue oceans:
1. Launch completely new industries e.g. Amazon
with online book retailing, ebay with online
auctions
1. Expand boundaries of existing industry e.g.
Cirque du Soleil
Read more: https://www.blueoceanstrategy.com/bos-moves/cirque-du-soleil/

36. Blue Ocean Strategy: Four Actions Framework
•Which factors that
the industry has long
competed on should
be eliminated?
•Which factors should
be created that the
industry has never
offered?
•What factors were a
result of competing
against other
industries and can be
reduced?
•What factors should
be raised well above
the industry's
standard?
Raise Reduce
EliminateCreate
1. Similar to Jobs Theory, considers when a
“job” or a “factor” is good enough or can
be eliminated
1. Modern take on classic Porter’s 5 Forces
1. Focus on strategic decision making than
competitor analysis

37. Case Study: iPhone
• Computing power
• Ease of use
• Easy to carry
• Always on
• Built-in camera
Source:
https://i.ytimg.com/vi/U3i7TTX5SRI/hqdefault.jpg
After years of competing with the PC, Steve Jobs expanded market boundaries to Mobile. Disrupted the camera
industry. Build incredible momentum and market cap bringing together Devices & Services

38. Jobs Theory & Blue Ocean: Recap
• Complementary frameworks
• Use Jobs Theory first to deeply understand what the customers are trying to get done, their
circumstances, desired outcomes, and constraints
• Use Blue Ocean to identify a unique market space that delights customers and is hard for
competitors to replicate
• Goal: Avoid head-to-head competition

39. From: businessnewsdaily.com
BCG Matrix
Useful for strategic planning to decide where to invest your $$
#3
1. Milk the Cash Cows
1. Divest in the Dogs
1. Spend on the Stars
1. Experiment with Question Marks

40. BCG Matrix in the Product Lifecycle
1. Most products start as question
marks
1. If market share is low, they will
absorb lot of cash
1. Some question marks, become
dogs while other grow to be
stars and cash cows
Source: https://www.slideshare.net/9579764806/bcg-matrix-with-example

41. Microsoft through the BCG Lens
Disclaimer: This is one PM’s perspective

42. BCG Matrix - Applications
1. Mini-CEO mindset
1. Macro - Managing product features within product, or products
within product area
1. Micro - manage attributes within a feature
1. Understanding and making career choices

43. From: mindtheproduct.com
Kano Model
Famous ASQ methodology to understand and classify customer needs
#4
Customers don’t
know to ask for
this
“Must” be met to
avoid
unhappiness
Customer
“wants” for
happiness

44. Surveys the KANO Way
Source: foldingburritos.com/kano-model
Feature present
How would you feel if our
battery life was longer 8
hours?
Feature NOT present
How would you feel if our
battery lasted less 8
hours?
Like This would be very helpful to me X
Expect This is a basic requirement for me
Don’t care This would not affect me
Live with This would be a minor inconvenience
Dislike This would be a major problem for me
Like This would be very helpful to me
Expect This is a basic requirement for me
Don’t care This would not affect me
Live with This would be a minor inconvenience
Dislike This would be a major problem for me X

45. Classifying customer needs
Like Expect Don’t
care
Live
with
Dislike
Like Q D D D S
Expect R I I I E
Don’t
care
R I I I E
Live
with
R I I I E
Dislike R R R R Q
FEATURE PRESENT
FEATURENOTPRESENT
D - Delighters
S – Satisfiers
E – Expectations
X – Don’t care
Q – Questionable
R – Reverse

46. Case Study: Disney Cruises
• Kids eat free
• Free room service
• Fireworks
• Special acts of kindness
• Towel Animals!
Source: cruises.priceline.com
Further reading: https://sigmazone.com/kanos_model

47. Finding Delighters
• Analogies
• Personal, social and emotional moments
• 5 Whys
See more: bing.com/explore

48. KANO: Takeaways and things to remember
• Rigorous approach to identifying and classifying customer needs
• Popularized by ASQ and used widely in LEAN / Six Sigma methodologies
• Supplemented with rigorous behavioral analyses to be effective
• Identifying delighters is tricky, use analogies

49. Lean UX#5
Source: https://www.agriya.com/blog/
Read more: https://www.interaction-design.org
• Quick and iterative
• Focus on users
• Early feedback
• Hypothesis testing
• Focus on
collaboration
Takeaway: If no data or evidence, walk away from an idea

50. Lean UX Canvas
Source: https://medium.com/@jboogie/the-lean-ux-canvas-d4da53263c40

51. Case Study: Bing
Source: https://www.bing.com/search?q=american+sign+language

52. Metrics
Plan
DefineExecute
Metrics: Prioritize features,
Validate hypotheses
3

53. A different perspective..

54. A Typical User Journey
DISCOVER RETURN BUY SHARE=+ENGAGE ++
Determine primary and secondary
goals for your feature / product
Pick the features and metrics that
will move the right goal

55. Measuring Feature Discovery
DISCOVER
1. How many total Users
1. How many total Active Users – Daily, Monthly using
feature, How many Active Days
1. What % of total users see feature X
1. Where are the users coming from – entry points,
marketing channels Product Level
User Level
Feature Level

56. Measuring User Engagement
DISCOVER ENGAGE+
1. How many Users are seeing feature X
1. How many Users are clicking or engaging with feature X
1. How many Users leave after seeing feature X (churn, bounce rate)
1. What is a user’s engagement before / after seeing feature X?User Level
Feature Level

57. Measuring User Retention
DISCOVER RETURNENGAGE ++
1. In a 30 day window, how many Users that saw feature X
returned to use feature X
1. In a 30 day window, how many Users that saw feature X
returned to the product AND used a different feature Y
1. For users that saw Feature X, did overall engagement
with product increase?
1. Corollary: Are engaged users of product using feature
X?
Product Level
User Level

58. Measuring Monetization
DISCOVER RETURN BUY+ENGAGE ++
1. What is the customer acquisition cost?
1. What is the average revenue per user?
1. What is the revenue per user for feature X compared
with rest of product?
Product Level
User Level

59. Measuring Advocacy & Fandom
DISCOVER RETURN BUY SHARE=+ENGAGE ++
1. What is the life-time value for the user?
1. Did user refer friends?
1. What is the user saying about the product?
1. What is the revenue per user for the product?Product Level
User Level

60. If you were the product manager for FB Events, what metrics
would you use to measure success?

61. RECAP
Plan
DefineExecute
User Research: Identify underlying
user needs
Frameworks: Structure and
frame problem spaces
Metrics: Prioritize features,
Validate hypotheses
13
2
Start with
Why
How
What

62. Recommended Reading & Viewing
What
How
Why

63. Q&A

64. Thank you

65. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New York, Austin,
Boston, Seattle, Chicago, Denver, London, Toronto
www.productschool.com