Weitere ähnliche Inhalte Ähnlich wie How to Use Quant and Qual Feedback to Rapidly Improve Your Product (20) Kürzlich hochgeladen (20) How to Use Quant and Qual Feedback to Rapidly Improve Your Product1. How to Use Quant & Qual Feedback
to Rapidly Improve Your Product
Dan Olsen Jan 2020
2. Engineering background: submarine design
Stanford MBA
Product Management leader at Intuit & startups
CEO & Cofounder, TechCrunch winner YourVersion
Product Management trainer and consultant
Founder: Lean Product Silicon Valley
Twitter: @danolsen
My slides: https://dan-olsen.com
My Background
Copyright © 2020 @danolsen
8. The 3 Phases of Your Product
Copyright © 2020 @danolsen
BEFORE
Product-Market Fit
AFTER
Product-Market Fit
BEFORE your
product is live
Phase 1
AFTER your
product is live
Phase 2
#ofCustomers
Time
0
lots
Phase 3
10. The 3 Phases of Your Product
BEFORE
Product-Market Fit
AFTER
Product-Market Fit
Growth
Mainly
Quantitative
BEFORE your
product is live
Phase 1
Test hypotheses,
gain confidence
before building
Mainly
Qualitative
AFTER your
product is live
Phase 2
Improve product-
market fit
Qualitative &
Quantitative
Goal:
Methods:
Phase 3
11. The 3 Phases of Your Product
BEFORE
Product-Market Fit
AFTER
Product-Market Fit
Growth
Mainly
Quantitative
BEFORE your
product is live
Phase 1
Test hypotheses,
gain confidence
before building
Mainly
Qualitative
AFTER your
product is live
Phase 2
Improve product-
market fit
Qualitative &
Quantitative
Goal:
Methods:
Phase 3
What phase are you in?
27. **InVision
Iterative Design & Test Workflow
Copyright © 2020 @danolsen
Hand
sketches
Interactivity
Fidelity
Clickable
Wireframes*
Clickable
Mockups**
Live
Product
Test Test
Test
*Balsamiq
28. The Lean Product Process:
MarketingReport.com Case Study
Copyright © 2020 @danolsen
29. MarketingReport.com Case Study
My client (CEO) had a new product idea
Team: me, CEO, VP marketing, UI designer
Goals
See if business opportunity exists
Do so quickly and inexpensively (no coding)
“Marketing report” would let consumers
control the direct mail that they receive
Analogous to credit report
Copyright © 2020 @danolsen
30. Mapping Out Customer Benefits
Reduce
Junk Mail
Learn why I
receive the junk
mail I receive
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Marketing
Report
Marketing
Score
Marketing
Profile
Save
Trees
“Marketing Shield”
Concept
“Marketing Saver”
Concept
Copyright © 2020 @danolsen
33. Learning from First Wave of User Tests
Reduce
Junk Mail
Learn why I
receive the junk
mail I receive
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Marketing
Report
Marketing
Score
Marketing
Profile
Save
Trees
Legend
Strong appeal
Low appeal
No appeal
“Shield” Concept “Saver” Concept
Copyright © 2020 @danolsen
36. Summary of Case Study
4 weeks to validated product concept
1 round of iteration
Zero coding
Reasonable cost
Customers willing to pay $10/month
Trimmed away non-valuable pieces
You can achieve similar results
Copyright © 2020 @danolsen
38. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2020 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Global
Level
Metric
Level
Select
Top Metric
Learn
& Iterate
39. Focus on right metric at right time
Identify your
“Metric that Matters Most”
(MTMM)
Copyright © 2020 @danolsen
What’s the right order to optimize?
acquisition
conversion
retention
40. Retention Rate
Retention rate tracks what % of your
customers are still active over time
~80%
never use
app again
Curve either
goes to zero
or flattens out
47. Abandonment Rate (7 Day Moving Average)
0%
10%
20%
30%
40%
50%
60%
70%
80%
10/7/02
10/14/02
10/21/02
10/28/02
11/4/02
11/11/02
11/18/02
11/25/02
12/2/02
12/9/02
12/16/02
12/23/02
12/30/02
1/6/03
1/13/03
1/20/03
AbandonmentRate(7DayMovingAverage)
Steps 1-2
Copyright © 2020 @danolsen
Case Study: Account Signup Process Redesign
48. Conversion Rate
Copyright © 2020 @danolsen
Users
exposed to
ability to
take action
Users that
took action
100%
x%
Can track conversion rates for many different actions
49. Copyright © 2020 @danolsen
Analyzed Drop-Off at Each Major Section
100%
62.3%
58.8%
50.9%
34.4%
32.7%
0%
20%
40%
60%
80%
100%
%ofUsers
Sign in /
Registration
Account
Type
Cash vs.
Margin
5 Partner
Pages
3 Partner
Pages
Focus on
biggest
drop
50. Copyright © 2020 @danolsen
Open
Account
Sign in
Account
Selection
Register
56%
44%
Forget
Password
Registration
Process
45% drop off
(20% of total)
36% overall
drop off for
this step
70%
(32% of Total)
17% drop off
(10% of total)
20% drop off
(6% of total)
30%
(14% of Total)
80%
(26% of Total)
55%
(24% of Total)
64%
of Total
Analysis of Sign In/Registration Flow
Change
Password
83%
(46% of Total)
51. Copyright © 2020 @danolsen
Open
Account
Sign in
Account
Selection
Register
56%
44%
Forget
Password
Registration
Process
45% drop off
(20% of total)
36% overall
drop off for
this step
70%
(32% of Total)
17% drop off
(10% of total)
20% drop off
(6% of total)
30%
(14% of Total)
80%
(26% of Total)
55%
(24% of Total)
64%
of Total
Analysis of Sign In/Registration Flow
Change
Password
83%
(46% of Total)
52. Copyright © 2020 @danolsen
Open
Account
Sign in
Account
Selection
Register
56%
44%
Forget
Password
Registration
Process
45% drop off
(20% of total)
36% overall
drop off for
this step
70%
(32% of Total)
17% drop off
(10% of total)
20% drop off
(6% of total)
30%
(14% of Total)
80%
(26% of Total)
55%
(24% of Total)
64%
of Total
Analysis of Sign In/Registration Flow
Change
Password
83%
(46% of Total)
53. Copyright © 2020 @danolsen
Open
Account
Sign in
Account
Selection
Register
56%
44%
Forget
Password
Registration
Process
45% drop off
(20% of total)
36% overall
drop off for
this step
70%
(32% of Total)
17% drop off
(10% of total)
20% drop off
(6% of total)
30%
(14% of Total)
80%
(26% of Total)
55%
(24% of Total)
64%
of Total
Analysis of Sign In/Registration Flow
Change
Password
83%
(46% of Total)
54. Abandonment Rate (7 Day Moving Average)
0%
10%
20%
30%
40%
50%
60%
70%
80%
10/7/02
10/14/02
10/21/02
10/28/02
11/4/02
11/11/02
11/18/02
11/25/02
12/2/02
12/9/02
12/16/02
12/23/02
12/30/02
1/6/03
1/13/03
1/20/03
AbandonmentRate(7DayMovingAverage)
Steps 1-2
Copyright © 2020 @danolsen
Redesigned User Flow Improved
Registration Conversion Rate
37% improvement
in conversion rate
Released
new design
55. Copyright © 2020 @danolsen
Sequencing Your Analytics Optimization
Retention
Rate
Conversion
Rate
Acquisition of
Prospects
Referral
Improve product-
market fit
Revenue:
Customer LTV
Convert prospects
to customers
Increase ARPU
Decrease churn
Decrease CAC
Scale marketing
Optimize
viral loop
Profitability:
LTV > CAC
56. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2020 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Metric
Level
Select
Top Metric
Learn
& Iterate
Global
Level
57. Geoffrey Moore, author of Crossing the Chasm
Tue Jan 14th at Intuit, Mountain View
Innovation advice from his newest book Zone to Win
https://meetup.com/lean-product