Many companies have already embraced Lean Startup principles. It's hard to argue with learning from customer feedback and iterating to achieve product-market fit. But how exactly are you supposed to do that? This discussion covers the following topics: * Understanding and prioritizing customer needs * Determining your product's value proposition * Principles of good User Experience (UX) design * Validating your product with user testing * Product analytics About the Speaker Dan Olsen is an entrepreneur and product expert. He started his high-tech career managing submarine design and then worked at Intuit where he led product management for Quicken. Dan also led product management at Friendster. He was CEO and cofounder of YourVersion, a personalized news discovery engine startup that won the People's Choice Award at TechCrunch50. Dan currently consults to companies, usually as interim VP of Product for startups. His former clients include Facebook, Box, Hightail (YouSendIt), Chartboost, Epocrates, and One Medical Group. Dan holds a BS in Electrical Engineering from Northwestern, a Master's in Industrial Engineering from Virginia Tech, and an MBA from Stanford. Upcoming Events Reserve your seat for the next AIPMM webinar. Visit: http://aipmm.com/aipmm_webinars/. Want To Certify Your Team? If you have a product team of 10 or more that you want to certify, visit: http://bit.ly/1bzjUYB. About AIPMM The AIPMM is the hub of all things product management. It is where product professionals go for answers. With members in over 65 countries, it is the worldwide certifying body of product team professionals. It is the world's largest professional organization of product managers, brand managers, product marketing managers and other product team professionals who are responsible for guiding their organizations, or clients, through a constantly changing business landscape. AIPMM's certification programs are internationally recognized because they allow product professionals to demonstrate their expertise and provide corporate members an assurance that their product management and marketing teams are operating at a high competency level. Visit http://www.aipmm.com. Call For Speakers: http://bit.ly/1b006vm Subscribe: http://www.aipmm.com/subscribe LinkedIn: http://www.linkedin.com/company/aipmm Membership: http://www.aipmm.com/join.php Certification: http://aipmm.com/html/certification Articles: http://www.aipmm.com/html/newsletter/article.ph

1. AIPMM
Webinar
Series
www.aipmm.com
©
AIPMM
2013
www.aipmm.com

2. ©
AIPMM
2013
www.aipmm.com

3. The Hub Of All Things Product Management
- Founded
1998,
AIPMM
has
cer5fied
product
team
professionals
in
nearly
70
countries
- World’s
largest
professional
associa5on
for
product
management,
product
marke5ng,
and
brand
management
- Provides
globally
recognized
cer5fica5ons:
• Cer5fied
Product
Manager
• Cer5fied
Product
Marke5ng
Manager
• Agile
Cer5fied
Product
Manager
• Cer5fied
Innova5on
Leader
• Cer5fied
Brand
Manager
©
AIPMM
2013
www.aipmm.com

4. Tweet!
Use
AIPMM:
Product Ninja:
#AIPMM
#ProductNinja
Follow
Dan Olsen
Hector Del Castillo:
AIPMM:
@DanOlsen
@hmdelcastillo
@AIPMM
©
AIPMM
2013
www.aipmm.com

5. Par=cipate
and
Win!
• Win
a
copy
of
the
Guide
to
the
Product
Management
and
Marke5ng
Body
of
Knowledge
(ProdBOK®)
©
AIPMM
2013
www.aipmm.com

6. Today’s
Speaker
Moderator:
Hector
Del
Cas5llo,
CPM,
CPMM,
PMP
Presenter:
Dan
Olsen
Product
Consultant
&
Interim
VP
of
Product
Olsen
Solu5ons
@DanOlsen
hXp://olsensolu5ons.com
©
AIPMM
2013
www.aipmm.com

7. +*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+
Building Great
Products the Lean
Startup Way
(aka How to be a
Lean Product Ninja)
Dan Olsen
Olsen Solutions
November 15, 2013
+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+

8. What
does
“Lean”
mean?
n
Lean
Startup
n Achieving
product-­‐market
fit
n Valida5ng
product
with
users
n Improving
&
itera5ng
your
product
quickly
n Tes5ng
hypotheses
&
learning
n Minimizing
waste
of
resources
n
Product-­‐market
fit
=
product
management’s
responsibility
Audience
Poll
Copyright
©
2011
YourVersion

9. Why
“Ninja”?
n High
level
of
exper5se
n Self-­‐sufficient
n Uses
wide
array
of
skills
to
get
the
job
done
Copyright
©
2011
YourVersion

10. What’s
the
Formula
for
Product-­‐Market
Fit?
n
A
product
that:
n Meets
customers’
needs
n Is
beXer
than
other
alterna5ves
n Is
easy
to
use
n Has
a
good
value/price
Focus of my
talk today
Will cover basics. See
my talk “How to be a
UX Design Army of
One”
Copyright
©
2013
Olsen
Solu5ons

11. Dan’s
Model
for
the
Causality
Underlying
Product-­‐Market
Fit
Customer
has
needs
Target
Customer
Customer
Needs
Product
is
designed
to
meet
needs
Customer
decides
how
well
product
meets
needs
(sa5sfac5on)
Product
Copyright
©
2013
Olsen
Solu5ons

12. The
Lean
Product
Ninja
Way:
Steps
to
Achieving
Product-­‐Market
Fit
1.
2.
3.
4.
5.
Understand
underserved
customer
needs
Define
your
product’s
value
proposi5on
Build
wireframe/mockup
Get
user
feedback
Iterate
Copyright
©
2013
Olsen
Solu5ons

13. Understanding
Customer
Needs

14.
Problem
Space
vs.
Solu5on
Space
n Problem
Space
n A
customer
problem,
need,
or
benefit
that
the
product
should
address
n A
product
requirement
n
Solu5on
Space
n A
specific
implementa5on
to
address
the
need
or
product
requirement
Example:
n Ability
to
write
in
space
NASA:
space
pen
n
(zero
gravity)
($1
M
R&D
cost)
n Russians:
pencil
Copyright
©
2013
Olsen
Solu5ons

15. Problem
Space
vs.
Solu5on
Space:
Product
Level
Problem
Space
(user
benefit)
Solu5on
Space
(product)
Prepare
my taxes
Pen and
paper
Check my
taxes
File my
taxes
TurboTax
Maximize
deductions
Reduce
audit risk
TaxCut
Copyright
©
2013
Olsen
Solu5ons

16. Problem
vs.
Solu5on
Space:
Feature
Level
Problem
Space
Help me
Solu5on
Space
prepare taxes
Save Money
Audit Risk
Analyzer
Tax Return
Error Checker
Save time
preparing taxes
Save Time
Reduce my
audit risk
Check my
return
Empowerment/
Confidence
Tax Interview
Wizard
Tax Data
Downloader
Save time filing
taxes
Electronic Tax
Return Filing
Maximize my
tax deductions
Tax Deduction
Finder
Copyright
©
2013
Olsen
Solu5ons

17. Importance
of
User
Need
Priori5zing
Customer
Needs:
Importance
vs.
Sa5sfac5on
High
Opportunity
Compe55ve
Market
Not
Worth
Going
Aper
Low
Low
High
User
Sa5sfac5on
with
Current
Alterna5ves
Copyright
©
2013
Olsen
Solu5ons

18. Kano
Model:
User
Needs
&
Sa5sfac5on
User
Sa5sfac5on
Delighter
(wow)
Performance
(more
is
beXer)
Need
not
met
Need
fully
met
Must
Have
Needs
&
features
migrate
over
5me
User
Dissa5sfac5on
Copyright
©
2013
Olsen
Solu5ons

19. Importance
vs.
Sa5sfac5on
Ask
Users
to
Rate
for
Each
Feature
100
Great
95
84 87
90
Bad
85
Importance
98
70
55
80
79
75
80
72
70
86
84
80
75
65
60
55
41
50
40
50
60
70
80
90
100
Satisfaction
Recommended
reading:
“What
Customers
Want”
by
Anthony
Ulwick
Copyright
©
2013
Olsen
Solu5ons

20. What
is
Your
Value
Proposi5on?
n Which
user
benefits
are
you
providing?
n How
are
you
beXer
than
compe5tors?
Compe=tor
A
Compe=tor
B
Must
Have
Benefit
1
You
Y
Y
Y
Performance
Benefit
1
High
Low
Med
Performance
Benefit
2
Low
High
Low
Performance
Benefit
3
Med
Med
High
Delighter
Benefit
1
Y
-­‐
-­‐
Delighter
Benefit
2
-­‐
-­‐
Y
Copyright
©
2013
Olsen
Solu5ons

21. Building
Wireframes/
Mockups

22. Geqng
Customer
Feedback:
Problem
Space
vs.
Solu5on
Space
n Customers
CAN’T
ar5culate
problem
space
n Customers
CAN
react
to
solu5on
space
n That’s
why
you
need
something
tangible
to
show
to
customers
to
get
good
feedback
Copyright
©
2013
Olsen
Solu5ons

23. Interactivity
Design
Ar5facts:
Interac5vity
vs.
Fidelity
Interactive
Prototype
Clickable
Wireframe*
Hand
sketch
Clickable
Mockup**
Static
Wireframe*
Mockup
Fidelity
*
Balsamiq:
balsamiq.com
**
Invision:
invisionapp.com
Copyright
©
2013
Olsen
Solu5ons

24. Design
Tools:
Fidelity
vs.
Effort
Design Fidelity
3
Photoshop
All Product People
2
1
Balsamiq
Visual
Designer
Hand
sketch
Effort to Create Artifact
Copyright
©
2013
Olsen
Solu5ons

25. Why
You
Should
Wireframe
n In
case
you
don’t
have
UX
designer
(open)
n To
clarify
and
refine
your
thinking:
n Informa5on
Architecture,
layout,
naviga5on
n To
communicate
your
product
ideas
to
others
n Modern
tools
make
it
easy
and
fast
Audience
Poll
Copyright
©
2013
Olsen
Solu5ons

26. Learning
from
Customers
Copyright
©
2013
Olsen
Solu5ons

27. What
Are
You
Going
to
Get
Feedback
&
Learnings
About?
Problem
Space
(your
mental
model)
Solu5on
Space
(what
users
can
react
to)
Feature
Set
Customer
Understanding
(needs
&
preferences)
UI
Design
Messaging
Copyright
©
2013
Olsen
Solu5ons

28. “Ramen”
User
Feedback
for
Startups
n Anyone
can
do
it!
n Ingredients:
n Solu5on-­‐space
product/mockup
to
test
n 1
customer
(with
laptop
if
tes5ng
code)
n 1
desk
n 1
person
to
conduct
the
session
n Pen
and
paper
n Op5onal
note-­‐taker
and
observers
Copyright
©
2013
Olsen
Solu5ons

29. Typical
Format
for
Customer
Session
5
-­‐
10
min:
Ask
ques5ons
to
understand
user
needs
and
solu5ons
they
currently
use
n 30
-­‐
50
min:
User
feedback
n
n Show
user
product/mockup
n Non-­‐directed
as
much
as
possible
n When
necessary,
direct
user
to
aXempt
to
perform
a
specific
task
n
5
-­‐
10
min:
Wrap-­‐up
n Answer
any
user
ques5ons
that
came
up
n Point
out/explain
features
you
want
to
highlight
n Ask
them
if
they
would
use
the
product
Copyright
©
2012
Olsen
Solu5ons

30. Dos
&
Don’ts
of
User
Feedback
Sessions
n
Do
n
Explain
to
the
user:
Their
feedback
will
help
improve
the
product
n Not
to
worry
about
hur5ng
your
feelings
n “Think
Aloud
Protocol”
n
n
n
n
n
Ask
user
to
aXempt
the
task,
then
be
a
fly
on
the
wall
Ask
non-­‐leading,
open-­‐ended
ques5ons
Take
notes
and
review
them
aperwards
for
take-­‐aways
Don’t
n
n
n
n
n
Ask
leading
ques5ons
“Help”
the
user
or
explain
the
UI
(e.g.,
“click
over
here”)
Respond
to
user
frustra5on
or
ques5ons
(un5l
test
is
over)
Get
defensive
Blame
the
user
Copyright
©
2012
Olsen
Solu5ons

31. Iterate
Your
Wireframes
Based
on
Feedback

32. Itera5ng
Your
Product
Vector
Based
on
User
Feedback
in
Solu5on
Space
Problem
Space
(your
mental
model)
Help
user
book
travel
Solu5on
Space
(what
users
can
react
to)
Help
user
plan
travel
Mockups
/
Code
Customer
Feedback
Copyright
©
2013
Olsen
Solu5ons

33. Case
Study
on
Product-­‐Market
Fit:
Marke5ngReport.com
Copyright
©
2013
Olsen
Solu5ons

34. Product-­‐Market
Fit
Case
Study:
Marke5ngReport.com
n My
consul5ng
client
(CEO)
had
an
idea
for
a
new
product
n Team:
me,
CEO,
head
of
marke5ng,
UI
design
consultant
n Goal:
n Validate
product-­‐market
fit
quickly,
cheaply
without
wri5ng
a
single
line
of
code
n Determine
if
there
was
a
business
opportunity
here
Copyright
©
2013
Olsen
Solu5ons

35. Product-­‐Market
Fit
Case
Study:
Developing
Product
Concept
Product
Concept
was
“marke5ng
report”
that
let
consumers
control
the
direct
mail
that
they
receive
n Concept
was
fuzzy
with
various
components,
so
I
broke
it
into
2
different
“flavors”:
n
n #1
“Marke5ng
Shield”:
Service
to
reduce/stop
junk
mail
n #2
“Marke5ng
Saver”:
Opt
in
&
receive
money-­‐
saving
offers
Copyright
©
2013
Olsen
Solu5ons

36. Clustering
Poten5al
User
Benefits
to
Create
Product
Concepts
“Shield” Concept
“Saver” Concept
Reduce
Junk Mail
Find out what
“they” know
about you
Save
Trees
Marketing
Report
Marketing
Score
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Marketing
Profile
Copyright
©
2013
Olsen
Solu5ons

37. Copyright
©
2013
Olsen
Solu5ons

38. Copyright
©
2013
Olsen
Solu5ons

39. Product-­‐Market
Fit
Case
Study:
Recrui5ng
People
n
Telephone
recruit
of
prospec5ve
customers
n Wrote
phone-­‐screen
ques5onnaire
to
create
rough
target
customer
segmenta5on
n Wanted
users
who
work
full-­‐5me
&
use
internet
n Fit
for
opt-­‐in
offers:
use
coupons,
Costco
membership
n Fit
for
an5-­‐junk
mail:
use
paper
shredder,
block
caller
ID
Paid
each
person
$75
n Scheduled
3
groups
of
2
or
3
people
to
discuss
each
product
concept
for
90
minutes
n Moderated
each
group
through
the
paper
mockups
to
hear
their
feedback
n
Copyright
©
2013
Olsen
Solu5ons

40. Product-­‐Market
Fit
Case
Study:
Findings
on
Concepts
&User
Benefits
“Shield” Concept
“Saver” Concept
Reduce
Junk Mail
Find out what
“they” know
about you
Save
Trees
Marketing
Report
Marketing
Score
Marketing
Profile
Money
Saving
Offers
Compare
Yourself
to Others
Social
Networking
Legend
Strong appeal
Some appeal
Low appeal
Copyright
©
2013
Olsen
Solu5ons

41. Product-­‐Market
Fit
Case
Study:
Learnings
from
Research
n
n
n
n
Learned
that
“Shield”
(an5-­‐junk
mail)
concept
was
stronger
than
“Saver”
People
didn’t
like
many
of
the
“Saver”
concept
components
Learned
users’
concerns
/
ques5ons
about
“Shield”
concept
Refined
“Shield”
concept:
n
n
n
Removed
irrelevant
components
Improved
messaging
to
address
user
concerns
/
ques5ons
Validated
revised
“Shield”
concept
with
quick
2nd
round
of
tests
n
n
No
customer
concerns
Clear
willingness
to
pay
Copyright
©
2013
Olsen
Solu5ons

42. Copyright
©
2013
Olsen
Solu5ons

43. Product-­‐Market
Fit
Case
Study:
Summary
n 4
weeks
from
1st
mee5ng
to
validated
product
concept
with
zero
coding
n Reasonable
cost
n 1
round
of
itera5on
on
product
concept
n Iden5fied
compelling
concept
that
users
are
willing
to
pay
$10/month
for
n Trimmed
away
non-­‐valuable
pieces
n You
can
achieve
similar
results
Copyright
©
2013
Olsen
Solu5ons

44. Metrics
to
Validate
Product-­‐Market
Fit
n
Survey
results
n
Importance
&
Sa5sfac5on
Net
Promoter
Score
n
Survey.io/Qualaroo
n
n
“How
would
you
feel
if
you
could
no
longer
use
Product
X?”
n
n
Very
disappointed,
Somewhat
disappointed,
Not
disappointed
User
behavior
n
n
n
Prospects
sign
up
(high
conversion
rate)
They
keep
using
it
(high
reten5on
rate)
They
use
it
open
(high
frequency
of
use)
Copyright
©
2013
Olsen
Solu5ons

45. Op5miza5on
with
Metrics:
Define
the
Equa5on
of
your
Business
Model:
Adver5sing
Business
Profit
=
Revenue
-­‐
Cost
nique
Visitors
x
Ad
Revenue
per
Visitor
U
mpressions/Visitor
x
Effec5ve
CPM
/
1000
I
isits/Visitor
x
Pageviews/Visit
x
Impressions/PV
V
ew
Visitors
+
Returning
Visitors
N
nvited
Visitors
+
Uninvited
Visitors
I
of
Users
Sending
Invites
x
Invites
Sent/User
x
Invite
Conversion
Rate
#
Copyright
©
2013
Olsen
Solu5ons

46. Sample
Signup
Page
Yield
Data
Daily Signup Page Yield vs. Time
New Registered Users divided by Unique Visitors to Signup Page
100%
Daily Signup Page Yield
90%
80%
70%
60%
50%
40%
Started requiring
registration
30%
20%
10%
Changed
messaging
Added questions
to signup page
0%
1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1
0
Copyright
©
2013
Olsen
Solu5ons

47. Iden5fying
the
“Cri5cal
Few”
Metrics
n
n
What
are
the
metrics
for
your
business?
Where
is
current
value
for
each
metric?
How
many
resources
to
“move”
each
metric?
n
n
Developer-­‐hours,
5me,
money
Which
metrics
have
highest
ROI
opportuni5es?
Metric
B
Bad
ROI
Return
Return
Metric
A
Good
ROI
Investment
Metric
C
Great
ROI
Return
n
Investment
Investment
Copyright
©
2013
Olsen
Solu5ons

48. Case
Study:
Op5mizing
Friendster’s
Viral
Loop
% of users
sending = 15%
invites
Active
Users
% of users
who are
active
Invites per
sender = 2.3
Invite
Prospective
Users
Invite
click-through rate
Click
Registration
Process
Fail
Succeed
Don’t
Click
Conversion
= 85%
rate
Users
•
Mul5plied
together,
these
metrics
determine
your
viral
ra5o
•
Which
metric
has
highest
ROI
opportunity?
Copyright
©
2013
Olsen
Solu5ons

49. The
Upside
Poten5al
of
a
Metric
?
100%
100%
85%
15%
0
Registra5on
Process
Yield
Max
possible
improvement
0.15
/
0.85
=
18%
2.3
0
%
of
users
sending
invita5ons
0.85
/
0.15
=
570%
0
Avg
#
of
invites
sent
per
sender
?
/
2.3
=
?%
Copyright
©
2013
Olsen
Solu5ons

50. Okay,
so
how
can
we
improve
the
metric?
How
do
we
increase
the
average
number
of
invites
being
sent
out
per
sender?
n For
each
idea:
n
n What’s
the
expected
benefit?
(how
much
will
it
improve
the
metric?)
n What’s
the
expected
cost?
(how
many
engineer-­‐
hours
will
it
take?)
n
You
want
to
iden5fy
highest
ROI
idea
Copyright
©
2013
Olsen
Solu5ons

51. Aper
Launching
Address
Book
Importer…
Copyright
©
2013
Olsen
Solu5ons

52. Aper
Launching
Address
Book
Importer…
Copyright
©
2013
Olsen
Solu5ons

53. Aper
Launching
Address
Book
Importer…
Copyright
©
2013
Olsen
Solu5ons

54. Op5miza5on
through
Itera5on:
Con5nuous
Improvement
Measure
the
metric
Learning
Analyze
the
metric
Gaining
knowledge:
•
Market
•
Customer
•
Domain
•
Usability
Iden5fy
top
opportuni5es
to
improve
Design
&
develop
the
enhancement
Launch
the
enhancement
Copyright
©
2013
Olsen
Solu5ons

55. Lean
Product
Ninja
Cheat
Sheet
Understand
the
problem
space
to
iden5fy
underserved
customer
needs
n Define
your
value
proposi5on
n Create
wireframes/mockups
n Get
customer
feedback
n Revise
your
feature
set,
UI
design,
and
messaging
to
improve
product-­‐market
fit
n Launch,
learn,
and
iterate
n
Copyright
©
2013
Olsen
Solu5ons

56. My
Background
n
Educa5on
n
n
n
n
n
BS,
Electrical
Engineering,
Northwestern
MS,
Industrial
Engineering,
Virginia
Tech
MBA,
Stanford
Web
development
and
UI
design
20
years
of
Product
Management
Experience
Managed
submarine
design
for
5
years
n 5
years
at
Intuit,
led
Quicken
Product
Management
n Led
Product
Management
at
Friendster
n CEO
&
Cofounder
of
YourVersion,
“Pandora
for
your
news”
n PM
consultant
to
startups:
Box,
YouSendIt,
Epocrates
My
slides
will
be
posted
to
hXp://buildinggreatproducts.com
n
Copyright
©
2013
Olsen
Solu5ons

57. Q
&
A
Moderator:
Hector
Del
Cas5llo,
CPM,
CPMM,
PMP
Presenter:
Dan
Olsen
Product
Consultant
&
Interim
VP
of
Product
Olsen
Solu5ons
@DanOlsen
dan@olsensolu5ons.com
hXp://buildinggreatproducts.com
hXp://slideshare.net/dan_o
©
AIPMM
2013
www.aipmm.com

58. Upcoming
Courses
November
26-­‐28,
2013
December
2-­‐3,
2013
Follow
the
links
above
for
more
informa5on,
or
contact
Katerina
Protonatoriou,
kpro
[at]
ait.gr
Athens
Informa5on
Technology
(AIT)
19
km,
Markopoulo
Ave.
GR
-­‐
19002
Peania,
Athens,
Greece
hXp://www.productmanagement.gr/
©
AIPMM
2013
www.aipmm.com

59. Upcoming
Courses
Zagreb,
Croa5a
Belgrade,
Serbia
Singapore,
Singapore
Washington,
DC
Singapore,
Singapore
Washington,
DC
Follow
the
links
above
for
more
informa5on,
or
visit:
Transform
Your
Career.
©
AIPMM
2013
www.aipmm.com

60. Please
Join
Us
Again!
AIPMM
Webinar
Series:
The
Art
of
a
Handoff
Mind
the
Agile
Business
Gap
Why
the
Way
You
Collect
the
Voice
of
the
Customer
MaXers
Nov
22nd,
12
pm
ET
Dec
4th,
3
pm
AEDT
Dec
6th,
12
pm
ET
hXp://aipmm.com/aipmm_webinars/
Global
Product
Management
Talk:
hXp://www.blogtalkradio.com/prodmgmXalk
Stay
Informed!
Announcements:
hXp://www.aipmm.com/subscribe
Call
For
Speakers:
LinkedIn:
hXp://www.boldpm.com/call_for_speakers
hXp://www.linkedin.com/company/aipmm
Membership:
hXp://www.aipmm.com/join.php
Cer=fica=on:
hXp://aipmm.com/html/cer5fica5on/
©
AIPMM
2013
www.aipmm.com