Weitere ähnliche Inhalte Ähnlich wie ProductCamp Vancouver 2013 (20) Mehr von Dave Sharrock (20) Kürzlich hochgeladen (20) ProductCamp Vancouver 20131. Lean Product Management
The Value of Experiments
@DaveSharrock
ProductCampVancouver 2013
Photo: ESA
http://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright ©2007-2013
agile42 | We advise, train and coach companies building software www.agile42.com | All rights
2. Assembly Line Manufacturing
has a high cost of change
http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspx
Archives of Ontario, Reference Code: C 190-5-0-0-21
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
3. Inclusive thinking
Feature usage in
• In software development high
cost of change leads to inclusive enterprise software
thinking Never
Rarely
Sometimes
Often
Always
• Any and every idea has to be 7%
captured in the first version of a 13%
requirements specification
• Creates waste - bloated 45%
documents, unwanted features 16%
and entitlement thinking
19%
Standish Group - CHAOS Report, 1995, 2002
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
4. Cost of Hypothesized
Change Requirements
Capture all
possible needs
Detail of
Requirements
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
5. Software Development has
commoditized cost of change
Object-oriented
languages
jquery
Ruby on Rails
Coffeescript
Continuous Delivery
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
6. Cost of Hypothesized
Change Requirements
Evolving
Requirements
Capture all
possible needs
Emerging needs
as development
progresses
Detail of
Requirements
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
7. Changing paradigms
• If change is cheap, requirements
can change continuously
• We can evolve our thinking as
we learn more about the product
we are building
• Maximizing Return-on-
Investment means validating new
features early, before the total
investment cost has been spent
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
8. Cost of Hypothesized
Change Requirements
Evolving
Requirements
Validated
Requirements Capture all
possible needs
Emerging needs
as development
progresses
Lean Startup
experiments
Detail of
Requirements
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
9. Shifting Paradigms
B.G.|A.G.
“I know that half of
my advertising
budget is wasted,
but I’m not sure
which half”
Lord Leverhulme
http://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditional
http://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.php
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
10. Validated Learning
• Understand the problem a
feature will solve
• Decide how to ask if your
customer cares
• Validate before investing
complete cost
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
11. The Scientific Method
Theory Use the theory
to make a
prediction
Modify or change
your theory
Observe Prediction
Design an
experiment to
test the
Run the prediction
experiment Experiment
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
12. The Theory of General Relativity
Gradual acceptance of
General Relativity over Einstein’s Theory of
General Relativity
Newtonian Mechanics Einstein’s
prediction
(1907)
Observations validated
Light bends in a
General Relativity
gravitational field
Wait for 1919 solar
Arthur Eddington eclipse to observe
observed that Light did whether or not light
bend around the sun bends around the sun
Solar Eclipse of 1919
http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
13. Applied to an Online Experience
User Model Use the model
to make a
prediction
Modify or change
your user model
Validate Prediction
Design an
experiment to
test the
Run the prediction
experiment Experiment
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
15. User Story (Journey) Mapping
Elements of Customer Journey
Different Customer
Your model for
Experiences
customer interaction covers
each step of the customer
Each row matches a different
journey
customer experience
Story Mapping © Jeff Patton, all rights reserved, www.AgileProductDesign.com
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
16. Vanity vs. Actionable Metrics
Vanity metrics (like web hits or number of
downloads) which only serve to
document the current state of the
product but offer no insight into how we
got here or what to do next.
An actionable metric is one that ties
specific and repeatable actions to
observed results.
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
18. Depersonalize
Decision Making
• “HiPPO’s” rule the business world
• It is never about you, it can’t be
about you
• Benchmarking is awesome
• Leverage competitive analysis
• Experimentation / testing rock
• Execution strategy:
• Transparency, standardization,
looking outside in
• Be a slave to customer
centricity (its about your
customers (internal & external))
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
19. “I can’t say it any better, HiPPO’s rule the world, they over rule
your data, they impose their opinions on you and your
company customers, they think they know best (sometimes they
do), their mere presence in a meeting prevents ideas from
coming up.
The solution to this problem is to depersonalize decision
making, simply don’t make it about you or what you think. Go
outside, get context from other places. Include external or
internal benchmarks in your analysis. Get competitive data (we
are at x% of zz metric and our competition is at x+9% of zz
metric).
Be incessantly focussed on your company customers and
dragging their voice to the table (for example via
experimentation and testing or via open ended survey
questions). Very few people, HiPPO’s included, can argue with a
customer’s voice, the customer afterall is the queen / king! : )”
Avinesh Kaushik, kaushik.net
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
20. Interaction Model Backlog Validation Execution
To Do Doing Observation Validated? Doing Deployed
For <target group>
Who <has a specific need> Idea
#1.1
The <product name>
Is A <product category>
That <key benefit>
Idea
Unlike <primary competitors> #1.2
Our Product <additional
advantages>
Idea
Idea #1.3
Idea
#1.5
Pathway #1.4
#1
Idea Idea
#1.6 #2.1
Idea
Pathway #2.4
Idea Idea
#2 #2.3 #2.2
Level of Detail / Close to done
•Visualize your User Interaction Model •Always revisit validation tests for size •The execution board is equivalent to
and adjust it as you learn the team’s task board, but at the Idea
•Come up with ideas for each pathway Card level
•Prioritize experimental testing
•Spend time on measurement
infrastructure (a test without ability to
measure = an unvalidated feature)
21. How Much Effort on Validation
Feature usage in
• In 2009, Google ran
approximately 12,000 enterprise software
experiments, of which about Never
Rarely
10% led to business changes Sometimes
Often
Always
7%
• Linus Pauling famously said, 13%
"The best way to have a good
idea is to have a lot of ideas."
45%
• Focus on minimizing the 16%
incremental cost of testing ideas
with your customers/users
19%
Standish Group - CHAOS Report, 1995, 2002
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
22. Feedback loops make
all the difference
In 1940, the chance of an
American being killed by
lightning was about 1 in
400,000.
Today it’s 1 in 11 million.
In November 2007,
economists in the Survey of
Professional Forecasters —
examining some 45,000
economic-data series —
foresaw less than a 1-in-500
chance of an economic
meltdown as severe as the
http://www.nytimes.com/2012/09/09/
magazine/the-weatherman-is-not-a-
one that would begin one
moron.html month later.
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
23. Lean Product Management
The Value of Experiments
• Validate requirements trump everything
• Incremental cost of validation is falling
• Feedback means nothing without a model
The race doesn’t go to the one with the best idea,
but to the one with the most tested ideas
Photo: ESA
http://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright ©2007-2013
agile42 | We advise, train and coach companies building software www.agile42.com | All rights
24. “Coming together is a beginning.
Keeping together is progress.
Working together is success.”
Henry Ford
thank you
dave.sharrock@agile42.com
skype: dave.sharrock
twitter: @davesharrock
slides: slideshare.net/davesharrock
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright © 2007 - 2009.