A startup is an organization created to deliver a new product or service under conditions of extreme uncertainty. Approximately 40 percent of all startups will cease operation with investors losing everything; 95 percent will fall short of their financial projections. And the number one cause of startup failure? No one wants to buy their product. Eric Ries, author of The Lean Startup, learned that under conditions of extreme uncertainty, classical management methods do not bring success. Based on his and others’ experiences, he formulated the Lean Startup methodology consisting of five important principles: (1) Build-Measure-Learn (BML) loop, (2) Minimum Viable Product (MVP), (3) Validated Learning, (4) Customer Development, and (5) One Metric That Matters. Lee Copeland believes these same Lean Startup ideas have great value for testers. Come and discover how the BML loop is similar to exploratory testing, how the MVP idea suggests a Minimum Viable Set of Tests, how Customer Development suggests developing clients for your testing services, and more. Learn how to apply Lean Startup ideas in your testing organization.
1. W1
Test
Management
10/4/17
11:30
The
Lean
Startup
Method:
Its
Value
for
Testers
Presented
by:
Lee
Copeland
TechWell
Corp.
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/
2.
Lee
Copeland
TechWell
Corp.
With
more
than
forty
years
of
experience
as
an
information
systems
professional
at
commercial
and
nonprofit
organizations,
Lee
Copeland
has
held
technical
and
managerial
positions
in
applications
development,
software
testing,
and
software
process
improvement.
At
TechWell,
Lee
has
developed
and
taught
numerous
training
courses
on
software
development
and
testing
issues,
and
is
a
sought-‐
after
speaker
at
software
conferences
in
the
United
States
and
abroad.
He
is
the
author
of
the
popular
reference
book,
A
Practitioner’s
Guide
to
Software
Test
Design.
3. 2
3
Lee Copeland
TechWell
lee@techwell.com
@grandpacopeland
Testing Lessons
from Lean Startups
4
4
Topics
• The Dismal Record of Startups
• Eric Ries
• Foundations of Lean Startup
• What Value for Testers?
4. 3
5
5
The Dismal Record of Startups
• A startup is an organization
created to deliver a new
product or service under
conditions of extreme
uncertainty
6
6
The Dismal Record of Startups
• Shikhar Ghosh of the Harvard Business School
reported these statistics:
– 95% of startups will fall short
of their financial projections
– 80% will fail to meet their
projected return on investment
– 40% will cease operations with
investors losing everything
5. 4
7
7
The Dismal Record of Startups
• The #1 cause of failure = No Market for their
Product
• It’s a great product or service that no one
wants to buy
8
8
Eric Ries
• An entrepreneur with an
impressive track record
– Product to connect students
with … employers
– Product to let people play in
a 3D virtual world
6. 5
9
9
Eric Ries
• Ries discovered that under conditions of
extreme uncertainty, classical management
methods do not bring success
‒ When we lack knowledge – we gather more
information
‒ When we lack alignment – we give more
detailed instructions
‒ When outcomes are not what we expected –
we impose more detailed controls
10
10
Eric Ries
• He formulated the Lean
Startup methodology
based on his and others’
experiences
7. 6
11
11
Foundations of Lean Startup
1. Customer Development
2. Build-Measure-Learn (BML) Loop
3. Minimum Viable Product (MVP)
4. Validated Learning
5. One Metric That Matters (OMTM)
12
12
1. Customer Development
• Learning and discovering who a company’s
initial customers will be, and what markets they
are in, requires a separate, distinct, and parallel
process from Product Development
― Steve Blank
9. 8
• The MVP is that version of the product that
enables a full turn of the Build-Measure-Learn
loop with a minimum amount of effort and the
least amount of development time
15
15
3. Minimum Viable Product (MVP)
• The minimum viable product lacks many
features that may prove essential later on – and
that’s OK
16
16
3. Minimum Viable Product (MVP)
(Final product)(Minimum
viable product)
10. 9
17
17
4. Validated Learning
• Validated learning is “the process of
demonstrating empirically that the team has
discovered valuable truths about the present
and future business prospects”
18
18
5. One Metric That Matters (OMTM)
• Most metrics gathered by organizations are
“vanity metrics.” They make us feel good (or
look good), but don’t really show progress
toward our goal
Number of
customers
Cost to obtain
each new
customer
Number of
customers
actually using
the product
11. 10
19
19
5. One Metric That Matters (OMTM)
• The One Metric That Matters measures the one
most important thing at the present state of the
startup
20
20
The Lean Startup Methodology
What Value for Testers?
12. 11
21
21
What Value for Testers?
• As software testers, we are rarely part of an
entrepreneurial startup team
• Are there lessons for us from the Lean Startup
methodology?
22
22
1. Customer Development
• Ask:
– Who are our customers?
– What are their problems
our testing solves?
– Do our customers perceive
these problems as
important?
– Are they willing to pay for
our testing services?
13. 12
23
23
1. Customer Development
• Customer Development suggests that we
should identify the “customers” of our
TESTING SERVICES:
– Developers
– Users
– Management
And serve THEM, not our testing process
24
24
1. Customer Development
• Or this could be our future …
Largest dot-com
flop in history
Raised $375 million
in first public
offering on revenue
of $395,000 and net
losses of $50
million
14. 13
25
25
2. Build-Measure-Learn Loop
• BML loop is equivalent to the Exploratory
Testing loop
This
is
where
we
start
Not
here!
26
26
3. Minimal Viable Product (MVP)
• The idea of an MVP
suggests we could
start with a Minimal
Viable Set of Tests
• Don’t strive for
“completeness” from
the beginning
• Add additional tests
as the need becomes
apparent and
warranted
Learn
what
your
customers
want/
need/value
15. 14
27
27
4. Validated Learning
“It isn't what we don't
know that gives us
trouble, it's what we
know that ain't so”
Will Rogers
American Humorist
28
28
4. Validated Learning
• The Lean Startup
methodology suggests
running frequent
experiments to determine
customer response
• As testers, we can run
frequent tests to determine
both system capabilities
and customer satisfaction
16. 15
29
29
5. One Metric That Matters
30
30
5. One Metric That Matters
• Is your metrics program an example of
“success theatre”?
17. 16
31
31
5. One Metric That Matters
• Replace the dozens of vanity metrics …
– Test cases planned
– Test cases implemented
– Test cases executed
– Test cases passed
– Test cases failed
– …
32
32
5. One Metric That Matters
• … with one that is truly indicative of something
vital
18. 17
33
33
5. One Metric That Matters
• Does your metric
– Measure your success at improving quality?
– Directly relate to product success?
– Tie to real customers of your service?
– Help you determine what to do next?
Ivory Madison, “Bonfire of the Vanity Metrics”
34
34
In Conclusion
• The Lean Startup methodology has many
insights to offer to software testers
• Become familiar with its precepts. Adopt its
ideas for your testing
19. 18
35
35
Thanks
Tạm biệt từ Việt NamGood bye from Viet Nam
36
36
One Final Bit of Advice
Be Open to Serendipity
“Finding something good
without looking for it”