1. A Product Improvement
Decision Making Process
for (SaaS) Startups
Jan König - @einkoenig

2. Lean Startup and Metrics
Is Build-Measure-Learn too simplistic?

3. Improving an existing Product
Where do the important ideas come from?

4. “Startups don’t starve,
they drown. ”
Shawn Carolan
Menlo Ventures

6. Lean Startup and Metrics
We all know
Measuring the right things is important.
We all know the difference between vanity metrics and actionable metrics.
We do cohorts, we do funnels. But how often does this lead us to actions that
could really improve our products?
my thesis
We lack a systematic approach that encourages
us to act on metrics on a regular basis.

7. Indication
Test In-depth
Process for Product Analysis
Improvement
Decisions
Hypothesis Diagnosis

8. Indication
Look at a dashboard of macro-metrics on a regular basis (daily, weekly,
monthly).
Is there something unusual?
Where do my users get stuck? On which lifecycle stage do I lose most of my
users?
Identify your key macro metric you want to improve (e.g. Activation,
Retention…).
Tools
USERcycle
to be updated

9. Dashboards
Indikation
Indication Macro Metrics
Funnel Analysis
Test In-depth
Analyse
Analysis
OK, something seems
to be wrong, here!
Hypothese
Hypothesis Diagnose
Diagnosis

10. In-depth Analysis
Do Customer Development!
Ask the users who get stuck why they get stuck.
Ask your happy users what makes your product special to them.
Do Lifecycle Messaging.
Dive deeply into data you already have (e.g. steps until Activation).
What particular step is a barrier?
Tools Intercom USERcycle
to be updated

11. Indikation
Indication
Test In-depth
Analyse
Get additional Analysis
information by diving
deeply into data and
customer questioning.
Micro Metrics
Lifecycle Messaging
Customer Development
Hypothese
Hypothesis Diagnose
Diagnosis

12. Diagnosis
Aggregate your data!
What is the real reason for the unsatisfying product usage?
Involve your team.
Tools
to be updated

13. Indikation
Indication
Test In-depth
Analyse
Analysis
Use data to identify
potential problems in
product usage.
Hypothese
Hypothesis Diagnose
Diagnosis
Brainstorming

14. Hypothesis
What do we have to change?
Again: Involve your team.
Is it a feature? An onboarding process? Do users need to be educated?
Define a falsifiable hypothesis:
„*Change of product] will increase user activation by *.+%“
Tools
to be updated

15. Indikation
Indication
Test In-depth
Analyse
Which product Analysis
changes could possibly
improve product
usage?
Brainstorming
Hypothese
Hypothesis Diagnose
Diagnosis

16. Test
Finally! We‘re at „Build“
Do Usability Tests.
Split-Test it.
Never just introduce a new feature to every single user!
Tools Analytics
Content Experiments

17. Usability Tests Indikation
Indication
Split Tests
Test In-depth
Analyse
Analysis
How does this change
affect consumer
behavior?
Hypothese
Hypothesis Diagnose
Diagnosis

18. Validation / (Indication, again)
Did something change?
Bravo! Let’s improve the next thing!
Maybe this wasn’t the right solution for your “Diagnosis”.
Bad! Go to “In-depth Analysis” to find out why!
Tools
USERcycle
to be updated

19. Indication
Test In-depth
Process for Product Analysis
Improvement
Decisions
Hypothesis Diagnosis

20. Feedback?
Hey, I’m Jan!
I am a German student and I’m currently writing
my bachelor’s thesis about metrics for startups.
I would love to get some feedback about this approach
(which is the “high-level part” of my thesis).
I will upload some other presentations about metrics
design in startups within the next weeks.
Glad to hear from you!
Jan
Twitter: @einkoenig
Mail: me@jan-koenig.com