Presentation by Hiten Shah (CEO, Kiss Metrics) at the 'Lean Startup, Lean Investor' event on November 3, 2010 (Produced by 500 Startups & Nokia/Nokia Growth Partners)
6. The Startup Pyramid
The Race to the Top
Growth
Transition to Growth
Product / Market Fit
Sean Ellis: Startup-Marketing.com
7. AARRR
Created by Dave McClure - 500startups.com Designed by
1. ACQUISITION
Customers come from various channels
2. ACTIVATION
Customers use product for first time
YOUR PRODUCT
3. RETENTION 4. REFERRAL
Customers come back multiple times Customers refer others
5. REVENUE
Customers make your business money
8. Minimize TOTAL time through the loop
IDEAS
Learn Faster Build Faster
LEARN BUILD Unit Tests
Split Tests
Customer Interviews Usability Tests
Customer Development Continuous Integration
Five Whys Root Cause Incremental Deployment
Analysis Free & Open-Source
Customer Advisory Board Components
Falsifiable Hypotheses DATA CODE Cloud Computing
Product Owner Cluster Immune System
Just-in-time Scalability
Accountability
MEASURE Refactoring
Customer Archetypes Measure Faster
Cross-functional Teams Developer Sandbox
Semi-autonomous Teams Split Tests Funnel Analysis Minimum Viable Product
Smoke Tests Clear Product Owner Cohort Analysis
Continuous Deployment Net Promoter Score
Usability Tests Search Engine Marketing
Real-time Monitoring Real-Time Alerting
Customer Liaison Predictive Monitoring
Eric Ries: StartupLessonsLearned.com
9.
10. Build Faster
• Test Driven Development (TDD / Unit Tests)
• Continuous Deployment
• Continuous Integration
• Just-in-time Scalability (Amazon Web Services)
“ On average we deploy new code fifty times a day.”
Timothy Fitz, IMVU
11. We have purposefully deployed:
• Poorly designed features
• Inflexible, short-sighted code
• Half-done features
• Known bugs
• Crappy code with major performance issues
12. Measure Faster
• Funnels & Cohort Analysis (KISSmetrics)
• Feedback (NPS, Surveys, KISSinsights)
• Usability Testing (UserTesting, Mockups, etc...)
• Split Tests (A/B Testing) (KISSmetrics + Internal)
“ It's about changing customer behavior -- for the better.”
Eric Ries, The Lean Startup
14. Learn Faster
• Fake it till you make it (Smoke Tests)
• Five Whys Root Cause Analysis
• Falsifiable Hypotheses
• Customer Development (Customer Interviews)
“ Customer Development will help you – force you – to make
better decisions based on tested hypotheses, rather than
untested assumptions.”
The Entrepreneur’s Guide to Customer Development, CustDev.com
15. Original Hypothesis for KISSinsights
“Our hypothesis is that product manager type people have a
problem doing fast/effective/frequent customer research.”
17. No Departments, Just Two Teams
?
Problem Team
What are the
☼
Solution Team
Find the
most important minimum solution
problems to solve? for a given problem.
I don’t know a single entrepreneur who would disagree with this statement. And many of you know, I know a lot of entrepreneurs...
I’ve failed a lot and I want to fail faster and fail smaller. By now, everyone knows that successful startups are a series of “failures”. You can’t succeed without trying a lot of things that don’t work.
In this picture everyone is out of the building talking to customers or...
Talking to one of these guys... Jokes aside, I am humbled to have these guys involved with KISSmetrics. I won’t talk about them, since you all probably know of them. Personally, I’ve learned a lot from them about startups and entrepreneurship.
Sean Ellis and his 3 stages of a startup Metrics of success are different. Reminds us where we are and our success metrics...
AARRR!!! Dave McClure’s pirate metrics. Helps you frame key areas of your business based on a customer’s lifecycle. Helps determine the tactics you should be thinking about. We use this for idea generation.
And of course, Eric’s lean startup graphic. It’s all about validated learning about customers... So we can build something a LOT of people want to buy.
So now i’ll go into our story at KISSmetrics on how we think of Building, measuring and learning faster....
Have the ability to ship code all day, every day. This leads to faster, more measurable iterations...
This has resulted in sub-par, sub-optimized code. But our customers don’t care, in fact they are delighted by our daily iterations. Customer’s don’t care how awful the code is and you shouldn’t either. Remember, it’s about getting validated learnings...
Here are some key areas and tools we use for measuring our business. Idea is to focus on a few key pieces of data at any given time. Here is one way we do that...
We came up with a model for our business. We then establish our actual baseline which became our macro metrics. Find the riskiest assumptions in your model and macro metrics. And attack them...
Learning is all about testing your assumptions. Put up landing pages, added navigation, whatever it takes... To get validated learnings about customers. Learn about the key risks in your model.
We always try to write down our hypothesis, so we know what we are trying learn. Question: Is this hypothesis in fact true or not?! Take the least amount of effort to find out... Ex. Orig. hypothesis for KISSinsights...
The principles of build, measure, learn help us think about how to get answers. The rules of communication and information exchange are much different in a lean startup. The tactical stuff to build, measure and learn faster is one side of things. Now I’m going to talk about the business processes and what we have learned.
Company is divided into two teams. No sales, marketing, etc... Problem team identifies the most important problems to solve. Solution team finds the minimum solution for each problem. Goal is to go through this cycle continuously.
Monday - Team meeting to describe most important problems and goals for the week. Metrics + Few tasks, Tuesday. Wednesday - Check in on how are we doing, solutions already getting implemented? What have we learned so far? Friday - Postmortem on learning and peek into next week from the Problem Team. Problem team is 1-week ahead.
Tool to increase autonomy and communication, focused on our validated learnings driven process. Weekly goal and metrics based on most important problems. Team visibility across all tasks, keep solution team focused on important problems. Provide tools for problem team to identify next things. 5 tasks person, only 1 is active (green-lighted), not in here then not exist, backlog.
We’ve learned some lessons in our efforts to apply lean startup principles. Here are three key ones.
Optimizing one area in the BML loop causes bottlenecks in others. Once we implemented the infrastructure to build faster, we then increased our ability to discover customer problems. But didn’t have a good process for organizing and finding solutions. Once we had that, we started discovering a lot about our team...
They became more accountable. Team members started “policing” themselves. One of our engineers likes to get himself in a hole and just code for days. Doesn’t come up for air and only 50% of time does he produce fantastic results. Our process helped him catch himself and in fact stop before he went on for a long time.
Quote from our CTO, John. A lot of discipline is required to stay focused on validated learnings. We now focus on iteration based on validated learnings instead of perfection. This was really hard, we are a team of perfectionists!