Slide KPIs & Metrics - Innlab RM 29-04-2013

1. KPIs & metrics
@matteopanfilo
matteopanfilo@gmail.com
InnLab RM 2013

2. About me
1
What I’ve done until 2012:
I studied economics at LUISS
I wrote a thesis on the VC
I published a research on the VC that has ended up on the IlSole24Ore
I won InnovAction Lab 2011
I went to Silicon Valley
I worked @ Principia SGR (8 of the last investments ≅ 15mln)
What I'm doing since the beginning of 2013
I work @ Boox Venture Incubator
I’m a member of several professional associations promoting entrepreneurship
I continue research activities
I continue to study (a lot)

3. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
2

4. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
3

5. 1. Different businesses, different metrics
Airbnb
Fiendfinder
Blogo
6sicuro.it
Privalia
4
Number of house with HD pics
Viral factor (K)
Pages per user
Generated quotes (lead)
Cost per shoting

6. Hundreds of different metrics
PV Page Views
UV Unique Visitors
CPM Cost per Thousand
BR Bounce Rate
CTR Click Trough Rate
CPR Cost Per Registration
CPL Cost Per Lead
CPA Cost Per Acquisition (or
Cost per Action)
CPC Cost Per Click
LTV Lifetime Value
CR Conversion Rate
Churn Churn Rate
5

7. Lean Analytics
6
http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/

8. Lean Analytics (alcuni esempi)
7
http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/

9. Scalability (basic formulas)
LTV
CPC
%CR
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/
Dipende fondamentalmente
da: soddisfazione dei clienti,
CRM, biz model, etc
<CPA
8
“LTV is the projected profit that a
customer will generate during his
lifetime”
General CPA = $ Adv cost / # Customers acquired
CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired
LTV formulas
• LT Revenues: # Purchases x $ Avg. basket
• LT Revenues: $ Avg. basket / % Churn rate
• LT Gross Profit: # Purchases x $ Avg. basket x % gross profit margin
• LT Gross Profit: $ Avg. basket / % Churn rate x % gross profit margin
• LT DCF: ($ Avg. basket margin x Life Expectancy) x (Customer retention rate / 1 + discount
rate – Customer retention rate)

10. Scalability (basic formulas)
LTV
CPC
%CR
<CPA
9
Small numbers,
huge impact:
CPC = 0,4€
%CR = 4%
CPA = 10€
CPC = 0,4€
%CR = 2,5%
CPA = 16€
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/

11. Scalability, make it happen!
10
Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
Note: CAC = CPA

12. Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
Scalability, how you can do it?
Note: CAC = CPA
11

13. Conversion
What’s the meaning of conversion rate?
What’s the meaning of funnel?
CPC
%CR
CPA =
Define what are you looking to measure!
i.e.: define who you consider a customer; define the difference between
CPL and CPA (if there is)
12
General CPA = $ Adv cost / # Customers acquired
CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired

14. The Funnel ..<offline mode>
13
1. A lot of people see the adv
wall
2. Some people recognize it
walking in front of a
sunglasses shop
3. Some of them decide to go
in and try them
4. One of them want to buy it

15. The Funnel ..<online mode - ie. ecommerce>
100
50
20
5
3
1
1. I paid to have some users on my
website
2. They arrive on the landing page
3. They leave their email
4. They put one product in the basket
5. They start the checkout process
6. They complete all the payment
info
7. They pay
50%
40%
40%
38%
33%1%
14

16. The Funnel ..<online mode - ie. ecommerce>
100
50
20
5
3
1
Paid users
Landing page
Leave the email
Product on the basket
Checkout process
Payment info
They pay
• Pictures
• Copy/text button
• Info required
• Password
• Email confirm
• Add button
• Related product
• Info required
• Shipping days
• Delivery cost
• Payment gateway
• Invoice/legal info
• CC circuit
• PIN request
Some drivers:
15

17. Why measure the funnel
You need to identify the bottleneck and solve it!
Examples:
- Pictures
- Checkout process
- Payment gateway 16

18. The Funnel ..(flash sales ecommerce)
Users that
arrive till
checkout
Registered users
Unique users
Users that complete
the form
Paying
users
1%-0,5%
Measure your CPL/CPA!
The CPA initially refers only to
the traffic resulting from paid
campaigns is the main KPI to
be monitored in an
ecommerce / SaaS, but in
the medium term it should
be monitored along with the
metrics of the traffic that
came from other channels
(email, facebook page,
etc. )
17

19. Customer journeyUsers
Customers
Source: IAB Seminar 2013, Ilaria Zampori Managing Director QUISMA, Individual Conversion Attribution
http://www.iabseminar.it/wp-content/uploads/2013/02/9_QUISMA.pdf
Awareness Consideration Conversion
Loyalty
ChoicheInformationDesire
18

20. Customer journey
Awareness Consideration Conversion
Loyalty
Source: IAB Seminar 2013, Marco Caradonna Simple Agency Dynamic Customer Journey
http://www.slideshare.net/mcarado/4-iab-seminar-performance-2013-simple-agency
ChoicheInformationDesire
19
instruments instruments instruments

21. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis, measure customer behaviour
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
20

22. Cohort Analysis referral source analysis
“A cohort is a group of people who share a common
characteristic over a certain period of time”
GoogleFacebook Blog
Customer spending by referral source
Direct
Source: www.cohortanalysis.com/
21

23. Cohort Analysis time-period analysis
Time period
1 2 3 4 5 6 7 8 9 10 11 12
Time-period cohorts
1 513 410 328 263 210 168 134 108 86 69 55 44
2 923 739 591 473 378 303 242 194 155 124 99
3 1.258 1.007 805 644 515 412 330 264 211 169
4 1.537 1.230 984 787 630 504 403 322 258
5 1.777 1.421 1.137 910 728 582 466 373
6 1.987 1.590 1.272 1.017 814 651 521
7 2.178 1.742 1.394 1.115 892 714
8 2.355 1.884 1.507 1.206 965
9 2.524 2.020 1.616 1.292
10 2.689 2.151 1.721
11 2.852 2.282
12 3.017
paying users 513 1.334 2.325 3.398 4.495 5.583 6.644 7.671 8.661 9.618 10.547 11.454
How much revenue do the users that I’ve acquired on the second months
generate?
How many users that I’ve acquired on the second month will I probably still
have on the ninth month?
Source: andrewchenblog.com
22

24. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate, do not throw the job done
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
23

25. Churn Rate
Your churn rate is the amount of customers or subscribers who cut ties
with your service or company during given time period.
These customers have “churned”
Recurring revenues model Traditional Ecommerce
# Subscribers lost
# Starting subscribers
= % Churn rate
# Customers who has
not made a
purchase in 90 days*
# Starting active
customers
= % Churn rate
Source: www.churn-rate.com
24
* it depend on
business model
and offer

26. Optimize..
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
25
Analyse
Change
LearnConsolidate
Iterate
Learning
process

27. Why you need to optimize..
• You have 0-8 seconds to make a compelling headline and
landing page. After 8 seconds, the majority of visitors leave.
• Approximately 96% of visitors that come to your website are
not ready to buy.
• The more landing pages you have, the more leads you are
likely to get.
• Product videos can increase purchases of the product by
144%.
• A 1 second delay in your site speed can result in a 7%
reduction in conversions.
• A/B testing is becoming the preferred method that has
brought a lot of the companies the most success.
Source: http://blog.kissmetrics.com/what-converting-websites-do/
26

28. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test, what do customers prefer?
6. Source channels, email/DEM
7. CRM
Conclusion
27

29. A/B test…
• Elaborate 2 different versions (text, colors, font, layout, etc)
• Propose versions to the target sample (with the same content)
• Measure conversion rate
• Profile users based on conversion rates (male/female, addicted, etc)
• (Visualize where the users click on the pages)
The same process could be applicable to web pages, email/DEM,
games dynamics, etc!
28

30. Headline..
Original
+30%
+15%
29Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

31. Layout..
30Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

32. Layout..
31Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

33. Layout..
Is not always a good
idea to write more…
32Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

34. A/B test are easy but..
Source: http://techcrunch.com/2013/01/12/current-conversion-rate-and-desired-
confidence-interval-will-help-you-avoid-analysis-paralysis-stop-running-stupid-tests/
Consider a site that has 10,000 visitors per month and has a 5 percent conversion
rate. The table below shows how long it will take to run a “conclusive” test (95
percent confidence) based on how much the change impacts conversion rate.
33

35. Other tips..
• Badging
up to +50%
• Message consistency
up to +100%
• International website
up to +100%
• Email sign-up
up to +1.000%
Fonte: http://www.slideshare.net/Emerce/emerce-performance-bryan-eisenberg
34

36. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
35

37. Email & DEM
1. Choose & Optimize email form
2. Choose & Optimize password request
3. Choose the confirmation process
4. Choose emails’ arguments and style
5. Choose & Optimize email subjects
6. Metrics, metrics, metrics:
• Open rate/Unique OR
• Click rate
• Click-through-rate
• Delivery time
• List growth
• Unsubscribe rate (or churn rate)
• Spam complaint rate
Drivers:
Ø Subject, time
Ø Content
Ø Layout
Ø All of the above
Ø Too many
Ø Frequency & content
Ø Too many
Email could become your first traffic source, improve the LTV of your
customers, decrease your customer acquisition cost !
36

38. Email & DEM
Source: http://retail-analytics.quora.com/Email-Open-Click-Through-Analytics-from-over-80-Billion-Emails
37

39. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
38

40. The true cost of bad support
39Source: https://www.helpscout.net/blog/bad-customer-service/

41. Interact with customers…
People now expect the same—if not better— level of
service from online storefronts and service channels as
they do from a visit to a retail store or a phone
conversation with an agent.
Source: “ E mail Customer Service in North American Small and Medium Businesses” by
BenchmarkPortal, 2005 and https://www.helpscout.net/blog/bad-customer-service/
40

42. Source: Communication Preference, Live Chat Effectiveness, A Survey of Internet Shoppers 2012
Interact with customers…
41

43. Source: “Email Customer Service in North American Small and Medium Businesses” by BenchmarkPortal, 2005,
Source: Forrester Research Inc., 2008
83% of small and medium-sized
businesses responded with
inaccurate or incomplete answers.
41% of consumers expect an e-mail
response within six hours.
Only 36% of retailers responded that quickly.
Interact with customers…
42

44. “Customer experience quality could
result in a swing of $184 million for a
large Internet retailer.
Source: Forrester Research Inc., 2008
Interact with customers…
43

45. The power of great video…
Video: http://www.youtube.com/watch?feature=player_embedded&v=ZUG9qYTJMsI
44

46. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
45

47. Recap & useful tools
1. Metrics, measure and understand your KPIs
2. Funnel, know the steps to improve your conversion
3. Cohort analysis, analyse customer behaviour
4. Churn rate, do not throw the job done
5. A/B test, what do customers prefer?
6. Source channels, email/DEM still a very interesting
channel
7. CRM, interact and learn from your customers
- Mixpanel
- Jirafe free
- Woopra
- Kissmetrics
- RJ metric
- Geckoboard
46

48. My 2 cents
Non pensate che fare una startup sia un gioco.
Cercate di creare valore, non fate le cose perché vanno di moda
e non sviluppate l’ennesima app per condividere le foto.
Non pensate a raccogliere soldi ma pensate a creare un prodotto
e ad avere una visione per i successivi 10 anni.
Studiate e documentatevi tanto prima di prendere ogni decisione,
c è un enorme quantità di informazioni online, basta saperle
cercare.
47
“I believe in the power of a well-operated, sophisticated organization
that generates social and economic value. My career goal is to found
and lead this type of organization.”
Hansoo Lee

49. • Steve blank tools (www.steveblank.com/tools)
• Avinash Kaushik (www.kaushik.net)
• Quora (follow Matthew Caroll)
• Slideshare (xerox, 500startups, etsy)
• Techcrunch
• Angel.co
• Italian Startup Scene
• Indigeni Digitali
• Startupnumbers.com ;)
48
Must read/view:

50. • Blog etsy (http://mcfunley.com/)
• Blog Fabrice Grinda
• Blog fab.com
• Blog google
• Blog a16z.com
• Foundat.io/n
• ecorner.stanford.edu
• avc.com
• Angel.co
Must read/view:
49

51. Primo esempio: wikli
+70%
Spesa tot: 200€
Strumenti utilizzati: opimizely
Utenti attuali >6.000

52. Secondo esempio..

53. Secondo esempio…iterazione

54. Secondo esempio…iterazione

55. Terzo esempio…non solo web

56. Quarto esempio (fashion subscription commerce)
Conversion rates:
From inception October 30th until December 31th
• Unique visitors to registered free members: 27%
Benchmark successful players: circa 22%
• Free members to paying subscribers: 2,6% in 2 mesi
Benchmark successful players: circa 3% in un anno
Absolute metrics:
Form inception November 5th until December 30th
• 26k unique visitors
• 7k registered members
• 200 subscription sales
• 7k $ revenues
Traffic sources:
• Direct: 59%
• Facebook 23%
• Daily Deal Sites 10%
• Others 9%

57. Always in test..
Video (da vedere!): http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313
“We put it in a feature that we can
build in a week. It's a ghetto build. We
AB test it. We flow test it. We put it out
to 1%, 10% of our users. We build a data
warehouse with the testing platform.
So, we're running several hundred tests
at any given time for every one of our
games. And no single user ever is
involved in more than one test. So, we
love testing. When we see that it moves
our metrics in a considerable way,
that's when we take it to be a full
feature roll out and then we do the full
2.0.
Capitalizzazione 3,6 $bn --- Ricavi 1,22 $bn --- 12% ricavi da facebook

58. Always in test..
Sources: http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313
“We don't want to go down those paths until we've test them
with our users. And what's amazing and this is a feature that you
will all have available to you as you enter this third Internet
business is that you're going to run a service and you are going
to every week be able to test things with your users and
something that I never have available to me in previous
companies. It won't be build it for three months and hope and
pray. It's test it now. And you can test it with - you don't need to
be a big company - you can test it by buying Google AdWords
or Facebook Self Service ads. You can put these links out. You
could spend 50 bucks a day and know most of what we know.
So, it's an awesome way...”
Mark Pincus – Founder & CEO @ Zynga