1. Development Metrics:
Measuring Your Site for Improvement
9 – 10 October 2013
Moscow, Russia

2. 2
Introduction
What is the one thing that you would like to know about your site
and what would you do if you knew it?
Some Examples:
 What kinds of content does my online audience want?
Action: Provide more stories like the ones they like
 What parts of my site are easy to use and what parts are difficult?
Action: Make my site easier to use
 Why is my site so slow?
Action: Make the improvements to make the site faster

3. 3
Metrics Overview
An online company relies on metrics to make decisions at three
different levels, based on the metrics purpose and frequency of
collection.
1. Foundation Metrics
2. Key
Performance
Indicators
3. Tactical
Metrics
Reported Regularly:
• Visits
• Unique Visitors
• Page Views
• Basic Demographics
Reported Regularly:
• Engagement Metrics
• Cost per Acquisition
Address a
Specific Problem
• A/B Testing
• Heat Maps
Website Metrics Structure

4. 4
Foundation Metrics
In the past we have talked about the basic metrics provided by
platforms like Google Analytics and Yandex Metrika.
Foundation Metrics:
- Visits
- Unique Visitors
- Page Views

5. 5
Foundation Metrics
A common Foundation Metric is a description of the audience –
audience demographics.
 In Russia, Yandex provides much better data on demographics than
Google on age and gender as well as purchase intent.

6. 6
Foundation Metrics
But neither Google or Yandex provide in-depth and customized
descriptions of your audience.
 To create a deeper picture of your audience, there are syndicated services
which will add you to their regular research for a fee or provide broader
market information
- Gemius Russia : www.gemius.ru
- TNS (Gallup) : www.tns-global.ru
- KissMetrics : www.kissmetrics.com
- WhyAnalytics : www.visualdna.ru; why.visualdna.com/analytics/
Among many others.
 If you are using your audience description to answer the questions of
advertisers, be aware that advertisers tend to prefer the data of certain
companies.

7. 7
Foundation Metrics
For customized data about your audience or to fulfill on an
advertiser’s request for a specific question, you may want to
consider creating your own online survey.
 Google Forms creates a simple, no-cost way to collect simple information
on web users – YouTube Intro (English)
 Survey Monkey (https://ru.surveymonkey.com/) provides a more
sophisticated set of tools for online surveys but will require an annual
payment (R10,000 to 35,000).
Main Features:
- Multiple delivery platforms (online, email, text, social networks, …)
- Integration with statistics software for analysis
Main Uses:
- Customer Satisfaction
- Special Demographics (Purchase Intent, …)
- New Product Feedback

8. 8
Foundation Metrics
Foundation metrics are a good first step, but are usually too general
to help make decisions.
 Foundation metrics only support very general observations:
- How big is the audience?
- How fast is it growing or declining?
 Foundation metrics alone don’t help you understand the cause of what’s
happening on your website?
- Why is the site growing?
- Who is the new traffic? Where do they come from?
 By the time, you see that visits are declining or growing, it is too late to
do anything about it.

9. 9
KPI’s
Key Performance Indicators are metrics that are collected regularly
that help you manage your online business.
1. Foundation Metrics
2. Key
Performance
Indicators
3. Tactical
Metrics
Reported Regularly:
• Visits
• Unique Visitors
• Page Views
• Basic Demographics
Reported Regularly:
• Engagement Metrics
• Cost per Acquisition
Address a
Specific Problem
• A/B Testing
• Heat Maps
Website Metrics Structure

10. 10
KPI’s
Key Performance Indicators (KPI’s) are regularly collected data that
help you make decisions about your online business.
 To create KPI’s for your business, you need to determine what actions are
most important to the success of your website. For news sites, the most
common types of measures are:
- Engagement (Bounce Rate, Time on Site,…)
Data Source: Web Analytics Service - Yandex or Google
- Content Dynamics: (Most Read Category, Most Shared Story)
Data Source: Web Analytics, Social Media Analytics, CMS Analytics
- Efficiency (Cost Per Acquisition – CPA, Revenue Per Thousand - RPM)
Data Source: Ad Server/Ad Network + Web Analytics + Accounting
Classified Sites will have different actions which define success.
 KPI’s will often involve bringing together data from many different
sources.

11. 11
KPI – Engagement
To understand how engaged your audience is with your website,
there are several industry standard measures.
Engagement Metrics
 Session Length: Length of time set to record metrics.
- Google default (30 minutes)
 Bounce Rate: The percent of visits that result in only one page.
- Calculated as total of single page visits to a page/total visits to a page.
- Source: Web Analytics
 Time on Page and Time on Site: The net time from the time you entered
the first page of the site to the time you landed on the last page you
visited before exiting. [Note: Bounced Pages are measured as 0 minutes.]
Source: Web Analytics

12. 12
KPI - Engagement
For Google Analytics, the main engagement metrics can be found in
three different locations.
Audience Overview Content Overview
Page Overview

13. 13
KPI - Engagement
In addition, Google provides histograms of the aggregate level of
engagement.

14. 14
KPI - Engagement
In addition, Google provides histograms of the aggregate level of
engagement.

15. 15
KPI - Engagement
Yandex also provides these standard engagement metrics.

16. 16
KPI – Engagement
For a news site, a common goal for web site improvements is
increased engagement, which is reflected in these KPI’s.
Engagement Metrics - Tendency
 Bounce Rate: Decreasing over time.
- When a site is new, traffic is from search engines, bounce rates are high.
- As search engine traffic declines as a percent, bounce rate declines.
- Story page improvements can also create new page view opportunities.
+ Most commented story widgets
+ Most read story widgets
+ Polls, interactive graphics
 Time on Page and Time on Site: Increasing over time.
- As site becomes more familiar to audience, time on site should increase.
- Bounce rate declines as time on site increases (mostly).
- Story page improvements also increase time on site and time on page.
+ Video/Audio
+ Polls and other interactive features.

17. 17
KPI – Content Dynamics
In order to understand what kind of content is most important to the
success of the site, there are three standard content dynamic KPI’s.
Content Dynamic Metrics
 Most Visited Category: The category of content that receives the most
visits during a period. Level of detail dependent on URL structure.
- Refine to focus on only direct and social traffic.
- Source: Web Analytics
 Revenue Per Thousand Pages (RPM): Total direct revenue resulting from
a page’s advertising divided by the number of pages served in thousands>
- Refine by page type: Home Page, Category Front, Story Page…
- Source: Web Analytics (Page Views), Ad Serving/Ad Network (Revenue)
 Social Media Engagement: Measure may vary by the social media tracked.
Can refer to the number of “shares” a posting receives.
Ideally you want your audience’s most visited type of content to generate the
the highest RPM and be the most shared content on the site.

18. 18
KPI – Content Dynamics
Most Visited Category is the core data in the content section of most
web analytics services.
Google Analytics
Yandex Metrika

19. 19
KPI – Content Dynamics
Let’s try an example:
Depends on URL structure!
http://portalkbr.com/berita/olahraga/index.html

20. 20
KPI – RPM
Revenue per thousand pages (RPM) requires information from both
your ad system and your web analytics service.
PortalKBR.com Sports Example
 Revenue: What was total revenue to Sports Section of PortalKBR for
August 2013?
• AdSense Revenue = $10
• Sold Advertising = $50
• Affiliate Revenue = $5
TOTAL = $65
Sources:
- Finance/Ad Serving System
- AdSense Dashboard
- Affiliate Dashboard
$
$
$
ILLUSTATIVE

21. 21
KPI – RPM
After creating the revenue total for the period (and the site or
section of the site), then your web analytics service will provide you
with the total page views.
 PortalKBR.com Page Views (August 2013)
- Sports = 10,750 or 10.75 (thousands)
 RPM for Sports in August = $65/10.75 = $6.05
 Average RPM for PortalKBR = $2.25
 Sports Page Views = 3.5% of total
If you want to increase revenue, what actions would you suggest?
If you want to increase revenue for the sports section, what actions
would you suggest? What data do you want to support your actions?
ILLUSTATIVE

22. 22
KPI – Social Media Engagement
If your site plans for social media sites (vK/FaceBook) to contribute
to site traffic, then measuring what creates social media activity is
also important.
Staying with the PortalKBR Example:
FaceBook Insights

23. 23
KPI – Social Media Engagement
In the PortalKBR example, what can we learn from the most
“popular” posts on FaceBook?
 Does content type make a difference?
- The highlighted story is an opinion piece?
 Does the wording of the post make a difference in whether your audience
on FaceBook will engage with the post?
- The highlighted post is written as a question to engage the audience.
 Other things you can try to determine about how to post that will lead to
more click through to the site and more sharing and “followers”.
- Time of day/Day of week
- Type of content (Link, Photo, Audio, Video, …)
- Role of Promotion

24. 24
KPI – Cost Per Acquisition
Understanding the cost of acquisition for different segments of your
audience can help you grow your audience without wasting money.
 Case Study: CPA
Your site has invested in three marketing channels to bring audience to
your website – an email newsletter, online advertising, and social media.
For next year, you need to develop a budget for how to invest your
marketing spending to be the most efficient and generate the most
audience.
 CPA will help you answer this question.

25. 25
KPI – Cost Per Acquisition
To answer this question, you will have to gather information from
multiple systems – finance, social media, email server, …
Based on these results, where would you spend your online
marketing budget?
Email Advertising Social Media
Impressions 1,000 10,000 2000
Clicks 50 100 200
Click through Rate (CTR) 5.0% 1.0% 10.0%
Average Cost Per Click (CPC) $1 $1 $1
Cost $50 $100 $200
Conversion 10 10 10
CPA $5 $10 $20
Online Marketing Channel CPA

26. 26
Tactical Metrics
Tactical Metrics are very specific to a particular website issue like
improving navigation or advertising placement.
1. Foundation Metrics
2. Key
Performance
Indicators
3. Tactical
Metrics
Reported Regularly:
• Visits
• Unique Visitors
• Page Views
• Basic Demographics
Reported Regularly:
• Engagement Metrics
• Cost per Acquisition
Address a
Specific Problem
• A/B Testing
• Heat Maps
Website Metrics Structure

27. 27
Tactical Metrics
Tactical metrics are the most sophisticated measurements used in
developing a website.
What kind of questions might you try to answer?
 Optimize advertising placement and inventory.
Sales has presented a new “floating” ad unit which will make us $100
each month. If we use the new ad unit, will we actually make more
money or not?
 Optimize the elements of navigation.
We have discovered that we will make more money, if we can create more
page views in Sports. If we highlight Sports in the Navigation, will it get
more clicks?

28. 28
Tactical Metrics – A/B Testing
A/B Testing allows your audience to interact with two versions of a
website to measure how their behavior will change.
For A/B Testing to be successful, there are some pre-conditions:
 What measure are you optimizing the site for?
- Bounce Rate?
- Page Views
- RPM
- Ad Click Through Rate
 Have you developed two (or more versions) of the changes that you believe
will achieve the change?
 Have you estimated the cost of implementing the versions you are testing?
- For small changes, you can test in a live market environment.
- For example, the cost of changing the ad placement or number is mostly small.
- If you want to test a new ad plan, you might just try it and watch what happens to
total RPM for those pages.

29. 29
Tactical Metrics – A/B Testing
A/B Testing can be a costly and technically complicated tool, but
there are services that can help manage the effort.
Common Features:
• Real-time reporting tools and dashboards
• Integration with Google Analytics
• Customer segmentation (Regular Visitors vs New Visitors vs …)
• Simple version creation

30. 30
Tactical Metrics – Heat Maps
Heat Maps provide a visual model for where your audience clicks on
individual pages.
Yandex Metrika Example:
• Tells you where on the
page your audience is
clicking
- Red = High Clicks
- Green = Fewer Clicks
• Helps you optimize for:
- Ad Placement
- Navigation Design
- Copy Details
Other Services Include:
- Google Analytics
- CrazyEgg.com
- MouseFlow.com

31. 31
Summary
Metrics should be designed to help you manage your website both
the technology aspects of the site but also the business elements of
the site.
1. Foundation Metrics
2. Key
Performance
Indicators
3. Tactical
Metrics
Reported Regularly:
• Visits
• Unique Visitors
• Page Views
• Basic Demographics
Reported Regularly:
• Engagement Metrics
• Cost per Acquisition
Address a
Specific Problem
• A/B Testing
• Heat Maps
Website Metrics Structure