Why no one knows how many people are visiting their websites.

1. Counting on confusion
Why no one knows how many
people are visiting their websites

2. What if everyone is wrong?

3. The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing

4. The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing
• Internal counts are
three times higher
than Nielsen ratings

5. The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing
• Internal counts are
three times higher
than Nielsen ratings
• Whose count is
accurate? Agencies’ or
content-providers’?

6. Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting

7. Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting
• Afternoon traffic is
undercounted, local
sites are a challenge

8. Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting
• Afternoon traffic is
undercounted, local
sites are a challenge
• Nielsen once gave
washingtonpost.com
10m and comScore 17m

9. comScore is now the standard
• About 2 million users
have tracking code on
their computers

10. comScore is now the standard
• About 2 million users
have tracking code on
their computers
• comScore uses polling
techniques to weight
and adjust its sample

11. comScore is now the standard
• About 2 million users
have tracking code on
their computers
• comScore uses polling
techniques to weight
and adjust its sample
• Some users may have
had code installed
without permission

12. Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?

13. Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?
• Visitors are counted by
device. Less accurate
if they clear cookies.

14. Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?
• Visitors are counted by
device. Less accurate
if they clear cookies.
• No way of separating
out robotic hits
from search engines

15. What is a unique visitor?
• Most agree it is
someone who visited
once during time span

16. What is a unique visitor?
• Most agree it is
someone who visited
once during time span
• Unique visitors per
month is most common
Internet traffic measure

17. What is a unique visitor?
• Most agree it is
someone who visited
once during time span
• Unique visitors per
month is most common
Internet traffic measure
• Other measures: return
visits, number of page
views, time spent

18. Dearth of free data
• SimilarWeb works like
comScore except its
data are more open

19. Dearth of free data
• SimilarWeb works like
comScore except its
data are more open
• Quantcast gets good
reviews from some, but
code must be installed

20. Dearth of free data
• SimilarWeb works like
comScore except its
data are more open
• Quantcast gets good
reviews from some, but
code must be installed
• Alexa, now owned by
Amazon, offers an
alternative approach

21. Was newspaper data better?
• No one knows how
many people actually
open up the paper

22. Was newspaper data better?
• No one knows how
many people actually
open up the paper
• Multipliers of “readers”
versus “subscribers”
is highly suspect

23. Was newspaper data better?
• No one knows how
many people actually
open up the paper
• Multipliers of “readers”
versus “subscribers”
is highly suspect
• Chicanery to make
numbers look higher
is not unusual

24. Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent

25. Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent
• The Washington Post
has adopted A/B testing
and “lead measures”

26. Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent
• The Washington Post
has adopted A/B testing
and “lead measures”
• In the age of distributed
journalism, the metrics
challenge has grown

27. Credits
• Much of this presentation is based on
“Confusion Online: Faulty Metrics and the
Future of Digital Journalism,” by Lucas Graves,
John Kelly and Marissa Cluck, Tow Center for
Digital Journalism, Columbia University
Graduate School of Journalism, September
2010.
• Updated on December 6, 2016.