1.
Making sense of analytics for
documentation pages
Kumar Dhanagopal
API The Docs Virtual 2023 - Feedback, Metrics, Analytics

2.
TOPICS
What analytics is and isn’t
Overview of key metrics
The analytics process
Interpreting metrics
Summary
2
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

3.
How analytics can (and can’t) help…
✓ Understand how users engage with our content
• Who and from where
• How often
• On what devices
• …
✓ Understand user behavior on our site
• How they navigate
• What they click
• Where they spend time
• …
• Analytics can’t help us learn about user satisfaction and sentiment
• What do users need?
• Did they find it?
• Were they satisfied?
• How (and how much) did they read?
• Need other sources (e.g., user ratings, comments)
• …
3
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

4.
Data != reality (correlation doesn’t imply causation)
4
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

5.
The key metrics
5
A B C
D E
F G H
Page Views Visits Entries Exits Bounces
A 1 1 0 0 0
B 2 2 0 0 0
C 1 1 1 0 0
D 3 2 0 0 0
E 1 1 1 1 1
F 2 1 1 0 0
G 2 2 0 1 0
H 1 1 0 1 0
Session-1
Session-2
Session-3
Note:
❖ Repeat visits and long visits are treated differently
❖ Cookie’s matter
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

6.
The analytics process
1. Define the problem or goal
2. Get the required data
• Metrics
• Dimensions
• Period
3. Prepare the data
• Remove noise
• Fix inconsistencies
• Aggregate data
4. Analyze, explore, visualize
5. Describe, diagnose, prescribe, predict
6
"If you torture the data long enough, it will confess – to anything" – Ronald H. Coase
Minimize the garbage-in-garbage-out risk.
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

7.
Interpreting metrics: General guidelines
• Validate with other data
• Avoid cross-product comparisons
• Look at trends, not just absolute numbers
• Remember: offline and second-hand are not tracked!
• Consider the content type, length, format, and structure
• Use analytics data as supplementary input for decision making
7
Page Views
A 15k
B 14k
Jan Feb Mar Apr May Jun
3358 2610 3025 2411 2316 1900
0 0 1583 3308 5015 5029
Page A 25k 25k
Page B 25K 25k
Page C -- 25k
Page D -- 25k
Chunking affects analytics!
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics
Doc 1 Doc 2
50k views 100k

8.
Interpreting low page views
• Popularity != page views
• Look at trends, not just absolute numbers
• Consider the “age” of the page
• Check whether the page is discoverable
• Keep in mind structural changes
• Don’t ignore zero-view pages
8
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

9.
Interpreting bounce rate
• Possibly the most misunderstood metric
• High bounce rate is not necessarily a problem
• Text-book guidance might not apply to
documentation websites
• Session time-out = bounce
• Analyze in conjunction with other metrics, like time
spent on page
9
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

10.
Interpreting time spent on page
• High time spent:
• Was the content useful/interesting?
• Difficult to understand?
• User left the browser open too long? ☺
• High time spent + high page views
• Too much content, or too complex?
• Opportunity to improve chunking
• High time spent + high bounce rate
• Landing pages: cause for concern
• Other pages: Opportunity to simplify content
• Low time spent + high bounce rates
• Reference pages: positive indicator?
10
Time spent is NOT calculated for exit pages!
Number
of users
Total time
spent (min)
Avg. time
spent (min)
Action after
reading…
Group A 50 500 10.0 Exit site
Group B 50 250 5.0 Another page in
the site
Total 100 750 7.5
2.5
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

11.
Analytics “targets”
Should we have analytics KPIs?
Documentation sites != e-commerce portals
Consider data availability and decision-making culture
11
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

12.
SUMMARY
✓ Data != reality
✓ Analysis requires non-trivial effort
✓ Metrics won’t give us all the answers
✓ Focus on users, business goals
✓ The forest AND the trees matter!
12
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

13.
SUGGESTED READING
Web Analytics - An Hour A Day, Avinash Kaushik
Practical Text Analytics, Steven Struhl
An Introduction to Data Science, Jeffrey S. Saltz, Jeffrey Stanton
13
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics

14.
THANK YOU!
https://www.linkedin.com/in/kumardhanagopal
14
Kumar Dhanagopal, API The Docs Virtual 2023 - Feedback, Metrics, Analytics