Slides from JUSP webinar on 10 November 2020

1. Reporting with COUNTER R4 and R5 data
10 November 2020
JUSP webinar

2. Outline
Reporting with COUNTER R4 and R5 data - 10 November 20202
What we will cover
• The transition to R5: what it means for reporting
• Data in JUSP
• Deciding what to report
• Comparing real R4 and R5
• Demo of useful reports, features and functionality in JUSP
• Impact of temporary access to content
• Your questions

3. The transition to R5: what it means for reporting
1. Report structure has changed
• Multiple metric types to deal with
• No totals in the standard views
• TR_J1 and TR_B1 exclude gold open access usage
• Lots more choice/flexibility = more decisions
2. Not all suppliers have yet switched to R5, some providing non-compliant reports, some switched off R4
3. “Ideal” R4 to R5 mappings don’t always work on the real data
Reporting with COUNTER R4 and R5 data - 10 November 20203
COUNTER R5 brings lots of benefits and improvements, but has caused
pain in the short term, especially for regular reporting

4. Confidence rating poll
How confident are you in understanding and working with the R5 usage data?
Scale of 1 to 10
1 = not at all confident
10 = extremely confident 1
0
4
4
4
6
5
3
1
0
0 1 2 3 4 5 6 7
1 = not at all confident
2
3
4
5
6
7
8
9
10 = extremely confident
Confidence Poll results

5. Data in JUSP
• Mix of R4/R5 data – historical and ongoing
• 'Supplier status' tab: updates, start and end dates
• Check Your data (R4) and Your data (R5)
• Data overlap
• Note that for R5 we use the label “Service (Platform)” (e.g. ScienceDirect). In R4
we used “Publisher” or “Supplier” (e.g. Elsevier).
• Restated data
• Data visualisations – R4 and R5
Reporting with COUNTER R4 and R5 data - 10 November 20205

6. R4 -> R5 transition update
• R4 datasets in JUSP: ~ 100
• R4 still being collected each month as of 31st October: 16
• R5 suppliers active in JUSP: 70 (12 DR, 58 PR, 67 TR)
• Missing R5 datasets - latest updates (Silverchair, EBSCOhost, Proquest x2, '2021' publishers)
• Handling of EBSCOhost and Proquest R5 reports
• Highwire – extract TR, no PR
Reporting with COUNTER R4 and R5 data - 10 November 20206

7. Deciding what to report

8. Taking a step back
Reporting with COUNTER R4 and R5 data - 10 November 20208
What to consider
• Who are you reporting to? What do they really want to know? Will the
data be re-used? How?Purpose
• When? How often? How long does it take?Time
• Previous years? Across suppliers? What’s your baseline?
SCONUL/benchmarking?Comparability
• Does it need to be comprehensive or indicative? Can you focus on a
sub-set?Coverage
• Reports, tables, charts, dashboards etc. What level of detail? What is
the indicator: total usage, CPU or % change? What are the caveats?Presentation

9. Choosing which R5 metric to report
Question Options Considerations
R4 or R5? • Stick with R4 as long as possible
• Switch to R5 as preferred
• Collect both where available
• R4 is being phased out
• Not always a choice
• Duplicate work, but opportunities for analysis
Exclude or include
OA_Gold?
• Exclude (TR_J1 and TR_B1)
• Include (TR_J3 and TR_B3)
• Report both separately (TR_J3 and
TR_B3)
• What did you do before? Why?
• What do you want to monitor? ROI, VfM,
engagement, uptake?
• R5 Standard Views support all but require
calculations
Which metric types? • Total or Unique
• Multiple metric types
• Can it be collected at the same time?
• How will the data be used?
• How will it be recorded?
How to collect and
collate?
• Standard Views
• Master reports
• JUSP reports
• SUSHI
• Who is collecting?
• Where from?
• What is supported?
Reporting with COUNTER R4 and R5 data - 10 November 20209
Decisions for COUNTER Release 5 reporting

10. Case study: Birmingham City University
• Background:
• Quarterly key indicator to university executive - % increase in e-resource usage.
• One headline figure lacked meaning, but lots of different metrics confusing and difficult to communicate internally. Initially
focus on journals as “downloads” can be widely understood.
• R4/5 review and report:
• Examined what usage data was available for 48 e-resources and presented summary - 2% (1) still COUNTER 3 (C3)
standard, 79% (38) on C4 and 92% (44) on C5
• Noted issues and challenges for reporting
• Recommendations
• Adopt C5 as the default reports from August 2020 onwards
• Contact suppliers/providers not yet providing C5 reports
• Use COUNTER 5 TR_J1 Journal Requests (Excluding OA_Gold) (Unique Item Requests) for journals
• For comparison with C4 usage data from the 2019/20 academic year, use JUSP TR: Transition (R4 - R5) reports or
COUNTER 5 TR_J1 Journal Requests (Excluding OA_Gold) (Total Item Requests)
Reporting with COUNTER R4 and R5 data - 10 November 202010
Report to inform decision to transition from COUNTER 4 to COUNTER 5
for default reporting

11. Poll: transition from R4 to R5
What is your library’s current position on reporting?
• Use R4 where available and R5 if not
• Use R5 where available and R4 if not
• Only use R5
• Only use R4
• Use both where available
4
20
0
0
3
0 5 10 15 20 25
Use R4 where available and R5 if not
Use R5 where available and R4 if not
Only use R5
Only use R4
Use both where available
Poll results

12. Poll: preferred R5 metric
1. Which count do you use for reporting journal usage?
2. Which count do you use for reporting book usage?
3. Which count do you use for reporting database usage?

13. Poll results
• Note that the report examples given in the journal and book questions options are incorrect.
Reporting with COUNTER R4 and R5 data - 10 November 202013
10
5
4
3
2 2
0
2
4
6
8
10
12
Total Item
Requests
including
OA Gold
(e.g. TR_J1)
Not
sure/don't
report this
Total Item
Requests
excluding
OA Gold
(e.g. TR_J3)
Unique Item
Requests
excluding
OA Gold
(e.g. TR_J3)
CombinationUnique Item
Requests
including
OA Gold
(e.g. TR_J1)
Poll results: journal metric
8
7
4
3
2 2
0
1
2
3
4
5
6
7
8
9
Total Item
Requests
including OA
Gold (e.g.
TR_B1)
Not
sure/don't
report this
Combination Unique Title
Requests
excluding
OA Gold
(e.g. TR_B1)
Total Item
Requests
excluding
OA Gold
(e.g. TR_B3)
Unique Title
Requests
including OA
Gold (e.g.
TR_B1)
Poll results: book metric

14. Poll results continued
Reporting with COUNTER R4 and R5 data - 10 November 202014
8
7
6
2
1 1 1
0
1
2
3
4
5
6
7
8
9
Total Item
Requests
Combination Not sure/don't
report this
Unique Item
Investigations
Total Item
Investigations
Searches
Regular
Unique Item
Requests
Poll results: database metrics

15. Comparing real R4 and R5 data

16. When it all works
• Slight variations expected e.g. double-click count window has changed, robot exclusions different etc.
• JUSP TR: Transition Report set filters for you
• Almost all suppliers have some overlapping data in JUSP – 24 have complete 2019 overlap - still work to be
done on a systematic comparison and analysis
• COUNTER have provided guidance in the Code of Practice: 13.3 Transitioning from COUNTER R4 to R5
Reporting with COUNTER R4 and R5 data - 10 November 202016
Closest equivalent metrics
R4 R5 closest match
JR1 full-text article requests • TR_J3 Total Item Requests with Controlled and OA_Gold added
together
BR2 full-text section requests • Master report: Total_Item_Requests AND Data_Type=Book AND
Section_Type=Chapter|Section
BR1 full-text book requests • Master report: Unique_Title_Requests AND Data_Type=Book AND
Section_Type=Book
DB1 result clicks / record views • DR_D1 Total_Item_Investigations

17. Known exceptions
• For most suppliers, Total_Item_Requests correspond well with the R4 section requests, but not all
• Known cases showing drops: Springer, DawsonEra, Askews & Holts, ProQuest Ebook Central
• Differences due to clarifications in the way sections are counted
• Some suppliers (e.g. Springer-Nature) provide additional custom metrics in their master reports to help
provide a point of comparison
• How do you handle these differences?
• Compare monthly trends or percentage change
• Take a new baseline
• Overlap R4 and R5 reporting
• Other suggestions?
Reporting with COUNTER R4 and R5 data - 10 November 202017
Some supplier show huge drops in e-book usage counts

18. JUSP demo
Useful reports, features and functionality

19. Temporary free content
• Still counted as “Controlled” but usage not captured in institutional reports, so counts are much lower than normal for
March-July 2020
• COUNTER message to libraries: https://www.projectcounter.org/message-to-libraries-about-counter-usage-during-
the-covid-19-pandemic/
Reporting with COUNTER R4 and R5 data - 10 November 202019
Case 1: publisher has made content freely available to all
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Total_Item_Request (Controlled + OA Gold) for journals
single institution
2019 2020

20. Temporary access content
Reporting with COUNTER R4 and R5 data - 10 November 202020
Case 2: publisher has expanded access for existing customers
• Still counted as “Controlled” and captured in institutional reports, so usage (hopefully) goes up
0
200000
400000
600000
800000
1000000
1200000
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Unique_Title_Requests (Controlled) for all UK HE
2019 2020

21. Further resources
• Evaluating and interpreting usage of temporary content
• View ‘What happens when content isn’t free anymore?’ at
https://www.jisc.ac.uk/events/managing-your-print-and-online-collections-in-a-covid-
19-world-15-jul-2020
• List of resources for Coronavirus crisis see
https://subscriptionsmanager.jisc.ac.uk/about/resources-for-coronavirus-crisis
• JUSP
• Events and training: https://jusp.jisc.ac.uk/events-training/
• JUSP guides: https://jusp.jisc.ac.uk/guides/
• COUNTER guides: https://www.projectcounter.org/friendly-guides-release-5/
Reporting with COUNTER R4 and R5 data - 10 November 202021

22. Questions

23. Feedback survey
https://jisc.researchfeedback.net/s.asp?k=160129985823&u=fzjq