With ever-shrinking library budgets it is more essential than ever to ensure that the library collection is targeted, relevant and well-used. Return on Investment (ROI) has become the mantra of library management and libraries need to show accountability for collection decisions. This webinar will focus on speakers who have successfully implemented assessment metrics (such as COUNTER 3, Eigenfactor and impact factors) as one determining factor of collection development decisions.

1. NISO Webinar: Assessment Metrics
December 14, 2011
Oliver Pesch
Chief Strategist, E-Resources
EBSCO Information Services

2. • What’s new with Release 4 of the COUNTER Code of
Practice
• How the COUNTER and SUSHI XML Schemas are
affected
• Facilitating implementation with the COUNTER-SUSHI
Implementation Profile
• Ensuring Compliance through a more robust audit
• Timing

3. What’s New

4. One Code of Practice
• Covers books, databases, journals and multimedia
collections
• Easier to manage for COUNTER
• Easier to manage for content providers –
particularly those offering multiple resource types.

5. Multimedia Report 1
• Reports on use of items in multimedia collections
• Applies to platforms like ArtStor
Consortium Report 3
• The multimedia report for consortia
Journal Report 5
• Usage of journal articles broken out by year of
publication
• No longer optional

6. DOI and Proprietary Identifier
• Improves identification of items listed in the reports
• Improves matching usage to the correct items in a
knowledge base
• Involves coordination with KBART so the same
identifier is on both the vendor’s Title List and their
COUNTER report
• Applies to most COUNTER reports

7. Institutional Identifier
• Utilizes the NISO I2 as the standard identifier for the
institution (being implemented as ISNI for
organizations)
• Improves comparative analysis– particularly for
usage processed by consortium
• Applies to all reports

9. Gold Open Access
• Covers articles accessible without subscription
• Included as a separate count in Journal Report
1and Journal Report 1a

10. Result Clicks and Record Views
• Result Click counts each time a user clicked a link
from the search result to the detailed view, a link
resolver, etc.
• Record View counts the abstract/detailed records
explicitly viewed from the database
• Both indicate an explicit expression of interest in the
result
• Provides a more direct measure of database usage,
in particular for A&I databases
• Applies to Database Reports 1 and 3

11. Access Denied: content item not licensed
• In addition to Turnaway s (simultaneous user limit)
• Records a user’s attempt to access full text for which
their institution does not have a subscription
• Applies to “Access Denied” reports:
o Journal Report 2
o Database Report 2
o Book Reports 3 and 4
• Useful input for collection development

12. Sessions
• Federated Search and Discovery products have
made “Sessions” an unreliable measure of use
• Reports affected:
o Database Report 1
o Database Report 3
o Consortium Report 2

13. Changes resulting from the new Code of Practice

14. COUNTER Schema
• Minor changes only
• ItemIdentifier, ItemPublisher now optional elements
• PubYr attribute ( used by Journal Report 5) now
defined as a year.
• PubYrFrom and PubYrTo attributes added to allow
Journal Report 5 to report ranges of years.

15. COUNTER Enumeration Schema
• Contains the controlled vocabulary for elements like
ItemDataType, Category, MetricType, etc.
• Added new values as required by the new and
updated reports

16. SUSHI Schema
• No changes required

17. Facilitating adoption of SUSHI

18. Why
• SUSHI and COUNTER tend to be abstract
• Abstraction leaves the opportunity for concepts to
be interpreted differently
• Different interpretations leads to inconsistencies
• Examples:
o Different authentication methods that require custom development
o Wrong capitalization on controlled data values
o Including “Totals” in the XML
o Including invalid ISSNs (e.g. “N/A”)

19. Authentication for SUSHI Client
• IP and use of requester and customer IDs allowed
• Use of SOAP extensions not allowed

20. Report Names
• Must exactly match the value in the “Name”
column on the SUSHI Reports Registry
Date Ranges for SUSHI Request
• Format must by yyyy-mm-dd
• Begin date must be first of the month
• End date must be last day of the month
• End date must be later than begin date

21. SUSHI Exceptions
• Standard error numbers must be used for standard
error conditions (see Table 17 in the SUSHI standard)
SUSHI Server Registry
• SUSHI Server must be registered on SUSHI Server
Registry

22. Data to Return in the COUNTER Report
• Only the months of usage asked for
• Include exception if some or all usage not available
• Only include items appropriate to the report (e.g.
don’t return book usage in a Journal Report)
• Do not include totals for the report or an item

23. Data Elements Values
• Use registered values with proper casing for
enumerated values (Item Data Type, Category,
Metric Type, etc.)
• Use values appropriate for the report

24. Designed for use by implementers
• Prescriptive in nature
• Lots of tables and charts

25. Designed for use by implementers
• Prescriptive in nature
• Lots of tables and charts

26. Monitoring compliance

27. • SUSHI and COUNTER XML will receive more focus in
the audit
• COUNTER-SUSHI Implementation Profile should server
as a guide for auditors
• Auditors will be expected to use automation to
reduce the cost of the audit

28. Trial use, comments and final implementation

29. January 20, 2012
• Last day for comments on the draft of release 4 of
COUNTER Code of Practice and the COUNTER-SUSHI
Implementation Profile
March/April 2012
• Final publication of Release 4 of COUNTER CoP and
the COUNTER-SUSHI Implementation Profile
December 31, 2013
• Deadline for compliance with Release 4 of the
COUNTER CoP

30. COUNTER CoP R4 http://www.projectcounter.org/code_practice.html
COUNTER-SUSHI http://www.niso.org/apps/group_public/document.php?
Implementation Profile document_id=7639&wg_abbrev=sushimaintenance
COUNTER Schema Date http://www.niso.org/workrooms/sushi/values_R4
Element Values
KBART http://www.uksg.org/kbart
NISO SUSHI website http://www.niso.org/workrooms/sushi
SUSHI Reports Registry http://www.niso.org/workrooms/sushi/reports_R4/
SUSHI Server Registry https://sites.google.com/site/sushiserverregistry/
Thank you