NISO IOTA presentation at the 2010 Charleston Conference.

1. NISO’s IOTA Working Group
Creating an Index for Measuring the Quality of
OpenURL Links
Charleston Conference - Nov. 5, 2010
Rafal Kasprowski, Rice U.
Susan Marcin, Columbia U.

2. Agenda
• Background: Full-text linking and Advent of OpenURL
• Problems with OpenURL linking
• IOTA: Created in response to OpenURL linking problems
• Overview of IOTA's work
• IOTA project site
• Community-derived data reports
• Publicizing IOTA: wiki, blog, Twitter
• Get involved and Take advantage of current effort

3. Before OpenURL: Proprietary Linking
• A&I database providers offered option for full-text linking
(e.g., CSA, PubMed, etc.) .
• Libraries manually activated linking to full-text providers
they had subscriptions with.
• A&I --> Full Text

4. Proprietary Linking: Cons and Pro
• Linking had to be activated manually by libraries for
each full-text provider.
• A&I providers offering this option were few.
• Selection of full-text providers was limited.
But...
• Once set up, the static links to full texts were
accurate.

5. Advent of OpenURL
• Objective: Deliver full texts unrestrained by proprietary silos.
• Open standard generating link at time of request.
• Library's holdings indicate provider of "appropriate copy".
• A-Z list (e.g., e-journal, e-books):
o Knowledge base (KB) with library's holdings.
o Intermediary in linking.
• A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")

6. OpenURL Syntax and Resolver
A, Bernand, et al. "A versatile nanotechnology to connect individual nano-objects
for the fabrication of hybrid single-electron devices." Nanotechnology 21, no. 44
(November 5, 2010): 445201. Academic Search Complete, EBSCOhost (accessed
October 24, 2010).
http://ps4ps6lm2r.search.serialssolutions.com/?issn=0957-
4484&volume=21&issue=44&date=20101105&spage=445201&title=Nanotechnology
&atitle=A+ versatile+nanotechnology+to+connect+individual+nano-
objects+for+the+ fabrication+of+hybrid+single-electron+devices.&aulast=A+
+Bernand
Source Citation
Target OpenURL (Source OpenURL structured similarly)

7. Pros & Cons of OpenURL
Pros:
• KB/Resolver vendors took over most of the linking setup:
Less work for libraries and providers.
• Participation by A&I platforms and full-text providers
exceeded proprietary linking: OpenURL scales better
Cons:
• Dynamic linking less predictable than static linking: more
difficult to pinpoint cause of link failures
• OpenURL linking not improved significantly last 10
years.
• No systematic method exists to benchmark OpenURLs.

8. Problem Statement & Methodology
"72% of respondents to the online survey either agreed or
strongly agreed that a significant problem for link resolvers is
the generation of incomplete or inaccurate OpenURLs by
databases (for example, A&I products)."
Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG,
p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf.
Recently, researchers have indicated the need for metadata
quality metrics, including:
• completeness;
• accuracy;
• conformance to expectations;
• logical consistency and coherence.
Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining,
Expressing, Exploiting. In Metadata in Practice. Ed. Diane I. Hillmann and Elaine L. Westbrooks.
Chicago: American Library Association, pp. 238-256.

9. Année philologique OpenURL Study
2008 Cornell study led by Adam Chandler*
• Problem: Too often links sent from Aph did not
successfully resolve to requested resource.
• Objective: Examine quality of OpenURLs offered to
users by Aph in order to improve the linking.
Aph Study investigated:
• Faulty citation metadata from source database.
• Method to evaluate the OpenURLs.
*Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation:
Mellon Planning Grant Final Report.
http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.

10. Scoring System & Aph Study Outcomes
Concept of scoring in Aph study (based on B. Hughes study)*
• establish a baseline for comparison;
• results to be shared with data providers;
• develop a best practice.
Problem analysis in Aph study limited to:
• source link
• presence/absence of citation metadata elements
Results:
• OpenURL quality model: compares elements in Aph
OpenURLs to those of other providers.
• No scores, but model is first step towards scoring system.
*Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives
Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng
Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.

11. Creation of IOTA
NISO accepts proposal to take Aph Study to wider community
• Improving OpenURLs Through Analytics (IOTA):
o Formed in January 2010.
Basic Assumptions:
• Results are achieved through an analytical investigation of
how OpenURL links work.
• Practical: Not OpenURL standard is addressed, but links
(OpenURLs) generated by standard.
• Selective changes to OpenURLs will lead to significant
improvement in linking success rate.
o Motto: "small changes. big improvements"

12. Desired Outcomes
• Develop community-recognized index for measuring the
quality of OpenURL links generated by content providers.
• Produce qualitative reports that will help OpenURL
providers quickly compare their OpenURL quality to that of
their peers.
• Method:
o fair;
o transparent;
o scalable across all OpenURLs and their providers.

13. Initial Focus
1. Core elements:
• Contained in IOTA's OpenURL reporting system;
• 9 million OpenURLs from libraries and providers.
2. Scoring system based on assumption:
• Correlation exists between
o # of core elements ("OpenURL completeness") &
o ability of OpenURLs to link to specific content.
3. Weighting assigned to core elements:
• Based on relative importance
o spage vs atitle
o issn vs jtitle
o doi/pmid vs date, etc.

14. Work in Progress
• Element weighting still in progress:
o E.g., importance of identifiers (doi, pmid) vs
bibliographic data (issn, volume, spage).
• Currently, IOTA focuses on OpenURLs from citation
sources only. OpenURL quality is also influenced by:
o knowledge base,
o resolver,
o full-text provider (target).
• High "completeness" score of OpenURLs not always
indicative of "success" in linking to full texts
o Combination of multiple indexes along linking nodes
may provide more complete picture.

15. Issues with openURL linking
• Why might "article" links not appear or work?
o target vendor does not support openURL linking
o problems with information was passed from source into
openURL string. ex:) not enough info, incorrect info

16. IOTA Project Site

17. OpenURL log file reports
Over 9 million openURL strings in system

18. Report Types

19. Report Types

20. Source Report - vendor or database

21. Source Report : Elements (vendor)

22. Source Report : Patterns (vendor)

23. Element & Pattern Frequency Report

24. Element & Pattern Frequency Report :
Choose Metric

25. Element & Pattern Frequency Report :
Metric = Issue

26. Click from Element & Pattern
Frequency Report to Source Report

27. Documentation Wiki
http://openurlquality.niso.org/

28. Glossary
Provides working definitions
of terms such as:
• Elements
• Link-to / Target
• Patterns
• SID
• Vendor

29. IOTA Blog & Twitter
http://openurlquality.blogspot.com/
Find us on Twitter
Hash Tag #nisoiota

30. How can I get involved in IOTA?
If you are a vendor --
• Look at IOTA data ... or...
• Point your tech folks to our data
• This data is meant to help make improvements in your
openURL linking.
If you are a librarian --
• Contribute data to IOTA
• Help spread the word to vendors
• More information on the IOTA project at:
http://openurlquality.niso.org/

31. Questions?
Group email: openurlquality@list.niso.org
More info... http://openurlquality.niso.org/