Web-scale Discovery Services are becoming an integral part of libraries' information gathering arsenal. These services are able to use a single interface to seamlessly integrate results from a wide range of online sources, emulating the experience patrons have come to expect from Internet search engines. But despite their ability to streamline searching, discovery services provide a wide set of challenges for libraries who implement them. This virtual conference will touch on both the potential of discovery services as well as some of the issues involved.

1. http://www.niso.org/news/events/2013/virtual/discovery
NISO Virtual Conference:
Web-Scale Discovery Services:
Transforming Access to
Library Resources
November 20, 2013
Speakers:
Lorcan Dempsey, Amy Hoseth,
Cody Hanson, Michael Kucsak,
Athena Hoeppner, Matthew Reidsma,
Marshall Breeding

2. NISO Virtual Conference:
Web-Scale Discovery Services:
Transforming Access to Library Resources
Agenda
11:00 a.m. – 11:10 a.m. – Introduction
Todd Carpenter, Executive Director, NISO
11:10 a.m. – 12:00 p.m. – Selecting a Web-scale Discovery Service: Evaluating the Options
Amy Hoseth, Coordinator for Onsite Services, Colorado State University Libraries
**12:00 p.m. – 12:45 p.m. – Keynote: Library Discovery: Past, Present and Some Futures
Lorcan Dempsey, Vice President, OCLC Research and Chief Strategist
** Due to technical difficulties, the Keynote appears second in the order of panelists.
12:45 p.m. – 1:30 p.m. – Lunch Break
1:30 p.m. – 2:00 p.m. – Why Web-scale Discovery Means the End of Build vs. Buy
Cody Hanson, Acting Director, Web Development, University of Minnesota University Libraries
2:00 p.m. – 2:30 p.m. – Implementation: Delivering the Goods
Michael Kucsak, Director of Library Systems and Technology, University of North Florida
2:30 p.m. – 3:00 p.m. – Zen and the Art of Discovery Maintenance
Athena Hoeppner, Electronic Resources Librarian, University of Central Florida Libraries
3:00 p.m. – 3:15 p.m. – Afternoon Break
3:15 p.m. – 3:45 p.m. – The Library with a Thousand Databases: Web Scale Discovery and The Hero's Journey
Matthew Reidsma, Web Services Librarian, Grand Valley State University Libraries
3:45 p.m. – 4:15 p.m. – Update on the NISO Open Discovery Initiative
Marshall Breeding, Independent Consultant and co-chair, NISO Open Discovery Initiative
4:15 p.m. – 5:00 p.m. – Conference Roundtable
Presenters return for a Q&A discussion lead by Todd Carpenter, Executive Director, NISO

3. Selecting a Web-scale Discovery
Service: Evaluating the Options
Amy Hoseth
Colorado State University Libraries
November 20, 2013

4. About Colorado State University
•
•
•
•
Land-grant institution located in Fort Collins, Colorado
Approximately 26,000 students, including 22,000
undergraduate and 4,000 graduate students
The CSU Libraries includes Morgan Library on the main
campus, and a small branch library on the veterinary campus
Carnegie Research University; ARL library

5. Web-scale research at CSUL
•
•
Summer 2009: Library / IT Task Force makes recommendations
Fall 2009: Library dean convenes Discovery Tools Committee
•
•
Six members represent key library departments: Tech services, metadata,
collections and contracts, college liaisons, and access services
Committee charge: Explore Web-scale discovery tools currently
available, and recommend plan and budget for moving forward

6. Review process
•
Four products were evaluated:
• EBSCO Discovery Service (EDS)
• Primo Central (Ex Libris)
• Summon (Serials Solutions)
• WorldCat Local

7. Research Process
•
•
•
•
RFIs issued to vendors
Vendor presentations at CSU
Other research: conferences, site visits, phone calls
Six month process, start to finish

8. Comparing products
•
Identifying key criteria for evaluation was first challenge
•
•
•
•
Useful, synonymous information needed for each product
Committee developed a “comparison matrix” to gather, report, and
compare information on each product
Matrix included 40+ points of comparison
Ultimately, the matrix could be distilled into five key criteria

9. Five key criteria
1.
2.
3.
4.
5.
Overall goal/purpose for acquiring a discovery tool
Cost
Coverage/content
Usability
Technology issues

10. 1. Goal/Purpose
•
•
•
•
What will the Web-scale tool allow users to accomplish?
What is the purpose of this tool for your library?
How will it work with existing resources?
Who are your users, and what kind of tool do they need?

11. 2. Cost
•
•
•
•
What are the costs associated with each product?
How will the implementation of this tool affect other
subscriptions?
Is consortium purchase/pricing an option?
Costs for local staffing and support

12. 3. Coverage/content
•
•
•
What existing resources are covered by the new tool?
How does it manage handoffs to native databases and
resources?
What resources are not indexed or included, and how will
users access them?

13. 4. Usability
•
•
•
Need to conduct both formal and informal testing
“Test drive” implementations as much as possible
Key elements to consider include:
•
•
•
•
•
User interface
Faceting
Relevancy ranking
Search options
Customization options

14. 5. Technology issues
•
•
•
•
How much local support is required for the Web-scale tool
to run smoothly?
How well does it integrate with existing library applications?
Is the API customizable? Is it robust?
Is the Web-scale discovery tool mobile-friendly? Does it
effectively serve users with disabilities?

15. Other practical recommendations
•
•
•
Involve individuals from across the library in your overall
analysis
Conduct a thorough, methodical review of each product
Contact institutions that are already using these products
for their advice and feedback

16. Outcome at CSU
•
•
Final report (June 2010) identified one web-scale tool as
the product most likely to meet our needs
Ultimately, no purchase was made
•
•
•
•
Concerns about cost
Questions about integration of tool with existing resources
Products still new, maturing
CSUL is now convening another committee to revisit the
possibility of purchasing a Web-scale discovery tool

17. Questions?
Amy Hoseth
Coordinator for Onsite Services
Colorado State University Libraries
970-491-4326
amy.hoseth@colostate.edu

18. Library
discovery:
past, present and some futures
@LorcanD
Lorcan Dempsey
20 November 2013
NISO Virtual Conference: Web-Scale Discovery
Services: Transforming Access to Library
Resources

19. Scope

20. Pretty general
20

21. Past:
Present:
Future:
Some drivers
Some responses
Some directions
21

22. Drivers 1
The user environment
The example of visitors and residents
Lynn Silipigni Connaway, David White, Donna Lanclos, and Erin Hood. 2013. Meeting the Needs of Digital Visitors
and Residents: Developing Engagement with Institutional Services Educause Annual Conference, 15-18 October
2013, Anaheim, California (USA).http://www.oclc.org/content/dam/research/activities/vandr/presentations/meetingthe-needs-of-digital-visitors-educause-2013.pptx
Lynn Silipigni Connaway, Donna Lanclos, and Erin Hood. 2013. "I Find Google a Lot Easier Than Going To the
Library Website."
Imagine Ways to Innovate and Inspire Students to Use the Academic Library. ACRL 2013: Imagine, Innovate, Inspire,
10-13 April 2013, Indianapolis, Indiana
(USA).http://www.oclc.org/content/dam/research/activities/vandr/presentations/acrl-vandr2013.pptx
22

23. “It’s convenience. It’s the immediacy of it.”
(UKF3, Experiencing, Male, Age 52, Artist & Technical Support)
Convenient Doesn’t
Always Mean Simple
Image: http://www.flickr.com/photos/dubpics/8685963533/

24. “...Google doesn’t
judge you.”
(UKF3, Male, Age
52)
Image: http://www.flickr.com/photos/cubmundo/6184306158/

25. “It’s like a taboo I guess with all teachers, they
just“I just type itknow,Google they see what
all say – you into when and explain the
paper comes up.” (UKS2)
they always say, “Don’t use Wikipedia.”
(USU7, Female, Age 19)
Learning Black Market
Image: http://wp.me/pLtlj-fH

26. Then: Resources were scarce and
time was abundant
Now: Resources are abundant and
time is scarce
Convenience is an important value.
26

27. Drivers 2:
The service environment
“How can building a website be so difficult?”
27

28. Until recently …..
… library websites were providing a
very thin layer of integration over
two sets of heterogeneous
resources, ….
…which map more to legacy
technical and business issues than
to user behaviours.
28

29. 1. Systems
2. Databases
Catalog/ILS
A-Z lists
Metasearch
Legacy database boundaries
map more to historically
evolved publisher
configurations and business
decisions than to user needs or
behaviors.
Resolver
Repository
29

30. Full text publishers
A&I publishers
Aggregators
Discovery layer
providers
30

31. Driver 3:
The network environment
Stuck in the middle
31

32. Webscale: operating at the scale of the web. We
have seen many service providers emerge in recent
years which operate at webscale. Facebook,
Amazon, Expedia, Etsy.
They concentrate capacity in platforms whose
benefits can be broadly shared. The platform
supports the aggregation of data and
infrastructure at scale.
Additionally, many of these services build strong
communities - networks of participants who
communicate, share, or trade on the platform.

33. Webscale and personal go
together.

36. Webscale and personal go together.
The institution is the squeezed
middle?

37. Researchers prefer to
adopt open source and
social media technologies
that are available in the
public domain rather than
institutional license-based
applications ….. First the
social media technologies
facilitate networking and
community building.
Second, researchers prefer
to use technologies that
will enable them access to
resources and their own
materials beyond their
institution-based PhD
research.
e.g. Mendeley, Zotero, Endnote

38. Then: users built their workflow
around the library.
Now: the library must build its
services around its users workflow.
38

39. Users value convenience.
Library destinations fragmented
and demand too much work
(cognitive and mechanical).
The institution is being squeezed
between the webscale and the
personal. Have to get into
workflows.
39

40. Discovery layer
Users value convenience.
Library destinations fragmented and
demand too much work (cognitive and
mechanical).
The institution is being squeezed
between the webscale and the personal.
Have to get into workflows.
The challenge of discoverability
40

41. Some responses
41

42. Some responses
1. Systems integration – unified discovery and
unified backoffice workflows
2. Website integration – an integrated
experience
3. Make discovery more like web search
4. Discoverability – a decentered network
presence
42

43. 1. Systems integration – unified discovery
and unified backoffice workflows
2. Website integration – an integrated
experience
3. Make discovery more like web search
4. Discoverability – a decentered network
presence
43

44. Worldshare MS
ALMA
Sierra
Intota
Kuali Ole

45. Worldcat Local
Worldshare MS
Primo
ALMA
EDS
Sierra
Summon
Intota
Kuali Ole

46. (Partial) systems integration
Cloud-based.
Workflow integration across purchased,
licensed, digital?
Central index.
Best-of-breed vs Trend to single supplier
ecosystems?

47. US Academic Libraries
N = 881
Based on data from Marshall Breeding’s Lib-Web-Cats technology profiles, August 2013.
Created by Constance Malpas, OCLC Research, using the Sankey template from Bruce McPherson.

48. 1. Systems integration – unified discovery
and unified backoffice workflows
2. Website integration – an integrated
experience
3. Make discovery more like web search
4. Discoverability – a decentered network
presence
48

49. Some examples ..
• Content management
systems
• Resource guides
• Locally controlled search
container
– VuFind
– Blacklight
• Discovery systems
– A unified view
– Move work from user
to system
49

50. 1. Systems integration – unified discovery and
unified backoffice workflows
2. Website integration – an integrated
experience
3. Make discovery more like web search
4. Discoverability – a decentered network
presence
50

51. “Simple” search box to find everything

52. “Simple” search box
Refine results
52

53. “Simple” search box
Refine results
A focus on fulfilment …
53

54. 1. Systems integration – unified discovery and
unified backoffice workflows
2. Website integration – an integrated
experience
3. Make discovery more like web search
4. Discoverability – a decentered network
presence
54

55. Network Presence
John Doe University Library
Website
Decoupled
Communication
John Doe
University
Library
External
Syndication
Cloud Sourced

56. Flickr
Decoupled
Communication
Blogs
Facebook
Twitter
Google
Youtube
Discovery
Knowledgebase
Libguides
Cloud Sourced
Resolver

57. WorldCat
ArchivesGrid
Summon
Metadata
Scirus
Blogs
Suncat
Catalogue
RSS
Mobilepp
Ethos
Proxy Toolbar
OAI-PMH
(Dspace)
Linked Data
(Catalog)
Dspace
Discovery
Services
Proxy
Widgets
Library APIs
Z39.50
External
Syndication
Data
Jorum
Digital
Archive
Europeana

58. Creating conversations
around collections:
Sleevefacing at
Bowling Green State Univ
http://blogs.bgsu.edu/librarysleevefacing/2012/08/15/bookends/

60. „Discoverability‟ needs to be
managed in parallel with „discovery‟
…
Get into research and learning
workflows …
60

61. Directions: some futures
61

62. Full library discovery
Fulfillment at the point of need
Ranking, relating, recommending
Outside-in and inside-out
From strings to things
62

63. Full library discovery

65. Bento Box?
Ranking?
Collections
Resource Guid
Website
People
Events
…
Image: http://www.flickr.com/photos/4563photo/3889124590/sizes/l/in/photostream/ 65

66. Ranking, relating,
recommending
Specialising to institution/person
66

67. Examples
• Analytics
– Improve user
experience
– Improve management
decisions
• Recommendations
• Reading lists/course
reserves
• Specialise to courses
67

68. Fulfillment at the point of need
…
Buy, borrow, connect, …
Demand driven acquisition …
68

69. 69

70. Outside-in vs inside-out

71. Outside in Bought, licensed
Discovery layer
Aim: to discover, to fulfill
Inside out
Institutional assets: special collections,
research and learning materials (IR), institutional records, …
Aim: to *have* discovered … discoverability

72. From strings to things

73. The social graph

74. Three benefits acc to Google:
1.Find the right thing
2.Get the best summary
3.Go deeper and broader

76. Examples
• Include „cards‟ about:
– People
– Works
– Institutions
• Linked data backbones?
• VIAF
76

77. Discoverability
Discovery
Image: http://www.flickr.com/photos/martindo/5975419889/
77

78. Thank
you
@LorcanD
©2013 OCLC. This work is licensed under a Creative Commons Attribution 3.0 Unported License. Suggested attribution: “This
work uses content from [presentation title] © OCLC, used under a Creative Commons Attribution license:
http://creativecommons.org/licenses/by/3.0/”

79. Delivering the Goods
Implementing Web Scale
Michael Kucsak
Director of Library Systems and Technology
University of North Florida
NISO Virtual Conference November 20, 2013

80. Promises
• Google-like searching
• ~98.5% coverage of
library content
• One click access to
full text content
• Everything is better
with <insert product
here>!!!

81. 3 Keys to a Successful
Implementation
1. Support from
management
2. Clearly defined goals
3. Diverse
implementation team

82. Support from Management
“We will be using the EDS first and foremost when we
search; we will be teaching it first in classes; we will be
using it first in one on one instructions; and we will be
promoting it at the new service desk. And, we will keep
very good statistics on all facets of the pilot.”
An email to all library faculty
from the Associate Dean of the Library

83. Clearly Defined Goals
1.
2.
3.
4.
5.
6.
Primary search tool by September
30, 2011
Include all physical and covered
eResources
Seamless integration with our
web site and EZProxy
Successful link to full text ≥90%
Staff trained
Problem reporting process

84. Diverse Implementation
Team
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Acquisitions
Cataloging
Instruction
Reference
Library Systems
Information Technology
Florida Center for Library
Automation
EBSCO Support

85. Changes to Make it Work
• Dropped Serials Solutions 360 for EBSCO’s
LinkSource and A-Z
– Reduce finger pointing
• Dropped ProQuest for EBSCO Databases
– Increase reliability of full-text links

86. Initial Challenges

88. Initial Challenges

89. Initial Challenges

90. Simple Design Philosophy
Connect our patrons to the
knowledge they seek with as little
effort on their part as possible

91. Solutions: Always Provide a link

92. Solutions: Always provide an option

93. Solutions: Always provide an option

94. Solutions: Always provide an option

95. Solution: Integrate core tools

96. Solution: Integrate core tools

97. Web Scale vs ILS
1.
2.
Eliminate MARC serial data loads
ILS access through a widget in the
EDS results page (auto-populated)

98. Web Scale vs ILS
1.
2.
3.
Eliminate MARC serial data loads
ILS access through a widget in the
EDS results page (auto-populated)
Direct upload of e-content into
EDS NOT ILS

99. Web Scale vs ILS
1.
2.
3.
4.
Eliminate MARC serial data loads
ILS access through a widget in the
EDS results page (auto-populated)
Direct upload of e-content into
EDS NOT ILS
EDS as primary search tool on
library home page

100. Testing
1.
2.
3.
4.
Inspected thousands of search
results
Analyzed individual content
provider reliability
Examined link resolver
performance
Confirmed remote access
performance
Achieved 90% reliability!

101. Training
1.
2.
3.
EBSCO provided onsite training
Library Systems/Instruction
trained staff about the mechanics
Library Instruction trained
librarians, students and faculty on
how to use the tool

102. Marketing

103. Blackboard Integration
Blackboard Home Screen

104. Blackboard Integration

105. Multi-Touch Surface

106. Summary
•
•
•
•
•
•
Links to full text
Auto-populate ILLiad
Ulrichs integration
Union catalog widget
Guest default/JIT EZProxy
Available in the Library by
default
• Commitment to EDS
•
•
•
•
•
EBSCO link resolver/A-Z
EBSCO content
EBSCO marketing
Eliminated MARC loads
Pushed reliable vendors
to the top
• Made vendors work for
their $$$

107. Fruits of our Labor
• 107% increase full text
downloads
• >40% reduction in ILL
• Collaborated with CS student
senior projects
• Opportunities to build
partnerships
• EBSCO Success story
• Jacksonville BizJournal Tech
Innovation Award

108. Thank you
Questions?
Michael Kucsak
Director of Library Systems and Technology
University of North Florida
NISO Virtual Conference November 20, 2013

109. Zen and the Art
of WSD
Maintenance
Web-Scale Discovery
Services: Transforming
Access to Library Resources
NISO Virtual Conference
Athena Hoeppner
20 November 2013

110. UCF’s Web Scale Discovery Journey

111. Soft launch was June 2012
•
•
•
•
•
•
•
•
Merged bib records from 11 libraries
Added new content providers to the central index
Implemented LinkSource and AtoZ
Migrated EZproxy to a new server
Upgraded to SFX v4 on a new server
Moved digital collections to a new server
Rolled onto the newest version of EDS
Created API to search and embed FT links in Canvas course system

112. Implementation Questionnaires

113. Some choices we live not
only once but a thousand
times over, remembering
them for the rest of our lives.
-Richard Bach

114. Expected
Maintenance
•
•
•
•
•
•
•
•
•
Branding
Central Index Contents
MARC loads
OAI loads
ILS integration
OpenURL and Links
Search forms
APIs
Any Customizations

115. Tip: Look Under the Hood
•
•
•
•
•
Searching
Databases
Viewing Results
Linking
Branding

116. Maintenance Team

117. Cataloging
Systems
Index Content
Local
• Kim
• Joel
• Bobby
• Subject librarians
FLVC
• Gerald
• Jean
• Ned
• Mark
• Claire
• Brian
• Brian
EBSCO • Brian

118. Technical Support
Greeting Line
•
•
•
•
•
•
•
•
•
•
•
•
Brian
Jessica
Elan
Joseph
Tyler
Ellini
Thomas
Monica
Susan
Sherry
Jillian
Heather

119. Staying Informed
Discussion List
•
•
•
•
New content
New features
Problem sharing
Grousing
Wiki
•
•
•
•
Content details
Partners spreadsheet
Technical specs
Innovative
applications
Support Pages
• FAQs
• Common approaches

120. Still Calm?

121. Expect Problems

122. 1.5 Year’s Worth of Problems
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Embedded search form asking for login
Shows MARC from other libraries
Prefer different relevancy rankings
English limiter eliminates results
Auto-enabling new content providers
Duplicate records showing
Dedupping algorithm changes hit count
Multiple full text/content links showing
Full text links showing from 500 fields
Bad data from content providers
Insufficient data makes bad OpenURL
Malformed DOIs from content providers
Slow response time
Duplicate/multiple login prompts
• 50+ authors showing in the brief view
• Full text HTML showing in the brief view
• Punctuation and stop words causing
•
•
•
•
searches to fail
EBSCOnet renewals turned off full text in
AtoZ and LinkSource.
Sub-locations not showing for catalog
items
Myriad cookies, pop-up, browser and
end-user setting problems
Myriad article access problems - EZproxy
dropped, host config problem, dropped
from aggregator, incorrect holdings
threshold set, etc.

123. KEEP
CALM
AND
INVESTIGATE

124. KEEP
CALM
AND
REPLICATE

125. KEEP
CALM
AND
COMMUNICATE

126. KEEP
CALM
AND
MITIGATE

127. KEEP
CALM
AND
ALLEVIATE

128. Problem Response
• Provide problem-report
forms and options
• Enlist a core team to
respond and investigate
• Replicate reported
problems
• Communicate the
problem in detail to the
vendor
• Change the system to
mitigate unresolvable
problems
• Fix resolvable problems

129. Example Maintenance and Problem Response
• Purchased 12 Alexander Street Press
video collections. Woot!
• Checked EBSCOadmin for option to
turn on indexing. It’s possible, BUT…
• Checked options for linking to videos
• Discussed options with Brian at
EBSCO
• Loaded ASP MARC into the catalog
(thanks, Kim!)
• Nightly update loaded the new MARC
into our EDS
• Checked 856s links in EDS –
PROBLEM!

130. Say Old Man, Do You Play the Fiddle?

131. Say Old Man, Do You Play the Fiddle?
• Checked representation in local view of catalog
• Captured images of the screen in EDS
• Reported problem to Kim, Gerald, Brian, with permalink and
images
• Brian passed the problem to a catalog specialist
• Kim explained the 856 subfields involved
• Gerald fixed the export and EBSCO fixed the processing
It’s like music to my ears!

132. Beyond Maintenance
 Placards
 Profiles for subjects and
content types
 Discipline limitors
 Specialized widgets for
subjects and content types
 Revamp header and footer
 Load LibGuides as a
content source
 Shibboleth
 More APIs

133. WSD Truisms

134. Everything is More Visible

135. Flaws are Magnified

136. Everything is Connected

137. Things Can Get Tangled

138. Tip: Cultivate a Team

139. Tip: Create a Sandbox

140. Tip: Seek Simplicity

141. Finis.
Web-Scale Discovery Services: Transforming Access to Library Resources
NISO Virtual Conference
20 November 2013
Athena Hoeppner
athean@ucf.edu
@cybrgrl

142. NISO Virtual Conference: Web-Scale Discovery Services:
Transforming Access to Library Resources
Update on the NISO
Open Discovery Initiative
Marshall Breeding
http://www.librarytechnology.org/
http://twitter.com/mbreeding
November 20, 2013
142

143. Index-based Discovery
(2009- present)
Search Results
Usagegenerated
Data
Customer
Profile
Digital
Collections
Consolidated Index
Search:
ILS Data
Web Site
Content
Institutional
Repositorie
s
Aggregated
Content
packages
…
Open
Access
E-Journals
Reference
Sources
Pre-built harvesting
and indexing

144. Bento Box Discovery Model
ILS Data
VuFind / Blacklight
Search Results
Web Site
Content
Digital
Collections
Institutional
Repositories
Consolidated Index
Search:
Aggregated
Content
packages
Open
Access
E-Journals
Central index &
search functionality

145. Web-scale search problem
ILS Data
Digital
Collections
Search Results
Consolidated
Index
Search:
Web Site
Content
Institutional
Repositorie
s
Aggregated
Content
packages
…
E-Journals
Problem in how to deal with resources
not provided to ingest into
consolidated index
??
?
Pre-built
harvesting and
Non
indexing
Participating
Content
Sources

146. Discovery Concerns
• Important space for libraries and
publishers
• Discovery brings value to library collections
• Discovery brings uncertainty to publishers
• Uneven participation diminishes impact
• Ecosystem dominated by private
agreements
• Complexity and uncertainty poses barriers
for participation
146

147. Heterogeneous Representations
• Content objects represented by
– MARC Records for books and journal titles
– Citation data for articles
– Full text for articles
– Full text for books
– Abstracts and Indexing products
– Other metadata or enrichment

148. Discovery index issues
• Citations or structured metadata provide
key data to power search & retrieval and
faceted navigation
• Indexing full-text of content amplifies
access
• Important to understand what is indexed
– Currency, dates covered, full-text or citation
– Many other factors
148

149. Library Perspective
• Strategic investments in subscriptions
• Strategic investments in Discovery Solutions to provide
access to their collections
• Expect comprehensive representation of resources in
discovery indexes
– Problem with access to resources not represented in index
– Encourage all publishers to participate and to lower
thresholds of technical involvement and clarify the business
rules associated with involvement
• Need to be able to evaluate the coverage and
performance of competing index-based discovery
products

150. Collection Coverage?
• To work effectively, discovery services need to
cover comprehensively the body of content
represented in library collections
• Why do some content providers not participate?
• How are A&I resources represented?
• Is content indexed at the citation or full-text
level?
• What are the restrictions for non-authenticated
users?
• How can libraries understand the differences in
coverage among competing services?

151. Evaluating the Coverage of
Index-based Discovery Services
• Intense competition: how well the index
covers the body of scholarly content stands
as a key differentiator
• Difficult to evaluate based on numbers of
items indexed alone.
• Important to ascertain how your library‟s
content packages are represented by the
discovery service.
• Important to know what items are indexed by
citation and which are full text

152. Some Key Areas for Publishers
1. Expose content appropriately
2. Trust that access to material will be
controlled consistent with subscription
terms
3. “Fair” Linking
4. Materials not disadvantaged or
underrepresented in library discovery
implementations
5. Usage reporting

153. ODI context
Facilitate a healthy
ecosystem among
discovery service providers,
libraries and content
providers

154. ODI Pre-History
• June 26, 2011: Exploratory meeting @
ALA Annual
• July 2011: NISO expresses interest
• Aug 7, 2011: Proposal drafted by
participants submitted to NISO
• Aug 2011: Proposal accepted by D2D
• Vote of approval by NISO membership
• Oct 2011: ODI launched
• Feb 2012: ODI Workgroup Formed
154

155. Organization
• Reports in NISO through Document to
Delivery topic committee (D2D)
• Staff support from NISO through Nettie
Lagace
• Co-Chairs
– Jenny Walker (Ex Libris)
– Marshall Breeding (Library Consultant)
• D2D Observers: Jeff Penka (OCLC)
Lucy Harrison (CCLA)
155

156. ODI Timeline
Milestone
Target Date
Appointment of working group
Dec 2011
Approval of charge and initial work plan
Mar 2012
Agreement on process and tools
Jun 2012
Completion of information gathering
Jan 2013
Completion of initial draft
Jun 2013
Completion of final draft
Sep 2013
Public Review Period commences
Sep 2013
Status
156

157. Balance of Constituents
Libraries
Marshall Breeding, Vanderbilt University
Jamene Brooks-Kieffer, Kansas State University
Laura Morse, Harvard University
Ken Varnum, University of Michigan
Sara Brownmiller, University of Oregon
Lucy Harrison, College Center for Library
Automation (D2D liaison/observer)
Michele Newberry
Publishers
Lettie Conrad, SAGE Publications
Roger Schonfeld, ITHAKA/JSTOR/Portico
Jeff Lang, Thomson Reuters
Linda Beebe, American Psychological Assoc
Aaron Wood, Alexander Street Press
Service Providers
Jenny Walker, Ex Libris Group
John Law, Serials Solutions
Michael Gorrell, EBSCO Information Services
David Lindahl, University of Rochester (XC)
Jeff Penka, OCLC (D2D liaison/observer)
157

158. ODI Project Goals:
• Identify … needs and requirements of the three
stakeholder groups in this area of work.
• Create recommendations and tools to streamline
the process by which information providers,
discovery service providers, and librarians work
together to better serve libraries and their users.
• Provide effective means for librarians to assess the
level of participation by information providers in
discovery services, to evaluate the breadth and
depth of content indexed and the degree to which
this content is made available to the user.

159. Subgroups for Info Gathering
• Level of Indexing + Communication of
Library Rights
• Technical formats
• Usage Statistics
• Fair Linking
159

160. Specific deliverables
• Standard vocabulary
• NISO Recommended Practice:
– Data format & transfer
– Communicating content rights
– Levels of indexing, content availability
– Linking to content
– Usage statistics
– Evaluate compliance
• Inform and Promote Adoption
160

161. ODI Stakeholder Survey
• Collected data from Sept 11 thru Oct 4,
2012
• Each subgroup developed questions
pertinent to it area of concern
161

162. Survey Responses
•
•
•
•
782 Librarians
74 Publishers
15 Discovery Services
871 Total
162

163. Selected results
• Libraries: do you use a discovery service?
– Yes: 74%, Planning to soon: 17%, No: 5%,
Don‟t know: 4%
• Smallest discoverable unit:
– Component title: 9%, Article: 25%, Collective
work record: 11%, All the above: 50%
• Linking from A&I entry: 75 prefer linking to
full text on original publisher‟s server
163

164. Librarian’s preferred Use
statistics
• Total Number of Searches
• List of search query terms
• Referring URLs
164

165. Content providers (74)
• Contribute data: Yes-All: 44%, Some: 48%,
No: 8%
– Current data: 12%, Current + back files: 85
• Barriers to contributing:
– IP concerns, technology, staff resources
• Challenges in delivery:
– Complicated formats: 15%, transmission of
data: 18, allocation of personnel: 23%, can‟t
automate: 12%, None: 20%
165

166. Issues surrounding A&I
resources
• Concern that A&I resources not be freely
available to non authenticated users and
only for subscribing institutions
• How to “credit” A&I data that contributes to
search results
– Example: Index entry produced by enhancing
full-text with A&I data
• Preservation of the value added by A&I in
the discovery ecosystem
166

167. ODO Survey Report
• Issued January 2013
• NOT the final report for ODI
• Survey findings, especially for those that
responded to survey
• One source of input for the ODI final report
of findings and recommended practices
167

168. ODI Final Report
• Issued for public Comment
• Comment period closed November 18,
2013
168

169. Report Topics
• Introduction
– In scope / out of scope
– Terms and definitions
• Evolution of Discovery
– Related initiatives
• Recommendations
169

170. General Recommendations
• Create oversight group
• Actions for content providers and discovery
service creators to assert conformance
170

171. Recommendations for
Content Providers
• Content providers should make items
available to discovery service providers.
– Basic: Citations: specific metadata elements
– Enhanced: additional metadata + Full-text
• Provide to Libraries: disclosure of
participation in discovery services
171

172. Recommendations for
Discovery Service Creators
• Disclosure of content indexed
– Specific metadata fields
• Fair / non-biased linking
– Mechanisms for libraries to choose versions
preferred for linking
– Annual statement regarding neutrality of
linking or relevance
– Provide links to A&I services when applicable
• Usage statistics to Publishers
– Searches
172

173. Report Highlights
• What is in and out of Scope
– Focus on content available to be indexed
– Quantity and form of content (citations /
fulltext)
– Metadata fields contributed
– Role of A&I products
– Controlled Vocabularies
• Out of Scope
– Relevancy algorithms
– User Interface issues
173

174. Technical recommendations
• Transfer of data from content providers to
discovery service creators
– Make use of existing standards and protocols
when possible
174

175. Current work Next Steps
• Review comments received
– Chairs + Workgroup members
– Make any needed revisions
– Submit for final approval by NISO D2D
175

176. Connect with ODI
• ODI Project website:
http://www.niso.org/workrooms/odi/
• Interest group mailing list:
http://www.niso.org/lists/opendiscovery/
• Email ODI:
odi@niso.org
176

177. NISO Virtual Conference
Web-Scale Discovery Services:
Transforming Access to Library Resources
Questions?
All questions will be posted with presenter answers on
the NISO website following the webinar:
http://www.niso.org/news/events/2013/virtual/discovery
NISO Virtual Conference • November 20, 2013

178. THANK YOU
Thank you for joining us today.
Please take a moment to fill out the brief online survey.
We look forward to hearing from you!