Given the decreasing budgets for collections at many libraries, librarians are looking at abstracting and indexing (A&I) resources and full-text databases with a more skeptic eye. In addition to traditional evaluation measures, such as costs, usage and faculty input, we looked at the overlap of indexing and/or full-text coverage. Those who have conducted such overlap studies have approached it at either the journal or article level. Article-level overlap studies demonstrate coverage of selected articles in the databases under study. Conversely, journal-level studies examine the extent of indexing of journals among the selected databases. Both methods are very time-consuming and require extensive resources. A simplification of the journal-level method is to compare lists of journals indexed. Two tools, Cufts and JISC's Academic Database Assessment Tool (ADAT) are very useful for this purpose, but do not include all databases. Many databases need to be manually collected. This presentation will describe the background to such a project, the specific tools and procedures used, how the results were used to address budget reductions, and the limitations of the results. Members of the audience will be able to consider using this method for evaluating their own abstract and index databases for budgetary purposes.
Presenter:
Karen Harker
Collection Assessment Librarian, University of North Texas
3.
Web-scale discovery systems are changing
the information ecosystem
Duplication of content is no longer a necessary
evil
Materials budgets being squeezed
Key Question:
What could we drop, if we had to?
What we wanted to know
4.
Done the hard way
Limited to Abstract & Indexes
Only
First Attempt, 2012
• Download Lists
• Import into
Database
Coverage
• Paired
Comparisons
• On ISSN
Match • Overlap Rate
• Unique Titles
Report
5.
Outcome of First Attempt
The Good
Confirm suspicions
Some databases cut
Updated information
Some additional info
coverage years
scope of coverage
Provided foundation
database
The Bad & the Ugly
Took a lot of time
Some lists not easily
transportable
Some lists without
ISSNs
Limited to one-on-one
comparisons
6.
Another round of budget cuts
Looking at full-text aggregator databases
Gathering more data about resources
Less time to devote
Graduate Student Assistant help
Second Attempt, 2014
8.
Academic Database Assessment Tool (ADAT)
Originally from JISC (UK)
Now administered by Center for Research Libraries
(CRL)
Lost some functionality
Selected resources
Paired comparisons only
No differentiation between indexing or full-text
JISC ADAT
9.
What ADAT used to
provide
Jacso, P. (2010). The JISC academic database assessment tool -
- virtues and vices. Online Information Review, 34(5), 806-814.
11.
Open Source Serials Management
Simon Fraser University, BC, Canada
Public tools: Resource Comparison
Broad selection of resources
Differentiates between indexing and
full-text
CUFTS
12.
Like CUFTS
Specific to UNT Libraries’ holdings
Compare one to any number of
resources
Full-text only – no information on
indexing alone
Serials Solutions
Overlap Analysis
19.
Number of…
Unique Titles
Titles Overlapped in Target Database
Unique Full-Text Titles
Full-Text Titles Overlapped in Target
Database
Kind of data collected
20.
Data derived
Indexed
Total number of titles
indexed
% of Unique titles
% of titles Overlapped
Full-Text
Total number of full-
text titles
% of Unique full text
titles
% of full-text titles
Overlapped
37.
Conclusions on Data
Gathering
Quick
Easy to run
Recently updated
JISC summary table
Serials Solutions
summary
Dirty
Data didn’t match
Data changed over time
CUFTS
Sloooow
Summary difficult to
find
Data requires tweaking
39.
The Good, the Bad
& the Ugly
JISC ADAT
The Good: Simple table
The Bad:
Limited number of databases
15 Bibliographic
9 Full-Text
Only paired-comparison
No pretty pictures
The Ugly: Unable to
download or export results
CUFTS
The Good:
Extensive list of databases and
versions.
Can download results.
Can compare indexing & full-
text separately
Can compare up to 4 databases
at once
The Bad: Some resources not
available
The Ugly: Had not been
updated at time of first
attempt.
40.
The Good, the Bad
& the Ugly
Serials Solutions Overlap
Analysis
The Good:
Easy to run
Nice summaries
Can download results
Compare any number of
resources
Serials Solutions Overlap
Analysis
The Bad: Limited to
full-text coverage only
The Pretty: Is BEST!
41.
Comparison of Sources
The Good
Mostly consistent
Within 10%
Most databases &
aggregators covered
About 15% not available
Some coverage details
Year
Scope
The Bad
Duplication of titles
Missing ISSNs
Lack of coverage details
Year
Scope
44.
The Good: Much faster
The Bad: Not all databases
available
The Ugly: Some data unreliable or
inconsistent.
Outcomes
45.
Need All Four Wheels
ADAT CUFTS
SerSol Manual
Hart, Alfred. Improvement in Wagon-Brakes., Patent, December 4, 1877; digital
images, (http://texashistory.unt.edu/ark:/67531/metapth166725/ : accessed April 24, 2014), University
of North Texas Libraries, The Portal to Texas History, http://texashistory.unt.edu ; crediting UNT
Libraries Government Documents Department, Denton, Texas.
Hinweis der Redaktion
In the days of silo’d databases – each with their own search interface – overlap of content was essential.
In the world of Discovery systems, this duplication is no longer necessary
This is what you get now – a simple table, yes, but of limited value. And while the linked numbers do provide a list of titles – that’s all – all of the info about each title is hidden behind another link. You can’t download the lists with the ISSN.
EconLit is the target database, the one we were interested in.
Clicking on the total overlap number gives you the full list of titles, which also provides details about the overlap, where they are overlapped and for which year. It can be downloaded into Excel.
And then there’s Coyote Ugly…For some resources, we ended up with wildly different number of titles reported by the systems. Here is the strange case of EconLit. When we compared the full-text coverage to Business Source Complete, each system we used to compare the two showed such different numbers that it was simply impossible to determine the true overlap. Interestingly, Serials Solutions & CUFTS were quite close and showed similar overlap rates. But this did not match the full-text journals list provided from EBSCO itself! It turns out that SS & CUFTS include reports and other kinds of content, not just journals, which explains the difference. Because the usually don’t have ISSN’s, it is quite difficult to match these kinds of content.