Recent history has taught us that we must begin assessing what it is we really do, alter our record-keeping to include an ever-widening group of new services and features, provide evidence that we are actually accomplishing our goals, and find open-ended assessment tools that anticipate future change in library operations. This type of rigorous self-examination makes it more difficult and perhaps unwise to use a one-size-fits-all statistical analysis. Accordingly, this presentation focuses on the process necessary for meaningful and dynamic statistical analysis, including: parsing your mission statement to discover categories of evaluation, brainstorming key indicators that relate directly to these categories, leveraging your organization's current statistical analyses, and evaluating your methods to ensure future adaptability.

1. Putting “Value” in Evaluation
Building Relevant, Dynamic Statistical
Analysis
Presented by Joshua Johnson @ #ULA2013
See Josh Blog: www.joshinglibrarian.info
See Josh Tweet: @JoshinLibrarian

2. Don’t We Already Use
Statistical Analysis?
We’ve been doing this for years;

3. Librarians are well known as gatherers of
statistics and other data. However, we do not
always use the information that we gather,
preferring instead to point to the numbers
themselves as evidence of our work.
Blake & Schleper, “From Data to Decisions…”
Quotable Research/Literature Review

4. What is the worth of the library in the networked-
computer age? How do shifts in use patterns reflect
changes in customers’ valuations of library
services, and how would customers prefer that
library resources be added or reallocated? What
benefits are conferred on different types of library
customers by their variant uses of … libraries?
And, how can those benefits be measured?
Holt & Elliott, “Measuring Outcomes…”
Quotable Research/Literature Review

5. Librarians may be good at counting, but as a whole
the profession is not trained to evaluate and
analyze statistics. Librarians offer numbers often as
proof of the value of their work with little thought as
to whether those numbers really establish anything
of value…Library Managers need to think critically
about which statistics are useful.
Tim Spindler, “Statistical Analysis Models:
Applications for Libraries”
Quotable Research/Literature Review

6. It doesn’t really matter whether you can quantify your
results. What matters is that you rigorously assemble
evidence—quantitative or qualitative—to track your
progress.
If the evidence is primarily qualitative, think like a trial
lawyer assembling the combined body of evidence. If
the evidence is primarily quantitative, then think of
yourself as a laboratory scientist assembling and
assessing the data.
Jim Collins, Good to Great and the Social Sectors
Quotable Research/Literature Review

7. Examine Current Methods of Analysis
• Are you basing reference staffing decisions on
circulation data?
• Are you comfortable extrapolating future
trends from your current data sets?
• Does the data you collect match the services
you currently provide?
• Is the process by which you collect and
synthesize your data easily transferred from
one staff member to another in the event of a
promotion or retirement?

8. WHERE DO I START?
Okay, I’ll bite. I’m rethinking my library’s statistical analysis anyway;

9. Evaluate Your Mission Statement
• Does your mission statement accurately
describe what your library does?
• What promises are you making in your mission
statement?
• Which of your services, collections, etc. are
related to each promise?
• Are you considering adding to your services
and unsure if you can handle the change
without adding staff?

10. Evaluate Your Mission Statement
For example, part of the mission statement
of the American Library Association:
The American Library Association was
created to provide leadership for the
development, promotion, and improvement of
library and information services and the
profession of librarianship in order to enhance
learning and ensure access to information for
all.

11. Evaluate Your Mission Statement
Let’s highlight a few of the promises that
reasonable people might assume based on
the quote:
• ALA members will be better at promotion.
• ALA members will be better librarians.
• Library patrons served by ALA members will
be better served than those served by non-
ALA members.
• Library patrons served by ALA members will
receive better access to information than those
served by non-ALA members.

12. Brainstorm Related Key Indicators
• What factors, practices, personnel influence your
ability to make good on these promises?
• What physical or monetary limitations could
impact your organization’s ability to fulfill
promises?
• How might you be able to assess the
organization’s perceived value or measure your
effectiveness at providing services or staffing?
• Would the organization be better able to fulfill
some promises if it changed the promises it made
or cut some services in favor of others?

13. In this case, the indicators represent daily, measurable tasks
performed by each set of employees. I consider the lists of
indicators “in process.”
What Do I Mean by Indicators?

14. TAKE THE PROCESS FOR A
SPIN
Now that we’ve talked about the process,

15. Parsing the Mission Statement
• Form groups and select a “scribe.”
• Examine the sample mission statement as a
group and write as many promises made as
possible in the time allotted.
• Was it difficult? Was it worth it?

16. Brainstorming Key Indicators
• Stay in the same groups.
• Take your sample mission statements and the list
of promises your group made in the last exercise
and switch them with another group.
• Read the mission statement as necessary, but
focus on the promises made by the statement.

17. Brainstorming Key Indicators
• Make a list of indicators (services, practices, etc.) that
relate to these promises.
• As your group brainstorms, keep in mind:
• What factors, practices, personnel influence the
organization’s ability to make good on these
promises?
• What physical or monetary limitations could
impact the organization’s ability to fulfill promises?
• In what ways might you be able to assess the
organization’s perceived value or measure your
effectiveness at providing services or staffing?
• Would the organization be better able to fulfill
some promises if it changed the promises it made,
or cut some services in favor of others?

18. Process Evaluation
• How difficult was this process?
• How productive was it?
• What other documents could be used in addition
to mission statements?
• What kinds of ideas did it spark for examining
your own workplace processes?
• How could we streamline the brainstorming
process?

19. What reference indicators might reasonably be
added, based on your experience in libraries?
Process Evaluation

20. WHERE DO I GO FROM HERE?
That was interesting, and perhaps useful, but

21. Leverage Current Statistical Analyses
• Reuse the data and statistical analysis you
already have. You almost certainly keep some
form of raw statistical record-keeping or statistical
analysis. There’s no need to completely reinvent
the wheel.
• Track additional data as needed - make sure it is
relevant.

22. Leverage Your Current Statistical
Analyses
• Nearly all of the information used in this project was pulled directly
from statistics already kept by the library system.
• Excel, for example, will pull information from other Excel files and
update it automatically; this makes pulling data for your own projects a
breeze.

23. “FIND & REPLACE” & MACROS SIMPLIFY REPETITIVE TASKS.
Leverage Current Statistical Analyses
• Take advantage of time/labor saving computer programs.
Spreadsheets often allow you to use macros and a wide
range of other helps.
• The example below contains instructions for
copying/updating monthly statistics using the “find &
replace” function available in many spreadsheets. Each
type of information can be updated in a slightly different
way.

24. Leverage Your Current Statistical
Analyses
• Be logical - Be certain the picture you paint with
statistics is as objective as possible; some
statistics are misleading or unethical.
• Compare apples to apples - This can be more
difficult than you think. Statistics make more
sense when you find ways to compare
commonalities.

25. Methods Should Allow Adapdability
• Think about how you lay out your information;
why organize it over and over?
• Will your methods be easily passed to your future
successor?

26. • What libraries “do” in 5-10 years will change, so expect it.
• When planning, leave room to grow, even if it is just a blank
space in the sheet.
• Review promises and indicators for relevance and value to
ensure the most valuable evaluation of your organization.
Evaluate Methods to Ensure Adaptability

27. Brainstorming Helps for Key Indicators
Here are some examples to get you going; they are
not exhaustive. Many come from Blake & Schleper
(2004).
Quantitative
• Circulation statistics (most commonly "check-outs,"
but also in-house uses, etc.)
• Website analytics (hits, unique visitors, duration, etc.)
• Subject/date analysis (examining average publication
dates in a given subject or collection)
• Cost-per-use analysis
• ILL requests

28. Brainstorming Helps for Key Indicators
Qualitative
•Comparison of collections to peer organizations
•User input
• Questionnaires
• Interviews
• Compare holdings to collection development policies
• Comparison of overall services to peer institutions
• Print collections
• Electronic holdings
• eBooks/vendors
• Study space
• Training/community outreach
• Staffing levels

29. Brainstorming Helps for Key Indicators
Qualitative
• Physical examination of collection (wear & tear or dust)
• Anecdotes (patrons' or colleagues' opinions)

30. QUESTIONS?
Are there any

31. Spindler, T (2009). Statistical analysis models: Applications for libraries.
Library Publications. Paper 11. http://docs.rwu.edu/librarypub/11
References & Suggested Resources
Blake, J C & Schleper, S P. (2004). From data to decisions: Using
surveys and statistics to make collection management decisions.
Library Collections, Acquisitions, & Technical Services 28(4).
Holt, G & Elliott, D. (2003). Measuring outcomes: Applying cost-benefit
analysis to middle-sized and smaller public libraries. Library Trends. 51(3).
424-440 <link to article>
Collins, J (2005). Good to great and the social sectors: Why business
thinking is not the answer. Boulder, CO: J. Collins. Print.

32. Presented @ #ULA2013
By Joshua Johnson
Josh’s Blog: www.joshinglibrarian.info
Josh’s Tweets: @JoshinLibrarian