This document outlines the Rare Book and Manuscript Library's (RBML) efforts to conduct a user assessment survey using the Archival Metrics Toolkit. It describes how the RBML lacked quantitative data on user experiences and services. The RBML conducted a year-long survey of 566 onsite researchers to understand user needs, evaluate services, and support a culture of assessment. The survey found that most users were satisfied with facilities and services, though some areas like website and finding aids could be improved. The RBML aims to use assessment data to strategically support user needs.

1. WHY IS THIS
ASSESSMENT DIFFERENT
FROM ALL OTHERS?
Assessment of Archives & Special Collections
NISO Virtual Conference
Wednesday April 20, 2016

2. About the RBML

3. Why A Special Kind of Assessment?
• Lack of information regarding RBML services from
existing LibQual+ surveys
• Major changes at RBML in 5 year period but no
quantifiable data to validate informal observations of
increased use and patron satisfaction
• Library administration increasingly invested in using data
to drive decision making

4. Preparations
• Review of current literature in both library assessment
and user satisfaction studies
• Institutional Research Board (IRB) training and approval
• Brainstorming information needs
• Soliciting input from stakeholders

5. Brainstorming Worksheet

6. Goals
• To understand users’ needs and experiences using the
RBML
• To evaluate service at the RBML
• To collect data in a more strategic way and support
Libraries’ commitment to assessment
• Build a culture of assessment into RBML in a
programmatic and repeatable way

7. Lots of
Data- No
Big
Picture
LibQual
In-house
Database
Reading
Room
Stats

8. Archival Metrics Toolkit

9. Archival Metrics Toolkit Surveys
• Onsite Users
• Students who have attended an orientation
session
• Instructors who use the archives for
teaching
• Online users of the website
• Online users of finding aids

10. ARCHIVALMETRICS.ORG

11. Survey Form and Collection Box

12. Methodology
• Instrument - Archival Metrics Toolkit Researcher
Questionnaire
• Data collection - September 1, 2011 to August 30,
2012
• Participants - visiting researchers who
volunteered to participate in the project.
• Response rate – 62% (566 respondents)
12

13. Are you affiliated with Columbia University?
Yes
37%
No
63%

14. What best describes your position?
Faculty member
or post-doc
24%
Other
23%
Undergrad.
15%
Master's
16%
Doctoral
22%
Students
53%

15. Which best characterizes the project that motivated this visit to
the Rare Book and Manuscript Library?
7 (1%)
9 (2%)
10 (2%)
29 (5%)
42 (7%)
49 (9%)
117 (21%)
134 (24%)
198 (35%)
Film or video
Curriculum development/teaching preparation
Family history project
Administrative or work-related product
Gathering information, but don't have a final
project in mind
Other
Class assignment
Dissertation or thesis
Publication (e.g. article, book)
RESPONSE COUNT (RESPONSE %)
TYPEOFPROJECT

16. How many times have you used the Rare Book and
Manuscript Library?
74%
16%
4%
7%
First time 2-5 times 6-10 times > 10 times

17. How long have you been using archival materials?
31%
11%
26%
32%
First time < one year 1-5 years > 5 years

18. Please indicate your level of satisfaction with the facilities, services and staff of the RBML.
Questionnaire items 1 (completely dissatisfied) 5 (completely satisfied)
No
opinion
Response
count
Study areas 0% 0% 4% 16% 72% 8% 495
Noise level 0% 2% 6% 21% 71% 0% 502
Furniture 1% 1% 7% 18% 71% 2% 498
Physical Access to the building 0% 1% 4% 18% 70% 7% 489
Lighting 1% 3% 8% 21% 67% 0% 504
Informational/ navigational signs 0% 2% 8% 20% 60% 9% 491
Temperature 2% 5% 12% 20% 60% 0% 502
Internet access 0% 1% 1% 7% 57% 34% 479
Hours of service 1% 5% 16% 26% 51% 1% 499
Website 0% 1% 6% 21% 50% 22% 477
Catalogs/ indexes/ findings aids 0% 2% 5% 20% 43% 30% 467
Exhibits 0% 0% 1% 10% 41% 47% 466
Reference books 0% 0% 2% 5% 28% 65% 465
Photocopying / duplication services 1% 2% 2% 5% 24% 67% 455
Microfilm and fiche viewing facilities 0% 1% 1% 3% 19% 76% 466

19. 19

20. A Culture of Assessment
20
Online registration
and request system
for special collections
Online reference tracking
system used by all CUL

21. Comments, Questions?
Jocelyn Wilk
jkw19@columbia.edu