2. Library Impact | The Challenge
How do we assess, measure, and report library impact?
3. Library Impact | The Challenge
How do we equate the information hidden in library
usage statistics to measures of student achievement?
4. Library Impact | The Challenge
How do we increase higher education’s collective
intelligence of library contribution to student success?
Let’s start with research….
5. Source: Oakleaf, Megan. The Value of Academic Libraries. Association of College and Research Libraries, 2010.
“Libraries must share information
with others by clearly aligning
library services and resources to
institutional missions.
Communicating that alignment is
crucial for communicating library
value in institutional terms.”
2010: Objective Defined
6. Source: Oakleaf, Megan. The Value of Academic Libraries. Association of College and Research Libraries, 2010.
“Libraries cannot demonstrate
their institutional value to
maximum effect until they define
outcomes of institutional
relevance and then work to
measure the degree to which they
attain them.”
2010: Task Declared
2012/13 2013/14 2014/15 2015/16
7. An evidence-based report substantiating library contributions
to student learning and student success is published:
2016: Research Released
Documented Library Contributions to Student Learning and
Success: Building Evidence with Team-Based Assessment in
Action Campus Projects. Prepared by Karen Brown,
Association of College and Research Libraries, 2016.
Now to the findings….
8. Library Impact | Findings Released
1. Students benefit from
library instruction in their
initial coursework.
2. Library use
increases student
success.
Three Year
Study
Results
Source: Brown, Karen. Documented Library Contributions to Student Learning and Success. Association of College and Research Libraries, 2016.
9. Library Impact | Findings Released
3. Collaborative academic
programs involving the
library enhance learning.
4. Information literacy
instruction strengthens
general education
outcomes.
Three Year
Study
Results
Source: Brown, Karen. Documented Library Contributions to Student Learning and Success. Association of College and Research Libraries, 2016.
10. Library Impact | Assessment Results
5. Student retention improves with library instructional services.
6. Library research consultation services boost student learning.
7. Library instruction adds value to a student’s long-term academic experience.
8. Use of library space relates positively to student learning and success.
More Findings2016
Source: Brown, Karen. Documented Library Contributions to Student Learning and Success. Association of College and Research Libraries, 2016.
11. Library Impact | Institutional Alignment
SRJC Mission
We focus on student learning...
by preparing students for transfer;
by providing responsive career and technical education;
and by improving students’ foundational skills.
12. Do our measures of library
usage reflect this mission focus?
What do we measure?
13. Library Impact | Reported Statistics
We measure these….
Collections – Volumes held, volumes added
Expenditures – Information expenditures, operating expenditures
Services – Items loaned and borrowed, library hours
Staffing – Full time equivalent staff, other paid staff
Electronic Services – Reference by e-mail or web, digitized documents
Information Literacy – Institution SLOs, information literacy in SLOs
Virtual Reference – Reference by e-mail, chat, and text messaging
14. Library Impact | Reported Statistics
….with great confusion
Electronic Services (Table 11). Percentage of academic libraries with
selected electronic services
Virtual Reference (Table 13). Percentage of academic
libraries with virtual reference services.
“The way you
organize things
says a lot about
you.”
--Abby Covert, Author
Are we clear on what we are
collecting?
16. Library Impact | Reported Statistics
Correlation of
library
collections to
student success
is not readily
evident.
17. What could we measure in
addition to usage statistics?
18. Library Impact | How We Use Data We Collect
We started by measuring # of students taught…and made adjustments
Source: SRJC Library Annual Report Data. 2010-2015.
Identify trends
Make resource and
program adjustments
19. But did we correlate the
instruction sessions to an
institutional outcome?
Much harder to do…..
20. …but it can be done.
U of Minnesota measured the
impact of reference interactions
and instruction sessions on GPA.
“Students who use the library
had an average GPA of 3.18
compared with the average GPA
of students who did not use the
library, which was 2.98.”
Library usage correlated
directly with academic success.
Source: Krista M. Soria, Jan Fransen, Shane Nackerud. “Library Use and Undergraduate Student Outcomes: New
Evidence for Students’ Retention and Academic Success.” Libraries and the Academy, 13. 2 (2013). 147–164.
22. Library Impact | Narrative Without Correlation to Outcome
349 students received
one-on-one instructional
assistance through a
Research Assistance
Program where they met
with a librarian for
focused support.
Source: SRJC Library Annual Report Data. 2010-2015.
23. Transform the underprepared into the prepared
Help students achieve course completion
Source: Head, Alison J. and Michael B. Eisenberg. "Truth Be Told: How College Students Evaluate and Use Information in the Digital Age." Project Information Literacy. 2010.
84% of students report difficulty just getting started
on assignments.
Library Impact | Framing the Message
Librarian research assistance offers a convenient way
for students to access one-on-one help:
24. Library Impact | Correlating Library Instruction with Student Success
In equal groups of students offered voluntary tutorial sessions:
No tutorial session with a librarian:Tutorial session with librarian:
Source: LIR 10 Summer 2015 Anonymized Assessment Data. L Esparza, August 2015.
No session with librarian
Received lower grades than the
lowest grade earned in the
group of students who met
with a librarian.
26. Library Impact | Correlating Library Use with Access
98,660 items
circulated, including
28,966 textbooks
2015 annual estimated cost of textbooks for
average community college student: $1,328
Source: CollegeBoard. Average Estimated Undergraduate Budgets. 2015; SRJC Libraries. 2014-15 Annual Report. 2015.
An infusion of student
equity funds allowed
the library to buy costly
textbooks.
27. Library Impact | Correlating Library Use with Student Achievement
At SRJC, students who
checked out textbooks
from the library had a .20
higher GPA than students
who did not.
Source: SRJC Libraries. Textbook Checkouts. July 1, 2015 to May 1, 2016.
29. Library Impact | Equity Measures
Provide environments to fully
develop student potential
Increase access
Achieve course completion
Ensure that underrepresented
groups have equal opportunity
Source: California Community Colleges Chancellor’s Office. Student Equity Fact Sheet. 2014
Do libraries have
ready data to
demonstrate ways
we contribute to
student equity?
30. Source: Head, Alison J. and Michael B. Eisenberg. "Truth Be Told: How College Students Evaluate and Use Information in the Digital Age." Project Information Literacy. 2010.
Library Impact | Helping Underrepresented Groups Succeed
The library increases access
for students in financial need,
and for first generation
students who may enter
college with limited
knowledge of academic
jargon, behaviors, and
expectations.
31. Library Impact | Creating Safety Nets to Help with Retention
The library started actively tracking all non-research questions to
better understand where our students are overwhelmed in the
academic experience and at risk of faltering.
Source: SRJC Libraries. 2014/15 Annual Report.
32. Library Impact | Profile of Students Who Use the Library
The library reaches a higher
concentration of under-represented
populations than exists in the SRJC
student population mix.
Source: SRJC Libraries. 2014/15 Annual Report.
33. Library Impact | Increased Access and Reduced Barriers
Source: SRJC Libraries. Reserves Checkouts from 1/20/15 – 2/10/15. Anonymized data.
The library directly
supports student equity
by increasing support to
under-served and at risk
populations.
34. Library Impact | Increased Access and Reduced Barriers
Doyle Library textbook
checkouts increased 15% in
2014/15 due to the availability
of textbooks from Student
Equity funds.
Library textbook reserve checkouts by
student population group
Libraries College
Latino 36.7% 30.7%
Asian 6.7% 4.4%
African American 5.6% 2.3%
Pacific Islander 0.9% 0.4%
Source: SRJC Libraries. Reserves Checkouts from 1/20/15 – 2/10/15. Anonymized data.
Increasing support to under-
served and at risk populations.
35. Library Impact | Bridging the Online Performance Gap
Source: Johnson, Hans and Marisol Cuellar Mejia. Online Learning and Student Outcomes in California’s Community Colleges. Public Policy Institute of California, p.9. May 2014.
Online Performance Gap
(percentage points)
We need to help lessen the
performance gap students
experience in the online classroom.
The bigger the gap, the
worse a student’s
performance is in online
courses compared to
traditional ones.
36. Library Impact | Underprepared Students Are Less Likely To Complete
SRJC Student
Completion by Level
of Preparedness
Source: Santa Rosa Junior College Student Success Scorecard. 2016.
Completion:
Degree
Certificate
Transfer
37. Library Impact | Underprepared Students Are Less Likely To Complete
Source: Hensen and Hern. Let Them In: Increasing Access, Completion, and Equity in College English. California Acceleration Project. 2014.
…prepared for college-level work in
English and math: completion and
transfer rate of 70%.
…unprepared for college-level work and
enrolled in remedial courses:
completion and transfer rate of 40%.
Students who are …
Not good because:
38. Library Impact | Opportunities for Increased Service
35% of students who checked out textbooks had taken Basic Skills
level courses in remedial English and Math.
SRJC Basic Skills Students Who First Enrolled
Below Transfer Level
38% Remedial Math 44% Remedial English
39. Using technology to bridge
barriers to learning in the online
environment.
With no budget, of course…
40. Library Impact | Increase Understanding of Complex Research Systems
California ranks lowest in the
nation in student exposure to
libraries: 1 librarian for every
7,657 students in public
schools.
Students enter college ill-
prepared to navigate the
complex research systems.
Integrate Active Learning Tutorials
to Provide On-Call eCoaching
41. Library Impact | Increase Understanding of Complex Research Systems
Source: Virtue, Alicia, Ellen Dean and Molly Matheson. “Assessing Online Learning Objects: Student Evaluation of a Guide on the Side Interactive Learning Tutorial Designed by SRJC Libraries.” Interdisciplinary
Journal of E-Learning and Learning Objects, 10. 2014.
96% of students reported
increased understanding of
library research interfaces after
taking the interactive Guide on
the Side coaching tutorials.
55% of students exhibited an
increase in correct responses to
tested assessment when
compared to those students
who had not taken the
coaching tutorials.
42. Library Impact | Use Videos to Personalize the Online World
Source: Kolowich, Steve. “The Human Element.” Inside Higher Ed. 29 March 2010.
Connecting student to
instructor is a critical
component in online
instruction. Failure can be
directly attributed to the
lack of human presence
students experience in the
online environment.
43. Library Impact | Use Videos at SRJC to Increase Familiarity and Confidence
Develop a series of brief online videos to provide a crash course in
library services, facilities, and support for students who are unfamiliar
with academic research libraries.
44. Library Impact | Library-LMS Integration
Integrating library resources and
support services directly into the
learning management platform may
be accomplished in several ways.
46. Library Impact | Promote Student Success
Student Success Librarian: Basic
Skills and College Skills
Implement and expand instruction
and academic engagement
programs aimed to assist new
students and basic skills students
in learning the tools of scholarship
while adjusting to college life.
47. Library Impact | Promote Student Success
Student Success Librarian: ESL/ELL
and Underserved Populations
Implement and expand instruction
and academic engagement
programs aimed to assist target
student groups in learning the
tools of scholarship while adjusting
to college life
48. Library Impact | Student Retention Factors
Consistently accessible and responsible staff
Frequent collaborative contact with faculty
Efficient, convenient, and responsive libraries
Curricular integration
Source: Oakleaf, Megan. The Value of Academic Libraries. Association of College and Research Libraries, 2010.
49. Library Impact | Information Literacy Instruction Strengthens GE Outcomes
Arts &
Humanities
Behavioral
Sciences
Business &
Professions
Culinary,
Agriculture
Health
Sciences
Early
Childhood
STEM
Learning
Resources
Language Arts
Public Safety
Kinesiology &
Athletics
Student
Services
Work
Experience
In one academic year,
librarians conducted 346
course integrated
instruction sessions,
reaching 13 clusters,
providing instruction
across 29 disciplines
working with 117
instructors.
Source: SRJC Library Annual Report Data. 2015/16 Academic Year.
50. Library Impact | Contributing to Student Retention
Source: SRJC Library Annual Report Data. 2010-2015.
Information literacy
instruction, along with
strong curricular
integration, and academic
collaboration with discipline
faculty, library instruction
reaches 40% of all students.
51. How do we share our impact
with the college community?
52. Library Impact | Increasing Awareness
Shift from statistical reports to visual messages that help our
college community see the connections between libraries and
student success.
Use the same vocabulary articulated in institutional goals.
53. Library Impact | Increasing Awareness
First attempt at an annual report was a step away from reporting
statistics, but did not directly address institutional goals.
54. Library Impact | Increasing Awareness
Then we added expanded our message to show the responsive,
consistently accessible services the library provides.
55. Library Impact | Increasing Awareness
The following year we
articulated the impact areas
into outcomes:
Foster learning and
academic excellence
Expand opportunities and
eliminate barriers
56. Library Impact | Increasing Awareness
More outcomes:
Provide responsive learner-
centered environments
Engage students and spark
intellectual curiosity
Emphasis shift: collection usage
no longer the primary message.
57. Library Impact | A Call to Arms
ACRL says “librarians have failed to
explain to those outside the field what
contributions they make.”
Source: Oakleaf, Megan. The Value of Academic Libraries. Association of College and Research Libraries, 2010.
ACRL: The transformed library seeks to fulfill campus goals. Libraries need to create connections between academic library services and institutional outcomes. This is very hard to do unless libraries have the capability to directly correlate performance of students with their library interactions
What are these measures? Completion rates, retention rates, transfer rates, efficiency rates, grade point averages…
Useful follow up to the 2010 published work of ACRL. The findings of a three year study sponsored by ACRL in a three year program called Assessment in Action: Academic Libraries and Student Success. 64 institutions participated, representing a variety of colleges and universities from 34 states in the US and participation from Canada.
With a focus on freshman and new students, the first finding correlated student acquisition of common set of competencies for undergraduate study. Takeaway: students receiving information literacy instruction perform better than those who do not.
By analyzing multiple data points, including circulation usage, online database access, study room use, and other library services), the findings show that students who use the library in some way achieve higher levels of academic success, measured as GPA and retention, than those who did not use the library.
Librarian partnership with writing centers, academic enrichment and speech labs yield positive results for students – higher grades, academic confidence, retention.
Libraries improve general education outcomes and contribute to inquiry-based and problem solving learning, including critical thinking, ethical reasoning, global understanding and civic enrichment.
The findings of a three year study sponsored by ACRL in a three year program called Assessment in Action: Academic Libraries and Student Success. 64 institutions participated, representing a variety of colleges and universities from 34 states in the US and participation from Canada.
With a focus on freshman and new students, the first finding correlated student acquisition of common set of competencies for undergraduate study. Takeaway: students receiving information literacy instruction perform better than those who do not.
By analyzing multiple data points, including circulation usage, online database access, study room use, and other library services), the findings show that students who use the library in some way achieve higher levels of academic success, measured as GPA and retention, than those who did not use the library.
Librarian partnership with writing centers, academic enrichment and speech labs yield positive results for students – higher grades, academic confidence, retention.
Libraries improve general education outcomes and contribute to inquiry-based and problem solving learning, including critical thinking, ethical reasoning, global understanding and civic enrichment.
Additional findings with building evidence of positive library impact in other related areas, although they have not been studied as extensively or findings may not be as consistently strong.
Additional findings
National Education Statistics, ACRL, other entities encourage statistics taking that is resource focused
National center for educational statistics data tracked.
Use data to identify trends and redirect resources, activity and programs. This is an internal assessment that is critical to ensure programs are aligned with instruction. However, it doesn’t tell the college the impact of these activities.
U of Minnesota library investigated the impact of library usage on retention and academic success of first year students. Library usage statistics and statistics about students who engaged with library staff through instruction sessions and reference interactions were gathered for a semester (fall 2011) for thirteen library services and access points.
Use data to identify trends and redirect resources, activity and programs. This is an internal assessment that is critical to ensure programs are aligned with instruction. However, it doesn’t tell the college the impact of these activities.
The online reservation service creates an in-person faculty-student instructional relationship.
89% of those who met with a librarian received an A grade, whereas in the group of those students who did not meet with a librarian, 56% received an A.
In the group that did not meet with the librarian, 44% of students received grades lower than the lowest grade earned in the group of students who met with a librarian.
Quality contact matters. Make it easy, make it convenient.
In a study of textbook usage, the average GPA of 3988 students (a subset of the total) was 2.83. The Fall 2015 GPA was 2.64.
In a study of textbook usage, the average GPA of 3988 students (a subset of the total) was 2.83. The Fall 2015 GPA was 2.64.
Although the BOG has made student equity planning a minimum standard for receipt of state funding since 1996 and has long recognized the importance of student equity, until the passage of the Student Success Act of 2012, student equity was not tied to any categorical program and did not receive formal funding through the legislative budget process. In January of 2014, in recognition and support of the importance of the need to identify and support equity and success for all students, the governor’s 2014-15 budget proposed to target $100 million of additional Student Success and Support Program (SSSP) funding to close achievement gaps in access and success in underrepresented student groups, as identified in local student equity plans.
Financial aid in form of BOG waiver. BOG waiver eligible students may be:
Low income households
Receiving Aid to Families with Dependent Children
Supplemental Social Security Income/State Supplementary Program or
Those who qualify for financial aid through other identified need-based profiles
27,308 questions from students needing general help navigating college in 2014/15.
Separate from the 17,496 Reference and Research questions our librarians answered. These 27,000 questions represent students needing direct assistance in everything from help finding rooms on campus to emergency need for calculators and computer access to a crash course in where to buy a bluebook when the bookstore is closed.
Outcomes: Consistently accessible and responsible staff; Efficient, convenient, and responsive libraries
The libraries give vital support to students who are unfamiliar with the academic environment. The library is often the only service consistently open with faculty available to students during the day, nights, and weekends.
The library supports student equity by reaching a higher concentration of underrepresented students than exists in the college population mix.
How are we responding with technology to support these populations?
Back to the underpreparedness. SRJC is just above the state average for completion. 2014/15 academic scorecared: 54.1% completion.
“California’s Student Success Scorecard shows a stark divide between college prepared and unprepared students. Statewide, more than 70% of incoming students are required to enroll in one or more remedial courses.”
There are some great research support tools offered by vendors, such as ProQuest research companion.
Their success will depend on their fluency in using the tools of the academic library.
In California, 70% to 85% of students who are new to the community college system are assessed as needing basic skills courses in one or more subject areas. Student Research Context: Ill-prepared, Limited Exposure Source: Johnson, Hans and Marisol Cuellar Mejia. Online Learning and Student Outcomes in California’s Community Colleges. Public Policy Institute of California, p.9. May 2014.
Their success will depend on their fluency in using the tools of the academic library.
In California, 70% to 85% of students who are new to the community college system are assessed as needing basic skills courses in one or more subject areas. Student Research Context: Ill-prepared, Limited ExposureSource: Johnson, Hans and Marisol Cuellar Mejia. Online Learning and Student Outcomes in California’s Community Colleges. Public Policy Institute of California, p.9. May 2014.
Share Course QuickLooks
Each video is integrated into the campus online orientations. Reference minute videos also done. Other ideas: library modules in Canvas, integration of library services into the online environment
Additional strategies that positively influence retention that were identified in the ACRL report included undergraduate research, capstone courses, common intellectual experiences, and learning communities.
Another finding of the recent ACRL report was the correlation of the library’s role in GE curriculum to success. Information literacy instruction advances aspects of inquiry based learning, problem solving learning, including critical thinking, ethical reasoning, global understanding, and civic engagement.
What can we show without the benefit if formal longitudinal studies tracking students under conformed consent conditions?
The information we need to proactively provide to offset current assumptions about libraries needs to be targeted to many key groups. We need to report data to institution decision makers and to legislators that directly link library value to known outcomes. Switching from simply reporting in state and federal documents to creating intentional marketing material. Processing print isn’t something the human brain was built for…Mother Nature has built into our brain our ability to see the visual world and interpret it.”
Students rely on campus libraries for convenient, responsive services that allow them to succeed.
Shape ideas through several communication exposures over a short period of time.
Libraries: make a brilliant annual report that translates library programs and services to institutional goals.
Add frequent communications about library relevance delivered through direct engagement with campus colleagues.
