4. Outline
• Value of a Library
– Personal value
• Direct measures
• Indirect measures
– Organizational value
• Academic libraries
• Public libraries
• National libraries
– Financial impacts (Afternoon))
5. Few libraries exist in a
vacuum, accountable only to themselves.
There is thus always a larger
context for accessing library
quality, that is, what and how well does
the library contribute to achieving the
overall goals of the parent constituencies?
Sarah Pritchard
6. There is no systematic evidence
collected which shows the value
of academic libraries
for teaching and research staff.
Claire Creaser and Valerie Spezi
7. Performance Measures
Library Individual Society
Services
Input Proces Output Outcome Outcome
s s s
Resource Capability Use Beneficial effects
s
Efficiency Effectivenes
s
Cost Impact
Effectiveness
Cost benefit
VALUE
8. Start with the end in mind:
work backwards
Refocus from the activity
to the impact
10. Library Control
How much? How many? How economical? How prompt?
Magnitude Magnitude Resources used Cycle times
% of change last Change Units processed Turnaround time
year
Anticipatory
% of overall change
Cost
11. Library & Customers Decide
How valuable? How reliable? How accurate?
Effort expended Dependability Completeness
Cost Access Comprehensiveness
Benefits obtained Accuracy Currency
12. Customers Decide
How well? How courteous? How responsive? How satisfied?
Accuracy Attentive Anticipatory Expectations met
Promptness Welcoming Helpful Materials obtained
Courtesy Empathetic Personal interaction
Expertise Ease of use
Environment
Comfort
Willingness to
return
14. Challenges
Lack of consensus about what should be
measured and how
Lack of understanding of performance
measurement and metrics
Organizational structural issues
Lack of precision in measuring performance,
and
alignment issues
Determining the “bottom line” is too far away
Majority of stakeholders are too far away
Library staff find it difficult to see the “big”
16. Lack of a Connection
• Budget and outputs (and outcomes) are
separated
• No “bottom line” measure for libraries
• Decision-making process is bigger than
the library
• Library has neither champions nor foes
• Library benefits are not widely self-evident
17. Orr’s Fundamental Questions
• How good is the library?
• What good does the library do?
• How well is the library managed?
18. We should be a bit wary of the “little library”
…For when it is good, it is very, very good
and when it is bad,
it’s a “pretty good library for a town this size.”
Eleanor Jo Rodger
19.
20. Levels of Assessment …
• Individual student
• Course
• Departmental/Program
• College or University
21. Types of Measures
• Direct
– Provide tangible, visible and
self-explanatory evidence of
what students have & have not
learned
• Indirect
– Capture students’ perceptions of
their knowledge & skills;
supplement direct measures;
sometimes called surrogates
22. Qualitative Tools
• Focus groups – open
ended
• Biography
• Phenomenology –
capture the “Aha!”
moment
• Grounded theory
• Ethnography
• Case study
23. Qualitative Assessment
• Provides in-depth understanding of user
responses and interactions
• Represents part of a long-term strategy of
formative evaluative
24. Quantitative Tools
• Surveys
• Transaction logs
• Statistics from systems
• Observations (count)
25. Quantitative Assessment
• Analyses to determine library impacts on
academic performance, retention rates
• Describe retention rates and GPAs in
defined populations over semesters and
users
• Compare users & non-users of library
services while adjusting for academic
preparation and background differences
• Conduct quasi-experimental designs
employing multivariate analysis of
covariance & hierarchical regression
28. The Issue
• Is it: Use library resources/services and
you will get better grades.
• Or: I want to do well and so I work hard to
achieve better grades (and one way I do
that is to use library resources/services).
29. “Not surprisingly, librarians are keen to show
that the use of expensive, scholarly
materials
positively correlates with higher grades,
although they cannot prove that this is so.”
Deborah Goodall & David Pattern
30. “There is growing pressure on all academic
library managers to be more accountable for
how they use limited resources and to
achieve institutional outcomes perceived as
important by college and university
stakeholders….”
Elizabeth Mezick
31.
32.
33.
34. Value of Information
• Expect value-in-use
• Library’s collection reflects a “potential
value”
• Collection also reflects a “future value”
• Value of local collection is declining
35. Valuable is not about our professional
values;
in the paradigm of the value of public
libraries,
we are the producers,
not the consumers of services.
Our sense of what is valuable
really doesn’t matter
much at all unless it
matches that our our customers.
36. Fundamental Changes
Libraries have changed more in the past two
decades than in the prior two centuries.
Technology is the major driver . . .
We need to recognize that all this change
has only begun, and that change is
irreversible.
37. Increasingly it is important
to remember that libraries
provide few unique services.
39. Quality of Information
This fast food approach to information consumption drives
librarians crazy. “Our information is healthier and tastes
better
too” they shout. But nobody listens. We’re too busy
Googling.”
Peter Morville
40. Key Characteristics of
Information
Uncertainty Knowledge
Ambiguity Indeterminacy
Redundancy System dependency
Sharing Timeliness
Compression Presentation
Stability Multiple life cycles
Leakability Substitutability
41. Criteria for Judging Value
Customer Criteria Value Added by the Service
Ease of use Browsing, formatting, mediation service,
orientation service, ordering, physical
accessibility
Noise reduction Access (item identification, subject
description, subject summary), linkage,
precision, selectivity
Quality Accuracy, comprehensiveness, currency,
reliability, validity
Adaptability Closeness to problem, flexibility,
simplicity, stimulatory
Time savings Response speed
Cost savings Cost savings
44. Nature of Information is
Changing Information
Information
was …. is ….
Scare, controlled All around us
Expensive Cheap or free
Shaped by elites Shaped by consumers
One-way, mass Designed for sharing,
consumption participation &
feedback
Slow moving
Immediate
External to our
worlds Embedded in our
worlds
47. If the physical proximity of
print collections had a demonstrable
impact on researcher productivity,
no university would hesitate to
allocate prime real estate
to library stacks.
49. Universities Provide
• Private goods &
services
– Courses exchanged for
tuition
– Research completed for
funding
• The value proposition
The value to an individual or an
organization determines
50. Academic Libraries Provide
• Public goods and
services
Print and online resources are
shared by all, usually without
the exchange of payment
• Value proposition
The collective value of all users
must be estimated to determine
if a good or service should be
52. Astin’s IEO Model
Programs
Institutional
characteristic
Input s Output
Library
Fellow students
Faculty
Place of residence
Environment
53. Student Learning Outcomes
Model
Abstract,
Process Intelligence
Oriented
General Reasoning
Broad Abilities
Knowledge, Understanding, and
Reasoning
Concrete,
Content-
Oriented
54. Define, develop, and measure
outcomes
that contribute to
institutional effectiveness
ACRL Standards for Libraries in Higher Education
55. Challenge
It is not how much a library
is used that matters,
rather how does the library
impact or benefit the
customer?
56. Perspectives on Value
Impacts
Direct Personal
Organizational
Use
Indirect Financial
Benefits
Option – Preservation of option for
future use by me
Nonuse Existence – Perceived value and
significance
to the community
Legacy – Value of preservation for
future generations
58. Why Use the Library?
Reasons Interactions Results
For a TASK Access COGNITIVE results
RESOURCES
For PERSONAL AFFECTIVE results
reasons Use of
RESOURCES or ACCOMPLISHMENTS
To get an SERVICES
OBJECT or EXPECTATIONS met
INFORMATION OPERATIONS
TIME aspects
To perform an ENVIRONMENT
ACTIVITY S MONEY estimates
61. Student Learning is Affected by
…
• Full-time students
• Live on campus
• Interact more with faculty
• Study more
• Collaborate with their
peers
62. NSSE
• 5 benchmarks of effective educational practice
– Level of academic challenge
– Active & collaborative learning
– Student-Faculty interaction
– Enriching educational experiences
– Supportive campus environment
• Student self-reported gains in intellectual &
personal development
• No overlap between self-reported data &
standardized objective tests
63. NSSE – Use of the Academic
Library
• 50% never used the library
• Use of libraries at small, academically
challenging liberal arts colleges are
correlated with other purposeful activities
• Library use less intensive at larger
universities
• Students who work harder use library
64.
65. Assessing Student Achievement
• Direct measures
– Capstone experience
– Use of a portfolio
– Standardized exam (Collegiate
Learning Assessment)
• Indirect measures
66. Assessment of Higher Ed
• Gains in student
performance are
quite low
• Individual learning
is characterized
by persistence
• Notable variation
within and across
institutions
67. Wabash National Study
• Different instrument – CAAP
• 2,212 students
• Nearly identical results to Academically
Adrift
• 44 percent no gains in the first year
• 33 percent no gains in 4 years
• Students only study about 15 hours per
week
69. Bibliographic Instruction
• Improvement in basic library skills is the
means and not the end
• Yet the means is the focus for evaluation
efforts
• Evaluation efforts focus on
– Opinion surveys
– Skills improvement
– Pre-test & post-test knowledge
• Not the impact on student achievement
70. Library experiences do not seem to
directly
contribute to gains in information
literacy,
to what students gain overall in college,
or
to student satisfaction.
Kuh & Gonyea
71. “One way to demonstrate the library’s
contribution is to assess whether students’
experiences with the library
directly or indirectly
contribute to desired outcomes of college.”
George D. Kuh & Robert M. Gonyea
72.
73. How to Demonstrate Impact in
…
• Student enrollment
• Student experiences
• Student learning
• Student grades (GPA) &
achievement
• Student retention & graduation
• Student career success
• Faculty productivity
• Institutional reputation
• The environment
78. Meta-analysis
• Entering student characteristics
– SES
– High school GPA
– ACT/SAT scores
• Environment - Psychosocial and study
skill factors
– Academic goals, skills and self-confidence
– Social support & engagement (acculturation)
79. Student Learning Occurs …
• in the classroom
• in the laboratory
• with peers
• in the student union
• in the dorm
• in the library (for some)
• online
• and
80.
81. Direct Measures
• Capstone Experience
• Portfolios
• Standardized Test
• Locally-developed
test
92. University of Minnesota
Gym Bags and Mortarboards
Use Campus Recreational Facilities
At least 25 times, first-
year retention increased
1%
&
5-year graduation rates
increased 2%
93.
94. University of Minnesota Library
• 5,368 first-year non-transfer students
• Use of library was associated with a .23
increase in students GPA
• More use of the library, GPA also goes up
96. Library Instruction and GPA
• Surveys of student opinions & habits
• Assessing student work for specific skills
• Analysis of grade point average
• Mixed results
97. Library Instruction and GPA
Hong King Baptist University
• 45 sample groups – N=31 to 1,223, study
majors
• Pairs of data
• One-fourth (11) had a positive relationship
• Results:
– 1 or 2 workshops – little impact on GPA
– 3 or 4 workshops – ½ show a positive impact
– 5 workshops (1 sample group) – 100% had a
higher GPA
98. Library Instruction and GPA
University of Wyoming Libraries
• Analysis of 4,489 transcripts
• Slight positive relationship between upper-
level library instruction courses and GPA –
0.075 GPA difference – that’s less than
1/10th of 1 percent
99. • Research statement – 44%
• Evaluate Web site – objectivity –
52%
– authority – 65%
• Presentation to persuade – 12%
103. Retention Concepts
• Institutional retention
– Enrolling & graduating from the same
institution
• Program retention
– Enrolling & graduating with the same
major/department/school
• System retention
– Students who leave one university yet
continue and complete post-secondary
studies elsewhere
104. Measures of Retention
• Persistence (Continuation rate)
– From first to second year? Entry to
graduation?
• Completion rate
– From entry to graduation (Student goals?)
• Graduation rates
– Are transfers included? Time period?
• Attrition
– Leaving university? Leaving higher ed?
105. Measures of Retention
• Stopout
– Leave university with the intention (and
action) of returning later to complete a
program
• Dropout
– Leave university with intention (and action) of
NOT returning
• Transfer
– Change institutions yet persist in higher
education
– May change type of institution
106. Why Students Leave?
• Students’ decision to leave University is influenced
by many personal factors
– Financial reasons
– Family responsibilities
– Lack of academic ability
– Poor fit, etc.
• Foundational Theories from Education / Psychology:
– Tinto’s “Model of Student Integration”
– Bean’s “Model of Student Attrition”
107. Tinto’s
Model of Student Integration
Pre-entry Goals / Institutional Integration Goals / Outcome
Attributes Commitment Experience Commitment
s s s
Academic System
Performance Academic
SES
Integration
Intentions Engagement Intentions
Stay
Skills & Goal or
Abilities
Institutional Leave
Extra- Institutional
Commitment Curricular Match
Quality s Social
of Peer group Integration
Educatio
n Social System
108. Bean’s
Model of Student Attrition
Loyalty Attitudes Practical
Certainty Value
Personal Organization
Variables al Variables
Grades
Courses
Educational Goals
Intent
Major & Job Certainty
Environment
Opportunity to Transfer
Variables
Dropout
Family Approval
al
109.
110. Student Retention & Graduation
• Important because … rankings,
revenues, educational achievement,
emotional well-being
• Many reasons for drop-outs are not
under the control of the university
• Engagement is the key
111. Indicators
• Student goal • Transfer rate & success
attainment • Employer assessment
• Course retention • Academic value add
• Subsequent course • Student satisfaction
work • Professional growth
• Fall-to-fall persistence • Student involvement
• Time to degree • Citizenship &
• Degree completion engagement
• Grad school
enrollment
117. Library Retention Studies
• Statistically significant relationships between
library expenditures, or staffing levels and
student retention
E.g. Hiscock, 1986
Hamrick, Schuh, & Shelley, 2004
Mezick, 2007
118. Graduation Rates & Library
Expenditures
• Used IPEDS data on institutional
characteristics & resource allocations
• Library expenditures was strongly
correlated with graduation rates – 1.77
percent increase in graduation rates
• Greatest payoff is attributable to enhanced
library expenditures (+0.92) and instruction
(+0.80) while increased non-library
contributions were quite modest (+0.27)
Hamrick, Schuh & Shelley
119. Library Retention Studies
• Relationships between library use
(collections) and student retention
– Student who borrowed books = more likely to
persist
• E.g. Kramer & Kramer, 1968
• Impact of instruction
– Students involved in library skills programs
showed lower attrition rates
• E.g. Knapp, 1966
120. Library Retention Studies
University of Minnesota
– 77% of undergrads made use of the
libraries, 85% of grad students made use of
the libraries
– Students who used the library at least once
were 1.54 times more likely to re-enroll
121. Library Retention Studies
• Some library involvement in first year
experience programs; specific programs for
“at risk” groups
– NOT proven to have significant effect
• E.g. Hollis, 2001
Colton, et al, 2002
Aguilar & Keating, 2009
Love, 2009
122. Library Retention Studies
• Relationship between library employment
& retention
– Higher completion rate among library student
workers
• E.g. Wilder, 1990
Rushing & Poole, 2002
123. “If strong linkages between libraries
and student retention can be made,
then the perceived value of the library
may indeed rise.”
Steven Bell
124. • Some groups, some majors & seniors
engage in more library-related activities
• Academic support expenditures tend to
correlate with increased engagement
• Institutional academic challenge
correlates with library use
128. Faculty Teaching
• Integration of library resources and
services into course
syllabi, Websites, lectures, labs, reserve
readings, etc.
• Faculty/librarian collaborations;
cooperative curriculum, assignment, or
assessment design
129. Perceived Benefits for Teaching
• Savings
– Of own time
– Of own money
– Of other resources
• Improvements
– Teaching
– Course-related materials
– Student performance
130.
131. Impact on Faculty
• Library is the source for most journal
articles (individual subscriptions are way
down)
• If library subscriptions were unavailable –
productivity would decrease 17%
• Library is not the source of book readings
• 42% of reading material is library provided
132. Time
• Academics spend a lot of time reading
• Article reading inspires new thinking,
improved results, changed focus
• Award-winning academics read more
• Academics who publish more use more
library resources
133. Ithaka Studies
• Library services not
understood
• Library services not valued
• The Library is
disappearing
134. Faculty Research Productivity
• Number of publications, number of
patents, value of technology transfer
• Tenure/promotion judgments
Faculty Grants
• Number of grant proposals (funded or
unfunded)
• Value of grants funded
135. Assessment of Research
• Payback model – form of ROI
• Research impact
• Research utilization ladder
• Lavis decision-making impact model
• Weiss logic model
• HTA organization assessment
framework
• Societal impact framework
• Research assessment exercise
• Becker medical library model
136. For Most Impact Models
• Indicators of research output
• Indicators of knowledge transfer
• Indicators of implementation
• Indicators of community benefit
140. Institutional Reputation
• Changes in reputational rankings affects
student & faculty recruitment
• University budget allocations to libraries
have decreased
Since the library absorbs a very small percentage of a
university budget, the contribution of the library is
disproportionately high relative to its cost to the institution.
Sharon Weiner
141. University & the Library Can
• Attract outstanding faculty
• Retain outstanding faculty
• Foster innovative research
• Align library activities with
university goals
142. • Indispensable for their research
• Maintain a high-level overview of their field
• Value for money is good
• Library not available, costs would increase
40%
• Take 31% longer to locate same information
143. Value of Special Libraries
• Time saved
• Money saved
• New revenues
• Other outcomes
144. In addition, to ROI
• Knowledge-Value Add
• Intranet Team Forums
• Intellectual Capital
Valuation
145. Measurements of value
were, in fact, a
key differentiator
between
successful
and unsuccessful corporate libraries
James Matarazzo
149. Libraries/Building/Communities
• Developing social capital
– Providing a welcoming environment
– Creating a pride of place
– Attracting users from all walks of life
– Reaching out to the community
– Appreciation of cultural differences
– Building bridges to government
– Encouraging collaboration across the
community
150. Libraries/Building/Communities
• Overcoming the digital divide
– Making technology accessible
– Exploiting technology to benefit the
community
• Creating informed communities
– Community information
– Government information
– Providing a gateway to the world of
information
151. Libraries/Building/Communities
• Convenient and comfortable places of
learning
– Developing information skills
– Stimulating ideas and discussion
– Supporting vulnerable learners
– Supporting students
152. Outcomes
Enhanced quality of life
Enhanced enjoyment from
hobbies
Able to obtain information
Facilitates lifelong learning
Support for children’s education
153. Contributions
Safe and pleasant place
Supporting educational facilities
Facilitating lifelong learning
Encouraging responsible social
behavior
Ensuring access to the Internet
154. Social Benefits
• Basic reading literacy • Local history &
genealogy
• Business/career
• Health & well-being
• Information literacy
• Social cohesion
• Library as place
• General information
• Summer reading
• Empowerment
155. Making Cities Stronger
• Building a stronger local economy
• Improving early literacy & school
readiness
• Building workforce participation
• Supporting small business
• The power of place
156.
157. Tracking Value
The Engaged Library:
Chicago Stories of Community Building
•Prove that public libraries build social capital
•Identify & connect the library’s assets to the
community
•Assess & strengthen the library’s connections with
and use of community assets
•Produce a toolkit for other libraries to adopt to
•Mapping tools to perform an inventory services,
identify areas for improvement and highlight library’s
contribution to the community’s wider social,
educational, cultural and economic goals.
158.
159. PLQIM
• Access to information
• Community & personal participation
• Meeting readers’ needs
• Learners’ experiences
• Ethos & values
• Organization & use of resources
• Leadership
160.
161. Valuing the Collection
Dewey Subclass 2010-2011 Avg
Number of Titles List Total Value ($)
001 - Knowledge $76.71 $0.00
002 - The book $62.45 $0.00
003 - Systems $129.77 $0.00
004 - Data processing. Computer science $89.82 $0.00
005 - Computer programming, programs, data $69.14 $0.00
006 - Special computer methods $83.60 $0.00
010 - Bibliography $73.65 $0.00
011 - Bibliographies $69.46 $0.00
012 - Bibliographies of individuals $0.00 $0.00
013 - Of works by specific classes of authors $0.00 $0.00
014 - Of anonymous and pseudonymous works $55.95 $0.00
015 - Of works from specific places $184.99 $0.00
016 - Of works on specific subjects $134.87 $0.00
017 - General subject catalogs $0.00 $0.00
018 - Catalogs arranged by author, date, etc. $0.00 $0.00
019 - Dictionary catalogs $0.00 $0.00
020 - Library and information sciences $56.06 $0.00
021 - Library relationships $62.60 $0.00
022 - Administration of the physical plant $65.50 $0.00
023 - Personnel administration $56.00 $0.00
025 - Library operations $77.02 $0.00
026 - Libraries for specific subjects $89.99 $0.00
027 - General libraries $66.25 $0.00
164. What Are the Results
Four Year
College
Full-time College degree – 19%
Four Year Graduate degree –
College Stopout/Transfer 11%
Part-time
AA degree – 8%
Two Year College
No college No college degree –
60%
Editor's Notes
You can download one and buy the other
Claire Creaser and Valerie Spezi. Working Together. June 2012. UK: Loughborough University.Libraries are busy measuring activity and not measuring value
Richard Orr
Focus on outcomes not process
Not all impacts are positiveNot all impacts are intendedNot all impacts are immediateDifficulty in separating library impact from other influences
The How Questions
Allan Pratt and Ellen Altman. Live by the Numbers, Die by the Numbers
2/3rds of managers responsible for library budgets have no idea of how to evaluate or value the library
How good is the library? Quality and capabilitiesWhat good does the library do? Benefits, Impacts - ValueHow well is the library managed? Efficiency - benchmarkingHow can we move from bad to good?
New Zealand Libraries, March 1990
What are some of the Tools for Assessment - Evaluation
Direct method – the financial savings method is used widely to monetize the direct use benefits of public libraries.As such, it undercounts benefits that have had an estimated value assigned to it Not so in academic libraries
Focus groups – open ended – standard, guided, exploratoryEthnography – work study, photo essays, mapping dairies, etc.University of Rochester –they “know” their customers
Survey - random sample best, set questions (no follow up), statistical analysis, representative sample, response ratesExplore - How was it?, What do you do?, What do you want?, What did you do?
Deborah Goodall & David Pattern . Academic library non/low use and undergraduate student achievement. Library Management, 32 (3), 2011, 159-170.p. 161
DefinitionsA nounExchange for or equivalenceMonetary or material worthUsefulness, utilityPrinciple, standard, or qualityToll, cost or priceDarkness or lightness of color A verbEstimate the worth of something (appraise)Regard highly (esteem)Assign a value to something Other definitions depending on the fieldRegardless of context, defining value is a complex issue with its own philosophical discipline; axiology (Cram, 1999, p.11). Axiology, or Value Theory defines three different dimensions of value; extrinsic value, systematic value and intrinsic value (Hartman, 1969, p. 114). Thus, there are a number of different value types, including personal value, aesthetic value, religious value, spiritual value; and ethical value. Bequest value – willingness to pay for the endowment of the good or service for future generationsGoogle images - add_value.jpgonproductmanagement.net
Adam SmithValue-in-exchangeValue-in-use (utility theory)Google image adam‑smith.jpgblog.braintraffic.com
Value is a moving targetWhat was valuable vs.What is valuable vs.What will be valuable
Eleanor Jo Rodger. Value & Vision. American Libraries, November 2002, 50-52.Ideas of value have changedMoving target, constantly needs assessmentUsefulness, quality, availability, imageHistoric is not valuableDoesn't’t correspond to staff ideas of importanceIs not about our professional valuesDoing wrong things well does not create value
Mobile technology is the needle, and social networks are the thread (materials being woven)
Libraries are so screwed Value of the local collections is being diminished
Douglas Badenoch et al
Robert Taylor
http://matthew.reidsrow.com/As a result Discovery happens elsewhere
Hope College Library
A large local inventory was a hallmark of academic reputationWe no longer live in that world.
Photo Flickr.com401K College
Flickr Timtom.ch Trinity College Library
Two different people receive different value when using the same item
Similar to Robert Orr’s Input-Process-Output-Outcomes modelImpact of any one characteristic of the University environment is clearly, at best, indirectInput are the entering student characteristicsOutput are the graduating student characteristicsSimilar to Orr’s Input, Process, Output, Process model
Richard Shavelson. Measuring College Learning Responsibility: Accountability in a New Era. Stanford, CA: Stanford University Press, 2010, pg 13
Libraries building communities – 75% of users and nonusers agreed that the library is a good place for facilitating social interaction
Personal Perspectives
TefkoSaracevic and Paul Kantor
Gates Foundation
Organizational Perspective
NSSE results
Pascarella, Seifert, and Blaich. How Effective are the NSSE Benchmarks in Predicting Important Educational Outcomes? Change, January 2010.NSSE data is reliable yet there are still concerns about use of the data
N=380,000 NSSE dataGeorge Kuh and Robert Gonyea. The Role of the Academic Library in Promoting Student Engagement in Leaning. College & Research Libraries, July 2003, 256-282.
The Halo EffectGary Pike – The Constant Error of the Halo – For freshmen – halo error accounts for half of the explained varianceFor seniors – one quarter to one half of the explained varianceSelf-reported gains in student achievement and college experiencesPhil Rosenzweig – The Halo Effect – Jim Collins Good to Great
Richard Arum and JosipaRoksa. Academically Adrift: Limited Learning on College Campuses. Chicago: University of Chicago Press, 2011.Almost half (45%) showed no improvement – using CLA before and after data – after 2 years35% showed no improvement over 4 yearsLiberal arts majors do better than other majorsTo gauge summative performance authentically, the CLA presents realistic problems that require students to analyze complex materials and determine the relevance to the task and credibility. Students' written responses to the tasks are evaluated to assess their abilities to think critically, reason analytically, solve problems and communicate clearly and cogently. Scores are aggregated to the institutional level to provide a signal to the institution about how their students as a whole are performing. Whether CLA actually measures what it says it measures is another matter
Pascarella et al. How Robust Are the Findings of Academically Adrift? Change, May-June 2011CAAP - Collegiate Assessment of Academic Proficiency
Megan Oakleaf – RAILS Project
Wong Chan and Chu. JAL, July 2006More recently,Reinsfelder. Citation Analysis as a Tool to Measure the Impact of Individual Research Consultants. C&RL, May 2012, 263-277. Found that as number of sources improved, the grade improved yet other research found either no correlations or negative correlations.
Kuh & Gonyea. The role of the academic library in promoting student engagement in learning.College & Research Libraries, 64 (7), July 2003, 256-82.300,000 student respondents – NSSE data
Sept 2010
Megan’s categories – all her suggestions rely on indirect measures or surrogates
One study – Gary Reynolds. The Impact of Facilities on Recruitment & Retention of Students. New Directions for Institutional Research, 135, Fall 2007.– showed that the library was the 2nd or 3rd most important reason why a university was selectedPartnering with Student Affairs – campus tour offices – Connecting with parentsMore recently, Lombard The Role of the Academic Library in College Choice. The Journal of Academic Librarianship, July 2012, 237-41.62% lib. no factor, & another 29% the lib. Was only a minor factor. 9% either an important factor or deciding factorGoogle backtoschool_XS.jpgusd261.com
ACRL Value Report possible surrogate measuresChoice of college (from the students perspective) has a big impact on how long it takes to graduate with a BA, onbeing accepted into a high quality grad schoolAs well as amount of income over your lifetime. Private vs public
Google C09LRN1.jpgcsuchico.edu
ACRL Value Report possible surrogate measures
Acculturation – learning how to function (and succeed) in their new environmentTechnology & bureaucracy are the biggest problems
So assessment of student learning is difficult So too is the assessment of the library’s contribution to student learning
Google 123RF Portrait of happy young people sitting in pub, drinking beer, looking at camera, smiling.
Standardized test – Collegiate Assessment of Academic Proficiency (CAAP)Measure of Academic Proficiency and Progress (MAPP)the Collegiate Leaning Assessment (CLA) test has three components:Make an argumentCritique an argumentPerformance task – prepare a briefing reportEssays are scored using a rubricThe institution is the primary unit of analysis
Derek Rodriquez – PhD uses rubrics to assess the impact of student capstone projects
De Jager 2002, Zhong & Alexander 2007, Julien & Boon 2004, Wong & Webb 2011Jim Self 1987 no correlation between use of reserve collections and gradesLibrary-related interactions – Dickensen 2006Behaviors – Poll 2003, Poll & Payner 2006
Good newsUK Library Impact Data Project
UK Library Impact Data Project
Book borrowing by students at Huddersfield University in the UK
University of WollongongData into the Library Cube – R squared = .91The Library Cube provides the information needed to support continuous improvement in three areas: collection development; academic relationships; and marketing.The Library has seen a positive correlation between borrowing activity and academic performance
Univ of Wollongong Aus School of CommerceAs use of library increases, grades go up
Reported in College & Research Libraries
Small study but interesting results
Regression Analysis -
led to $59 million expansion
Hong King Baptist University – Wong and Cmor CR&L Sept 2011
Melissa Bowles-Terry. Library Instruction and Academic Success: A Mixed-Methods Assessment of a Library Instruction Program. Evidence Based Library and Information Practice, 7(1), 2012.
6,300 students in the sample ICT skills 2007
Many confusing terms to describe the same thingRetention rates range from 60 to 80%Do you know the retention rate for your university?
ACRL Value Report surrogate measures
In the 2005 book “College Student Retention” edited by Alan Seidman, Linda Hagedornhas a great chapter that explains the differences between various student retention concepts
Defined by US National Center for Education Statistics institutionsretain; individualspersistPersistence: from entry to graduation? From first to second year?Completion: from entry to grad? What about student goals upon entering?Graduation rates: time period? What about transfers? Can only really belong to one institution – so transfer school X captures the student in their graduation rates; beginning school Y calls the student a non-persister or a dropoutAttrition: leaving the system? Leaving the institution?So just a few more things to think about when you are exploring the student retention literature.
Retention and drop out rates are NOT dichotomous – students might leave during one time period, but then come backStopout; Drop out; TransferVoluntary / Involuntary – how is this (IS this?) accounted for in student retention stats?Non-persistence is not always a bad thing. Retention theorist Alexander Astin believes student GOALS coming in, and INTENTIONS are most significant. (e.g. did that student only intend to pick up one language course at the community college? Did that student travelling far from home only intend to spend one year at the institution for the experience before transferring to a more affordable institution?
On-campus undergraduate students
On-campus undergraduate students
Carroll et al – graduate distant education student Retention Model
Develop a personal connection with the university – especially in the first yearGlasgow Caledonia Univ – highest retention and fastest progression rates among students who have high use of eResources Crawford et al 2004Social integration, academic integrationTemple University – Reasons to drop outLost financial aid/change in financial situation reason for dropping outAlternative textbook project – eBooks
Association for the Study of Higher Ed (ASHE) report
2008 AUSSE data – more you use the library more likely to NOT consider leavingDeparture intention“How libraries and librarians can support student engagement”
Curtin University – sample of 4,461 students66% had NOT borrowed an item were more likely to withdrawThose that HAD accessed eResources were more likely to remain in schoolStudents with low SES DO use computer workstations in the library
Early warning system! In the UK Compare Current vs Dropout
Engagement is particularly important for the first-year student
Why – data is easily accessibleJane E Hiscock, 1986Long Island UniversityLibrary Expenditures increased retentionHamrick, Schuh, and Shelley, 2004Iowa State University Predicting graduation rates based on resource allocation and other institutional characteristics.Found library expenditures (not counting materials) significantly related to graduation ratesCAUSAL RELATIONSHIP: very difficult to prove!! Focus instead on showing statistically significant differences.Mezick, 2007Long Island University (Brookville, NY)Analyze relationships to both library expenditures and number of professional library staff to student persistence. Strongest relationships:Library expenditures (overall) and retention at undergraduate institutionsLibrary professional staff and retention at doctoral institutions
Lloyd A. Kramer & Martha B. Kramer, 1968California State Polytechnic CollegeStatistically significant correlation between library use (in terms of book borrowing) and persistence from 1st to 2nd year<CLICK>Patricia B. Knapp, 1966Monteith College (Wayne State University, MI)Students involved in library skills program showed lower attrition, higher grades
Large body of work that supports the idea that campus jobs generally promote persistence – Astin, TerenziniStanley Wilder, 1990; Darla Rushing & Deborah Poole, 2002Louisiana State University & Loyola University, New OrleansRelationship between employment in the library and student completionWill Weston San Diego State Univ 2010
The problem however is that almost all studies make the library the focus of the investigation and not student retention (and student success)
NSSE - Students tend to over-report their experiences and their skills
ACRL Value Report surrogate measures
What is the impact of the library on the faculty?Flickr AKMA Seabury Faculty
Creaser & Spezi. Working Together: Evolving value for academic libraries. June 2012.
Tenopir and Valentine 2012Back files are important
Time spent reading represents an exchange value – about 25 days per yearCarol Tenopir ALA Midwinter 2012
Ithaka plus Cluff & Murrah 1987 JALFlickr Ben Heine We All Disappear Someday
ACRL Value Report surrogate measures
Creaser & Spezi. Working Together: Evolving value for academic libraries. June 2012.For researchers - Help with lit searching – huge opportunity
Jason Priem – Univ of North Carolina – Chapel Hill Altmetrics– use social media
US News & World ReportStudents from better schools get into better graduate schoolsWeiner JAL Jan 2009 Total library expenditure is related to US News Peer Assessment ScoresBudgets down from 3.7% to 2.5% over 10 years
Work together to …
AustraliaLibrary provides access to information resources that are:
Outsell report 2007Avg time saved per interaction – 9.5 hours (35% of all library interactions)Money saved – projects + centralized purchasing to reduce duplication – Avg $3,000 per interaction (20% of lib interactions) Revenue generated - $7,000 per interaction (19% of all lib interactions)Decisions supported – 68% of the respondentsConfidentiality – in-house serviceLibrary as a revenue generatorContribution to decision makingImproved productivityImproving qualityFacilitating communication (honest broker)Role in risk managementTimeliness of information
Frank Portugal. Valuing Information Intangibles. Washington, DC: Special Libraries Association, 2000.Knowledge-Value Add – helps managers create value. Uses a surrogate measure for intangible value to determine how a sub-process contributes to the final product or service (time required to learn a sub-process, number of words needed to describe a sub-process). Value is determined by assessing the cost of each sub-process.Intranet Team Forums – The use of forum and chat software – track how information evolves into knowledge and gets incorporated in products and services. Estimate information and knowledge costs and compare to expected revenues (cost savings). Identify library’s contribution.Intellectual Capital Valuation – Measures are developed for four areas:Customer FocusProcess FocusRenewal & Development FocusHuman Focus
The Value of Corporate Libraries
Public Library Assets
Australian project
Top 5 OutcomesAverage time less than 30 minutes. Attracted by the collections.
Top 5 Contributions2008. Average time less than 30 minutes. Attracted by the collections and services.
Urban Libraries Council. Making Cities Stronger: Public Library Contributions to Local Economic Development, 2007.
Urban Libraries Council 2006represent a step towards a more ethnography based approach to measuring the value of public libraries and digital services.
Scotland<CLICK>
6 point scale 1 = inadequate, 6 = excellentEthos & values = high quality services to allLeadership – innovative & entrepreneurial
List your take aways
LC
85% of US population HS grads; 50+% some college1980 data AA = 6%; College degree = 19% no degree – 75%2010 data AA = 8%, College degree – 19%,