Presentation made by Lorna Dodd, User Services Manager, University College Dublin Library, at ANLTC Seminar "Library Impact and Assessment", held on Tuesday, 7th May 2013 at Trinity College Dublin Library.

1. Measuring the impact of
information literacy instruction:
A starting point for moving
forward
Lorna Dodd
User Services Manager, UCD Library
Lorna.dodd@ucd.ie
Twitter:@LornaDodd

2. Why?
…snapshot of impact to date & best
path forward

3. Looking back

4. Large University with no single strategy for
the development of Information
Literacy
Deliver a heavy load of information skills,
driven by individuals
Sometimes we evaluate but we rarely measure.
with no clear indication in meeting learning
outcomes
Looking back

5. Looking Forward
New Library Strategic Plan 2010-2014 aims to
“monitor, measure and evaluate the Library’s teaching and learning
strategy and activities.” (UCD Library, 2010)
As a result there has been a realignment
of staffing structure.
We have shifted from module to programme
approach
And from a responsive to consultative role
A key goal is to measure our impact on student
learning

6. Focus on process &
outputs
Limitations of traditional
approach in measuring
success
Important to use results of
any study which measures
the impact of Library
Service
Focus on Library use rather
than instruction
Common tools include
surveys, behavioural
observation and pre/post
tests
Literature Review

7. Challenges
Data protection rules
Measuring long-term impact
It takes time to measure impact of Library
Instruction

8. Promote new service
Identify what has been successful
Gather feedback from those not
engaged
Identify useful methods going
forward
Measuring impact retrospectively
Perceived impact is subjective
External factors influence skills
development
Opt for the familiar

9. Survey all module coordinators

10. Promotion

11. Survey
Results

12. Yes 43%
No 35%
Don’t know
22%
Has the Library ever delivered instruction in your module?

13. 35%
Lecture
33%
Workshop
25%
Tour
7%
Other
What kind of instruction did your students have?

14. Impact of Library Instruction

19. “Students have always
received instruction to
date and so difficult to answer the
comparison questions below”

20. Should ALL students have Library Instruction?
Yes
86% Don’t
Know
9%
No
5%

21. Should ALL students have Library Instruction?
Yes
96% Don’t
Know
3%
No
2%
Those who had Library Instruction

22. Should ALL students have Library Instruction?
Yes
77% N0
14%
Don’t
Know
10%
Those who did NOT have Library Instruction

23. Should Library Instruction happen in every year?

24. “Instruction in Year
one should be
built on in
subsequent years to
enhance
skills by
year”

25. Preferred Services

26. 120
Subject
specific
workshops
62
Library
Tours by
peer
mentors
58
Generic
workshops
50
Online
tutorials
48
Online
instructional
videos
45
Tutorials
First choice for undergraduates

27. 66
Generic
workshops
52
Tutorials
50
Online
instructional
videos
45
Online
tutorials
42
Subject
specific
workshops
36
Library
tours with
peer
mentors
Second choice for undergraduates

28. First choice for taught postgraduates
154
Subject
specific
workshops
62
Library
Tours by
peer
mentors
57
Generic
workshops
53
Online
instructional
videos
51
Online
tutorials

29. Influence of instruction
on preference?

30. Influence of instruction
on preference?

31. Influence of instruction
on preference?

32. “…create materials that are reusable and for
any student, regardless of level.
Just because a student moves up a level
doesn't mean they've acquired all of the
learning they should have”
“…really good online video
materials, available
on demand is the way
to go”.

33. Qualitative Feedback 20% of respondents

34. Themes
In line
with
strategy
Online
Options
Current &
Future
Services
Comments
on service
Comments
on survey

35. “…same 'introductory' talks
by librarian using lecture slots of
various modules
for at least three times
(the third time he skipped
as he found it
ridiculous)
“Students just skip the
guest lecture.”

36. “…requires personal
contact with a friendly
face early on.
“we really love our librarians; thank you”

37. “…it makes a noticeable difference in the
quality of assignment students produce. The
Library Tutorial is typically rated as one of the
most important lectures they have in that
Stage 1 module.”

38. Implications for
the Future

39. Impact of
Instruction
Definite benefit
in Library
instruction
Stronger
evidence in
discovery and
use of resources

40. Active learning
tailored to specific
subject needs
Tours perceived as
ineffective
Preferred delivery
of instruction

41. Asked to identify how
effective instruction was
at improving students’
information literacy
skills.
When they request
instruction, do they
usually consider this?
Questions

42. More Questions
Did respondents choose workshop
environment over online version due to
familiarity?
Then why did those who had no
instruction choose workshop?
If delivering workshops on this scale is
unsustainable,
What can we do ensure effectiveness?
What work do we need to do in terms
of marketing?

43. We have a shared understanding…
…how do we move forward?
Conclusion
Lorna.dodd@ucd.ie
@LornaDodd

44. Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1
impact projects’, Library & Information Research (LIR) vol. 29 (91)
Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address
sustainability in the delivery of information literacy components in UCD’: Librarians'
Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.
Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK
university libraries on student learning and research: A facilitated action research
approach’, Journal of Information Management vol. 26 pp. 3-15
Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’,
Library Hi Tech vol. 24 (4) pp. 547-562
Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a
comparison of the efficacy of commonly used measures in library instruction’, Journal
of the Medical Library Association, vol. 100 (4) pp. 258-269
Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report
on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22
UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’
http://www.ucd.ie/library/news_publicity/showcase_strategy/
Lorna Dodd: ANLTC 2013
References

45. Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/
why can't i breathe whenever i think about you? http://www.flickr.com/photos/breannajosephine/5098047755/
Looking Into the Past - Custom House http://www.flickr.com/photos/coreytempleton/5893927056/
To go to http://www.flickr.com/photos/rubensmj/5399226001/
Literature review http://www.flickr.com/photos/carowallis1/4297460415/
Nissan Qashqai Challenge London http://www.flickr.com/photos/parmiter/2521273890/
Playing chess http://www.flickr.com/photos/bburt33/5930699148/
Surveying for Central Station, 1900 http://www.flickr.com/photos/state-records-nsw/4010230662
Shouting kid http://www.flickr.com/photos/jamessarmiento/4353936770/
Results of elections http://www.flickr.com/photos/ntr23/4650249185/
Four stages of the moon http://www.flickr.com/photos/leehaywood/6607585765/
Meteor crater http://www.flickr.com/photos/homer4k/113472442/
Writing http://www.flickr.com/photos/jjpacres/3293117576/
Used books http://www.flickr.com/photos/babblingdweeb/23816128/
Quality Barnstorm http://commons.wikimedia.org/wiki/File:Quality_Barnstar_v2.0.png
Apples & Oranges - They Don't Compare http://www.flickr.com/photos/thebusybrain/2492945625/
Lorna Dodd: ANLTC 2013
Images

46. Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832
Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475
Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-
white-background-image28694065
One gooseberry fruits isolated http://www.dreamstime.com/stock-photography-one-gooseberry-fruits-
isolated-image20607952
Construction Tunnel http://www.flickr.com/photos/joshuarothhaas/2250657031 Online
http://www.flickr.com/photos/orkomedix/5398470372/
Conversation http://www.flickr.com/photos/stijnnieuwendijk/3098445189/
Backyard fresh http://www.flickr.com/photos/jim_belford/8544552591/
Abandoned amphitheatre http://www.flickr.com/photos/pixarman/5456444943/
Lego love http://www.flickr.com/photos/evaysucamara/5419530024/
Student working http://www.flickr.com/photos/92399131@N07/8399835572/
Future http://www.flickr.com/photos/maniya/4020026753/
Punch http://www.flickr.com/photos/timcaynes/148110505/
teach me http://www.flickr.com/photos/giovannamatarazzo/182108786
Question proposal http://www.flickr.com/photos/eleaf/2536358399/
Light Bulb http://www.flickr.com/photos/polifemus/561986434/
walk ahead and cross the sky http://www.flickr.com/photos/ldawei/4920681478/
Lorna Dodd: ANLTC 2013
Images