1. Information Literacy
Assessment
Challenges & Opportunities for
One-shot Library Instruction
Presented by
Diana Myers Hyatt
October 23, 2012

2. Introduction
Examine assessment process and
best practices for assessing LI sessions
Planning for learning and assessment,
from the program to the session level
Specific assessment tools suitable for
sessions, their strengths and weaknesses
Specific examples of incorporating
assessment
How can we leverage limited resources for
maximum learning impact?

3. Challenges
Challenges of assessing in one-shot instruction
sessions
 Limited class time. How do you effectively measure
learning in such a short time?
 Librarian staffing and workload
 Limited librarian training in teaching and
assessment
 Limited support from faculty/college administration
 UCR trends? Budget cuts?

4. Why assess?
 Improve instructional programs and teaching
methods to better meet learning goals.
 Improve student learning. Gives them a chance to
review learning goals, see their progress and what
they still need to learn.
 Part of accountability, meeting department and
institutional learning goals.
 Demonstrate value of library with evidence.

5. What is assessment?
 Assessment to improve learning:
How well are students learning what we intend them to learn?
Measurement of student learning based on established learning outcomes.
 Continuous, cyclical process of incremental improvement and change
Assessment Primer: How to do
assessment, University of Connecticut
www.assessment.uconn.edu/primer/how1.html

6. Assessment Best Practices (ACRL)
Program evaluation:
 Develops a process for program planning, evaluation, and revision.
 Measures the progress of meeting the program’s goals and objectives
 Integrates with course and curriculum assessment, institutional evaluations and
regional/professional accreditation initiatives.
 Uses appropriate assessment/evaluation method for relevant purposes, for
example formative and summative and/or short-term and longitudinal.
Student outcomes:
 Acknowledge differences in learning and teaching styles in the outcome
measures.
 Multiple methods: needs assessment, pre-tests, post-tests, portfolio assessment,
oral defense, quizzes, direct observation, peer and self review, and experience.
 Focus on learner performance, knowledge acquisition, and attitude appraisal.
 Assess the learners’ process and product.
 Include learner-, peer-, and self-evaluation.
Characteristics of Programs of Information Literacy that Illustrate Best Practices: A Guideline
http://www.ala.org/acrl/standards/characteristics

7. Assessment Best Practices
Assessment program should:
 be continuous: drive change and innovation in an
ongoing, iterative process, pinpointing areas for
improvement and development
 be learner-centered: focus on student learning rather
than on us
 measure what students are learning as accurately and
directly as possible
 learning goals should be flexible enough to be adaptable
to specific courses, integrating current program goals
(implicit and explicit) with ACRL standards
 When possible, allow for immediate teaching
improvement by informing teaching librarian of current
student skill levels through assessment opportunities
prior to the workshop

8. Strategic plan for learning and
assessment
 Moving from the general level to the specific (session level)
 Library develops main learning goals integrating university
and library missions, ACRL standards and input from
librarians and faculty
 From these goals, articulate specific learning outcomes which
can be assessed and measured
◦ target groups, scaffolding (e.g. first-year English, graduate
art students)
◦ Delivery vehicles (LI sessions, online tutorials, embedded
teaching)
 Determine which specific learning outcomes are best
addressed in LI sessions.

9. Mapping learning outcomes to
specific courses
Oakleaf, M. (2009). WRITING
INFORMATION LITERACY
ASSESSMENT PLANS: A guide to best
practice. Communications in Information
Literacy, 3(2), 80-90.

10. Learning plan for LI sessions
Learning outcomes for specific LI sessions
based on:
 Library-level goals for information literacy
 Learning goals of the specific course and
professor. Instructor collaboration, course
syllabus and library curriculum map
 Students’ current course assignments and IL
mastery level. Assignment description, pre-
session assessment (assignment or pre-test)

11. Assessment component of
learning plan
 A good plan gives teaching librarians time to
focus on teaching and engaging students,
because learning objectives and assessment
tools for each workshop are clear.
 Articulate what specific results will show that
students have mastered a learning outcome.
 How will assessment measures be gathered and
reported? How will the gathered data be reported
and shared?
 Assessment plan is reviewed on a regular basis.
 New assessment methods should be tested
before using on a widespread basis.

12. Assessment planning for LI sessions
 Rubric-based assessment plan combining learning outcomes paired
with appropriate assessment tools and scoring. Done collaboratively
within the library.
Learning outcome Assessment Acceptable
instrument performance criteria
Explores general Learning activity: Find Appropriate source,
information sources to and cite one reason given, proper
increase familiarity with appropriate article from citation format
the topic online subject
encyclopedia
 Assessment resources developed (test banks, learning activities
with worksheets, etc.)
 How to measure success. What scores will be considered
adequate?
 Teaching librarian now has appropriate learning outcomes and
ready-to-use assessment tools that can be easily adapted for
specific LI sessions.

13. Types of assessment instruments

14. Summary of assessment techniques
Technique Potential Strength Potential Weakness
Locally developed multiple Can provide direct evidence of Reliability and validity generally
choice tests student mastery of learning are unknown
objectives Creating effective exams requires
Appropriate mixes of items allow time and skill
faculty to address various types Traditional testing methods may
of learning objectives not provide authentic
Can be centrally developed and measurement
then adapted for specific
workshops
Easily scored and tabulated
Embedded assignment and Can provide direct evidence of Requires time to develop and
activities student mastery of learning coordinate and score, need
objectives simple grading rubrics
Involves active learning, possibly Requires faculty trust that the
more retention program will be assessed, not
Can provide authentic individual teachers
assessment of learning Reliability and validity generally
objectives are unknown
Data collection is unobtrusive to Norms generally are not available
students
University of Connecticut Assessment Primer, Assessment Primer, http://assessment.uconn.edu/primer/how1.html

15. Summary of assessment techniques
Technique Potential Strength Potential Weakness
Surveys, one-minute Are flexible in format and can include Provide indirect evidence about
papers, 1-2-3 mode, questions about many issues student learning
Survey monkey, clicker Can be administered to large groups of Validity depends on the quality of
technology, Poll respondents the questions and response
Anywhere options (avoid guessing)
Can easily assess the views of various
stakeholders What people say they do or know
Questions generally have a clear relationship may be inconsistent with what
to the objectives being assessed they actually do or know
Can be conducted relatively quickly and Open-ended responses can be
easily difficult and time-consuming to
analyze
Responses to closed-ended questions are
easy to tabulate and to report in tables or
graphs
Open-ended questions allow faculty to
uncover unanticipated results
Can be used to track opinions across time to
explore trends
Polls can be used for quick pre-tests with
instant feedback, encourage active
participation

16. Embedded performance-based
assessment
 Our assessment methods should support problem-
based and other active learning scenarios rather than
straight lectures.
 Combining active learning activities with assessment
 Could include collaborative learning exercises, creating
3-item bibliography (encyclopedia article, book, journal
article), other focused library activities with
worksheets, concept maps or other concrete product
that can be assessed -- better learning using
constructivist methods.
 Rubric for quantitative measurements. Or multiple
choice post-test afterwards for easier assessment.
 Trade-off: more complex learning, but harder to
measure in class.

17. LI session evaluation by students
 3-2-1 model using Google forms.
More structured than one-minute
paper.
 Indirect measure, but can reveal
important information. How well are
responses aligned with our teaching
goals?
 Gives students a chance to reflect
on and articulate what they’ve
learned for better retention
 Although open-ended questions
are harder to compile for statistical
purposes, they give better feedback
to librarians to improve instruction.
 Provides an opportunity for
personal follow-up with students
who want it by providing space for
questions and email info.
Nardine, J. and Meier, C. (2010), "Assessing the one-shot instruction session: Leveraging technology for optimum results", Virginia
Libraries, Vol. 56 No. 3, pp. 25-8

18. Assessment outside workshops
 Library/Faculty collaboration on class
assignments integrating IL learning
outcomes.
 Target key classes and assignments such as
English composition, introductory classes.
◦ bibliographies, annotated bibliographies
◦ research blogs/journals on their research
experience - very qualitative but could be eye-
opening.
◦ shared assignments with linked learning
outcomes
◦ students complete online IL module with tutorials,
activities and assessments.

19. Evaluating our assessment methods
 Is the method accurately measuring the
learning outcomes it’s supposed to?
 Is it valuable? Does it improve our
teaching?
 Does it put reasonable demands on our
staff and resources for the value it brings?
 Comparison of multiple forms of evidence
for the same learning outcome (including
class assignments)

20. Assessment method examples
ACRL standard 1.1: The information literate student
defines and articulates the need for
information.
 Small group activity: hypothetical information need situation
(buying a car, taking a trip), articulate the information need
and specifics of what information is needed, also brainstorm
possible sources (friends, consumer reports, review or travel
websites, travel guides).
Assessment: state general information need, 3 questions that
need to be answered, 3 possible sources in Google Docs
 Multiple choice post-test using Google form: what is the most
important first step in library research?
◦ What is the most important first step in library research?
a) identify key authors on research problem.
b) know research problem
c) Locate key articles on research problem
d) Locate key books on research problem

21.  Give them a broad topic such as “commuters and Internet crime”.
Have them come up with a good research question and identify keywords for
first stage of research. Could use Google spreadsheet or Google form. Could
use concept map or Wordle for keyword brainstorming.

22. Sample use of Google Docs for learning activity
Standard 1.1c Explores general information sources to increase familiarity with
the topic.
Performance based exercise: Given a topic by librarian, find an appropriate
encyclopedia article that gives background information on aspect of the topic and
create citation.
Answers cut and pasted into Google spreadsheet or fill out Google Docs form.
Shannon R. Simpson, Google Spreadsheets and real-time assessment: Instant feedback for library instruction, Coll. res. libr. news October 2012 73:528-549.

23. Reliability and Validity of
Assessment Methods
Reliability
 Consistent responses over time. Variation in scores is actual difference
in what is being measured, such as knowledge
 Students have adequate time to take assessment, guessing is
minimized
 Consistent methods for all sessions (e.g. same post-test questions).
Importance of collaboration and buy-in in developing assessment
methods.
 Must have consistent scoring or rating scale, so easy to score and
guidelines for scoring are clear
Validity
 The assessment tool measures what we want to measure?
 Do our results correlate with other proven measures that assess the
same learning outcome?
 Does our method accurately predict future student performance?

24. Conclusion
 No one best way to assess student learning. Use a
variety of methods, direct and indirect.
 Trade-offs between more thorough assessments and
time needed to do them. Have to prioritize learning
outcomes and make sure they are realistic given time
and staff constraints.
 Developing a “learning bank” of active learning activities
and specific assessment tools, such as a test bank, can
give teaching librarians more time to focus on improving
their teaching methods.
 Program-level learning plan helps us see the big
picture. Learning objectives in one LI session are
scaffolded with other sessions. Complement other
learning vehicles such as class assignments, tours.
 Use assessment methods that involve a learning
activity. Learning and assessment together.

25. “Good teaching is inseparable from
good assessing. The question,
therefore, is not whether to evaluate
students, but how to measure
performance in ways that will enrich
learning, rather than restrict it.”
Ernest L. Boyer, The Basic School: A Community for Learning (New York: The Carnegie Foundation for the
Advancement of Teaching, 1995), 104.