How to Improve Your Library Instruction: Assessment in Five Minutes (November 2017)
How to Improve Your
Assessment in Five Minutes
November 15, 2017
During Today’s Session, We Will…
• Review the purpose and benefits of quick
assessment in one-shot instruction sessions.
• Draft learning outcomes for a session.
• Compose assessment instruments based on
our learning outcomes.
• Select appropriate assessment tools based on
your instruction goals, time, and population.
What Can Quick Classroom
• It can identify instructional gaps or disconnects.
• It can help you determine how you spend class
• It can build your confidence.
• It can provide evidence of efficacy and impact.
• It can inspire and motivate your learners.
• It can (& should) be a basis for change.
What Will We Assess?
• To assess:
– Comprehension of integral concepts
– Achievement of learning outcomes
• Not to assess:
– Your personal shortcomings
– Your speaking mistakes
– Your hairstyle
Have you ever written learning
objectives or outcomes?
• Nope, not yet
• Yes, one or two times
• Yes, regularly
• I’m not sure
Outcome Versus Objective
“A Learning Outcome is a statement of what
the student should understand and be able to
do as a result of what she has learned ... ‘the
essential and enduring knowledge, abilities,
and attitudes or dispositions’ that enable a
learner to practice and apply her learning in
the real world.”
-Valencia Community College
• Global: large, meant to excite and inspire long-
• Educational: mid-level, meant to describe how a
student will interact with a topic or content.
• Instructional: specific, narrow, focus on day-to-day
“slices of learning”
-Anderson & Krathwohl
A Good Learning Outcome Will…
• Identify the audience.
• Set a time frame and a context.
• Be jargon-free.
• Be measurable.
• Be action-oriented.
• Be brief.
• Be linked to learner needs.
• Be appropriately narrow.
Determine the Class Priorities
Choose two to five.
– What must the students accomplish?
– What must they comprehend in order
to accomplish it?
– What is likely to trip them up?
– What do they already know/find
Add Product or Outcome
What do the students need?
• A thesis statement
• Keyword search strategies (basic or advanced)
• Understanding of the research process
• Ability to critically analyze a source
What is the context?
• A class assignment
• A real-world scenario
All Together Now!
Stem + Verb + Product/Outcome
By the end of this class, you will be ready to
construct a thesis statement for your paper.
This session will equip you with the tools you
need to critically analyze whether or not a
source is credible.
By the end of class, you will be able locate
scholarly articles on your topic and will
choose to seek help from your librarian, if
What’s Wrong Here?
Today I’ll talk about…
1. The library website.
2. Database searching for peer-reviewed
journals using boolean logic and the
subject heading thesaurus.
3. Where you click to search the catalog,
search the databases, and place an ILL
request, and every step you’ll take to
accomplish these processes.
Have you ever assessed
student comprehension or
learning in class?
• Nope, not yet
• Yes, but not regularly
• Yes, regularly
• I’m not sure
• Convert your learning outcomes to
• Limit the number of questions.
• Eliminate overly easy or “all of the
• Avoid nebulous or complicated
• Work in terminology to ensure it’s
Formative or Diagnostic
• Interactive exercises or games
• Interactive exercises or games
• Minute papers & muddiest point
By the end of this class, you will be
able to construct a flexible and usable
thesis statement for your paper.
1. Have them identify/list the qualities of a useful
2. Have them look at some sample thesis
statements and consider whether they are too
broad or too narrow.
3. Have them draft a thesis statement. Ask them
to think about how they might broaden or narrow
it, and/or submit synonym ideas.
• How many of you have had a library
instruction class here before?
• Is this source peer-reviewed? / Is this
source scholarly or popular?
• Who is the author of this book?
• Are you confused about topic x?
• Write learning outcomes.
• Base your assessment questions on
the outcomes and on student needs.
• Review and reflect on the answers
you get in class with the students,
• Ambrose, Susan et al. How Learning Works: Seven Research-Based Principles for Smart
Teaching. Jossey-Bass, 2010.
• Anderson, Lorin W. and David R. Krathwohl. A Taxonomy for Learning, Teaching, and
Assessin: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman,
• Bloom et al.’s Taxonomy of the Cognitive Domain.
• Cahoy, Ellyssa Stern, and Robert Schroeder. “Embedding Affective Learning Outcomes in
Library Instruction.” Communications in Information Literacy, 2012. 6(1): 73-90.
• Clay, Ben. Is This a Trick Question? A Short Guide to Writing Effective Test Questions.
• Cornell University Center for Teaching Innovation, “Setting Learning Outcomes.”
• Oakleaf, Megan. “A Roadmap for Assessing Student Learning Using the New Framework for
Information Literacy for Higher Education.” In The Journal of Academic Librarianship, 40.5,
• The Teaching Center, “Asking Questions to Improve Learning.”
• Utah State University Library, “Assessment.”
Hinweis der Redaktion
Choose your questions and model based on the outcomes you need to meet.
Everyone has been watching the new Information Literacy framework develop. I agree with Megan Oakleaf that the framework doesn’t overthrow the importance of learning outcomes, though it might shift the way you were writing them if you were already doing them.
We will incorporate the framework’s tenets today and talk about how you can assess the threshold concepts that they’re comprised of.
Example: terminology—article, journal, database
This type of assessment is not an end in itself—it supports change and best practice in teaching.
Did they take away what you thought they were going to take away? If not, why? Threshold concepts: “core tenets in a particular discipline that are transformative, irreversible, integrative, bounded, and potentially troublesome” MEYER and LAND
They tend to be highly constructive
In many cases, these words are used interchangably. Some people are moving in favor of Outcomes over objectives, because the term objective has become muddled over the years, so that now, objectives are sometimes written as teacher-centric instead of user-centric.
That means that the language is geared toward what I will do (as the teacher), rather than what you will be able to do (at the end of the class session).
Our first step in good, quick assessment is to write learning outcomes. The assessment questions which you choose will, in general relate specifically to those learning outcomes. They will also provide a roadmap for you, as the instructor, in terms of what you must cover. Whatever assessment method you choose later will tie back to these.
A good learning outcome will state what the student will be able to do, and WHY.
First step: consider the top three or four class priorities—two to five on the outside. Write them down. If you are going to do some basic things then you can have more, like four or five. If you want to start mixing in larger concepts—things that are likely to trip them up and require some deep thought to understand--then you may find that you can’t fit in as many—but also, you may find that the way you teach is transformed, and that those more specific pieces aren’t necessary.
Think more broadly than “they will know which ten clicks will lead them to a perfectly refined search. Rather, help them understand information types and why they care about them in the first place. Don’t be afraid to focus on “Big ideas and enduring understandings” Do they understand why they would use a scholarly article search versus a Google search? Do they understand why the scholarly conversation is critical? Can they identify a scholarly article and its parts?
Learning outcomes should stem from a needs assessment. The needs assessment will determine the gap between the students’ existing condition and a desired condition—that condition is usually an increase in knowledge, skills, or attitudes.
Remember that these are just a pithy starting place for you. One outcome may take almost the full hour to unpack.
Future tense—timeframe Identifies learner
Benjamin Bloom, an American educational psychologist, helped to write these in 1956. Domains of learning. This chart represents the updated version. Cognitive domain.
Factual knowledge: bits of information Conceptual knowledge: more complex, organized knowledge forms Procedural knowledge: knowledge of how to do something Metacognitive knowledge: “knowledge about cognition in general as well as awareness of and knowledge about one’s cognition.” page 27, Anderson and Krathwohl-- can take place in all, but they chose to separate.
This should include context as often as possible without being too wordy or jargon-filled.
So, for example, in this last I’ve said “advanced techniques” rather than saying boolean logic, subject header searches. You may want to cover those things, but probably not in your outcomes.
You don’t have to have the context, but I like it. If you don’t include it in your written outcomes, still think about it and be able to articulate the context to the students.
Stem and outcome Demonstrable/assessable Bloom’s cognitive domain—high level No jargon
Often INCREMENTAL—step-based. They fit together to make a whole. They also, if you show them, provide a sort of roadmap for the students. Giving them a meaningful outline of the topics you’re going to cover is shown to help students engage with the class more thoroughly.
Let me again address the threshold concept piece of this. I hope it’s clear from each of these how they can be jumping off points for in-depth and transformative conversations, rather than point and click sessions. You can use them as a basis for teaching them the WHYs and WHENs of applying these skills, rather than just the HOWs.
Learner not identified—teacher focus Not assessable Poor verb No verb Jargon, too specific HUGE in scope
We’re going to talk about each and which might be best for you, given your setting.
Polls are good for feedback throughout the session--this kind of assessment is called formative assessment OR just at the end, for summative assessment.
Polls are great if you have information that builds and is cumulative. So, you can get them to answer a question or two before you move on to the next topic. They’re also a fun active learning exercise and they give you immediate feedback.
The questions have to be simpler, with no short answer or essay, but you can have more of them.
You can set them up to be embedded on a site/LibGuide (if your students have computers), or use free tools to set them up as text messages. can take those polls and embed them into PowerPoints, and then the answers will show up there (PollEverywhere). They can also just raise their hands, of course!
Regardless of what method you choose to deliver the content (phone, embedded poll, clicker) you can have different question types. Multiple choice, yes/no.
These probably seem basic, and that’s because they are, but they are often the basis for everything you’re trying to share with your class. If they have a shaky foundation, it means you can’t build a house on it and your efforts will be a failure, so these aren’t always exciting, but they’re critical.
Substantive or friendly—can also help you to set the stage for what you’ll cover in the beginning. I know these aren’t hifalutin, but it’s hard to build on a shaky foundation. These can be your foundation questions.
These can be great, because they are anonymous in the moment. So, if someone feels shy about raising a hand to say s/he is confused, this can help them to do it without drawing attention to themselves.
These can get into those higher level thinking skills and get students to really demonstrate understanding, application, evaluation, etc.
You can position these however you like in your class time. I used to deliver them about ten minutes from the end of class, give the students five minutes, and then use the last five minutes to go over the answers quickly and take one last stab at fixing any confusion issues that arise.
You may want to have them answer the questions right when they come in, then discuss what they wrote as a group. It’s a great way both for you to get a sense of where the students are from the start, and to help them apply your teaching to something they have written in front of them.
You can also do a pre and post questionnaire.
With both quizzes and polls, you can easily convert your learning outcomes to question, if they’re carefully drafted. So, “You will know how to construct a search statement becomes “construct a search statement.”
Not more than five “meat” questions, maybe a couple of close out questions that are a bit more free-form.
The intent of this first question is to learn what they’re anxious about and what I should focus on. The second one is to hear their thoughts on why the library is important. Often they come up with abstractions like “my teacher says I have to” and that can actually be a fabulous jumping off point for a discussion about source credibility. The third one tends to go much the same—they say things like “I click the peer-reviewed button.”
When covering these, I’m always careful not to shut them down when their answers are off target—rather, I talk about how we are now operating in a different framework—the research framework—and that that is different from the open web.
The fourth one has a specific function, and that is to ask them to apply what we are about to talk about to something already in their minds. This kind of anchoring technique can help them to relate what you’re saying to their reality,a nd therefore, find more meaning in it.
Pre and post, or just post. Pre and post can gauge learning, post can gauge what they’re walking out with. If you can get teacher cooperation, you can also do them again as a follow up, to gauge retention on various time scales.
Some of these are open ended, some multiple choice. It’s a good idea to have a variety. Not too many easy ones, or you may not learn anything useful.
You can also format these as quick assignments. Great for classes with one set task—evaluate and cite websites, for example.
You can also treat these like scavenger hunts, if you have enough time, and ask the students to get up and locate things.
You don’t need to be perfect your first time. If your questions aren’t perfect the first time or your instruction isnt perfect, that’s okay
You will realize even as you get more accomplished, that you sometimes don’t say things perfectly! You can correct that on the spot.
You can use answers as an opportunity to congratulate them for their awesomeness.
What is a minute paper? You have them write or type for just one minute—only two real questions (and maybe some basic information). On paper or online is your choice. It’s easier to tabulate things online though.
Drawbacks: It’s hard to evaluate these in the moment, but you can.
It’s also very hard to tie these in with your objectives—however, they are useful. I like to do a combination of hand-raising polling, with a post quiz that has a minute paper attached.
Gives them space to clarify and explain, if they want to. You can also invite them to leave contact info.
With polls and quizzes, if they’re brief, it’s easy to tabulate and review responses immediately. I like to do this. Skim over the answers and address problems right there. “It seems like I was unclear on Point X. Can someone describe the answer?
HAVE YOUR ELEVATOR SPEECHES READY.
This is the most important step. If you don’t integrate the information, it’s not useful to anyone.
We’re not perfect—this kind of thing can be jarring at first.
Change can relate to the way the questions are worded, ordered, or what is included, and also in the ways you teach.
Create a warm and welcoming environment by walking around in the beginning of class and during. Insure the students that the process is difficult and messy, and the assessments are intended to insure that they have what they need and have a chance to ask questions while you’re there to help.
Let them know the things you learn from their struggles help you to improve the classes for future students. Appeal to altruism=awesome.
Most of these can be used for both.
Basic: free 10 questions per survey 100 responses per survey No fancy things like skip logic
Select: $26 per month ($312 per year) Unlimited questions Unlimited responses Custom urls Skip logic etc.
Can handle short answer, you can embed images, and it does quizzes and surveys in addition to polls.
If you’re asking what the difference is between a quiz and a survey in terms of this software, if you structure your setup as a quiz, you can set it to know the correct answers in advance.
Free account: Used to be limited to 100 responses per month, but they took that away. So, the free version should be everything you want unless you want custom urls.
This is usable with any computer, tablet, or phone—even old ones, and it can handle short comment answers. You can embed in PowerPoint.
Free version 25 responses per poll or 40 responses per poll if you are in K through 12 or higher education. They don’t love public libraries, apparently.
The pricing beyond that point gets kind of complicated based on your situation, but it’s $14 per month and up.
Free, but somewhat clunky to use.
List of verbs: http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html
Offenbar haben Sie einen Ad-Blocker installiert. Wenn Sie SlideShare auf die Whitelist für Ihren Werbeblocker setzen, helfen Sie unserer Gemeinschaft von Inhaltserstellern.
Sie hassen Werbung?
Wir haben unsere Datenschutzbestimmungen aktualisiert.
Wir haben unsere Datenschutzbestimmungen aktualisiert, um den neuen globalen Regeln zum Thema Datenschutzbestimmungen gerecht zu werden und dir einen Einblick in die begrenzten Möglichkeiten zu geben, wie wir deine Daten nutzen.
Die Einzelheiten findest du unten. Indem du sie akzeptierst, erklärst du dich mit den aktualisierten Datenschutzbestimmungen einverstanden.