8. THE SOLUTION
Low-stakes,
Online,
Repeatable
Learning Checks
9. WHY?
• Online assessment
• K-12, and other standardized tests already online
(Bennett, 2002)
• Easier access for students (Brooks, Nolan, &
Gallagher, 2003)
• Can be used to enhances self-regulated learning (Nicol &
Milligan, 2006)
• Can be used to remediate deficiencies, automate homework
grading, and guide study time toward problem solving
(Toback, Mershin, & Novikova, 2005)
10. WHY?
• Repeatable quizzing
• Keller Plans (1968)
• Personalized Systems of Instruction = UDL (Brothen &
Wambach, 2003)
• Found most valuable tool in online microeconomics course (Brown
& Liedholm, 2004)
• Encourages mastery learning (Brooks, Nolan, & Gallagher, 2003)
• Mastery learning extremely successful (Bloom, 1976)
• Formative assessment with feedback promotes student learning
(Nicol & Milligan, 2006)
• Shown to aid in decreasing DFW rates (Twigg, 2004)
11. DECISIONS TO MAKE
• Give correct answer or not?
• With or without feedback?
• Graded or ungraded?
• Timed or untimed?
• Number of questions each time?
• Limited or unlimited number of attempts?
12. STEPS
1. Build item pool
2. Enter item pool into Blackboard
3. Set up repeatable quiz parameters on
Blackboard
Let Blackboard to the rest!
13. SIDE NOTE…
Most instructors and test bank writers do NOT write good
test questions
&
Many students are test savvy
14. SIDE NOTE…
• Some hints for writing good test questions
• Align the questions with learning outcomes
• Match the question type to the Bloom‟s taxonomy
level
• Avoid using negatives in the question
• Avoid giving information in the stem that can give
the answer to another question
• Avoid clues in the stem (e.g., grammatical)
15. SIDE NOTE…
• General test question patterns that test savvy students
pick up on
• The middle answer (or C) is usually correct
• The longer answer is usually correct
• The „different‟ answer is usually correct
• “All of the above” or “None of the above” is usually
the correct answer
16. LESSONS LEARNED
• Use Respondus to forbid screen capturing
• Shorten time for quizzes
• Limit number of attempts
• Larger item pools are better
• Simple item analysis limitations
• Scores positively correlated with effort
(Johnson, Joyce, & Sen, 2002)