A presentation given to the 2009 Teaching and Learning Conference at London Metropolitan University, 7th July.

1. Practice makes perfect: Testing the testing
effect in a naturalistic setting
Dr David Hardman
School of Psychology
London Metropolitan University
d.hardman@londonmet.ac.uk
Twitter: @davidkhardman
Presentation given to the 2009 Teaching and Learning
Conference, London Metropolitan University, 7th July.

2. 100%
The forgetting curve
Initial learning
Schematic based on Ebbinghaus
(1885) and many subsequent
Amount remembered
studies
20 mins later
1 hour later
2 days later
One month later
0
0 Time

3. How can we improve long-term retention?
• Elaborative rehearsal (rather than maintenance
rehearsal).
• Self-reference effect.
•Distributed practice (not massed practice, or cramming).
• Testing, rather than rereading (after an initial period of
reading; Karpicke & Roediger, 2008).
BUT --- rereading is the most common revision strategy
(Karpicke et al, 2009) and students predict it will be more
successful than recall testing (Karpicke and Roediger,
2008).

4. Can we generalise from laboratory studies
to naturalistic settings?
Gurung & Daniel (2006)…
• Revision tests in unsupervised settings were associated
with poorer exam performance.
• Supervised tests were associated with better exam
performance.

5. Overview of the current studies
• Revision tests were presented at regular intervals in WebLearn (the
London met VLE).
• Students were informed about the benefits of revision testing and
warned against looking up the answers.
• On submitting a test, students received performance feedback: a score,
their answers, and the correct answers (if different).
• Tests were initially available for a limited period; but later made
permanently available.
• Engagement was monitored using the tracking function.
• Students completed a summative test at the end of the semester, and
an essay-based final exam about a month later.

6. Study 1. Autumn 2009.
Certificate level undergraduates taking Cognitive
Psychology 1

7. Study 1. Relationship between number of online revision tests taken and
score for the in-class test.

8. Study 1. Relationship between the summative MCQ and essay exams

9. Study 1. Two other potential correlates of MCQ test
performance (indirect measures of ability and motivation).

10. Regression analysis of data (1)
• Reading WebLearn discussions was not associated with MCQ test
performance.
•Together, numeracy and revision testing accounted for 38% of the variation in
summative MCQ test scores.
The standardized regression coefficients were:
• Numeracy. ß = .49 (p < .001)
• Revision tests. ß = .34 (p < .001)

11. Regression analysis of data (2)
• Neither WebLearn discussions nor numeracy were predictive of performance
on the essay-based exam.
• MCQ revision testing accounted for 9% of the variation in essay exam
performance, (p = .002), ß = .29

12. Study 2
• Spring semester: comparison of undergraduate
and graduate students.
• As before, MCQ revision was tracked through the
teaching period.
• At the end-of-teaching period, a month prior to the
final (essay) exam, paragraph questions were
added to WebLearn.

13. Study 2. The relationship between MCQ revision and
the summative MCQ test.

14. Study 2. The relationship between MCQ revision and
performance on the final essay exam

15. Study 2. The relationship between number of MCQ &
paragraph tests and essay exam performance.

16. Study 2. Statistical analyses.
Comparison of Undergraduate and Graduate students,
with revision testing as a covariate.
2. Summative MCQ test.
Both type of student and revision testing were significant
factors (ps < .001)
2. Summative essay exam.
Type of student was only marginally significant (p = .05)
Revision testing was significant (p = .001)

17. Conclusions
• Results are consistent with lab research into the
testing effect.
• Practice leads to better test results, even when
indirect measures of motivation/ability are taken into
account.
• Level of revision practice was the main feature
distinguishing undergraduates and graduate students.
Limitation
Unable to determine what students did “offline”

18. References
Ebbinghaus, H. (1885). Memory: A contribution to experimental
psychology. Translated by Henry A. Ruger & Clara E. Bussenius (1913).
Originally published in New York by Teachers College, Columbia
University. Available at:
http://psychclassics.yorku.ca/Ebbinghaus/index.htm
Gurung, R.A.R., and Daniel, D. (2006). Evidence-based pedagogy: Do text
based pedagogical features enhance student learning? In D.S. Dunn and
S.L. Chew (Eds.), Best practices for teaching introduction to psychology
(pp. 41-55). Mahwah, N.J.: Erlbaum.
Karpicke, J.D., & Roediger, H.L. (2008). The critical importance of retrieval
for learning. Science, 319, 966-968.
Karpicke, J.D., Butler, A.C., & Roediger, H.L. (2009). Metacognitive
strategies in student learning: Do students practice retrieval when they
study on their own? Memory, 17, 471-479.