Preliminary findings on a mixed-methods study.

1. Exploring
Formative E-Assessment in
Leicester Medical School
Terese Bird
PedR Group 4th April 2016

2. Context
• Summative examinations in Medicine
• Every semester
• Integrated
• One Single Best Answer (SBA)
• One Short Answer Question (SAQ)
• Previous semesters’ formatives
• Document of questions distributed on Blackboard
• Face-to-face discussion

3. Groupwork sessions
• Students divided into groups
• Workbooks with questions
• Answers given the week after or never
• Searching for the answer is the point
• Facilitated by junior doctors

4. Problems
• NSS feedback scores
• Students don’t know where they are
• Tutors don’t know where students are

5. Student study issues
• Lecture notes + workbook “answers” = holy canon of notes on iPad
• Holdover from coursework
• Memorise the form of words for SAQs
• Writing notes is good but can’t write it all
• Screenshotting is enough = having it in my notes is enough = having the right
answer is enough
• “Revision” happens in the weeks before exams
• Not much day by day learning
• Not much verbalising or writing of what they know
• And if they did, how would we give feedback?

6. Laurillard’s Conversational Framework
Laurillard’s Conversational Framework (Laurillard, 2008) Dashed line box added by this author to show
segment relevant to e-assessment of this study

7. ExamSoft £30 / student / year

8. Examsoft formatives GI Unit Autumn 2015
Exam Type When Content Circumstance
Session-by-session
exams
Weekly, from Oct
through Dec 2015
Material studied in
the past week
Exam done in
groupwork class
sessions, not
under exam
conditions
Christmas 1
Christmas 2
New Year 1
New Year 2
During Christmas
break Dec ember
2015–Jan 2016
All questions from
session-by-session
exams, put
together and
randomised
Four separate
exams with the
same exam
content, offered 4
times throughout
Christmas break
for self-study
whenever students
wanted.

9. Instant answers with additional information

10. Students can log into portal to check progress

13. How did the formatives change students’
study if at all?
• “A mini-revision before the proper revision” – forced earlier study
• Uncovered trouble spots which caused them to seek solutions
• Back-and-forth communication
• Active study

14. Thinking and learning on higher levels of
Bloom’s Taxonomy
• If questions were written higher on Bloom’s, students learnt higher
on Bloom’s – constructive alignment
• Single Best Answer leads to Evaluation and Analysis
• Students reacted with Create to using exams in revision
• Created document of info on uncovered trouble spot
• Created answers when they tested themselves without the multiple-choice

15. “It’s not clear that I performed
better on summatives because
of these exams, as the
summatives are integrated, but
they helped me to learn GI
better.”
“It was a nice way of doing the
workbook questions in
groupwork”

16. What next?
• Cost of ExamSoft against other costs – Top Hat
• ExamSoft is assessment
• TopHat is interactive live learning
• Cost of time to create questions
• Consider summatives – tech and staff issues