Simon Bates' slides from the LTKB workshop in 2011 - looking at peer assessment of presentations - and reviewing Barwell and Walker's 2009 paper.

1. Physics Education Research
The University of Edinburgh
PeerWise in Physics 1A Semester 1 2010
Simon Bates s.p.bates@ed.ac.uk
1

2. Physics Education Research
The University of Edinburgh
Motivation – why
bother?
Implementation – what
was done and how?
Suggestions for local
implementation
2

3. Physics Education Research
The University of Edinburgh
Motivation – why bother?
Richer student-generated feedback on presentations
Student engagement with presentations other than their
own
Need to be time efficient
Little previous use of this application of EVS
3

4. Physics Education Research
The University of Edinburgh
Implementation – class details
Hons (year 3) Media and Eng Lit class
No previous use of EVS handsets
3 sessions with seminar groups of 15-20 students
Student assessment of their peers was not summative
4

5. Physics Education Research
The University of Edinburgh
Implementation – assessment details
Instructor assessed summatively – content, delivery,
timing visuals
Student assessment questions criteria determined
through discussion with previous year’s class
Based on tutor Qs but adapted for student perspective
Most Qs scored on numerical scale of 1-10
5

6. Physics Education Research
The University of Edinburgh
Implementation – marks
6

7. Physics Education Research
The University of Edinburgh
Implementation – marks
Student marks comparable
But all very high with not much
differentiation
Ranges : 71-90 (students) 72-89 (tutor)
Means identical to nearest %
Interesting case is student 6 – misinterpreted question but lively
presentation (low tutor score, high peer score)
7

8. Physics Education Research
The University of Edinburgh
Implementation – student feedback
Anonymity of handsets appreciated
Engagement - more attentive to session
Assessment over several sessions – ‘fixing’
Serial Q responses – slow
10Qs per student – ‘button fatigue’
Unanimous that it should not be summative
8

9. Physics Education Research
The University of Edinburgh
Evaluation – my thoughts
An interesting study in an appropriate area
Achieved some of its aims
Were I to do it, I would do things slightly differently
9

10. Physics Education Research
The University of Edinburgh
1. Can’t assume students are confident
assessors, partly because they rarely
assess
Need to run a dummy exercise as a training
session
Illustrate with short examples of ‘staged’ good,
average and poor presentations
10

11. Physics Education Research
The University of Edinburgh
2. Students should feel some ownership of
the peer assessment component
Develop assessment criteria and questions with
whole class discussion input (this year’s class!)
Trial with training exercise presentations
Discuss and if necessary refine
11

12. Physics Education Research
The University of Edinburgh
3. Simplify the questions / choices
10 point scale is far too fine
Suggest replacing it with statements rated on a
5 point Likert scale (or 6 point Osgood)
Reduce the number of questions to the
minimum needed to cover all the assessment
issues
12

13. Physics Education Research
The University of Edinburgh
4. Consider letting students assess different
things to tutor
If similar criteria assessing same components of
presentation, won’t you always expect similar
scores?
Why not let students assess suitability of e.g.
content, subject to development of criteria
through training?
13

14. Physics Education Research
The University of Edinburgh
Consequences
+ Students as partners in the development of
PA criteria
+ Normalisation of expectations ahead of real
session: ‘on the same page’
+ Encourages reflection before students create
and deliver their own presentations
- Takes 1-2 hrs additional class time
(but probably time well spent)
14

15. Physics Education Research
The University of Edinburgh
Finally…..
I think you could allocate a proportion of the
summative assessment to this process, if
criteria transparent and students trained as
suggested.
15

16. Physics Education Research
The University of Edinburgh
16