Reflections on use of PeerMark to facilitate peer review. Given at eAssessment Scotland 2011. Dundee 26th Aug 2011.

1. Harnessing the Potential of PeerMark to Enhance and Enrich the Student Experience of Assessment and Feedback “Students don't just learn from instructors they also learn from one another. PeerMark facilitates peer review so that students can evaluate each other's work and learn from their classmates.” iParadigms Dr Neil McPherson Lecturer in Sociology School of Social Sciences University of the West of Scotland neil.mcpherson@uws.ac.uk @neilgmcpherson

2. Why peer review? “if students are to be prepared for learning throughout life they must be provided with opportunities to develop the capacity to regulate their own learning as they progress through higher education” (Nicol & MacFarlane-Dick 2006: 15) “Students need a sound working knowledge of three concepts in particular – task compliance, quality and criteria – if interactions between teachers and learners are to be formatively effective, and capability in complex appraisal is to be developed. These assessment concepts must be understood not as abstractions but as core concepts that are internalised, operationalised and applied to concrete productions” (Sadler 2010: 548)

3. Why peer review? “..if we want students to develop critical thinking, judgement and autonomy in assignment production they should be provided with high-level evaluative experiences similar to those of experts. Peer review, students evaluating and commenting on each other's work, is one way to achieve this” PEER

5. encouraging students to think critically and reflect on learning strategies and approaches to assessment through the consideration of the work of others

6. enabling students to self-assess and self-evaluate their work through the assessment and evaluation of the work of others

8. Setting up PeerMark? 3: Set no. of reviews (self & instructor reviews) 1: Identify document for peer-review 2: Identify timings & marks available – if any 4: Set questions for peer reviewers

9. The PeerMark workflow Instructor may also review/ comment at this point Students submit assessment online through Turnitin 5 1 Assessments distributed automatically for review Students revise assessment in light of reviews 2 6 Students carry out assigned reviews online Students submit revised assessment through Turnitin. Instructors mark final piece of work using defined rubric within GradeMark 3 On receipt of peer reviews, students carry out self review 7 4

10. Instructor reflections Benefits of employing PeerMark in peer review Harnesses the integration internal to the Turnitin suite of tools - single-screen navigation between tools for student & instructor Automated distribution of submissions for review Provides an accessible, safe & stable environment for students to undertake peer review System interface is unobtrusive, leaving students to concentrate on the process of peer review Facilitates a much deeper student engagement with and understanding of assessment criteria and marking rubric

12. Production of dialogue

13. Passive -> active engagement

14. Makes student more aware of academic expectations

15. Parallel development of disciplinary expertise and academic skills(see, e.g. Liu & Carless 2006; Nicol 2011, O'Brien, R. & L. Sparshatt; PEER 2011; REAP 2011; Rust, Price & O’Donovan 2003; Sadler 2010):

16. Student reflections Students involved in peer review – 10 (7 provided feedback) No issues highlighted in using PeerMark All agreed that system was straightforward to navigate - both in terms of carrying out peer & self review and accessing reviews “It was relatively straightforward, and quick to use” “easy to use” All agreed that undertaking peer review increased awareness of assessment criteria and benefited their own work – 6 agreed that it raised awareness of the assessment process in general

17. Student reflections “Sometimes pointing out flaws in others work highlights that these issues are also present in your own work” “It should be introduced at as early a stage as possible as it does benefit students’ work and understanding of assessment criteria” “[I] feel comfortable criticising peoples word if it's going to help them improve and being able to take criticism knowing it's going to improve my work” [Peer assessment] would help transferable skills going into the work place, as self evaluating and evaluating others work, can form part of building a strong team “It may often be the case in the workplace that you will have to give constructive criticism to fellow colleagues. Having taken part in such a peer review system at undergraduate level can only stand you in good stead for these types of situations in the future”

18. Student reflections Peer marking or peer review? Would students be comfortable awarding marks? Would students be comfortable having marks awarded?

20. enable and encourage the development of students’ potential for critical thinking, self-evaluation and self-regulation

21. facilitate the stimulation of metacognition in the the processes of assessment & feedback – thereby closing the gap between staff expectations and student understanding of those expectationsA tool with which to achieve many of the benefits of student engagement in the assessment, feedback & peer review processes highlighted in the contemporary literature

22. References Bloxam, S. & West, A. 2004. Understanding the rules of the game: marking peer assessment as a medium for developing students' conceptions of assessment. Assessment & Evaluation in Higher Education. 29;721-733. BoudD, & Associates. 2010. Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian Learning Council. Carless, D. 'Differing Perceptions in the Feedback Process'. Studies in Higher Education. 2006; 31: 219-233. Gielen, S., F. Dochy & P, Onghena. ‘An inventory of peer assessment diversity.’ Assessment & Evaluation in Higher Education. 2011; 36(2): 137-155. Lui, N. & D. Carless. ‘Peer feedback: the learning element of peer assessment.’ Teaching in Higher Education. 2006; 11(3): 279-290. Nicol, D. 2007. Reap: Re-Engineering Assessment Practices in Scottish Higher Education. Glasgow: JISC. Nicol, D.J. & D. MacFarlane-Dick. 'Formative Assessment and Self-Regulated Learning: A Model and Seven Principles of Good Feedback Practice.'. Studies in Higher Education. 2006; 31: 199-218. O'Brien, R. & L. Sparshatt. 'Mind the Gap! Staff Perceptions of Student Perceptions of Assessment Feedback'. Higher Education Academy Annual Conference 2007. 2007; Harrogate, 3-5 July: Available at: http://www.heacademy.ac.uk/events/conference/papers [Accessed 21 Aug 2011]. PEER. 2011. Peer Evaluation in Education Review. Website. Online: University of Strathclyde. Available at: http://www.reap.ac.uk/PEER.aspx [Accessed 21 June 2011]. REAP. 2011. Re-Engineering Assessment Practice in Higher Education. Website. Online: University of Strathclyde http://www.reap.ac.uk/reap/index.html [Accessed 21 Aug 2011]. Sadler, D.R. 'Beyond Feedback: Developing Student Capability in Complex Appraisal'. Assessment & Evaluation in Higher Education. 2010; 35: 535-550.