AHE Exemplars

1. Managing dialogue around
exemplars
David Carless, Masterclass
AHE Manchester,
June 28, 2017
http://davidcarless.edu.hku.hk/
Twitter: @CarlessDavid
The University of Hong Kong

2. Overview
1. Exemplars: rationale and benefits
2. Challenges and how they might be
tackled
3. Managing dialogue
4. Implementation suggestions
The University of Hong Kong

3. Teaching Enhancement Project
Enhancing dialogic use of exemplars amongst 10
teachers in a Faculty of Education
The University of Hong Kong

4. What are exemplars?
Samples used to illustrate dimensions of quality
Usually assignments from a previous cohort
The University of Hong Kong

5. Why used?
Exemplars convey messages
that nothing else can (Sadler,
2002)
The University of Hong Kong

6. Promotes self-evaluation
Illustrate what good
work looks like
Benchmark for
comparison with
own performance
The University of Hong Kong

7. Benefits for students
Gain experience in making
judgments
Apply insights to own work &
improve learning outcomes
The University of Hong Kong

8. Hendry et al., (2012)
Students positive about exemplars
Balanced teacher-led discussion of exemplars as
crucial
The University of Hong Kong

9. Student views
Rubrics: vague, unclear and do teachers
really use them?
Exemplars: useful & concrete; students
want more of them
(Carless, 2015, 2017)
The University of Hong Kong

11. The University of Hong Kong

12. ‘Model answer syndrome’
May reduce student
creativity
May lead to copying
(Handley & Williams,
2011)
The University of Hong Kong

13. Scaffolded use of exemplars
Students work on assessment task prior to
exposure to exemplars
The University of Hong Kong

14. Scaffolding steps
Instructional sequences combining:
- Student work in progress;
- Interaction with peers and teachers;
- Analysis of exemplars.
(Carless et al., 2018)
The University of Hong Kong

15. Time issues
• Collecting them
• Obtaining consent
• Time taken from
instruction
• How many exemplars?
The University of Hong Kong

16. Content vs process
Too much time spent teaching content,
insufficient attention to learning processes
The University of Hong Kong

17. It’s challenging!
Difficulties for students in evaluating exemplars
accurately
The University of Hong Kong

18. Have I addressed challenges satisfactorily?
Any other thoughts or queries?
The University of Hong Kong

19. DIALOGIC USE OF EXEMPLARS
The University of Hong Kong

20. Key assumption
The quality of dialogue about exemplars is crucial
The University of Hong Kong

21. The University of Hong Kong

22. Research question
What are the main features of a specific
exemplars dialogue and how is it orchestrated?
The University of Hong Kong

23. Context of the case
Participants:
• Trainee Science teachers
• BEd/BSc double degree
Teacher-researcher:
Dr Kennedy Chan
The University of Hong Kong

24. Data collection
• Classroom observation
• Open-ended student survey
• 2 focus group interviews
• Interview with teacher-researcher
• Teacher journal
• Student artefacts: ‘exit slips’, assignments etc.
The University of Hong Kong

25. Coding scheme
Main teacher moves:
- Eliciting student views
- Summarizing student views
- Elaborating student views
The University of Hong Kong

26. Exemplars provided
2 high quality exemplars
– Same format (i.e. reflective essays) but different
content focus
– Design aimed at reducing ‘copying’
The University of Hong Kong

27. Management of dialogue
1. Prior to class, students analyzed exemplars;
2. Students discussed exemplars in pairs;
3. Students elicited views from peers and made
mini-presentations;
4. Teacher-orchestrated dialogue;
5. Students submitted exit slip of reflections.
The University of Hong Kong

28. Dilemmas
Students
constructing
notion of quality
VS
Telling students
about quality
 Time for students
to talk with
peers
VS
Time for
developing shared
understandings

Exemplars as
guide
VS
Exemplars as
model
Using students’
voices
VS
Making
teacher’s voice
explicit

The University of Hong Kong

29. Good Exemplars dialogue
• Airs multiple & divergent viewpoints
• Shows linkages between peer talk & whole-
class discussion
• Evidences development of student views
• Makes explicit some key qualities of exemplars
(Carless & Chan, 2016)
The University of Hong Kong

30. Implications
The University of Hong Kong

31. Implementation blueprint
Assessment task
Students devise or
engage with criteria
for good task
response
Two samples read
before class
Peer discussion
Teacher-led
dialogue
Student ownership
of insights
The University of Hong Kong

32. Exemplars & feedback
By developing capacities to make judgments,
students are learning to decode feedback
(Sadler, 2010)
Analysis of exemplars facilitates student uptake
of feedback (Carless et al., 2018; To & Carless,
2016)
The University of Hong Kong

33. Engaging with quality
Student need to engage with what quality looks
like and develop capacities in making judgments
The University of Hong Kong

34. COMMENTS,
QUESTIONS
The University of Hong Kong

35. References
Carless, D. (2015). Excellence in University Assessment: learning from award-
winning teachers. London: Routledge.
Carless, D. (2017). Students’ experiences of Assessment for Learning. In D.
Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up
Assessment for learning in Higher Education. Singapore: Springer.
Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars.
Assessment and Evaluation in Higher Education,
http://dx.doi.org/10.1080/02602938.2016.1211246
Carless, D., Chan, K.K.H., To, J., Lo, M. & E. Barrett (2018). Developing
students’ capacities for evaluative judgement through analysing
exemplars. In D. Boud, R. Ajjawi, P. Dawson & J. Tai (Eds), Developing
Evaluative Judgement in Higher Education: Assessment for knowing and
producing quality work. London: Routledge.
Handley, K. & Williams, L. (2011). From copying to learning: Using exemplars
to engage students with assessment criteria and feedback. Assessment
and Evaluation in Higher Education, 36(1), 95-108.
The University of Hong Kong

36. References
Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing
standards‐based assessment effectively: Incorporating discussion
of exemplars into classroom teaching. Assessment and Evaluation
in Higher Education, 37(2), 149-161.
Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb
(Eds.), Assessment: Case Studies, Experience and Practice from
Higher Education (p.130-136). London: Kogan Page.
Sadler, D. R. (2010). Beyond feedback: Developing student capability in
complex appraisal. Assessment and Evaluation in Higher Education,
35(5), 535-550.
To, J. & Carless, D. (2016). Making productive use of exemplars: Peer
discussion and teacher guidance for positive transfer of strategies.
Journal of Further and Higher Education, 40(6), 746-764.
The University of Hong Kong

37. The University of Hong Kong

38. The University of Hong Kong
Productive assessment
task design
Appreciating the nature of
quality work
Student engagement
with feedback
Learning-oriented assessment framework

39. The University of Hong Kong