Experience of using formative assessment and students perception of formative assessment
1. Experience of using formative
assessment and students
perception of formative
assessment
Paul Ong
Greg Benfield
Margaret Price
2. Evaluation of the Oxfor d Brookes
Univer sity Assessment Compact
How satisfied are students with assessment and
feedback practices in their courses?
Which assessment and feedback practices do you
think have the greatest impact on learning?”
How do students and staff relate specific
assessment and feedback practices to impact on
their learning?
3. Integration of Compact
to Module U43702
Important principles adopted by team aligned with
Compact:
• Assessment is integral to teaching and learning
• Timely feedback provided
• Actively involving students in assessment and
dialogue (with peers and tutors)
4. Introducing a formative
assessment
Three concepts integrated in support of these
principles:
1. Establishing Peer Support Groups
2. Identifying learning needs through timely formative
feedback from: Peers, Seminar leader and
Turnintin
3. Support with developing own academic skills
5. Question 1 - Most value to learning (all 31 responded)
Learning what is expected of me
Being able to send a draft to turn it in and also to the
tutor, as it showed me the level I am at as I had no idea.
(Resp 12)
Getting direction on how to progress
the formative feedback as it gives direction as to where
to go next with the essay (Resp 04)
Learning how to improve their grade or performance
The feedback I have received is clear and will help me
to hopefully rework my part one before final deadline,
so I can achieve a better standard. (Resp 03)
Gaining confidence
The understanding of how to write an essay has
improved a lot and the knowledge of where I have gone
wrong and how to improve. It has given me more
confidence. (Resp 26)
12 respondents
(39%)
Learning what I’ve done wrong
helped us understand where we were going wrong and
gave us a lot of confidence,,, (Resp 02)
4 respondents
(13%)
8 respondents
(26 %)
8 respondents
(26 %)
5 respondents
(16%)
6. Question 2(a) - Perceptions of the value of peer learning
(81% responded positively)
Seeing other approaches
Reading others was useful to you; get other ideas of
what information to include. (Resp 14)
6 respondents
(19%)
Sharing ideas
It was useful for someone else, who also has been
working on the same information, to share their ideas
+ seeing their work. (Resp 12)
Gaining a better understanding of content
Presentation: encouraged me to get in-depth
knowledge of one of the symptoms. This then helped
me to plan how I would research each of the other
symptoms. Other groups presentations then gave me
an overview of the physiology of other symptoms to
start with….. (Resp 21)
4 respondents
(13%)
Getting feedback from peers
Getting feedback was helpful as you are able to learn
more from them. (Resp 14)
1 respondent
(3%)
3 respondents
(10%)
7. Question 2(b) Perceptions of the value of self-assessment
(Only aspect negatively perceived by majority)
.
10 respondents (32%) expressed positive perceptions
18 (58%) expressed negative perceptions about self-assessment
Remainder (3) did not comment on this aspect
Most common reasons for participants’ dissatisfaction with this aspect
were lack of perceived relevance and lack of skill:
‘Self assessing = not valuable as we are not teachers or taught to
mark others’ work!’ (Resp 03)
8 (26% of all respondents) of the negative comments indicated
respondents felt ill equipped to do self-assessment. For example:
‘I don’t think that this was useful as none of us really had the
confidence or knowledge to mark each other’s work ‘(Resp 08)
8. Question 2(c) - Perceptions of the value of tutor feedback
(Highly valued by all 31 respondents)
Typical comments include:
‘Excellent feedback which has given me great insight into
where I need to improve and how. I am no longer worried
about my writing skills and this is down to such great tutor
feedback’ (Resp 20)
‘The comments you wrote on the side where you referred
to the text I found very useful and a big advantage to me’
(Resp 28)
‘Very detailed feedback. I know what to change to
improve my assignment’ (Resp 10)
9. Question 2(d) - Perceptions of the value of Turnitin (all 31
students made favourable comments about Turnitin)
Typical comments include:
‘Receiving the Turnitin report and the tutor feedback on
how to improve. It lets you know that you are on the right
track and how you were doing in terms of knowledge and
writing as well as using references correctly’ (Resp 14)
‘Showed me how to paraphrase better so I don’t get too
much highlighted from my text’ (Resp 31)
‘Confused, but I know that I shouldn’t get too much in my
text to be highlighted’ (Resp 31)
A second respondent commented that they didn’t learn much
from Turnitin because their referencing was already good.
10. Analysis of module U43702 student
feedback about formative assessment
Results
There are three main findings.
1.Formative assignment was highly valued by all the
respondents
2.The self-assessment element is problematic. Many, if not
most, respondents felt they lacked the skills to self-assess
their work effectively and/or did not see the reason for doing
so
3.The vast majority (81%) views the peer learning elements
very favourably
11. Participants’ recommendations
for changes (27 students)
•opportunity for a second draft for those who did poorly on
their first one (x1 resp)
•Extend the formative process into part two of the assignment,
(x3 resp)
•Try to include more content in the weeks 5 to 7 sessions
(x1 resp)
•Reduce the amount of time spent on Turnitin (x1 resp)
•Schedule the assignment earlier in the semester (3 resp)
12. Recommendations:
Peer assessment is highly regarded but perhaps
its full value is still to be realized.
Actions that might improve perception about value to be
gained include:
• Highlighting for students in subsequent runs the value
that past participants have found from seeing a range of
responses to the same issue or problem
• This sharing of ideas and approaches allows them to
improve their own work.
• Stressing with participants that giving feedback is a skill
they will develop over time
• Providing some additional in-class training in what to
look for (using criteria) and how to offer feedback
13. Recommendations
Establish processes for providing a rationale for selfassessment and for supporting students with it
Some possibilities include:
• Make clearer the link between developing self-assessment skills
and the well-regarded peer assessment component. Students need
to understand that the main way in which they will improve their selfassessment skills is by being exposed to and assessing or giving
feedback on a variety of other peoples’ work
• Clarify and make explicit the relationship between self- (and peer-)
assessment skills and the graduate attribute of personal literacy and
critical self-awareness
• Incorporate additional self-assessment elements so that students
become more familiar with the process and (hopefully) see their
skills develop.
• Advocate opportunities for formative Turnitin use in other (at least
first-year) modules.
14. Evaluation of the Oxfor d Brookes
Univer sity Assessment Compact
How satisfied are students with assessment and
feedback practices in their courses?
Which assessment and feedback practices do you
think have the greatest impact on learning?”
How do students and staff relate specific
assessment and feedback practices to impact on
their learning?
15. References
Davis M (2007) ‘The Role of Turnitin within the
formative process of academic writing – A tool for
learning and unlearning. Brookes eJournal of
Learning and Teaching. Vol.2 Issue 2.
Davis M (2009) ‘Using Turnitin Formatively’ OBU
Hinweis der Redaktion
Prior to implementing Compact:
Only had summative assessment, module team looked at A4 plan, all taught content focused on physiology/pathophysiology
Resit rate around 10% - so quite high
Feedback from team suggested issues more about academic/literacy skills rather than understanding of subject content
Compact gave us platform to try and address these issues
What we wanted to do was raise students awareness of their own academic skills, develop these skills and their abilities to make their own informed judgements:
introduce activities e.g. marking exercises, self and peer-assessment etc. specifically designed to involve students in the assessment process
encourage dialogue between students and their tutors and students and their peers
Notes:
Peer Support Groups (PSG) set up in week 1 – Introductory Seminar (2 hours)
Generating a context for learning where the student feels supported and where students feel more comfortable with self and peer-assessment activities
Establishing links between assessment and module content early on.
Tasks set for PSG:
Find out who is in your Peer Support Group
Arrange to meet
Read assignment question
Read ‘Assignment Guidance’
Read ‘How to Reference for Health and Social Care Students’ (Harvard System)
Learn how to access the electronic journals
Select journal articles and access them via the Electronic Library
Go through the ‘Assignment Tutorial’ (focuses on interpretation of module marking criteria (explored in week 5 assignment seminar)
Prepare 10 minute presentation on one of symptoms in assignment to present at week 5 assignment seminar
Plan draft of Assignment for formative feedback from peer(s) at week 5 assignment seminar
2. Provision of formative feedback
Week 5 – Assignment Seminar 1 (2 hours)
Actively involving students in assessment and dialogue to enhance understanding (marking exercises, self and peer-assessment)
Seminar activities:
Peer support groups give 10 min. presentation on one of the allocated assignment symptoms to rest of group – opportunity for seminar leader to use presentations on symptoms to ‘sign post’ how module content relates to assignment
Students bring individual assignment drafts (Part 1 only) to discuss/mark in peer support groups (with seminar leader support) – students have opportunity to use marking criteria and mark a draft of a person in their PSG, give feedback to that person and engage in general discussion about meaning/significance of each criteria.
Students develop draft further through peer support activities in seminar and seminar leader input
Support: what is Turnitin, how will it be used on the module, how to submit draft for formative feedback to Turnitin
Week 6:
Submit electronic draft of assignment to Turnitin for formative feedback on academic writing skills
Submit hard copy of draft for feedback from seminar leader
Week 7 – Assignment Seminar 2 (2 hours)
Timely feedback provided
Actively involving students in assessment and dialogue
Seminar activities:
Students receive own copies of Originality Reports and seminar leader feedback on drafts
Seminar leader presentation on Interpretation of ‘Originality Reports’
Activities focusing on development of individual academic writing skills: identifying individual learning needs and where support can be found (use of post-its to add degree of anonimity – examples include paraphrasing, referencing, analysing etc)
Group/individual support on formative feedback: students read feedback, students discuss feedback in PSG and PSG collate questions to discuss with seminar leader (opportunity at end for individual students to discuss any questions have about feedback with seminar leader)
Seminar leader summarises key themes relating to academic writing or subject knowledge
Seminar leader discusses how/where to seek support with academic writing skills
Deadline for final summative submission – Week 11
Assessment Compact Evaluation
Analysis of module U43702 student feedback about formative assessment 2012 (31 anonymous respondents)
The questionnaire asked for student reactions to the following:
Please think about your formative assignment during weeks 5-7, consisting of presentation of a draft to your peer support group, followed by submission of your draft via Turnitin and receiving tutor feedback on it. Regarding this:
1. Which parts of this process were of most value to your learning? Why?
2. What did you learn from
a) giving and getting feedback from peers?
b) self-assessing your work?
c) tutor feedback?
d) feedback from Turnitin?
3. How, if at all, do you think this formative assignment process could be
improved?
Question 1 Most value to learning
All 31 respondents answered item 1. The learning benefits explicitly referred to are shown in Table 1, along with a representative quote to illustrate the idea.
Table 1: major benefits to learning
Additionally, the following processes were explicitly referred to within this item as being especially valued:
Peer support10 respondents (32%)
Tutor feedback - 4 respondents (13%)
Using Turnitin - 12 respondents (39%)
Question 2a Perceptions of the value of peer learning
This is a highly valued aspect of the assignment. 25 of the 31 respondents (81%) responded positively to item 2a about the peer learning aspects of the assignment.
Actions that might improve perception about value to be gained include:
Highlighting for students in subsequent runs the value that past participants have found from seeing a range of responses to the same issue or problem
Few participants explicitly commented on the value of getting feedback from their peers.
It isn’t clear whether this is because they did not rate the quality of such feedback highly, whether they didn’t actually get much feedback from their peers.
Participants may have found quantity and/or quality of peer feedback insufficient because many found it hard to give good feedback to their peers It might be worth stressing with them that this is a skill they will develop over time. Some in-class training in what to look for (using criteria) and how to offer feedback would be worth incorporating if this is not already done.
This sharing of ideas and approaches allows them to improve their own work.
Stressing with participants that giving feedback is a skill they will develop over time
Providing some additional in-class training in what to look for (using criteria) and how to offer feedback
Question 2b Perceptions of the value of self-assessment
The self assessment component of the assignment was the only aspect that was negatively perceived by a majority of participants.
Suggests module team need to make clearer the link between developing self-assessment skills and the well-regarded peer assessment component. Students need to understand that the main way in which they will improve their self-assessment skills is by being exposed to and assessing or giving feedback on a variety of other peoples’ work
Clarify and make explicit the relationship between self- (and peer-) assessment skills and the graduate attribute of personal literacy and critical self-awareness
Incorporate additional self-assessment elements so that students become more familiar with the process and (hopefully) see their skills develop.
Advocate opportunities for formative Turnitin use in other (at least first-year) modules
Tutor feedback on their assignments and presentations was universally regarded (i.e. every single respondent) as of tremendous value and usefulness. The features of tutor feedback that were valued were gaining a sense of the required level and getting direction on how to improve their work.
Question 2(d) - Perceptions of the value of Turnitin
Formative use of Turnitin was the other universally acclaimed assignment element after tutor feedback. All 31 respondents made favourable comments about the Turnitin aspect. Typical remarks were:
Receiving the Turnitin report and the tutor feedback on how to improve. It lets you know that you are on the right track and how you were doing in terms of knowledge and writing as well as using references correctly (Resp 14)
Showed me how to paraphrase better so I don’t get too much highlighted from my text (Resp 31)
As well as positive perceptions of the value of Turnitin to participants also expressed difficulties. For example, as well as the above remark respondent 31 also said of Turnitin:
Confused, but I know that I shouldn’t get too much in my text to be highlighted (Resp 31)
A second respondent commented that they didn’t learn much from Turnitin because their referencing was already good.
This feedback suggests that it would be worthwhile for the course team to advocate opportunities for formative Turnitin use in other (at least first-year) modules.
Results
There are three main findings.
1.This formative assignment was highly valued by all the respondents. A substantial majority explicitly recommend making NO changes to it in future while those changes recommended were of a relatively minor nature. Further details about what respondents valued most are given below.
2.The self-assessment element is problematic. Many, if not most, respondents felt they lacked the skills to self-assess their work effectively and/or did not see the reason for doing so. Many described this part of the task as ‘hard’. Future runs of this assignment will need to establish processes for providing a rationale for this aspect and for supporting students with it.
3.The vast majority (81%) views the peer learning elements very favorably, but some gain or perceive less value in it than they should. This might be partially explained by members having been absent for group presentations and/or being un- or under-prepared to give each other feedback.