“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
Rob howe - assessment strategies in a digital age
1. Assessment strategies in a digital age
Rob Howe (@rjhowe)
The University of Northampton
Presentation at Blended Learning Conference, London.
26th October, 2016
2. Have we got it sorted?
No – but 2% up on HEI sector average
for A&F in NSS
JISC – digital student tracker
Online assessments are delivered
and managed well (%).
Agree – 61.8 (Npton) 54.8 (Sector)
Neutral – 23.6 30.2
Disagree – 5.5 9.5
NSS trend
3. Need to join up University approaches
Timings:
2007 Turnitin introduced to Northampton
<2010 – Small scale trials (Turnitin, SoundsGood Rotheram (2009) and Looks
Good)
2010-2011 – Piloting full e-submission
2012 – Whole University approach to submission and marking
JISC (2016), Assessment and
Feedback Lifecycle.
4. • We know many of the solutions but may be lacking time,
motivation and skills to make a difference
• Quality processes can regulate delivery and provide
consistency but can also restrict innovation – fear of failure!
6. Creating feedback opportunities
Do students value feedback (or just the grade ?)
“Staff complain that feedback does not work
(Weaver 2006) and that students do
not act on feedback (Mutch 2003). It is
claimed that students are only concerned
with their grades (Wojtas 1998; Nesbit and
Burton 2006), see feedback as a means
to justify the grade (Price and O’Donovan
2008; Price et al. 2010) or only read the
qualitative comments if the quantitative mark
is outside of their expectations
(Duncan 2007).”
(Hepplestone et al, 2011)
@rjhowe
7. Some benefits for students
• Allows students to read it at a time convenient to them
• Allows them to concentrate more deeply on the comments
in the absence of their peers.
• Students able to access feedback whenever and wherever
they complete future assessments.
• Where grades are presented alongside feedback, students
can use this information to inform their performance in
future assessment tasks.
• Feedback returned electronically may be returned quicker
as time may be saved during administrative processes.
• Device agnostic
8. Is it working? Feedback from students
•Email
• Blog
• Spot surveys
• Focus groups
I also feel that lecturers
returned grades to students
much more promptly when
they were on paper
I do think that
electronic feedback is
good because it is
easy and quick.
It was beneficial as it is easier
to hand in assignments
instead of having to travel
into the university however
feel there is less feedback
given
It would be better if it
was more uniform.I think its brilliant
and it saves time
and money on
travelling.”
9. Did staff like it ?
• Many did (after adapting their style) – Northampton
Outside the Box Assessment practices:
• Criminal Justice students produce videos and leaflets for
public benefit
• Paramedic Science students use videos to support peer
assessment of applied skills
• Distance Engineering students enjoy creative problem
solving to test their knowledge and understanding
• Engaging and assessing Sport students through digital
storytelling
• Mobile formative assessment for Foundation Art students
• ......others.....
10. Support for assessment enhancement
• JISC – Effective Assessment in a digital age
• JISC - Transforming assessment and feedback with
technology guide
• JISC - Electronic management of assessment readiness tool
http://ji.sc/emaready
• Race (2015) The Lecturer’s Toolkit
• Northampton’s Assessment and Feedback portal
11. Top five tips
1. Pilot extensively on robust tools and ensure all areas of the
institution are represented. Assess your starting point using the
JISC readiness tool - http://ji.sc/emaready
2. Consistent University policies and procedures – do assessments
have to be bunched at the end / moderation or second marking ?
3. Mandatory (re)training for all staff involved and phased rollout.
4. Clear instructions for students.
5. Review survey feedback – and keep refining.
13. References etc.
Useful links
• Hepplestone, S., Holden, G., Irwin, B., Parkin, H., Thorpe, L., (2011) Using
technology to encourage student engagement with feedback: a literature review .
Research in Learning Technology Vol. 19, No. 2, July 2011, 117–127.
• JISC (2010) Effective Assessment in a digital age. HEFCE.
• JISC (2016) Transforming assessment and feedback with technology guide, HEFCE.
• Race, P. (2015) The Lecturer’s Toolkit: A practical guide to assessment, learning and
teaching. 4th edition. Routledge.
• Rotheram, B. 2009. Sounds good: Quicker, better assessment using audio feedback.
JISC funded project. http://sites.google.com/site/soundsgooduk/downloads.
Image credits
• Photography by Rob Farmer, University of Northampton
Hinweis der Redaktion
Welcome and introductions
There will be a test at the end – those that pass will be allowed to go to the drinks reception!
Items considered in the assessment and feedback life cycle;
Basic Comments
Rubrics
Voice Comments
Quickmark comments
Video feedback
Peer Assessment
Second marking
External examiners
Submission points
Letter grading vs numeric grading
Ensure robust functionality of the tools
Archive results in line with University policy
Significant research in the area over a long period. Phil Race (2015) documents many of the common types of assessments – their benefits and their pitfalls.
This was the standard workflow at Northampton where we guided tutors on the stages from creating an assessment to finalising the grades and passing them to the correct records teams.
We have now broadened the range of assessment types:
Computer marked assignments allow you to identify the discrimination index and facility value of the questions to refine the question pools which can be used.
VLEs allow for a standard question to be setup with different variables each time based on a standard calculation.
Peer marking may involve students in the marking process.
Videos can be used in presentations, projects and eportfolios.
Consider gamification – aim for a level then get a reward.
There is a need to educate students about the need to review feedback and not just the grade. We need to consider the turn around times for assessment. 4 weeks can be too long for marking of TCAs. Modular structures can mean that students move onto another unit rather than having the opportunity to build on previous feedback.
Rubrics allow tutors to be explicit with students at the outset about the criteria for marking and what standard is needed to achieve a higher grade. Marking may also be simplified as tutors need to complete the rubric and then provide some comments at the end. Rubrics and marking toolkits may be shared within the teaching tem to improve consistency.
In the module Crime and Criminality, part of the assessment is to produce a 30 second
video and supporting leaflet around a crime prevention strategy. This could be
something already operating, so to raise awareness or a completely new idea about
how the public can protect themselves from being a victim of a particular crime. Once
assessed, Superintendent Dave Hill will be looking at them to choose one that will
feature in the Northants Police Website.
In the Paramedic Science division we use Objective Structured Clinical Examinations
(OSCEs) to assess practical application of skills. For example, advanced life support can
be demonstrated on mannequins using all of the actual equipment required. In practice
sessions we asked the students to video their attempts on SMART phones, iPads or
other devices, which can then be uploaded to NILE (the University’s Virtual Learning
Environment). These could then be viewed for self, peer and tutor review as formative
assessment and feedforward to improve performance. Initially, the students found this
to be daunting but eventually came to value the opportunity to self assess and refer
back to their performance on past videos.
Distance Engineering Students - created assessments that include a range of question types from formulae to essays to cater for students with different skills.
Sport students - a Blackboard journal tool for each group, and they are required to write a formal journal entry for each of the 7 weeks where they document and justify (with the aid of appropriate references) the choices they are making in relation to topic, methodology, design etc.
Upon completion of their journal diaries, I ask them to create a 5 to 10 minute movie
documentary which should present their research project using these 7 stages as ‘chapters’ in their movie. I offer 30% of the mark for creative communication of their work, and students have come up with some fantastically original ideas (last year’s
highlight was the use of Lego figures to present a group meeting with Emile Durkheim, a
famous sociologist).
Attendance for this part of the module is always high, and students often comment that
they have never worked harder than on this assignment (a good thing?!), and take a lot
of pride in getting their movie just right.
There is significant research and support available for A&F.
There can be problems as noted by JISC where there is variation in approach across the institution; technology is bolted on and not blended and there may be organisation myths about policy and process…..it’s always been done that way and hence it must be right? This can lead to variations about how staff apply the policy in the first place.