This document summarizes a study on assessment workload at an unnamed institution. It profiles summative continuous assessment workloads for students and staff during one semester. Key findings include that multiple assessments often cluster towards the end of semester. Students generally spend 1 day or less on assessments regardless of weighting, while time spent varies greatly for group work. Staff are often unaware of workloads in other modules. Recommendations include better communication across programmes on assessments and asking students why they spend certain amounts of time on assessments.
1. Find us on Twitter: @glynnmark @clare_gormley @LostelloeFind us on Twitter: @glynnmark @clare_gormley @Lostelloe
More Than The Metrics
Our deep dive into assessment workload
Laura Costelloe, Mark Glynn, Clare Gormley
2. Overall Project Goal
To profile summative continuous
assessment workloads (staff and student)
during one semester (Semester 2,
2017/18 academic year)
To understand and improve the student
experience on the basis of evidence
3. What does the literature tell us?
- “Assessment is probably the most important determinant of
the character of a course and of student learning within the
course” (Scott, 2015);
- A modular course structure and largely unregulated
numbers of internal assessments has resulted in an
assessment “arms race” (Harland et al, 2015);
- Students will generally say they are over-worked - but it’s
important to have conversations around expectations
(Fielding, 2008);
4. What metrics should we use?
- There are methodological challenges with measuring
assessment workload, e.g. what metrics should be used?
should we include both formative and summative assessment
tasks? How should we classify different assessment types?
- Scott (2015) considers the benefits of examining the
following: number of assessment tasks; word count and
equivalences; student time-on-task; time involved in
designing an assessment task and in marking
- Ultimately, “there are no simple answers as to how student
assessment workload should be measured and standardised”
(Fielding, 2008)
5. Methodology
- 5 programmes from 5 faculties investigated (Humanities,
Science, Education, Engineering, Business)
- Initial data drawn from centralised Course Information system
- Staff questionnaire + interview (11 lecturers)
- Focus on perceptions of workload + time on design, marking, feedback
- In-class student surveys (8 modules, n=129 student
respondents)
- Focus on time on task
6. Padlet Q 1
What are the challenges of measuring the assessment
workload of students and staff?
https://padlet.com/clare_gormley/q1
7. DCU Business School:
Number of assessments
1
2
3
4
4
5
6
7
1 2 3 4 5 6 7 8 9 10 11 12
9 Assessments in Total
(excluding attendance/participation)
Week
Numberofassessments
55% of assessments due in the
same week. or 77% in the space of
two weeks
13
9. 5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Presentation
Essay / Case Study
Reflection
Poster
MCQ
Participation/Attendance
Week
%ofmarktowardsfinalmark
13
DCU Business School:
Type and weighting
10. DCU Business School:
Type and weighting
Presentation
Essay / Case Study
Reflection
Poster
MCQ / Short answer
Participation/Attendance
%ofmarktowardsfinalmark
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Week
13
50
60
70
80
90
100 E
E
E
E EE
E
11. Faculty of Science and Health:
Number of assessments
1
2
3
4
4
5
6
7
1 2 3 4 5 6 7 8 9 10 11 12
22* Assessments in Total
(excluding attendance/participation)
Week
Numberofassessments
23% of assessments due in the
same week. or 36% in the space of
two weeks
12. Faculty of Science and Health:
Type and weighting
Presentation
Essay / Case Study
Reflection
Poster
MCQ / Short answer
Participation/Attendance
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Week
%ofmarktowardsfinalmark
50
13
60 E
E
E
E
14. Key Findings - Student Perspective
-Multiple CAs are clustering around same time,
typically towards end of semester
-Notably majority of students spend ≤ 1 day on
assessment preparation, regardless of weighting
-As expected, there are substantial variations in
student time-on-task across all assessments eg For
one group presentation task, data shows
- 44% of students spend ≤ 1 day
- 24% of students spend ≥ 3 weeks
15. What students are saying...
I had an Oral CA the
same week as
[Module X] CA.
Had many assessments
and exams at this time,
was overwhelmed and
exhausted
Normally I would spend more
time but there was a lot of
work due around the same
time, so, as this was a mock-
exam, it was not my priority
Lot of CAs towards
end of term
16. Key Findings - Staff Perspective
-Thoughtful consideration of own module workload but
generally unaware of assessments elsewhere (some
exceptions)
-Aware of insufficient communication/visibility across the
entire programme
-Variations in marking and feedback time - feedback
available but often unavailed of; lecturers not strongly
influenced by design, marking and feedback time
-Aware of group work free-rider issue and possible
implications for student workload
17. What staff are saying...
I don’t know how much
lecturers know about how
much is going on in the bigger
picture for students because
we just tend to be
responsible for [our own]
modules.
The majority of the content
(90%) is done in class so
when they go home, if they
spend 10-15 minutes
finishing up every week, that
will be it. If they let it go,
then it becomes a massive
burden.
Within the past few years, I set
up this Google Drive
spreadsheet so that for all the
modules all the staff can see
how and when they are being
assessed. We try not to have
everything due in week 12, for
example... That has definitely
helped.
I would think that a student who embraces
every single week and does the assigned
reading every week has a fairly hefty
workload. But I imagine that most students
don’t do that... They can choose how
heavy to make their workload but it is
difficult to gauge it.
I don’t think that they are over assessed within
this module. But I am always wary, when you are
doing any form of group work, that there is that
potential for some people to have more of an
assessment workload than others simply because
they have to carry the slack for people who don’t
equally contribute.
18. Padlet Q 2
How does your institution currently compare workloads
of different modes of assessment across disciplines?
https://padlet.com/clare_gormley/q2
19. Recommendations for similar studies
Go for it.. but be aware of ethical considerations and possible
reluctance to participate (Programme Chair may help with buy-in.)
(Re)Consider the word count question
Ask staff what amount of time they would expect students to spend
on their assessments
Clarify assessment info from centralised system before approaching
students (esp regarding assessment type categorisation)
Provide hard and online copies of survey - we received more
comments on printed copies
Ask students the Why? question regarding their time spent
20. Padlet Q 3
How should we engage with staff to take a more
programmatic view of assessment?
https://padlet.com/clare_gormley/q3
23. Bibliography
Fielding, A. (2008) ‘Student assessment workloads: a review’, Learning and
Teaching in Action 7, 3: 7-15.
Harland, T., McLean, A., Wass, R., Miller, E and Sim, K. N. (2015) ‘An
assessment arms race and its fallout: high-stakes grading and the case for
slow scholarship’, Assessment & Evaluation in Higher Education, 40(4): 528-
541.
Scott, S.V. (2015) ‘Quantifying the assessment loads of students and staff:
the challenge of selecting appropriate metrics’, Journal of Further and
Higher Education, 39(5): 699-712.
25. Programme Calendar
- Moodle plugin
- Tag with programme code
- Viewable to relevant staff
- Combine with “my feedback”
plugin
- Beta test : September 2018
26. Programme in
Faculty of Engineering and
Computing:
Type and weighting
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Presentation
Essay / Case Study
Reflection
Poster
MCQ
Participation/Attendance
Week
%ofmarktowardsfinalmark Labs