SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Find us on Twitter: @glynnmark @clare_gormley @LostelloeFind us on Twitter: @glynnmark @clare_gormley @Lostelloe
More Than The Metrics
Our deep dive into assessment workload
Laura Costelloe, Mark Glynn, Clare Gormley
Overall Project Goal
To profile summative continuous
assessment workloads (staff and student)
during one semester (Semester 2,
2017/18 academic year)
To understand and improve the student
experience on the basis of evidence
What does the literature tell us?
- “Assessment is probably the most important determinant of
the character of a course and of student learning within the
course” (Scott, 2015);
- A modular course structure and largely unregulated
numbers of internal assessments has resulted in an
assessment “arms race” (Harland et al, 2015);
- Students will generally say they are over-worked - but it’s
important to have conversations around expectations
(Fielding, 2008);
What metrics should we use?
- There are methodological challenges with measuring
assessment workload, e.g. what metrics should be used?
should we include both formative and summative assessment
tasks? How should we classify different assessment types?
- Scott (2015) considers the benefits of examining the
following: number of assessment tasks; word count and
equivalences; student time-on-task; time involved in
designing an assessment task and in marking
- Ultimately, “there are no simple answers as to how student
assessment workload should be measured and standardised”
(Fielding, 2008)
Methodology
- 5 programmes from 5 faculties investigated (Humanities,
Science, Education, Engineering, Business)
- Initial data drawn from centralised Course Information system
- Staff questionnaire + interview (11 lecturers)
- Focus on perceptions of workload + time on design, marking, feedback
- In-class student surveys (8 modules, n=129 student
respondents)
- Focus on time on task
Padlet Q 1
What are the challenges of measuring the assessment
workload of students and staff?
https://padlet.com/clare_gormley/q1
DCU Business School:
Number of assessments
1
2
3
4
4
5
6
7
1 2 3 4 5 6 7 8 9 10 11 12
9 Assessments in Total
(excluding attendance/participation)
Week
Numberofassessments
55% of assessments due in the
same week. or 77% in the space of
two weeks
13
Legend
Presentation
Essay / Case Study
Reflection
Poster
MCQ
Participation/Attendance
Individual Group
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Presentation
Essay / Case Study
Reflection
Poster
MCQ
Participation/Attendance
Week
%ofmarktowardsfinalmark
13
DCU Business School:
Type and weighting
DCU Business School:
Type and weighting
Presentation
Essay / Case Study
Reflection
Poster
MCQ / Short answer
Participation/Attendance
%ofmarktowardsfinalmark
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Week
13
50
60
70
80
90
100 E
E
E
E EE
E
Faculty of Science and Health:
Number of assessments
1
2
3
4
4
5
6
7
1 2 3 4 5 6 7 8 9 10 11 12
22* Assessments in Total
(excluding attendance/participation)
Week
Numberofassessments
23% of assessments due in the
same week. or 36% in the space of
two weeks
Faculty of Science and Health:
Type and weighting
Presentation
Essay / Case Study
Reflection
Poster
MCQ / Short answer
Participation/Attendance
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Week
%ofmarktowardsfinalmark
50
13
60 E
E
E
E
Student time on task
Key Findings - Student Perspective
-Multiple CAs are clustering around same time,
typically towards end of semester
-Notably majority of students spend ≤ 1 day on
assessment preparation, regardless of weighting
-As expected, there are substantial variations in
student time-on-task across all assessments eg For
one group presentation task, data shows
- 44% of students spend ≤ 1 day
- 24% of students spend ≥ 3 weeks
What students are saying...
I had an Oral CA the
same week as
[Module X] CA.
Had many assessments
and exams at this time,
was overwhelmed and
exhausted
Normally I would spend more
time but there was a lot of
work due around the same
time, so, as this was a mock-
exam, it was not my priority
Lot of CAs towards
end of term
Key Findings - Staff Perspective
-Thoughtful consideration of own module workload but
generally unaware of assessments elsewhere (some
exceptions)
-Aware of insufficient communication/visibility across the
entire programme
-Variations in marking and feedback time - feedback
available but often unavailed of; lecturers not strongly
influenced by design, marking and feedback time
-Aware of group work free-rider issue and possible
implications for student workload
What staff are saying...
I don’t know how much
lecturers know about how
much is going on in the bigger
picture for students because
we just tend to be
responsible for [our own]
modules.
The majority of the content
(90%) is done in class so
when they go home, if they
spend 10-15 minutes
finishing up every week, that
will be it. If they let it go,
then it becomes a massive
burden.
Within the past few years, I set
up this Google Drive
spreadsheet so that for all the
modules all the staff can see
how and when they are being
assessed. We try not to have
everything due in week 12, for
example... That has definitely
helped.
I would think that a student who embraces
every single week and does the assigned
reading every week has a fairly hefty
workload. But I imagine that most students
don’t do that... They can choose how
heavy to make their workload but it is
difficult to gauge it.
I don’t think that they are over assessed within
this module. But I am always wary, when you are
doing any form of group work, that there is that
potential for some people to have more of an
assessment workload than others simply because
they have to carry the slack for people who don’t
equally contribute.
Padlet Q 2
How does your institution currently compare workloads
of different modes of assessment across disciplines?
https://padlet.com/clare_gormley/q2
Recommendations for similar studies
Go for it.. but be aware of ethical considerations and possible
reluctance to participate (Programme Chair may help with buy-in.)
(Re)Consider the word count question
Ask staff what amount of time they would expect students to spend
on their assessments
Clarify assessment info from centralised system before approaching
students (esp regarding assessment type categorisation)
Provide hard and online copies of survey - we received more
comments on printed copies
Ask students the Why? question regarding their time spent
Padlet Q 3
How should we engage with staff to take a more
programmatic view of assessment?
https://padlet.com/clare_gormley/q3
Attitudes to Excessive Workload
Contact us
clare.gormley@dcu.ie
laura.costelloe@dcu.ie
mark.glynn@dcu.ie
Bibliography
Fielding, A. (2008) ‘Student assessment workloads: a review’, Learning and
Teaching in Action 7, 3: 7-15.
Harland, T., McLean, A., Wass, R., Miller, E and Sim, K. N. (2015) ‘An
assessment arms race and its fallout: high-stakes grading and the case for
slow scholarship’, Assessment & Evaluation in Higher Education, 40(4): 528-
541.
Scott, S.V. (2015) ‘Quantifying the assessment loads of students and staff:
the challenge of selecting appropriate metrics’, Journal of Further and
Higher Education, 39(5): 699-712.
Additional Slides
Programme Calendar
- Moodle plugin
- Tag with programme code
- Viewable to relevant staff
- Combine with “my feedback”
plugin
- Beta test : September 2018
Programme in
Faculty of Engineering and
Computing:
Type and weighting
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12
Presentation
Essay / Case Study
Reflection
Poster
MCQ
Participation/Attendance
Week
%ofmarktowardsfinalmark Labs

Weitere ähnliche Inhalte

Was ist angesagt?

Looking atstudentwork, michelle swanson theron cosgrave
Looking atstudentwork, michelle swanson theron cosgraveLooking atstudentwork, michelle swanson theron cosgrave
Looking atstudentwork, michelle swanson theron cosgrave
NAFCareerAcads
 
Gr4009 application of quality standards and instruments v.1.1
Gr4009 application of quality standards and instruments v.1.1Gr4009 application of quality standards and instruments v.1.1
Gr4009 application of quality standards and instruments v.1.1
Ibrahim Suliman
 

Was ist angesagt? (18)

Agile Practices in Higher Education: A Case Study
Agile Practices in Higher Education: A Case StudyAgile Practices in Higher Education: A Case Study
Agile Practices in Higher Education: A Case Study
 
2019.05.10 Noticing Progress IH Bydgoszcz Sandy Millin
2019.05.10 Noticing Progress IH Bydgoszcz Sandy Millin2019.05.10 Noticing Progress IH Bydgoszcz Sandy Millin
2019.05.10 Noticing Progress IH Bydgoszcz Sandy Millin
 
100 Days of EE
100 Days of EE100 Days of EE
100 Days of EE
 
Test Your Instructional Design IQ
Test Your Instructional Design IQTest Your Instructional Design IQ
Test Your Instructional Design IQ
 
3.7 Evaluating impacts of VLE use
3.7 Evaluating impacts of VLE use3.7 Evaluating impacts of VLE use
3.7 Evaluating impacts of VLE use
 
Sally Brown 2008
Sally Brown 2008Sally Brown 2008
Sally Brown 2008
 
Shades of Instructional design
Shades of Instructional designShades of Instructional design
Shades of Instructional design
 
Supervising and working with PhD students - July 2020
Supervising and working with PhD students - July 2020Supervising and working with PhD students - July 2020
Supervising and working with PhD students - July 2020
 
Outcomes Targeting Introduction
Outcomes Targeting IntroductionOutcomes Targeting Introduction
Outcomes Targeting Introduction
 
New SP coaching for induction 1 2013
New SP coaching for induction 1 2013New SP coaching for induction 1 2013
New SP coaching for induction 1 2013
 
Self assessment (Portfolio) By Miss Muntira Wongsawieng TESOL 7
Self assessment (Portfolio) By Miss Muntira Wongsawieng TESOL 7Self assessment (Portfolio) By Miss Muntira Wongsawieng TESOL 7
Self assessment (Portfolio) By Miss Muntira Wongsawieng TESOL 7
 
Looking atstudentwork, michelle swanson theron cosgrave
Looking atstudentwork, michelle swanson theron cosgraveLooking atstudentwork, michelle swanson theron cosgrave
Looking atstudentwork, michelle swanson theron cosgrave
 
Qcl 15-v4 [3]-[banasthali university]_[prabh simran kaur]
Qcl 15-v4 [3]-[banasthali university]_[prabh simran kaur]Qcl 15-v4 [3]-[banasthali university]_[prabh simran kaur]
Qcl 15-v4 [3]-[banasthali university]_[prabh simran kaur]
 
T pi he workshop 3
T pi he workshop 3T pi he workshop 3
T pi he workshop 3
 
First Principles of Instruction
First Principles of InstructionFirst Principles of Instruction
First Principles of Instruction
 
SBAC Performance Task Overview
SBAC Performance Task OverviewSBAC Performance Task Overview
SBAC Performance Task Overview
 
SlOAN November 2013 Handout for Presentation
SlOAN November 2013 Handout for PresentationSlOAN November 2013 Handout for Presentation
SlOAN November 2013 Handout for Presentation
 
Gr4009 application of quality standards and instruments v.1.1
Gr4009 application of quality standards and instruments v.1.1Gr4009 application of quality standards and instruments v.1.1
Gr4009 application of quality standards and instruments v.1.1
 

Ähnlich wie More than the metrics - SEDA 2018

Construction Technology Course outline
Construction Technology Course outlineConstruction Technology Course outline
Construction Technology Course outline
Est chua
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
Quo Ming
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
Louise Foong
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
chloeesim
 
Acc30205newcourseoutline jan2015semester-151210044427
Acc30205newcourseoutline jan2015semester-151210044427Acc30205newcourseoutline jan2015semester-151210044427
Acc30205newcourseoutline jan2015semester-151210044427
Kenneth Tan
 
Course Outline (Accounting).doc
Course Outline (Accounting).docCourse Outline (Accounting).doc
Course Outline (Accounting).doc
Xying Lee
 
Bk course outline september 2015 semester
Bk course outline   september 2015 semesterBk course outline   september 2015 semester
Bk course outline september 2015 semester
Tung97Michelle
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
AdeleLu
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
Yip Xiaojung
 
Acc new course outline
Acc new course outline Acc new course outline
Acc new course outline
LY97
 
ACC30205 New Course Outline - September 2015 Semester.doc
ACC30205 New Course Outline - September 2015 Semester.docACC30205 New Course Outline - September 2015 Semester.doc
ACC30205 New Course Outline - September 2015 Semester.doc
Jia San Oljs
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
gjj97
 
ACC30205 New Course Outline - September 2015 Semester
ACC30205 New Course Outline - September 2015 SemesterACC30205 New Course Outline - September 2015 Semester
ACC30205 New Course Outline - September 2015 Semester
ChengFern
 

Ähnlich wie More than the metrics - SEDA 2018 (20)

Construction Technology Course outline
Construction Technology Course outlineConstruction Technology Course outline
Construction Technology Course outline
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
 
Acc30205 new course outline jan 2015 semester
Acc30205 new course outline   jan 2015 semesterAcc30205 new course outline   jan 2015 semester
Acc30205 new course outline jan 2015 semester
 
Acc30205newcourseoutline jan2015semester-151210044427
Acc30205newcourseoutline jan2015semester-151210044427Acc30205newcourseoutline jan2015semester-151210044427
Acc30205newcourseoutline jan2015semester-151210044427
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Course Outline (Accounting).doc
Course Outline (Accounting).docCourse Outline (Accounting).doc
Course Outline (Accounting).doc
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Bk course outline september 2015 semester
Bk course outline   september 2015 semesterBk course outline   september 2015 semester
Bk course outline september 2015 semester
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Acc
AccAcc
Acc
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Acc new course outline
Acc new course outline Acc new course outline
Acc new course outline
 
ACC30205 New Course Outline - September 2015 Semester.doc
ACC30205 New Course Outline - September 2015 Semester.docACC30205 New Course Outline - September 2015 Semester.doc
ACC30205 New Course Outline - September 2015 Semester.doc
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
ACC30205 New Course Outline - September 2015 Semester
ACC30205 New Course Outline - September 2015 SemesterACC30205 New Course Outline - September 2015 Semester
ACC30205 New Course Outline - September 2015 Semester
 
Acc30205 new course outline september 2015 semester
Acc30205 new course outline   september 2015 semesterAcc30205 new course outline   september 2015 semester
Acc30205 new course outline september 2015 semester
 
Tbl, pbl, ebl, scale up, buzz, virtual or what?
Tbl, pbl, ebl, scale up, buzz, virtual or what?Tbl, pbl, ebl, scale up, buzz, virtual or what?
Tbl, pbl, ebl, scale up, buzz, virtual or what?
 
Teamwork seda may 2018
Teamwork seda may 2018Teamwork seda may 2018
Teamwork seda may 2018
 

Mehr von Dublin City University

Mehr von Dublin City University (20)

The Future Teacher
The Future TeacherThe Future Teacher
The Future Teacher
 
Text matching - Carrot or Stick approach
Text matching - Carrot or Stick approachText matching - Carrot or Stick approach
Text matching - Carrot or Stick approach
 
Reflections on spending other peoples money edtech 2018
Reflections on spending other peoples money   edtech 2018Reflections on spending other peoples money   edtech 2018
Reflections on spending other peoples money edtech 2018
 
Using video to facilitate cpd in the workplace edtech 2018
Using video to facilitate cpd in the workplace   edtech 2018Using video to facilitate cpd in the workplace   edtech 2018
Using video to facilitate cpd in the workplace edtech 2018
 
Testing knowledge of one software using another! edtech 2018
Testing knowledge of one software using another!   edtech 2018Testing knowledge of one software using another!   edtech 2018
Testing knowledge of one software using another! edtech 2018
 
Citations for President's awards for excellence in Teaching & Learning - 2018
Citations for President's awards for excellence in Teaching & Learning -  2018Citations for President's awards for excellence in Teaching & Learning -  2018
Citations for President's awards for excellence in Teaching & Learning - 2018
 
Using Moodle and Mahara to promote academic integrity
Using Moodle and Mahara to promote academic integrity Using Moodle and Mahara to promote academic integrity
Using Moodle and Mahara to promote academic integrity
 
Handling Video submission with Moodle
Handling Video submission with MoodleHandling Video submission with Moodle
Handling Video submission with Moodle
 
Transforming education with technology citywest expo
Transforming education with technology    citywest expoTransforming education with technology    citywest expo
Transforming education with technology citywest expo
 
Erasmus+ Presentation - The integrity project
Erasmus+ Presentation - The integrity projectErasmus+ Presentation - The integrity project
Erasmus+ Presentation - The integrity project
 
Turning off TurnItIn
Turning off TurnItInTurning off TurnItIn
Turning off TurnItIn
 
AAIM project Advanced Analytics in Mahara
AAIM project   Advanced Analytics in MaharaAAIM project   Advanced Analytics in Mahara
AAIM project Advanced Analytics in Mahara
 
citation and referencing - is our training making a difference
citation and referencing - is our training making a differencecitation and referencing - is our training making a difference
citation and referencing - is our training making a difference
 
Rethinking our approach to plagiarism
Rethinking our approach to plagiarismRethinking our approach to plagiarism
Rethinking our approach to plagiarism
 
Y1 feedback Symposium "Predicted" Presentation
Y1 feedback    Symposium "Predicted" PresentationY1 feedback    Symposium "Predicted" Presentation
Y1 feedback Symposium "Predicted" Presentation
 
Dstep project national forum presentation nov 2016 slide deck
Dstep project  national forum presentation nov 2016 slide deckDstep project  national forum presentation nov 2016 slide deck
Dstep project national forum presentation nov 2016 slide deck
 
Altc winter conference 2016 unicam presentation
Altc winter conference 2016   unicam presentationAltc winter conference 2016   unicam presentation
Altc winter conference 2016 unicam presentation
 
What Works & Why
What Works & Why What Works & Why
What Works & Why
 
New approaches to old learning spaces
New approaches to old learning spacesNew approaches to old learning spaces
New approaches to old learning spaces
 
Why is the vle not dead in dcu? edtech 2016
Why is the vle not dead in dcu? edtech 2016Why is the vle not dead in dcu? edtech 2016
Why is the vle not dead in dcu? edtech 2016
 

Kürzlich hochgeladen

Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
KarakKing
 

Kürzlich hochgeladen (20)

General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - English
 
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfUnit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
 
21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptx21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptx
 
Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
 
Interdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxInterdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptx
 
Basic Intentional Injuries Health Education
Basic Intentional Injuries Health EducationBasic Intentional Injuries Health Education
Basic Intentional Injuries Health Education
 
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
 
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdfUGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
UGC NET Paper 1 Mathematical Reasoning & Aptitude.pdf
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptxHMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
 
Single or Multiple melodic lines structure
Single or Multiple melodic lines structureSingle or Multiple melodic lines structure
Single or Multiple melodic lines structure
 
Exploring_the_Narrative_Style_of_Amitav_Ghoshs_Gun_Island.pptx
Exploring_the_Narrative_Style_of_Amitav_Ghoshs_Gun_Island.pptxExploring_the_Narrative_Style_of_Amitav_Ghoshs_Gun_Island.pptx
Exploring_the_Narrative_Style_of_Amitav_Ghoshs_Gun_Island.pptx
 
Sociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning ExhibitSociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning Exhibit
 
latest AZ-104 Exam Questions and Answers
latest AZ-104 Exam Questions and Answerslatest AZ-104 Exam Questions and Answers
latest AZ-104 Exam Questions and Answers
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 
Understanding Accommodations and Modifications
Understanding  Accommodations and ModificationsUnderstanding  Accommodations and Modifications
Understanding Accommodations and Modifications
 

More than the metrics - SEDA 2018

  • 1. Find us on Twitter: @glynnmark @clare_gormley @LostelloeFind us on Twitter: @glynnmark @clare_gormley @Lostelloe More Than The Metrics Our deep dive into assessment workload Laura Costelloe, Mark Glynn, Clare Gormley
  • 2. Overall Project Goal To profile summative continuous assessment workloads (staff and student) during one semester (Semester 2, 2017/18 academic year) To understand and improve the student experience on the basis of evidence
  • 3. What does the literature tell us? - “Assessment is probably the most important determinant of the character of a course and of student learning within the course” (Scott, 2015); - A modular course structure and largely unregulated numbers of internal assessments has resulted in an assessment “arms race” (Harland et al, 2015); - Students will generally say they are over-worked - but it’s important to have conversations around expectations (Fielding, 2008);
  • 4. What metrics should we use? - There are methodological challenges with measuring assessment workload, e.g. what metrics should be used? should we include both formative and summative assessment tasks? How should we classify different assessment types? - Scott (2015) considers the benefits of examining the following: number of assessment tasks; word count and equivalences; student time-on-task; time involved in designing an assessment task and in marking - Ultimately, “there are no simple answers as to how student assessment workload should be measured and standardised” (Fielding, 2008)
  • 5. Methodology - 5 programmes from 5 faculties investigated (Humanities, Science, Education, Engineering, Business) - Initial data drawn from centralised Course Information system - Staff questionnaire + interview (11 lecturers) - Focus on perceptions of workload + time on design, marking, feedback - In-class student surveys (8 modules, n=129 student respondents) - Focus on time on task
  • 6. Padlet Q 1 What are the challenges of measuring the assessment workload of students and staff? https://padlet.com/clare_gormley/q1
  • 7. DCU Business School: Number of assessments 1 2 3 4 4 5 6 7 1 2 3 4 5 6 7 8 9 10 11 12 9 Assessments in Total (excluding attendance/participation) Week Numberofassessments 55% of assessments due in the same week. or 77% in the space of two weeks 13
  • 8. Legend Presentation Essay / Case Study Reflection Poster MCQ Participation/Attendance Individual Group
  • 9. 5 10 15 20 25 30 35 40 1 2 3 4 5 6 7 8 9 10 11 12 Presentation Essay / Case Study Reflection Poster MCQ Participation/Attendance Week %ofmarktowardsfinalmark 13 DCU Business School: Type and weighting
  • 10. DCU Business School: Type and weighting Presentation Essay / Case Study Reflection Poster MCQ / Short answer Participation/Attendance %ofmarktowardsfinalmark 5 10 15 20 25 30 35 40 1 2 3 4 5 6 7 8 9 10 11 12 Week 13 50 60 70 80 90 100 E E E E EE E
  • 11. Faculty of Science and Health: Number of assessments 1 2 3 4 4 5 6 7 1 2 3 4 5 6 7 8 9 10 11 12 22* Assessments in Total (excluding attendance/participation) Week Numberofassessments 23% of assessments due in the same week. or 36% in the space of two weeks
  • 12. Faculty of Science and Health: Type and weighting Presentation Essay / Case Study Reflection Poster MCQ / Short answer Participation/Attendance 5 10 15 20 25 30 35 40 1 2 3 4 5 6 7 8 9 10 11 12 Week %ofmarktowardsfinalmark 50 13 60 E E E E
  • 14. Key Findings - Student Perspective -Multiple CAs are clustering around same time, typically towards end of semester -Notably majority of students spend ≤ 1 day on assessment preparation, regardless of weighting -As expected, there are substantial variations in student time-on-task across all assessments eg For one group presentation task, data shows - 44% of students spend ≤ 1 day - 24% of students spend ≥ 3 weeks
  • 15. What students are saying... I had an Oral CA the same week as [Module X] CA. Had many assessments and exams at this time, was overwhelmed and exhausted Normally I would spend more time but there was a lot of work due around the same time, so, as this was a mock- exam, it was not my priority Lot of CAs towards end of term
  • 16. Key Findings - Staff Perspective -Thoughtful consideration of own module workload but generally unaware of assessments elsewhere (some exceptions) -Aware of insufficient communication/visibility across the entire programme -Variations in marking and feedback time - feedback available but often unavailed of; lecturers not strongly influenced by design, marking and feedback time -Aware of group work free-rider issue and possible implications for student workload
  • 17. What staff are saying... I don’t know how much lecturers know about how much is going on in the bigger picture for students because we just tend to be responsible for [our own] modules. The majority of the content (90%) is done in class so when they go home, if they spend 10-15 minutes finishing up every week, that will be it. If they let it go, then it becomes a massive burden. Within the past few years, I set up this Google Drive spreadsheet so that for all the modules all the staff can see how and when they are being assessed. We try not to have everything due in week 12, for example... That has definitely helped. I would think that a student who embraces every single week and does the assigned reading every week has a fairly hefty workload. But I imagine that most students don’t do that... They can choose how heavy to make their workload but it is difficult to gauge it. I don’t think that they are over assessed within this module. But I am always wary, when you are doing any form of group work, that there is that potential for some people to have more of an assessment workload than others simply because they have to carry the slack for people who don’t equally contribute.
  • 18. Padlet Q 2 How does your institution currently compare workloads of different modes of assessment across disciplines? https://padlet.com/clare_gormley/q2
  • 19. Recommendations for similar studies Go for it.. but be aware of ethical considerations and possible reluctance to participate (Programme Chair may help with buy-in.) (Re)Consider the word count question Ask staff what amount of time they would expect students to spend on their assessments Clarify assessment info from centralised system before approaching students (esp regarding assessment type categorisation) Provide hard and online copies of survey - we received more comments on printed copies Ask students the Why? question regarding their time spent
  • 20. Padlet Q 3 How should we engage with staff to take a more programmatic view of assessment? https://padlet.com/clare_gormley/q3
  • 23. Bibliography Fielding, A. (2008) ‘Student assessment workloads: a review’, Learning and Teaching in Action 7, 3: 7-15. Harland, T., McLean, A., Wass, R., Miller, E and Sim, K. N. (2015) ‘An assessment arms race and its fallout: high-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education, 40(4): 528- 541. Scott, S.V. (2015) ‘Quantifying the assessment loads of students and staff: the challenge of selecting appropriate metrics’, Journal of Further and Higher Education, 39(5): 699-712.
  • 25. Programme Calendar - Moodle plugin - Tag with programme code - Viewable to relevant staff - Combine with “my feedback” plugin - Beta test : September 2018
  • 26. Programme in Faculty of Engineering and Computing: Type and weighting 5 10 15 20 25 30 35 40 1 2 3 4 5 6 7 8 9 10 11 12 Presentation Essay / Case Study Reflection Poster MCQ Participation/Attendance Week %ofmarktowardsfinalmark Labs