This document provides an overview of the TESTA (Transforming the Experience of Students through Assessment) research project. It discusses key findings from auditing assessment practices across various university programmes. Some programmes had clear goals and feedback that drove student effort, while others lacked clarity and feedback. The research found formative assessment was underused and feedback was often untimely and disjointed. TESTA cases studies showed how increasing formative work and dialogue about standards can boost learning. Overall, the project revealed assessment patterns influence student experience and outcomes significantly.

1. Programme Assessment:
Insights about student learning
from TESTA
Dr Tansy Jessop, TESTA Project Leader
Presentation to the SDG Course Leaders
University of West of Scotland
12 May 2014

2. 1) Assessment drives what students pay
attention to, and defines the actual
curriculum (Ramsden 1992).
2) Feedback is significant (Hattie, 2009; Black
and Wiliam, 1998)
3) Programme is central to influencing change.
TESTA premises

3. Thinking about modules
modulus (Latin): small measure
“interchangeable units”
“standardised units”
“sections for easy constructions”
“a self-contained unit”

4. How well does IKEA 101 packaging
work for Sociology 101?
Furniture
 Bite-sized
 Self-contained
 Interchangeable
 Quick and instantaneous
 Standardised
 Comes with written
instructions
 Consumption
Student Learning
 Long and complicated
 Interconnected
 Distinctive
 Slow, needs deliberation
 Varied, differentiated
 Tacit, unfathomable,
abstract
 Production

5.  HEA funded research project (2009-12)
 Seven programmes in four partner universities
 Maps programme-wide assessment
 Engages with Quality Assurance processes
 Diagnosis – intervention – cure
What is TESTA?
Transforming the Experience of Students through Assessment

6. TESTA ‘Cathedrals Group’ Universities

7. Edinburgh
Edinburgh Napier
Greenwich
Canterbury Christchurch
Glasgow
Lady Irwin College University of Delhi
University of West Scotland
Sheffield Hallam

8. TESTA
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs

9.  Time-on-task
 Challenging and high expectations
 Students need to understand goals and standards
 Prompt feedback
 Detailed, high quality, developmental feedback
 Dialogic cycles of feedback
 Deep learning – beyond factual recall
Based on assessment principles

10. TESTA Research Methods
(Drawing on Gibbs and Dunbar-Goddet, 2008,2009)
ASSESSMENT
EXPERIENCE
QUESTIONNAIRE
FOCUS GROUPS
PROGRAMME AUDIT
Programme
Team
Meeting

11.  Number of assessment tasks
 Summative/formative
 Variety
 Proportion of exams
 Oral feedback
 Written feedback
 Speed of return of feedback
 Specificity of criteria, aims and learning outcomes.
Audit in a nutshell

12.  Quantity of Effort
 Coverage of content and knowledge
 Clear goals and standards
 Quantity and Quality of Feedback
 Use of feedback
 Appropriate assessment
 Learning from exams
 Deep and surface learning
Assessment Experience
Questionnaire

13. Focus Groups
 Student voice and narrative
 Explanation
 Corroboration & contradiction
 Compelling evidence with the stats

14. Case Study X: what’s going on?
 Mainly full-time lecturers
 Plenty of varieties of assessment, no exams
 Reasonable amount of formative assessment (14 x)
 33 summative assessments
 Masses of written feedback on assignments (15,000 words)
 Learning outcomes and criteria clearly specified
….looks like a ‘model’ assessment environment
But students:
 Don’t put in a lot of effort and distribute their effort across few topics
 Don’t think there is a lot of feedback or that it very useful, and don’t
make use of it
 Don’t think it is at all clear what the goals and standards are
 …are unhappy

15. Case Study Y: what’s going on?
 35 summative assessments
 No formative assessment specified in documents
 Learning outcomes and criteria wordy and woolly
 Marking by global, tacit, professional judgements
 Teaching staff mainly part-time and hourly paid
….looks like a problematic assessment environment
But students:
 Put in a lot of effort and distribute their effort across topics
 Have a very clear idea of goals and standards
 Are self-regulating and have a good idea of how to close the gap

16. Two paradigms…

17. Transmission Model

18. Social Constructivist model

19.  Audit
 AEQ
 Focus Groups
Research Methods

20. 1) Any interesting patterns?
2) Anything particularly striking?
3) Any dangling questions, curiosities,
scepticisms?
4) Any predictions, hunches, thoughts about
what other data might throw up?
Task 1: Audit data

21. Audit Variables Humanities
(Mean)
Sciences
(Mean)
Professional
(Mean)
Total number of assessments 54 67 42
Number of summative assessments 42 43 32
Number of formative assessments 12 31 10
Varieties of assessment (n) 11 15 14
Proportion of examinations 14.3% 30.6% 15%
Time to return feedback 23 days 20 days 23 days
Amount of oral feedback 3hrs 17mins 4hrs 56mins 10hrs 33mins
Amount of written feedback 7,382 words 3,615 words 7,040 words
Discipline Effects

22. a) Fill in an AEQ from the vantage point of being a
student in the past, or one of your own students
b) Cluster the questions into scales using the
Scales Exercise.
c) What’s a ‘good’ score?
d) Any thoughts, issues, comments, questions?
Task 2: The AEQ

23. In pairs, explore programme audit and AEQ
data from one programme.
Triangulation: Audit and AEQ data

24.  In pairs/groups, read through quotes from student
focus group data on a particular theme.
 What problems does the data imply?
 What solutions might a programme develop to
address some of these challenges?
 A3 sheets provided to tease out challenges and
solutions.
Focus Group data

25. Challenges Solutions
Student voice data

26.  If there weren’t loads of other assessments, I’d do it.
 If there are no actual consequences of not doing it, most
students are going to sit in the bar.
 I would probably work for tasks, but for a lot of people, if
it’s not going to count towards your degree, why bother?
 The lecturers do formative assessment but we don’t get
any feedback on it.
Theme 1: Formative is a great idea
but…

27. We could do with more assessments over the course of the year
to make sure that people are actually doing stuff.
We get too much of this end or half way through the term essay
type things. Continual assessments would be so much better.
So you could have a great time doing nothing until like a month
before Christmas and you’d suddenly panic. I prefer steady
deadlines, there’s a gradual move forward, rather than bam!
Theme 2: Assessment isn’t driving
and distributing student effort

28.  The feedback is generally focused on the module.
 It’s difficult because your assignments are so detached from
the next one you do for that subject. They don’t relate to each
other.
 Because it’s at the end of the module, it doesn’t feed into our
future work.
 You’ll get really detailed, really commenting feedback from
one tutor and the next tutor will just say ‘Well done’.
Theme 3: Feedback is disjointed
and modular

29.  The assessment guidelines is a formal document so the language
is quite complex and I’ve had to read it a good few times to kind
of understand what they are saying.
 Assessment criteria can make you take a really narrow approach.
 They read the essay and then they get a general impression, then
they pluck a mark from the air.
 It’s a shot in the dark.
 We’ve got two tutors – one marks completely differently to the
other and it’s pot luck which one you get.
Theme 4: Students are not clear
about goals and standards

30. 1. Too much summative; too little formative
2. Too wide a variety of assessment
3. Lack of time on task
4. Inconsistent marking standards
5. ‘Ticking’ modules off
6. Poor feedback: too little and too slow
7. Lack of oral feedback; lack of dialogue about standards
8. Instrumental reproduction of materials for marks
Main findings

31. 1. Students and staff can’t do more of both.
2. Reductions in summative – how many is enough?
3. Increase in formative – and make sure it is valued and
required.
4. Debunking the myth of two summative per module.
5. Articulating rationale with students, lecturers, senior
managers and QA managers.
1. Summative-formative issues

32. The case of the under-performing engineers (Graham,
Strathclyde)
The case of the cunning (but not litigious) lawyers (Graham,
somewhere)
The case of the silent seminar (Winchester)
The case of the lost accountants (Winchester)
The case of the disengaged Media students (Winchester)
1. Examples of ramping up formative

33. The case of low effort on Media Studies
The case of bunching on the BA Primary
2. Examples of improving ‘time on task’

34. The case of the closed door (Psychology)
The case of the one-off in History (Bath Spa)
The case of the Sports Psychologist (Winchester)
The conversation gambit
3. Engaging students in reflection
through improving feedback

35.  The case of the maverick History lecturer (a dove)
 The case of the highly individualistic creative
writing markers
4. Internalising goals and standards

36. Programmatic Assessment Design
Feedback Practice
Paper processes to people talking
Changes

37.  Improvements in NSS scores on A&F – from bottom
quartile in 2009 to top quartile in 2013
 Three programmes with 100% satisfaction ratings post
TESTA
 All TESTA programmes have some movement upwards
on A&F scores
 Programme teams are talking about A&F and pedagogy
 Periodic review processes are changing for the better.
Impacts

38. www.testa.ac.uk

39. Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning.
Learning and Teaching in Higher Education. 1(1): 3-31.
Gibbs, G. & Dunbar-Goddet, H. (2009). Characterising programme-level assessment environments
that support learning. Assessment & Evaluation in Higher Education. 34,4: 481-489.
Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112.
Jessop, T. and Maleckar, B. (in press). The Influence of disciplinary assessment patterns on student
learning: a comparative study. Studies in Higher Education.
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-
scale study of students’ learning in response to different assessment patterns. Assessment and
Evaluation in Higher Education. 39(1) 73-88.
Jessop, T, McNab, N & Gubby, L. (2012) Mind the gap: An analysis of how quality assurance processes
influence programme assessment patterns. Active Learning in Higher Education. 13(3). 143-154.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517
Sadler, D.R. (1989) Formative assessment and the design of instructional systems, Instructional
Science, 18, 119-144.
References