TESTA at UNSW, Sean Brawley, TESTA Summit 16 Sept 2013
•Als PPTX, PDF herunterladen•
1 gefällt mir•459 views
TESTA is Faculty of Arts and Social Sciences. Efficiency in Assessment and Feedback. Data-driven approach to fast-tracking quality assurance to make responsive changes to assessment.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie TESTA at UNSW, Sean Brawley, TESTA Summit 16 Sept 2013
Ähnlich wie TESTA at UNSW, Sean Brawley, TESTA Summit 16 Sept 2013 (20)
Mehr von Tansy Jessop
Mehr von Tansy Jessop (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
TESTA at UNSW, Sean Brawley, TESTA Summit 16 Sept 2013
- 1. The UNSW experience with TESTA Faculty of Arts and Social Sciences Professor Sean Brawley Associate Dean (Education)
- 3. UNSW Assessment Project Goals 1. Improve efficiency in the conduct of assessment in order to reduce academic workloads and ensure that student’s time on tasks is not spent on unnecessary activities, which have the sole purpose of generating a grade. 2. Improve the educational effectiveness of assessment tasks to ensure that they engage and motivate students in their learning efforts.
- 4. 3. Increase the alignment of assessment tasks with the disciplinary goals and the learning outcomes of individual courses such that student efforts are directed towards achieving the key learning objectives. 4. Improve the alignment of assessment activity and processes across the faculty. 5. Increase the congruence between Faculty assessment processes and practices and the University’s assessment policies, procedures and aspirations.
- 5. Assessment Tasks in H&P Final Exam 10% Tut Particip/Prep 11% Individual Present 3% Tests/Quizzes 8% Group Work 3% Short writing 11% Tut Writing 18% Extended Writing 36%
- 6. Assessment Tasks in EMPA
- 7. UNSW HP assessments tasks against those of the University of Sydney, University of Melbourne, University of Nottingham, George Mason University and UCLA combined. 50 45 40 35 30 25 UNSW Others 20 15 10 5 0 Extended Writing Short Writing Exams Tut particip Indiv Present Group Work Tests/Quizzes
- 8. No. Assessment Tasks per Course 8 7 6 5 4 3 2 1 0 UNSW USYD MELB NOTT UCLA GM
- 9. Amount of words per level 6000 5000 4000 Level 1 3000 Level 2 Level 3 2000 1000 0 USYD MELB GM NOTT
- 10. FASS Assessment Project Faculty Working Party Asses. Tool Prototype Student Time on Task Trial Trial S1, 2012 15 courses 1000+ students 53 staff diaries 170 student diaries Efficiency / Quality Assurance Innovation / Quality Improvement Faculty Assessment Tool TESTA 1700 Students Complete AEQ 60 Students Focus Groups • Incorporates GA Tool • Operational S2, 2012 • Aids governance, • Mainstreams innovation • Self-pop course guides Reduce average student time on task from 114 to 85 hours Establish staff marking expectations (time/words per student)
- 12. Workshop Feedback • “Interesting project that will enable innovation in both efficiency and pedagogy” • “The work shop was much more useful than I thought it was going to be …the information about TESTA has reassured me that there will be a quality/pedagogy component to the exercise, rather than only a resources/efficiency drive. I now have a better idea how TESTA brings the student voice to the fore, and think this is a valuable undertaking”
- 13. Workshop Feedback • “Thanks for the opportunity to do this workshop. Ive learned much and realise how I can improive my own practice and help steer colleagues in productive directions” • “The challenge is how to service implementation with very limited resources and competing demands and performance expectations … Positive education frame of reference is a good place to start … with a proven model of change. Thanks for a good experience in collective problem solving”
- 14. Workshop Feedback • “I fear that everyone knows that ultimately what we decide will have very little impact on decisions that are made higher up, and that [these decisions] are not based on pedagogical considerations. On that basis, its very frustrating … The 2 days were really interesting for me. I do think, however, that the major purpose for which it ran, is unlikely to be achieved. It is difficult not to be cynical about the pressures coming from above — and they are not pedagogical”
- 15. Workshop Feedback “Learnt heaps — so good to get an opportunity to discuss assessment” “TESTA sessions did not inspire enthusiasns when I said I would attend but I have been pleasantly surprised by how useful learning about the system and process has been. I now want to revisit the program I convene — this would be so useful for anyone who developing a course or program”
- 16. Workshop Feedback • “I thought the TESTA methodology was very good and thorough but not realistic for the time and resource constraints that UNSW operates under”
- 17. Assessment Experience Questionnaire • Slight changes to some of the 28 AEQ questions
- 18. “The teachers made it clear from the start what they expected from students” Became “My lecturers/tutors made it clear from the start what they expected from students”
- 19. “It was often hard to discover what was expected of me in this course” Became “It was often hard to discover what was expected of me in my major/program”
- 20. “I learnt new things while preparing for the exams” Became “I learnt new things while preparing for the exams/class tests”
- 21. Assessment Experience Questionnaire • Slight changes to some of the 28 AEQ questions • Online instrument
- 22. Assessment Experience Questionnaire • Slight changes to some of the 28 AEQ questions • Online instrument • Explode email to nearly five thousand undergraduate students
- 25. Assessment Experience Questionnaire • Slight changes to some of the 28 AEQ questions • Online instrument • Explode email to nearly five thousand undergraduate students • 1820 responded (37%)
- 26. AEQ responses by Program
- 27. Demographics • 85% of students domestic, 15% international – would rep the average • Age 17-21 — 55% 22-30 — 37% 31 and over 6% • Gender 72% were female (higher than Faculty average) • Level 100 level — 28% 200 level — 22% 300 level — 27% 400 level above — 21% (greater than average) • GPA pass average 16% credit average 48% a distinction average 31% 4% high distinction average 1% failing average
- 28. 1: I used the feedback I received to go back over what I had done in my work 1: I used the feedback I received to go back over what I had done in my work 2% Strongly Disagree 2% Strongly Disagree 7% Disagree 7% Disagree 21% Neutral 21% Neutral 56% Agree 56% Agree 15% Strongly Agree 15% Strongly Agree 0% 10% 0% 20% 10% Strongly Agree 30% 20% Strongly Agree Agree Agree 30% Neutral Neutral 40% Disagree Disagree 40% Strongly Disagree Strongly Disagree 50% 50% 60% 60%
- 29. 3: I received hardly any feedback on my work 5: The assessment system made it possible to be quite selec ve about what parts of courses you studied 10% Strongly Disagree 45% Strongly Disagree Disagree 3% 20% Disagree 23% Neutral 28% Neutral 16% Agree 43% Agree 5% Strongly Agree 6% Strongly Agree 0% 5% 0% 10% 5% 15% 10% Strongly Agree 20% 15% Strongly Agree Agree Agree 25% 20% Neutral Neutral 30% 25% Disagree Disagree 35% 30% 40% 35% Strongly Disagree Strongly Disagree 45% 40% 50% 45%
- 30. 5: The assessment system made it possible to be quite selec ve about what parts of courses you studied 5: The assessment system made it possible to be quite selec ve about what parts of courses you studied 3% Strongly Disagree 3% Strongly Disagree 20% Disagree 20% Disagree 28% Neutral 28% Neutral 43% Agree 43% Agree 6% Strongly Agree 6% Strongly Agree 0% 5% 0% 10% 5% 15% 10% Strongly Agree 20% 15% Strongly Agree Agree Agree 20% Neutral Neutral 25% 25% Disagree Disagree 30% 30% 35% 35% Strongly Disagree Strongly Disagree 40% 40% 45% 45%
- 31. 11: It was possible to be quite strategic about which topics you could affo d r not to study 11: It was possible to be quite strategic about which topics you could affo d r not to study 3% Strongly Disagree 3% Strongly Disagree 25% Disagree 25% Disagree 34% Neutral 34% Neutral 35% Agree 35% Agree 3% Strongly Agree 3% Strongly Agree 0% 5% 0% 10% 5% 10% Strongly Agree Strongly Agree 15% 15% Agree Agree 20% 20% Neutral Neutral 25% Disagree Disagree 25% 30% 30% Strongly Disagree Strongly Disagree 35% 35% 40% 40%
- 32. 17: The way the assessment worked in this major/program meant you had to study every topic 17: The way the assessment worked in this major/program meant you had to study every topic 4% Strongly Disagree 4% Strongly Disagree 34% Disagree 34% Disagree 30% Neutral 30% Neutral 28% Agree 28% Agree 5% Strongly Agree 5% Strongly Agree 0% 5% 0% 10% 5% 10% Strongly Agree Strongly Agree 15% Agree Agree 15% Neutral Neutral 20% 20% Disagree Disagree 25% 25% Strongly Disagree Strongly Disagree 30% 30% 35% 35%
- 33. 7: It was always easy to know the standard of work expected 5: The assessment system made it possible to be quite selec ve about what parts of courses you studied 6% Strongly Disagree 22% Strongly Disagree Disagree 3% 20% Disagree 25% Neutral 28% Neutral 41% Agree 43% Agree 6% Strongly Agree 6% Strongly Agree 0% 5% 0% 10% 5% 15% 10% Strongly Agree 20% 15% Strongly Agree Agree Agree 20% Neutral Neutral 25% 25% Disagree Disagree 30% 30% 35% 35% Strongly Disagree Strongly Disagree 40% 40% 45% 45%
- 34. 10: Thesacademic staff e emed more interested in tes ng what I had memorised than what I understood 10: Thesacademic staff e emed more interested in tes ng what I had memorised than what I understood 16% Strongly Disagree Strongly Disagree Disagree 16% 46% 46% Disagree 19% Neutral 19% Neutral 16% Agree 16% Agree 3% Strongly Agree 3% Strongly Agree 0% 5% 0% 10% 5% 15% 10% 15% Strongly Agree Strongly Agree 20% 20% Agree Agree 25% 25% Neutral Neutral 30% 30% Disagree Disagree 35% 35% Strongly Disagree Strongly Disagree 40% 40% 45% 45% 50% 50%
- 35. 16: Whatever feedback I received on my work came too late to be useful 16: Whatever feedback I received on my work came too late to be useful 6% Strongly Disagree 6% Strongly Disagree 32% Disagree 32% Disagree 29% Neutral 29% Neutral 25% Agree 25% Agree 7% Strongly Agree 7% Strongly Agree 0% 5% 0% 10% 5% 10% Strongly Agree Strongly Agree 15% Agree Agree 15% Neutral Neutral 20% 20% Disagree Disagree 25% 25% Strongly Disagree Strongly Disagree 30% 30% 35% 35%
- 36. 26: I learnt new things while preparing for the exams/class tests 26: I learnt new things while preparing for the exams/class tests 4% Strongly Disagree 4% Strongly Disagree 15% Disagree 15% Disagree 29% Neutral 29% Neutral 42% Agree 42% Agree 9% Strongly Agree 9% Strongly Agree 0% 5% 0% 10% 5% 15% 10% Strongly Agree 20% 15% Strongly Agree Agree Agree 20% Neutral Neutral 25% 25% Disagree Disagree 30% 30% 35% 35% Strongly Disagree Strongly Disagree 40% 40% 45% 45%
- 37. 27: I understood things be er as a result of the exams/class tests 27: I understood things be er as a result of the exams/class tests 6% Strongly Disagree 6% Strongly Disagree 18% Disagree 18% Disagree 36% Neutral 36% Neutral 33% Agree 33% Agree 7% Strongly Agree 7% Strongly Agree 0% 5% 0% 10% 5% 10% Strongly Agree Strongly Agree 15% 15% Agree Agree 20% 20% Neutral Neutral 25% Disagree Disagree 25% 30% 30% Strongly Disagree Strongly Disagree 35% 35% 40% 40%
- 38. 24: O en I found I had to study things without having a chance to really understand them 24: O en I found I had to study things without having a chance to really understand them 5% Strongly Disagree 5% Strongly Disagree 29% Disagree 29% Disagree 23% Neutral 23% Neutral 33% Agree 33% Agree 10% Strongly Agree 10% Strongly Agree 0% 5% 0% 10% 5% 10% Strongly Agree Strongly Agree 15% Agree Agree 15% Neutral Neutral 20% 20% Disagree Disagree 25% 25% Strongly Disagree Strongly Disagree 30% 30% 35% 35%
- 40. Feedback
- 51. 4: You had to study the en re syllabus to do well in the assessment comparison of language major student responses to overall fig res u 4: You had to study the en re syllabus to do well in the assessment comparison of language major student responses to overall fig res u 1% Strongly Disagree 5% 1% 12% Strongly Disagree Disagree 5% 33% 12% Disagree 31% 33% Neutral 28% 31% Neutral 40% Agree 27% Agree 28% 40% 27% 15% Strongly Agree 7% Strongly Agree 15% 7% 0% 5% 0% 10% 5% 10% 15% 20% 15% 20% Language major streams Language major streams 25% 25% Overall Overall 30% 30% 35% 35% 40% 40% 45% 45%
- 52. 17: The way the assessment worked in this major/program meant you had to study every topic - comparison of language major student responses to overall figures 17: The way the assessment worked in this major/program meant you had to study every topic - comparison of language major student responses to overall figures 0% Strongly Disagree 4% 0% 18% Strongly Disagree Disagree 4% 34% 18% Disagree 28% 34% Neutral 30% 28% Neutral 41% 30% Agree 28% Agree 41% 28% 13% Strongly Agree 5% Strongly Agree 13% 5% 0% 5% 0% 10% 5% 10% 15% 20% 15% 20% Language major streams Language major streams 25% 25% Overall Overall 30% 30% 35% 35% 40% 40% 45% 45%
- 53. 18: To do well in the major/program all you really needed was a good memory - comparison of language major student responses to overall u fig res 18: To do well in the major/program all you really needed was a good memory - comparison of language major student responses to overall u fig res 13% Strongly Disagree 19% 13% 33% Strongly Disagree Disagree 19% 41% 33% Disagree 27% 41% Neutral 21% 27% Neutral 16% Agree 14% Agree 21% 16% 14% 11% Strongly Agree 5% Strongly Agree 11% 5% 0% 5% 0% 10% 5% 10% 15% 20% 15% 20% Language major streams Language major streams 25% 25% Overall Overall 30% 30% 35% 35% 40% 40% 45% 45%
- 54. 4: You had to study the en re syllabus to do well in the assessment comparison of Humani es student responses to overall fig res u 5% Strongly Disagree 4: You had to study the en re syllabus to do well in the assessment comparison of Humani es student responses to overall fig res u 5% 5% 40% Strongly Disagree Disagree 5% 33% 40% Disagree 29% 33% Neutral 28% 29% Neutral 21% Agree 27% Agree 28% 21% 27% 6% Strongly Agree 7% Strongly Agree 6% 7% 0% 5% 0% 10% 5% 10% 15% 20% 15% Humani es Humani es 20% Overall Overall 25% 25% 30% 30% 35% 35% 40% 40%
- 55. 25: Doing exams/class tests brought things together for me - comparison of Humani es student responses to overall figr es u 25: Doing exams/class tests brought things together for me - comparison of Humani es student responses to overall figr es u 11% Strongly Disagree 8% 11% 33% Strongly Disagree Disagree 8% 21% 33% Disagree 30% 21% Neutral 37% 30% Neutral 24% Agree 31% Agree 37% 24% 31% 1% Strongly Agree 3% Strongly Agree 1% 3% 0% 5% 0% 10% 5% 10% 15% 20% 15% Humani es Humani es 20% Overall Overall 25% 25% 30% 30% 35% 35% 40% 40%
Hinweis der Redaktion
- Foundation member Group of Eight Research intensive52K student population85th Times Higher Ed Rankings52nd QS World Rankings
- First stage – stocktaking and bencmarking
- The broader projectHow we came to TESTA — Paul Hyland
- Responbdents chance to win 10 shopping vouchers valued at $50 or an IPad
- 4913 undergrads, 999 postgrad coursework then HDR
- 32% of students from the BA30% of students provided additional comments — may be less than if we had done paper but probably got more responses85% of students domestic, 15% international – would rep the average55% 17-2137% 22-306% 31 and over72% were female (higher than Faculty average)28% 100 level22% 200 level27% 300 level12% 400 level (greater than average)9% 500 level (Eductaion combined degrees)16% pass average48% credit average31% a distinction average (not the case!)4% high distinction average (slightly under)1% failing averageArts students 32%
- 85% of students domestic, 15% international – would rep the average72% were female (higher than Faculty average) 400 level above — 21% (greater than average)a distinction average 31% (not the case!)4% high distinction average (slightly under)
- Things I feared
- Disappointed
- disappointed
- Surprised
- pleased
- Provoked reflection
- Bit of a concern.
- Statement 24 (‘Often I found I had to study things without having a chance to really understand them’) is concerning, with more students agreeing with this than disagreeing (43% compared to 34%). Even more Level 1 students agreed with this (50% strongly agree/agree with only 25% strongly disagree or disagree).
- Good students did use feedback.
- Students who do not achieve well in the program do not perceive they have received feedback on their work as much as those that do well. Is this due to differing feedback levels? Or with difficulties in appreciating/utilising the feedback they have been given?
- Mature age students appear less strategic in their approach to assessment than do younger students. Particularly true for those older students who are in 100 level (see below).
- International students also felt they needed to study the entire syllabus more than domestic students.
- Mature age students felt the need to put the hours in regularly.
- International students appear to have felt they got more out of exams than domestic students did.
- Not much difference
- Felt studying entire syllabus was very important, much more agreement than for students overall (4)
- Also felt that assessments meant they had to study every topic, more important than for students overall (17)
- Linked to this, more felt that a good memory was important than overall (18)
- Did not feel entire syllabus needed to be studied as much as other students – more strategic learning perhaps (4)
- Exams were less important in bringing coherence to the course than it was for other students (25)
- Qual feedback very useful
- 62 students in 12 focus groupsInteresting data
- Provide some headlines for Schools