Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie App pgr workshop3
Ähnlich wie App pgr workshop3 (20)
Mehr von sarahattersley
Mehr von sarahattersley (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
App pgr workshop3
- 1. APP PGR: Workshop 3 Assessment and Feedback; Planning and evaluation
- 2. Morning! Grab a coffee/tea and write your name on a large post-it note. Talk to your APP PGR peers about… (if you can’t remember what that is, look it up…..!)
- 3. Learning outcomes To identify key themes in successful planning for teaching and learning, including constructive alignment, assessment and feedback and their relevance to your current work. To critically consider the place and modes of assessment and feedback in HE and in your subject discipline. To discuss real student feedback and collectively think through how that might be used.
- 6. The verb: Bloom’s taxonomy Define, duplicate, list, memorise, recall, repeat, state Classify, describe, discuss, explain, identify, locate, recognise, report, select, translate, paraphrase Choose, demonstrate, dramatise, employ, illustrate, interpret, operate, schedule, sketch, solve, use Appraise, compare, contrast, criticise, differentiate, distinguish, examine, experiment question, test Appraise, argue, defend, judge, select, support, value, evaluate Assemble, construct, create, design, develop, formulate, write
- 7. What’s the point? Writing intended learning outcomes By the end of the module you should be able to: (verb) (object) (context) Calculate statistical power and required sample size for situations that can be analysed using one or two sample t-tests. Demonstrate how your academic teaching practice can be informed by relevant educational and disciplinary research.
- 8. Assessment
- 9. screening initial diagnostic formative summative Different assessment types
- 10. Knowledge types Declarative knowledge Functioning knowledge Core knowledge Troublesome knowledge (Biggs and Tang, pp.81-87)
- 11. Troublesome knowledge “There are ‘conceptual gateways’ or ‘portals’ that lead to a previously inaccessible, and initially perhaps ‘troublesome’, way of thinking about something. A new way of understanding, interpreting, or viewing something may thus emerge – a transformed internal view of subject matter, subject landscape, or even world view.” (Meyer and Land 2005)
- 12. 6 tenets of assessment (HEA): promote assessment for learning; ensure assessment is fit for purpose; recognise that assessment lacks precision; standards constructed within communities; integrate assessment literacy into course design; ensure professional judgements are reliable.
- 13. Asking the right kind of questions Activity: what questions could you ask of these images which promote different levels of thinking skill?
- 15. • Wheeler et al (2005) discuss the notion of distributed problem-based learning online and ‘communities of practice’. • Discursive online dialogue can lead to a ZPD (Vygotsky, 1978) – an incremental shift of an individual’s cognitive development, through peer interactions. • DPBL can be achieved in synchronous and asynchronous environments. • Notions of online identity and its relationship to actual, professional identity. Wheeler, S (2005) The influence of online problem-based learning on teachers’ professional practice and identity ALT-J Research in Learning Technology Journal. v. 13. no. 2. 2005. p. 125 Problem-based learning online
- 16. Students say that… • knowing what is expected of them, through clear briefs and criteria enabled them to learn more effectively. • assessment is marked on implicit criteria • Student say that they don’t understand the criteria There are indications that… • students don’t fully understand what is meant by commonly used assessment terminology Understanding the assessment criteria
- 17. Feedback is “the most important aspect of the assessment process in raising achievement” (Bloxham and Boyd, 2007:7)
- 18. Seven principles of good feedback Good feedback practice: 1. helps clarify what good performance is (goals, criteria, expected standards) 2. facilitates the development of self-assessment (reflection) in learning 3. delivers high quality information to students about their learning 4. encourages teacher and peer dialogue around learning 5. encourages positive motivational beliefs and self-esteem 6. provides opportunities to close the gap between current and desired performance 7. provides information to teachers that can be used to help shape the teaching. Taken from: Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 34 (1), 199-218.
- 19. How good is your feedback? Activity: Using your own feedback to students, or an example provided, evaluate its quality against the seven principles described. • What is done well and is clear? How might this affect the student/recipient? •What is omitted or less successfully delivered? How might this affect the student/recipient? • What are the challenges in ensuring good quality feedback in your teaching context? • What alternative models might you use?
- 20. What Warwick students think about assessment and feedback BSc (Hons) Chemistry BEng (Hons) Automotive Engineering https://unistats.direct.gov.uk/
- 22. Recent research has found audio feedback to have advantages over written in terms of retention of content and likelihood of engagement. London Met University (Lunt and Curran, 2010) found audio feedback to be beneficial to both staff and students, with the latter 10 times more likely to read their feedback. Audio feedback can be achieved easily through podcast techniques or combined with written feedback through screencast approaches. Innovative approaches to giving feedback
- 23. Involving students Students can use their online voice for reciprocal teaching or to provide evidence of group/seminar activity, which you can listen to and assess later.
- 25. Session planning: considerations Time Learning environment Resources Timing Theme characteristics Feedback opportunities Student preparation Tutor notes Cohort characteristics Learning outcomes Module context Content Style of delivery Assessment Evaluation Activities
- 26. 10 specific planning questions 1. What is the overall purpose of this session? 2. Where does this session fit? 3. For whom is this module designed? 4. What specifically should students learn and be able to do by the end? 5. What standards will be used to assess their learning? 6. How will their learning be assessed? 7. What specific content will be taught and assessed? 8. What will motivate students to learn deeply and well? 9. What work will students do to learn? 10. What work will teachers and others do to help students learn? Angelo 2013: 101.
- 27. Evaluating your teaching: considerations WHY do you want to get feedback? WHAT do you want to get feedback on? WHEN is the best time to collect feedback? WHO will you get evidence from, and who is it for? HOW will you do it and how will you act on the outcome?
- 28. How can you best evaluate teaching?
- 29. What can we make of this?
- 30. • Share and reflect on feedback results, with students, if possible. • Examine any ‘mismatch’ between feedback and YOUR perception. • Act on feedback – show how you have or explain how you are going to. • Use feedback (good and bad) to grow the curriculum.
- 31. Acting on the feedback Spend time with the data, reflecting on whether it matches your expectations. Enable the ‘feedback loop’ to be completed by sharing findings with your students and colleagues. Draw together common themes/problems and seek actions “ You said; I did”. Explain where change isn’t possible. Consider feedback in relation to departmental context and NSS data.
- 32. Activity: Create a ‘wall’ of responses about the APP PGR by sticking post-it notes under the headings STOP, START and CONTINUE. What other mechanisms for in-class feedback could you use (think about the learning environment and resources around you)? Feedback exercise
- 33. Students learn better when they: they become aware of their own relevant prior knowledge, preconceptions, beliefs and values; set and maintain realistically high and personally meaningful learning goals and expectations for academic success; learn how to study and learn effectively; understand criteria, standards and methods used to assess them and how to use feedback on performance against those standards; collaborate regularly and effectively with other learners and their teachers to achieve shared, meaningful goals; invest adequate time and effort, effectively and efficiently in their academic work seek and find connections to and applications of the concepts and skills they are learning to their lives and work. Angelo, 2013, p.100.
- 34. Critical closing questions How far is your curriculum ‘constructively aligned’? Do students understand what they are being assessed on? Can you identify one (or more) teaching/learning methods or resources that you have used today that you could take forward into your own practice? What is your priority, for planning for learning?
Hinweis der Redaktion
- Biggs and Tang, 2007, pp. 52-54 Suggest that you start in a different place. Constructively align your module/learning activities/curriculum. CA has two aspects. The ‘constructive’ aspect refers to what the learner does, which is to construct meaning through relevant learning activities. Comes from constructivist learning theory. Predicated on notion that in order to learn the student needs to be active in their learning. The ‘alignment’ aspect refers to what the teacher does, which is to set up a learning environment that supports the learning activities appropriate to achieving the desired learning outcomes. The key is that the components in the teaching system, especially the teaching methods used and the assessment tasks, are aligned to the learning activities assumed in the intended outcomes. The learner is in a sense ‘trapped’, and finds it difficult to escape without learning what is intended should be learned. This covers alignment within a module, but in a well-designed curriculum individual modules are also aligned horizontally – relating to other modules that students are studying the same year across a programme – to avoid redundancy (students covering the same material or skills in more than one module). And vertically between years – previous educational experiences and looking forward to their next modules/work. Backward design – start at the end – defines what students must demonstrate that they know and can do (intended learning outcomes) and how well they know and can do it (standards) – then works backwards to determine how best to get there and where to start.
- Alignment: again. If your learning outcome is for them to assemble an All the verbs in some way address understanding – which is why using the verb ‘understanding’ isn’t helpful. It doesn’t tell your students how they are expected to understand Written to include an activity, not just declarative knowledge of a topic. Alignment is achieved by ensuring that the intended verb in the outcome statement is present in the teaching/learning activity, and in the assessment task.
- So, if the module aims are written from the perspective of the tutor – the learning outcomes are entirely directed at the student. Fetishisation of learning outcomes. Folded into the norms of quality rules have grown up. We all learn to write our learning outcomes, which are documented in module handbooks, but then which don’t trouble us or our students ever again. Consider subject knowledge, level of understanding that you want them to perform, key skills, cognitive skills (e.g. critical analysis, formulate and test concepts etc), core knowledge, threshold concepts, subject-specific, professional skills. All that is relevant to what you want them to learn during the module/learning activity. They should express the minimum requirement (standard) for the credit not the maximum goal. Should be vertically and horizontally aligned.
- 12.20 - Often some downsides of assessment. What are your thoughts on formative and summative assessment? When and how are each used? Bring in discussions from group exercise introduced in previous slide. Think about your own experiences of assessment and feedback, what’s been good/bad? What can we learn from your experiences? My own personal experiences (Emma) As an undergraduate not understanding the assessment criteria and how they were apparent in a piece of work (couldn’t see the differences between two pieces) so learning to mimic good work instead. As a postgraduate not getting much feedback (because it’s all fine) but consequently not really knowing if you’re doing enough, too much or too little. As a postgraduate when you’re getting high grades not being given ideas for how to take work further.
- Nature of knowledge. Declarative knowledge is knowing about things, knowing what Freud, or Marx, or Keynes said, that this is how a blast furnace works, that the battle of Hastings took place in 1066; that Moby Dick was published on 18 October 1851, that Harry Potter’s first broomstick was a Nimbus 2000. Only one thing that you can really do when confronted with declarative knowledge – that’s receive it, internalise it – fit it into the meaning that you make of the world. Knowledge to be accommodated. You will find this knowledge verified and reproduced in books, online. Knowing that. Functioning knowledge puts declarative knowledge to work. Functioning knowledge informs action, where performance is informed by understanding. Functioning knowledge requires declarative knowledge, but doesn’t necessarily follow that declarative must be instilled first, e.g. problem-based learning. Knowing how. When truly understand something see the world differently, and act differently. A core concept is a conceptual ‘building block’ that progresses understanding of the subject; it has to be understood but it does not necessarily lead to a qualitatively different view of subject matter. So (Biggs and Tang, pp.81-87)
- These are: transformative – occasioning a sudden shift in the perception of a subject; irreversible – unlikely to be forgotten and learned with considerable effort; integrative – exposing previously hidden interrelatedness of something. Possibly often bounded – any conceptual space will have terminal frontiers, bordering with thresholds on new conceptual areas. Also, they may be troublesome – often counter-intuitive, or even the opposite of what they have been taught. “as students acquire threshold concepts, and extend their use of language in relation to these concepts, there occurs also a shift in the learner’s subjectivity, a repositioning of the self . . . . Threshold concepts lead not only to transformed thought but to a transfiguration of identity and adoption of an extended discourse.” (Meyer and Land, 2005: 375-375) Distinct from core knowledge – which students also need to know. A core concept is a conceptual ‘building block’ that progresses understanding of the subject; it has to be understood but it does not necessarily lead to a qualitatively different view of subject matter. So, for example, the concept of gravity – the idea that any two bodies attract one another with a force that is proportional to the product of their masses and inversely proportional to the distance between them – represents a threshold concept, whereas the concept of a centre of gravity does not, although the latter is a core concept in many of the applied sciences.
- Evolution of these ideas, through: Carless, 2007 ‘Learner oriented assessment’. Work of the ASKe Pedagogy Research Centre at Oxford Brookes. Weston Manor Group. HEA. Look at each of these in more detail. Assessment for learning: Advocate a shift of focus from discussion of standards which champion to consider how high standards of learning can be achieved though assessment. Learning and assessment should be fully aligned. Price et al 2011 argue that: Assessment of learning places deep and complex learning under threat due to understandable demand for high levels of reliability and consistency. If measure what is easy to measure, often fall back on lower orders of Blooms’ taxonomy: remember, understand, apply, analyse, evaluate create. What is ‘high standard of learning’? Critical thinking Independent thinking Active learning Better understanding of subject matter Ability to apply understanding Development skills and attributes: self-efficacy, self-confidence, self-motivation, reflection? Dovetail with employability. What is low standard of learning? Strategic learning Surface learning Regurgitation of learned facts – remembering and repeating Passive learning Poor academic integrity Rather than assessment of learning need to place emphasis upon assessment for learning, i.e. assessment promotes student learning. “The need to provide a reliable, verifiable mark for each individual for each assignment can either limit the methods we have used or create justifiable concerns about consistency or fairness in marking.” So assessment methods which have demonstrable value for learning, such as feedback on drafts, group assessment, peer learning and work-based assessment have been side-lined. Rebalance from summative to formative and diagnostic - which enables exchange information on student achievement – allowing teaching to be responsive. Process becomes dialogic, students given opportunities for preparation and practice before summative assessment – encouragement and support to be their best. Unlocking potential of approaches previously viewed with some suspicion – enables students to develop desirable graduate attributes, self-monitoring, evaluative skills. Key points Assessment must be designed to develop high standards of learning. Students’ learning is enhanced when assessment builds on previous learning and requires demonstration of higher order learning and integration of knowledge. A high quality learning process requires a balance between formative and summative assessment ensuring that summative assessment does not dominate. One of the roles of formative assessment is to give students opportunities for preparation and practice before they are summatively assessed. A range of approaches to feedback in addition to tutor comments on submitted work need to be in place. Students need to develop the capacity to use feedback effectively Integration assessment literacy into course design Active engagement with assessment standards needs to be integrated and seamless part of course design – interwoven into fabric of their course. This enables students to develop their own internalised conceptions of standards – to monitor and supervise their own learning. Students need to understand recognised standards – this understanding will enable them to become autonomous, reflective learners. Assessment literacy: students and staff understand the nature and purposes of assessment and assessment standards Takes time Active involvement in educational community – students as partners Iterative process – course design and implementation to provide ‘unhurried opportunities and time’ – within and across programmes – progression pathways Students need opportunities to learn about, understand, internalise and apply relevant standards – achievable through observation, modelling, discussion, reflection and practice. Dialogue – with staff and with peers (peer assessment) – learn more about subject, themselves and assessment of their performance. Assessment literacy also Indicator 6 QAA chapter B6 on assessment: “staff and students should engage in dialogue to promote a shared understanding of the basis on which academic judgements are made”. “Facilitating students' assessment literacy includes illustrating the way in which standards are communicated and applied within the relevant subject to enable staff to make judgements about student performance in different types of assessment task. It also enables students to develop an awareness of the complex nature of professional judgement, and of the way in which standards are derived from the descriptors in the relevant higher education Qualifications Framework and Subject Benchmark Statements, and from the degree-awarding body's regulations, policies and processes for assessment. (See Indicators 1, 2 and 13 of this Chapter, and Part A: Setting and Maintaining Academic Standards.)” Explanation Courses and assessments need to be designed in ways that help students to achieve understanding of the recognised standards. Understanding will also help them to become autonomous learners who can readily reflect on and review their own progress, development and learning. Appropriately involving students in the design of courses will help this be more easily realised. Key points Assessment literacy is essential to everyone involved in assessment practice. It takes time to develop understanding and skills in assessment. These can be gained by active involvement in an educational community in which students are contributing partners. Students are able to realise complex and sophisticated outcomes when they have the opportunities to learn about, understand, internalise and apply the relevant standards. This can be achieved through observation, modelling, discussion, reflection and practice. Assessment literacy is an iterative process, and therefore course design and implementation should provide unhurried opportunities and time – within and across programmes – to develop complex knowledge and skills, and to create clear paths for progression. Encouraging self- and peer assessment, and engaging in dialogue with staff and peers about their work, enables students to learn more about the subject, about themselves as learners, as well as about the way their performance is assessed.
- 12.30 Just talk them through this slide
- 13.20 Share this list What does this mean in practice? Even if you’re marking summative assessment tell them when to expect it back? Mention feedback sandwich. Also making it related to their previous and future work Are there strategies we can implement for making it more efficient? What should feedback be doing? How do we ensure reliability? Plagiarism +Collusion Introduce the idea of ‘feedback dialogue’ and hand out flyers. What does good feedback do?
- 12.35 NSS data 2015 – taken from HEFCE spreadsheets - http://www.hefce.ac.uk/lt/nss/results/2015/ May need to be checked. what students have been saying, what can we do something about, also acknowledge the extent to which we can meet these. Need to look for efficient methods, especially after the fees hike where these issues might become bigger. These are percentages agreeing. Discuss
- Angelo, 2013, p.101. Angelo identifies 10 specific questions to be addressed in this fixed order. Forefronts: Overall purpose of the module: Discipline-specific knowledge and skills General graduate attributes Work/career based skills Basic academic language/learning skills Personal awareness, development or growth Something else (if so what?) Where module fits in programme – relation to programme learning outcomes? Vertical and horizontal alignment Who is module designed for? Intended learning outcomes? Standards Assessment All before get to the actual content. More than half way down the list. The final thing on the list is what teachers do. Learning-centred approach.
- 14.10 Discuss and talk them through Starting with Why Evaluate Quick fire round – collate on a flipchart Course/Module/Session Learning Outcomes - Teaching Methods – Assessment (F &S) – CONSTRUCTIVE ALIGNMENT Compare and contrast with previous runs Summative evaluation – at the end of the module – evaluating the ‘success of the module’ against objectives. Formative evaluation – as the course or module is progressing – emphasis is on improvement/changes to meet objectives for that Diagnostic evaluation – we sense something is wrong – want to sort it We have tried something different Student Perspective Expectations What they think they have learnt We want to try something new Teaching Teaching style To benchmark ourselves with colleagues / NSS survey – trigger something Teaching development QAA Department, Faculty, Institutional
- 14.20 10 minutes collectively how might you respond to these three. Now think from two different angles – how would you respond affectively (feelings) and then how will you actually use the comments?
- Angelo, 2013, p.100. Which of the above seem most persuasive and relevant to your own teaching and module design?