Modified Sloan-C presentation for Research in Learning and Development class.

1. Feedback, Media, and Cognitive Load Making Feedback Understandable Albert Ingram, PhD Betzi Bateman, MLIS Kent State University

2. Background on Feedback Background on Cognitive Load Pilot Study Future Plans Agenda

3. Small-Scale Feedback Large-Scale Feedback Types of Feedback?

4. What are the functions of feedback?

5. What is effective feedback? Students should learn from it Students should improve products with it

6. Further Reading on Feedback Tagg, J. (2003). The Learning Paradigm College. Bolton, MA: Anker Publishing Company, Inc. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189.

7. Why is Cognitive Load important in instruction? Working memory capacity is limited Learning occurs when relevant information is connected and operated on in working memory If too much of the processing capacity is taken up with extraneous factors, learning is impaired

8. What does the research show? Research on integrating visuals and text, visuals and narration Forcing people to divide resources between different sources of information interferes with learning Using methods that work together improves learning

9. Feedback and Cognitive Load Reducing cognitive load when delivering feedback should increase effectiveness

10. Further Reading on CL Mayer, R. E. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. In Bruning, R., Horn, C. A., &PytlikZillig, L. M. (Eds.), Web-based learning: What do we know? Where do we go? Charlotte, NC: Information Age Publishing. Paas, F., Renkl, A., &Sweller, J, (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1-4. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction,4, 295-312.

11. Hypotheses Research Design Results Pilot Study

12. Hypotheses Decreasing the extraneous cognitive load imposed by how feedback is presented will increase learning from the feedback. Delivering feedback (on textual assignments) as audio will reduce cognitive load because there is less need to switch among stimuli Delivering feedback embedded into an assignment will reduce cognitive over delivering it separately

13. Research Design Use intact classes and existing, real-world assignments that must be handed in as drafts and revised for a grade Repeated measures, within-subjects design in which people receive feedback on multiple assignments in different ways

14. Instrument Adapted from:Yao, Y. & Gill, M. (2009). The effect of hypertext annotation presentation formats on perceived cognitive load and learner control. Journal of Interactive Learning Research, 20, 359-373.Yeung, A. S., Lee, C. F., Pena, I. M., & Ryde, J. (2000, January). Toward a subjective mental workload measure. Paper presented at the International Congress for School Effectiveness and Improvement, Hong Kong, China.

15. Instrument Four categories measuring perceived cognitive load Difficulty (“I think the feedback was easy to follow.”) Incompetence (“I was not able to tell what to modify based on the feedback.) Negative Affect (“I like receiving feedback in this format.”) Lack of Effort (“I worked hard on my revision.” 5-point Likert-type scale (many items reverse-coded)

16. Results of Pilot Study Educational Technology course (N=64; 48% response rate) Series of linked assignments (but no revision of individual assignments) Very simple feedback = very low perceived cognitive load Audio feedback: 31.75, 28.27 Text feedback: 23.7, 21.67 (out of possible score of 80) Feedback was not embedded as initially planned

17. More Questions Four respondents REALLY disliked audio feedback Gender or style of narrator? Entity vs. Incremental Theories? Is it even harder for those sensitive to negative feedback to receive it in audio format? Procedural feedback not conducive to audio format? Text or graphics more conducive for procedural feedback?

18. Plans English Composition Classes Larger N Revised assignments Rich feedback Detailed rubrics Balanced quasi-experimental design Embedded vs. Separate Document Perceived cognitive load and student achievement

19. Time for YOUR Feedback! Is this a DUH! Study? Does embedding vs. separate document matter? What do you think of the survey instrument on perceived cognitive load?