Kevin Gumienny ELS 2015 Presentation: The Brainy Way to Better Training
•Download as PPTX, PDF•
1 like•938 views
In this discussion, we’ll explore abstract scientific studies and tie them down to methods, tools, and techniques that will create better elearning—elearning that succeeds because it’s tailored to the way that the brain works. We’ll use practical examples to demonstrate ways in which science can be deftly applied to create training that will change student behavior, enabling them to do what they hadn’t been able to do before.
Recommended
More Related Content
Similar to Kevin Gumienny ELS 2015 Presentation: The Brainy Way to Better Training
Similar to Kevin Gumienny ELS 2015 Presentation: The Brainy Way to Better Training (20)
Recently uploaded
Recently uploaded (20)
Kevin Gumienny ELS 2015 Presentation: The Brainy Way to Better Training
- 1. Kevin Gumienny The Brainy Way to Better Training
- 4. What does elearning look like, anyway?
- 29. 3/4 of 2/3 cup
- 32. Check Your Understanding After a survey of utility bills, you discover that the building uses almost as much current in the winter months as it does in the summer. However, the facility has gas heating. What would be the best first step to solve the problem? Post signs reminding employees to turn off lights when they leave a room. Install data loggers on all fuse boxes. See if the facility has a building automation system.
- 34. 1. What (exactly) are you doing? 2. Why are you doing it? 3. How does it help you? (Can you describe it precisely?) (How does it fit into the solution?) (What will you do with the outcome when you get it?)
- 35. Pre-Knowledge Check In this module, we’ll discuss how data classification can help protect your organization. Take a moment and reflect: What dangers might result from the lack of data classification? Type your response here, and click submit to see some possibilities. Your response Some possibilities If data leaks, it could lead to the public losing trust in the organization. If data leaks, it could result in criminal or civil penalties. If the information is federally regulated, then a breach might lead to a federal investigation, or even the loss of the right to collect revenue. Submit
- 39. Phase angle Apparent power (volt-amperes) Reactive power (volt-amperes reactive) True power (watts) Power Triangle Apparent power is 140 VA True power is 120 W Here’s a hint: A2 + B2 = C2 Need another? What is reactive power? Submit Here’s a hint: W2 + VAR2 = VA2
- 41. Incorrect OK That is incorrect. Please try again. Correct OK That’s right! You’ve selected the correct response.
- 45. Annie Murphy Paul Will Thalheimer Ruth Clark Richard E. Mayer Julie Dirksen More Information
Editor's Notes
- A lot of names—cognitive psychology, learning and instruction, experimental psychology, educational psychology. Much of this work has been developed in the last couple of decades or so. And much of it in an experimental setting. Much of it for K-12 and college situations. Is it ready for the classroom? Is it ready for online training? Why focus on science at all? ----- Science of learning: scientific study of how people learn Science of instruction: scientific study of how to help people learn—informed by learning Science of assessment: scientific study of how to determine what people know—measuring how effective instruction has been
- Why take this approach? Because science is cool. We know about this, but we don’t practice it—we can be cutting edge by applying it. 65% of learning professionals are aware of the impact that deep practice can have on learning. 13% use it. (https://www.td.org/Publications/Blogs/Science-of-Learning-Blog/2015/03/Boost-Your-Training-with-the-Neuroscience-An-Infographic). After all, what’s the alternative? Opinions? Slogans? Quotations from experts? Learning styles? Being able to explain why you’re repeating information; why you’re hiding information; why you’re including that type of question right there can go a long way in convincing your stakeholders, including your subject matter experts, that there’s a method behind your madness. Now I can’t promise you success in convincing your SMEs and stakeholders that having a narrator reading the on-screen text to the learner makes training less effective. But when you’re able to point to studies that demonstrate this is the case (in one 2002 study, students who saw animation, listened to narration, with no text scored a median of 79 percent better than students who saw the animation, heard the narration, and saw redundant text).
- What I mean. It can mean video, having PDFs available to be downloaded. Potted plant is what I’ve seen a lot—sit down in front of a computer and click on the screen for an hour.
- Typical layout.
- Typical layout. But generally, it’s around an hour, they sit through it at their desks, and then get on with life. Have to pass final exam with a cut score. Often 70. Why? It’s higher than 69 and lower than 71. Now, we’re trying to move beyond this format, to get rid of bullet points, to build interactive learning experiences, to get away from telling learners what they need to know and enabling them to do what they need to do. Stop pushing knowledge at them, and create situations where they pull the knowledge that they need to learn. But… one challenge at a time. The techniques I’m discussing here can apply to your standard potted-plant courses and courses that function at a much higher level.
- Why do we train people, anyway? Well, Clark pointed out, to change behavior, improve performance. There’s compliance training, of course. This is often to ensure that employees are following applicable rules and regulations correctly. Sometimes it’s to cover the organization—Fred took the compliance training and should have known not to use the copier for his kid’s birthday invitations.
- Why do people take training, anyway? Because they have to, often. New hire training, ethics, HR rules… Some sign up for fun—Lynda.com and MOOCs. Some for personal improvement, such as certifications. Sometimes just to get credit hours (for certification). If we’re going to improve performance, if we’re going to change behavior, if we’re going to address both those who are bored, and those who are engaged, then…
- So what do our learners need to be able to do for training to be successful? Sometimes they need to be able to click a next button for an hour, and then guess 70% of the right answers for the final quiz. What makes it successful, on a general level? There’s a ton of research out there, entire shelves of libraries dedicated to this. I’d like to focus on three areas, and within those three areas, what years of research and studies have told us works best.
- If learning isn’t in memory, it can’t be recalled. What does research tell us about designing training that more effectively encodes to memory? If learners can’t transfer to the job, or behavioral change, or the next environment, then it’s ineffective. If learners aren’t motivated to achieve goals set within training, then they have no impetus to recall or transfer the information. Some areas easily translate to an online environment. You’ll see this with recall. Transfer and motivation, on the other hand, aren’t quite as direct. Still, research done on those topics can positively inform ways that we can approach training. These are only three—but then again, this is only an hour presentation. I’ll give you some resources at the end that you can use to find more ways in which to make online courses more effective.
- Under the category of what will help memory, I’d like to talk about testing, spaced repetition, and interleaving. These are three of the most effective strategies for helping people remember the learning. By the way—chunking content—a lot of work done that shows us that’s a good way to help people remember information.
- The basic idea is that the act of recalling information helps embed information more firmly in memory. And testing is all about recall.
- The most basic of these experiments take two premises: which is better? Study, study, study, and then test. Or study, then test, test, test? It turns out that study once, and then testing repeatedly, offers real benefits when encoding to long term memory. “The memorial benefits of practice tests have been documented across a wide range of materials including foreign language word pairs (e.g., Pyc & Rawson, 2009), text passages (Roediger & Karpicke, 2006b), statistics (e.g., Lyle & Crawford, 2011), general knowledge facts (e.g., Carpenter, Pashler, Wixted, & Vul, 2008), world history (e.g., Roediger, Agarwal, McDaniel, & McDermott, 2011), science (McDaniel, Agarwal, Huelser, McDermott, & Roediger, 2011) and map learning (e.g., Carpenter & Pashler, 2007).” “Compared to simply rereading a given piece of information (or doing nothing at all), actively retrieving information from memory improves the likelihood that the information will be recalled at a later time. The benefits of practice tests have been documented hundreds of times and the benefits of tests may last up to nine months after information has been initially learned (see Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). “Practice tests have been shown to improve long-term retention across the lifespan, from children (e.g., for a review see Agarwal, Bain, & Chamberlain, 2013) to older adults (e.g., Logan & Balota, 2008) and have been documented using a wide range of materials and experimental designs, from word lists and foreign language word pairs to expository texts and maps (e.g., for a review see Roediger & Karpicke, 2006a).” And to emphasize that student’s don’t always know best: “In particular, students need to practice retrieval several more times after they think they know the issue at hand well (Karpicke, 2009)”
- Original.
- Here’s one that retests. Notice that we’re bringing in learning checks from module 1.
- Can easily do this on the course level. Here, we’d drop in learning check. We could also include quiz questions from earlier modules. Sort of like a cumulative cumulative exam. One approach might be for each quiz to contain half questions on new information, half questions on old information.
- And, with proper feedback, the entire course could be a series of questions. I have seen it used as a test prep, where the course was built out of a series of questions. The learner could refer to external documents to supplant their knowledge, then spend time with testing and recall. Worked really well.
- Give the same information over time. It has been found to be effective in various domains, from sales training to language learning to medicine (Caple, 1997; Castel, Logan, Haber, & Viehman, 2012; Grote, 1995; Kerfoot et al., 2007; Lambert, 2009; Landauer & Ros..s, 1977; Lehmann-Willenbrock & Kauffeld, 2010; Toppino & Cohen, 2010). (Spaced Learning) Yet this often the hardest to wrap my head around. Most of the training I design will be consumed in one sitting. How do I do spaced repetition on a 30-minute course? Of course, one answer is that you don’t. Use a different, more appropriate research-based strategy, instead. You don’t have to cram every technique in every course. Mathematics for adults—10 problems in one day, versus 5 one, seven days later, 5 another, test four weeks later, 64% passed spaced versus 32% passed massed Memory retention seems to be best when spacing gap is about 10-20% of test delay (134)
- You can do this on a course level. And many of us actually do. When you introduce information, convey the information, and then summary the information, you’ve doing spaced repetition.
- One way, using our example here, is to allow the learner to see the information throughout the course. Repeated testing is another form of spaced repetition.
- And, of course, you can plan to repeat information in different formats—bringing items you covered as bullet points in a previous slide as part of a story. For example, say you’re designing a course that for adults who deal with troubled teens, and you want to point out how contributing to their community can help develop a sense of personal fulfillment. Later in the course you can bring in social contributions as part of a story reinforcing the idea. You can branch out with different forms of media—videos, interaction. One bit of feedback that I’ve gotten is that it’s hard to come up with new and engaging ways to present information. I’m afraid this doesn’t help that—now you have to come up with presenting the same information in new and interesting ways.
- In fact, there are ways in which online environments offer greater opportunities for spaced learning than traditional instructor-led training. ILT is usually tied down to a single session—four or eight hour training sessions. It’s hard to get people to come to a classroom for fifteen-minute sessions spaced over three weeks. But it may be much easier to ask someone to log into a system on their desktop for fifteen minutes spaced over six weeks. If you have greater control over the course format, though. You have more opportunity to take advantage of spaced repetition. Offer time to learn and practice material across time, by having sessions separated by days, including a recap of previous learning. Memory retention seems to be best when spacing gap is about 10-20% of test delay. So if you’re going to delay a test for four weeks, then the repetition events should take place every 3-6 days (134).
- Is it better to block knowledge, giving chunks of related information together; or is it better to interleave knowledge?
- One study (Kang and Pashler, 2013) worked with adults to determine if it was more effective to categorize birds by studying them in blocks.
- Or by interleaving different categories together. Generally speaking, interleaving is better. And once again, it’s a technique that benefits multiple learning areas. It’s been effective in learning the styles of artists and mathematical problems. Yet there are limits. A recent study by Carpenter and Mueller (2013) on pronunciation guides showed that students learned better if learning ()done by seeing and hearing example words that represented the rules) was done by blocked examples of a single rule, rather than mixing the rules. A theory for the difference is that interleaving works better when the point of study is to notice difference (between categories of birds) rather than similarities (similar pronunciation of words). When college students were given five algebraic rules to learn (2002), those who worked on problems that interleaved the problems (worked with one rule, then a series of problems, worked with second rule, worked problems with rule 1 and rule 2), outperformed those who worked with blocks of rules, 97% correct to 85% correct on an assessment.
- The design that keeps on giving. This also has the benefit of interleaving. Recall—testing, spaced repetition, and interleaving—these are all techniques that have been demonstrated to benefit recall. And they can be effectively used in online learning.
- My learner can now remember information effectively. But can they transfer it to a situation that exists outside of the learning environment. There’s been work done helps us think about how to make transfer more effective. Two areas I’d like to talk about are context and metacognition.
- What helps learners transfer information from one context to another…from an online learning environment to actual work. The use of tools in practical environments helps people work almost error free (e.g., Cohen, 1983; Schliemann and Acioly, 1989; Simon, 1972; see also Norman, 1993). New technologies make it possible for students in schools to use tools very much like those used by professionals in workplaces (see Chapter 8). Proficiency with relevant tools may provide a way to enhance transfer across domains.
- The actual activity performed is going to be different in the two contexts. If I’m given an example of a recipe as a word problem in a class, I’m going to work the formula. How do you do it in life? One example is of a man needed three-fourths of two-thirds of a cup of cream cheese to create a dish. How does he do it? Well, he measures out 2/3 of a cup, then shapes the cheese into a square, cuts it twice, and then include three of the four quarters. (Study of people in a weight watcher’s program, Lave, et al, 1972, HPL 74) Not how you would do it in school. No fractions, no multiplying, no reducing to the common denominator. The abstract reasoning of many learning environments is quite unlike the contextualized reasoning used in real life (Resnick, 1987, HPL 74). Embedding abstract logical arguments in concrete contexts can improve reasoning (Wason and Johnson-Laird, 1972, HPL 74).
- Essentially, we get the learner as close to real life as possible. If we were designing a course that trained building technicians how to ensure that their buildings were energy efficient, the best option would be to create an activity where the technician walks through a real building and looks for inefficiencies and opportunities for improvement. "Chiller" by P199 - Own work. Licensed under Public Domain via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Chiller.jpg#/media/File:Chiller.jpg Learners of all ages are more motivated when they can see the usefulness of what they are learning and when they can use that information to do something that has an impact on others—especially their local community (McCombs, 1996; Pintrich and Schunk, 1996). Sixth graders in an inner-city school were asked to explain the highlights of their previous year in fifth grade to an anonymous interviewer, who asked them to describe anything that made them feel proud, successful, or creative (Barron et al., 1998). HPL, 61
- http://www.energystar.gov/sites/default/uploads/tools/esplant.swf Of course, we’re doing elearning, and that might make a walk through impractical. How about a simulation? Here’s one from EPA. It’s not a real environment, but it’s contextualized enough so that connections can be easily made. And sometimes diagrams are better than the real thing. It’s unlikely you’d be able to see leaks in a compressed air pipe, but here you have a visual cue to help.
- If we can’t simulate it, then a scenario-based question with good context is the next best thing. Notice this questions doesn’t ask the learner to recall information (recognize, really). It asks them to evaluate a situation, one that might occur in real life. One note of caution: Simply learning to perform procedures, and learning in only a single context, does not promote flexible transfer. The transfer literature suggests that the most effective transfer may come from a balance of specific examples and general principles, not from either one alone.
- Metacognition is thinking about thinking. It’s a strong component of expert thinking. Experts are aware of what they know and what they don’t know, so that they can efficiently and effectively expand their knowledge based when faced with new problems. “Explicit, flexible metacognitive functioning appears to be a strong component of expert thinking. Experts monitor their own understanding as they work so that they are explicitly aware when their own knowledge limits are reached, and additionally verify and analogize new information within their existing knowledge set in order to adaptively grow their understanding” (NRC, 2000). When you think about your thinking, it helps you map new information to different contexts.
- Alan Schoenfeld, mathematics profession (1985). Found that students would read and explore a solution come hell or high water, even long after it was found unproductive. He began to ask students to ask themselves these questions, and modeled the behavior in class. As he did, more students asked themselves these questions. And as they did, they spent more time on planning , implementing, and verifying a solution than just reading, analyzing, and exploring it. “A similar experiment in a classroom setting showed instructing students in mathematics using a metacognitive focus, as opposed to traditional teaching, lead to students outperforming the control group (Mevarech and Fridkin, 2006).” “Such metacognitive prompting improved students’ accuracy and efficiency of problems that were more complex (no effect on easier problems), and induced more awareness of unproductive solutions, when compared to a control group (Hoffmana and Spatariu, 2008). How does this map onto the transfer of knowledge? “Students in the control section learned as much factual knowledge as did those in the experimental section (as measured on an exam), but the experimental group did significantly better in applying knowledge to new problems. Even more tellingly, a greater proportion of students from the experimental section passed the next advanced course compared to the control students, and with better grades.” In short, student who became skilled at metacognition were able to evaluate their own thinking and determine which solution best fit a new problem. Or, in more detail: 1. Have you solved similar problems before? 2. What strategy can you use to solve these problems? 3. What steps are you using to solve the problem? 4. Can your answer be checked for accuracy? 5. Are you sure your answers are correct? 6. Can the problem be solved in steps? 7. What strategy are you using to solve the problems? 8. Is there a faster method to solve the problem? 9. Are these problems similar to addition in any way?
- “Preassessments that prompt students to examine their initial thinking — e.g., an initial "What do I already know about this topic that could guide my learning?" self-question along with explicit metacognitive prompts to guide students' planning.” Now, when I start learning, I’m already thinking about how the new knowledge fits into my current knowledge. Some examples of data breach: worked at Target, Home Depot, the health insurer Anthem, or the comptroller’s office of the State of Texas.
- Some things you can’t fix. But we can ensure that we’re not creating difficulties, and unintentionally making maintaining motivation more challenging than it needs to be. Challenges, however, must be at the proper level of difficulty in order to be and to remain motivating: tasks that are too easy become boring; tasks that are too difficult cause frustration. Easy test questions versus challenging test questions. We can help mitigate frustration and maintain motivation by playing attention to the expertise reversal effect and feedback.
- Experts and novices learn differently. Experts map information into what they already know; novices don’t have that support, so need a greater level of guidance. Kaluga, 1998—novice electrical trainees learned better with diagrams with text; more experienced learner learned best with diagram only. They were able to absorb the information more efficiently. More experienced learners don’t need as much handholding. They don’t need the level of guidance. In fact, that extra layer of guidance can make learning more difficult for experienced learners. 1. Experts notice features and meaningful patterns of information that are not noticed by novices. 2. Experts have acquired a great deal of content knowledge that is organized in ways that reflect a deep understanding of their subject matter. 3. Experts’ knowledge cannot be reduced to sets of isolated facts or propositions but, instead, reflects contexts of applicability: that is, the knowledge is “conditionalized” on a set of circumstances. 4. Experts are able to flexibly retrieve important aspects of their knowledge with little attentional effort. 5. Though experts know their disciplines thoroughly, this does not guarantee that they are able to teach others. 6. Experts have varying levels of flexibility in their approach to new situations. (HPL, 33)
- As an example, consider Shakespeare. Joss Weadon’s 2012 version of Shakespeare’s Much Ado About Nothing. “Oksa, Kalyuga and Chandler (2010) found an expertise reversal effect caused by co-referring text with another piece of text when comprehending complex literary text. Novices learned better in a condition that provided original Shakespearean play extracts with embedded line by line modern English translation and explanatory interpretations than in a condition that provided the original text only. The relative effectiveness of the two conditions reversed with highly skilled Shakespearean experts. “The Shakespearean experts in the integrated explanatory notes group reported higher subjective ratings of cognitive load. Retrospective verbal reports indicated that the experts were unable to avoid cross- checking the accuracy and validity of the explanations with their internalized interpretations and those activities distracted them from directly analysing the original text.” Instead of working memory being spent processing new information, the external supports conflicts with existing information and cognitive load has to be use reconciling the two. What happens if you have both novices and experts in the same course? Or, to moderates the extremes, learners with different levels of skill.
- If we provide too much information, we make the learning environment unpleasant for experts, both too easy, and with a high level of cognitive load. If we don’t provide enough, then our novices will become frustrated and unmotivated. Ideally, we can prejudge the level of learning that a learner needs. Are they experts or novices? And tailor to that. But, remember, learners are notorious about misjudging their own abilities. One approach: If there’s no instructor to regulate the content, can we design something so that learners can regulate it themselves? Power triangle http://www.allaboutcircuits.com/vol_2/chpt_11/2.html
- “From a review of 12 meta- analyses that have included specific information on feedback in classrooms (based on 196 studies and 6972 effect-sizes), the average effect-size was d = .79 (see Bertsch & Pesta, this volume, for a discussion of effect sizes and meta-analysis), which is twice the average effect (Hattie, 2009). This gives credence the common aphorisms that feedback can double the rate of learning, and that feedback is among the top 10 influences on achievement.” “As highlighted in Kluger and DeNisi's (1996) review, the variance of feedback effects, however, is considerable, indicating that some types and circumstances of feedback are more powerful than others.”
- Standard feedback form. ------ “Although prior research has firmly established the memorial benefits of practice tests (compared to study), research has shown that the benefits of practice tests are even greater when tests are followed by feedback (e.g., Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Cull, 2000), especially for items that are incorrectly recalled on test trials (Pashler, Cepeda, Wixted, & Rohrer, 2005).”
- And, of course, you can incorporate the learning event into a learning experience. With this idea, the learner gets active feedback depending on the choices that the learner makes. It’s more in depth than feedback that merely gives a positive/negative response or even one that presents consequences. “There exists a substantial literature base concerning the value of goals and goal setting as base-level ingredients in motivation, effortful striving, and in achieving self-control (Locke & Latham, 2002; 2013). It has been shown that setting challenging goals is more motivating than either possessing easy goals or urging people to 'do your best'. Available studies with adults suggest, however, this proposition is valid only when individuals are provided with feedback as to their progress towards the challenging goal.” “Studies with high school students indicate that feedback becomes more effective when it reflects progress made in terms of actual accomplishments, rather than normative standards such as letter grades (Harks, Rakoczy, Hattie, Besser, & Klieme, 2013). Such process-oriented feedback is rated by students as more useful than grade information, and is linked to enhanced interest in the material being taught.” Feedback can also be given during the scenario, guiding the user as they work through the course.
- One last word about feedback—it’s important to consider the tone of feedback. Notice the tone of this quiz from Fast Company—a great site, if you’re looking to… fruitfully spend some time exploring interests. It doesn’t say that you’re right or wrong, but that you’ve agreed or disagreed with experts. http://www.fastcodesign.com/3035993/innovation-by-design/quiz-can-you-spot-great-design “On the other hand, many classroom 'process-product' studies conducted in the 1970s repeatedly found high levels of teacher criticism to have a negative impact on many classroom indices, including student achievement test scores (Brophy, 1981, 1986). Punitive teachers contribute to negative classroom climate, which is linked with a range of undesirable outcomes, often in a manner suggesting snowball or cascade effects (Demanet & Van Houtte, 2012).”
- These three areas, along with the specific examples Memory: Testing, spaced repetition, interleaving Transfer: Context, metacognition Motivation: Expertise retrieval effect, feedback
- Annie Murphy Paul: Currently writing a book on the science of learning, past blog posts and articles still useful Will Thalheimer: Blog posts on what research tells us about learning Ruth Clark and Richard E. Meyer: E-learning and the Science of Instruction Richard E. Meyer: Work on the effective use of multimedia Applying the Science of Learning in Education: Society for the Teaching of Psychology, Division 2 of the American Psychological Association How People Learn: National Academies Press, John D. Bransford, Ann L. Brown, and Rodney R. Cocking, eds. Why are we going to plow through all kinds of research material to figure out how to design effective research-based elearning?
- Because science is cool.