A novel web-based learning platform for accelerated knowledge retention in higher education

1. A novel web-based learning platform for accelerated
knowledge retention in higher education
Jeremiah R. Cohen, B.S.
David Geffen School of Medicine at UCLA
Massachusetts Institute of Technology, 2009
Memorang.io
George J. Courtsunis, B.S.
Massachusetts Institute of Technology, 2009
Memorang.io
June 5, 2014

2. The Problem
• Increasing volumes of knowledge to retain
• Memorization tools have not fundamentally changed in 10 years
• Most of these tools center around “flashcards”
• Flashcards represent knowledge in a one-to-one relation, while integrated
knowledge requires many-to-many relations.
• Lack of detailed feedback for metacognition
• Many students lack structure
Create
Evaluate
Analyze
Apply
Understand
Rote memory

3. Jeremiah
George
(formerly “Enki Learning”)

4. Why not automate Gagne’s 9 Levels of Learning?
• Gain attention (Reception)
• Informing learning of the objective (Expectancy)
• Simulating recall of prior learning (Retrieval)
• Presenting the stimulus (Selective perception)
• Providing learning guidance (Semantic encoding)
• Eliciting Performance (Responding)
• Providing Feedback (Reinforcement)
• Assessing Performance (Retrieval)
• Enhancing Retention and Transfer (Generalization)

6. Strategy

7. Content

12. Retrospective Cohort
• Inclusion criteria
• Studying for USMLE Step 1 Board Exam
• 100+ questions answered
• 2+ study sessions
• Last 4 weeks
• Data items recorded
• Answers, mode of study, accuracy, study sessions, mastery score, specific
content studied

13. Overview
• 558 students identified
• 552,631 facts studied
0
500
1000
1500
2000
2500
3000
3500
25th 50th 75th
FactsStudiedPerStudent
Percentile

14. Facts Studied per Learning Mode
1. Multiple Choice
2. Flashcards ~ Matching
3. Candy Cannon
79%
10%
10% 1%
Multiple Choice Flashcards Matching Candy Cannon

15. Accuracy
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
25th 50th 75th
Accuracy
Percentile
Overall
Multiple Choice
Flashcards
Matching
Candy Cannon

16. Weighted Accuracy
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
25th 50th 75th
Accuracy
Percentile
Overall
Multiple Choice
Flashcards
Matching
Candy Cannon

17. Accuracy per Subject Mastery
R² = 0.9928
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20% 40% 60% 80% 100%
Accuracy
Mastery

18. Accuracy per Subject Mastery per Mode
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20% 40% 60% 80% 100%
Accuracy
Mastery
Multiple Choice Flashcards Matching Candy Cannon

19. Case Study: “Cellular Proteins”
• Inclusion criteria
• Set with a large amount of study activity
• Reached exactly 70% mastery
• Data items recorded
• Answers, mode of study, accuracy, study sessions, mastery score, specific
content studied
• Selection:
• 17 students
• 196 facts per student to reach 70%

20. Case Study: Facts Studied per Learning Mode
61%
24%
14%
1%
Multiple Choice Flashcards Matching Candy Cannon

21. Case Study: Accuracy per Subject Mastery
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20% 40% 60% 70%
Accuracy
Mastery

22. Case Study: Facts studied to reach 70%
80%
82%
84%
86%
88%
90%
92%
94%
96%
98%
100%
0 50 100 150 200
Accuracy
Number Facts Studied to reach 70% Mastery

23. Conclusions
• When presented with multiple learning modes, medical
students show an overwhelming preference for multiple choice
questions over other modes
• Mastery scores can be used to predict accuracy
• At lower levels of subject mastery, matching is the easiest
• At moderate-to-high levels of mastery, all modes are equal
• Flashcards are not more effective than any other learning
mode, and students may benefit from having more flexibility.

24. Next steps
• Correlate study data with exam scores
• Real-time data tracking for students and educators
• Leverage data to offer personalized recommendations
• Determine optimum answer events to thoroughly master a concept
• Experiment with different strategies and modalities (e.g. spaced repetition vs.
original order vs. randomized shuffle).
• Compare mobile device vs. desktop/laptop studying

25. Thank you!
We welcome your feedback and questions
yermie@memorang.io
george@memorang.io