PASTEL: Evidence-based Learning Engineering Method to Create Intelligent Online Textbook at Scale
1. COMPUTER SCIENCE
PASTEL: Evidence-based Learning
Engineering Method to create Intelligent
Online Textbook at Scale
Noboru Matsuda & Machi Shimmei
Center for Educational Informatics
Department of Computer Science
North Carolina State University
2. COMPUTER SCIENCE
Current MOOC Challenge
• Lack of individualization
– Ineffective learning (no learning!)
– Disengagement / drop-out
• Lack of systematic content creation & validation
– Where should we start from?
– How can we iteratively make it better?
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 2
3. COMPUTER SCIENCE
Current ITS Challenge
• Scalability / Generality
– Too expensive to build
– Mostly good for procedural skill acquisition
• What about conceptual learning?
• Robustness of Learning
– Luck of learning to solve with justifications
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 3
4. COMPUTER SCIENCE
Summary of Challenges
• To overcome the issues of MOOC and ITS, there
is a critical need to innovate a technology that
– provides adaptive instruction while promoting
synergetic learning
• An evidence-based curriculum development is
essential
– to build a large scale online course
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 4
5. COMPUTER SCIENCE
Our Solution
• Evidence-based learning engineering methods
– PASTEL (Pragmatic methods to develop Adaptive and Scalable
Technologies for next generation E-Learning)
• Adaptive Online Courseware
– CyberBook
= MOOC + Cognitive Tutors + Adaptive Control
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 5
6. COMPUTER SCIENCE
CyberBook
• Online courseware that provide macro-
adaptive scaffolding
– Problem balancing
– Mastery practice (cognitive tutoring)
– Proactive detection of unproductive failure
– Scaffolding on failure
• Incorrect quiz attempt and/or wheel-spinning
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 6
16. COMPUTER SCIENCE
Results – Num. Assessments
• Adaptive >> Non-adaptive for Science, but
not for Math
• No correlation between # assessments and
post-test when pre-test was controlled
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 16
17. COMPUTER SCIENCE
Results – Dynamic Link
• Surprisingly low use of dynamic link
– 0.4±1.8 in average
• Students figured out that corresponding
resources were almost always on the same
page
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 17
18. COMPUTER SCIENCE
Results – Video Use
• Adaptive == Non-adaptive
• Doer/Non-doer effect was not observed
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 18
(Koedinger et al., 2015)
19. COMPUTER SCIENCE
Results – Hint Use
• Hint on Failure Ratio
• Adaptive >> Non-adaptive for Science, but not for
Math. In particular low-prior students
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 19
20. COMPUTER SCIENCE
Conclusions
• CyberBook with the macro-adaptive scaffolding facilitated
learning
– Controlling the amount of formative assessment and
Dynamic link
– Only for the middle school Science, but not for high
school Math
• Hint use on failure among low-prior students was associated
with learning
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 20
21. COMPUTER SCIENCE
Future Works
• Redo the study with the balanced online
courseware
– Add cognitive tutors to Science
– Add videos to Math
• Redesign Dynamic Link
Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 21
