«Conversational agents in MOOCs: what’s the point?» / Stavros Demetriadis, associate professor at the School of Informatics, Aristotle University of Thessaloniki (AUTh), Greece.
A study about the impact of rewards on student's engagement with the flipped ...eMadrid network
Weitere ähnliche Inhalte
Ähnlich wie «Conversational agents in MOOCs: what’s the point?» / Stavros Demetriadis, associate professor at the School of Informatics, Aristotle University of Thessaloniki (AUTh), Greece.
Ähnlich wie «Conversational agents in MOOCs: what’s the point?» / Stavros Demetriadis, associate professor at the School of Informatics, Aristotle University of Thessaloniki (AUTh), Greece. (20)
«Conversational agents in MOOCs: what’s the point?» / Stavros Demetriadis, associate professor at the School of Informatics, Aristotle University of Thessaloniki (AUTh), Greece.
1. ARISTOTLE UNIVERSITY OF THESSALONIKI
Conversational agents in MOOCs:
What’s the point?
Stavros Demetriadis
Assoc. Professor
School of Informatics
Aristotle University of Thessaloniki
Greece
2. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Aristotle University
of Thessaloniki (AUTh)
Est. 1926
42 Schools in 11 Faculties
Academic Staff: 2024
Students: ~ 75000
3. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
School of Informatics
Software and Interactive Technologies
Lab (SWITCH)
SWITCH Lab
Software Technology
Learning Technologies
Music Informatics
School of Informatics
Est. 1992
Academic Staff: 30
Students: ~ 900
4. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Overview
Peer interaction: productive dialogue for learning
Conversational Agents
MentorChat: Research Evidence
colMOOC project: Agents in MOOCs
Design, Expectations, Research
Erasmus+
Knowledge Alliances
5. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Peer interaction
Peer interaction is the key learning mechanism
for knowledge building in collaborative learning
settings
The real generative processes of the emergence
of mind and the production of knowledge can
be usefully modeled as multilevel conversations
between conversants…
Conversation Theory (Boyd, 2001)
6. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Free = suboptimal collaboration
But no guarantee that these interactions will actually occur
Dillenbourg & Tchounikine (2007)
Various studies have identified patterns of suboptimal
collaboration in free (non-supported) collaboration
conditions
(e.g., Liu & Tsai, 2008).
7. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Scripting collaborative learning
Scaffolds are needed to increase the probability that
productive peer interactions occur…
… such as consensus building, explicit explanation, mutual
regulation, argumentation, conflict resolution, etc…
For example ‘Make it Explicit!’: …when asking students to
proactively articulate their own positions explicitly, then
improved peer interaction is triggered in a subsequent
collaborative session
Papadopoulos, Demetriadis & Weinberger, 2013
8. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
When students discuss…
Teachers may intervene…
This kind of intervention fosters productive dialogue and
deeper learning: triggers students to elaborate, recall, make
connections, argue, etc.
Do you agree…?
Do you think this
relates also to…?
Do you think this
also relates to… ?
………………..
9. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
“Academically Productive Talk” (APT)
Framework originating from the teachers’ community
emphasizing the orchestration of classroom discussions
(Michaels et al. 2010).
Addresses a set of teachers’ discussion practices (‘moves’)
that can lead to participation of all learning partners.
Prioritizes reasoning over correctness
Highlights ‘sharing their reasoning out loud’ as an effective
teaching strategy for the elicitation of learners’ perspectives
(Papadopoulos, Demetriadis, & Weinberger, 2013).
10. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
APT Move Example
Link Contributions
Agree-Disagree
Add-On
A. “Do you agree with what your partner said…?”
B. “Would you like to add something to …?”
Re-voice “So, are you saying that … Is that correct?”
Ask for Accuracy
Ask for Credibility
Ask for
Completeness
A. “Could you identify that in a reference book?”
B. “This is probably true, but how could we get
more evidence on that?”
Build on Prior
Knowledge
“How does this connect with what we know about
…?”
Ask for Reasoning
“What are the arguments in favor of that?”, “Why
do you think that?”
Expand Reasoning
Take your Time
Say More
A. “Please take your time before answering”
B. “That's interesting! Can you elaborate on that?”
APT Moves
11. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
APT move efficacy
The efficacy of each APT move depends on
various factors like:
the teacher authority
the student background
the education level
the difficulty of the instructional domain
the students’ knowledge background.
Michaels et al. (2008)
12. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
When you have too many students…
MOOCs are
popular today
Open
Educational
Resources
How can we
advance peer
interaction and
productive
dialogue in
MOOCs? Can the Teacher make APT interventions
in MOOC chats? No! :-(
13. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
…but someone else can do it for you…
a conversational agent
14. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Agents are friendly creatures that live
onscreen…
A ‘pedagogical agent’ is considered
an autonomous computer-generated
virtual character aiming to fulfill
specific pedagogical purposes in a
learning environment
Gulz, Haake, Silvervarg, Sjödén, & Veletsianos (2011).
15. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Conversational Agents (CAs):
Agents that talk to learners
Agents attempting to
engage learners in a
conversation through
natural language
16. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Can CAs successfully support
peer interaction? Yes!
Agent directed learners through prescribed lines
of reasoning substantially improved learning
Chaudhuri (2009)
Agent displayed reflective prompts while learners
took turns tutoring each other increased the
conceptual content in students’ utterances
Walker et al. (2011)
Agent interacted with peers during collaborative
brainstorming significantly enhanced learners’
creativity (more ideas as compared to
brainstorming with a human peer)
Wang et al. (2007)
17. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Emerging research direction:
Agile form of conversational agent to support social
interaction by inducing constructive mental processes
Conversational agent to scaffold collaborative learning
discussions through the approach of ‘Academically
Productive Talk’ (APT)
18. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
MentorChat
A configurable conversational agent
APT-based: Makes APT interventions during synchronous online
student discussion (chatting)
Configurable: (1) various APT moves (2) teacher enters the domain
19. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Configurable-1: can be setup to make
various moves
Build on Prior Knowledge
Link contributions
1. Agree/Disagree
2. Add on
20. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Configurable-2: The teacher enters the
domain
21. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
MentorChat: Key design decisions and
models
Moderate AI level
Agent does not enter into
full fledged discussion
Models and triggers
student dialogue by
making APT moves
Development efficiency
Teacher configures
the domain
Agent as a teacher-
configured learning tool
NOT teacher substitute
22. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Study 1: Linking Contributions
(Tegos, Demetriadis & Karakostas, 2015)
Participants: 43 undergraduate CS students
Randomly assigned to groups and 2 conditions:
Treatment (received agent LC interventions, 9 dyads and 1 triad)
Control condition (no Agent, 11 dyads)
23. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Example: Agent LC intervention
User Student Dialogue with Agent Interventions
George What is the most relevant design principle here?
Anna
That preferably the information should be displayed using different
representation codes, such as images, diagrams, etc., in order to achieve a
better understanding. I think this is the principle of spatial contiguity.
George I see
Agent
George, do you agree or disagree with Anna’s statement regarding the
spatial contiguity principle? Please, elaborate.
George
[Submitted Response to Agent] Well, I think the principle of spatial
contiguity suggests that the text and the relevant image should be put
close to each other
Anna
Ah yes! That’s true. I believe this design principle should be our
number one priority because Mayer has shown that students tend to learn
better when text and graphics are placed close to one another than when
they are placed far from each other (avoid screen scrolling).
24. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Study 1: Key Results
Learning: Treatment outperformed control in both individual and
group learning measures (conceptual knowledge)
Explicitness: Treatment significantly increased frequencies of
explicitness (explicit positions and explicit arguments during
discussions)
Agent intervention triggered on average 1.18 subsequent explicit
contribution (Explicit Response Ratio, ERR = 1.18)
Treatment students used considerably more key domain concepts in
their group discussions
Frequency of explicit arguments was identified as mediator variable
25. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Explicitness as mediator variable
(Tegos, Demetriadis & Karakostas, 2015)
26. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Study 2: Building on Prior Knowledge
(Tegos & Demetriadis, 2015)
Participants: 72 undergraduate CS
students
Randomly assigned to groups and 2
conditions:
Treatment (received agent BPK interventions,
19 dyads)
Control condition (no Agent, 17 dyads)
Confirmed all study-1 results
Improved Individual and Group learning
Increased explicitness
Frequency of explicit arguments was identified
as mediator variable
27. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Study 3: Solicited (A) vs. Unsolicited (B)
Interventions (Tegos, Demetriadis, & Karakostas, 2014)
Unsolicited
interventions:
Perceived by the
students as more
intrusive
But found to be
more effective
in stimulating
explicit
reasoning
A B
28. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Study 4: Undirected (A) vs. ‘Weak’ Directed (B)
Interventions (Tegos, Demetriadis, & Tsiatsos, 2014)
‘Weak’ Directed interventions increase the transactive quality
of a peer dialogue and, leads to improved learning
Transactivity: Reasoning on each others contributions
29. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Lessons from MentorChat
Conversational Agent interventions based on APT can
improve learning outcomes in online student discussions
Interventions trigger students’ explicit reasoning results in
improved learning
Unsolicited interventions >> Solicited
‘Weak’ Directed interventions >> Undirected
30. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Limitations
Controlled conditions: Participants were informed that
discussions would be reviewed by the course instructor…
.. learning benefits though were observed without any feedback from
the instructor
Moderate Agent AI: Not full-fledged discussion with students
but only trigger peer interaction
Objective: easily constructed and deployed agents with a minimum
required level of AI
Text-based communication (use of keyboard):
Easier for tech-savvy students
Can be extended to voice-based dialogue?
31. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Portrait of a Scholar
by Domenico Feti, Italian painter
Gemäldegalerie, Dresden
What
next?
The colMOOC project to integrate similar
agent-based tools in MOOCs
(Demetriadis et al. 2018)
32. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
The point is….
We expect….
Learner engagement and satisfaction through social interaction
Learning benefits at cognitive (domain) and metacognitive level
Developing a community of educators to further explore the
impact of agents on learning
To engage MOOCs learners in
academically productive
dialogue triggered by agent
interventions
33. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
colMOOCs to test-bed the approach
Develop pilot MOOCs in
3 domains
With agent-based chat
integrated as learning
activity
34. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Key design features:
1) Teacher configures the agent domain
The teacher as a creator-developer, the animator of the
agent
35. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Key design features:
2) The agent as an interaction mediator
The agent operates as a TIM (“Teacher-configurable
Interaction Mediator”) mediating teacher-group interactions
36. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Key design features:
3) Analytics component
Empowered also by learning analytics tools to visualize
peer-agent interaction data
37. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Key design features:
4) Community building
Agent-based chat software freely available also independent
of a MOOC platform
Developing their own agent…
Upload it to an agent service/repository…
Use agent they find useful…
Extend agent domain models or change ‘behavior’ (APT
moves)
In the future: Integrate additional cognitive or metacognitive
capabilities in the agent
38. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Educational Design Software development, System
integration, Pilot MOOCs development (M1-M18, June 2019)
Pilot MOOCs Deployment and Evaluation Community
building Dissemination (M19-M36, December 2020 and beyond..)
39. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Open design/research issues
Teacher tools: How to setup the agent and model the domain?
Group formation: How to match peers? Group size?
Learners’ feedback: How to provide it? Peer assessment?
Gamification? Increase engagement?
Modalities of peer-agent communication? (verbal)
Support all forms of knowledge? (conceptual, procedural)
Synchronous vs. Asynchronous peer discussions?
40. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
Thank you! Questions?
sdemetri@csd.auth.gr
http://mlab.csd.auth.gr/sdemetri/
41. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
References 1/2
Boyd, G. M. (2001). Reflections on the conversation theory of Gordon Pask. In R. Glanville & B. Scott (Eds.), Festschrift in
celebration of Gordon Pask. Kybernetes: The International Journal of Systems and Cybernetics 30(5–6), 560–570.
Chaudhuri, S., Kumar, R., Howley, I., & Rosé, C. P. (2009). Engaging collaborative learners with helping agents. In V.
Dimitrova, R. Mizoguchi, B. du Boulay, & A. Graesser (Eds.), Proceedings of the 14th International Conference on Artificial
Intelligence in Education (pp. 365–372). Amsterdam: Ios Press.
Demetriadis, S., Karakostas, A., Tsiatsos, Th., Caballé, S., Dimitriadis, Y., Weinberger, A., Papadopoulos, P.M.,
Palaigeorgiou, G., Tsimpanis, C., & Hodges, M., (2018). Towards Integrating Conversational Agents and Learning
Analytics in MOOCs. In L. Barolli et al. (Eds.): Proceedings of EIDWT 2018, Springer, LNDECT 17, pp. 1–12.
https://doi.org/10.1007/978-3-319-75928-9_98
Dillenbourg, P., & Tchounikine, P. (2007). Flexibility in macro-scripts for computer-supported collaborative learning.
Journal of computer assisted learning, 23(1), 1–13.
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer–supported
collaborative learning. Educational Psychologist, 48(1), 56–66.
Gulz, A., Haake, M., Silvervarg, A., Sjödén, B., & Veletsianos, G. (2011). Building a social conversational pedagogical
agent–design challenges and methodological approaches. In D. Perez–Marin & I. Pascual–Nieto (Eds.), Conversational
agents and natural language interaction: Techniques and effective practices (pp. 128–155). Hershey, PA: IGI Global.
Liu, C. C., & Tsai, C. C. (2008). An analysis of peer interaction patterns as discoursed by on–line small group problem–
solving activity. Computers & Education, 50(3), 627–639.
Michaels, S., O’Connor,M. C., Hall, M.W., & Resnick, L. B. (2010). Accountable talk sourcebook: For classroom that works
(v.3.1). University of Pittsburgh Institute for Learning. Retrieved March 1, 2015, from
http://ifl.pitt.edu/index.php/download/index/ats
42. ARISTOTLE UNIVERSITY OF THESSALONIKIStavros Demetriadis @ eMadrid, May 11, 2018
References 2/2
Michaels, S., O’Connor, C., & Resnick, L. B. (2008). Deliberative discourse idealized and realized: Accountable talk in the
classroom and in civic life. Studies in Philosophy and Education, 27(4), 283–297.
Papadopoulos, P. M., Demetriadis, S. N., & Weinberger, A. (2013). ‘Make it explicit!’: Improving collaboration through
increase of script coercion. Journal of Computer Assisted Learning, 29(4), 383–398.
Tegos, S., Demetriadis, S. N., & Karakostas, A. (2014). Conversational agent to promote students’ productive talk: The
effect of solicited vs. unsolicited agent intervention. In IEEE 14th International Conference on Advanced Learning
Technologies (pp. 72–76). Athens, Greece.
Tegos, S., & Demetriadis, S. (2015). Promoting academically productive talk with conversational agent interventions in
collaborative learning settings. Computers & Education, 87, 309–325.
Tegos, S., Demetriadis, S., & Tsiatsos, T. (2014). A configurable conversational agent to trigger students’ productive
dialogue: A Pilot Study in the CALL domain. International Journal of Artificial Intelligence in Education, 24(1), 62–91.
Walker, E., Rummel, N., & Koedinger, K. R. (2011). Designing automated adaptive support to improve student helping
behaviors in a peer tutoring activity. International Journal of Computer-Supported Collaborative Learning, 6(2), 279–306.
Wang, H. C., Rosé, C. P., Cui, Y., Chang, C. Y., Huang, C. C., & Li, T. Y. (2007). Thinking hard together: The long and
short of collaborative idea generation in scientific inquiry. In Proceedings of the 8th International Conference on Computer–
Supported Collaborative Learning (pp. 754–763). International Society of the Learning Sciences. New Brunswick, NJ.