Unit-IV; Professional Sales Representative (PSR).pptx
PhD Defense: Computer-Based Support and Feedback for Collaborative Chat Conversations and Discussion Forums
1. Computer-Based Support and Feedback for
Collaborative Chat Conversations and
Discussion Forums
PhD Candidate: Traian Rebedea
Scientific Advisor: Ștefan Trăușan-Matu
2. Contents
• Introduction
• Inter-animation in Online Conversations
– Types of Interactions between Utterances
• Discourse Models for Online Conversations
– Conversation Graph
• PolyCAFe – A Chat & Forum Analysis and Feedback
System
• PolyCAFe Validation Experiments
• Using Semantics for Improving Relevance of Text
Documents
• Conclusions
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
3. Introduction
“Time changes all things; there is no reason why
language should escape this universal law” (
de Saussure, 1915, p. 77)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
4. Introduction
• Evolution of online communication (and the
linguistic artifacts): chat conversation, online
forums, blogs, social networks
• New type of discourse
– Highly collaborative
– Many participants
– Different from narratives, but also from dialogues
• Parallel discussion threads, topic shifts, new
language, etc.
• Not investigated enough, there is no linguistic
theory built for this type of discourse
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
5. Introduction
• Inter-disciplinary research topic
– Computer Supported Collaborative Learning
(CSCL)
– Natural Language Processing
– Collaborative Knowledge Building
• CSCL
– Uses lots of online communication
– Massive Open Online Courses (MOOCs)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
6. Research Question
• How to design and implement a system to provide
automatic feedback and support for learners and
tutors involved in online discussions?
– How to (best) analyze collaborative online discussions
using chat and forums?
– What elements of the proposed theoretical framework
can be used to design a system for automatic analysis of
chats and forums in a CSCL situation?
– How accurate is the feedback offered by such a system?
– Is the system efficient for the learners and tutors that are
using it?
– Can semantics be used in other domains as well?
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
7. Why?
• Which of the previous two messages does
reply 68 continue?
– Does Loredana continue her own message?
– Does she continue the previous message (67)?
66 yes, i think chat is more for brainstorming Loredana
67 chat's greatest advantage is the simplicity it proposes Alex
68 or maybe finding out faster and direct ways of dealing Loredana
with a certain domain
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
8. Why?
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
9. Why?
• Collaboration & parallel discussion threads
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
10. Novelty
• Inter-animation framework for online conversation
analysis
• Study for detecting implicit links in discussions
• Building a conversation graph using natural language
processing (NLP) techniques
• Development of PolyCAFe
– POLYphony-based system for Collaboration Analysis and
Feedback generation
• Validation experiments and results
• Using Wikipedia to extract semantics that may be used
to improve the relevance of online text documents
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
11. Inter-animation in Collaborative
Chat Conversations
“The text lives only by coming into contact with
another text. Only at the point of this contact
between texts does a light flash…” (Bakhtin,
1986)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
12. Scope
• Inter-animation framework of analysis
– Developed for multi-party online conversations
(especially chat and other short text streams)
– Focus on discovering interactions/links/references
among utterances
– Determining discussion threads
– Compute interactions between threads and the
degree of inter-animation or collaborative
discourse
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
13. The Utterance
• “unit of speech communication […] determined by a
change of speaking subjects, that is, a change of
speakers.” (Bakhtin)
• A unit of speech communication – not be related to
existing boundaries in written text, such as phrases,
sentences, paragraphs, etc.
• An utterance does not have a specific length (may vary
from a word to a novel or a treaty).
• Any utterance should be meaningful on its own.
• “Responsive understanding” – other utterances are
linked to previous ones in the same discourse.
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
14. Inter-animation
• Inter-animation: “the meaning of an utterance is
not reducible to the intentions of the speaker or
to the response of the addressee but emerges
between these two.” (Wegerif, 2008)
• Polyphony, heteroglossia, inter-animation
• Intertextuality
• Interdiscourse
• Ideal text, memex, hypertext, WWW
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
15. In Online Conversations
• Interactions or links among utterances and
among different discussion topics
• Explicit links
– #tag
– @username
– Other means for linking utterances
• Implicit links
– Must be determined automatically
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
16. Problems in Online Conversations
• Multiple parallel discussion floors
• Different discussion threads, running in parallel,
inter-animating or competing
• “competition for attention or control of the
discourse” (Cherny, 1999)
• Patterns that have been identified in specific
conversations: speaker-and-supporter,
collaborative and multiple conversational floors
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
17. Similar Problems
• Identifying implicit links in
• Newsgroups
• Blog posts
• Chats
• Thread detection in chat and chat-like
conversations
• Thread reconstruction in discussion forums
• Uptake analysis framework
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
18. Classification of Utterances
• Speech acts
• Dialog acts
• Argumentation acts
• Rhetoric structures
• Transacts
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
19. Example: Argumentation in Chat
34 Mona
Conversation
Ok! So... main the reason I sustain this technology is it's accesibility CLAIM
35 Mona Every person that knows how to type can succesfully be part of a chat DATA
(LINK TO 34)
36 Cristi indeed .... chat is everywhere DATA
(LINK TO 34)
37 Cristi nowadays almost everyone uses one chat client DATA
QUALIFIER
(LINK TO 34)
38 Stefan Why do you say that? Every other techology is everywhere. CLAIM2
REBUTTAL
(LINK TO 34, 36)
39 Corina true.... (LINK TO 39)
40 Mona By accesibility I was also referring to the way you use it... chat is one of the WARRANT
most simple means of communication nowadays (LINK TO 34)
41 Stefan that's true. (LINK TO 40)
42 Cristi is not very complex ... mainly is just typing BACKING
(LINK TO 40)
43 Mona Even a 7 year old kid can use messenger for example BACKING
(LINK TO 40)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
20. Classification of Interactions between
Utterances
• Interactions based on Lexical Cohesion
– Reiterations
– Collocations
– Lexical chains
• Adjacency pairs
• Coherence relations
• Interactions based on Semantic Similarity
• Anaphora and Coreference Resolution
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
21. Lexical Chains in Chats
64 and if you do not have a good internet connection, clasic chat chould also be a solution... Mona
65 agree...videochat requires some bandwidth Cristi
66 ok...but what about information, what do you think it is the most reliable source of Diana
information?
67 chat is fast, simple to use and it does not require such a good internet connection Mona
68 if it is an inside network i don't think this should be a problem Stefan
69 well depends on the type of the meeting... Corina
70 what do you mean by inside network? Mona
71 in the same building? Mona
72 LAN Stefan
73 nowadays a fast internet connection is everywhere :) Cristi
74 or VPN Stefan
75 true.... internet connection is not really a problem nowadays... Mona
76 so we all agree that videochat is the best solution for a "virtual" meeting Mona
77 undoubtedly Cristi
78 yes! Corina
79 so.... one activity solved.... Mona
80 December 2012, meetings we have video chat
14 ok so for PhD Defense - Computer-Based Support and Feedback for
Stefan
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
22. Semantic Similarity
• Knowledge-based methods
– Dictionaries, thesauri, ontologies, other semantic
networks (DOLCE, SUMO, Cyc)
• The length of the path between the two concepts
• The depth of the node in the IS-A taxonomy
• The lowest common ancestor (LCA or lowest super-
ordinate – LSO)
• The information content of a concept
• The commonality and difference between the two
concepts in the IS-A taxonomy
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
23. Semantic Similarity
• Corpus-based methods
– Statistical analysis of occurrences and co-occurrences of
words in a large collections of documents
• PMI (Point-wise mutual information)
• GND (Google Normalized Distance) WWW-based
• LSA (Latent Semantic Analysis)
• LDA (Latent Dirichlet Allocation)
• ESA (Explicit Semantic Analysis) Wikipedia-based
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
24. Discourse Models for Web
Conversations like Chats & Forums
“Practically everything is a graph”
XKCD comics:
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
25. Types of Discourses
• OLAC (Open Language Archives Community)
• Dialogue: “conversations, interviews, correspondence, consultations, greetings
and leave-takings”
• Drama: “plays, skits, and enacted scenes”
• Formulaic: “prayers, curses, blessings, charms, curing rituals, marriage vows, and
oaths”
• Ludic: “play languages, jokes, secret languages, and speech disguises”
• Oratory: “Public speaking, or of speaking eloquently according to rules or
conventions”
• Narrative: “historical, traditional, and personal narratives, myths, folktales, fables,
and humorous stories”
• Procedural: “recipes, instructions, and plans”
• Report: “news reports, essays, and commentaries”
• Singing
• Unintelligible speech
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
26. Interactions/Links among Utterances
• “The base condition governing the operation
of meaning in any utterance. It is that which
insures the primacy of context over text. At
any given time, in any given place, there will
be a set of conditions – social, historical,
meteorological, physiological – that will insure
that a word uttered in that place and at that
time will have a meaning different than it
would have under any other conditions”
(Bakhtin, 1981, p. 428)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
27. Modelling Conversations as a
Graph
• Many discourse theories define links between
discourse units
– Rhetorical structure
– Coherence relations
– Adjacency pairs
• Stavrianou: post-opinion graph for online
forums
• Suthers: contingency graphs, uptake graph
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
28. Conversation Graphs
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
29. Conversation Graphs
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
30. Explicit Links
• Corpus with 8 chats, 2800+ utterances,
1900+ explicit links
0.25
0.2
0.15
Distribution of explicit links
Poly. (Distribution of explicit
0.1
links)
0.05
2
R = 0.9927
0
0 5 10 15 20
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
31. Shallow Links
• Broken utterances / split turns
• Reasons:
– Limitations in the length of the utterance caused by the
environment
– Incomplete turns due to haste
– Trying to grab the discussion floor and the attention of the
other participants although an utterance is still not finalized
– Lengthy turns are seldom broken into multiple shorter ones
– Allowing the other participants to continue the utterance that is
not finalized or even incomplete ideas
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
32. 29 how could we forget the Michael Example 1
30 blog Michael LINK TO 29
…
36 let's try to focus on the subject matter Michael Example 2
37 if you had to choose ONE reason Michael
38 for choosing your technology Michael LINK TO 37
39 what would that one reason be ? Michael LINK TO 37
…
66 yes, i think chat is more for brainstorming Loredana Example 3
67 chat's greatest advantage is the simplicity it proposes Alex
68 or maybe finding out faster and direct ways of dealing with a Loredana LINK TO 67
certain domain
…
70 true, however, in most projects you need a data format with Michael Example 4
more structure
71 and that is why i think Wave offers a good trade-off between Michael LINK TO 70
structure and interactivity
…
75 A chat or a chat room has a more private nature than blogs or Alex Example 5
waves or wikis
…
78 but privacy isn't really the best choice if you have an urgent Cristi LINK TO 75
matter and you're looking for answers
…
123 of course Michael Example 6
124 but to make things interesting Michael LINK TO 123
125 we're in different cities/countries/timezones Michael LINK TO 124
126 i.e. we can't be in the same room Michael LINK TO 125
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
33. Lexical Links
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
34. Semantic Links
0.12
0.1
0.08
0.06 LSA similarity score
between utterances
0.04
0.02
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
35. WordNet – Resnik
0.25
0.2
0.15
WordNet similarity score
between utterances
0.1
0.05
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
36. 148 ok.... so what we were talking about before the Mona
connection issue?
149
150 meeting board
LSA for implicit links
as long as it depends on an internet connection... Mona
Cristi
LINK TO
147
LINK TO score(150, 148) =
148 0.18 = max
151 about different stages of a project... a client must Corina LINK TO
know about them..so wiki is a good solution 148
152 meeting? Corina LINK TO score(152, 150) =
150 1.00 = max
153 ok Stefan LINK TO
151
154 so we agree that wiki is a good solution when we want Corina score(154, 151) =
to present a product/the evolution of a project to a 0.5 = max
client ?
155 what about selling our products..what technologies we Diana
should use for this?
…
166 to sell our products blog is the best solution Corina LINK TO score(166, 155) =
155 0.38 = max
…
169 A blog would be a good way to advertise our products Mona LINK TO score(169, 166) =
166 0.48 = max
…
180 A forum would be useful for offering solutions to some Mona
problems that our customers have
181 I agree...also other people can offer solutions, not only Corina LINK TO score(181, 180) =
us 181 0.27 = max
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
37. How useful are semantic links for
implicit links’ detection?
131 132 133 134 135 136 143 An5 Tot
References greater than
0.44 0.47 0.41 0.45 0.42 0.43 0.45 0.43 0.44
10-turn median
References greater than
0.40 0.47 0.41 0.47 0.42 0.45 0.46 0.45 0.44
20-turn median
Pearson correlation
between number of
0.83 0.90 0.81 0.88 0.76 0.92 0.78 0.83 0.84
references and average
similarity score
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
38. Open Issues in Building the
Conversation Graph
• Automatic Identification of Utterance Types
– Not just speech acts, but also argumentation
– Machine learning vs rule-based
• Questions and Answers in Online
Conversations
– Questions are easier to identify
– Pairing answers to questions is much more
difficult
• Conversation-Focused Coreference Resolution
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
39. Developing PolyCAFe
“In theory, theory and practice are the same. In
practice, they are not.” (attributed to various, in
press)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
40. Problems
• P1. Need to integrate feedback on different
levels: conversation, utterance, participant
• S1. Mixture of techniques:
– Natural Language Processing
– Social Network Analysis
– Information Retrieval
• P2. Supplementary, focus on measuring
collaborative discourse as a key characteristic of
successful online textual interaction
• S2. New theory of discourse for chat
conversations with multiple participants
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
41. Proposed Solution - PolyCAFe
• Chat and Forum Analysis and Feedback System
• PolyCAFe = Polyphony-based Collaboration Analysis
and Feedback generation
• Provides feedback to learners, tutors and teachers
related to the interaction of students in online
discussions
• Takes into account both:
– Content of the conversation (related to a domain or topic)
– Collaboration (related to the conversation, participation,
etc.)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
42. Implicit Links in PolyCAFe
• Repetitions (expanded using ontologies)
• Lexical chains (using WordNet, LSA and ESA)
• Semantic similarity (including lexical chains)
• Adjacency pairs (using speech acts)
• Cue phrases (“that’s a good idea”, “as <x> said”, …)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
43. Technical architecture
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
44. Utterance Evaluation & Threads
Thread Coherence Content Future Impact
Relevance
Completeness
Utterance
Centrality
In-Degree Social Out-Degree
Past Current Future
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
45. Utterance evaluation
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
46. Lexical Chains Using ESA
employees employe employee employer employers
audit auditing statistics audits business
provides provide service communication
executives state world constitution
companies company corporate market
redundant information technology
chats clients irc chat client
comes google come news refers
friends live life status work
policies prfit public policy
real complex analysis number
programs computer software
marketing profitability
discussing forums forum
personal inter-personal
users interface window
question ask questions
room rooms chat-rooms
research university
brainstorming mind
music song artist
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
47. Conversation Visualization
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
48. Utterance Feedback
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
49. PolyCAFe Validation Experiments
«Validation ensures that “you built the right
thing”. Verification ensures that “you built it
right”.» (in press and in project meetings)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
50. Validation Experiment
• 35 students part of the HCI course
– Experimental group: 25
– Control group: 10
• Divided in teams of 5
• Had two distinct assignments that were correlated
• 6 tutors had to provide manual feedback for the
students: using and not using PolyCAFe
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
51. Tutor Efficiency
• VT1: Tutors/facilitators spend less time preparing
feedback for learners compared with traditional means
• Likert questionnaire: everyone agreed that “they find
the information needed to write the feedback for the
learners more quickly using PolyCAFe than without it”
(average=4.7, agree=100%)
• Comparison between average time needed to prepare
feedback for a conversation with and without the
system:
– without PolyCAFe: 84 minutes
– with PolyCAFe: 55 minutes
– Improvement: 35%
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
52. Quality and Consistency of Feedback
• VT2: Learners perceive that the feedback
received from the system contributes to
informing their study activities
• Logging: 285 visits to PolyCAFe and 1447
page-views, that results in more than 40 page-
views in average per student.
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
53. Quality and Consistency of Feedback
% Agree /
Validation statement Average
Strongly agree
The information the system provides me is
accurate enough for helping me perform 3.7 60%
my learning tasks.
PolyCAFe's feedback is sufficiently
accurate to inform my study activities. 3.8 64%
PolyCAFe provides feedback that is useful
to my study activities. 3.8 72%
PolyCAFe provides feedback that is
relevant to my study activities. 3.9 72%
I trust PolyCAFe to provide helpful
feedback. 4.0 80%
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
54. Quality of Educational Output
• VT3: Learner performance in online
discussions is improved in the areas of content
coverage and collaboration when using
PolyCAFe
• Measurements computed for the second chat
assignment, by comparing experimental with
control groups
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
55. Quality of Educational Output
Experimental Control Improvement over
group group control group
Average score for a chat
conversation
(collaboration + content) 6.80 6.37 6.8%
Average importance of
the most important 20 0.194 0.192 1.2%
concepts
Average number of
utterances 351 338 3.8%
Average distribution of
(implicit and explicit) 1.12 0.87 29%
links between utterances
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
56. Verification Experiments
• Utterance scoring
• Participant ranking
• Speech acts classification
• Evaluation of collaboration score
• & more
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
57. Utterance Scoring
• Chat 1 (331 utterances):
• Scores: 1 (not important) – 4 (very important)
• Tutor 1–Tutor 2 (inter-rater) correlation:
61%
• Tutor 1 – PolyCAFe correlation: 60%
• Tutor 2 – PolyCAFe correlation: 51%
• Tutor average – PolyCAFe correlation: 57%
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
59. Participant Ranking
Rankings Average
Correlation Precision
compared distance
Tutors – System 94% 77% 0.23
Students – System 84% 66% 0.43
Tutors – Students 84% 71% 0.40
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
60. Using Semantics for Improving
Relevance of Text Documents
“All our work, our whole life is a matter of
semantics, because words are the tools with
which we work, the material out of which laws
are made […]. Everything depends on our
understanding of them.” (Felix Frankfurter)
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
61. Wikipedia-Based Semantics for
Improving Search Relevance
Refined Rank = Rank(Google) + p * S
score
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
62. Improved Semantics
Source Concept Destination Concept Score
avl tree self balanced binary tree 0.703385
avl tree amortization analysis 0.605471
avl tree donald knuth 0.557814
avl tree b tree 0.546726
avl tree binary tree 0.527878
avl tree computer science 0.527694
avl tree persistent data structure 0.500780
avl tree red black tree 0.441281
avl tree associative array 0.437683
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
63. Relevance for Ranking Video
Comments on YouTube
• The Beatles- Here Comes The Sun
• On YouTube we have two options
1. Top comments
– Really? Who would've thought?
– i think the beatles are better then one derection
1. Chronologic
– Pathetic attempt to get thumbs up
– Why are people comparing The Beatles to all these other
musicians? one direction and Justin Bieber are musicians
too, but we prefer The Beatles.
– yes of course they are?!
–…
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
64. Solution
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
65. Our results
Comment Relevance
maybe your friend should know that being english, have a picture in abbey road 662
and "sing" all you need is love" won't make one direction? a group like the
beatles...
my mom said she doesn't like the beatles and she said that john was only good 968
to look at? not to hear. my dad said, " haha so true!." i'm an orphan now.
you shouldn't be listening to the beatles since these seem to turn your friends 983
into enemies! beatles are all about peace!? you are not getting their message!
please read this ! hey i know u just wanna listen to the song but i still have to 1309
write this hoping someone will see it and that someone will care .i'm a? young
musician from croatia so this spam is my only chance to get noticed.please check
out my channel and i promise u won't be sorry.i appreciate your time because
music means everything to me, thank you! ?
i didn't mean fight other places. i meant focus on the hurt people in your own 1639
country first, then expand to the others. if people don't agree with peace that's
an opinion. not a fact, and people often take offense to opinions. there isn't?
anything to take offense to, they say something that's all it is. they said it, don't
put meaning to it. world peace - i meant the whole world having peace there
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
66. Overall Conclusions
“I'm sorry I wrote you such a long letter; I didn't
have time to write a short one.” (attributed to
Blaise Pascal, in press)
Inter-animation, implicit links,
theoretical framework, conversation graph,
PolyCAFe system,
validation experiments, measurements,
semantics for improved relevance
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
67. •
•
Conference proceedings and journal articles
Publications
Trausan-Matu S., Dascalu M., Rebedea T. - A System for the Automatic Analysis of Computer-Supported Collaborative Learning Chats, accepted at ICALT 2012 (ISI Web of Knowledge,
Category B in ARC Conference Ranking)
• Rebedea T., Dascalu M., Trausan-Matu S., Armitt G., Chiru C.G. - Automatic Assessment of Collaborative Chat Conversations with PolyCAFe. EC-TEL 2011, LNCS 6964, Springer, 2011, pp.
299-312 (ISI Web of Knowledge)
• Dascalu M., Rebedea T., Trausan-Matu S., Armitt G. - PolyCAFe: Collaboration and Utterance Assessment for Online CSCL Conversations, in CSCL 2011 Conference Proceedings, Hong Kong,
vol 2, pp. 781-785 (ISI Web of Knowledge, Category A in ARC Conference Ranking)
• Banica D., Trausan-Matu S., Rebedea T. - Detecting Collaboration Regions in a Chat Session, in CSCL 2011 Conference Proceedings, vol 2, Hong Kong, pp. 771-775 (ISI Web of Knowledge,
Category A in ARC Conference Ranking)
• Dessus P., Trausan-Matu S., Wild F., Dupré D., Loiseau M., Rebedea T., Zampa V. - Un environnement personnel d'apprentissage évaluant des distances épistémiques et
dialogiques. Distances & Savoirs, 9(4), 473–492
• Chiru C., Cojocaru V., Trausan-Matu S, Rebedea T., Mihaila D. - Repetition and Rhythmicity Based Assessment for Chat Conversation. ISMIS 2011, LNAI 6804, Springer, 2011, pp 513-523.
• (ISI Web of Knowledge, Category C in ARC Conference Ranking)
• Wild F., Ullmann T., Scott P., Rebedea T., Hoisl, B. (2011) - Applicability of the technology acceptance model for widget-based personal learning environments. In: 1st Workshop on
Exploring Fitness and Evolvability of Personal Learning Environments (EFEPLE'11), 2011, France, http://www.ceur-ws.org/Vol-773/EFEPLE-10-Wild.pdf
• Trausan-Matu S., Rebedea T. - A Polyphonic Model and System for Inter-Animation Analysis in Chat Conversations with Multiple Participants, CICLING 2010, LNCS 6008, Springer, 2010,
pp.354-363 (ISI Web of Knowledge, Category B in ARC Conference Ranking)
• Rebedea T. et al. - Overview and Preliminary Results of Using PolyCAFe for Collaboration Analysis and Feedback Generation, EC-TEL 2010, LNCS 6383, Springer, 2010, pp. 420-425 (ISI Web
of Knowledge)
• Rebedea T., Posea V., Dascalu M., Trausan-Matu S. - Web Services for Supporting the Interactions of Learners in the Social Web, accepted at 9th International Conference Roedunet 2010
(ISI Web of Knowledge)
• Scheau C., Rebedea T., Costin C., Trausan-Matu S. - Improving the Relevance of Search Engine Results by Using Semantic Information from Wikipedia, 9th International Conference
Roedunet 2010, pp. 151-156 (ISI Web of Knowledge)
• Rebedea T., Trausan-Matu S. - Automatic Feedback System for Collaborative Learning Using Chats and Forums, CSEDU 2010 (ISI Web of Knowledge)
• Chiru C.G., Cojocaru V., Rebedea T., Trausan-Matu S. - Malapropisms Detection and Correction Using a Paronyms Dictionary, a Search Engine and WordNet, ICSOFT 2010 (ISI Web of
Knowledge, Category B in ARC Conference Ranking)
• Chiru C.G., Hanganu A., Rebedea T., Trausan-Matu S. - Filling the Gaps Using Google 5-Grams Corpus, ICSOFT 2010 (ISI Web of Knowledge, Category B in ARC Conference Ranking)
• Trausan-Matu S., Posea V., Rebedea T., Chiru C. - Using the Social Web to Supplement Classical Learning. In Advances in Web Based Learning – ICWL 2009, LNCS 5686, Springer, 2009, pp.
386-389 (ISI Web of Knowledge)
• Rebedea T., Trausan-Matu S. - Computer-Assisted Evaluation of CSCL Chat Conversations, CSCL 2009 (poster) (ISI Web of Knowledge, Category A in ARC Conference Ranking)
• Dessus P., Trausan-Matu S., Zampa V., Rebedea T., Dascalu M. - Vers un environnement-tuteur d’apprentissage dialogique, 2e Colloque Echanger pour Apprendre en Ligne (EPAL’09),
Grenoble : France, http://hal.archives-ouvertes.fr/hal-00404842/fr/
• Trausan-Matu S., Rebedea T. - Ontology-Based Analyze of Chat Conversations. An Urban Development Case, Towntology Conference 2009. In Teller (ed.) Proceedings of Towntology
Conference, Liege, 2009
• Rebedea T., Trausan-Matu S., Chiru C.G. - Extraction of Socio-semantic Data from Chat Conversations in Collaborative Learning Communities, EC-TEL 2008. In P. Dillenbourg and M. Specht
(Eds.): EC-TEL 2008, LNCS 5192, Springer, 2008, pp. 366–377 (ISI Web of Knowledge)
• Rebedea T., Trausan-Matu S. - Autonomous News Clustering and Classification for an Intelligent Web Portal, ISMIS 2008. In A. An et al. (Eds.): ISMIS 2008, LNAI 4994, Springer, 2008, pp.
477–486 (ISI Web of Knowledge, Category C in ARC Conference Ranking)
• Trausan–Matu S., Rebedea T., Stahl G. - Polyphonic Analysis of CSCL Chats, ICLS Workshop: A Common Framework for CSCL Interaction Analysis, Utrecht, The Netherlands, 2008
14 December 2012,
• Trausan-Matu S., Rebedea T., Dragan A., Alexandru C. - Visualisation of Learners’ Contributions in Chat Conversations, Workshop on Blended Learning at ICWL 2007. and Feedback for
PhD Defense - Computer-Based Support In Fong., J., Wang, P.
BucharestBlended Learning, Pearson Prentice Hall, 2007, pp. 215-224
(Eds.), Romania
Collaborative Chat Conversations and Discussion Forums
68. Publications (2)
• Book Chapters
• Rebedea T., Trausan-Matu S., Chiru C. - Inter-animation between Utterances in Collaborative
Chat Conversations, submitted for Innovative Methods for Electronic Discourse Analysis, IGI
Global, 2013, to appear
• Trausan-Matu S., Rebedea T., Dascalu M. - Analysis of discourse in collaborative Learning
Chat Conversations with Multiple Participants, ỉn Tufis D., Forascu C. (eds.), Multilinguality
and Interoperability in Language Processing with Emphasis on Romanian, Editura Academiei,
2010
• Trausan-Matu S., Rebedea T. - Polyphonic Inter-Animation of Voices in VMT, in Stahl G. (ed.),
Studying Virtual Math Teams, Springer, 2009
• Rebedea T., Trausan-Matu S. - Identification of Discussion Threads in Chat Conversations, in
Trausan-Matu S. (ed.), Conversational Analysis in Online Collaborative Systems, Matrix-Rom,
2008 (in Romanian)
• More than 20 papers co-authored in international workshops, Romanian conference
proceedings and journals
• 11 deliverables co-authored within the FP7 ICT Language Technologies for Lifelong Learning
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
(LTfLL) project
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
69. THANK YOU!
Questions
&
Feedback
14 December 2012, PhD Defense - Computer-Based Support and Feedback for
Bucharest Romania Collaborative Chat Conversations and Discussion Forums
Hinweis der Redaktion
Utterance = interventia
“ Within the arena of almost every utterance an intense interaction and struggle between one's own and another's word is being waged, a process in which they oppose or dialogically interanimate each other.” (Bakhtin, 1981)
A fost un drum lung, interesant si cateodata anevoios, Insa am cunoscut multi oameni extraordinari, cu care am mers mai departe – colegi din facultate, din proiecte, studenti Lor trebuie sa le multumesc, dar si familiei si prietenilor, dl.prof. Trausan pentru sfaturile primite in ultimii multi ani, precum si comisiei de doctorat pentru efortul intreprins in a intelege si imbunatati anumite aspecte ale tezei.