1. Toward Defining, Justifying,
Measuring, and Supporting
Social Deliberative Skills
Tom Murray
UMass Amherst
At AIED July 2013, Memphis:
Workshop on Self-Regulated Learning in Educational
Technologies: Supporting, modeling, evaluating, and fostering
metacognition with computer-based learning environment
5. Meaning Negotiation
Conflict Resolution
Skills needed to bridge different perspectives to
build mutual understanding and mutual regard
Social Deliberative Skills:
The capacity to deal productively with
heterogeneous goals, values, or perspectives in
dialogue and deliberation
– Including: collaboration, problem solving, knowledge
building, inquiry learning...
6. Overview
• Background
• Supporting them online
– Participants
– Facilitators
• Measuring them
– Human coding
– Machine classification
• Deeper exploration of their meaning
– And issues with construct definitions (ontology)
10. Examples of Social Deliberative
Skills/Behavior
From authentic dialogues in our online
corpora
“ I am probably extremely biased because I am
under 21 years old and in college. I wonder if as a
45 year old I will feel differently. ” (self reflection)
“I can’t help but imagine what that is like, for her
and for her family.” (perspective taking)
10
11. Code Frequencies in Several Domains
Exp. Group Total_
SD_Skill
Intersubjective
speech acts
Vanilla (N = 8) 0.29 (0.07) 0.20 (0.09)
Reflective Tools (N = 8) 0.40 (0.08) 0.30 (0.08)
11
• A significant difference and main effect between
Total-SD-Score and grouping, F(1, 14) = 6.89, p =
0.02*, d = 1.46 (a large effect) in favor of the
Reflective Tools group
• A significant relationship between Intersub and
grouping, F(1, 14) = 4.81, p = 0.05*, d = 1.05 (a large
effect) in favor of the Reflective Tools group
12. Support/Scaffolding
(vs. “Education”)
Online Dialogue &
DELIBERATION
Outcomes:
- Agreements/solutions
- Relationship, Trust (social capital)
- SKILL USE (and practice)
Existing
Skills
Adaptive
Support
(4th party)
Passive
Support
(interface)
Facilitator
Support
(Dashboard)
13. [CURRENT] WEEK 1: Discuss the pros and cons of leg...
UPDATE PROFILE
LOG OUT
HOME
Logged in as tomm
[CURRENT] WEEK 1: Discuss the pros and cons of legalizing marijuana.[CURRENT] WEEK 1: Discuss the pros and cons of legalizing marijuana.
To focus the conversation, we invite you to assume you are on an advisory panel for the state
legislature, having some preliminary conversations online, and you will eventually be drafting
a group recommendation. Consider not only your own preferences but what is best for the
state (or society).
edit delete
CONTRIBUTE YOUR THOUGHTS
14:53 EDT Sunday, November 13 by tomm
tomm has joined the conversation
23:53 EDT Saturday, November 12 by ines- v
ines-v added a resource: 'Getting a Fix'
23:52 EDT Saturday, November 12 by ines- v
I have to disagree with your third point that marijuana is a gateway drug. Of
all the people I know that smoke marijuana, they do not do any hard drugs.
I do agree that gateway drugs exist, however I feel like that typically
happens from one hard drug to another when one doesn't seem to be
enough. But if you want to talk about gateway drugs we would also have to
mention alcohol and cigarettes which many people consume and smoke.
Alcohol and cigarettes are also drugs and often considered gateway drugs.
They are both legal so that option is void in regards to marijuana.
You also mentioned cancer and other lung related issues. Marijuana is a
natural plant. Cigarettes are made up of extremely harmful chemicals that
cause lung related issues and cancer much faster than marijuana ever could.
Yet, they are still legal. If anything, cigarettes should be illegal when
considering public health. Marijuana is a lot safer than cigarettes.
I do appreciate you playing Devil's advocate though!
I'd like to explain how I see it differently (ines-v)
18:26 EDT Friday, November 11 by arthur- x
It seems like the vast majority is supportive of the legalization of marijuana,
so I'm going to play devil's advocate in order to bring the opposition's side
to the table.
First off, research has demonstrated that marijuana use reduces learning
ability by limiting the capacity to absorb and retain information. A 1995
study of college students discovered that the inability of heavy marijuana
users to focus, sustain attention, and organize data persists for as long as 24
hours after their last use of the drug. Earlier research, comparing cognitive
abilities of adult marijuana users with non-using adults, found that users fall
short on memory as well as math and verbal skills. Although it has yet to be
proven conclusively that heavy marijuana use can cause irreversible loss of
intellectual capacity, animal studies have shown marijuana-induced
ines-v
arthur-x
joseph-t
laura-t
rtwells
matthew-s
tomm
DIALOGUE TABLE
Everyone (no demographics set)
13
32. Primary SD-Skills
• Differentiating facts/opinions (knowing how
to reason about each)
• Reflecting on biases and assumptions (mostly
one's own but also others)
• Perspective taking (of actual interlocutors; of
other groups/identities/cultures, etc.)
• Reflecting on the dialog as a whole (meta-
dialogue)
33. Social Deliberative Skills V-2
Social/Emotional/Reflective
• 1. Social perspective taking
(cognitive empathy, reciprocal role
taking...)
• 2. Social perspective seeking (social
inquiry, question asking skills...)
• 3. Social perspective monitoring
(self-reflection, meta-dialogue...)
• 4. Social perspective weighing
(reflective reasoning; comparing and
contrasting views...)
33
35. Social Deliberative Skill:
application of HOSs to me/you/we
Higher Order Skills
• argumentation
• critical thinking
• explanation & clarification
• inquiry/curiosity (questioning)
• reflective judgment
• meta-cognition
• epistemic reasoning
Apply these skills, not to
EXTERNAL REALITY (“IT”/problem
domain) but to the
INTERSUBJECTIVE domain
Higher Order Skills applied to:
SELF
goals; level of certainty;
feelings, values, assumptions…
YOU
goals, assumptions, feelings,
values; perspective taking;
"believing" & cognitive empathy…
WE
agreements, goals; quality of
the discourse/collaboration;
differences and similarities in
values, beliefs, goals, power, roles…
39. A journey through
ontological conundrums
• Below is additional material related to the
unavoidability of construct overlap and
indeterminacy in defining and using abstract
concepts such as metacognition, inquiry, etc.
(See overlap in Kuhn slide above)
40. “ ...to Understand, Measure &
Support SD-skills”
• Reviewer: “How can study them if you don’t
understand them and can’t define them
precisely?”
41. • Scholarly work is
"notoriously fraught
with definitional
disagreement"
(Shermer, 2011)
An invitation to co-explore
Ontological Indeterminacy
42. Kurt Fischer’s Dynamic Skill Theory
• Skills develop in response to real tasks
• Higher order skills are built doing complex
tasks; no isolated task, therefore no isolated
skill (compare “leadership” skill with “territoriality” biological
drive)
• ...skills [and knowledge] are not isolated units, but rather
function together in complex structures of
inter participation...an ecosystem...any given skill
requires the existence of various others as component [or
interacting] parts...
44. Example: “intelligence” and IQ
• Real life target Task
• Skills (& knowledge)
• Measurement
Caution:
the construct definition can be reduced to the
measurement definition or enactment
45. SD-Skills: task-oriented definition
Skills needed to bridge different perspectives to
build mutual understanding and mutual regard
Social Deliberative Skills:
The capacity to deal productively with
heterogeneous goals, values, or perspectives in
dialogue and deliberation
– Including: collaboration, problem solving, knowledge
building, inquiry learning...
46. Concept Indeterminacy
(Lakoff)
• Abstract Concepts:
–Graded (fuzzy)
–Have “metaphorical pluralism”
–Metaphorical (limited by embodiment)
• More abstract => More indeterminate
Tom Murray | www.perspegrity.com |
August 2010
46
47. Two sources of indeterminacy
• 1. Task structure: Higher order skills are built
doing complex tasks; no isolated task
therefore no isolated skill
• 2. Nature of categories: Abstract concepts are
indeterminate by nature
52. Debate, Dialogue and Deliberation
Deliberation: “thoughtful, careful, or lengthy consideration by individuals; and
formal discussion and debate in groups” (Davies & Chandler 2011)
Editor's Notes
New domain, still catching up on interdscipl literature; hear to learn
& meta-memory; meta affect;
when we slice the world into categories we simplify and may ignore what is between or outside them
so we should take caution; or periodically revisit our construct definitions and ontologies
at the algorithm level: adjusting class weights, cost, or priors for different classifiers, including support vector machine (SVM) with different hyper parameters (e.g., kernels, degree), SVM-one class, naïve Bayes, decision tree, random forest, bagging trees, and boosting treesat the data level: using different sampling techniques (e.g., up-sampling and down-sampling) to balance the data before training a classifier. When using a SVM classifier with a polynomial kernel, we have achieved similar result as we have done using the first approach.A high variance model – training error is much lower than testing errorBagging is often an effective technique for solving the high variance problem However, because our data is highly skewed, it is not surprising that bagging does not work wellA new adaptive sampling algorithm for imbalanced datathe algorithm works as follows: (1) it iteratively performs down-sampling without replacement in order to fully utilize the training data and ensure that there are great overlapping in different sampling spaces, (2) it runs SVM multiple times using different samples, and (3) it decides the class labels for testing data based on majority vote using classifiers built in step (2). Co-training algorithms for imbalanced multi-class classificationthe key idea of co-training is that two classifiers trained on two different views train each other using the unlabeled data two views: cohesion-based features (coh-metrix), lexical features(LIWC)Latent Deliberative Skill Model (LDSM) for Skill Classificationa mixture of only 5 skill labels are modeled, e.g, perspective taking, big-picture thinking the generative processchoose a distribution over 5 skill labels for a documentpick up a skill label from the skill label distribution draw a word according to the skill label-word distribution.
