1. Multi-Level Microanalysis
A Methodology for Tracing the Co-Evolution
of Group and Individual Phenomenon in
Asynchronous Online Discussions
Alyssa Friend Wise
Simon Fraser University
Supported by the Social Sciences and Humanities Research Council of Canada
2. Some Background, Goals & Relevant Issues
Background
Context
• Asynchronous Online Discussion Forums
• Higher Education
Prior Studies
• Connections between Group and Individual Actions
(Knowledge Construction Study)
• Ground-Truthing via a Microanalytic Approach
(Patterns in Learners’ Online Listening Study)
2
3. Influence of Individual Posts on Group Dynamics
(Knowledge Construction Study - Wise & Chiu, ijCSCL, 2011)
Level of Knowledge Construction Contribution by Post
Agreeing &
5
Applying
Testing &
4
Modifying
Negotiating
3
Meaning
Exploring
Dissonance
2
Sharing
Information
1
0 5 10 15 20
3
4. Group Dynamics as Context for Individual Posts
(Knowledge Construction Study - Wise & Chiu, ijCSCL, 2011)
Level of Knowledge Construction Contribution by Post
Agreeing &
5
Applying
Testing &
4
Modifying
Negotiating
3
Meaning
Exploring
Dissonance
2
Sharing
Information
1
0 5 10 15 20
4
5. Ground-Truthing via a Microanalytic Approach
(Patterns in Learners’ Online Listening Study - Wise et al., in press)
Cluster Analysis
Case Study
19 students characterized as
Student 82 (and others!)
“Broad Listeners”
• Initial listening was brief,
• Many sessions, long time in the
punctuated and scattered
system (avg 13.6, 27 min)
• Few intensive sessions at
• Viewed almost all posts at least the end of the week
once (96%, avg 109 views)
• Small number of posts
• Moderate time spent attending to attended to extensively
others posts (avg 4.6 min) • Visited own and other
discussions in off week
6. Some Background, Goals & Relevant Issues
Goals
Macro-goal
• Study the co-evolution of individual- and group-level phenomena
in asynchronous online discussions over time
• Theorize about / model / explain relations between levels of
learning.
Micro-goals
• Identify useful constructs at each level
• Develop methods to trace their trajectories over time
• Identify the ways these constructs can be used to inform each
other / describe how processes at each level situate, constrain,
and influence each other
6
7. Some Background, Goals & Relevant Issues
Relevant Issues
Conceptually
• What kinds of claims can we make about influences across levels
and what kinds of evidence / warrants are needed to support them?
• Are the mechanisms by which group-level phenomena affect
individual-level phenomena different from that through which
individuals influence the group?
– Pivotal posts change the tenor / mode of a group’s conversation
– Ideas move from individual claim to become group “fact” (or visa versa)
– Collective lack of dissent encourages individuals to give up arguments
– Disjointed discussion allows group to accept false “consensus”
7
8. Some Background, Goals & Relevant Issues
Relevant Issues
Methodologically
• How do we know that influences across levels is happening?
Can we pinpoint when? (Is this important?)
– Detection (rapid screening protocols)
– Modelling states and actions (HMM)
• Need for methods that explain not only the “what” but the “how”
– Ground-truthing
• Differing timescales at different levels of analysis
– Nested Timelines in Asynchronous Online Discussions
8
10. Analytical Framework – Dimensions Considered
Doing (Actions) Thinking (Ideas)
Static Dynamic Static Dynamic
I. II. III. IV.
E.g.: How much E.g.: How does the E.g.: What idea(s) E.g.: How does the
Group does the group group participate in does the group group’s idea(s)
participate in the the discussion over have about the change over time?
discussion? time? discussion topic?
V. VI. VII. VIII.
E.g.: How much E.g.: How does a E.g.: What ideas E.g.: How does a
Individual does a student student participate does a student student’s idea(s)
participate in the in the discussion contribute to the change over time?
discussion? over time? discussion?
10
11. Analytical Framework – Dimensions Focused On
Doing (Actions) Thinking (Ideas)
Static Dynamic Static Dynamic
I. II. III. IV.
E.g.: How much E.g.: How does the E.g.: What idea(s) E.g.: How does the
Group does the group group participate in does the group group’s idea(s)
participate in the the discussion over have about the change over time?
discussion? time? discussion topic?
V. VI. VII. VIII.
E.g.: How much E.g.: How does a E.g.: What ideas E.g.: How does a
Individual does a student student participate does a student student’s idea(s)
participate in the in the discussion contribute to the change over time?
discussion? over time? discussion?
11
12. Overview of Analysis Process
Log-file data Group participation Individual participation
patterns patterns
of Actions
Content of Group’s ideas Individual’s ideas
posts
Narrative of Narrative of
the group each individual
Coordinate analyses across
levels and over time
12
13. Log-file Data of Student Actions
Date Time Session Action Duration Length Message #
(min) (words)
6/3/2011 23:46 1 Read 44.43 413 447
6/3/2011 23:52 1 Read 1.73 60 455
6/4/2011 00:08 1 Scan 0.23 117 459
6/4/2011 00:09 1 Read 12.51 413 460
6/4/2011 23:49 2 Post 3.18 120 477
• Actions: View (read or scan), Post, Review (and edit)
– Scan vs. Read (6.5 words per second threshold) (Hewitt 2003)
• Session closed when action length > 60 min
• Summary variables calculated for each student / the group
– e.g. # of actions, # of sessions, time spent reading
13
14. Visualizing Group Participation Patterns
• Left side shows all group posts in threaded structure distributed over time on the
vertical axis. Letters are student initials, P = professor. Right side shows each
individual’s reading actions by day. Column height indicates the total time spent
reading; labels are [# of sessions, # of posts read].
14
15. A Tool to Trace Individual Participation Patterns
Date Time Session Action Duration Length Message #
(min) (words)
6/3/2011 23:46 1 Read 44.43 413 447 Log-file Data
6/3/2011 23:52 1 Read 1.73 60 455
of Student
Actions
6/4/2011 00:08 1 Scan 0.23 117 459
6/4/2011 00:09 1 Read 12.51 413 460
6/4/2011 23:49 2 Post 3.18 120 477
Dynamic discussion
map: a record of the
discussion to show the
historical appearance
of the discussion forum
at any point in time
15
16. Post Content Analysis
• Currently using an argumentation coding scheme
adapted from Clark and Sampson (2008)
– Students may propose, support or rebut ideas
– They may also analyze the problem and/ or synthesize
the group’s position
• Other coding schemes could be used
• Results visualized as argumentation flows for the
group and individuals
16
17. Visual Timelines: Group & Individual Ideas
Group’s collective idea trajectory Individual’s idea trajectories
(each line tracks an ideas) (each line tracks one person’s ideas)
17
18. Overview of Analysis Process
Log-file data Group participation Individual participation
patterns patterns
of Actions
Content of Group’s ideas Individual’s ideas
posts
Narrative of Narrative of
the group each individual
Coordinate analyses across
levels and over time
18
19. Sample Analysis
• Small group online discussion of business
challenges in large blended undergraduate course
– Two solutions given; addt’l ones could be proposed
– Need consensus on one solution
– Discussions ran Friday to Tuesday
– Postings worth 5% of course grade + => presentation
• Sample analysis of one group of five students
– Focus on three students (A, B, C) over two key days
19
21. Arlene
• Started early, set focus for discussion,
reacted to Ben’s initial position
• Strongly committed to own opinions;
little time attending to other’s ideas
• Insistent in driving the group to her sol’n
“Earlier we agreed there are 3 main problems… [but] I just realized that in
our solution, we actually only address problem #2…The face-to-face
meeting with the senior exec might clear up [this] misunderstanding.”
21
22. Ben
• Prolific speaker and active listener;
equally influenced / influenced by group
• Engaged in argument with Arlene but
quickly abandoned his position. Was
still engaged in listening on Sunday
• Posts later that he hadn’t realized that the group needed
to agree on a sol’n
22
23. Celia
• Spent considerable time listening,
but not strong in taking a stance,
emphasized “identify the problem”
• Diplomatic approach to debate: little
preference / opposition to solutions
• In the end was content to follow the
group’s will
23
24. Connections Between Levels
Individual(s) Group
One student took a stand for a solution
Debate dynamic
Prompting another to rebut his idea
Reluctance of others to join-in
Collective lack of dissent
First student abandoned his argument
Initiated many new threads Disjointed discussion
One student presumed consensus
Group set on a solution,
Other students don’t contest other ideas / alternatives
abandoned w/o rebuttal 24
25. Reflections on Multi-Level Microanalysis
Interpretativity
• Log-file data: objective, rule-based processing
• Post content coding: clear, theoretically-grounded scheme
• Narrative creation: repeated negotiation by multiple researchers
Generalizability
• Looking at specific situations to understand how generalizations play
out or can be seen in them.
• Generate hypothesis to be tested more generally at higher levels
Scalability
• By hand > 100 hours to conduct complete analysis
• Exploring graphical / coding automation
• Pinpoint critical time-spans worthy of analysis
25
26. Alyssa Friend Wise
alyssa_wise@sfu.ca
Educational Technology & Learning Design Program
Simon Fraser University | Vancouver, Canada
