An invited keynote talk given at the Intelligent Tutoring Systems (ITS) conference in Honolulu, 2014. Begins with some fun observations about being an academic in Hawaii. Motivated both by my early work studying dyadic interaction with Belvedere and a theoretical view of the multi-dimensionality of distributed learning in socio-technical networks and consequent analytic challenges, outlines a framework called "Traces" that addresses these challenges. Most of the examples are of analysis of Tapped In, a successful online network of educational professionals from 1997-2013. Probably the most comprehensive overview of my research to date.

1. Multilevel Analysis of Socially
Embedded Learning
Dan Suthers
University of Hawaii
Supported by the National Science Foundation

3. “Do you go to the beach all
the time?”

4. No.
We do not always go to the beach.

5. We also go to the mountains.

6. “Health hazards on
Mauna Kea:
Altitude sickness.
At the summit
elevation of 13796
feet (4200 m), the
atmospheric
pressure is 40
percent less than
at sea level …”

10. Major Motivations and Ideas
! Learning (particularly in socio-technical
settings) is a complex and embedded
phenomenon
! Multiple theories and levels of analysis are
needed
! Distributed and multimediated nature of socio-technical
systems present analytic challenges
! Approaches illustrated with my work:
! Generalized concept of interaction and the
contingency of acts on their setting
! Abstract transcript and analytic hierarchy

11. Let’s start with Learning
in Socio-Technical Networks …
… and the idea that Learning is
“Embedded” in multiple ways.

12. Learning in Socio-Technical Networks
How do social settings foster learning?
Agency
Who or what is the agent
that learns?
! Individual
! Small groups
! Networks (communities,
cultures, societies)
Epistemologies
What is the process of
learning?
! Acquisition
! Intersubjective meaning-making
! Changes in participation
and Identity
The correspondence is not strict. Epistemologies
can be applied at local or network levels
Based on ! Suthers (ijCSCL 2006)

13. Levels of Agency and Epistemologies
! Acquisition Epistemologies
Learning as acquisition of information, knowledge or skills
! Local: contribution theory, given/new contract, explanation,
conceptual change, practice of skills, etc.
! Network: weak ties, diffusion theories (contagion theory,
diffusion of innovations)
! Intersubjective epistemologies
Learning as intersubjective meaning-making
! Local: co-construction, collaborative inquiry, group cognition
! Network: knowledge building, communities of scientists
! Participatory epistemologies
Learning as changes in social participation and identity
! Local: identity, apprenticeship & mentoring (LPP)
! Network: expansive learning (CHAT)

14. A Complex Multilevel Phenomenon
Claim: individuals participate in the foregoing
forms of learning simultaneously
! One might choose to focus on one form, or
! Grapple with a fundamental question:
How does learning take place through the
interplay between individual and collective
agency in socio-technical networks?
! Requires coordinated multi-level analysis
! Requires coordinated multi-level theorizing

15. Learning is Embedded
! Interactionally embedded
! Learning accomplishments are contingent on
their interactional setting
! Socially embedded
! Social as source of resources
! Social entity as agent of learning
! Technologically embedded
! Affordances influence processes
! Artifacts sustain practices and activity
structures

16. Analytic Challenges
! Embedded: need to say how activity is
contingent on setting
! Multimediated: need media independent
unit of interaction, while being media
aware
! Distributed: need to unify diverse data
streams
! Hierarchical: need multiple levels of
theory and analysis

17. Traces Analytic Hierarchy
Addressing (some of) the needs
Activity is distributed across multiple media
" Abstract transcript representation collects distributed
events from multiple media into a single analytic
artifact, reassembling fragmented record of activity
Local activity is hierarchically embedded in network
settings, calling for coordinated multilevel analysis
" Analytic hierarchy that supports multiple levels of
description (interaction, mediated associations, ties)
and analysis
! Suthers (HICSS 2011)
! Suthers & Rosen (LAK 2011)

18. Interaction Affiliations
Uptake Ties
Contingencies
Mediated Associations

19. Uptake and
Contingencies
How these ideas developed, picking
up where we left off in about 2003 ….
Thanks to NSF, Chris Hundhausen,
Laura Girardeau, Nathan Dwyer, Richard
Medina and Ravi Vatrapu

20. Example 1: First Encounter of Needs

21. Distributed Interaction

22. Motivated concept of “Uptake”
! Needed a cross-media unit out of which
to construct analytic accounts of
interaction
! Media specific concepts (“adjacency pair”,
“edit”, “reply”) are too specific
! Not a new unit, but rather a name given
to all such constructs taken collectively
! Generalize beyond interaction as
“reciprocal action or influence” to other
forms of association

23. Uptake
Minimal requirement for two acts to form part
of an interaction: that the existence of the
first act is consequential in some way for the
second act:
Uptake is present when an act takes some
aspect of a prior act (or event) as relevant
for ongoing activity.
Flexible and Broad: Opens up our thinking about
how interaction might be accomplished

24. Example 2: Asynchronous Dyads
! Asynchronously interacting dyads
! Public heath problem with hidden profile
materials
! Original study: representational guidance of
evidence maps vs. threaded discussion

25. Example: Asynchronous Dyads

26. Closeup

27. Example 2: Interactional Pattern (“W”)
! Information Sharing / Round Trip in Evidence Map
! Subsequent Negotiation in Threaded Discussion

28. Connecting Uptake to Evidence
Motivations
! “How do you know it’s really uptake?”
! Problem of intentionality but also
! Separate evidence from claim
! Manual analysis is slow
! Sufficiently “objective” evidence would also
be computable
! Action is contingent on its setting in
many (observable) ways: let’s use
computational tools to leverage this!

29. Contingencies
Any observed relationship between events
that may evidence how one event may
have enabled or influenced other events
(acts)
! Include “many metaphysical shades
between full causality and sheer
inexistence” (Latour, 2005)
! Contingencies record how each act is
embedded in a history of interaction
and a social and technological setting

30. Some Types of Contingencies
Media Dependency
ei operates on object created
or modified by ej
Temporal Proximity
ei took place soon after ej
Spatial Organization
ei takes place in
configurational context created
by ej
Inscriptional Similarity
ei creates inscriptions similar
to those created by ej
Semantic Relatedness
The meaning of inscriptions
created by ei and ej overlap
Contingencies of ei on ej
(! Suthers Dwyer, Medina & Vatrapu, ijCSCL 2010)

31. Example 3: Early Contingency Analysis
! Analysis originally undertaken to explain convergence
& divergence, but discovered emergence of
representational practices
! First automated construction and visualization of
contingency graph
(# Medina & Suthers, RPTEL 2009)

32. Example 3: Asymmetry in Roles

33. Example 3: Representational Practices

34. Example 3: Episodic View of Interaction
Abstraction to uptake between episodes of specific acts

35. Example 3: Multi-level Analysis
Lemke: "look at at least one organizational level below
the level we are most interested in (to understand the
affordances of its constituents) and also one level
above (to understand the enabling environmental
stabilities)"

36. Traces Analytic Approach

37. Testbed: Tapped In
SRI’s Network of education professionals: PD and peer
support (Mark Schlager, Patti Schank, Judi Fusco)
1997-2013: longest running educational online
community
! 20K educators/year
! 800 user spaces
! 50 tenants
! 40-60 volunteer-run
community-wide
activities/month
! Chats, discussions, wikis, resource sharing ...
Good Testbed: Heterogeneous network of diverse small
groups interacting with multiple media

39. Automatic Discovery of
Distributed “Sessions” and
Influences Between Sessions

40. Overview of Analysis: Process Trace

41. Overview of Analysis: Events

42. Overview of Analysis: Contingencies

43. Overview of Analysis: Uptake

44. Overview of Analysis: Sessions

46. Process Model

47. XML Scripts driving Java, NLTK, iGraph
<!-- ========== Content Preprocessing ========== -->!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.PythonScriptStep" >!
!<stepconfig scriptref="nltk/lancaster_stemmer.py" />!
</step>!
<!-- ========== Contingencies ========== --> !
<step bundlename="apps.analyzer"
classname="apps.analyzer.script.ReadDiscussionMessageRule" />!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.LexicalRule" />!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.ReplyRule" />!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.AddressRule" />!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.SameActorRule">!
!<stepconfig windowsize="300" tag="SA300" />!
</step>!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.TimeWindowRule">!
!<stepconfig windowsize="120" tag="TW120s" />!
</step>!
<!-- ========== Activity Structure (Finding Sessions)========== -->!
<step bundlename="apps.analyzer" classname="apps.analyzer.script.ActivityRule"> !
!<stepconfig graphName="activity">!
! !<weighter fileref="weights/activity_weights.xml" />!
!</stepconfig>!
</step>!
. . . !

48. Case Study
Analyze 3 days of chat,
centered on a session of
interest

49. Teaching Teachers Session
184 23:35 Mary: are all good teachers good mentors?
185 23:38 Amber: some people will take a while to get to that point
186 23:42 Amber: No..not all
187 23:51 Erica: definitely not
188 23:55 Lara: Training can help, but I think some is personality
189 24:09 Amy: some people are excellent teachers but are horrible mentors
190 24:09 Erica: some great teachers can not hold a decent conversation with an
adult
191 24:11 Amber: i had to co-ops who would be awful mentors
192 24:24 Lara: Nods
193 24:27 Dianne: That is an interesting question Maria, ... I would probably say
'yes' first off, and then wonder some more
194 24:42 Mary: it is something I have thought about often Lisa
195 24:47 Amber: I think its alot of personality
196 25:17 Dianne: one thing a mentor has to know is how to operate with a peer,
and how to be intentional about handing over, or encouraging
greater independence
197 25:18 Mary: observation has made me think that it takes an extra “special
ingredient” to tip the scales
198 25:34 Erica: I think if you have the passion for teaching you will want
everyone else to feel the same
199 25:35 Amber: agree

50. Contingencies computed
! Time Window (recency): all chats within 120
seconds
! Last Contribution: last chats by same actor in
300 seconds
! Address: Actor chats ... chat addresses actor
! Reply: Chat addresses actor ... actor chats
! Lexical Overlap: weighted count of overlapping
lexical items (NLTK Lancaster Stemmer)
Weighted sum of counts of above $ estimate
of uptake

51. Uptake Graph for 3 Days of Chat

52. Rooms
One session across
two rooms
Two sessions
in one room

53. Sessions (Modularity Partitions)
One session across
two rooms
Two sessions
in one room

54. Session Partitions Collapsed

55. Inspect group in Data Laboratory

56. Teaching Teachers Session Begins

57. Teaching Teachers Session Ends

58. Sociogram
Folding contributions by actor
to expose actor-actor uptake

59. Session 74, Rooms

60. Session 74, Contributions Colored by
Actor, ForceAtlas2 Layout
Can we characterize
“good” sessions by
structural patterns?
Nodes are contributions, Colors are actors, Node size is weighted indegree

61. Session 74 Sociogram
Nodes are actors
Node size is weighted indegree

62. Selecting second group

63. Same actors in NTraining Session

65. How “Communities” are
Embedded in Technological
Media
Mediated Associations and
Community Detection
! Suthers, Fusco, Schank, Chu & Schlager (HICSS 2013)

66. Interaction Affiliations
Uptake Ties
Contingencies
Mediated Associations

67. Characterization of Community Structure
! “I don’t know what communities are
there”
! Organizational “tenants” and unsponsored
! Multiple, fluid forms of participation
! An empirical matter
! Don’t assume that the network is one
community
! Don’t assume that external communities are
replicated within the sociotechnical system

68. Communities: Technologically Embedded
! Multiple technologies for participation, each with their
own interactional and social affordances
! Choice of technologies reflect and reaffirm the
relationship between interlocutors (Licoppe and
Smoreda, 2005)
! Apply this idea to collective rather than dyadic level:
Communities are embedded within and make use of
technological media for interaction in ways that
reflect and reaffirm their nature
! Our approach identifies cohesive subgroups of actors
and of actants (mediational means) simultaneously
! Suthers & Chu, LAK 2012

69. Intermediate level of representation
! Actor-Actor ties: useful
abstraction, but hide
how enacted
! Intermediate granularity:
mediated association
! Interaction traces (e.g., contingency
graphs): overwhelming detail!

70. Portion of an Associogram
actors
discussions
files

71. Cohesive subclusters in associogram
Modularity
Partitioning
• 234 Partitions
• Modularity: 0.828
Open Ord Layout
in Gephi
Cohesive
subgraphs of
actors and
artifacts via
which they
interact

72. Interpretations of Top 6 Partitions
After School
Online
Events
Associations
via TI
Reception and
other public
rooms
Chat-based
CoP in a
Midwestern
school district;
Discussion-based
professional
development in
the Southern
US
Chat-based
Language
Arts in the US
Midwest;
Pre-service
program in
Western US

73. Myriad of Small Clusters

74. Size distribution of Largest 86 partitions

75. Average weighted degree by actor size
(sample of every 10th partition)

76. Artifact/Actor ratios by actor size

77. Use of media in large and small partitions
! Tenant and
unsponsored are
similar in large
partitions
! In small partitions,
tenants are strongly
chat based while
unsponsored rely on
asynchronous media

78. Summary & Comments
! Purely structural (graph theoretic) computations
identified cohesive subgroups that have interpretations
as communities
! Diversity demonstrates vibrancy of Tapped In as
“transcendent community” (# Joseph et al., CSCL 2007)
! Value to learning analytics: identify social units that
are the setting or agent of learning
! Can “dive in” to examine activity of high-degree
actors, structure of chat sessions in rooms, etc.
! Need algorithm for overlapping cohesive clusters
! Clique percolation fails on bipartite graphs
! Edge communities and flow compression promising
! Suthers, Fusco, Schank, Chu & Schlager (HICSS 2013)

79. Productive Multivocality
Bringing multiple theoretical
and methodological traditions
to bear

80. Productive Multivocality Project
The complexity of learning requires multiple
analytic “voices” (theories and methods): How
to bring them into productive dialogue?
! 5 year project sharing/comparing approaches
to analyzing collaborative learning
! 37+ researchers analyzed 5 corpora
! Suthers, Lund, Rosé, Teplovs & Law
(Springer 2013)

81. Strategies for Productive Multivocality
! Dialogue about the same data, from different
perspectives
! Share an analytic objective (e.g., “pivotal
moments”)
! Bring analytic representations into alignment
with each other and the original data
! Eliminate inconsequential differences and
Iterate
! Push the boundaries of traditions without
betraying
! Reflect on Practice: dialogue about methods
as object-constituting, evidence-producing and
argument-generating tools

82. Brief Comments on Design
Mediating between individual
and group

83. Individual !" Small group
Representational affordances for
intersubjective meaning-making:
! (Im)Mutable Mobiles
! Negotiation Potentials
! Referential Resources
! Reflector of subjectivity (awareness)
! Persistence (reflection)
!S uthers & Hundhausen (JLS 2003)
!S uthers (ijCSCL 2006)

84. Individual !" Network
!J oseph, Lid & Suthers (CSCL 2007)

85. act
persist
find
care
care
act
persist
find
care
act
persist
persist find
find
care
act
Thanks to Viil Lid
for diagrams

86. Key Ideas
! Learning is interactionally embedded
% Contingency and Uptake analysis of
sequential structure
! Learning is socially embedded
% Empirically identify the social units in a STN
! Learning is technologically embedded
% Identify the mediational means (mediated
associations)
! Generalized concepts, abstract transcript ,
and analytic hierarchy help

87. Summary of Concepts
! Mediation and Associations
! All interaction is mediated; actors are associated via media
! Understand how social phenomena are technologically
embedded (! Licoppe & Smoreda, SN 2005; ! Suthers & Chu,
LAK 2012)
! Uptake: (! Suthers, ijCSCL 2006)
! Taking some aspect of (the trace of) a prior act or event as
relevant for ongoing activity
! A generalized unit of analysis for “interaction” broadly
understood (multi/cross-media; inter/intra-subjective)
! Contingencies: (! Suthers Dwyer, Medina & Vatrapu, ijCSCL 2010)
! Manifest relationships between acts and their setting
(including other events)
! Evidence for Uptake

88. Traces
Analytic
Hierarchy
" Abstract transcript
representation that
collects relevant
events into a single
analytic artifact
" Analytic hierarchy
that supports
multiple levels of
analysis
!S uthers, HICSS 2011
!S uthers & Rosen, LAK
2011
Interaction Affiliations
Uptake Ties
Contingencies
Mediated Associations

89. Mahalo!
Dan Suthers, Dept. of ICS
suthers@hawaii.edu
Lilt.ics.hawaii.edu