This document discusses the relationship between multidisciplinarity and multivocality in the context of learning analytics research. It argues that while learning analytics began as a multidisciplinary field, approaches like productive multivocality that involve performing multiple analyses from different frameworks can help drive the field towards greater integration and interdisciplinarity. The document uses an example from a prior multivocality study to illustrate how comparing analyses across frameworks can lead researchers to refine concepts, make positions more explicit, and potentially achieve some methodological integration.

1. Multidisciplinarity vs. Multivocality
the case of “Learning Analytics”
Nicolas Balacheff1
, Kristine Lund2
,
CNRS1,2
, University of Grenoble1
, University of Lyon2
Learning Analytics & Knowlege Conference
April 8-12, 2013
Leuven, Belgium
Laboratoire d’Informatique de Grenoble

2. 2
Learning Analytics / Educational Data Mining
- Conceptually grounded, coined to respond to research needs
- Socially grounded, adopted as common conceptual flagships
Multidisciplinarity,
multivocality and
interdisciplinarity
To what extent can the different disciplines involved in the TEL
community be integrated on methodological and theoretical
levels?
Origin of concepts and methods in the TEL research area
Problématique,
theoretical framework
and methodology
In which way does each expression solve problems
identified in the TEL research area and how specific are
they? What relations do they have with other concepts in
the domain?
The case of
LA and EDM

3. 3
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
The evolution of TEL research is rapid and motivations are diverse
The language is often ill-defined
Differences in terminology: variations among communities or conceptual differences
Difficult to ensure that the wheel is not being reinvented
The case of LA and EDM
- LA: introduced in 2009 -- first conference 2011 -- no identified endogenous precursors but
strong heterogeneous (analytics)
- EDM: introduced in 2000 -- first conference 2008 – precursor workshops jointly held with
ITS, AIED, ICALT, etc. – evidence of historical roots in learner modeling
Different histories, but does that imply semantic differences?
http://www.tel-thesaurus.net/

4. 4
(partial view of the links).
Colors represent
the clusters centered on
the most important
keywords
Data (orange)
link
Analytics (blue)
and
Educational
Data (red)
Source: Stellar Grand
Challenge problems
http://www.tel-thesaurus.net/maps/contexteGCP/
Keywords relating to
learning analytics

5. 5
(partial view of the links).
Colors represent
the clusters centered on
the most important
keywords
Data (orange)
link
Analytics (blue)
and
Educational
Data (red)
Source: Stellar Grand
Challenge problems
http://www.tel-thesaurus.net/maps/contexteGCP/
Keywords relating to
learning analytics
Closer look at the DataTEL and the Productive Multivocality
workshops at the Alpine Rendez Vous 2011
- although data is at the heart of both, there is almost no shared
vocabulary, apart from cognates of “learning”
- in addition, there is a great difference in terms of scope in the
two workshop’s objectives

6. 6
DataTEL
“The research on TEL recommender systems can contribute to
decreasing the drop-out rate”
“customize existing recommendation algorithms for learning,
employ recommender systems in real-life scenarios and develop
suitable evaluation criteria for different kinds of recommender
systems”.
Productive Multivocality
“supportive structure for a dialogical interpretation of the data in
order to make the community and stakeholders aware what results
converge among the different data sets and different interpretations
and in order to identify open questions”.
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
Data is at the core of both communities, but in different ways
- One focuses on improving algorithms to treat data (recommender systems)
- The other focuses on interpretation of shared data
Sharing data is a potentially productive move for TEL research, but not an easy one
What « data » means might be the next question
a challenge illustrated in the second part of this communication

7. 7
Initial conditions for Productive Multivocality
using the pivotal moment as
a boundary object
X 5 Editors: Suthers, D., Lund, K., Rosé, C., Law, N. & Teplovs, C.

8. 8
Multidisciplinarity, interdisciplinarity and multivocality
• Neither theoretical perspectives nor actual results from different
participating disciplines are integrated during multidisciplinarity
 subject approached from different angles, using different disciplinary perspectives (van
den Besselaar & Heimerik, 2001)
 Each research group stays within their own boundaries (Choi & Pak, 2001)
• Interdisciplinary research integrates contributing disciplines by creating
its own theoretical, conceptual and methodological identity
 analyzes, synthesizes and harmonizes links between disciplines into a coordinated and
coherent whole (van den Besselaar & Heimerik, 2001)
• Multivocal research
 performing multiple analyses from different epistemological and methodological
frameworks on a shared corpus (e.g. group interactions in pedagogical contexts)
 Productive : analytical concepts were refined, epistemological positions were made
explicit, and the conditions under which learning occurs were characterized, but with
different perspectives, thus allowing discussion about learning

9. 9
Our argument
• The LA community is much like the CSCL community
 Multidisciplinary with a potential for interdisciplinarity
• (Our version of ) multivocality is closer to interdisciplinarity than to
multidisciplinarity
 We will use an example from the Productive Multivocality Initiative to illustrate this
 Multivocality and interdisciplinarity are approaches that move research fields
forward
– The communities researching “Learning Analytics” are nicely positioned to
benefit from such approaches, much in the same way that CSCL has been

10. 10
How multivocality can tend towards interdisciplinarity
• Step 1
 3 researchers each designate the moment they call pivotal
– Different visions of learning are made explicit
– “Moments” are of differing length (cf. unit of analysis/interaction)
1 Trausan-Matu
2 Shirouzu
3 Chiu
1 Trausan-Matu
1 Trausan-Matu
“Fold, then cut out the 3/4 of 2/3 of the
origami paper”

11. 11
Multivocality without convergence
• The comparison of two researcher’s pivotal moments lead to
progress in each other’s problématiques, but not to integrating on
either a theoretical or methodological level
 Taking another researcher’s pivotal moments and interpreting them in one’s own
framework (e.g. Chiu : breakpoints in frequency of new ideas corresponded to
when and how the pedagogical designer’s intentions were actualized by students’
behavior - Shirouzu)
 Neither methodological nor theoretical convergence is achieved, but a discussion
has begun

12. 12
Multivocality with convergence
• The comparison of two other researcher’s pivotal moments lead to
progress in one researcher’s problématique, but also to integrating
one of the researchers’ approaches on a methodological level
 Trausan-Matu extended the definition of an analytical concept (e.g. Bakhtinian
“voices” include gestures)
 extended the domain of application (e.g. from just chat to face-to-face interactions)
 Deeper theoretical integration is more difficult
– We do not always aspire to that because tension can be productive

13. 13
“Data” as a boundary object for learning analytics and
educational data mining
What is shared by a teacher having to manage a lesson on ratio and proportion, and
by the dean of the university having to ensure the success of the freshman class?
How far is the meaning of “learning” the same in both cases?
We join Siemens and Baker in a call for cooperation with the suggestion of
an analysis of…
- the nature of data
- the problématiques driving underlying commonalities and differences
perhaps for the sake of a new theoretical, conceptual and methodological identity for both
Learning Analytics and Educational Data Mining

14. 14
LA
“Learning analytics is the measurement, collection,
analysis and reporting of data about learners and
their contexts, for purposes of understanding and
optimizing learning and the environments in which it
occurs” (Long and Siemens, EDUCAUSE 2011).
EDM
“Educational Data Mining is a term used for
processes designed for the analysis of data from
educational settings to better understand students
and the settings which they learn in.” (Desmarais
and Baker, TEL Dictionary 2012)
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
- Are the Learning Analytics tools imported from analytics sufficient for relevantly
analyzing learning data?
- Should all data attached to the activities of a student be considered as learning data?
- Isn’t Learning Analytics reducing successful learning to the academic success of
students in their institutions, limiting de facto the problématique of TEL research?
- Compared to the classical problématique of “learner modeling”, what are the specific
contributions of “Learning Analytics”?

15. 15
Afterword
From…
- 42 papers from LAK12
- 24 papers from LAK11 76 docs / articles
- 10 papers from JETS12
article's
subject
article's
body
"learning
analytics" 10 564
"educational
data mining" 3 40
 LA "learninganalytic“
learning (1000.0) learn (909.0) analytic
(498.0) learner (372.0) activity (276.0)
research (257.0) context (230.0) social (222.0)
design (218.0) provide (214.0) community
(205.0) knowledge (203.0) practice (196.0)
tool (192.0) individual (180.0) datum (175.0)
model (169.0) development (160.0)
environment (159.0)
 EDM "educationaldatummining"
cluster (140.0) use (125.0) datum (94.0) teacher
(91.0) student (85.0) user (81.0) clustering
(76.0) project (72.0) mining(66.0) system (64.0)
theme (62.0) pattern (57.0) lecture (57.0) study
(55.0) video (51.0) online (49.0) feature (43.0)
model (43.0) class (41.0) group (40.0) question
(39.0) final (39.0) classroom (38.0) tool
(37.0) level (35.0) teaching (33.0)
instructional (32.0)
1. Latent Dirichlet Allocation on the preprocessed corpus (stopwords elimination,
discarding parentheses and phrases of less than 2 words & lemmatizing) -- to
enforce the search for the specific concepts in the topic model – to consider them
compound words.
after this step emerges: "learninganalytic" (without s) and
"educationaldatummining" (with datum instead of data)
2. Trained incremental LDA topic models starting from only the LAK corpus (1.3MB)
with 20 topics (a topic contains all words, with their corresponding weights in
descending order with concepts semantically related that emerge from co-occurence
relations
3. adding chunks of TASA corpus  results to come
Thanks! To Mihail Dascalu
for bringing his expertise within
very short delay! Work in progress!