Learning Analytics at Large: the Lifelong Learning Network of 160, 000 European Teachers

TeLLNet

EC-TEL
EC TEL 2011

Learning Analytics at Large:
the Lif l
th Lifelong Learning Network
L i N t k
of 160, 000 European Teachers

Ergang Song, Zinayida Petrushyna, Yiwei Cao, and Ralf Klamma
Information Systems and Databases, RWTH Aachen University

Palermo, Italy
September 23 2011
23,
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-SPCK-0911-1

TeLLNet

Motivations
 How to support lifelong learning (LLL)?
– New means for LLL with rapid development of ICT
(Meta-)
– Competence assessment methods for LLL in demand Competence
management
– Self-monitoring f LLL needed
S lf it i for d d
– Still lack of large data sets Self-
monitoring
– Tools are needed instead of a concept
 Case study: eTwinning Network Learning analytics
for lifelong learning

– Continuous professional development for teachers
p p
– Aiming to promote collaborations among schools
– Competence gap to recognize and to bridge
– Meta-competence
 Learning analytics is needed
– Vi l analytics for self-monitoring
Visual l ti f lf it i
Prof. Dr. M. Jarke
– Multiple levels (individual, community, and network)
I5-SPCK-0911-2

TeLLNet

Learning Analytics
Learning analytics is the measurement, collection, analysis and reporting of data about learners and their
ea ning s e easu e e , co ec o , a a ys s a d epo g o da a abou ea e s a d e
contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
(Siemens, 2011)

 Visual analytics
– It is easier for teachers to understand visualization than statistics
(Breuer et al., 2009)
 Data analysis
 Learning context analysis
(Cao et al., 2010)
 Network analysis
 The EC-TEL communities
as an e ample
example
Prof. Dr. M. Jarke (Pham et al., 2011)
I5-SPCK-0911-3

TeLLNet
Learning Analytics Contributions
to EC TEL so far
EC-TEL
 2006 - Klamma, Spaniol, Cao, Jarke: Pattern-Based Cross Media Social Network
Analysis for Technology Enhanced Learning in Europe
– Media Bases as research tools for TEL
– SNA as research methodology for TEL
 2008 - Petrushyna, Klamma: No Guru, No Method, No Teacher: Self-Observation and
Self-Modelling of E-Learning Communities
– In-depth Analysis of a Media Base for TEL
– Combination of SNA and content-based measures
 2009 - Breuer, Klamma, Cao, Vuorikari: Social Network Analysis of 45.000 Schools: A
Case Study of Technology Enhanced Learning in Europe
– eTwinning database of European cooperation between schools
– SNA as a tool for teachers
– Visualization and Usability
 2010 – Petrushyna: Self-modeling and Self-reflection of E-learning communities
(Doctoral Consortium)
Prof. Dr. M. Jarke
I5-SPCK-0911-4

TeLLNet
TeLLNet Project
Teachers' Lifelong Learning Networks

eTwinning
T i i TwinSpace
T i S TeLLNet
T LLN t

• Founded in 2005 • Since 2008? •3-year-project within the EU
• Coordinated by European • Subject to eTwinning Lifelong Learning
Schoolnet
S h l t • Web 2 0 for T i i
W b 2.0 f eTwinning Programme (2009-2012)
• Internet platform with • Blogs •Project obejctives:
workspace and Competence development
• Quality labels
(communication) tools for teachers in learning
• Desktop tools networks with social network
• P j t must be done b
Projects tb d by
two or more partners from analysis and scenario
different countries building based on eTwinning
• Offline activities: • Partners
Workshops across Europe • European Schoolnet
• RWTH Aachen University
• Open University of the
Netherlands
• Institute for Prospective
Technological Studies (IPTS)
–Joint Research Centre of
the European Commission

Prof. Dr. M. Jarke
I5-SPCK-0911-5

TeLLNet
Competence and Meta-Competence
 Developed in lots of areas: Author Definition
Human resource management
management, M Cl ll d
McClelland Th k l d kill i i d
The knowledge, skills, traits, attitudes,
vocational education ... (1973) self‐concepts, values, or motives directly
related to job performance or important
 Different definitions in literatures life outcomes and shown to differentiate
between superior and average
b i d
 Common points performers.
 A set of human characteristics Brown and A meta‐competence is the overarching
McCartney ability under which competence shelters.
(knowledge, skills, abilities...)
(knowledge skills abilities ) (1995)
( ) It embraces the higher order abilities
b h h h d bl
 The performances to enhance which have to do with being able to learn,
adapt, anticipate and create. Meta‐
 Categorized into different types competences are a prerequisite for the
 Assessment methods development of capacities such as
d l f h
judgment, intuition and acumen upon
 Explicit assessment (questionnaire, test) which competences are based and
without which competences cannot
 Implicit assessment flourish
fl i h
 Events to monitor Cheetham Meta‐competence is the competence that
and Chivers is beyond other competences, and which
 Algorithms to design (2005) enables individuals to monitor and/or
 Competence to computer
C t t t develop other competences
d l h
(Informationssysteme)  Automated executable without participation
Prof. Dr. M. Jarke
I5-SPCK-0911-6 of questionnaires

TeLLNet

Teachers’ Competence in eTwinning
 eTwinning Network
g  Our meta-competence
(as of the end of 2010) – Higher order competence
Teacher Amount %
Sum 135,351 100%
– Competence to monitor and
Project with projects 26,365 19.4%
develop other competences
with QLs 2,093 1.55% – Depends on context
with EQLs 616 0.46% – Ability to self-monitoring is
y g
with prizes 655 0.48%
meta-competence in the context
Wall post Wall posts sent 10,104 7.47%
of LLL Meta
Wall posts 18,986 14.03% competence
received
i d
Self- monitoring
Blog Posts written 4,508 3.33%
ability

etence
Post comments 441 0.33%
Language Wall-post writing
written

onal compe
competence
t ability
bilit

ompetence
e
Post comments 727 0.54%
received Blog writing
Project
Comment Project comments 1,531 1.13% performance ability

Professio

Social co
written
itt
Lehrstuhl Informatik 5 Prize comments 354 0.26% Project efficiency Comment writing
Prof. Dr. M. Jarke written etc. ability, etc.
I5-SPCK-0911-7

TeLLNet
Data Analysis: Large-Scale Data Set
of eTwinning
 New tables generated Table Name
Table Name Records
Records Error
Error
 New data for Web 2.0 number Rate

– Blogs (TwinBlogPost) Affectation 99886 0.00002
Institution 71988 0
– Comments (TwinBlogComment, MyContact 464780 0.000037
PrizeComment) Prize 892 0.0045
– Labels (QualityLabel) PrizeComment 441 0.0091
0 0091
– Tagging, etc. Project 17392 0
ProjectGuestBook 3460 0.009
– ProjectMember
ProjectMember 66145 0 0045
0.0045
– ProjectGuestBook QualityLabel 4886 0
 Data cleaning Teacher 133693 0
TeacherWall 34900 0.00014
0 00014
 Data dumps TwinBlog 15235 0.00013
– 1st Dump (June, 2010) TwinBlogComment 2950 0.2783

– 2nd Dump (November, 2010)
2 ( TwinBlogPost
T i Bl P t 31163 0.00064
0 00064
Sum 947811 0.0013
Prof. Dr. M. Jarke – 3nd Dump (May, 2011)
I5-SPCK-0911-8

TeLLNet

Competence Assessment
 Indicator model in Entity Relationship Diagram

 Performance Indicator

I  
fF
w f  Norm ( f )
Prof. Dr. M. Jarke
I5-SPCK-0911-9

TeLLNet
System Architecture of
Prototype CAfe

Prof. Dr. M. Jarke
I5-SPCK-0911-10

TeLLNet
Self-monitoring of Teacher Network
in CAfe
 Target users
– European teachers (teachers‘ workshops)
– Administrators & policy-makers

Prof. Dr. M. Jarke
I5-SPCK-0911-11

TeLLNet
Self-Monitoring of Competence
Management

Prof. Dr. M. Jarke
I5-SPCK-0911-12

TeLLNet
Self-Monitoring of Competence
Management
 Community level ->

 Teacher level

Prof. Dr. M. Jarke
I5-SPCK-0911-13

TeLLNet
Dynamic Network Analysis in Progress

 The Development Model (Pham et al. 2011 )

 Applied to collaboration (project, email) and social media(blog) networks
pp ed o co abo a o (p ojec , e a ) a d soc a ed a(b og) e o s
- To detect the development pattern of project partner community
- To compare different networks
Prof. Dr. M. Jarke
I5-SPCK-0911-14

TeLLNet
Learning Analytics: EC-TEL
Community among TEL Communities
 ICALT, ICWL, EC-TEL, IST, AIED (Pham,, Derntl and Klamma 2011)
, , , , ( )
4 (a) Densification law (b) Clustering Coefficient
10 1

0.95

Clustering coefficient
Number of edges
3
10
0.9
1.1976
ICALT: 0.34889*x

g
r
1.0544 ICALT
ICWL: 1.1149*x 0.85
10
2 ICWL
1.2415
ECTEL: 0.40338*x ECTEL
1.3817 0.8
ITS: 0.15818*x ITS
1.1197
AIED: 1.0128*x AIED
1
10 0.75
10
1
10
2
10
3
10
4 1 2 3 4 5 6 7 8 9
Number of nodes Age
(c) Maximum Betweenness (d) Largest connected component
0.08
0 08 0.7
07
ICALT ICALT

Largest connected component
ICWL 0.6 ICWL
mum betweenness

0.06 ECTEL ECTEL
0.5
ITS ITS
AIED 0.4 AIED
0.04
0.3
Maxim

c
0.2
02
0.02
0.1

0 0
1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
Age Age
(e) Diameter (f) Average Path Length
20 8
ICALT ICALT
ICWL ICWL

Average path length
15 6
ECTEL ECTEL
Diameter

ITS ITS
10 AIED 4 AIED

5 2

0 0
1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
Lehrstuhl Informatik 5 Age Age

Prof. Dr. M. Jarke
I5-SPCK-0911-15

TeLLNet
Node Level Analysis: Structural Holes,
Closure and Social Capital
 Structural holes (Burt, 1992)
- Nodes are positioned at the interface
between groups (gatekeepers, e.g. node B)
- Informational nodes: access to information
from different parts of networks
- Novel ideas by combining information from
different groups
- Control the communication between groups
 Closure:
Cl
- Nodes with high clustering coefficient (e.g. node A): embedded in tightly-knit
groups
- More trust and security within coherent communities
 Social capital (Coleman,, 1990)
p ( )
- Individuals and groups deriving benefits from social relationships
Prof. Dr. M. Jarke
I5-SPCK-0911-16 - Network structural property: can be either structural hole or closure

TeLLNet

Conclusions
 SNA & visualization as tools for competence development in
learning networks
– Competence assessment is still limited in performance indication
 eTwinning case study
– Complex data management issues
– Visual complexity of networks vs. teachers’ competence
– Experimenting with web based tools
web-based
 Learning analytics is the solution for large scale network

Data Visual Context Network Learning
analysis analytics analytics analysis analytics
Prof. Dr. M. Jarke
I5-SPCK-0911-17

TeLLNet
Learning Analytics for Conference
Participants
 At academic conferences/workshops
– Whi h t lk t attend?
Which talk to tt d?
– To whom to talk to?
 CAMRS – Mobile Context-aware Recom-
mendation Services for Conference Participants

? ?
? ? ?

Auditorium: keynote Room 342: workshop

Prof. Dr. M. Jarke
I5-SPCK-0911-18
Room 204: paper session Hall: poster session Room 048: round table

Learning Analytics at Large: the Lifelong Learning Network of 160, 000 European Teachers

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Ähnlich wie Learning Analytics at Large: the Lifelong Learning Network of 160, 000 European Teachers

Ähnlich wie Learning Analytics at Large: the Lifelong Learning Network of 160, 000 European Teachers (20)

Mehr von Ralf Klamma

Mehr von Ralf Klamma (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Learning Analytics at Large: the Lifelong Learning Network of 160, 000 European Teachers