Keynote at the 10th JTEL Summer School 2014 in Malta Ralf Klamma ACIS Group @ RWTH Aachen University April 28, 2014

1. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
1
Learning
Layers
This slide deck is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Community Learning Analytics –
A New Research Field in TEL
Ralf Klamma
Advanced Community Information Systems (ACIS)
RWTH Aachen University, Germany
klamma@dbis.rwth-aachen.de
JTEL Summer School, Malta, April 28, 2014

2. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
2
Learning
Layers
Abstract
Learning Analytics has become a major research area recently. In
particular learning institutions seek ways to collect, manage, analyze
and exploit data from learners and instructors for the facilitation of
formal learning processes. However, in the world of informal learning at
the workplace, knowledge gained from formal learning analytics is only
applicable on a commodity level. Since professional communities
need learning support beyond this level, we need a deep understanding
of interactions between learners and other entities in community-
regulated learning processes - a conceptual extension of self-
regulated learning processes. In this presentation, we discuss scaling
challenges for community learning analytics and give both
conceptual and technical solutions. We report experiences from
ongoing research in this area, in particular from the two EU integrating
project ROLE (Responsive Open Learning Environments) and
Learning Layers (Scaling up Technologies for Informal Learning in
SME Clusters).

3. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
3
Learning
Layers
Responsive
Open
Community
Information
Systems
Community
Visualization
and
Simulation
Community
Analytics
Community
Support
WebAnalytics
WebEngineering
Advanced Community Information
Systems (ACIS) Group @ RWTH Aachen
Requirements
Engineering

4. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
4
Learning
Layers
Agenda
LearningAnalytics
CommunityLearningAnalytics
ROLE&LearningLayers
ExpertsinCommunityInformation
Systems
Conclusions&Outlook

5. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
5
Learning
Layers
A PHD STUDENT VIEW ON
THE RESEARCH FIELD

6. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
6
Learning
Layers
Motivations for Doing
PhD Research in TEL
■ Some reasons (more?)
– My supervisor told me … (research interest of person paying me)
– My own research interest
– Good career perspectives (get famous, get rich, or both)
■ Formal Learning
– Close to my own practice and experience as a teacher, researcher
– Research settings easier to control (classroom as a lab)
■ Informal Learning
– Better funding opportunities (H2020, industry)
– More innovative (mobile, Web, micro, games)
– Real impact expected

7. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
7
Learning
Layers
LEARNING ANALYTICS

8. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
8
Learning
Layers
Self- and Community Regulated
Learning Processes
Based on [Fruhmann, Nussbaumer & Albert, 2010]
Learner profile
information is
defined or
revised
Learner finds
and selects
learning
resources
Learner works
on selected
learning
resources
Learner reflects
and reacts on
strategies,
achievements
and usefulness
plan
learnreflect
The Horizon Report – 2011 Edition

9. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
9
Learning
Layers
The Long Tail of Personal Knowledge
in Lifelong Learning
■ Zillions of new learning opportunities
■ Abundance of learning materials
■ But: Extremely challenging to find & navigate
High-quality, specially designed,
learning materials like books or
course material
Gaps in personal knowledge
identified mostly by real-world
practice

10. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
10
Learning
Layers
Personal Learning Environment (PLE)
PLE describes the tools, communities, and services that constitute the
individual educational platforms learners use to direct their own learning and
pursue educational goals
LMS – course-centric vs. PLE – learner-centric:
• Extension of individual research
• Students in charge of their learning process
• self-direction, responsibility
• Promotes authentic learning (incorporating expert feedback)
• Student’s scholarly work + own critical reflection + the work and voice of
others
• Web 2.0 influence on educational process
• customizable portals/dashboards, iGoogle, My Yahoo!
• Learning is a collaborative exercise in collection, orchestration, remixing,
& integration of data into knowledge building
• Emphasis on metacognition in learning

11. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
11
Learning
Layers
ROLE Approach to the Design
of Learning Experiences
guidance &
freedom of
learner
motivation of
learner (intrinsic,
extrinsic)
stimulation of
learner’s meta-
cognition
collaboration &
good practice
sharing among
peers
personalization
& adaptability to
learner & context What is the impact of these
findings from behavioral &
cognitive psychology on
design of learning?
Goal setting
Planning
Reflection
Control & Responsibility
Recommendation

12. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
12
Learning
Layers
ROLE Approach to the Design
of Learning Experiences
What is the impact of these findings from behavioral & cognitive psychology on
design of Personal Learning Environments?
learner profile information
is defined and revised
learner finds and selects
learning resources
learner works on selected
learning resources
plan
learnreflect
learner input regarding
goals, preferences, …
creating PLE
recommendations
from peers or tutors
assessment and
self-assessment
evaluation and
self-evaluation
feedback
(from different sources)
learner should understand and
control own learning process
ROLE infrastructure should
provide adaptive guidance
attaining skills using different
learning events (8LEM)
learner reflects and reacts
on strategies, achievements,
and usefulness
monitoring
recommen-dations
be aware of

13. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
13
Learning
Layers
Learning Analytics vs. Community
Learning Analytics
Formal Learning Learning Analytics Community
Regulated
Learning
Community
Learning Analytics
Environment LMS EDM/VA CIS/ROLE DM/VA/SNA/Role
Mining
Tools Fixed LMS Specific Eco-System Tool Recommender
Activities Fixed Content
Recommender
Dynamic Content
Recommender /
Expert
Recommender
Goals Fixed Progress Dynamic Progess / Goal
Mining / Refinement
Communities Fixed Not applicable Dynamic (Overlapping)
Community
Detection
Use Cases Courses Learning Paths Peer Production /
Scaffolding
Semantic Networks
of Learners /
Annotations

14. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
14
Learning
Layers
COMMUNITY LEARNING
ANALYTICS – A GENERAL
APPROACH

15. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
15
Learning
Layers
Communities of Practice
■ Communities of practice (CoP) are groups of people
who share a concern or a passion for something they
do and who interact regularly to learn how to do it
better (Wenger, 1998)
■ Characterization of experts in CoP
– Shared competence in the domain
– Shared practice over time by interactions
– Expertise based on gaining and having reputation within the CoP
– Being an expert vs. being a layman, a newcomer, an amateur etc.
– Informal leadership
– Identity as an expert depends on the lifecycle of the communities
Expertise in highly dynamic, locally distributed multi-disciplinary
and heterogeneous communities?

16. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
16
Learning
Layers
Proposed Development of the
Community Learning Analytics Field
■ Will happen J Big Data by Digital Eco Systems (Quantitative Analysis)
– A plethora of targets (Small Birds)
– Professional Communities are distributed in a long tail
– Professional Communities use a digital eco system
– An arsenal of weapons (Big Guns)
– A growing number of community learning analytics methods
– Combined methods from machine intelligence and knowledge representation
■ May not happen L Deep Involvment with community
(Qualitative Analysis)
– Domain knowledge for sense making
– Passion for community and sense of belonging
– Community learns as a whole
→ Community Learning Analytics for the Community by the Community

17. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
17
Learning
Layers
Web 2.0 Competence Development
Cultural and Technological
Shift by Social Software
Impact on
Knowledge Work
Impact on
Professional
Communities
Web 1.0 Web 2.0 Microcontent
Providing
commentary
Personal knowledge
publishing
Establishing personal
networks
Testing Ideas
Social learning
Identifying competences
Emergent Collaboration
Trust & Social capital
personal
website and
content
management
blogging and
wikis
User generated
content
Participation
directories
(taxonomy)
and
stickiness
Tagging
("folksonomy")
and syndication
Ranking
Sense-making
Remixing
Aggregation
Embedding
Emergent Metadata
Collective intelligence
Wisdom of the Crowd
Collaborative Filtering
Visualizing Knowledge
Networks

18. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
18
Learning
Layers
Interdisciplinary Multidimensional
Model of Communities
■ Collection of CoP Digital Traces in a MediaBase
– Post-Mortem Crawlers
– Real-time, mobile, protocol-based (MobSOS)
– (Automatic) metadata generation by Social Network Analysis
■ Social Requirements Engineering with i* Framework
for defining goals and dependencies in CoP
Social Software
Cross-Media Social Network
Analysis on Wiki, Blog, Podcast,
IM, Chat, Email, Newsgroup, Chat
…
Web 2.0 Business
Processes (i*)
(Structural, Cross-media)
Members
(Social Network Analysis: Centrality,
Efficiency, Community Detection)
Network of Artifacts
Content Analysis on Microcontent, Blog entry, Message,
Burst, Thread, Comment, Conversation, Feedback (Rating)
Network of Members
Communities of practice
Media Networks

19. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
19
Learning
Layers
Community Learning Analytics
in CoP
■ User-to-Service Communication
• CoP-aware Usage Statistics
• Identification of successful CoP services
• Identification of CoP service usage patterns
■ User-to-User Communication
• CoP-aware Social Network Analysis
• Identification of influential CoP members
• Identification of CoP member interaction/learning patterns
+

20. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
20
Learning
Layers
Supporting Community Practice
with the MobSOS Success Model

21. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
21
Learning
Layers
Community SRE Processes–
i* Strategic Rationale

22. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
22
Learning
Layers
RESPONSIVE OPEN
LEARNING ENVIRONMENTS

23. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
23
Learning
Layers
Responsive Open Learning
Enviroments (ROLE) 2009-2012
• Empower the learner to build their
own responsive learning environment
ROLE Vision
• Awareness and reflection of own
learning process
Responsiveness
• Individually adapted composition of
personal learning environment
User-Centered

24. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
24
Learning
Layers
ROLE
Technical Infrastructure
■ Sucessfully deployed in industry and education
■ Open Source Software Development Kit
■ ROLE Widget Store (role-widgetstore.eu)
■ ROLE Sandbox (role-sandbox.eu)

25. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
25
Learning
Layers
ROLE Sandbox – Geospatial &
Temporal Access
§ Users: 5787 (95% external)
§ Widgets: 1475 (71.5% external)
§ Spaces: 1283 (64.3% external)
§ Shared Resources: 18922 (6% external)

26. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
26
Learning
Layers
ROLE Requirements Bazaar –
Community-aware Requirements Prioritization
Factors influencing
requirements ranking
User-controlled weighting
of ranking factors
Community-dependent
requirements ranking lists
http://requirements-bazaar.org

27. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
27
Learning
Layers
Learning Analytics Visualization –
Dashboards
1. Database Selection
2. Filter Selection/
Definition
3. Adapted Visualization

28. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
28
Learning
Layers
LEARNING LAYERS –
SCALING UP TECHNOLOGIES
FOR INFORMAL LEARNING IN
SME CLUSTERS

29. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
29
Learning
Layers
Maturing
Interacting with People at the
workplace
Paul discovers a problem at the
construction site with PLC equipment ...
Generating dynamic Learning
Material
The regional training center observes the
Q&A and links it to their course
material ...
Q: How to use PLC equipment …?
• I have seen this before here …
• Last time I did it, I …
• Here is something helpful
Social Semantic Layer
Emerging shared meaning,
giving context
Energy
Consump.on
Lightning
X3-­‐PVQ
X3-­‐PJC
X3-­‐POZ
PLC
Equipment
Instructional Taxonomy
• What is …
• How to …
• Example of …
Tutorial: How to Use PLC
What is PLC
How to use it?
Examples
Further Information
Hot Questions and
Answers
Work Practice Taxonomy
• Installation
• Testing
• Operation
Peter
Paul
Mary
Interacting in the Physical
Workplace
Physical workplace is equipped with QR
tags, learning materials are delivered just
in time ...
A list of helpful resources
• Tutorials: How to use …
• Persons: Peter, Mary, …
• Work Practice: Installation,..
• Concepts: PLC, Lightning
• Q&A: …,
Learning Layers in the
Construction Industry

30. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
30
Learning
Layers
Learning Layers – Scaling Technologies for Informal Learning
Learning Layers – Scaling up Technologies for
Informal Learning in SME Clusters

31. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
31
Learning
Layers
Space (shared by multiple users)
Using the ROLE Framework for
Semantic Video Annotation
Web application (composed of widgets)
Widget (collaborative web
component)
http://role-sandbox.eu/

32. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
32
Learning
Layers
SeViAnno Prototypes
SeViAnno (Web)
SeViAnno 2.0 (Widgets)
AnViAnno (Android)
AchSo! (Android)

33. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
33
Learning
Layers
COMMUNITY LEARNING
ANALYTICS –
EXPERT IDENTIFICATION

34. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
34
Learning
Layers
Experts in
Learning Communities
■ In learning communities
many experts from
different fields meet
– Intergenerational learning
– Interdisciplinary learning
■ New Openness for Amateur
Contributions
■ Methods, Tools & CoP
co-develop
– Expert role models needed
– Expert identification based
on complex media traces

35. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
35
Learning
Layers
YouTell - A Web 2.0 Service for
Collaborative Storytelling
§ Collaborative storytelling
§ Web 2.0 Service
§ Story search and
“pro-sumption”
§ Tagging
§ Ranking/Feedback
§ Expert finding
§ Recommending
Klamma, Cao, Jarke: Storytelling on the Web 2.0 as a New Means of Creating Arts
Handbook of Multimedia for Digital Entertainment and Arts, Springer, 2009

36. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
36
Learning
Layers
Expert Finding – Computation of
Actual Knowledge
■ Data vector consists of
– Personal data vector
– Competences, skills,
qualification profile
– Self-entered data
– Story data vector
– Visits of stories
– Involvement in projects
– Expert data vector
– Advice given
– Advice received
– Value = #Keywords – Date
Decay – Feedback
Motivation
PESE:
Web 2.0 –Anwen-
dung für community-
basiertes Storytelling
Der PESE-
Prototyp
Evaluierung des
Prototypen
Zusammen-
fassung
Ausblick
Find the most appropriate expert
Data vector represents knowledge of the expert

37. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
37
Learning
Layers
Knowledge-Dependent
Learning Behaviour in Communities
Renzel, Cao, Lottko, Klamma: Collaborative Video Annotation for Multimedia Sharing between Experts and Amateurs,
WISMA 2010, Barcelona, Spain, May 19-20, 2010
§ Expert finding algorithm: Knowledge value of community sorted by keywords
§ Community behavior: Experts spent more time on the services
§ Experts prefers semantic tags while amateurs uses “simple” tags frequently
§ Community tags: Experts use more precise tags

38. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
38
Learning
Layers
Threads to Expert Finding
■ Compromising techniques
— Sybil attack [Douc 2002], Reputation theft, Whitewashing attack, etc..
— Compromising the input and the output of the expert identification algorithm
■ Example: Sybil attacks
— Fundamental problem in open collaborative Web systems
— A malicious user creates many fake accounts (Sybils) which all reference the user to
boost his reputation (attacker’s goal is to be higher up in the rankings)
Sybil
region
Honest
region
ABack
edges

39. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
39
Learning
Layers
Conclusions & Outlook
■ Community Learning Analytics
– Informal learning more challenging for learning analytics
– New research challenges and funding opportunities
– Highly interdisciplinary and multi-method research
■ Case Studies
– Responsive Open Learning Environments
– ROLE SDK for Near Real-Time Widget-Based Web Applications
– Learning Layers - Scaling up Technologies for
Informal Learning in SME Clusters
– Informal Learning on the Workplace
– Collaborative Semantic Video Annotation