Sven Charleer developed learning dashboards in 7 different designs across 3 learning settings - classrooms, study advice sessions, and general student use - involving over 100 students, 20 instructors, and 17 study advisers. The dashboards aimed to provide visualizations of learning analytics data to support students, instructors, and advisers. Evaluations of the dashboards led to 19 publications and ongoing interest from universities to deploy the dashboards more widely.

1. Sven Charleer
Student-facing Learning Dashboards
Preliminary PhD Defence
May 23, 2017

2. LEARNING ANALYTICS
2
“The measurement, collection, analysis, and reporting of data about learners
and their contexts, for purpose of understanding and optimising learning and
the environments in which it occurs”
J. L. Santos. Learning Analytics and Learning Dashboards: a Human- Computer Interaction Perspective. PhD dissertation, KU Leuven, 2015.
G. Siemens. “Learning analytics: envisioning a research discipline and a domain of practice”. Proceedings of the 2nd
International Conference on Learning Analytics and Knowledge . ACM. 2012, pp. 4–8.
session
course
degree
year
Microlevel
intro

3. 3
EMPOWER
predictions
recommendations
LEARNING ANALYTICSintro

4. LEARNING DASHBOARDS
4
“A Learning Dashboard is a single display that aggregates different indicators
about learner(s), learning process(es) and/or learning context(s) into one or
multiple visualisations.”
B. A. Schwendimann, M. J. Rodríguez-Triana, A. Vozniuk, L. P. Prieto, M. S. Boroujeni, A. Holzer, D. Gillet, and P. Dillenbourg. Understanding learning at a glance: An overview of learning
dashboard studies. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pages 532–533. ACM, 2016.
K. Verbert, E. Duval, J. Klerkx, S. Govaerts, and J. L. Santos. Learning Analytics Dashboard Applications. American Behavioral Scientist, 57(10):1500–1509, 2013.
Perceived benefits
Design guidelines
intro

5. External input
Rapid prototyping
Multiple dependent variables
Inform
5
DESIGN-BASED RESEARCHintro

6. 6
CONTEXTintro
ch2
ch3
ch4

7. RQ1: What are relevant learning traces, and how should we
visualise these data to support students to explore the path from
effort to outcomes?
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
7
CREATING EFFECTIVE LEARNING DASHBOARDSch2
abundance of data - effort - outcome

8. 8
CONTRIBUTIONSch2

9. Abstract the LA data
Provide access to the artefacts
Augment the abstracted data
Provide access to teacher and peer feedback
9
RESULTSch2
RQ1: What are relevant learning traces, and how should we visualise these
data to support students to explore the path from effort to outcomes?

10. 10
RESULTSch2
RQ2: How can we promote students, inside and outside the classroom, to
actively explore this effort to outcomes path?
Visualise the learner path
Integrate LA into the workflow
Facilitate collaborative exploration of the LA data

11. 11
RESULTSch2
Integrate LA into the workflow
Augment the abstracted data

12. 12
RESULTSch2

13. 13
CONTRIBUTIONSch2
Guidelines published at EC-TEL
(25% acceptance)
S. Charleer, J. Klerkx, E. Duval, T. De Laet, and K. Verbert. Creating effective learning analytics dashboards: Lessons learnt.
Adaptive and Adaptable Learning: 11th European Conference on Technology Enhanced Learning, EC-TEL 2016, Lyon,
France, September 13-16, 2016, Proceedings, pages 42–56, Cham, 2016. Springer International Publishing
LARAe
test beds across Europe, 461 students
14 papers
Stanford University - University of Technology Sidney - University of the Basque
Country - Murdoch University Perth - Curtin University Perth
Visual Learning Analytics workshop
Learning Analytics Summer Institute 2014, Harvard

14. 14
BALANCED DISCUSSION IN THE CLASSROOMch3
RQ3: What are the design challenges for ambient Learning
Dashboards to promote balanced group participation in
classrooms, and how can they be met?
RQ4: Are ambient Learning Dashboards effective means for
creating balanced group participation in classroom settings?
over- and under-participation

15. oup 1 Group 2
oup 5 Group 3
EVALUATION SETUPch3
case study 1
# participants 12 students
deployment
1 3h session with dashboard
1 3h session without dashboard
evaluation
class discussion
questionnaires (perceived distraction/
awareness/usefulness)
activity/quality logging
case study 2
# participants 19 students
deployment
half 3h session without dashboard
half 3h session with dashboard
evaluation
questionnaires (perceived importance
feedback/motivation)
activity/quality logging
15

16. Visualise balance in an abstract and neutral way
Add the qualitative dimension to the visualisation
Create a realistic picture of the classroom situation
16
RESULTS
RQ3: What are the design challenges for ambient LDs to promote balanced
group participation in classrooms, and how can they be met?
ch3

17. Ambient dashboards as support for teacher/presenter
Ambient dashboards raise awareness of the invisible
Ambient feedback information can activate students
17
RESULTS
RQ4: Are ambient LDs effective means for creating balanced group
participation in classroom settings?
ch3

18. Ambient dashboards as support for teacher/presenter
Ambient dashboards raise awareness of the invisible
Ambient feedback information can activate students
18
RESULTS
RQ4: Are ambient LDs effective means for creating balanced group
participation in classroom settings?
ch3

19. EVALUATION SETUPch3
-0.2
0
0.2
0.4
0.6
0.8
%distancefromaverage
-0.2
0
0.2
0.4
0.6
0.8
0 10.5
% of time of feedback session passed
%ofdistancefromaverage
without visualisation
with visualisation
group 1
group 2
group 3
group 4
group 5
group 6
19

20. 20
CONTRIBUTIONS
Published Special Issue on Awareness and Reflection in
Technology-Enhanced Learning, IJTEL
(8/21 submissions accepted)
Charleer, S., Klerkx, J., Duval, E., De Laet, T. and Verbert, K. (2017) ‘Towards balanced
discussions in the classroom using ambient information visualisations’, Int. J. Technology Enhanced
Learning, Vol. 9, Nos. 2/3, pp.227–253.
Basis for new research collaboration
with the University of Sidney
ch3

21. SUPPORTING ADVISER-STUDENT DIALOGUEch4
RQ5: What are the design challenges for creating a Learning
Dashboard to support study advice sessions, and how can they be
met?
RQ6: How does such a Learning Dashboard contribute to the role
of the adviser, student, and dialogue?
21
lack of data-based feedback

22. EVALUATION SETUPch4
design
# participants
17 study advisers (preliminary feedback)
5 study advisers (iterative feedback)
approach
brainstorms/observations
iterative design
evaluation
# participants 5 study advisors
deployment
Engineering Science, Engineering Science:
Architecture
97 sessions (15-30min per session)
evaluation
15 sessions observed
questionnaires perceived usefulness
22

23. Data Confidence
Collaboration
Adviser’s role
23
RESULTS
RQ6: How does such a Learning Dashboard contribute to the role of the
adviser, student, and dialogue?
ch4
RQ5: What are the design challenges for creating a Learning Dashboard to
support study advice sessions, and how can they be met?
Authorship
Visual Encoding
Ethics

24. RESULTSch4
24

25. RESULTSch4
25

26. RESULTSch4
S. Claes, N. Wouters, K. Slegers, and A. V. Moere. Controlling In-the-Wild Evaluation Studies of Public Displays. pages 81–84, 2015.
26

27. 27
CONTRIBUTIONSch4
Conditionally accepted to IEEE Transactions on Learning
Technologies with minor revisions
11% acceptance rate, IF most recent: 1.129, 5-year IF: 1.608
S. Charleer, A. Vande Moere, J. Klerkx, K. Verbert, and T. De Laet. Learning analytics dashboards
to support adviser-student dialogue. IEEE Transaction on Learning Technologies, conditionally
accepted with minor revisions, 18 pages
Deployed at
Engineering Science, Engineering Science: Architecture, Maths,
Biology, Physics, Geology, Geography, Biochemistry, Informatics,
Bio-engineering, Engineering Technology (3 campuses)
15 study advisers during 165 sessions

28. 28
SUMMARYwrp
3 learning settings
7 dashboards
100+ students, 20 instructors, 17 study advisers
19 publications

29. 29
Citation indices Since 2012
# citations 112
h-index 6
i10-index 3
CITATIONSwrp
29

30. 30
FUTURE (ONGOING) WORKwrp
Ground work for long-term evaluations/deployments
Leiden University
Student union requests faculty deployment
KU Leuven has shown interest in dashboard university-wide

31. Student-facing Learning Dashboards
Sven Charleer
Preliminary PhD Defence
May 23, 2017