«Diseño Centrado en el ser humano y Learning Analytics: ¿dónde estamos y hacia dónde deberíamos dirigirnos?», Roberto Martínez-Maldonado

1. Roberto Martinez-Maldonado
Roberto.MartinezMaldonado.net
human-centred design and
learning analytics
where are we at and where should we go
next?
twitter: @RobertoResearch

2. learning analytics in Australia
(non-comprehensive list of LA research clusters)

3. learning analytics in Australia
(non-comprehensive list of LA research clusters)
+ quite a few more researchers in various universities/organisations
Some thoughts:
HD and schools are very interested in LA
Learning Analytics SIG (ASCILITE)
ALASI – Australian Summer Institute (December)
There is at least one PHD in LA program
Institutional and industry funding
LA as a Field-of-Research under review by the ARC

4. …there has been a growing interest in developing
evidence-based practices in education
©Shutterstock/M-SUR
P. Davies. (1999). What is evidence‐based education?
British Journal of Educational Studies, 47(2), 108-121.

5. learning analytics offer great promise in providing
evidence for teachers and learners to make informed decisions and transform the
educational experience

6. is the measurement, collection, analysis and reporting
of data about learners and their contexts, for
purposes of understanding and
optimising learning and the environments in which it
occurs.
Learning Analytics (definition back in
2011)
1st International Conference on
Learning Analytics and Knowledge,
Banff, Alberta, February 27–March 1,
2011

7. “The potential of learning analytics is arguably far more
significant than as an enabler of data-intensive educational
research, exciting as this is. The new possibility is that
educators and learners — the stakeholders who constitute
the learning system studied for so long by researchers — are
for the first time able to see their own processes and
progress rendered in ways that until now were the preserve
of researchers outside the system.
Knight, S. and Buckingham Shum, S. (2017). Theory and Learning Analytics. The Handbook of Learning Analytics.
What makes Learning Analytics unique?

8. The Learning
Analytics loop
Clow, D. (2012). The learning analytics cycle: closing the loop effectively.
In Proc. LAK’12.
DEFINITIONS
Learning Analytics:
“…the process of developing actionable
insights…” (Cooper, 2012, p. 3).
Actionable insight:
“data that allows a corrective procedure,
or feedback loop, to be established for a
set of actions” (Jørnø & Gynther, 2018).

9. “The potential of learning analytics is arguably far more
significant than as an enabler of data-intensive educational
research, exciting as this is. The new possibility is that
educators and learners — the stakeholders who constitute
the learning system studied for so long by researchers — are
for the first time able to see their own processes and
progress rendered in ways that until now were the preserve
of researchers outside the system.
Knight, S. and Buckingham Shum, S. (2017). Theory and Learning Analytics. The Handbook of Learning Analytics.
What makes Learning Analytics unique?

10. “The potential of learning analytics is arguably far more
significant than as an enabler of data-intensive educational
research, exciting as this is. The new possibility is that
educators and learners — the stakeholders who constitute
the learning system studied for so long by researchers — are
for the first time able to see their own processes and
progress rendered in ways that until now were the preserve
of researchers outside the system.
Knight, S. and Buckingham Shum, S. (2017). Theory and Learning Analytics. The Handbook of Learning Analytics.
What makes Learning Analytics unique?

11. limitationsof most of the current LA systems are also coming under critical scrutiny
students face difficulties in interpreting and acting upon
data to improve learning (Jivet et al., 2018; Matcha et al., 2019), and
the same applies to teachers (Mangaroska & Giannakos, 2018)
poor design decisions in LA
(Gibson & Martinez-Maldonado, 2017)
– or not reportedlack of involvement of teachers and
learners in the design of LA tools
(Buckingham Shum, 2019, Holstein et al., 2017, 2018,
Wise & Jung, 2019)

12. evidence generated by LA systems is commonly incomplete and…..
… often irrelevant

13. This confirms Biesta’s concern about evidence-based
teaching practice
… the role that evidence can play
should be subordinated to the
values that constitute the
educational practice.

14. Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: a consolidated model of a field of
research and practice. Learning: Research and Practice, 3(1), 63-78.
The importance of Design

15. The importance of Design
Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: a consolidated model of a field of
research and practice. Learning: Research and Practice, 3(1).

16. Emerging interest on Human centred learning analytics

17. Emerging interest on Human centred learning analytics

18. Emerging interest on Human centred learning analytics

19. Martinez-Maldonado, R., Pardo, A., Mirriahi, N., Yacef, K., Kay, J. and Clayphan, A. (2016) LATUX: an Iterative Workflow for Designing,
Validating and Deploying Learning Analytics Visualisations. International Journal on Learning Analytics, JLA, 2(3)
Emerging interest on Human centred learning analytics

20. What is human-centredness?

21. Human centredness has been identified in
other fields as a characteristic of systems that have
been carefully designed by:
identifying the critical stakeholders,
their relationships, and
the contexts in which those systems will function
.
Buckingham Shum, S., Ferguson, R. and Martinez-Maldonado, R. (2019). Human-Centred Learning Analytics.
Journal of Learning Analytics, JLA, 6(2)

22. the human centred (not centric)
All the human factors,
social factors and
technology factors
interact together under the
human activity umbrella.

23. “Human-centred design is concerned less with assuring
that artifacts work as intended (by their producers, designers, or
other cultural authorities) than with enabling many individual or
cultural conceptions to unfold into uninterrupted interfaces
with technology.” Klaus Krippendorff
Krippendorff, K. 2004, Intrinsic motivation and human-centred design,
Theoretic Issues in Ergonomics Science, Vol. 5, No. 1, pp 43-72.

24. Human-centred design landscape
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.

25. Human-centred design landscape
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.

26. Human-centred design landscape
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.

27. Human-centred design landscape
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.

28. Simplified design process
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.

29. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.
User-centred design Co-creation (co-design)
User
Researcher
Designer

30. Final evaluation with
teachers and students
What we commonly do in Learning Analytics

31. Initial consultation
with teachers and students
What we sometimes do in Learning Analytics

32. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.
User-centred design Co-creation (co-design)
User
Researcher
Designer

33. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.
What a co-design approach for Learning Analytics
should ideally look like

34. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5-18.
What a co-design approach for Learning Analytics
should ideally look like

35. the present
the future
human-centred learning
analytics

36. Is there a working definition of
human-centred learning analytics?

37. “LA may be 10 years old but HCLA is a Toddler”

38. 5 critical current take-home messages in HCLA
1- Learning is complex
The more we embrace this complexity the
more we will realise the challenges of
building effective LA interfaces.
For example: Teachers have many more needs than
data needs

39. 5 critical current take-home messages in HCLA
2- Many current LA tools and dashboards
require a level of data literacy
From a HCD perspective, it is more sensible to change the tool
to suit teachers’ and learners’ needs than training them to suit
the LA tools.

40. 5 critical current take-home messages in HCLA
3- Non-data experts are unlikely to be
aware of the implication of LA design
choices.
This suggests the need for new methods particularly
tailored to engage with teachers and students in the
design of data-intensive and pedagogical meaningful
LA innovations.

41. Teachers and students should be considered
non-data experts
Maltese, Adam V, Joseph A Harsh, and Dubravka Svetina.
(2015). Data visualization literacy: Investigating data
interpretation along the novice—expert continuum.
Journal of College Science Teaching, 45(1).
HE STEM students find it hard to interpret
charts and data visualisations…

42. 5 critical current take-home messages in HCLA
4- Involving stakeholders may initially
be perceived as difficult, time-
consuming and expensive.
Yet, this can make a key difference between an
unsuccessful prototype and a system that is effective
and successfully adopted

43. Simplified OrLA framework
Prieto, L. P., Rodríguez-Triana, M. J., Martínez-Maldonado, R., Dimitriadis, Y., & Gašević, D. (2019). Orchestrating
learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the
classroom level. Australasian Journal of Educational Technology.

44. 5 critical current take-home messages in HCLA
5- Ethics is a burning topic in LA.
HCD has the potential to shift LA from something
imposed on teachers and learners to some done with
them.

45. “I would say it more straightforwardly:
automated adaptation is antinomical to an emancipatory perspective.
Learners are not to be seen as passive beneficiaries of a superior
control entity. With respect to software adaptations, if Learning
Analytics has to play a role, it should be limited to one of awareness
and recommendation.”
An emancipatory perspective for Learning Analytics?
Tchounikine, Pierre. (2019). Learners’ agency and CSCL technologies: towards an emancipatory perspective. International
Journal of Computer-Supported Collaborative Learning, 14(2).

46. Papers in this special section
were mostly focused on teachers

50. other works

51. This paper provide guidance on developing new learning analytics applications to more
holistically address values pertinent to stakeholders, educators, and the society., such as:
 Fairness
 Autonomy
 Social well-being
 Freedom from bias
 Self-image
 Ease of information seeking

52. Learner-data journey mapping
Prieto-Alvarez, C. G., Martinez-Maldonado, R., & Shum, S. B. (2018, December). Mapping learner-data journeys: evolution of
a visual co-design tool. In Proc. OzCHI’18.

53. LA-Deck: LA design cards
Alvarez, C. P., Martinez-Maldonado, R., & Shum, S. B. (2020, March). LA-DECK: A card-based learning analytics co-design
tool. In Proc. LAK’20

54. LA-Deck: LA design cards
Group 1 Group 2 Group 3

55. the present
the future
human-centred learning
analytics

56. HCLA is a burning topic!
https://youtu.be/VF1IkC6DJl8

57. Much more involvement of students!!
Two recommendations for considering Students as partners in LA:
1- Training and professional development for staff.
2- Think of our institutions as communities and the role of data for
such communities

58. HCLA in Multimodal Learning Analytics

59. Inductive step with teachers

60. Deductive step: filling the holes from theory

61. “What aspects of the classroom or the
learning activity happening in the classroom
more visible?”
The LATEP process
Prestigiacomo et al. (2020) Lerning-centred Translucence: an Approach to
Understand How Teachers’Talk About Classroom Data. Proc. LAK’20

62. The LATEP process
1- Generative phase: enable educational stakeholders to externalise their ideas
without constraints using tools for dreaming’
What information is currently available (visible)?
How the information should be made available?
What kind of metrics would be interesting to see?
E. B. Sanders. (2000). Generative tools for co-designing. In S. S.A.R., B. L.J. & W. A., Eds.,
Collaborative design. Springer, London, UK, 3-12.

63. 1- Generative phase.
2- Norming and prioritising phase. A converging phase is needed which
includes
norming (assessing and categorising); and
prioritising (ranking), which involves ranking the most critical ideas or
Osborn. (1953). Applied Imagination: Principles and Procedures of Creative Problem Solving. New York: Charles
Scribener’s Sons.
E. De Bono. (2017). Six thinking hats. London, UK: Penguin.
The LATEP process

64. 1- Generative phase.
2- Norming and prioritising phase.
3- Translucence elicitation phase.
Which information is needed to create well-informed understanding (awareness) of
What information would be useful to have before/during/after the learning activity?
Which information is needed for every role to be held accountable for actions? Who
Should the data be made available to different roles (e.g. novice teachers, students)?
Martinez-Maldonado, R. R., Elliott, D., Axisa, C., Power, P., Echeverria, E. and Buckingham Shum, S. (2020).
Designing translucent learning analytics with teachers: an elicitation process. Interactive Learning Environments
The LATEP process

65. human-centred design and
learning analytics
where are we at and where should we go
next?
overview

66. human-centred design and
learning analytics
where are we at and where should we go
next?
overview

67. human-centred design and
learning analytics
where are we at and where should we go
next?
overview

68. THANKS!
CONTACT:
Roberto.MartinezMaldonado@monash.edu
twitter: @RobertoResearch