The document describes a Game Learning Analytics model called GLAID for analyzing learning in users with cognitive disabilities. GLAID collects interaction data during game sessions and analyzes it at three levels - individualized, collective, and predictive. It relates the data to the game design and educational goals. The model was applied to the serious game Downtown designed to teach subway navigation to people with Down syndrome. Observables like help button clicks were tracked over sessions to provide individualized and collective analysis of learning progress.

1. GLAID: Designing a Game Learning Analytics Model to
Analyze the Learning Process in Users with Cognitive
Disabilities
Baltasar Fernández-Manjón
Ana R. Cano, Álvaro J. García-Tejedor
Grupo e-UCM: www.e-ucm.es
balta@fdi.ucm.es @BaltaFM
SGames Conference, Porto, 16/06/2016
http://www.slideshare.net/BaltasarFernandezManjon/

2. LA & GLA 101
• Learning Analytics: Improving education based on Data Analysis
̶ Data driven
̶ Evidence-Based Education
• Game Learning Analytics application of LA to Serious Games
̶ Interaction data in a Serious Game is collected and analyzed for improving the
learning process supported by the game
̶ Educational game not as “black boxes”
̶ But LA & GLA is not “informagic”
̶ We need to relate data with what happens in the game and with the
educational design!

3. The GLA Problem
• Ok, we are collecting ALL the interaction data in a video game but…
IT IS A HUGE AMOUNT OF DATA!
Now what?
• What are the relevant observables?
• How do I analyze the data collected?
• How do I translate it into useful
information about the learning
process?

4. And the problem gets bigger…
…If the user has an intelectual condition or disability
(e.g. Down Syndrome)
User Features:
• Interaction with the game (motor skills)
• Ordering thoughts and language in a “logical” layout
• Listening and taking turns in conversations
• Communication in an interactive sense
• Relating objects and actions to spoken or written words

5. H2020 Beaconing project
• BEACONING stands for ‘Breaking Educational Barriers with Contextualised,
Pervasive and Gameful Learning’
• Started in january 2016, 15 partners, 9 countries, 6M
• Global goal is learning ‘anytime anywhere’
• Exploitation of technologies for contextual pervasive games and use of gamification
techniques
• Problem based approach to learning
• Enriching the Gaming Learning Analytics data model with
the contextual, geolocalized and accessibility information
• Large pilots in real settings with content providers
• Formal and informal learning across virtual and physical spaces
• GLA is a key element in the games and pilots evaluation
• Using RAGE infrastructure and extending it for these
new requirements and applications

6. Our approach: The GLAID Model
Present
Individualized
Learning Analysis
Collective
Learning Analysis
Predictive
Learning Analysis
….
Group 1
Group 2
Group 3
Game Sessions
LearningProgress
d1.a d1.n
d2.a
d3.a
d2.n
d3.n
*d = Data collected during a game session
GLAID (Game Learning Analytics for Intellectual Disabilities) Model
Analytics Framework
User 1
User 2
User n
User 1
User n
User 3 User 2
User 5
User 4
User 1
Data Handling
Designer Perspective Educator Perspective
User cognitive
restrictions
Formal
Requirements
Game & Learning
Design
Group of
Observables
Group of
Observables
Descriptive
Analytics
Clustering
Analytics
Predictive/Prescriptive
Analytics

7. First Step: From the User Restrictions to a Game Design
• Challenges:
1) Transform the user characteristics
into formal requirements
2) Develop a learning game design
adequate for users with intelectual
disabilities (such as Down Syndrome,
mild cognitive impairments, ASD
Autism Spectrum Disorders,…)
3) Select a group of
observables/variables that measure
the learning outcome of the user for
future assessment
….
*
User
User
User
User cognitive
restrictions
Formal
Requirements
Game & Learning
Design
Group of
Observables
Group of
Observables

8. 1st Level Analysis: Individualized Learning Analysis
• Goal: Describe and analyze historical
learning data from the student’s
perspective
• Outcome: Gives an overview of the user’s
learning behaviour through several game
sessions
• Observables collected individually
• Timestamps
• Level changes
• Achievements vs. Fails
• User interactions (number of clicks, heatmaps,
time between clicks,…)
Individualized
Learning Analysis
….
d1.a d1.n
d2.a
d3.a
d2.n
d3.n
*d = Data collected during a game session
User 1
User 2
User n

9. 2nd Level Analysis: Collective Learning Analysis
9
• Goal: Identify causes of trends and learning
outcomes for a group of users segmented
by disability or cognitive skills
• Outcome: Learning patterns
• Observables collected collectively
• Timestamps
• Level changes
• Achievements vs. Fails
• User interactions (number of clicks, heatmaps,
time between clicks,…)
Collective
Learning Analysis
Group 1
Group 2
Group 3User 1
User n
User 3 User 2
User 5
User 4

10. 3rd Level Analysis: Predictive Learning Analysis
• Goal: Analyze current and historical data to
make predictions about future learning
outcomes
• Outcome: assure the effectiveness of a game
as a learning tool for a user with an specific
disability
• Observables colected individually and
collectively
• Timestamps
• Level changes
• Achievements vs. Fails
• User interactions (number of clicks, heatmaps, time
between clicks,…)
Predictive
Learning Analysis
Game Sessions
LearningProgress
User 1

11. Data Handling: stakeholders
• 2 Data handling perspectives:
Game Designer’s Perspective
• Collect and analyze all the states that
the user can reach in a game session
• Are the mechanics of the game
appropiate for the user?
Educator’s Perspective
• Learning experience of each user
• Are the users learning or struggling
with the game?

12. Collecting data with xAPI
• We can collect the relevant data in a standard format using xAPI
• We are working in a xAPI serious games profile with ADL
• This will simplify the analysis and visualization of data (e.g. dashboards)
12
xAPI

13. Case study: Downtown
• Serious Game designed and develop
to teach young people with Down
Syndrome to move around the city
using the subway
• Status: Designed and developed.
Analysis pending
• Type of game: Serious Game
• Audience: People between 15 and 30 y/o with
Down syndrome
• Platform: PC and Android (work in progress)

14. Case Study: Downtown
• From user requirements to a game
design and its observables
• Standards: W3C cognitive
accessibility, accessibility guidelines
14

15. Case Study: From user requirements to a game design
User
Requirement
Game Restrictions Game Design & Mechanics Observable
Limited
intellectual
autonomy
The game should be able to
guide the user during the
learning session through
interactive help, pop-up tips or
other mechanics
There will be a "help" button
permanently in the screen where the
user can ask for help at anytime during
the game session
Clicks in the Help
buttons during a
game session
If the user doesn't perform any
interaction for more than 2 minutes, a
pop-up aid will appear providing guide,
tips and advices
Total inactivity time
Inactivity time after
pop-up help appears
The phone will act as a help
button. If the user needs tips or
advices, he can call the police
asking for clues to complete the
ongoing task

16. Case Study: From user requirements to a game design
User Requirement Game Restrictions Game Design Observable
Difficulty in the process
of abstractions,
conceptualization,
generalization and
learning transfer
The game should explain any
action to do, even the easiest,
without assuming that the
user already know how to
complete it
Tutorials: The description about how
to achieve the goals in the game will
be performed as a video explanation
before the task starts
Time consumed in
completing the task
Previous research prove that
visual explanations help to
understand the assignments
better than hearing or
reading.
Savidis, Grammenos and Stephanidis "Developing
inclusive e-learning and e-entertainment“. 2007

17. Case Study: Applying GLAID to the game
• Observable: Clicks in the “Help” button during a game session
Session #1 Session #n
User #1 3 clicks 1 click
Group of
users #1
5 clicks (avg) 4 clicks (avg)
GLAID
Individualized Learning Analysis Collective Learning Analysis
•Game designer’s persp: The user improved in
the use of the game through sessions
•Educator’s persp: The learning experience of
the user seems to improve through sessions
(measure with other observables)
•Game designer’s persp: Users with XX
disability slightly improved in the use of the
game through sessions. May reconsider
game design and mechanics for certain tasks
•Educator’s persp: The learning experience of
the user slightly improved through sessions.
May reconsider the learning experience
User#1 Assessment:
•The user is able to use the game as a
learning tool better than other users
•His intellectual autonomy seems to be
above the average for his type of
disability
•His learning experience seems to be
improving through game sessions

18. Just another BEACONING initiative …
18

19. Thanks!
Questions?
Mail: balta@fdi.ucm.es
Twitter: @BaltaFM
GScholar: https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en&oi=ao
ResearchGate: www.researchgate.net/profile/Baltasar_Fernandez-Manjon
Slideshare: http://www.slideshare.net/BaltasarFernandezManjon