Gaming Learning Analytics: using data for improving serious games applicability” This talk will present how to use a data-driven approach to improve how serios games are applied in different domains and how H2020 RAGE project is trying to simplify this analytics processes by creating an open software infrastructure.

1. Gaming Learning Analytics: using
data for improving serious games
applicability
Real Colegio Complutense at Harvard
Baltasar Fernandez-Manjon
balta@fdi.ucm.es , @BaltaFM
e-UCM Research Group , www.e-ucm.es
Cambridge, 07/04/2016
Cátedra Telefónica-Complutense
Educación Digital y Juegos Serios

2. Serious Games: definition and use
• Any use of digital games with purposes other than
entertainment (Michael & Chen, 2006)
• Serious Games have probed to be educationally
effective in several domains
– Medicine, military, business, corporate training
• But still is a low adoption of Serious Games in
mainstream education
• Serious Games considered usually as a complementary content
– Mainly used for motivational purposes
– No actual impact on the final mark
2

3. Data in Education: LA and GLA
• Learning Analytics Improving education based on data
analysis
– Data driven
– Evidence-based education
• Gaming Learning Analytics is a specific case when all
interaction data is used in serious games for improving
the learning process supported by the games
– Educational games not as “black boxes”

4. What happens when
new technology is used
in education?
Serious Games
….
4

5. Educational games in the
classroom: game
• The Radix Endeavor
– https://www.radixendeavor.org
• MMOG with Science,
Technology, Engeneering and
Math (STEM) topics
– Many different quests
• Adapted to middle and high
school curriculum
• Facilities for teachers
– Class configuration
– Quest assignation
– Tested in actual school
settings

6. Educational games in the
classroom: settings
• Master students in the e-
learning class
– 13 students, 8 nationalities
• No previous info about the
game
– If stuck ask for help or watch
the help videos
• One hour free play
– Who perform “better” at 30
minutes and at the end
• Publish screen captures in the
class Moodle course forum

7. Educational games in the
classroom: outcome
• Students were really engaged for the whole hour
– They enjoyed the experience
– Consider the game a commercial quality game
– No one asked for help
• All of them published the screenshots in the Moodle
class forum
• Educational outcome
– Only 4 of the students complete more than 2 quests
– Most of the students just spend the time exploring the
environment but not doing anything “useful” or
learning anything
• Actual data can be contradictory with simple observation

8. Other experiments ..
• Experiment: Gamified experience in the classroom
Goals
– First to complete the game
– Maximum “points” in the game
but
– Initially students were not informed that Learning
Analytics was in place
After some playing time LA data was
showed to the students
8

9. Using data interaction in the
classroom
GLA simplify SG deployment in the classroom
‣ Teacher knows what is happening in the class
‣ Teacher “keep the control”
Behavior change when students know that LA was in
place
• More oriented to success and less exploration or
“playing”
9

10. Serious games at school (USA)
Takeuchi, L. M., & Vaala, S. (2014). Level up learning: A national survey on teaching with digital games. New York: The Joan
Ganz Cooney Center at Sesame Workshop
• 62% teachers have used games to simulate and
supplement learning.
• (Harris Poll, 2014 –Pearson-, 1000 teachers K12, USA)

11. How teacher selects SG
11

12. Uses of Gaming Learning
Analytics in educational games
• Game testing – game analytics
– It is the game realiable?
– How many students finish the game?
– Average time to complete the game?
• Game deployment in the class – tools for teachers
– Real-time information for supporting the teacher
– “Stealth” student evaluation
– Knowing what is happening when the game is deployed in
the class
• Different uses for different stakeholders
– Teachers, students, administrators,
– Game developpers, researchers

13. Can we generalize and
simplify the adoption of
Gaming Learning
Analytics?
RAGE Project
….
13

14. Prosoci
alLearn
RAGE
No-one-
left-
behind 3d-
Tune-In
BEACONING
XX XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
EU H2020 serious games projects

15. H2020 RAGE in a nutshell
• RAGE will deliver advanced technology and know-how to
support the European Applied Games industry build-up and
job creation
– February 2015, 4 years, 20 partners, 9M €
• Creating a new serious games ecosystem by making
available a set of reusable technology components for
developing advance serious games easier, faster and more
cost-effectively
• Open source gaming learning analytics framework
– provide all the required services (e.g. game tracker,
learning analytics server, visualization of analytics
information)
– easy inclusion of gaming learning analytics techniques in
the new games
Realising an Applied Gaming Eco-System

16. RAGE: creating the Gaming Learning Analytics
infrastructure
H2020 RAGE project will simplify the process of
Serious Games creation with ready to use assets
• Game trackers (Unity3D, javascript)
• LA server infrastructure and services
• Basic analysis and visualization of LA data
• Extensible architecture
• Standards support (e.g. xAPI)
• Open code (github)
Using interaction data for assessment
Realising an Applied Gaming Eco-System

17. RAGE GLA Open Architecture
17
Clients AA Applications
Games
Analytics
Frontend
AA
Frontend
PlayersDevs,Students,
Teachers.
Admins
Users
Roles
Resources
Permissions
Applications
Authentication
Authorization
JWT
JSON
WEB
TOKEN
Games
Sessions
Results
TopologyAnalytics Backend
Games
Sessions
Collector
Results
Application 1
Application N
KafkaQueue
LRS
We reuse APEREO Open LRS

18. New Reference Models for GLA
18
Implications of
GLA in the design
of the game

19. RAGE, serious games and xAPI
• Define interactions model to assess serious games
• Create a xAPI vocabulary to represent the model
– Repurpose existing verbs and activities
– Create verbs and activities where needed
– Create extensions where needed
• Define analysis for the model
– Simple metrics processing statements (serious
game agnostic)
– More advanced analysis based on serious game
goals

20. H2020 Beaconing project
• BEACONING stands for ‘Breaking Educational Barriers with Contextualised,
Pervasive and Gameful Learning’
• Started in january 2016, 15 partners, 9 countries, 6M
• Global goal is learning ‘anytime anywhere’
• Exploitation of technologies for contextual pervasive games and use of
gamification techniques
• Problem based approach to learning
• Enriching the Gaming Learning Analytics data model with
the contextual, geolocalized and accessibility information
• Large pilots in real settings with content providers
– Formal and informal learning across virtual and physical spaces
• GLA is a key element in the games and pilots evaluation
• Using RAGE infrastructure and extending it for these
new requirements and applications

21. New models for specific uses:
games for people with cognitive
dissabilities
21

22. Cytopathology Challenge
GOALS
• Educational application for an Introduction to Cytopathology
course in cooperation with Harvard Med School and MGH
• Train cytologists both in Harvard U and in limited-resource
countries
CONSTRAINTS
• Work in Low-cost Android tablets (50$)
• Work in web browsers
• No always wifi-connection to play
• Limited budget
22

23. Cytopathology Challenge
23
2.Send traces
1.Play Cytopathology
Challenge
3.Analyze data
4.Recover results
5.Show analysis
to evaluate the students

24. Scientific Goals: from formal pre-
post evaluation to GLA?
• Off-line learning analytics
• Formal evaluation of
games is very complex
and expensive
– Pre-test
– Post-test
• Very few games have
been formally probed to
be effective
• Similar results with
Learning Analytics than
with pre-post test? 24

25. Conclusions
• GLA in Serious Games has a great potential from the
application and research perspective
• Still complex to implement GLA in SG
– Increases the (already high) cost of the games
– New specifications could simplify the task (e.g. xAPI)
• New open frameworks produced by EU projects and new
standards specifications could greatly simplify GLA
implementation and adoption
• And this is just the beginning (BYOD, VR, new
interactions devices..)
– Games are driving the disruption ….

26. 26
Thank you!
•¿Questions?
• Mail: balta@fdi.ucm.es
• Twitter: @BaltaFM
• GScholar: https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en&oi=ao
• ResearchGate: www.researchgate.net/profile/Baltasar_Fernandez-Manjon
• Slideshare: http://www.slideshare.net/BaltasarFernandezManjon