SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Downloaden Sie, um offline zu lesen
LEARNING ANALYTICS
 A PRACTICAL APPROACH WITH VIDEOGAMES


          ÁNGEL SERRANO
JUST ANOTHER DATA MINING PROCESS
SERIOUS GAMES ASSESSMENT
  Not many serious games include in-game evaluation
Serious games with integrated assessment usually rely in
                    Q&A structures
 … but games produce a lot of data that can be analyzed
         with educational/assessment purposes
LEARNING ANALYTICS + VIDEOGAMES
            Web Analytics
         Business Intelligence
           Game Analytics
WHAT DATA CAN WE TRACK?
INTERACTION TRACES     LOGIC TRACES
  Low level events    High level events
  GUI interactions      Player score
    Mouse clicks       Phases changes
    Keystrokes       Completed missions
Joystick movements     Player "deaths"
EVERY GAME IS DIFFERENT...
ONE SET OF TRACES TO
   RULE THEM ALL
START GAME
      Whenever a student launches the game
How many students played the game, who they were and
                when they played.
END GAME
    Whenever a student successfully the game.
Who accomplished the goals established for the game
         Does the optimal goal attain?
QUIT GAME
Whenever a student quits the game, before finishing it
Who abandoned the game before finishing it, and with
    the appropriate context, where he quitted.
PHASE CHANGES
 In an educational videogame, these phases can mark
              several educational sub-goals.
         Identifying most time-consuming phases
Understand how each part of the game is being accessed
    (if the phase exploration sequence is not linear)
SIGNIFICANT VARIABLES
   Games rely on variables to represent their state
   Some of those variables can be relevant for the
                    assessment
Track when and with which values these variables are
                      updated
USER INTERACTION
Raw user interaction (mouse clicks, screen touches, keys
    pressed…) can be used to retrieve some useful
                      information
SOME REQUIREMENTS
           Most of games are black boxes.
    No access to what is going on during game play.

            We need access to game “guts”

Or… the game must communicate with the outside world,
           using some logging framework
GLEANER
GAME LEARNING ANALYTICS FOR EDUCATIONAL RESEARCH
SOME EXAMPLES
THE BIG PARTY




                1:12
REPORTS
BUT...
ARE THEY LEARNING?
GAME DESIGN FOR LEARNING ANALYTICS
1. What do we exactly want to teach in the game?
       Is a skill, is a concept?
2. Can we measure if the students learned it?
3. If so, how are we going to measure it?
       Direct methods: Q&A
       Indrect methods: accomplish a mission that requires
       the skill/concept
A SIMPLER APPROACH
LOST IN SPACE <XML>




                  1:04
DID THEY LEARN WITH THE GAME?
CAN WE RELATE THESE RESULTS WITH
  TRACES COLLECTED IN THE GAME?
In-game metrics: correct and incorrect xml documents
 Students with best performance in post-tests had less
                 incorrect documents
  They also had a smaller number of documents sent
CONCLUSIONS
Our "universal" traces model is useful to extract web-
analytics style statistics.
Game design is key to enable effective learning
analytics
We need more concrete analysis tools to measure the
learning process
   Simple approaches to measure simple things
   Generalization and abstraction
QUESTIONS?
ÁNGEL SERRANO LAGUNA
  angel.serrano@fdi.ucm.es
         @anserran

Weitere ähnliche Inhalte

Was ist angesagt?

Game Data Science: The State of the Art
Game Data Science: The State of the ArtGame Data Science: The State of the Art
Game Data Science: The State of the Art
Africa Perianez
 
Sa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inalSa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inal
Vivek Bhaskaran
 

Was ist angesagt? (18)

Gameplay Concept Tool
Gameplay Concept ToolGameplay Concept Tool
Gameplay Concept Tool
 
Game Data Science: The State of the Art
Game Data Science: The State of the ArtGame Data Science: The State of the Art
Game Data Science: The State of the Art
 
Design feedback
Design feedback Design feedback
Design feedback
 
Star Wars: The Old Republic (A Critical Analysis of Game Design Elements)
Star Wars: The Old Republic (A Critical Analysis of Game Design Elements)Star Wars: The Old Republic (A Critical Analysis of Game Design Elements)
Star Wars: The Old Republic (A Critical Analysis of Game Design Elements)
 
Week 4 Decisions & Flow
Week 4 Decisions & FlowWeek 4 Decisions & Flow
Week 4 Decisions & Flow
 
Mobile Game Analytics: Presentation by Outplay at Games Analytics and BI Forum
Mobile Game Analytics: Presentation by Outplay at Games Analytics and BI ForumMobile Game Analytics: Presentation by Outplay at Games Analytics and BI Forum
Mobile Game Analytics: Presentation by Outplay at Games Analytics and BI Forum
 
Lecture 15 Game Analytics in the Age of Big Data
Lecture 15 Game Analytics in the Age of Big DataLecture 15 Game Analytics in the Age of Big Data
Lecture 15 Game Analytics in the Age of Big Data
 
Building Gameplay
Building GameplayBuilding Gameplay
Building Gameplay
 
A List of Some of the Tools Available to Create Digital Learning Games
A List of Some of the Tools Available to Create Digital Learning GamesA List of Some of the Tools Available to Create Digital Learning Games
A List of Some of the Tools Available to Create Digital Learning Games
 
Dsdt meetup 2018 02-12
Dsdt meetup 2018 02-12Dsdt meetup 2018 02-12
Dsdt meetup 2018 02-12
 
User Experience 8: Business, Ethics and More
User Experience 8: Business, Ethics and MoreUser Experience 8: Business, Ethics and More
User Experience 8: Business, Ethics and More
 
LAFS Game Design 8 - Playtesting
LAFS Game Design 8 - PlaytestingLAFS Game Design 8 - Playtesting
LAFS Game Design 8 - Playtesting
 
Sa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inalSa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inal
 
Checking the difficulty of evolutionary-generated maps in a N-Body inspired m...
Checking the difficulty of evolutionary-generated maps in a N-Body inspired m...Checking the difficulty of evolutionary-generated maps in a N-Body inspired m...
Checking the difficulty of evolutionary-generated maps in a N-Body inspired m...
 
Tools for Mobile UX Design
Tools for Mobile UX DesignTools for Mobile UX Design
Tools for Mobile UX Design
 
Learning Analytics in serious games
Learning Analytics in serious gamesLearning Analytics in serious games
Learning Analytics in serious games
 
Playability & Player Experience Research
Playability & Player Experience ResearchPlayability & Player Experience Research
Playability & Player Experience Research
 
Андрій Доценко “Game Designer’s Journey”
Андрій Доценко “Game Designer’s Journey”Андрій Доценко “Game Designer’s Journey”
Андрій Доценко “Game Designer’s Journey”
 

Andere mochten auch

First community mortgage marketing department account executive
First community mortgage marketing department account executiveFirst community mortgage marketing department account executive
First community mortgage marketing department account executive
Wes McGaffin
 

Andere mochten auch (11)

2013 06 13 (uc3m) emadrid rcohen 2u not a mooc
2013 06 13 (uc3m) emadrid rcohen 2u not a mooc2013 06 13 (uc3m) emadrid rcohen 2u not a mooc
2013 06 13 (uc3m) emadrid rcohen 2u not a mooc
 
2010-04-16 eoi mit ocwc stephen carson
2010-04-16 eoi mit ocwc stephen carson2010-04-16 eoi mit ocwc stephen carson
2010-04-16 eoi mit ocwc stephen carson
 
2011 06 01 (uned) emadrid fcardinali exactls mobile learning myth or realityr...
2011 06 01 (uned) emadrid fcardinali exactls mobile learning myth or realityr...2011 06 01 (uned) emadrid fcardinali exactls mobile learning myth or realityr...
2011 06 01 (uned) emadrid fcardinali exactls mobile learning myth or realityr...
 
2011 06 02 (uned) emadrid mspecht ounl linking sensors actuators and learning
2011 06 02 (uned) emadrid mspecht ounl linking sensors actuators and learning2011 06 02 (uned) emadrid mspecht ounl linking sensors actuators and learning
2011 06 02 (uned) emadrid mspecht ounl linking sensors actuators and learning
 
First community mortgage marketing department account executive
First community mortgage marketing department account executiveFirst community mortgage marketing department account executive
First community mortgage marketing department account executive
 
2tor Twin Day July 2010
2tor Twin Day July 20102tor Twin Day July 2010
2tor Twin Day July 2010
 
EdTech Europe 2015 [Track 1]: [2U], ([Rob Cohen], [President])
EdTech Europe 2015 [Track 1]: [2U], ([Rob Cohen], [President])EdTech Europe 2015 [Track 1]: [2U], ([Rob Cohen], [President])
EdTech Europe 2015 [Track 1]: [2U], ([Rob Cohen], [President])
 
TWOU_Pitch
TWOU_PitchTWOU_Pitch
TWOU_Pitch
 
Visual Design with Data
Visual Design with DataVisual Design with Data
Visual Design with Data
 
3 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 20173 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 2017
 
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheHow to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your Niche
 

Ähnlich wie 2012 12 14 (ucm) emadrid aserrano ucm learning analytics vision practica aplicada videojuegos educativos

Gamification - ASTD RTA
Gamification - ASTD RTAGamification - ASTD RTA
Gamification - ASTD RTA
trickyraymer
 
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
Staffan Björk
 
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
Use8.net
 
Sa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inalSa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inal
Vivek Bhaskaran
 

Ähnlich wie 2012 12 14 (ucm) emadrid aserrano ucm learning analytics vision practica aplicada videojuegos educativos (20)

Gamification: Future or Fail?
Gamification: Future or Fail?Gamification: Future or Fail?
Gamification: Future or Fail?
 
Introduction to Gamification
Introduction to GamificationIntroduction to Gamification
Introduction to Gamification
 
Skill Mapping
Skill MappingSkill Mapping
Skill Mapping
 
Primer on Play: Case Study for Knowledge Guru
Primer on Play: Case Study for Knowledge GuruPrimer on Play: Case Study for Knowledge Guru
Primer on Play: Case Study for Knowledge Guru
 
A Primer On Play: How to use Games for Learning and Results
A Primer On Play: How to use Games for Learning and ResultsA Primer On Play: How to use Games for Learning and Results
A Primer On Play: How to use Games for Learning and Results
 
Gamification - ASTD RTA
Gamification - ASTD RTAGamification - ASTD RTA
Gamification - ASTD RTA
 
Fun and the MDA framework
Fun and the MDA frameworkFun and the MDA framework
Fun and the MDA framework
 
[Pandora 22] Boosting Game Design with Analytics - Nikola Vasiljevic
[Pandora 22] Boosting Game Design with Analytics - Nikola Vasiljevic[Pandora 22] Boosting Game Design with Analytics - Nikola Vasiljevic
[Pandora 22] Boosting Game Design with Analytics - Nikola Vasiljevic
 
WEEF/GEDC eMadrid_Systematizing Game Learning Analytics for Improving Serious...
WEEF/GEDC eMadrid_Systematizing Game Learning Analytics for Improving Serious...WEEF/GEDC eMadrid_Systematizing Game Learning Analytics for Improving Serious...
WEEF/GEDC eMadrid_Systematizing Game Learning Analytics for Improving Serious...
 
Icce21 systematizing game learning analytics for improving serious games life...
Icce21 systematizing game learning analytics for improving serious games life...Icce21 systematizing game learning analytics for improving serious games life...
Icce21 systematizing game learning analytics for improving serious games life...
 
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
Gameplay design patterns presentation at dragon's lair, stockholm, sweden 201...
 
Carole Bagley - Elements of Effective Instructional Learning Game Design
Carole Bagley - Elements of Effective Instructional Learning Game DesignCarole Bagley - Elements of Effective Instructional Learning Game Design
Carole Bagley - Elements of Effective Instructional Learning Game Design
 
Fundamental of game design part 2
Fundamental of game design part 2Fundamental of game design part 2
Fundamental of game design part 2
 
Game Ethology 2
Game Ethology 2Game Ethology 2
Game Ethology 2
 
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
Serco Usability Research, Ben Weedon, The challenge of measuring game play ex...
 
Sa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inalSa zynga-vivek-kevin-f inal
Sa zynga-vivek-kevin-f inal
 
More Than Points: Architecting Engagement Through Game Design Thinking
More Than Points: Architecting Engagement Through Game Design ThinkingMore Than Points: Architecting Engagement Through Game Design Thinking
More Than Points: Architecting Engagement Through Game Design Thinking
 
User Experience 6: Qualitative Methods, Playtesting and Interviews
User Experience 6: Qualitative Methods, Playtesting and InterviewsUser Experience 6: Qualitative Methods, Playtesting and Interviews
User Experience 6: Qualitative Methods, Playtesting and Interviews
 
Using Game Learning Analytics to Improve the Design, Evaluation and Deploymen...
Using Game Learning Analytics to Improve the Design, Evaluation and Deploymen...Using Game Learning Analytics to Improve the Design, Evaluation and Deploymen...
Using Game Learning Analytics to Improve the Design, Evaluation and Deploymen...
 
Learning Analytics Serious Games Cognitive Disabilities
Learning Analytics Serious Games Cognitive DisabilitiesLearning Analytics Serious Games Cognitive Disabilities
Learning Analytics Serious Games Cognitive Disabilities
 

Mehr von eMadrid network

Mehr von eMadrid network (20)

Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo TovarRecognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
Recognizing Lifelong Learning Competences: A Report of Two Cases - Edmundo Tovar
 
A study about the impact of rewards on student's engagement with the flipped ...
A study about the impact of rewards on student's engagement with the flipped ...A study about the impact of rewards on student's engagement with the flipped ...
A study about the impact of rewards on student's engagement with the flipped ...
 
Assessment and recognition in technical massive open on-line courses with and...
Assessment and recognition in technical massive open on-line courses with and...Assessment and recognition in technical massive open on-line courses with and...
Assessment and recognition in technical massive open on-line courses with and...
 
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
Recognition of learning: Status, experiences and challenges - Carlos Delgado ...
 
Bootstrapping serious games to assess learning through analytics - Baltasar F...
Bootstrapping serious games to assess learning through analytics - Baltasar F...Bootstrapping serious games to assess learning through analytics - Baltasar F...
Bootstrapping serious games to assess learning through analytics - Baltasar F...
 
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
 
Best paper Award - Miguel Castro
Best paper Award - Miguel CastroBest paper Award - Miguel Castro
Best paper Award - Miguel Castro
 
eMadrid Gaming4Coding - Possibilities of game learning analytics for coding l...
eMadrid Gaming4Coding - Possibilities of game learning analytics for coding l...eMadrid Gaming4Coding - Possibilities of game learning analytics for coding l...
eMadrid Gaming4Coding - Possibilities of game learning analytics for coding l...
 
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdfSeminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
Seminario eMadrid_Curso MOOC_Antonio de Nebrija_Apología del saber.pptx.pdf
 
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
eMadrid-Opportunities and Design Challenges in the Gaming4Coding Project_Pete...
 
Open_principles_and_co-creation_for_digital_competences_for_students.pdf
Open_principles_and_co-creation_for_digital_competences_for_students.pdfOpen_principles_and_co-creation_for_digital_competences_for_students.pdf
Open_principles_and_co-creation_for_digital_competences_for_students.pdf
 
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
Competencias_digitales_del_profesorado_universitario_para_la_educación_abiert...
 
eMadrid_KatjaAssaf_DigiCred.pdf
eMadrid_KatjaAssaf_DigiCred.pdfeMadrid_KatjaAssaf_DigiCred.pdf
eMadrid_KatjaAssaf_DigiCred.pdf
 
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdfPresentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
Presentazione E-Madrid - 12-01-2023 Ruth Kerr.pdf
 
EDC-eMadrid_20230113 Ildikó Mázár.pdf
EDC-eMadrid_20230113 Ildikó Mázár.pdfEDC-eMadrid_20230113 Ildikó Mázár.pdf
EDC-eMadrid_20230113 Ildikó Mázár.pdf
 
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
2022_12_16 «“La informática en la educación escolar en Europa”, informe Euryd...
 
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
2022_12_16 «Informatics – A Fundamental Discipline for the 21st Century»
 
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
2022_12_16 «Efecto del uso de lenguajes basados en bloques en el aprendizaje ...
 
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
2022_11_11 «AI and ML methods for Multimodal Learning Analytics»
 
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
2022_11_11 «The promise and challenges of Multimodal Learning Analytics»
 

2012 12 14 (ucm) emadrid aserrano ucm learning analytics vision practica aplicada videojuegos educativos

  • 1. LEARNING ANALYTICS A PRACTICAL APPROACH WITH VIDEOGAMES ÁNGEL SERRANO
  • 2. JUST ANOTHER DATA MINING PROCESS
  • 3. SERIOUS GAMES ASSESSMENT Not many serious games include in-game evaluation Serious games with integrated assessment usually rely in Q&A structures … but games produce a lot of data that can be analyzed with educational/assessment purposes
  • 4. LEARNING ANALYTICS + VIDEOGAMES Web Analytics Business Intelligence Game Analytics
  • 5. WHAT DATA CAN WE TRACK? INTERACTION TRACES LOGIC TRACES Low level events High level events GUI interactions Player score Mouse clicks Phases changes Keystrokes Completed missions Joystick movements Player "deaths"
  • 6. EVERY GAME IS DIFFERENT...
  • 7. ONE SET OF TRACES TO RULE THEM ALL
  • 8. START GAME Whenever a student launches the game How many students played the game, who they were and when they played.
  • 9. END GAME Whenever a student successfully the game. Who accomplished the goals established for the game Does the optimal goal attain?
  • 10. QUIT GAME Whenever a student quits the game, before finishing it Who abandoned the game before finishing it, and with the appropriate context, where he quitted.
  • 11. PHASE CHANGES In an educational videogame, these phases can mark several educational sub-goals. Identifying most time-consuming phases Understand how each part of the game is being accessed (if the phase exploration sequence is not linear)
  • 12. SIGNIFICANT VARIABLES Games rely on variables to represent their state Some of those variables can be relevant for the assessment Track when and with which values these variables are updated
  • 13. USER INTERACTION Raw user interaction (mouse clicks, screen touches, keys pressed…) can be used to retrieve some useful information
  • 14. SOME REQUIREMENTS Most of games are black boxes. No access to what is going on during game play. We need access to game “guts” Or… the game must communicate with the outside world, using some logging framework
  • 15. GLEANER GAME LEARNING ANALYTICS FOR EDUCATIONAL RESEARCH
  • 20. GAME DESIGN FOR LEARNING ANALYTICS 1. What do we exactly want to teach in the game? Is a skill, is a concept? 2. Can we measure if the students learned it? 3. If so, how are we going to measure it? Direct methods: Q&A Indrect methods: accomplish a mission that requires the skill/concept
  • 22. LOST IN SPACE <XML> 1:04
  • 23. DID THEY LEARN WITH THE GAME?
  • 24. CAN WE RELATE THESE RESULTS WITH TRACES COLLECTED IN THE GAME? In-game metrics: correct and incorrect xml documents Students with best performance in post-tests had less incorrect documents They also had a smaller number of documents sent
  • 25. CONCLUSIONS Our "universal" traces model is useful to extract web- analytics style statistics. Game design is key to enable effective learning analytics We need more concrete analysis tools to measure the learning process Simple approaches to measure simple things Generalization and abstraction
  • 26. QUESTIONS? ÁNGEL SERRANO LAGUNA angel.serrano@fdi.ucm.es @anserran