In an interactive digital-game, traces of a learner’s progress, problem-solving attempts, self-expressions and social communications can entail highly detailed and time-sensitive computer-based documentation of the context, actions, processes and products. This talk will present measurement and analysis considerations that are needed to address the challenges of finding patterns and making inferences based on these data. Methods based in data-mining, machine learning, model-building and complexity theory form a new theoretical foundation for dealing with the challenges of time sensitivity, spatial relationships, multiple layers of aggregations at different scales, and the dynamics of complex behavior spaces. Examples of these considerations in game-based learning analytics are presented and discussed, with implications for game-based e-learning design.

1. Knowing what students know
from game-based learning
David Gibson
Curtin University

2. The Premise
In an interactive digital-game, traces of a
learner’s progress, problem-solving attempts,
self-expressions and social communications can
entail highly detailed and time-sensitive
computer-based documentation of the context,
actions, processes and products.

3. Example
Clarke-Midura & Gibson, 2013
• Contexts: Farm, Playground, Science Lab
• Actions: Talking, Testing, Walking to…
• Processes & Products: Test Results, Explanations

4. Interaction Traces = Evidence
There is a need for new frameworks, concepts
and methods for measuring what someone
knows and can do based on game interactions
and artifacts created during serious play
Why? (It’s a mouthful) Ubiquitous, unobtrusive,
interactive big data created by people working
in digital media performance spaces

5. Example
• Ecological rationality & Empirical probability
Clarke-Midura & Gibson, 2013

6. Sensors
• Wireless EEG
– Facial muscles, emotional
clusters, raw EEG
• Wireless Galvanic Skin
Conductance
– Arousal level
• Eye Tracker
– Gaze-point, duration,
mouse-clicks
• Haptics
– Button presses, head tilt

7. Anatomy of the System
Helen Chavez & Javier Gomez, ASU

8. Challenge: New Psychometrics
• What are some of the measurement and
analysis considerations needed to address the
challenges of finding patterns and making
inferences based on data from digital learning
experiences?

9. Biometric Sensor Nets
• What patterns do we
find?
• How do they change
over time?
• How do they relate to
baseline and
experimental
activities?

10. Network Graphs
Digraphs illustrate
structural
relationships in the
causative factors
during a time slice or
event frame.

11. Network Analysis
AF3 F7
AF3
Adjacency tables
Centrality
F7
F3
FC5
T7
P7
O1
O2
P8
Digraphs
T8
FC6
F4
F8
AF4
GX
GY
F3
FC5 T7
P7
O1 O2 P8
T8
FC6 F4
F8
AF4 GX GY

12. Symbolic Regression
Automated
search for
algorithms
Clarke-Midura & Gibson, 2013

13. New Space for Performance
• Unfold in time
• Cover a multivariate space of possible actions
• Assets contain both intangible (e.g. value,
meaning, sensory qualities, and emotions)
and tangible components (e.g. media,
materials, time and space)
NOTE: Asset utilization during performance
provides evidence of what a user knows and
can do

14. Example
Clarke-Midura & Gibson, 2013
Students who had
this pattern of
resources were
most likely to
show evidence of
forming a
hypothesis

15. Performance Space Features
• Unconstrained complex multidimensional
stimuli and responses
• Dynamic adaptation of items to user, which
entails interactivity and dependency
• Nonlinear behaviors with both temporal and
spatial components
NOTE: Higher order and creative thinking is
supported in such a space

16. Research Questions
• What patterns are
found within &
between sensors?
• How do these patterns
relate to baseline and
experimental
activities?

17. Data Dashboard at ASU
Helen Chavez and Javier Gomez

18. Thinking States
Rise in
uncertainty
and interest
During thinking
Agreement &
concentration
drop

19. The Game-Based Psychometric
Landscape
• A “do over” for performance assessment
• New ways of performing = new methods of
data capture, analysis and display
• Complex tasks and artifacts containing
– higher order thinking (e.g. decision sequences)
– physical performances demonstrating skills
– emotional responses

20. What Games & Sims Teach
•
•
•
•
•
•
•
Understanding big ideas - systems knowledge
Dealing with time and scale
Practice in decision-making
Active problem-solving
Concepts, strategies, & tactics
Understanding processes beyond experience
Practice makes improvement
(Aldrich, 2005)

21. Conclusion
Methods based in data-mining, machine
learning, model-building and complexity theory
form a theoretical foundation for dealing with
the challenges of time sensitivity, spatial
relationships, multiple layers of aggregations at
different scales, and the dynamics of complex
behavior spaces.