Slides of the paper presented at the Ideas and Innovation session in First International Conference on Learning Analytics and Knowledge.

1. Stepping out of the box
Toward analytics outside the LMS
Abelardo Pardo, Carlos Delgado Kloos
University Carlos III of Madrid, Spain
www.slideshare.net/abelardo_pardo

2. Warning: From macro to micro
Apply analytics to
everyday operations
in a course.
Toward highly
focused quick
actuators
Micro does not mean simple!
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 2

3. The scenario
• Second year engineering
• Class meets twice a week
• Active learning (U work, I
work)
1.Learn how to program in C
2.Use “industrial” tools
3.Work in teams
4.Increase your self-learning
• Work in pairs and in teams
• Significant out-of-class activity
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 3

4. The problem
• LMS = Forum
• Intense activity outside LMS.
• Can we record the interaction?
• Can we take measures?
• Can we act over 250 students?
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 4

5. The solution
• Think as a student
• Offer fully configured
environment for the course
• Offer doc. sharing
• Attract course activity to
that environment
• Capture, report, predict, act
and refine
• Total transparency!
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 5

6. Virtual Machines
• A (guest) computer in a
file (8 Gigabytes)
• Runs as a regular
application in your
(host) PC
• When in full window,
almost as running
another operating
system
• Access to files in “host”
machine
• Available 1st day
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 6

7. Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 7

8. Internals
• Instrumentation of main
tools
• Record when tools are
used
• Relay data to server when
updating shared
documents
• Transparent to the user
• User confirms terms of use
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 8

9. Advantages
• Closer insight on
procedural activities
• Limited to the course
environment
• Limited in time
• Observe some
“elusive” behavior
– Ability to use modern
engineering tools
– Ability to function in
multidisciplinary
teams
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 9

10. On-going work
• First data set collected:
Sep./Dic. 10. Observed 248
Students
• Used by 85% of them
• Encoded in CAM
• 48K events
• Beginning to report,
towards “act”
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 10

11. Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 11

12. Browsing outside the LMS
• 48K events: 15,5K are URLs
(32.07%)
• 8,669 unique pages
• 28.51% point to the LMS
• Conclusion: 3 out of 4 pages
are outside the LMS.
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 12

13. Conclusions
• VMs offer a good
trade-off for course
environment
• Advantages to
students
• Potential for high
observability
• Oriented toward
quick actuators
www.it.uc3m.es/abel
Twitter: @abelardopardo
Abelardo Pardo. Stepping out of the Box. Analytics outside the LMS 13