How can learning analytics be taken from its design to its deployment in an educational institution? What are the issues, limitations, strategies? This presentation includes a descirption of Learning Analytics, examples, how to tackle systemic deployment and suggestions on how to build institutional capacity.

1. Will Scullin Flickr
Learning and Behavioral Analytics
From concept to reality
Dr Abelardo Pardo (@abelardopardo)
Associate Head of Learning and Teaching
School of Electrical and Information Engineering
slideshare.net/abelardo_pardo
Taylor’s University
Academic Leaders Retreat
4 March 2014

2. jbgeronimi flickr.com
“How to measure the effectiveness of fye activities
and ensure we r on d right track?” Yesterday by CT
“No data, no talk” Yesterday by DVC Pradeep Nair
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
2

3. Unhindered by Talent flickr.com
Many colleges and universities have demonstrated
that analytics can help significantly advance an
institution in such strategic areas as resource
allocation, student success, and finance
(Bichsel, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
3

4. cygnus921 Flickr
What is LA?
Current
Projects
Systemic
Deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Building
Institutional
Capacity
4

5. cygnus921 Flickr
What is LA?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
5

6. NYTimes 16/02/12
Is she pregnant?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
6

7. Daily Mail UK 20/03/12
TV is watching you
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
7

8. WSJ 19/07/12
How do you read?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
8

9. NY Times 24/11/12
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
9

10. Humans are Subjective
Mind Sights: Original Visual Illusions, Ambiguities, and Other Anomalies, With a Commentary on the Play of Mind in Perception and Art, Roger N. Shepard, 1990
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
10

11. Learning Analytics
Measurement, collection, analysis
and reporting of data about learners
and their contexts, for purposes of
understanding and optimizing
learning and the environments in which
it occurs.
(US Dept. of Education)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
11

12. Kurafire flickr.com
Feedback as a dialogue
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
12

13. Discipline in its early steps,
but with large potential.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
13

14. (Manyika et al., 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
14

15. http://www.learninganalytics.net/?p=131 (last visited Sep 2013)
(Siemens & Long, 2011)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
15

16. www.solaresearch.org
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
16

17. Werner Kunz Flickr.com
Predict
Refine
Act
The five
steps of
analytics
Report
Collect
(Campbell, De Blois, Oblinger 2007, Academic Analytics, EDUCAUSE)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
17

18. Thomas Leth-Olsen flickr.com
Knowledge
Discovery in
Databases
Technology
enhanced
learning
Social
network
analysis
Business
Intelligence
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Educational
modelling
18

19. zzpza flickr.com
• Statistical prediction
• Clustering and profiling
• Relationship mining
• Link prediction in
networks
• Text analysis
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
19

20. Secretlondon123 flickr.com
1
2
3
4
Abelardo Pardo
Context
Questions
Data + algorithms
Level of intervention
Learning and Behavioral Analytics. From concept to reality
20

21. measure Joe flickr.com
1
2
3
4
Abelardo Pardo
Context: Student
retention
Questions: Why do
they leave? Can it be
avoided?
Data + algorithms: ?
Level of intervention: ?
Learning and Behavioral Analytics. From concept to reality
21

22. Shaylor flickr.com
1
2
3
4
Context: Student success
Questions: How did they
learn?
Data + algorithms: ?
Level of intervention: ?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
22

23. cygnus921 Flickr
What is LA?
Abelardo Pardo
Current
Projects
Learning and Behavioral Analytics. From concept to reality
23

24. Joe Fruchey flickr.com
MYTH: It is a highly sophisticated technique
It can be deployed gradually
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
24

25. tudedude flickr.com
MYTH: LA is a tool to fix a problem.
LA is a process to build/refine a model
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
25

26. HikingArtist flickr.com
MYTH: Once we have a
model, we apply it to all
scenarios
Models are derived from
the scenarios
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
26

27. DullHunk flickr.com
Reduce attrition. Phone 10 students this semester.
Which students? When?
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
27

28. colecamp flickr.com
A/B Testing: Space usage after change in
course/curriculum
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
28

29. (Macfadyen & Dawson, 2010)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
29

30. (Romero, Ventura & García, 2008)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
30

31. www.itap.purdue.edu/studio/signals (Last accessed Oct. 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
31

32. Signals
Nottsexminer
• Run by the instructor
• Points so far (Gradebook)
• Time on task (LMS)
• Past performance (SIS)
• Instructor chooses
intervention
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
32

33. Rio Salado (AZ, USA)
• Enrolment figures
• LMS variables
• Thirty factors considered
• Separate course models
• Naïve Bayes classification
method
• Model is applied to classify
students
http://campustechnology.com/Articles/2011/12/14/Monitoring-the-PACE-of-Student-Learning-Analytics-at-Rio-Salado-College.aspx?p=1
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
33

34. www.snappvis.org (Last accessed Oct. 2013)
Bookmarklet in your browser
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
34

35. Learning Glass
(Leony et al., 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
35

36. Data uses
• Multiple levels of intervention
• Variety of data sources
• Commonalities vs differences
• Require multidisciplinary
work
(Bichsel, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
36

37. Instructors
Student Enrolment
Abelardo Pardo
Curriculum development
Learning and Behavioral Analytics. From concept to reality
37
Zilverbat flickr.com
CPSC

38. cygnus921 Flickr
What is LA?
Current
Projects
Systemic
Deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
38

39. (Greller and Drachler, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
39

40. paral_lax flickr.com
Optimize
Predict
What if
scenarios
Understand
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
40

41. puthoOr Photography flickr.com
LMS interactions, quizzes, response times, sessions,
hints requested, questionnaires, interviews, requests,
demographics, high school grades, etc.
(Bienkowski, M., Feng, M., & Means, B., 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
41

42. (Campbel, DeBlois, Oblinger, 2007)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
42

43. deepwarren Flickr
Observe in and out of the classroom
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
43

44. www.smh.com.au/technology/technology-news/universities-monitor-online-activity-20130413-2hsj2.html (last accessed Sep. 2013)
44
Learning and Behavioral Analytics. From concept to reality
Abelardo Pardo

45. www.uts.edu.au/research-and-teaching/our-research/advanced-analytics-institute
45
Learning and Behavioral Analytics. From concept to reality
Abelardo Pardo

46. www.crlt.umich.edu/slam
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
46

47. (Powell & MacNeil, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
47
Salfalko flickr.com
Data Wrangler: someone comfortable handling
statistics, manipulating data for visualisation, and
capable of engaging with academics about the student
experience and course design. It is their job to
experiment with different tools to interpret, visualise
and share information with academics as a basis for
gaining actionable insights.

48. (Buckingham Shum, 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
48

49. cygnus921 Flickr
What is LA?
Current
Projects
Systemic
Deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
Building
Institutional
Capacity
49

50. Impact Levels
gregoriosz
• Micro: at risk students,
insight to learners,
recommendations
• Meso: clear understanding of
objectives, increase
productivity, help leaders
• Macro: institutional
transformation
(Buckingham Shum, 2012)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
50

51. Phase 1
Phase 2
Derpunk flickr.com
Static Reports
Dynamic
Analysis and
Intervention
Optimization
Phase 3
(Norris & Baer, 2013)
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
51

52. Allie Caufield flickr.com
Student Success and Retention
Practices
Workflows
Processes
Policies
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
52

53. juhansonin flickr.com
1) Develop culture of using data to make decisions
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
53

54. Rosellsgirl flickr.com
2) IT dept. are key to initiate connection
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
54

55. Not so much flickr.com
3) Multidimensional from the start
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
55

56. Abraham C flickr.com
4) Gradual deployment
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
56

57. Fhemerick flickr.com
5) Interfaces that provide value to stakeholders
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
57

58. Doug Woods flickr.com
6) Refinement as intrinsic component of the process
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
58

59. Laverrue flickr.com
7) Privacy and Ethics
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
59

60. Pennstatenews flickr.com
8) Keep the student at the center
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
60

61. Will Scullin Flickr
Learning and Behavioral Analytics
From concept to reality
Dr Abelardo Pardo (@abelardopardo)
Associate Head of Learning and Teaching
School of Electrical and Information Engineering
slideshare.net/abelardo_pardo
Taylor’s University
Academic Leaders Retreat
4 March 2014

62. References
Arnold, K. E. (2010).
Signals: Applying Academic Analytics.
EDUCAUSE Quarterly, 33(1), 10.
Bichsel, J. (2012).
Analytics in Higher Education Benefits, Barriers, Progress, and Recommendations.
EDUCAUSE Center for Applied Research
Bienkowski, M., Feng, M., & Means, B. (2012).
Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. US
Department of Education.
Buckingham Shum, S. (2012).
Learning analytics. UNESCO policy brief.
Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdfhttp://
iite.unesco.org/pics/publications/en/files/3214711.pdf, March 2014
Buckingham Shum, S. (2013).
Building Analytics Capability @open.edu
Retrieved from
http://www.slideshare.net/sbs/building-analytics-capacity-openeduhttp://www.slideshare.net/sbs/buildinganalytics-capacity-openedu, March
2014
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007).
Academic Analytics.
Educause Review (Vol. 42). EDUCAUSE White Paper.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
62

63. References 2
Dawson, S. (2010).
“Seeing” the learning community: An exploration of the development of a resource for monitoring online
student networking.
British Journal of Educational Technology, 41(5), 736–752.
Ferguson, R. (2012).
The State of Learning Analytics in 2012: A Review and Future Challenges a review and future challenges.
Technical Report Knowledge Media Institute KMI–12–01). The Open University UK.
Greller, W., & Drachsler, H. (2012).
Translating learning into numbers: A generic framework for learning analytics.
Educational Technology & Society, 15(3), 42–57.
Macfadyen, L. P., & Dawson, S. (2010).
Mining LMS data to develop an “early warning system” for educators: A proof of concept.
Computers & Education, 54(2), 588–599
Manyika, J., Chui, M., Groves, P., Farrell, D., Kuiken, S. Van, & Doshi, E. A. (2013).
Open data: Unlocking innovation and performance with liquid information
McKinsey Global Institute (p. 116).
Norris, D. M., & Baer, L. L. (2013).
Building Organizational Capacity for Analytics.
Educause.
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
63

64. References and 3
Powell, S., & MacNeil, S. (2012).
Analytics Series Institutional Readiness for Analytics (Vol. 1, pp. 1–11).
JISC Center for Educational Technology & Interoperability Standards, Analytics Series, 1(8).
Romero, C., Ventura, S., & Garcia, E. (2008).
Data mining in course management systems: Moodle case study and tutorial.
Computers & Education, 51(1), 368–384.
Siemens, G., & Long, P. (2011).
Penetrating the Fog: Analytics in Learning and Education.
Educause Review, 48(5),
Siemens, G. (2013).
Learning Analytics: The Emergence of a Discipline.
American Behavioral Scientist, 57(10), 1380–1400.
Siemens, G., Dawson, S., & Lynch, G. (2013).
Improving the Quality and Productivity of the Higher Education Sector. Policy and Strategy for
Systems-Level Deployment of Learning Analytics.
Technical Report, Office for Learning and Teaching, Australian Government
Abelardo Pardo
Learning and Behavioral Analytics. From concept to reality
64