Workshop held at the Australian Learning Analytics Summer Institute (A-LASI) run by Lori Lockyer, Tim Rogers and Shane Dawson

1. Getting Started with
Learning Analytics
Lori Lockyer
Tim Rogers
Shane Dawson

2. What about today?
•
•
•
•
•
•
•
Introductions and background
From base camp to summit
Data – it seems important
Analytics for teachers
Wrap up
Beer and cookies
Questions, concerns or issues

3. Learning Analytics
…is the collection, collation, analysis and
reporting of data about learners and their
contexts, for the purposes of understanding
and optimizing learning

4. Learning Analytics
Ed theory, Ed practice, SNA, Data
mining, Machine learning, semantic,
data visualisations, sense-making,
psychology (social, cognitive,
organisational), learning sciences

5. Examine large data sets – trends/ patterns or
anomalies.
What do patterns indicate and what do
changes in habit indicate?

6. Current State
Higher education:
• Lots of isolated work targeting attrition. Very few
large enterprise egs.
• Commercial – IBM, SAS, Hobsons, D2L Insight,
BB analytics

7. Education Examples

8. Education Examples

9. Education Examples

10. Education - Purdue

11. Education - UMBC

12. Education – UniSA

13. Current Focus
Pass/Fail, Retention
Concept understanding
Kentucky: 1.3% - 80 stds approx 400k

14. Where next?
Beyond dashboards
Predictive and recommender states

15. Future

16. Learner control

18. NLP – video annotations

19. Future
Emotions/ face tracking
Frustrated
Confusion
Engaged
Activity
modified
Continuous
state of
challenge

20. First steps – the why,
what and how of data
Improving feedback in mass higher education

21. Data, data,
everywhere…
• Where data is accessible it is usually
lagged, scattered, indecipherable,
requires manipulating, lacks context…
• Yes, there are BI reports, but they are
mostly for the converted and don’t flag
exceptions

22. …but not a digit of
use
• Currently, despite all the data,
• Students often don’t know how they are
going
• Academics don’t know if their teaching is
effective
• Program/degree owners don’t know how
their students navigate their way through
• Management don’t know if the Uni is on
track

23. Outline of data
thinking process
• What is the purpose for the data?
• What data is needed (and who ‘owns’
it)
• How to work with the data?
• How to make the data actionable?

24. Data for what
purpose?
• Student level support (success and
retention)
• Educator needs – improving teaching
and learning
• Program designer/owner needs –
curriculum flows
• Management/QA requirements – are
courses/subjects meeting standards
and improving?

25. Who owns the data…
• …aka where do you get it? IT,
Business Intelligence, Admin?
• And others, e.g. class rolls, library data,
orientation attendance, in-class
formative and summative assessments
etc

26. Working with data
• All data will need various degrees of
extraction and transformation
• All data needs contextualisation, and a
decision about how fine-grained that
needs to be
• For example, is this a problem?…

27. 42

28. 42
Student

29. 42
Student
Test score
/100

30. 42
Student
Test score 10%
/100
course
mark

31. 42
Student
Test score 10%
Degree:
/100
course Chemical
mark
Engineering

32. 42
Student
Test score 10%
Degree:
Course:
/100
course Chemical
Shakesp
mark
Engineering eare and
Society

33. 42
Student
Test score 10%
Degree:
Course: Class
/100
course Chemical
Shakesp Position
mark
Engineering eare and : 1/36
Society

34. Making data
actionable
• Visualising the data for summary and
exception highlighting
• Trends, key junctures, cumulative risk
• Tools for action, e.g. CRMs, and
business processes

35. Visualising critical
metrics
The work of Stephen Few
Context is everything

37. Data and analytics to support
learning design and implementation

38. Learning Analytics:
… measurement, collection, analysis and reporting
of data about learners and their contexts, for
purposes of understanding and optimizing learning
and the environments in which it occurs.
(LAK11 - https://tekri.athabascau.ca/analytics/)

39. Where and how does learning
occur in HE in Australia?
• Within courses/units
• which are designed predominately by
teachers (not instructional designers)
• who interact with students as they are
learning
• who can, may, may not, intervene in
the learning process.

40. How might a university teacher
use data and analytics?
• Analytics to inform design decisions
• Just-in-time analytics to understand learner
activity and experience during
implementation
• Recommendations for learner action
• Analytics for post-implementation reflection
and revision
• Support scholarship of teaching

41. What can data help us with?
• Moe than…
– retention/attrition
– “… and they liked it”
• To are they…
–
–
–
–
–
doing what you intended?
understanding the task?
on-task/off-task?
motivated, engaged?
actually learning anything?

42. Learning analytics can only help us
answer these questions if they are:
- specific to the learning outcomes of
the unit
- related to how we think learning
occurs for such outcomes, in the
discipline…
- relevant to the learning design we
have put in place

43. In other words…
… the teaching and learning context
matters.

44. Mapping a design

45. Case Based Learning Design adapted from Bennett (2002) available at
http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

46. LMS log: Student log in;
access case
Network diagram: even
pattern of participation
Network diagram: teachercentred pattern
Network diagram: even
pattern of participation
Document sharing logs of
contribution
LMS log: access to teacher
feedback
LMS log: submission of
reflection template
Content analysis: depth of
reflection
Case Based Learning Design adapted from Bennett (2002) available at
http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

48. LMS log: Student log in;
access case
Network diagram: even
pattern of participation
Network diagram: teachercentred pattern
Network diagram: even
pattern of participation
Document sharing logs of
contribution
LMS log: access to teacher
feedback
LMS log: submission of
reflection template
Content analysis: depth of
reflection
Case Based Learning Design adapted from Bennett (2002) available at
http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

50. LMS log: Student log in;
access case
Network diagram: even
pattern of participation
Network diagram: teachercentred pattern
Network diagram: even
pattern of participation
Document sharing logs of
contribution
LMS log: access to teacher
feedback
LMS log: submission of
reflection template
Content analysis: depth of
reflection
Case Based Learning Design adapted from Bennett (2002) available at
http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

51. Now it is your turn:
Sample design or
Your learning design?
•
•
•
•
•
•
What are the learning outcomes?
What does the design look like? Map it?
What do you want to know?
What data will inform these?
What patterns do you anticipate?
What can you do about it?

52. Summary
• We are already capturing a lot of data
• There’s a lot of information we are not
systematically capturing
• Current or possible answer might
answer our questions
• First we have to have relevant
questions and know what we are
prepared to do with the answers