Slides presented at ALT-C 2016; updated after the presentation. What can we learn from learning analytics? A case study based on an analysis of student use of video recordings [paper 1247]. Moira Sarsfield, and John Conway

1. What can we learn from
learning analytics?
A case study based on an analysis
of student use of video recordings
John Conway
Senior Learning Technologist
Physical Sciences
Imperial College London
Moira Sarsfield
Senior Learning Technologist
Life Sciences
Imperial College London

2. What is LA?
JISC
Learning analytics refers to the measurement,
collection, analysis and reporting of data about the
progress of learners and the contexts in which learning
takes place.
Predictive & Retention
HEA
Learning Analytics is the process of measuring and
collecting data about learners and learning with the
aim of improving teaching and learning practice
through analysis of the data.
Improving learning through better informed teaching practice

3. Background
• The analysis covers 18 UG modules
– Across Mathematics, Chemistry, Physics and Life Sciences
– For the academic year 2014-2015
– Covering year 1 and year 2 cohorts
• A module = a single block of teaching ending in an
examination
• Students = those who took the module and exam for
the first time in 2014-15
• Use of lecture capture measured by accesses and by
minutes viewed

4. • How much use is made of video recordings by
students?
• Is the use of recordings different for different:
– modules and degrees?
– groups of students?
– types of content?
• How does the use of recordings in a course vary over
time?
Key research questions

5. Factors in project design
• Ethics/privacy important when combining student
marks and grades with server access data
– Mark and grade information must remain within
departments; requires security
– Student SpLD status is ‘sensitive data’ under Data
Protection Act
– Student anonymity must be preserved – anonymisation
must be a one way process
• Flexibility and ease of use
– Excel for departmental use (known product)
– R for processing (allows automated proccessing,
standardised reporting)

6. Actionable insights leading to change in practice
Process

7. Do students use the recordings?
YES – but use varies considerably

8. Is there a pattern associated with grades?
Not in general

9. Is there a pattern associated with grades?
But, YES, when we look in detail

10. More/less use by 1st class students?
No significant difference between grades

11. More use by SpLD students?
No significant difference observed

12. More/less use by 1st class students?
Yes – where viewing is required
There is a pre-recorded Panopto lecture
for you to look at before Lecture 2 as we
will be flipping the class during the 2pm
Lecture 2 session slot.

13. How much do students generally watch?
Varies considerably by degree stream

14. Are students watching whole lectures?
Differs by degree stream and over time

15. Are students watching whole lectures?
Differs by degree stream and over time

16. Are students watching whole lectures?
Differs by degree stream and over time

17. Does the pattern of use differ by degree?
Yes – use in Life Sciences is very different

18. Timing of use in Life Sciences

19. Unexpected insights
If recordings are released
late, they are accessed
much less than those
released immediately after
the lecture.
Space in the timetable may
be needed to allow
students to consolidate
learning from one lecture
before the next.

20. Actionable insights
Advice for students
High-performing students:
• View recordings where the lecturer says it is required (e.g. a
flipped lecture).
• View recordings early, right after the lecture rather than in the
revision period.
• Maintain their application right through the course. They don’t
slack off as the end is in sight.
• May or may not use the lecture recordings; likewise poorer
students may or may not use them. Success is not directly
correlated with lecture recording viewing.

21. Actionable insights
Advice for lecturers and course/degree organisers
• Do not delay the release of your recordings – this will result in
lower usage.
• Think about how recordings are presented – should they be
given more/less/equal weight than other learning materials?
• Give advice to students on the way you expect them to use
lecture recordings.
• Consider how lectures are timetabled. Complex lecture
content requires time for students to assimilate.
• If the pattern of use of recordings is not as expected,
investigate why this is so, e.g. look at timing of lectures,
content, pattern of assessment, etc.
• You can check online which parts of each recording are being
viewed most. Training on this functionality will be provided.

22. • Investigate the reasons for the difference in use of
lecture recordings in Life Sciences
• Investigate how students attaining different grades
use the recordings
• Build on the lecture recording analysis, e.g. look at
other departments and changes in teaching
methods, e.g. implementing flipped classroom or TBL
• Apply the methodology and processes more widely
to investigate use of other learning materials, e.g.
formative quizzes in Blackboard, PeerWise.
Future Plans

23. Lessons for other LA projects
• Define what is to be analysed
• Define the data to be used
• Consider ethics/confidentiality
• Consider needs of different stakeholders
• Gather data in a structured way
• Drill down and analyse at a fine level
• Verify the results
• Automate the analysis and report writing
• Compare, analyse, make recommendations

24. Contact form
Contact Us
John Conway
Senior Learning Technologist – Physical Sciences
j.conway@imperial.ac.uk
Moira Sarsfield
Senior Learning Technologist – Life Sciences
m.sarsfield@imperial.ac.uk