Learning Analytics for Student Support

Innovate • Design • Enhance
Analytics are everywhere

“Analytics for Action – enabling
in-presentation evidence-based change
on Open University modules”
15 September 2017
• Rafael Hidalgo - Senior TEL Designer
LTI – The Open University

The Open University context
• The UK's largest university (approximately 125,000 students)
– 40% of part-time undergraduates in the UK study with us
• 180 qualifications and 600 modules – most of our
undergraduate courses have no formal entry requirements
• 76% of students study alongside (full or part-time) work
• 20,000+ people with one or more declared disabilities study
with us each year – probably more than any other university
in Europe

What is Analytics for Action (A4A)?
• A4A is a project to enable evidence-based change in presentation –
Active Presentation
• It started as part of the Learning and Teaching Centre’s Analytics Project
(2014-2016), encompassing:
• Real-time Student Feedback, Student progression reporting, SAS
Visual Analytics, Ethics, Predictive modelling (OU Analyse) and
Analytics 4 Action
• TEL Design provided a natural long-term home for Analytics for Action .
• A4A is run by TEL LTI since 2016/2017- Data is business as usual

“Using the data as
prompts, module teams
can initiate a range of
response actions,
including pedagogic
change and student
support”
- Bart Rienties, IET,
The Open University

Analytics for Action Framework

The Active Presentation Toolkit
• Based on the A4A
framework
• Available to all staff
in the Open
University intranet
• Step-by-step
instructions for
running an active
presentation

The Active Presentation Toolkit
• Staged approach

What support is available?
For all modules
• Toolkit provides for a DIY approach
• Training enables staff to get started with this,
• Surgery support at Learning Systems drop-ins
• Mailbox support: LTI-TEL-Design@open.ac.uk
For selected modules:
• Data support meetings: 3 per presentation

Which modules receive full support?
• Agreed with Faculties
• Modules identified as underperforming by the AQR – Z scores
• High student population
• Pedagogic challenges and/or innovations
• Qualifications needs
• Faculty strategic interest

Training
• 2 hours hands-on training sessions run by Senior TEL Designers, using
real and current data.
• Priority places for MTC’s and CM’s of modules selected for formal support
• Main focus: Active Presentation Toolkit and Student Progression
Dashboards
• Small groups to allow enough discussion and specific questions

Data support meetings
• Senior TEL Designers will support the relevant faculty modules
• Academics provide in-depth knowledge of the module,
• STELDS provide wider view as they are exposed to other data,
• Together we agree on the issues to be investigated,
• STELDs suggest actions and help in the implementation (this may
involve other units)
• Comprehensive reports on discussions and agreed actions,
• Regular updates to the faculty stakeholders about progress

What sort of data do we look at support meetings?
Student progression dashboards
• Student profile
• Concurrency
• Assessment submission rates
• Withdrawer profiles
• Online engagement
• Online tools and resource
usage
• Retention
• Pass and retention rates

Student profiles and concurrency
• 267 students registered at
FLP25
• Roughly half registered for
Q72 and a quarter for Open
Degree
• 66% male, most common
age group is 30-39 (37%)
• 16.5% have declared a
disability
• 73.8% were studying at least
another module, mostly
T313, with 38 students
studying 120 credits
• 79% are employed part or
full time
T219
• 187 students registered at
FLP25
• Q72 is the most popular
qualification (30%)
• 64% male, most common age
group 30-39 (37%)
• 11% have declared a disability
• 55% were studying at least
one other module with 20
students studying 120 credits
• 87% are employed part or full
time
T319

Assessment Submission Rates
93,1
87,0
96,5
83,6
94,9
62,1
55,2 63,9
65,1 59,6
0,0
10,0
20,0
30,0
40,0
50,0
60,0
70,0
80,0
90,0
100,0
S104 16J SDK100 U116 S111 S10415J
% of TMA Submissions - Level 1 Science
TMA01 TMA02 TMA03 TMA04 TMA05 TMA06

Assessment Submission Rates
87,3
70,9
59,3
57,1
87,6
76
60,3
0
10
20
30
40
50
60
70
80
90
100
TMA01 TMA02 TMA03 Project
% of TMA Submissions - T219 – by year
13J 14J 15J 16J

Retention
80
82
84
86
88
90
92
94
17 18 19 20 21 22 23 24 25
% Retention
T219 T213 DST206 AVE

Online engagement
VLE use for T219 in 16J

Use of online resources
Use of Block 1 Forum in T319

Completion and Pass rates

Data insight: Conversations and actions
• “Significant numbers of students are studying other modules concurrently. The module team
don’t advise on multiple module study.”
 Student Support Teams need to be made aware of significant number of students taking
120 points at once. A faculty discussion is also needed at qualification level.
• Are extensions any good?
• Is the gap between TMA02 and Project too long?
TMA02 weight and timing needs to be assessed
• More support from the tutor at the start of collaborative work
 Module Team to consider an intro collaborative activity
• Tutors should be trained in handling the forums, are they?

Data insight: Conversations and actions
Are collaborative activities really an issue?
Pass: Module v- Faculty Average Difference
12J av 13J av 14J av 15J av 16J av 12J 13J 14J 15J 16J
T219 61% 64% 60% 63% 51% 62% -3 -3 -11
T319 67% 69% 69% 68% -2 1
T215 74% 64% 72% 63% 73% 62% 10 11 11
TD223 58% 65% 58% 67% 61% 68% -7 -9 -7
U101 71% 65% 63% 65% 63% 64% 57% 63% 6 -2 -1 -6
B205 n/a
H817 (B)* 86% 79% 88% 79% 90% 79% 84% 76% 13 11 11 12
W302 n/a
DD206 80% 67% 78% 65% 71% 67% 73% 68% 13 13 4 5
DD313 78% 75% 76% 74% 3 2
E207 80% 68% 82% 68% 78% 68% 75% 69% 12 14 100 6

Data Support Meetings Reports

What can we do about it?
• Based on our findings we could decide:
• A change to the materials
• Some additional support to everyone
• Some targeted support
• Improvements upstream to get the right people onto the module

What can
we do
about it?
http://article.iet.open.ac.uk/Q/QE%20report%20series/2015-QE-
learning-analytics.pdf

Success examples
KJY113 16B: Improved pass rates
• Registered students dropped 12-13 %
from start to 25FLP. Pass rates under
faculty average for 3 years
• Additional information via dedicated
website and A5 postcard
• AL’s contacted students at risk
• Pass rate up 12%. Completion rate up
10%. TMA submissions improved. Drop
between start and FLP25 was only 3%

Success examples
S350 16J: Managing extensions
• Concurrency levels very high: 73.4%
• 75% of students were employed
• Students unexpectedly taking Sxx390
• AL’s were given list of high workload
students and their TMA dates for
concurrent modules, to manage
extensions.
• No historical data to compare but early
intervention by AL’s was enabled by
data support meeting

Success examples
T192 16J: design to evaluation
• In the design, Module Team addressed
a key mathematical concept –
“rearranging equations”-
• TEL Design helped to set up an
adaptive learning package of iCMAs
with support built-in via the website and
module tutors.
• Monitoring the data allowed to provide
the module team chair with the
information required to decide whether
to take action on these individual
quizzes.
-10
0
10
20
30
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16 18 20
Number of students with scores under 40% and 50%
Scores < 40 Scores < 50

Analytics for action: Data is business as usual
In 2016/17
• 29 OU modules fully supported – reaching almost 25,000
students
• Over 100 staff trained in 23 training sessions
• 75 data support meetings held
• Coverage expected to grow in 2017/18

Analytics for action: The future
For 2017/18: growing numbers and quality
• Up to 49 OU modules fully supported
• Improved tools and systems
• More than 120 staff to be trained and over 100 data
support meetings to be held
• Increased scholarship

Analytics for action: What are they saying?
About Data Training Tool sessions:
“The initial question I had was answered and I found out other useful things also”
“The trainers were able to support individual learners with their unique issues”
“It was informative and with help easy to navigate”
“Hands on appropriate support and guidance using real live data”
About Data Support Meetings:
“Being able to go “off grid” and actually – finally! – get some insights and answers
into presentation questions we have had for ages”
“Thank you that gives us something to think about for the future presentations”
“Really useful, and I’ll follow things up here”
“I was updating VB on the great work that has been happening with T219/T319”

The Open University
LTI TEL Design team
Level 7 Perry B
lti-tel-design@open.ac.uk
@TEL_Design_OU
Get in touch!

Learning Analytics for Student Support

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (19)

Ähnlich wie Learning Analytics for Student Support

Ähnlich wie Learning Analytics for Student Support (20)

Mehr von EADTU

Mehr von EADTU (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Learning Analytics for Student Support