Presentation to Data Matters conference on the 16th Jan 2019, entitled 'Using data to support the student digital experience'. Also included presentations by Marc Griffiths of LSBU and Marieke Guy at RAU
Defining Constituents, Data Vizzes and Telling a Data Story
2019 01 16 data matters - v6 - Using data to support the student digital experience
1. Data Matters conference 16th Jan 2019
Using data to support the student digital experience
Ruth Drysdale and Mark Langer-Crame (Jisc), Marieke Guy and Alex Norris
(RAU) Marc Griffith (LSBU)
2. AGENDA
14:00 Introduction: Ruth Drysdale and Mark Langer-Crame (Jisc)
14:10 Case study 1 Marieke Guy (LearningTechnologist) and Alex
Norris (Student) Royal Agricultural University
14:25 Case study 2 Marc Griffith (Head of Digitally Enhanced
Learning) London South Bank University
14:40 Groups of 6-8 What data sources do you use?
14:55 Plenary and close
3. 1. In the next 20 years, 90% of jobs
need digital skills
2. Significant investment in technology
3. Expectations from students that
universities will offer an authentic and
relevant learning environment
4. To maintain a competitive advantage
Why do we need to know how
students are using technology?
4. Not just ‘another survey…’
https://www.jisc.ac.uk/digital-experience-insights animation
https://digitalinsights.jisc.ac.uk
5. 1. Student and teaching staff surveys with:
- Closed questions that can be benchmarked
- Open questions for local analysis
- Can add institutional questions
2. A student and staff engagement process,
governed by our guidance
3. A community of practice around the
insights process and findings (including
student representatives)
4.Compare student feedback with teaching
staff views and organisational factors
What is digital experience insights?
6. What are the ‘student and staff digital experiences’?
7. UK student insights 2018
report
At a glance: summary of 2018 insights
findings: ji.sc/dig-exp-insights-summary-18
37,720 students surveyed
across 83 institutions
8. UK teaching staff insights
2018 pilot report
Digital experience insights survey 2018: findings
from the pilot of teaching staff in UK further and
higher education
1,921 teachers surveyed
across 15 institutions
9. A ‘taster’ of the main findings in
the student and staff insights
reports…
10. 3%
25%
46%
20%
4%
1% 1%3%
39%
45%
10%
2% 0% 0%
Best
imaginable
Excellent Good Average Poor Awful Worst
imaginable
FE students
HE students
1%
9%
35%
32%
18%
5%
1%0%
11%
41%
33%
13%
2%
1%
Best
imaginable
Excellent Good Average Poor Awful Worst
imaginable
College
University
Overall teaching staff are less satisfied than students…
… when it comes
to teaching
spaces, software,
and the digital
environment
overall
Students
Teaching Staff
11. Teaching staff want more digital technology
within teaching
65.1%
31.4%
3.5%
61.1%
33.5%
5.4%
More than they
are now
Same as they are
now
Less than they are
now
College University
32%
63%
5%
37%
60%
4%
More than they are
now
Same as they are
now
Less than they are
now
FE students HE students
Students Teaching staff
12. Teaching staff rate their digital CPD most commonly as
average to good – room for improvement?
2%
11%
32%
34%
16%
4%
1%1%
8%
29%
37%
21%
4%
1%
Best
imaginable
Excellent Good Average Poor Awful Worst
imaginable
How would you rate the support you receive from your organisation
to develop the digital aspects of your role?
College
University
13. Key findings/opportunities
•Continued investment in digital technology CPD for teachers
•Provide time, recognition and rewards for teachers who want
to use more digital technology
There are issues that relate to the digital environment, eg:
• the effective use of VLEs
• Well designed teaching rooms
• access to up-to-date and industry standard software
14. Difference in perception in relation to student online
safety within HE between students and teaching staff
53%
39%
8%
41%
50%
8%
Agree Neutral Disagree
My institution helps me stay safe online
FE students HE students
67%
29%
4%
16%
39%
45%
Agree Neutral Disagree
I am informed about my
responsibilities with regard to ensuring
students behave safely online
College University
Students Teaching staff
15. • See https://digitalinsights.jisc.ac.uk
• 2019 insight surveys for teaching staff and
students
• Case studies
https://digitalinsights.jisc.ac.uk/case-study-
listing/
• Full guidance on carrying out surveys and
analysis https://digitalinsights.jisc.ac.uk/our-
service/advice-and-guidance/
• Join the insights community of practice:
jiscmail.ac.uk/jisc-digitalinsights-cop
Guidance and support
16. Get in touch…
Except where otherwise noted,
this work is licensed under CC-BY
Ruth Drysdale
Ruth.Drysdale@jisc.ac.uk
and
Mark Langer-Crame
Mark.Langer-Crame@jisc.ac.uk
https://digitalinsights.jisc.ac.uk
#digitalstudent
17. AGENDA
14:00 Introduction: Ruth Drysdale and Mark Langer-Crame (Jisc)
14:10 Case study 1 Marieke Guy (LearningTechnologist) and Alex
Norris (Student) Royal Agricultural University
14:25 Case study 2 Marc Griffith (Head of Digitally Enhanced
Learning) London South Bank University
14:40 Groups of 6-8 What data sources do you use?
14:55 Plenary and close
18. Data matters conference, London, 16th January 2019 Marieke Guy (staff) and Alex Norris (student)
The RAU
Student digital
experience tracker 2018
21. The Royal Agricultural University
• At the forefront of agricultural education for more than 170
years
• Around 1,200 students – undergraduate, postgraduate,
international
• Courses cover agriculture, animal science, business,
environment, equine science, farm management, food, real
estate and rural land management
• New blended learning courses being developed
• Have digital aspirations - https://digitalrau.wordpress.com/
22. • January 2018 - tracker working group established (staff
& students)
• IT survey had been run in past
• April 2018 - Tracker run for the first time
• All students in all year groups asked
• Students sent the link by email, also advertised on
Facebook, Twitter and through internal news
mechanisms
• Poster campaign across campus
• Students entered into prize draw to win £50 Amazon
voucher
• Weekly update emails sent to students and to staff
indicating response rate by programme
Running the ‘tracker’
23. • 218 of our students responded to the tracker (18% response rate)
• Gender split (Q2)
• Male (44%)
• Female (56%)
• Stages of study (Q3)
• 1st year (56%)
• Middle year (15%)
• Final year (18%)
• Masters/postgrad (10%)
• 20% self-identify as needing to use assistive technologies (Q6)
Student sample
24. Tracker results
59% rate the quality of digital teaching and
learning on their course as good or above
(Q18)
34% agree their course prepares them for the
digital workplace (Q17d)
50% agree software used on their course is
industry standard and up-to-date (Q16c)
• 74% rate the quality of our digital provision
(software, hardware, learning environment)
as good or above (Q13)
• 69% agree we support them to use their
own digital devices (Q9a)
• 79% can access reliable WIFI whenever
they need it (Q7)
74%
69%
79%
59%
34%
50%
Quality of
digital
provision
Own device
support
Reliable
WIFI
Quality of
digital
teaching…
Preparation
for digital
workplace
Up-to-date
software
25. • The amount of technology available at RAU is fine – students want us to
make it more user-friendly and get better at training staff and students in
how to use it
• Wifi is very important to students but they actually think our coverage is OK
• Students like consistency – on Gateway (VLE), in lessons, in tools
• Students want us to prioritise online, free, up-to-date resources
• Students don’t feel they are getting the right amount of digital training in
tools they need for their course or in the skills they need for the work place
• Students like computer rooms, printers and charging points
• Students want more multimedia - videos (of lectures) and images
• Students appreciate IT support and would like more help with technical
issues
Key findings
26. Make sure everyone knows what software is available to
support them and how to use it to its full potential. Ensure
that the dyslexia disability support department is more
switched on about the technology available and ensure
students who do not know what is available for them can be
supported to find technology based solutions to their
problems.
To improve your experience of digital teaching and learning ... what one thing should we DO?
Lecturers should improve
the quality of the resources
put on Gateway, as
although some are good,
often they are very out of
date and re-used.
Step by step guides and more coverage at the
beginning of the year as it was rushed and I
found myself learning to use as I went along.
Lecturers should assume that part time
post graduate students may not be
able to attend lectures. Therefore
lecture content must be
comprehensive allowing students like
myself to choose whether or not to
drive 3 hours to attend lectures.
27. • Data analysed and shared with working
group, management and academics
• Action plan created & shared with relevant
people
• 31 page PDF shared with students (as
document and as ‘you said, we did/we
will/an explanation’ posters)
• Covered digital literacy, accessibility, digital
spaces, Gateway, IT support, learning and
teaching, library and mobile
• Blog post on SU website and Digital
transformation blog
Sharing the results
29. Next steps
• Plan to run the tracker again
• Want to add in additional questions on digital skills, RAU
Resource lists and app
• Improved internal communications – will integrate with app
• New GDPR requirements – need to rethink prize situation
• Possible eChamps work
• Understanding expectations and experiences of digital
should become a more integrated part of student voice
cycle
31. AGENDA
14:00 Introduction: Ruth Drysdale and Mark Langer-Crame (Jisc)
14:10 Case study 1 Marieke Guy (LearningTechnologist) and Alex
Norris (Student) Royal Agricultural University
14:25 Case study 2 Marc Griffith (Head of Digitally Enhanced
Learning) London South Bank University
14:40 Groups of 6-8 What data sources do you use?
14:55 Plenary and close
33. Background
• My role focuses on developing the university’s learning and teaching environments
– encompassing both physical and digital
• Limited initial data for developing a comprehensive digital strategy
• The student digital experience tracker pilot (student digital experience insights)
provided a tool for gathering the data required
34. Why the student digital experience insights?
The student digital experience insights pilot promised three key outcomes:
1. Establish the current digital experiences of students
2. Identify key areas for development and investment
3. Benchmark against the sector
35. What we did
• Established working group
• Scheduled its delivery (over the Christmas period)
• Advertised, made it available and offered a gift via draw
• Launched it via the VLE, intranet, direct email and posters in the Library and
Learning Resource Centre
• Engaged academic staff to encourage students to complete it
36. Analysing, interpreting and acting
• Do not underestimate the work involved to fully understand the data and report
findings
• Be selective in what you present to different audiences
• Drilling down and mapping data to other sources has proven difficult
• Decide what you want to do based on findings in advance – e.g. where to report
findings and closing the loop
37. Key findings
• The quality of digital teaching and learning compared
favourably with other UK HE institutions (69% rated good or
above)
• Variability in the use of the digital on courses is seen
between schools which requires further investigation
• Students are using a wide range of digital tools and devices
to support their learning (i.e. not only what the institution
provides) – youTube, Google, Lynda
• More needs to be done to address student views about how
well courses are preparing them for the digital workplace
(46% well prepared)
• Across the sector more needs to be done to provide support
for student owned devices if they are increasingly expected
to use them for learning
38. Biggest surprise
• Given the general perception of the proliferation of smartphones it is quite revealing
that a quarter of students report not using a personally owned phone for learning
• 76% at LSBU versus 80% national benchmark data
39. Key outcomes
1. Informed investment in lecture capture for the institution (in pilot)
2. Focused discussion / development around our digital ecosystem to incorporate
more industry standard tools (oneNote and teams)
3. Informed the development of a new institutional U/I for the VLE and a new module
template
4. Incorporated into the development of an institution wide digital strategy that looks
holistically at out digital environment
40. AGENDA
14:00 Introduction: Ruth Drysdale and Mark Langer-Crame (Jisc)
14:10 Case study 1 Marieke Guy (LearningTechnologist) and Alex
Norris (Student) Royal Agricultural University
14:25 Case study 2 Marc Griffith (Head of Digitally Enhanced
Learning) London South Bank University
14:40 Groups of 6-8 What data sources do you use?
14:55 Plenary and close
41. In small groups, nominate a spokes person
to feedback in the plenary, then discuss;
What data sources do you use to
evaluate the use of digital technologies
with students and teaching staff?
42. AGENDA
14:00 Introduction: Ruth Drysdale and Mark Langer-Crame (Jisc)
14:10 Case study 1 Marieke Guy (LearningTechnologist) and Alex
Norris (Student) Royal Agricultural University
14:25 Case study 2 Marc Griffith (Head of Digitally Enhanced
Learning) London South Bank University
14:40 Groups of 6-8 What data sources do you use?
14:55 Plenary and close
Hinweis der Redaktion
How are HE and FE students using technology?
Ruth to introduce and present slides 1 - 6
As an outcome of the digital student work and the need to gather quantitative data on students digital experience at an organisational level and at a sector level, we developed the student digital experience tracker as a survey tool with a robust set of student tested questions delivered in BOS. See http://bit.ly/jiscdigidataservice
This evidence supports discussions with senior managers
The report containing the summary findings from 2017 surveys will be available from 20th June from web link on this slide.
As student all we want is easily navigated and simple to use technology. Pretty much a system that does what it says on the tin.
The RAU does this by simplistic means of having a easily navigated and simplistic website called Gateway, this allows for students to find information themselves. The System incorporates the use of Module pages this is where course and module specific information can be found, including the PPTs for lectures, coursework briefs as well extra reading. This means that as students we don’t have to chase our tails around looking for information, and that the lectures can easily communicate with us. The University also has a policy of realising PPTs the day before this allows the students to assimilate the information before it is delivered, it also allows for easier note taking, especially for myself as I have DSA’s which allow for recordings, this coupled with the PPTs means I have exceptionally comprehensive notes.
Due to the RAU’s courses being rather practical the use of professional software is essential to prepare students for the job roles they’ll be undertaking. For myself the RAU allows me to have access to Promap, mapping software, Metropix for creating floorplans and Argus a valuation software. The lecturers do teach us how to use these software's however an aspect the RAU could improve is by educating students on how to use basic word and excel skills. As during some course works it was trail and error as well as the long route to the answer.
The Library also has extensive online resources and the access to them anywhere in the world is great, as I have written coursework in a multitude of random places including on the English Channel. The Ability to reserve as well as read books also means that you can quickly and efficient gather resources, the variety of course specific material as well as general knowledge is great and allows for extra reading in many spheres.
Overall the centralised system the RAU uses means that the Student IT experience is great for a IT Idiot like myself, icing on the cake is the contact you can have with IT if you don’t understand or cant find any information and their long opening hours and range of communication methods means that there is always an answer.
Joined 3 years ago
Limited data of staff and student digital use
Existing data was anecdotal and based on individual’s perception and difficult to assess which parts of the infrastructure needed most work
We need a more concrete starting point on which to prioritise development and investment decisions
I needed a reliable mechanism to hold discussions with our Executive and other stakeholders about where we were institutionally, what investment was needed and in what.
For example, a common refrain was how far behind the rest of the sector we were in relation to digital learning platforms, which coming in to the role did not resonate
Instead like other institutions there appeared to be areas where we were quite advanced and others where we lagged behind
To get this moving we established a small working group involving
Myself
Head of Library and Learning resources
Vice president (academic) from the SU
Scheduling is crucial and we had to find a window where it did not overlap with other institutional surveys such as our MEQ period of NSS – over xmas the library and its services are quite heavily used as students get ready for exams in the new year
The SU ran an effective social media campaign to encourage engagement
The community was invaluable and saved us significant time by sharing their experiences and even their promotional materials that we were able to modify for our use
We kept it quite low key but were hoping to get about 1000 responses
Do not under estimate the work involved in analysing and interpreting – we were lucky in that our Business Intelligence team were on board and interested in the data the outcomes
Our BI team produced a comprehensive 112 page report (including appendices), however the JISC report template does save a lot of time and highlights key focus areas
Given our cohort we were particularly interested in establish any patterns from:
Schools
Age
Gender
69% rated the overall of digital teaching and learning on their course as good or above – an interested pattern that emerged was that this rating declined as students progressed through the stages
Analysis of the responses shows that a high incidence of content focused digital tools such as YouTube, Google and Lynda.com were cited. The university currently invests quite a lot of time on content creation and this data may indicate that a shift to content curation may be better use of finite resources
46% of students felt well prepared for a digital workplace. While this compares favourably with the national benchmark of 41%, preparing students for employment is a key driver for the university
63% agree we support them to use their own digital devices (Q9a)