Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Implementing analytics - Myles Danson, Shri Footring, David Matthews, James Foster

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Wird geladen in …3
×

Hier ansehen

1 von 39 Anzeige

Implementing analytics - Myles Danson, Shri Footring, David Matthews, James Foster

Herunterladen, um offline zu lesen

Led by Myles Danson, senior co-design manager and Shri Footring, senior co-design manager - enterprise, both Jisc.

With contributions from:

David Matthews, VLE development manager, Rose Bruford College of Theatre and Performance
James Foster, planning analyst, University of Kent

Connect more in London, 28 June 2016

Led by Myles Danson, senior co-design manager and Shri Footring, senior co-design manager - enterprise, both Jisc.

With contributions from:

David Matthews, VLE development manager, Rose Bruford College of Theatre and Performance
James Foster, planning analyst, University of Kent

Connect more in London, 28 June 2016

Anzeige
Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Andere mochten auch (20)

Anzeige

Ähnlich wie Implementing analytics - Myles Danson, Shri Footring, David Matthews, James Foster (20)

Weitere von Jisc (20)

Anzeige

Aktuellste (20)

Implementing analytics - Myles Danson, Shri Footring, David Matthews, James Foster

  1. 1. Implementing analytics LearningAnalytics and Business Intelligence
  2. 2. Session outline » Overview of the Learning Analytics service » The user voice » Overview of the Business Intelligence project » The user voice » Group exercise Learning Analytics 329/06/2016
  3. 3. Learning Analytics
  4. 4. Effective Learning Analytics Challenge Rationale » Universities and colleges wanted help to get started and have access to a standard set of tools and technologies to monitor and intervene Priorities identified » Code of Practice on legal and ethical issues » Develop a basic learning analytics service including an app for students » Provide a network to share knowledge and experience Timescale » 2015-16 -Test and develop the tools and metrics » 2016-17 -Transition to service (Freemium) » Sept 2017 – Launch. Measure impact on retention and achievement Learning Analytics 529/06/2016
  5. 5. What do we mean by Learning Analytics? » The application of big data techniques such as machine based learning and data mining to help learners and institutions meet their goals: » For our project: › Improve retention (current project) › Improve attainment (current project) › Improve employability (future project) › Personalised learning (future project) Learning Analytics 629/06/2016
  6. 6. Toolkit and community » Blog: http://analytics.jiscinvolve.org » Reports › Code of Practice for Learning Analytics › The current state of play in UK Higher and Further Education › Learning Analytics in Higher Education: A review of UK and international practice » Mailing: analytics@jiscmail.ac.uk » Network meetings Learning Analytics 729/06/2016
  7. 7. Learning Analytics Architecture Learning Analytics 829/06/2016
  8. 8. Current engagement » Expressions of interest: 85 » Engaged in activity: 35 » Discovery to Sept 16: agreed (28), completed (18), reported (17) » Learning Analytics Pre-Implementation: (12) » Learning Analytics Implementation: (7) Learning Analytics 929/06/2016
  9. 9. Future Engagement From Sept 2016 » “ReadinessToolkit” with a diagnostic set of questions and support materials leading to implementation » Start-up guidelines to get ready for learning implementation Further details will be announced via analytics @jiscmail.ac.uk Learning Analytics 1029/06/2016
  10. 10. Connect More With Jisc 2016 Embedding Learner Analytics David Matthews VLE Development Manager david.Matthews@Bruford.ac.uk
  11. 11. RBC is a small specialist HEI, focusing on all aspects of theatre/performance training.  Large online/distance learning cohorts, blended courses such as PGCTLHE, plus off- campus MA and significant study abroad elements (e.g. Erasmus and ATA).  Want to use LA especially to help those students who are not on campus for extended periods of time during the working week.  TDAP.  Heard about the project through MASHEIN – off-shoot of the Leadership Foundation – but also have a long-standing relationship with Jisc/RSC/Evan and Martin 29/06/2016 Leanring Analytics 12
  12. 12. Existing use of Learner Data  Reporting out of Moodle – (not very good or satisfactory) – and other VLE services  Data held in Registry, e.g. HESA returns, DLHE, NSS and first impressions etc.  Data held by IT and LRC, e.g. logins from College usernames etc.  Google Analytics on College corporate websites and VLE  Data collected by programmes, e.g. end of module questionnaires, surveys 29/06/2016 Leanring Analytics 13
  13. 13. Learner Analytics Project Joining the project… • Introduced to Rob Wyn Jones and Paul Bailey who have made site visits (joined by Evan Dickerson) • Involved Registry, Student Records and IT. Approval from SMC – who are very keen on the project. • Site visits and Skype/Google Hangouts meetings 29/06/2016 Leanring Analytics 14
  14. 14. Learner Analytics Project Current status  Transformation of student data into Jisc’s LA data model is now taking place(being done in close collaboration with Jisc)  Student data - LA data model is similar to HESA student return structure/ field specifications – documented online at https://github.com/jiscdev/analytics-udd/  Moodle data o Historical data has been extracted from Moodle logs on our ULCC- hosted Moodle (via internal Jisc software) o This is being push into the Jisc Learning Records Warehouse o A LIVE data plug-in for Moodle (extracting engagement data moving forwards) has been evaluated and installed on our ULCC-hosted Moodle (Learning Locker screengrab) 29/06/2016 Leanring Analytics 15
  15. 15. Anticipated Outcomes  Improved use of data across the College  Retention is already very good, but one student leaving a small cohort makes a big dent in statistics and in funding!  Enable Registry to function effectively/grow post-TDAP  Better support for online/blended students and those on placements/Erasmus visits. Better or more timely interventions  A genuine development project that, as a small institution, we would not have been able to resource or support ourselves  Excited to be part of an important project in an emerging field  Very happy to continue our working relationship with Jisc… 29/06/2016 Leanring Analytics 16
  16. 16. HESA and Jisc Business Intelligence
  17. 17. About HESA 18 Learning Analytics 1829/06/2016
  18. 18. Learning Analytics 1929/06/2016
  19. 19. Heidi Plus  The new business intelligence service for UK Higher Education  Replaces Heidi (which will be decommissioned in November 2016)  Launched in November 2015 offering:  Improved data content and functionality  Delivery of data sets through commercial data explorer tool  New visualisations and dashboards  New training programme and support materials  Available to HE institutions with a full HESA subscription  Over 80% of current Heidi subscribers have started the Heidi Plus application process (40% completed) Learning Analytics 2129/06/2016
  20. 20. Heidi Lab overview Learning Analytics 2229/06/2016
  21. 21. Secure data processing environment Technical infrastructure bound by legal agreements to ensure data and dashboards are secure Learning Analytics 2329/06/2016
  22. 22.  Information improvement manager UEL with; Kent, Middlesex, Brunel, Royal Holloway  Strategic planning and BI manager Sunderland with; Glasgow, Glasgow Caledonian, St Andrews, Sunderland  Director of planning, Kent with; Birkbeck,Cardiff, Oxford, Southampton, Southampton  Strategic Planning Manager, MMU with; Leicester, Leicester,Cambridge, Bishop Grosseteste Winter teams Learning Analytics 2429/06/2016
  23. 23.  Upskilling of staff resource across sector  Opening up of collaborative relationships across other organisations  Value, saving and efficiency gains from the creation and delivery but also the actions subsequently taken due to the insights gained across research, student, staff and estates and possibly internationally  Opening up access to disparate data sets and making sense of them in an HE context  Possible national licensing deals for paid access to data Team member experiences Learning Analytics 2529/06/2016
  24. 24. Winter team dashboards Learning Analytics 2629/06/2016
  25. 25. Dashboard: Course Market Analysis for Institutions What is it? An Overview Movie Purpose: This dashboard is designed to support a university’s strategic planner in designing course by allowing comparison across the sector. Use case: As a Strategic planner when working out which courses to teach I want to examine competition to my course offerings to ensure I target recruitment activity most effectively. Data sources: National Pupil Database: http://bit.ly/224CU8I Key Information Sets: http://bit.ly/1ZYnG5z National Pupil Database: http://bit.ly/224CU8I HESA Data What needs to be done and issues Time and Effort to Market Where there is scope for improvement: • Generally very polished • Some work on the interface required perhaps to sign-post the features • Licencing issues for league table data need to be negotiated. • Data sources would need updating each year – particularly the school data. Learning Analytics 2729/06/2016
  26. 26. Dashboard: University Finder for Students What is it? An Overview Movie Purpose: This dashboard is targeted at students who are looking for a university course to fit their needs. By needs we don't only mean course but also: cost, employability, location and entry tariff. Use case: As a student when working out which university course offers best fit my needs, I want to understand factors of relevance to me (course, cost, employability, location, cost of living, rural/urban and entry tariff) to compare and match offers to my circumstances. Data Sources: Key Information Sets: http://bit.ly/1ZYnG5z HESA Data What needs to be done and issues Time and Effort to Market This dashboard supplies a unique perspective on data and services that are already available to students. In some ways this is a crowded marked. So the unique selling point of this product would need to be promoted – that is that the data already available to students is amalgamated and drawn together to create a” wizard like app” for students to find courses. What would need to be done: • Identify appropriate vehicle for delivery • Market uniqueness of the the product • Negotiate data licences for league table data Learning Analytics 2829/06/2016
  27. 27. Dashboard: Finding Comparable Institutions What is it? An Overview Movie Purpose: This dashboard can be used to identify a university’s relative performance against a benchmark of similar institutions. Use case: As a Planning Manager I want to select similar institutions based on metrics I choose so that I can determine the best institutions to compare with my own university to understand if our performance is relatively good or bad Data Sources: HESA data from Heidi Key Information Sets: http://bit.ly/1ZYnG5z League Table Data – will require licensing What needs to be done and issues Time and Effort to Market Where there is scope for improvement: • Data – a relatively narrow data set was used for prototyping; a production version could accommodate a far more comprehensive data set. • Filters – searching and filtering could be enhanced • Licencing – Makes use of some league table data to benchmark against entry tariff. Licence for this need to be negotiated. Learning Analytics 2929/06/2016
  28. 28. BI Group exercise Learning Analytics 3029/06/2016
  29. 29.  As a: Strategic Planning Manager  When: Reviewing current course provision  I want to: Enable course/curriculum management planning to match national and local demand  So I can: Grow or at least maintain student recruitment Data Sources:  HESA student, DLHE, Award data, KIS,CUG  School/College performance data (A level results and numbers, School Age Populations Forecasts, etc.)  Labour market data from NOMIS (Employment rates, earnings, SOC, SIC… Group exercise Learning Analytics 3129/06/2016
  30. 30. Feeding back 29/06/2016 Learning Analytics 32
  31. 31. What next? 29/06/2016 Learning Analytics 33
  32. 32. Library analytics labs » Teams working on Library BI Stories at 0.2 FTE, total estimated effort 15 days from July - Oct 2016 » Both Product Owners and Sector Data Experts invited: › Product Owner from the sector to steer which stories are of interest › Sector Experts to understand what data sources are available and what is in the data › Jisc Contracted Data transformation specialist (CETIS) › JiscAgile Scrum Master andTableau User » Teams receive experience and guidance of Agile working » Option forTableau Desktop training to help with creating visualisations » Apply at http://bit.ly/jisc_library_data_labs_applications » Queries to siobhan.burke@jisc.ac.uk or myles.danson@jisc.ac.uk Learning Analytics 3429/06/2016
  33. 33. FE analytics labs » Shri- can you add an overview and description of the three clusters Learning Analytics 3529/06/2016
  34. 34. Analytics academy – a Jisc beta service - October 2016 » Business intelligence offers value, savings and efficiencies to Universities through data informed enhanced planning / decision making » Many problem spaces are commonly felt, while the data landscape to support insights is vast » Some universities have little access to good BI at all, while those with capability are often duplicating effort » There is no higher education focused CPD offer to train up BI expertise » Analytics academy addresses these problems by providing expertise and tools for analysts (planning officers and others) to identify suitable problem areas (student, staff, research, estates etc), exploring the data landscape for insights and producing interactive dashboards for the sector Learning Analytics 3629/06/2016
  35. 35. Keep in touch » business-intelligence.ac.uk » Subscribe via jiscmail.ac.uk/JISC-HESA-BUSINESS-INTEL » Twitter @HESA @jisc #hesajiscbi Learning Analytics 3729/06/2016
  36. 36. Dashboard: University Research Benchmarking What is it? An Overview Movie Purpose: This dashboard is designed to answer a range of questions around the university’s research profile and potential vulnerability/strength. Use case: As a Research Planning Officer, when influencing research policies, I want to assess individual cost centres' relative research strengths against national/mission group norms; so that I can help support the financial sustainability of the institution. Data Sources HESA data from Heidi Key Information Sets: http://bit.ly/1ZYnG5z What needs to be done and issues Time and Effort to Market The research dashboard was a proof of concept using the limited research data that was available to the team. Identifying further data sources would make this into a powerful tool. • Low level HESA data on research • Research Grant Data from • Funding Councils • EU • Other • Ref Data Learning Analytics 3829/06/2016
  37. 37. jisc.ac.uk Myles Danson Senior co-design manager Shri Footring Co-design manager for enterprise Myles.Danson@jisc.ac.uk Shri.Footring@jisc.ac.uk 29/06/2016 Learning Analytics 39

Hinweis der Redaktion

  • Overview of LA service – Shri (5 mins)
    User voice – David Matthews (10 mins)
    Overview of BI - Myles (5 mins)
    The user voice - James (10 mins)
    Group exercise - Shri (20 mins)
    What’s coming next - Shri and Myles (5 mins)
  • The effective learning analytics challenge was initiated from consultation with stakeholders, senior manager and practitioners who felt the sector need support to get up to speed with learning analytics. They prioritised three main areas, a Code of Practice to address legal and ethical issues of using learning analytics; a set of basic learning analytics tools to allow institutions to get started and make informed decisions; and a network to allow institutions to share practice and learn from each other.
    The current project has procured suppliers to provide a learning analytics service which are currently being tested by several institutions. This will be developed into a full service next year and provided as a new Jisc service from Sept 2017.
  • What do we mean by learning analytics. The service we are developing will collect data and undertake statistical analysis of historical and current data derived from the learning process to create models that allow for predictions that can be used to improve learning outcomes.
    Models are developed by “mining” large amounts of data to find hidden patterns that correlate to specific outcomes
    E.g. Mine VLE event data to find usage patterns that correlate to course grades
    The service will provide predictive models initially for retention (identify students at risk of failing) and attainment (identifying students at risk of not achieving a specified level of attainment).
    In the future we will look to offer predictive models to support employability and personal/adaptive learning.
  • The project consists of the learning analytics architecture (next slide), a toolkit and community.
    These consist of a blog with reports and information to assist institutions with readiness to implement learning analytics and technical implementation of the Jisc service.
    There are three reports all linked from the blog a Code of Practice for Learning Analytics, A report from 18 months ago that reviewed current state of learning analytics in the UK and a more recent report on the evidence base for the effectiveness of learning analytics with 12 international case studies.
    If you want to be involved and keep informed about the development of the service then join the analytics jiscmail list
    We also hold quarterly network meetings which are promoted via the blog and jiscmail list
  • Overview of learning analytics architecture.
    Red items are components that will include the tools in the project (Tribal student insight, Unicon/Apereo LAP and Student Success Plan, Student App) but also alternative third party or institutional tools.
  • We have ~400 people on the Jiscmail list and a pipeline of interested institution's (50+ HE, 20+FE). We are actively engaging with 35 institutions, 28 in discovery institutional readiness and 12 in beta implementations.
  • From Sept 16 we’ll be introducing a new institutional readiness process to help institutions get ready for implementing learning analytics. This will consist of an overview workshop to introduce the service and an diagnostic assessment tool, institutions will complete the assessment tool and then undertake appropriate actions to address recommendations.
    For institutions who are ready to start implementation there will be set of guidelines to get set-up with data collection and visualisations, ready to implement a predictive analytics solution and the student app.
    Details will be announced via the jiscmail list – so join it to participate.
  • Myles
    Jisc and HESA are collaborating to develop new national shared services for business intelligence, making better use of the national data landscape, reducing repetitive activities across universities, brining the benefits of BI to all Univerisits regardless of capability / expertise
  • Myles
    HESA is a not for profit subscription organisation, so similar to Jisc in that sense. As well as a mandatory subscription, members are mandated to provide data collections covering the broad themes of Student, Staff, Destinations (of graduates) and Estates data. This is annual but in year collection is under consideration. HESA cleanse the data and provide back full data sets, published statistics and undertake bespoke analysis. Jisc and HESA membership is similar.
  • Myles
    Heidi Plus is depicted on the left – highlight the trucks driving in to the HESA data warehouse. HESA mandates that all publicly funded HEPs provide performance data on students, destinations of leavers, staff, finance and estates. Currently an annual collection they are moving to more frequent in year collections. The data is cleansed and a new team undertake dashboard development. Quality is assured as the dashboards are offered throught the radio mast in the middle – a new national BI dashboard delivery service offered to all HESA customers (currently 180 HEPs and associated organisations and departments). Built with Jisc and launched as a HESA service in November. Includes legal framework and national training programme. Replaces a system with 6.5K users. Lowers the bar to usage through the interactive dashboards so could take BI to a woder range of staff than is currently possible.
    Heidi Lab is depicted to the right. A Jisc led alpha July 15 – July 16. Highlight the trucks again and note it’s a two way street – a data sharing agreement allows HESA data into the Heidi Lab secure data processing environment. Agile analysis teams are created from multiple universities and given access. They identify commonly felt problems spaces, explore the wider national data landscape, acquire non-HESA data and cleanse, link and transform it creating new proof of concept dashboards. Highlight the trucksa driving from the Lab to the Radio Mast. Successful dashboards will be branbded produced by Jisc and delivered via Heidi Plus.
    Piece in the middle is the beta service – what comes next – Heidi Plus is sustained by HESA as a service. We have proved there is real merit in Heidi Labs and will launch a beta service July 16 – July 17.
  • James
  • James
    HESA’s current data delivery service is known as HEIDI (Higher Education Information Database for Institutions) developed in house in 2007. Jisc and HESA collaborated to replace this with a more up to date service. We procured Tableau, market leading data exploration software and now offer Heidi Plus
    Feedback has been extremely good across the sector

  • Myles
    Heidi Lab as a Jisc Alpha project (proof of concept) engaged with 290 individuals from 130 universities to develop a successful model of agile analysis. 50 analysts (planners, directors of planning from 44 universities volunteered to join cross institutional agile analysis teams for three Heidi Lab cycles of 3 months each at just 0.2 FTE. Teams were supported as they identified and refined widely felt problem areas (see example on the slide – covered student, staff, research, estates etc) linked to national policy. They explored the data landscape for supportive insights, recording the issues encountered in our data catalogue. Finally they produced interactive dashboards using Tableau software as proofs of concept to offer through Heidi Plus
  • Myles


  • Led by a senior staff member with knowledge of the information needs of a wide range of staff and institutions as well as national policy and what is ‘up stream’
  • James – adjust to suit your own experiences
  • James to lead
  • James to continue with these (as many as time permits)
  • James to continue with these (as many as time permits)
  • James to continue with these (as many as time permits)
  • Shri or James to lead?
  • Nominate a person from each table to feed back on a user story, explain we will photograph them all and feed into further cycles for consideration
  • Myles – just to note we are running a set of teams from the library area to prove the concept transfers
  • Shri – very brief note to say we are running a College Labs experiment
  • Myles – a new Jisc offer to explode whether there is a sustainable service in this
  • In case anyone wants to see anther dashboard after the session

×