Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Learning dashboard for supporting students: from first-year engineering to MOOC students

40 Aufrufe

Veröffentlicht am

Presentation used during the SEFI 2018 workshop in Copenhagen, Denmark.

Veröffentlicht in: Bildung
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Learning dashboard for supporting students: from first-year engineering to MOOC students

  1. 1. Learning dashboard for supporting students: from first-year engineering to MOOC students Tom Broos & Tinne De Laet Tom.Broos@kuleuven.be @TomBroos Tinne.DeLaet@kuleuven.be @TinneDeLaet
  2. 2. “Learning analytics is about collecting traces that learners leave behind and using those traces to improve learning.” - Erik Duval Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012, https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/ 2 Learning Analytics?
  3. 3. Learning Dashboards? 3Dashboard Confusion, Stephen Few, Intelligent Enterprise, March 20, 2004 “A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.” - Stephen Few
  4. 4. Two (external) examples 4
  5. 5. Successful Transition from secondary to higher Education using Learning Analytics enhance a successful transition from secondary to higher education by means of learning analytics  design and build analytics dashboards,  dashboards that go beyond identifying at-risk students, allowing actionable feedback for all students on a large scale. 10 www.stela-project.eu @STELA_project 562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
  6. 6. STELA 11 actionable feedback student-centered program level inclusive first-year experience institution-wide Learning Analytics actual implementation
  7. 7. [!] Feedback must be “actionable”. 12 Warning! Male students have 10% less probability to be successful. You are male. Warning! Your online activity is lagging behind. action? ? action? ? 
  8. 8. [!] Start with the available data. Lots of data may eventually become available in the future … …. already start with what is available 13 (*) (*) Zarraonandia, T., Aedo, I., Díaz, P., & Montero, A. (2013). An augmented lecture feedback system to support learner and teacher communication. British Journal of Educational Technology, 44(4), 616-628.
  9. 9. [!] Involve stakeholder expertise. 15 visualization experts practitioners / end-users researchers LA researchers first-year study success Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue. In IEEE Transactions on Learning Technology (http://ieeexplore.ieee.org/document/7959628/).
  10. 10. Student-facing learning dashboards REX – LASSI – Learning Tracker
  11. 11. founded 1425 research-intensive university 16 faculties → general university > 55 000 students 2016-2017-2018 Europe’s most innovative university (Reuters ranking)
  12. 12. learning dashboards @KU Leuven interaction self-reflection STUDENT ADVISOR STUDENT LASSI – learning skills REX - scoresPOS – future students
  13. 13. 20 - MOOCLearning Tracker
  14. 14. Explore one of the dashboards (10 min) 21 inspiration https://learningdashboards.eu/ Learning Tracker Be ready to explain what the dashboard is doing in 2 minutes! (next round)
  15. 15. Explain what the dashboard is doing in your group (10 minutes) 22 inspiration https://learningdashboards.eu/ Learning Tracker
  16. 16. [!] Start with the available data. 23 data already available? administrative (examples) student records course grades systems (examples) LMS access logs advisor meetings ) Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Small data as a conversation starter for learning analytics: exam results dashboard for first-year students in higher education. Journal of Research in Innovative Teaching & Learning, , 1-14. demo: https://learninganalytics.set.kuleuven.be/static-demo-rex/ (en) or https://learninganalytics.set.kuleuven.be/demo/rex-1718jan-ir (nl)
  17. 17. Stakeholder involvement: example 2 24
  18. 18. Stakeholder involvement: example 2 25
  19. 19. Stakeholder involvement: example 2 26 Student is starter? Study efficienc y? Mail invitation Student visits dashboard Reminder LOW GLOBAL SE TIPS REGULATIONS MID GLOBAL SE HIGH GLOBAL SE ADVICE INTRODUCTION MAIL REMINDER SCORE
  20. 20. Stakeholder involvement: example 2 27
  21. 21. Evaluation - REX 28
  22. 22. Evaluation - REX 29 571 76 245 220 353 258 274 1060 187 283 843 242 163 38 130 120 200 151 162 647 128 230 771 269 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% clicked non clicked
  23. 23. What can we learn from dashboard usage? 30Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Low-investment, Realistic-Return Business Cases for Learning Analytics Dashboards: Leveraging Usage Data and Microinteractions. accepted for ECTEL 2018 p<1e-5 p<1e-9p-test
  24. 24. Dashboard learning skills - LASSI 31 students complete LASSI questionnaire students received personalized email with invitation for dashboard 4367 students in 26 programs in 9 faculties @KU Leuven demo: https://learninganalytics.set.kuleuven.be/lassi-1718/ (KU Leuven login) 2 programs @TU Delft
  25. 25. Response 32 3868 students (89%) used dashboard
  26. 26. Student feedback? 33 http://blog.associatie.kuleuven.be/tinnedelaet/learning-dashboard-for-actionable-feedback-on-learning-and-studying-skills/ How CLEAR is this info? stars stars
  27. 27. Students that click through Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness. In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham. 34  better learning skills
  28. 28. More intense users Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness. In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham. 35  worse learning skills
  29. 29. [!] Acceptance precedes impact. 36 • Involve stakeholders from the start and value their input! COmmunication COoperation • Demonstrate usefulness. • Take care of ethics and privacy. • Best scenario: students & study advisors as ambassadors COCO
  30. 30. Impact? survey before intervention  2nd year students 2016-2017  experiences first-year feedback  41 vragen, 5-point Likert scale  pen & paper dashboards  LISSA  LASSI (learning skills)  3 x REX (grades) Survey after intervention  2nd year students 2017-2018
  31. 31. Impact? During the first year I received sufficient information regarding my academic achievements. 38 Engineering Science (p<0.001)
  32. 32. Impact? The information I received helped to position myself with respect to my peers. 39 Engineering Science (p<0.001)
  33. 33. Impact? 40 The information I received made me reflect. The information I received made me adapt my behaviour.
  34. 34. Learning analytics for learners Do learners change their behaviour when confronted with their learning performance relative to that of successful learners? Tracker deployed in the Drinking Water MOOC Dan Davis, Guanliang Chen, Ioana Jivet, Claudia Hauff, and Geert-Jan Houben. Encouraging Metacognition & Self-Regulation in MOOCs through Increased Learner Feedback. LAL Workshop 2016. 5,462 learners were exposed to the intervention WIS Research Project
  35. 35. Learning Tracker - results WaterX C 1,268 160 MOOC COND. N # PASS PASS RATE T 1,251 188 12.6% 15.0% UrbanX C 771 136 T 746 165 17.6% 22.1% BusinessX C 164 46 T 160 54 28.0% 33.8% OVERALL C 2,203 342 T 2,157 407 15.5% 18.9% **
  36. 36. Learning Tracker - results Control 1,150 45 CONDITION N # PASS PASS RATE 3.9% Indiv. - Promotion 1,147 62 5.4% Collect. - Prevent. 1,118 51 4.5% Note: If activity threshold is set from 5 mins to 1 hour, CalcX Pass rate changes from 5% to 16%
  37. 37. [!] Context matters! • available data • national and institutional regulations and culture • educational vision • educational system, size of population .. • … Don’t just copy existing LA solutions! 44
  38. 38. Recommendations
  39. 39. Key messages ETHICS & PRIVACY [!] Use all available expertise (study advisers) [!] Don’t oversimplify. Show uncertainty. [!] Beware of predictive algorithms. [!] Wording matters. [!] Give students “the key” to their data. [!] Learning Analytics is an opportunity within GDPR: “useful” view to the data IMPACT [!] Acceptance precedes impact. [!] Impact is hard to measure/obtain. [!] large scale through: “general” and “specifc” interventions DATA [!] Start with the available data. (small data approach) [!] Look beyond the obvious data. [!] Not all data is usable. [!] Keep Learning Analytics in mind when designing learning activities. [!] Learning dashboards themselves create new learning traces SCALABILITY & TRANSFERABILITY [!] Context matters! [repeat] Start with available data (small data approach) [!] modularity of software solutions more open important than open-source [!] realistic long-term deployments require embedding in existing IT choices and knowhow of higher education institute [!] Portability of data (documentation, meta data) is more important than standardization
  40. 40. Thank you for participating! www.stela-project.eu @STELA_Project

×