SlideShare ist ein Scribd-Unternehmen logo
1 von 42
Downloaden Sie, um offline zu lesen
Tracking and monitoring learners in MOOCs
Social surveys in EMMA
Chiara M Ferrari,
Group Director, social research department at
Ipsos Public Affairs, Milano
part of the Ipsos Group
Participants Profiling: HOW IT WORKS
REGISTRATION FORM
Email __________
Password_________
Age__________
Gender___________
______
Please complete your registration
answering to a short questionnaire:
http://research.ipsosinteractive.com/mrIWeb/mrIWeb.dll?I
.Project=S1036981&Id=101&i.User1=39253324&i.User2=CIB
eCCBd
INDIVIDUAL Link matched to the
users ID , which redirects to the
IPSOS platform
REGISTRATION FORM
Thank you!
Your registration is
completed.
The registration data are recorded in
the EMMA SERVER . Each
student/batch of data gets an
individual and unique ID which will
allow to connect the data from
different sources: surveys, tracking and
learning analytics
Data on expectations will
be stored in the IPSOS
SERVER and will receive the
same individual and unique
ID, to allow connection and
subsequent analysis
QUESTIONNAIRE ON EXPECTATIONS
•Email
•Gender
•Age
•Employment status:
• Working full time
• Working part time
• Looking for a work
• In full time education
• Retired
• Other “don’t work”
•Education Level: (ISCED LEVEL)
What is the highest level of education that you have completed?
Level 0 - [Pre-primary education]
Level 1 - [Primary education or first stage of basic education]
Level 2 - [Lower secondary or second stage of basic education]
Level 3 - [(Upper) secondary education]
Level 4 - [Post-secondary non-tertiary education]
Level 5 - [First stage of tertiary education] University/college
Level 6 - [Second stage of tertiary education] Doctorate level
• Country of birth/Country of residence
• Mother tongue/language used in studies/language of
choice for EMMA
Participants Profiling: WHAT WE COLLECT
Expectation questionnaire optional and one-off, a red toolbar which is there
during the entire user navigation and reminds the student he /she should complete
the survey
Specific questions about the EMMA offer
how did they hear about EMMA
what are the main reasons to participate, e.g..
• experience a MOOC
• look and experience the design of the MOOC
• gain knowledge in a new domain
• keep up to date with my domain
• want a certificate
personal learning objectives  single course vs. wider path of studies (e.g. several courses in connection) vs. focus on a
specific topic / theme vs. general info to gain general knowledge on a topic / theme vs. …
Previous experience with e-learning (as a learner)
How much time they expect to invest in this MOOC
In which of the learning activities they expect to take part
In which of the interactions they expect to participate
How much time they have available for this MOOC
From what location (home, university, work, library, ...)
On which support (PC, laptop, tablet, mobile…)
Expectations from EMMA users
Gender
AVG 36.5
Age
Education
59% 41%
Occupation
SOCIODEMOGRAPHIC PROFILE
Was this in context of your formal education?
Have you had any previous experience with e-learning?
What is your overall level of satisfaction with e-learning so far?
90% 10%
Very satisfied = 29%
E-learning experience
What are the reasons why you decided to register on EMMA?.
Reasons for visiting EMMA
Mini questionnaire -> optional one for each MOOC, it is entered freely by the teacher
as a simple unit (see the picture below). Teachers can decide when and where they
submit it
Emma evaluation package webinar
Emma evaluation package webinar
Emma evaluation package webinar
1414
OUNL_E-learning
MOOC QUESTIONNAIRE REPORT
Milan, March 2016
Ipsos – II round of MOOCs
© 2016 Ipsos. All rights reserved. ContainsIpsos' Confidential and Proprietaryinformation and may not be disclosed or reproducedwithout the prior written consent of Ipsos.
15
INDEX
Advice
Your experience with the MOOC E-learning
Tasks and assignments
Translation functionality
Your point of view
16
YOUR EXPERIENCE WITH THE MOOC E-LEARNING
The course in general
The audiovisual material
Recordings of expert interviews
Masterclasses on specific topics
Interaction with other learners
Interaction with teacher/tutor
Learning activities and assignments
The opportunity to keep up with the domain “e-learning” in a flexible manner
12%
38%
25%
12%
38%
38%
12%
25%
25%
38%
25%
38%
12%
12%
12%
12%
12%
12%
38%
50%
50%
50%
62%
62%
38%
50%
Strongly like Like Neutral Dislike Strongly dislike Didn't use/ does not apply
Base casi: totale rispondenti
Please rate from "strongly dislike" to "strongly like" how you would rate your experience
with this MOOC
17
50%
38%
12%
Fully in English
Fully in Dutch
Partly in English and partly in
Dutch
TRANSLATION FUNCTIONALITY
Base casi: totale rispondenti
a) Let’s now discuss the translation functionality. Did you follow the course…?
b) How much do you use the translation functionality? I used it…
25%
75%
Always
Sometimes
Not at all
c) How much do you agree or disagree with the following statements
about the translation functionality?
It’s effective
It needs more languages to be really
effective
It’s totally new and different
It enhances the overall value of the
EMMA platform
It makes EMMA a truly European
experience
50%
50%
50%
100%
50%
50%
50%
50%
50%
Completely agree
Fairly agree
Neither agree nor disagree
Fairly disagree
Completely disagree
18
TASKS AND ASSIGNMENTS
Base casi: totale rispondenti
How much do you agree or disagree with the following statements about the tasks and the assignments
which were requested?
They are a good way to experience the course
They are too time consuming
I found them educational
They are engaging
They motivated me
They are a good way to create a collaborative activity
They are a good way to improve individual learning
I received the expected feedback
I liked receiving feedback from other participants
I liked giving feedback to other participants
If I was left behind with the tasks, I was still motivated to continue with the course
If I was left behind with the tasks, I wanted to quit the course
12%
12%
12%
12%
12%
12%
12%
25%
25%
38%
38%
38%
25%
50%
12%
12%
25%
12%
25%
12%
12%
12%
25%
12%25%
12%
38%
12%
12%
12%
12%
12%
25%
25%
12%
12%
12%
12%
12%
38%
25%
38%
38%
25%
25%
25%
62%
62%
50%
38%
38%
Completely agree Fairly agree Neither agree nor disagree Fairly disagree Completely disagree Does not apply
19
ADVICE
Verbatim
I engaged in this MOOC but there is no subtitle in English.
Part of the course was in Dutch and not in English as expected. The very first video about what is e-learning was in Dutch with no
subtitles. There were too many videos in Dutch so even if you activate the subtitles, it is not easy to follow as there is too much text and
what is being said is not well translated. There were some bugs with the activities so after the week 2/3, I just left it as for me, if the
technology supporting the course is not reliable, then, it is a bit pointless. I guess that this platform needs a lot of improvements as well
as the course design. From the moment you accept any participant, just make sure that all the material is accessible in proper English
and that the activities can be made without any technical hassle. Overall, navigating into the platform is not intuitive, it takes too long. I
also felt that there was too much text and that some resources was out of date. Another thing, the platform is not supported well by all
the browsers
When I started the course, the automatic translation was not working well and some functions were not working well. This was
commented by the participants, but the facilitators were not reaction on this in the first days. After two days it was still not working, I
somehow stopped the course, because I simply forget to check in ....
cursus eigenlijk niet gevolgd (andere prioriteiten)
so bad because mi mother tongue is spanish for that reason i dont understood nothing
The course is not available after the end date which is a significant weakness for those pushed for time and who can't complete the
course in the timescale advised. I wasn't able to complete the course and the platform doesn't tell you this.
Timing was niet ideaal. Start tijdens paasvakantie, daarna een weekje conge en dan was het moeilijk om nog op te volgen!
Een enkele expert zou moeten werken aan haar presentatietechniek. Zij komt ondanks haar hoog opleidingsniveau tamelijk moeizaam
aan een vlotte redenering.
In your opinion, which are the weaker aspects of the course? Why?
20
YOUR POINT OF VIEW
Is there anything you would like to report or highlight, after having analyzed the data?
Should you re-run this same MOOC, is there anything you would do differently? Please explain
_____________________________________________________________________________________________________________________________________
_____________________________________________________________________________________________________________________________________
_____________________________________________________________________________________________________________________________________
_____________________________________________________________________________________________________________________________________
Exit questionnaire -> optional and and one-off, automatically embedded at the end of
each slide in all units of the last lesson of each course
65% 66% 57% 61% 73% 71%
It's engaging It's immediate
It's self-explaining It's rich in offer
It offers cultural diversity It offers flexibility in learning paths
disagree
9% 9% 22% 13%
7% 11%
The EMMA platform is … (% of agreement)
35%
46%
13%
5% 1% Extremely good
Fairly good
Neither good not bad
Fairly bad
Extremely bad
Evaluation of the experience
1%3%3%
3%
3%
3%
4%
4%
4%
4%
6%
9%
9%
11%
11%
21%
Sport
Arts
Education
Law and commerce
Marketing
Creative writing
Social Science
Sciences
Literature
Languages
Economy/Finances/taxes
Computer /web programming
Communication culture
History
Health/medical school
Food and wine
Areas of interest for the future
Learning Analytics in EMMA
Kairit Tammets, PhD
Senior researcher, head of the studies
Centre for Educational Technology
School of Digital Technologies
Tallinn University
Why Learning Analytics?
• LA allows to understand:
– how MOOC participants learn, engage with the
content and with the other participants;
– how MOOC designers and facilitators can improve
their course designs and pedagogies
• LA was implemented in EMMA as an
additional evaluation tool;
Why Learning Analytics? (2)
• Current approaches of learning analytics in
MOOCs focus on:
– Calculation of drop-out rates;
– Prediction of drop-out;
– Clustering on participants based on their online
behavior
• EMMA approach:
– To establish feedback loop between facilitators and
participants in the form of dashboard;
– To encourage evidence-based course design and
pedagogical interventions
Learning Analytics methodology
• What are the research questions?
• What data is needed for answering research
questions?
• How this data can be collected?
• How to meaningfully present data?
• How to design feedback loop?
• How to learn from the process?
How Learning Analytics work?
• Tracking tool – tracks and collects users
traces;
• Learning Record Store – stores learning
experiences (xAPI statements) and sends them
to dashboard application (Learning Locker);
• Dashboard – visualization of the interactions
How Learning Analytics work?
•
•
•
•
•
•
EMMA dashboards
• Learner view:
– Progress based on access of materials and
submissions of assignments;
– Overview of usage of the learning resources;
– Social network analysis based on comments and
posts;
– Activity stream;
– Participants – enrolled MOOC participants
Emma evaluation package webinar
Emma evaluation package webinar
Emma evaluation package webinar
EMMA dashboards
• Facilitator view:
– Overview of different interactions – accessed
materials, time spent on activities, submissions
and posts
– Overview of the usage of the learning resources;
– Social network analysis based on comments and
posts;
– Activity stream
– Participants – enrolled MOOC participants
Emma evaluation package webinar
Emma evaluation package webinar
Emma evaluation package webinar
Emma evaluation package webinar
Addition to dashboard
• Additional analysis:
– Cluster of the participants:
• Enrolled;
• Observers;
• Contributors;
• Active
– In-depth overview of usage of the course content
– Additional inquiries by the MOOC facilitators
Emma evaluation package webinar
What next?
• Beyond the dashboards – what happens after
using the dashboards, how MOOC designers
improve their pedagogies, technologies and
media
• Combining survey data and LA data;
• Translation of the numbers for contributing to
European MOOC model

Weitere ähnliche Inhalte

Was ist angesagt?

Customer interview – SkillOut.com
Customer interview – SkillOut.comCustomer interview – SkillOut.com
Customer interview – SkillOut.comOlya Kollen, PhD
 
Going online with education
Going online with educationGoing online with education
Going online with educationTom Hutchison
 
LeadTeam start-up: step 11 Validating hypothesis
LeadTeam start-up: step 11 Validating hypothesis LeadTeam start-up: step 11 Validating hypothesis
LeadTeam start-up: step 11 Validating hypothesis Clémence Lagaüzère
 
Integrating Tutors And Call In Efl
Integrating Tutors And Call In EflIntegrating Tutors And Call In Efl
Integrating Tutors And Call In Eflgetchan
 
EdTechJoker IST 402 - Syllabus day
EdTechJoker IST 402 - Syllabus dayEdTechJoker IST 402 - Syllabus day
EdTechJoker IST 402 - Syllabus daybtopro
 
Workshop finalpghs
Workshop finalpghsWorkshop finalpghs
Workshop finalpghscarena
 
Transformational online and hybrid teaching%28 sjc%29 (1)
Transformational online and hybrid teaching%28 sjc%29 (1)Transformational online and hybrid teaching%28 sjc%29 (1)
Transformational online and hybrid teaching%28 sjc%29 (1)prennertariev
 
837 lesson
837 lesson837 lesson
837 lessonkpatric1
 
Teaching hacks
Teaching hacksTeaching hacks
Teaching hacksMatt Clare
 
Terri Feedback Ppt
Terri Feedback PptTerri Feedback Ppt
Terri Feedback PptTerri Weiss
 
Online Learning and Student Success
Online Learning and Student Success Online Learning and Student Success
Online Learning and Student Success Pearson North America
 
QM Standards 5 & 6: Interaction and Technology
QM Standards 5 & 6: Interaction and TechnologyQM Standards 5 & 6: Interaction and Technology
QM Standards 5 & 6: Interaction and TechnologyStaci Trekles
 
Six Pack to the Rescue: Third Party Integrations
Six Pack to the Rescue: Third Party IntegrationsSix Pack to the Rescue: Third Party Integrations
Six Pack to the Rescue: Third Party IntegrationsD2L Barry
 

Was ist angesagt? (20)

Adobe connect technology_infusion
Adobe connect technology_infusionAdobe connect technology_infusion
Adobe connect technology_infusion
 
Notes
NotesNotes
Notes
 
Customer interview – SkillOut.com
Customer interview – SkillOut.comCustomer interview – SkillOut.com
Customer interview – SkillOut.com
 
Going online with education
Going online with educationGoing online with education
Going online with education
 
LeadTeam start-up: step 11 Validating hypothesis
LeadTeam start-up: step 11 Validating hypothesis LeadTeam start-up: step 11 Validating hypothesis
LeadTeam start-up: step 11 Validating hypothesis
 
Integrating Tutors And Call In Efl
Integrating Tutors And Call In EflIntegrating Tutors And Call In Efl
Integrating Tutors And Call In Efl
 
LeadTeam Pitch
LeadTeam PitchLeadTeam Pitch
LeadTeam Pitch
 
EdTechJoker IST 402 - Syllabus day
EdTechJoker IST 402 - Syllabus dayEdTechJoker IST 402 - Syllabus day
EdTechJoker IST 402 - Syllabus day
 
Workshop finalpghs
Workshop finalpghsWorkshop finalpghs
Workshop finalpghs
 
Sandra Luna handout
Sandra Luna handoutSandra Luna handout
Sandra Luna handout
 
Uwcsea day 1v2
Uwcsea day 1v2Uwcsea day 1v2
Uwcsea day 1v2
 
Transformational online and hybrid teaching%28 sjc%29 (1)
Transformational online and hybrid teaching%28 sjc%29 (1)Transformational online and hybrid teaching%28 sjc%29 (1)
Transformational online and hybrid teaching%28 sjc%29 (1)
 
837 lesson
837 lesson837 lesson
837 lesson
 
Teaching hacks
Teaching hacksTeaching hacks
Teaching hacks
 
Terri Feedback Ppt
Terri Feedback PptTerri Feedback Ppt
Terri Feedback Ppt
 
Online Learning and Student Success
Online Learning and Student Success Online Learning and Student Success
Online Learning and Student Success
 
Webinar REC:all
Webinar REC:allWebinar REC:all
Webinar REC:all
 
Session1
Session1Session1
Session1
 
QM Standards 5 & 6: Interaction and Technology
QM Standards 5 & 6: Interaction and TechnologyQM Standards 5 & 6: Interaction and Technology
QM Standards 5 & 6: Interaction and Technology
 
Six Pack to the Rescue: Third Party Integrations
Six Pack to the Rescue: Third Party IntegrationsSix Pack to the Rescue: Third Party Integrations
Six Pack to the Rescue: Third Party Integrations
 

Ähnlich wie Emma evaluation package webinar

Student Survey Nov 2022
Student Survey Nov 2022Student Survey Nov 2022
Student Survey Nov 2022Peter Windle
 
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01Kandis Johnston
 
Strategies and Tips for Engaging Today’s Students
Strategies and Tips for Engaging Today’s StudentsStrategies and Tips for Engaging Today’s Students
Strategies and Tips for Engaging Today’s StudentsCengage Learning
 
EDUC5199G Session 5 Presentation
EDUC5199G Session 5 PresentationEDUC5199G Session 5 Presentation
EDUC5199G Session 5 PresentationRobert Power
 
Web 2.0 professional development program
Web 2.0 professional development programWeb 2.0 professional development program
Web 2.0 professional development programColleen Hodgins
 
From tools to pedagogy: extending language learning using the Internet
From tools to pedagogy: extending language learning using the InternetFrom tools to pedagogy: extending language learning using the Internet
From tools to pedagogy: extending language learning using the InternetPhilip Saxon
 
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317IGNIS 2017 - Marcia Pedersen and Anita Peng 052317
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317SBCTCProfessionalLearning
 
User reports uf coe m ed_fran siracusa
User reports uf coe m ed_fran siracusaUser reports uf coe m ed_fran siracusa
User reports uf coe m ed_fran siracusaFran Siracusa
 
My addie model
My addie modelMy addie model
My addie modelFattimaaaH
 
Mr M.S Mkize Final Portfolio (exam)
Mr M.S Mkize Final Portfolio (exam)Mr M.S Mkize Final Portfolio (exam)
Mr M.S Mkize Final Portfolio (exam)Motheo Steven Mkize
 
International Certificate in Fundamentals of Early Childhood Education
International Certificate in Fundamentals of Early Childhood EducationInternational Certificate in Fundamentals of Early Childhood Education
International Certificate in Fundamentals of Early Childhood EducationAsian International College - India
 
TechChat: Enhancing your units
TechChat: Enhancing your unitsTechChat: Enhancing your units
TechChat: Enhancing your unitsdebbieholley1
 
Blended learning hand book by Vuolearning v.1.0
Blended learning hand book by Vuolearning v.1.0Blended learning hand book by Vuolearning v.1.0
Blended learning hand book by Vuolearning v.1.0Johanna Pellinen
 
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes How Blended Learning Can Pack More Objectives in Your EAP Writing Classes
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes Courtney King
 
Nbhs teacher technology_survey_results
Nbhs teacher technology_survey_resultsNbhs teacher technology_survey_results
Nbhs teacher technology_survey_resultsLaura Hearnsberger
 
Making Blended Learning Work in Vocational Education & Training (VET)
Making Blended Learning Work in Vocational Education & Training (VET)Making Blended Learning Work in Vocational Education & Training (VET)
Making Blended Learning Work in Vocational Education & Training (VET)Circulus Education
 

Ähnlich wie Emma evaluation package webinar (20)

Student Survey Nov 2022
Student Survey Nov 2022Student Survey Nov 2022
Student Survey Nov 2022
 
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01
Strategiesandtipsforengagingtodaysstudents 140328150510-phpapp01
 
Strategies and Tips for Engaging Today’s Students
Strategies and Tips for Engaging Today’s StudentsStrategies and Tips for Engaging Today’s Students
Strategies and Tips for Engaging Today’s Students
 
EDUC5199G Session 5 Presentation
EDUC5199G Session 5 PresentationEDUC5199G Session 5 Presentation
EDUC5199G Session 5 Presentation
 
Web 2.0 professional development program
Web 2.0 professional development programWeb 2.0 professional development program
Web 2.0 professional development program
 
From tools to pedagogy: extending language learning using the Internet
From tools to pedagogy: extending language learning using the InternetFrom tools to pedagogy: extending language learning using the Internet
From tools to pedagogy: extending language learning using the Internet
 
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317IGNIS 2017 - Marcia Pedersen and Anita Peng 052317
IGNIS 2017 - Marcia Pedersen and Anita Peng 052317
 
User reports uf coe m ed_fran siracusa
User reports uf coe m ed_fran siracusaUser reports uf coe m ed_fran siracusa
User reports uf coe m ed_fran siracusa
 
My addie model
My addie modelMy addie model
My addie model
 
Nbvtalkonmoocs
NbvtalkonmoocsNbvtalkonmoocs
Nbvtalkonmoocs
 
Mr M.S Mkize Final Portfolio (exam)
Mr M.S Mkize Final Portfolio (exam)Mr M.S Mkize Final Portfolio (exam)
Mr M.S Mkize Final Portfolio (exam)
 
International Certificate in Fundamentals of Early Childhood Education
International Certificate in Fundamentals of Early Childhood EducationInternational Certificate in Fundamentals of Early Childhood Education
International Certificate in Fundamentals of Early Childhood Education
 
TechChat: Enhancing your units
TechChat: Enhancing your unitsTechChat: Enhancing your units
TechChat: Enhancing your units
 
Blended Learning Essentials - 35,000 participants and counting
Blended Learning Essentials - 35,000 participants and countingBlended Learning Essentials - 35,000 participants and counting
Blended Learning Essentials - 35,000 participants and counting
 
Blended learning hand book by Vuolearning v.1.0
Blended learning hand book by Vuolearning v.1.0Blended learning hand book by Vuolearning v.1.0
Blended learning hand book by Vuolearning v.1.0
 
2015 c3 online learning part 1 & 2
2015 c3 online learning part 1 & 22015 c3 online learning part 1 & 2
2015 c3 online learning part 1 & 2
 
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes How Blended Learning Can Pack More Objectives in Your EAP Writing Classes
How Blended Learning Can Pack More Objectives in Your EAP Writing Classes
 
Nbhs teacher technology_survey_results
Nbhs teacher technology_survey_resultsNbhs teacher technology_survey_results
Nbhs teacher technology_survey_results
 
Youngstown session 2 online learning
Youngstown session 2 online learningYoungstown session 2 online learning
Youngstown session 2 online learning
 
Making Blended Learning Work in Vocational Education & Training (VET)
Making Blended Learning Work in Vocational Education & Training (VET)Making Blended Learning Work in Vocational Education & Training (VET)
Making Blended Learning Work in Vocational Education & Training (VET)
 

Mehr von EUmoocs

Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...
Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...
Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...EUmoocs
 
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platform
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platformRosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platform
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platformEUmoocs
 
Firssova, Brouns - Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...
Firssova, Brouns -  Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...Firssova, Brouns -  Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...
Firssova, Brouns - Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...EUmoocs
 
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs? - Da...
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs?  - Da...Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs?  - Da...
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs? - Da...EUmoocs
 
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...EUmoocs
 
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...EUmoocs
 
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...EUmoocs
 
Mart Laanpere - Task-centred approach to mooc design
Mart Laanpere - Task-centred approach to mooc designMart Laanpere - Task-centred approach to mooc design
Mart Laanpere - Task-centred approach to mooc designEUmoocs
 
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...EUmoocs
 
Milena Popova - Taste Europeana
Milena Popova - Taste EuropeanaMilena Popova - Taste Europeana
Milena Popova - Taste EuropeanaEUmoocs
 
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...EUmoocs
 
Eleonora Pantò - Using Social Media effectively in your MOOC
Eleonora Pantò - Using Social Media effectively in your MOOCEleonora Pantò - Using Social Media effectively in your MOOC
Eleonora Pantò - Using Social Media effectively in your MOOCEUmoocs
 
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...EUmoocs
 
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...EUmoocs
 
Is there value of learning analytics in MOOCs?
Is there value of learning analytics in MOOCs?Is there value of learning analytics in MOOCs?
Is there value of learning analytics in MOOCs?EUmoocs
 
EMMA Services - EMOOCs 2016 conference
EMMA Services - EMOOCs 2016 conferenceEMMA Services - EMOOCs 2016 conference
EMMA Services - EMOOCs 2016 conference EUmoocs
 
Governing by data: Considerations on the role of learning analytics in education
Governing by data: Considerations on the role of learning analytics in educationGoverning by data: Considerations on the role of learning analytics in education
Governing by data: Considerations on the role of learning analytics in educationEUmoocs
 
EMMA services at EMOOCs Conference 2016
 EMMA services at EMOOCs Conference 2016 EMMA services at EMOOCs Conference 2016
EMMA services at EMOOCs Conference 2016EUmoocs
 
What EMMA stands for?
What EMMA stands for?What EMMA stands for?
What EMMA stands for?EUmoocs
 
Fra sperimentazione e policy istituzionale
Fra sperimentazione e policy istituzionaleFra sperimentazione e policy istituzionale
Fra sperimentazione e policy istituzionaleEUmoocs
 

Mehr von EUmoocs (20)

Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...
Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...
Carmen Padrón-Nápoles - EMMA webinar: Sharing the experience of the EMMA plat...
 
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platform
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platformRosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platform
Rosanna De Dosa - EMMA webinar: Sharing the experience of the EMMA platform
 
Firssova, Brouns - Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...
Firssova, Brouns -  Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...Firssova, Brouns -  Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...
Firssova, Brouns - Hoe ontwerpt u een effectieve MOOC? Voorbeelden uit de pr...
 
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs? - Da...
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs?  - Da...Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs?  - Da...
Brouns, Firssova, Kalz - Is there value of learning analytics in MOOCs? - Da...
 
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...
Brouns, Firssova - Bootcamp EMMA MOOC Assessment for learning in practice - E...
 
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...
Ilaria Merciai, Marco Cerrone - Monitoring a Learning Community in a Hybrid E...
 
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...
Rosanna De Rosa, Alessandro Bogliolo - Teaching to teachers. A MOOC based hyb...
 
Mart Laanpere - Task-centred approach to mooc design
Mart Laanpere - Task-centred approach to mooc designMart Laanpere - Task-centred approach to mooc design
Mart Laanpere - Task-centred approach to mooc design
 
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...
Olga Firssova - Task-centred approach to MOOC design - challenges and opportu...
 
Milena Popova - Taste Europeana
Milena Popova - Taste EuropeanaMilena Popova - Taste Europeana
Milena Popova - Taste Europeana
 
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...
Anna Maria Tammaro, Getaneh Alemu - Using Europeanafor learning & teaching: E...
 
Eleonora Pantò - Using Social Media effectively in your MOOC
Eleonora Pantò - Using Social Media effectively in your MOOCEleonora Pantò - Using Social Media effectively in your MOOC
Eleonora Pantò - Using Social Media effectively in your MOOC
 
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...
Deborah Arnold - EMMA webinar: Capturing and delivering effective video as pa...
 
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...
Mathy Vanbuel - EMMA webinar: Capturing and delivering effective video as par...
 
Is there value of learning analytics in MOOCs?
Is there value of learning analytics in MOOCs?Is there value of learning analytics in MOOCs?
Is there value of learning analytics in MOOCs?
 
EMMA Services - EMOOCs 2016 conference
EMMA Services - EMOOCs 2016 conferenceEMMA Services - EMOOCs 2016 conference
EMMA Services - EMOOCs 2016 conference
 
Governing by data: Considerations on the role of learning analytics in education
Governing by data: Considerations on the role of learning analytics in educationGoverning by data: Considerations on the role of learning analytics in education
Governing by data: Considerations on the role of learning analytics in education
 
EMMA services at EMOOCs Conference 2016
 EMMA services at EMOOCs Conference 2016 EMMA services at EMOOCs Conference 2016
EMMA services at EMOOCs Conference 2016
 
What EMMA stands for?
What EMMA stands for?What EMMA stands for?
What EMMA stands for?
 
Fra sperimentazione e policy istituzionale
Fra sperimentazione e policy istituzionaleFra sperimentazione e policy istituzionale
Fra sperimentazione e policy istituzionale
 

Kürzlich hochgeladen

AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICESayali Powar
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesMohammad Hassany
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice documentXsasf Sfdfasd
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and stepobaje godwin sunday
 

Kürzlich hochgeladen (20)

AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICE
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming Classes
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice document
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and step
 
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdfPersonal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
 

Emma evaluation package webinar

  • 1. Tracking and monitoring learners in MOOCs
  • 2. Social surveys in EMMA Chiara M Ferrari, Group Director, social research department at Ipsos Public Affairs, Milano part of the Ipsos Group
  • 3. Participants Profiling: HOW IT WORKS REGISTRATION FORM Email __________ Password_________ Age__________ Gender___________ ______ Please complete your registration answering to a short questionnaire: http://research.ipsosinteractive.com/mrIWeb/mrIWeb.dll?I .Project=S1036981&Id=101&i.User1=39253324&i.User2=CIB eCCBd INDIVIDUAL Link matched to the users ID , which redirects to the IPSOS platform REGISTRATION FORM Thank you! Your registration is completed. The registration data are recorded in the EMMA SERVER . Each student/batch of data gets an individual and unique ID which will allow to connect the data from different sources: surveys, tracking and learning analytics Data on expectations will be stored in the IPSOS SERVER and will receive the same individual and unique ID, to allow connection and subsequent analysis QUESTIONNAIRE ON EXPECTATIONS
  • 4. •Email •Gender •Age •Employment status: • Working full time • Working part time • Looking for a work • In full time education • Retired • Other “don’t work” •Education Level: (ISCED LEVEL) What is the highest level of education that you have completed? Level 0 - [Pre-primary education] Level 1 - [Primary education or first stage of basic education] Level 2 - [Lower secondary or second stage of basic education] Level 3 - [(Upper) secondary education] Level 4 - [Post-secondary non-tertiary education] Level 5 - [First stage of tertiary education] University/college Level 6 - [Second stage of tertiary education] Doctorate level • Country of birth/Country of residence • Mother tongue/language used in studies/language of choice for EMMA Participants Profiling: WHAT WE COLLECT
  • 5. Expectation questionnaire optional and one-off, a red toolbar which is there during the entire user navigation and reminds the student he /she should complete the survey
  • 6. Specific questions about the EMMA offer how did they hear about EMMA what are the main reasons to participate, e.g.. • experience a MOOC • look and experience the design of the MOOC • gain knowledge in a new domain • keep up to date with my domain • want a certificate personal learning objectives  single course vs. wider path of studies (e.g. several courses in connection) vs. focus on a specific topic / theme vs. general info to gain general knowledge on a topic / theme vs. … Previous experience with e-learning (as a learner) How much time they expect to invest in this MOOC In which of the learning activities they expect to take part In which of the interactions they expect to participate How much time they have available for this MOOC From what location (home, university, work, library, ...) On which support (PC, laptop, tablet, mobile…) Expectations from EMMA users
  • 8. Was this in context of your formal education? Have you had any previous experience with e-learning? What is your overall level of satisfaction with e-learning so far? 90% 10% Very satisfied = 29% E-learning experience
  • 9. What are the reasons why you decided to register on EMMA?. Reasons for visiting EMMA
  • 10. Mini questionnaire -> optional one for each MOOC, it is entered freely by the teacher as a simple unit (see the picture below). Teachers can decide when and where they submit it
  • 14. 1414 OUNL_E-learning MOOC QUESTIONNAIRE REPORT Milan, March 2016 Ipsos – II round of MOOCs © 2016 Ipsos. All rights reserved. ContainsIpsos' Confidential and Proprietaryinformation and may not be disclosed or reproducedwithout the prior written consent of Ipsos.
  • 15. 15 INDEX Advice Your experience with the MOOC E-learning Tasks and assignments Translation functionality Your point of view
  • 16. 16 YOUR EXPERIENCE WITH THE MOOC E-LEARNING The course in general The audiovisual material Recordings of expert interviews Masterclasses on specific topics Interaction with other learners Interaction with teacher/tutor Learning activities and assignments The opportunity to keep up with the domain “e-learning” in a flexible manner 12% 38% 25% 12% 38% 38% 12% 25% 25% 38% 25% 38% 12% 12% 12% 12% 12% 12% 38% 50% 50% 50% 62% 62% 38% 50% Strongly like Like Neutral Dislike Strongly dislike Didn't use/ does not apply Base casi: totale rispondenti Please rate from "strongly dislike" to "strongly like" how you would rate your experience with this MOOC
  • 17. 17 50% 38% 12% Fully in English Fully in Dutch Partly in English and partly in Dutch TRANSLATION FUNCTIONALITY Base casi: totale rispondenti a) Let’s now discuss the translation functionality. Did you follow the course…? b) How much do you use the translation functionality? I used it… 25% 75% Always Sometimes Not at all c) How much do you agree or disagree with the following statements about the translation functionality? It’s effective It needs more languages to be really effective It’s totally new and different It enhances the overall value of the EMMA platform It makes EMMA a truly European experience 50% 50% 50% 100% 50% 50% 50% 50% 50% Completely agree Fairly agree Neither agree nor disagree Fairly disagree Completely disagree
  • 18. 18 TASKS AND ASSIGNMENTS Base casi: totale rispondenti How much do you agree or disagree with the following statements about the tasks and the assignments which were requested? They are a good way to experience the course They are too time consuming I found them educational They are engaging They motivated me They are a good way to create a collaborative activity They are a good way to improve individual learning I received the expected feedback I liked receiving feedback from other participants I liked giving feedback to other participants If I was left behind with the tasks, I was still motivated to continue with the course If I was left behind with the tasks, I wanted to quit the course 12% 12% 12% 12% 12% 12% 12% 25% 25% 38% 38% 38% 25% 50% 12% 12% 25% 12% 25% 12% 12% 12% 25% 12%25% 12% 38% 12% 12% 12% 12% 12% 25% 25% 12% 12% 12% 12% 12% 38% 25% 38% 38% 25% 25% 25% 62% 62% 50% 38% 38% Completely agree Fairly agree Neither agree nor disagree Fairly disagree Completely disagree Does not apply
  • 19. 19 ADVICE Verbatim I engaged in this MOOC but there is no subtitle in English. Part of the course was in Dutch and not in English as expected. The very first video about what is e-learning was in Dutch with no subtitles. There were too many videos in Dutch so even if you activate the subtitles, it is not easy to follow as there is too much text and what is being said is not well translated. There were some bugs with the activities so after the week 2/3, I just left it as for me, if the technology supporting the course is not reliable, then, it is a bit pointless. I guess that this platform needs a lot of improvements as well as the course design. From the moment you accept any participant, just make sure that all the material is accessible in proper English and that the activities can be made without any technical hassle. Overall, navigating into the platform is not intuitive, it takes too long. I also felt that there was too much text and that some resources was out of date. Another thing, the platform is not supported well by all the browsers When I started the course, the automatic translation was not working well and some functions were not working well. This was commented by the participants, but the facilitators were not reaction on this in the first days. After two days it was still not working, I somehow stopped the course, because I simply forget to check in .... cursus eigenlijk niet gevolgd (andere prioriteiten) so bad because mi mother tongue is spanish for that reason i dont understood nothing The course is not available after the end date which is a significant weakness for those pushed for time and who can't complete the course in the timescale advised. I wasn't able to complete the course and the platform doesn't tell you this. Timing was niet ideaal. Start tijdens paasvakantie, daarna een weekje conge en dan was het moeilijk om nog op te volgen! Een enkele expert zou moeten werken aan haar presentatietechniek. Zij komt ondanks haar hoog opleidingsniveau tamelijk moeizaam aan een vlotte redenering. In your opinion, which are the weaker aspects of the course? Why?
  • 20. 20 YOUR POINT OF VIEW Is there anything you would like to report or highlight, after having analyzed the data? Should you re-run this same MOOC, is there anything you would do differently? Please explain _____________________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________________ _____________________________________________________________________________________________________________________________________
  • 21. Exit questionnaire -> optional and and one-off, automatically embedded at the end of each slide in all units of the last lesson of each course
  • 22. 65% 66% 57% 61% 73% 71% It's engaging It's immediate It's self-explaining It's rich in offer It offers cultural diversity It offers flexibility in learning paths disagree 9% 9% 22% 13% 7% 11% The EMMA platform is … (% of agreement)
  • 23. 35% 46% 13% 5% 1% Extremely good Fairly good Neither good not bad Fairly bad Extremely bad Evaluation of the experience 1%3%3% 3% 3% 3% 4% 4% 4% 4% 6% 9% 9% 11% 11% 21% Sport Arts Education Law and commerce Marketing Creative writing Social Science Sciences Literature Languages Economy/Finances/taxes Computer /web programming Communication culture History Health/medical school Food and wine Areas of interest for the future
  • 24. Learning Analytics in EMMA Kairit Tammets, PhD Senior researcher, head of the studies Centre for Educational Technology School of Digital Technologies Tallinn University
  • 25. Why Learning Analytics? • LA allows to understand: – how MOOC participants learn, engage with the content and with the other participants; – how MOOC designers and facilitators can improve their course designs and pedagogies • LA was implemented in EMMA as an additional evaluation tool;
  • 26. Why Learning Analytics? (2) • Current approaches of learning analytics in MOOCs focus on: – Calculation of drop-out rates; – Prediction of drop-out; – Clustering on participants based on their online behavior • EMMA approach: – To establish feedback loop between facilitators and participants in the form of dashboard; – To encourage evidence-based course design and pedagogical interventions
  • 27. Learning Analytics methodology • What are the research questions? • What data is needed for answering research questions? • How this data can be collected? • How to meaningfully present data? • How to design feedback loop? • How to learn from the process?
  • 28. How Learning Analytics work? • Tracking tool – tracks and collects users traces; • Learning Record Store – stores learning experiences (xAPI statements) and sends them to dashboard application (Learning Locker); • Dashboard – visualization of the interactions
  • 31. EMMA dashboards • Learner view: – Progress based on access of materials and submissions of assignments; – Overview of usage of the learning resources; – Social network analysis based on comments and posts; – Activity stream; – Participants – enrolled MOOC participants
  • 35. EMMA dashboards • Facilitator view: – Overview of different interactions – accessed materials, time spent on activities, submissions and posts – Overview of the usage of the learning resources; – Social network analysis based on comments and posts; – Activity stream – Participants – enrolled MOOC participants
  • 40. Addition to dashboard • Additional analysis: – Cluster of the participants: • Enrolled; • Observers; • Contributors; • Active – In-depth overview of usage of the course content – Additional inquiries by the MOOC facilitators
  • 42. What next? • Beyond the dashboards – what happens after using the dashboards, how MOOC designers improve their pedagogies, technologies and media • Combining survey data and LA data; • Translation of the numbers for contributing to European MOOC model