Inside MOOCs: Student Expectations and Achievement #aln13
1. Inside MOOCs: Student Expectations and Achievement
Clark Shah-Nelson,
Sr. Instructional Designer
Center for Teaching and Learning
cshahne@jhsph.edu
Sukon Kanchanaraksa,
Director
Center for Teaching and Learning
skanchan@jhsph.edu
http://tinyurl.com/insidemoocs-aln13
2. Johns Hopkins Bloomberg School of Public Health
Founded: 1916
Graduate School
Students: 2164 from 87 nations
Faculty: 619 FT; 785 PT
Research: on-going in more than 130 countries
First institution of its kind worldwide
Largest school of public health in the world
Receives 20 percent of all grants and contracts awarded to the 50 accredited U.S. schools of public health
Ranked No. 1 by U.S. News & World Report since 1994
5. JHSPH OCW
As an important component of the School’s mission, JHSPH publishes the materials used in
the teaching of actual courses freely and openly available on the Web. (Since 2005)
Creative Commons license is used with the conditions – attribution, non-commercial and
share alike. (creativecommons.org)
No permission is needed. We encourage students, educators, and public health
practitioners to use OCW content.
6. JHSPH OCW
Content from 112 academic courses, trainings, and symposia is published on JHSPH OCW.
In the past 5 years (Oct 28, 2009 – Oct 27, 2013)
- 1.36 million visits
- 1 million unique visitors
China is the country with the third most visits (after US and India)
- 48,358 visits
- 41,150 unique visitors
7. JHSPH OCW -> Coursera
2012: CTL Looking to make OCW more interactive, started discussion OCW+
Began relationship with Coursera – moved rapidly to prepare first round of courses
Groundwork in place:
intellectual property & copyright
instructors on board
openness culture
faculty with online teaching experience
existing online course development/instructional design/OCW infrastructure
8. Coursera First Run: Total Enrolled Students
Course
Instructor
Enrolled Students
Bio Statistics
Brian Caffo
16,872
Computing for Data Analysis
Roger Peng
55,558
Community Change
Bill Brieger
15,386
Data Analysis
Jeff Leek
101,747
Food Systems
Bob Lawrence, Keeve Nachman
17,164
Obesity Economics
Kevin Frick
23,410
Health For All
Henry Perry
13,145
Vaccine Trials
Karen Charron, Amber Cox
11,546
Total Students
254,828
9. Coursera First Run: Statements of Achievement (SOA)
Bio
Sta(s(cs
Comp
Data
Community
Change
Data
Analysis
Total
Students
Enrolled
16,872
55,558
15,386
101,747
17,164
23,410
13,145
11,546
Total
SOA
749
3,940
768
4,260
2,556
1,908
1,033
1,865
Percent
4%
7%
5%
4%
15%
8%
8%
16%
SOA = Statement of Achievement
Food
Obesity
Health
Vaccine
Systems
Economics
For
All
Trials
10. The Data
•
Surveys administered at opening/closing of course
•
Multiple surveys with same email address
•
Used “conditional formatting” in Excel to identify entries
•
Color coded surveys to indicate properties of duplicate entries
• Looked at time between entries, number of entries and number of differences
•
Kept first entry to capture participants’ initial reaction
•
Sorted out entries after the first survey submission
11. The Survey Data
Total Students: 254, 828
Aggregate
of
all
courses
Total
Students
earned
an
SOA
Total
Students
who
submiNed
Pre-‐Survey
Total
Students
who
submiNed
Post-‐Survey
Total
Students
who
submiNed
both
surveys
Percent
17079
6.70%
53459
21.00%
10046
3.90%
4695
1.80%
12. How do students intend to participate in our MOOCs?
Pre-Survey Intentions Response
87.90%
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
9.50%
Watch all
videos
2.40%
Watch videos
of interest
0.20%
Watch all No Response
videos,
complete
assignment &
earn SOA
13. Poll on Intent and Achievement 1:
http://www.polleverywhere.com/multiple_choice_polls/yLOHh2ekLAqG6DT
http://pollev.com/aln13 TXT code to 37607
14. Of those who intended to earn an SOA, how many did?
Students Who Intended to Earn an SOA
8%
Earned SOA
Didn't Earn SOA
92%
15. Poll on Intent and Achievement 2:
http://www.polleverywhere.com/multiple_choice_polls/B9EdfZbRhXp15w6
http://pollev.com/aln13 TXT code to 37607
16. Of those who did NOT intend to earn an SOA, how many did?
Students Who Didn't Intend to earn an SOA
31%
Earned SOA
Didn't earn SOA
69%
17. What are some of the factors motivating student participation in our
MOOCs?
Why Students Enrolled
40.00%
35.00%
30.00%
Very Important
25.00%
Quite Important
20.00%
Moderately important
15.00%
Slightly important
10.00%
Not important
5.00%
No Response
0.00%
Academic
Job/Career
Field Relevant
Skills
Earn a
Credential
Prestigious
University
Fun and
Enjoyable
Curiousity
about online
courses
18. What are some of the factors motivating student participation in our
MOOCs?
●
Top 3 reasons for enrollment:
●
●
The course will be enjoyable and fun
●
●
The course teaches skills to help participant’s career
The course is directly related to the participant’s academic field of study
One Exception: Obesity Economics Top 3 Reasons
●
The course will be enjoyable and fun
●
The participant is curious about what it is like to take an online course
●
The course teaches skills to help participant’s career
19. How many students submitted one or all assignments?
Total Students in 7 courses: 153,081
Aggregate
of
7
courses
Percentage
SOA's
earned
12822
8.40%
Students
submiNed
all
required
assignments
13321
8.70%
SubmiNed
at
least
one
assignment
32653
21.30%
*Data Analysis omitted
20. Forum Participation and SOA’s
ParScipated
at
least
once
in
forum
ParScipated
at
least
once
in
forum
&
earned
SOA
ParScipated
in
Forum
&
didn't
earn
SOA
Didn't
parScipate
in
forum
&
earned
SOA
Percent
of
Aggregate
across
total
students
5
courses
enrolled
6062
3.80%
2935
1.80%
3125
2.00%
5827
3.70%
Total Students: 158,988
48% of students who
participated in the forums
earned an SOA
52% of students who
participated in the forums
did not earn an SOA
21. More information
White paper:
“Massive Open Online Courses in Public Health” in Frontiers in Public Health
http://tinyurl.com/moocs-in-ph-provisional
Gooding I, Klaas B, Yager JD and Kanchanaraksa S(2013) Massive Open Online Courses in
Public Health. 1:59. doi:10.3389/fpubh.2013.00059
http://www.frontiersin.org/Journal/Abstract.aspx?s=503& name=public%20health
%20education%20and%20promotion& ART_DOI=10.3389/fpubh.2013.00059
22. “Funnel of Participation” – 9 JHSPH MOOCs
Gooding I, Klaas B, Yager JD and Kanchanaraksa S(2013) Massive Open Online Courses in Public Health. 1:59. doi:10.3389/fpubh.2013.00059
23. “Funnel of Participation” 9 JHSPH MOOCs with Peer Assessments
Gooding I, Klaas B, Yager JD and Kanchanaraksa S(2013) Massive Open Online Courses in Public Health. 1:59. doi:10.3389/fpubh.2013.00059
24. Gooding I, Klaas B, Yager JD and Kanchanaraksa S(2013) Massive Open Online Courses in Public Health. 1:59. doi:10.3389/fpubh.2013.00059
25. How valuable were each of the following factors in helping students
complete the course?
Total Students:
7,206
Interaction with Professor
Related personal background and
experience
No Response
Appropriate length of course
Strongly disagree
Disagree
Appropriate difficulty level
Neutral
Sustained engagement with videos
Agree
Strongly agree
Interaction with others
Regular Assessment Deadlines
0.00%
10.00% 20.00% 30.00% 40.00% 50.00% 60.00%
26. How satisfied were students with various aspects of the course?
Total Students:
7,206
60%
50%
40%
30%
Strongly agree
20%
Agree
10%
Neutral
Disagree
0%
I am
Course
Want to Found this
happy with materials
take a
course
what I
presented
more
personally
learned
in
advanced fulfilling
engaging
course
manner
Learned
Will use
what I was what I
hoping to learned in
learn in this course
this course in my job
Strongly disagree
No Response
27. What would have made students more likely to complete the course?
Total Students:
2,797
More Direct Access with Instructor
Make Course Length Shorter
No Response
Credential More Valuable
Strongly disagree
Disagree
Make Course Material More Difficult
Neutral
Agree
Make Course Material Easier
Strongly agree
Reduce Time Commitment
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
28. Did they find the course useful, even if they didn't complete it?
Total Students:
2,797
Relevant to Job
No Response
Met Learning Expectations
Strongly
disagree
Personally Fulfilling
Disagree
Desire to take a more advanced
course
Neutral
Course Materials Engaging
Agree
Happy with what I learned
Strongly agree
0%
5%
10% 15% 20% 25% 30% 35% 40% 45% 50%
29. Enrollment in MOOCs
Updated: October 31, 2013
*Total enrollment includes enrollment in planned, in-progress, and completed offerings.
30. Updated SOA Numbers through 3 Offerings
Course
1st
Training
&
Learning
Programs
for
Volunteer
CHWs
Saving
Lives
Millions
at
a
Time
2nd
SOA
%
3rd
SOA
%
8,685
8,552
MathemaScal
BiostaSsScs
Bootcamp
2
%
7,194
7,122
Design
&
InterpretaSon
of
Clinical
Trials
StaSsScal
Analysis
of
fMRI
SOA
8,514
StaSsScal
Reasoning
for
Public
Health
10,393
Major
Depression
in
the
PopulaSon
16,406
IntroducSon
to
the
US
Food
System
17,164
2,556
14.89%
4,171
Case-‐Based
IntroducSon
to
BiostaSsScs
19,631
2,712
13.81%
2,249
Vaccine
Trials:
Methods
and
Best
PracSces
11,546
1,868
16.18%
9,471
1,033
10.91%
Health
for
All
Through
Primary
Health
Care
13,145
1,033
7.86%
12,706
718
5.65%
7,532
436
5.79%
Principles
of
Obesity
Economics
23,193
1,908
8.23%
6,355
MathemaScal
BiostaSsScs
Bootcamp
1
16,398
749
4.57%
22,153
1,427
6.44%
18,270
1,109
6.07%
Community
Change
in
Public
Health
15,271
767
5.02%
12,046
716
5.94%
1,105
CompuSng
for
Data
Analysis
50,883
3,938
7.74%
44,661
6,271
14.04%
83,441
6,768
8.11%
101,747
4,254
4.18%
72,202
Data
Analysis
31. Updated SOA Numbers through 3 Offerings
No.
of
Completed
Offerings
To
Date
1
End
of
Course
Enrollment
in
Completed
Offering(s)
17,164
Case-‐Based
IntroducSon
to
BiostaSsScs
1
19,631
2,712
13.81%
Vaccine
Trials:
Methods
and
Best
PracSces
2
21,017
2,901
13.80%
Health
for
All
Through
Primary
Health
Care
Principles
of
Obesity
Economics
3
1
33,383
23,193
2,187
1,908
6.55%
8.23%
MathemaScal
BiostaSsScs
Bootcamp
1
Community
Change
in
Public
Health
CompuSng
for
Data
Analysis
Data
Analysis
3
2
3
1
56,821
27,317
179,015
101,747
3,285
1,483
16,977
4,254
5.78%
5.43%
9.48%
4.18%
Total
17
479,288
38,263
7.98%
Course
IntroducSon
to
the
US
Food
System
SOAs
Issued
in
Passing
Rate
in
Completed
Completed
Offering(s)
Offering(s)
2,556
14.89%
32. Thank you!
Additional Credits
Questions?
Instructors:
Bill Brieger, Brian Caffo, Karen Charron, Amber Cox, Kevin Frick,
Bob Lawrence, Jeff Leek, Keeve Nachman, Roger Peng, Henry
Perry
cshahne@jhsph.edu
JHSPH CTL OWC/MOOC Coordinator:
Ira Gooding
JHU School of Education Intern:
Hilliary Googash
http://tinyurl.com/insidemoocs-aln13
