A unique partnership between a computer science and education faculty member has provided an opportunity to create a MOOC for middle and high school teachers designed to provide high quality online professional development in computer science education. The MOOC provided teachers with instruction and support in learning App Inventor, a free web-based application that allows users to learning programming by creating mobile apps. The MOOC was designed around the principles of evidence-based practices in online learning and included design features that addressed social presence and building community with participants. Two unique features of this MOOC included learning communities facilitated by mentor teachers and required Google Hangout sessions. This online workshop, funded by a grant from Google, has allowed the faculty to research learner behavior by quantifying and analyzing analytics in the MOOC. Data was collected through Course Builder, the learning management system, and Google Analytics. Interesting patterns and trends have been identified, that have provided information for online learning in any capacity as well as the development of future MOOCS. For example, the presenters have identified the different ways that certain participants have engaged with course material. The MOOC was designed to allow any participant to have access to the material with accountability measures required of "certificate completers". These participants showed differences in how they accessed and engaged with the material in the course. In this session, the presenters will provide analytics from the MOOC and will ask participants to help engage in analyzing the data for trends. Ideas for using analytics for continuous improvement and course design in online learning will be included in the discussion.

1. Diving into Data
Trends and Patterns from MOOC Analytics
Online Learning Consortium Conference
Orlando, Florida
October, 2014
Chery Takkunen, PhD-School of Education
Jen Rosato, MA -Department of Computer Science/Information Systems
The College of St. Scholastica
www.css.edu
SoTL Commons Conference- 2014- Georgia

2. Goals of the Session
Questions
Background
Course Design
Data
Implications
College of St. Scholastica, www.css.edu

3. The College of St. Scholastica
College of St. Scholastica, www.css.edu
Location
College
Growth Strategy 10% Gr

4. Background
CS + EDU= Unique Partnership
Computer Science Education
Professional Development Workshops
Experience in Online Teaching and Learning
Grants= TAG, Google CS4HS, Local/Regional
*New- 2015- National Science Foundation
*New- Certificate in Computer Science Education
College of St. Scholastica, www.css.edu

5. CS4HS is an annual grant program promoting computer science education
worldwide by connecting educators to the skills and resources they need to
teach computer science & computational thinking concepts in fun and relevant
ways. Traditionally, these have been in-person workshops.
College of St. Scholastica, www.css.edu

6. MOOC defined *
A Massive Open Online Course (MOOC; English pronunciation: /muːk/) is
an online course aimed at unlimited participation and open access via the
web. In addition to traditional course materials such as videos, readings,
and problem sets, MOOCs provide interactive user format that help build a
community for students, professors, and teaching assistants (TAs). MOOCs
are a recent development in distance education.
Wikipedia.org: http://en.wikipedia.org/wiki/Massive_open_online_course
College of St. Scholastica, www.css.edu

7. http:mfeldstein.comemerging_student_patterns_in_moocs_graphical_view/
Phil Hill
College of St. Scholastica, www.css.edu

8. Conceptual Framework
Community of Inquiry
The Community of Inquiry model. Garrison, R., Anderson, T, Archer, W. and Rourke, L et al. (2007).
College of St. Scholastica, www.css.edu

9. Teacher Professional Development
1) Long-term and intensive
2) Clear outcomes
3) Collaboration and community
4) Use of online tools for effective PD
5) Five core features:
a) content and pedagogy
b) consistency with reforms
c) coherence with educational goals
d) active learning- reflection and inquiry
e) aligned with standards
College of St. Scholastica, www.css.edu

10. Applying Principles to CS4HS Course
MOOC-like Design
LMS- Course Builder
Participation Levels
Building Community
Professional Learning Communities (PLC)
Mentors
Google Hangouts
Google Hangout on Air with guest speakers
Discussion forums
Narrated presentations
College of St. Scholastica, www.css.edu

11. Participant Levels
Casual Participant
vs.
Certificate Completer
College of St. Scholastica, www.css.edu

12. App Inventor
College of St. Scholastica, www.css.edu
Dave Wolber
Professor, Computer Science
University of San Francisco

13. Course
Unit Page Types:
1. Objectives
2. CS Unplugged*
3. Hangout On Air
4. App Inventor Tutorial, Part 1*
5. App Inventor Tutorial, Part 2*
6. Pedagogy*
7. Group Hangouts
8. Discussion
9. Additional Resources
*Included activities (formative assessments)
College of St. Scholastica, www.css.edu

14. Analytics Data
...the discovery and communication of meaningful patterns in data
GA - Google Analytics
GCB - Google Course Builder
GG - Google Groups
College of St. Scholastica, www.css.edu
http://en.wikipedia.org/wiki/Analytics

15. Participants
We were planning on 50
Over 400 participants
Over 40 states
Over 40 countries
College of St. Scholastica, www.css.edu

16. Google Analytics (GA)
Other: Canada, Puerto Rico, Tunisia
College of St. Scholastica, www.css.edu
90 % of visits from the US
*Kristen Donahue, Kassandra Quick & Alvaro Hernandez-Feris

17. GCB: Registration Data
College of St. Scholastica, www.css.edu

18. http:mfeldstein.comemerging_student_patterns_in_moocs_graphical_view/
Phil Hill
College of St. Scholastica, www.css.edu

19. GCB: Student Progress
College of St. Scholastica, www.css.edu

20. GA: Student Pageviews
College of St. Scholastica, www.css.edu
What questions does this
raise? What else would you
like to know? How would you
investigate?

21. GCB: Completion Rates
College of St. Scholastica, www.css.edu

22. GCB: Assessment Progress
College of St. Scholastica, www.css.edu

23. GA: Page Types
College of St. Scholastica, www.css.edu
What questions does this
raise? What else would you
like to know? How would you
investigate?

24. GG: Discussion Post Data
College of St. Scholastica, www.css.edu

25. GG: Thread Interactions
College of St. Scholastica, www.css.edu
Avera
ge
Total
Replie
s
3.9 67
Views 32.5 552
What questions does this
raise? What else would you
like to know? How would you
investigate?

26. GG: Thread Interactions
College of St. Scholastica, www.css.edu

27. Summary of Trends
Completion rates follow the typical MOOC behavior (w/higher rates in ours)
Combine analytics and registration data for a more complete picture
Completion rates increase later in course (students are more invested)
Assessment participation follows completion rates
Discussion participation follows completion rates
Some content appears to be more engaging than others
(pedagogy & guest speakers vs unplugged lessons)
College of St. Scholastica, www.css.edu

28. Continuous Improvement
How have we used the data from summer 2013?
Multiple certificate levels
Online office hours & Community manager (contact for those not in a PLC)
Paid close attention to the Unit 2 drop off
Unit 2 lighter than following
Mentors checked in with all participants in their PLC groups
Monitored forum more closely
College of St. Scholastica, www.css.edu
Page types
Kept a separate pedagogy page in units
Quizly questions AI tutorial pages

29. Recommendations
Analytics Recommendations:
Know what data you can get from where
(GA vs LMS and other tools)
Learn what the data means
Pilot the analytics before the course
Set time limits for collecting data
General Recommendations:
Be intentional
Give teachers a range of options
Match intent with support levels
Plan design around learner behavior
Provide opportunities to informally interact
w/content & each other
Require discussion - tempers the superposter
phenomenon
College of St. Scholastica, www.css.edu