This document provides instructions for participants on an informational webinar about the NEBHE Developmental Math Demonstration Project. It outlines how participants can access audio and ask questions. It then introduces the presenters and provides an overview of the project goals, which include increasing success in developmental math courses using Khan Academy, driving reform, and generating research findings to expand knowledge of new approaches. Implementation will involve several community colleges piloting the use of Khan Academy in developmental math courses over three years.
Bringing together internal and external students on Blackboard - Brett Fyfiel...
Nebhe nov 19 2012 webinar
1. Welcome to the NEBHE Developmental Math
Demonstration Project Informational Webinar
Participant Instructions
• Participants should be able to access the webinar’s audio through
computer speakers or via telephone. If you have any difficulties
accessing the webinar, please email Daren Follweiler at
dfollweiler@nebhe.org.
• To ensure good audio quality, please mute your audio when you
are not speaking.
• All participants are automatically muted upon joining. To ask a
question, please use one of the following:
• Type into the Question Box
• Raise Your Hand and we will unmute you
• Unmute by clicking the microphone icon
(at the side of your control panel)
NEBHE Developmental Math Demonstration Project
November 19, 2012
2. NEBHE Developmental Math
Demonstration Project
Presenters:
Stafford Peat, Senior Consultant & Project Director
Monnica Chan, Director of Policy&Research
NEBHE Developmental Math Demonstration Project
November 19, 2012
3. Project Overview
• Lumina Foundation awarded NEBHE a three-year grant
to demonstrate the effectiveness of using Khan Academy
math content and online tools to improve Developmental
Math (DM) outcomes
• Over three years NEBHE plans to work with 8-12
community colleges
• Targets post-matriculation traditional and non-traditional
DM courses (multiple levels) and pre-matriculation
programs as pilots
• Supports on-going developmental education reforms
NEBHE Developmental Math
3
Demonstration Project
4. Project Goals
• Increase postsecondary persistence and success in
DM courses using Khan Academy content and
tools
• Drive reform of DM using new no cost tools
• Create regional engagement and a community of
learners to promote and support developmental
education reform
• Generate high-impact data and research findings
and expand knowledge of new approaches
NEBHE Developmental Math
4
Demonstration Project
5. Project Goals (cont.)
• Produce a NEBHE document that reports on
findings and best practices
• Use demonstration project to enhance Khan’s
usefulness for postsecondary education
NEBHE Developmental Math
5
Demonstration Project
6. Project Implementation
November 8th: Letter of
Project Intent sent to
On-going Work Community Colleges
• Research existing NE DM
courses and learning objectives
November 19th: Host
webinar information session
• Map and align Khan online
resources/Common Core to a set December 15th: Letter of
of identified DM models Institutional Intent due
December 21st: 8-12
community colleges selected
and notified
NEBHE Developmental Math
6
Demonstration Project
7. Project Implementation
January: Orientation Webinar
for Demonstration Sites
March: Two-day training for
participating faculty and
institutions
Summer Semester: Project
start-up using Khan
Academy with students
Fall Semester: Full
implementation across all
participating institutions
NEBHE Developmental Math
7
Demonstration Project
8. Incentives for Colleges to Participate
• Khan Academy is no cost vs. other CAI programs
• Khan can be used in a variety of instructional settings and
pedagogical approaches including flipped
classroom/blended learning and to prepare individuals to
take Accuplacer
• NEBHE will provide coaching and ongoing technical
assistance and support
• Each site will receive a grant and financial incentives based
upon number of students served. Funds can be used to
defray costs including data collection or staff travel. Funds
can be also used for other purposes to be determined by
pilot institution
NEBHE Developmental Math
8
Demonstration Project
9. Letter of Project Intent – Highlights
• A single community college, a consortium of colleges or state
community college system is eligible to submit a Letter of
Institutional Intent
• Project funding includes $5K for a single institution and $15K for a
consortium or state system
• An additional $25K in incentives based upon number of students
served will be distributed at the end of the project
• Selection criteria used to select a diverse set of institutions
• Project participation will include a MOU between NEBHE and
participating institutions and a simple data share agreement
• Student level data will be collected at the institution and sent to
NEBHE through a simple data share agreement (no student names
or SS #)
NEBHE Developmental Math
9
Demonstration Project
10. Letter of Project Intent - Highlights
• Letter of Institutional Intent is designed so that
its completion is not burdensome
• Has nine basic questions
• Can be either mailed or emailed
NEBHE Developmental Math
10
Demonstration Project
11. Watch
Practice
Coach
Over 3300 videos
210 in Arithmetic and Pre- Algebra
717 in Algebra
NEBHE Developmental Math
11
Demonstration Project
12. Khan Academy June, 2012
by the numbers
5,072,554 Videos
Unique Visitors
March 25 – April 25 150 million videos
65% watched to date
42.5 1,357,377
35%
Exercises
500 million
million Hours spent on
Khan Academy
March 25 – April 25
problems solved
to date
The Khan Academy iPad app skyrocketed
Unique #1 to be the #1 education
app within days of launching. Over
Visitors Education App 600,000 downloads to date.
To Date
471,000 14,000 2,000 25 13%
Users involved in Total Pilot Active users
Classrooms
coaching groups of size Schools Schools in schools are
10 or greater international
13. About Khan Academy
• Data available via dashboard
Watch for students and teachers on
videos watched and
problems practiced, as well
Practice as goals set and attained.
• Tool for individualized
Coach instruction
NEBHE Developmental Math
13
Demonstration Project
15. Demonstrating Impact
• To what extent, and in what ways, is student
performance in DM impacted by implementation of
new delivery formats and Khan technology-based
tools?
• What are the most (least) effective utilizations of
Khan-based tools?
• What is the impact of the pilot implementation on
DM at the institution?
NEBHE Developmental Math
15
Demonstration Project
16. Data Reporting – Sample Requirements
NEBHE Developmental Math
16
Demonstration Project
17. Data Reporting – Sample Requirements
NEBHE Developmental Math
17
Demonstration Project
With that, I’d like to close by discussing next steps. We will be sending out a communiqué to all community colleges in the region, to gauge and identify interest in participation. In addition, we’ll be putting together an advisory committee comprised of faculty, academic deans, institutional researcher and others for feedback and guidance on this project. Certainly there’s much experience in the room here—if there are nominations, self or otherwise, we’d love to hear them. For now, I’d like to [LOOK AT WATCH] invite questions [OR] invite those interested in learning/discussing this more to approach Stafford or myself—particularly if you’d be interested in participating within the first cohort. We’ll be around after the session. Recognizing that we’ve promised a 4 PM end date—I’d like to turn it over to Michael Thomas for some closing remarks. Thanks so much for your time and attention! Hope to work with you!
As a snapshot of Khan Academy’s reach, this slide provides an overview of various usage metrics as measured by Khan Academy. I’ll only point out a few today—but these slides and a recording will be available. By June 2012, Khan Academy had 42.5 million users and has been used in at least 14,000 classrooms across 2,000 schools. To date, a majority of these classroom have been in the K-12 education space. This demonstration project is one attempt to determine the utility and effectiveness of Khan Academy materials in the postsecondary arena and build a postsecondary learning community, as Stafford mentioned earlier.
Something that I’ve noticed in my work at NEBHE, is a shift towards self-paced, individualized instruction—particularly in the developmental education “redesign” efforts that are happening across region. With Khan Academy, each of the three major activities—watch, practice, coach—have real-time data affiliated with them. This data that’s available, via a “dashboard”, on the videos watched and problems practiced, combined with the open-source nature of Khan Academy materials, could be a useful tool for these redesign efforts (certainly it’s a tool that we’re hearing more and more colleges take advantage of).
To look at this “dashboard” more concretely, this is a screenshot from the Khan Academy Dashboard of my attempts to multiply polynomials through Khan Academy exercises. If I had added one of you, or a professor, as a coach, they would have access to this same data. What you see on the x-axis are the problems that I attempted. The y-axis measures the amount of time I’ve spent working on each problem. Blue bars indicate that the problem was successfully solved. Red bars indicate the problems I’ve solved incorrectly. When I hover over a bar—this is the action that’s depicted in the image you see. I was hovering over the red bar—a few facts come up. The box tells you whether or not I solved the problem, whether or not I used a hint (hints are built into the site), as well as the length of time used to solve the problem. If I were click on this bar, you would see the actual problem and in this case, the hint that I used and when I used it.In addition to this specific data, Khan Academy allows coaches to export aggregate information on a student. Such as, how many videos have been completed? How many exercises has a student solved correctly? How much has been spent watching videos and solving exercises? This aggregate information is especially of interest to us because [CLICK] of our goal to demonstrate the impact of Khan Academy materials.
Specifically: is Khan usage correlated with student success in developmental education? Many of these objectives are outlined on pages 5 – 6 on the copy of the Letter of Intent each of you should have received. I won’t go into as much detail as the project objectives outlined in the table on pages 5 through 6; however, I did want to briefly cover our primary research questions:what extent and in what ways is student performance in developmental math impacted by the implementation of new deliver formats and Khan technology-based tools? [Are more students completing a demonstration course than when Khan Academy materials aren’t used? Are students more successful in subsequent math courses?]What are the most (and least) effective utilizations of Khan-based tools? [If one demonstration site it using Khan Academy in a pre-matriculation model, another is using a flipped classroom model, and another is using blended instruction, are student results the same across each site? Or is one model perhaps more effective than another?]What is the impact of the pilot implementation on developmental math at the institution? [Part of this information will be collected through surveys—we’re planning at the moment to survey both students and faculty their experience in a way that will hopefully collect good feedback and capture any unanticipated effects. Another aspect to this question will be cost, such as the impact Khan Academy has on the cost of delivering a developmental course.]
In building on these research questions, Stafford and I wanted to give you all a visual of how NEBHE plans to go about answering these questions through partnerships with demonstration sites. This slide is thus the first of two showing a sample reporting form. While still a draft, we anticipate that some of the basic information outlined here will be the same as what we’ll ask of demonstration sites during the implementation phase—there are some tentative dates outlined on page 2 of the letter of intent as to when this might occur.Going through this, you’ll see that this form is specific to a semester. We are asking that demonstration sites provide individual student information. As Stafford mentioned, we aren’t asking for names or SSNs. Rather, each demonstration site should be able to generate a random number that would allow us get individual data on some student characteristics—like gender and age—course taking, Khan Academy usage [those middle columns on the number of videos completed, time of videos, time on exercises], and final grade.
This next form is what we anticipate being a longitudinal report that gives information on subsequent student enrollment beyond the demonstration course. You’ll see this outlined in the additional columns to the right. As mentioned in that timeline on page 2 of the LOI, this longitudinal data won’t be due until 2015—close to the end of the project. Something to note is that although both of these forms ask for student level data, we’ll only be reporting in the aggregate. The individual data is needed because if multiple sites are implementing Khan Academy in the same format, then the individual data will allow us to group the data and aggregate the information accordingly. Individual data will also allow us to slice the data differently, say by student age, the number of video completed, etc. The data and feedback that we plan to collect through this project is—If I may be immodest—what makes this project really exciting to me. It’s an opportunity for us to partner as a region and learning community towards greater student success in developmental mathematics. With the level of interest we’ve seen from institutions and states, I think it’ll be a fantastic next few years. Stafford and I are really excited for what we’ll have to review from all of you next month. But rather than gush further as an invisible webinar presenter, let me thank you for listening and now turn it over to all of you for questions and comments.