1. PREDICTIVE ANALYTICS
OVERVIEW / PREVIEW
Matthew D. Pistilli, Ph.D.
Research Scientist
Office of Institutional Research, Assessment & Evaluation
Purdue University
mdpistilli@purdue.edu | @mdpistilli
March 22, 2014
2.
3.
4.
5. Challenge: How do you find the student at risk?
http://www.youthareawesome.com/wp-content/uploads/2010/10/wheres-waldo1.jpg
8. • Actionable intelligence
• Moving research to practice
• Basis for design, pedagogy,
self-awareness
• Changing institutional culture
• Understanding the limitations
and risks
Analytics is
about...
9. DEFINITIONS
Using analytic techniques to help target
instructional, curricular, and support resources to
support the achievement of specific learning goals
(van Bareneveld, Arnold, & Campbell, 2012)
the process of developing actionable insights
through problem definition and the application of
statistical models and analysis against existing
and/or simulated future data (Cooper, 2012)
11. THE BIG QUESTIONS
What can institutions do to improve student
success?
How can institutions help students take
advantage of existing campus resources?
What existing information on campus can be
utilized to better identify students at risk?
How can students become self-aware of what
effort is necessary to be successful in college?
How can analytics make a strategic impact at
scale?
14. THE DEVELOPMENT PROCESS
Basic model constructed
Four institutions to provide data for model building
and testing
Model to be tested, revised, retested, revised, etc.
Anticipated roll out early summer
Anticipated use by institutions this fall
15. MODEL BASICS
5 “buckets” of data
Each bucket weighted
Largest weight placed on current academic performance
and interaction with the course
The buckets:
Student academic effort
Current student performance
Historical student performance
Student demographics
Student behavior out of class
Specific data to be used TBD based on model
testing
18. EXPECTATIONS REALITY
Plug and Play
Immediate results
Solve every problem –
ever!
Universal adoption
Everyone would love it!
Fits, starts, reboots
Mostly long term outcomes
Solve some problems,
create some new problems
Lackluster use
Not everyone loved it
19. RESULTS… A LONG TIME COMING
Immediate
Few
Maybe noticed by instructors
Possibly noticed by help centers
Short term (1 term out)
Some
Based in final grades earned compared to previous terms
Medium term (2 terms out)
A few more
Success of students in sequential courses
One-year retention now available
Long term (3-4 years out)
Retention over time knowable
Graduation rates now available
20. INSTITUTIONAL CHALLENGES
Data in many places, “owned” by many
people/organizations
Different processes, procedures, and regulations
depending on data owner
Everyone can see potential, but all want something
slightly different
Sustainability – “can’t you just…”
Faculty participation is essential
Staffing is a challenge
21. NEW POSSIBILITIES
Using data that exists on campus
Taking advantages of existing programs
Bringing a “complete picture” beyond academics
Focusing on the “Action” in “Actionable Intelligence”
22. PREDICTIVE ANALYTICS
OVERVIEW / PREVIEW
Matthew D. Pistilli, Ph.D.
Research Scientist
Office of Institutional Research, Assessment & Evaluation
Purdue University
mdpistilli@purdue.edu | @mdpistilli
March 22, 2014
Editor's Notes
Large spaceIsolationGroup sizeImpersonal, remote instructorTheater settingHenricus de Alemannia Lecturing his StudentsLaurentius diVoltolina, ca. 1359
Large spaceIsolationGroup sizeImpersonal, remote instructorTheater settingGleason 1986
The third question was the impetus for using Blackboard usage data in the algorithm to predict success. We were already using grades, demographics, and student characteristics to make determinations and evaluations of student success. By adding the Blackboard behavior into the mix, we were able to get a better prediction and understanding of why students weren’t succeeding. By sending messages to students, they learn to become more aware of what they need to do be successful and where the resources on campus exist to help them in that effort. (question 4)This leads back to question 2 – just getting students to simply use what’s provided to them.