Presentation to the Board on evidence based programming

1. The Case for Evidence-Based
Programs
Del Elliott, Ph.D.
Center for the Study and Prevention of
Violence
University of Colorado, Boulder

2. Evidence: Something that
furnishes or tends to furnish
proof (Webster)

3. Reviewing the Scientific Evidence:
The Blueprints Strategy
 A systematic review of individual program
evaluations to identify violence, drug abuse and
delinquency prevention programs that meet a
high scientific standard of effectiveness
 Individual programs meeting this standard are
certified as Model or Promising evidence-based
programs
 Only Model programs are considered eligible for
widespread dissemination

4. Scientific Standard for
Certification as Evidence-Based*
 Experimental Design/RCT
 Statistically significant & substantive positive
effect
 Effect sustained for at least 1 year post-
intervention
 At least 1 external replication with RCT/QED
 RCT’s/QED’s adequately address threats to
internal validity
 No known health-compromising side effects
*Adapted from Blueprints and Working Group for the Federal Collaboration on What Works , 2004.
www.ncjrs.gov/pdffiles1/nij/220889.pdf

5. Evidence-Based
Classification*
 Model: Meets all standards
I. Model: Meets all standards
 II. Effective: Internal rather than external
RCT replication(s)
 III. Promising: Lacks any RCT replication
 IV. Inconclusive: Contradictory findings or
non-sustainable effects
 V. Ineffective: Meets all standards but with no
statistically significant effects
 VI. Harmful: Meets all standards but with
negative main effects or serious side effects
 VII Insufficient Evidence: All others
*Adapted from Blueprints and Hierarchical Classification Framework for Program
Effectiveness, Working Group for the Federal Collaboration on What Works,
2004.

6. Review of Evaluation Evidence*
Crime and Drug Prevention Programs
 Most Programs Have No Credible Evaluation
 Those With Credible Evaluations:
 Most Don’t Work
 40 to 45 Clearly Work or Have Promise
 A Few Appear to be Harmful
 Most Model Programs Don’t Have Capacity to
Go to Scale
*Over 900 programs. Center for the Study and Prevention of Violence

7. What Doesn’t Work!
*Negative Effects
 Waivers to Adult (Criminal Courts)*
• Boot Camps
• Traditional and New DARE*
• GGI and Positive Peer Culture
• Gun Buyback Programs*
• Peer Counseling Programs
• Summer Job Programs (At Risk Youth)
Adapted from Sherman et al., 2002; Aos et al.,2004; Sloboda et al, 2009; Redding,
2008; Sparks, 2010; James-Burdumy et al, 2005.

8. What Doesn’t Work!
*Negative Effects
• Neighborhood Watch
• Arrests of Unemployed Domestic
• Abusers*
Scared Straight*
• Shock Probation/Parole*
• Home Detention with Electronic Monitoring
• Casework/Counseling*
• Character Education
.

9. What Doesn’t Work!
*Negative Effects
 Intensive Supervised Probation/Parole
(A,J)
 Case Management Substance Abuse
Program (A)
 Community Policing-Increasing
Information Flow Strategy
 Diversion from Court to Job Training (A)
 Drug Market Arrests

10. What Doesn’t Work!
*Negative Effects
 Juvenile Wilderness Programs
 Peer Counseling Programs
 Short-Term Non-Residential Training
 Urine Testing
 Job Training Partnership Act
 21st Century Community learning Centers*
 Rehabilitation Programs –Non Directive
Unstructured Counseling

11. Evidence-Based Programs
Effective Prevention and
Intervention Programs

12. Evidence-Based Model Programs
(Blueprint Programs)
Nurse Home Visitation Midwestern Prevention Project
PATHS Functional Family Therapy
Bullying Prevention Program Multisystemic Therapy
Big Brothers Big Sisters Project TND
Life Skills Training Treatment Foster Care
Incredible Years

13. The Incredible Years
Description: Parent, teacher, and child training to promote
child emotional and social competence
Target: Children ages 2-8
Cost: $1,300 BASIC, $775 ADVANCE, $995 SCHOOL,
$975 Child Training, $1,250 Teacher
Contact: Carolyn Webster-Stratton
University of Washington, School of Nursing
Department of Family and Child Nursing
1107 NE 45th St., Suite 305
Seattle, WA 98105-4631
(206) 543-6010

14. The Incredible Years
Evidence of Effect
Parent Program:
• Reduced conduct problems.
• Increases in positive affect and compliance to parental
commands.
Teacher Program:
• Reductions in peer aggression in the classroom.
• Increases in positive affect and cooperation with teachers,
positive interactions with peers school readiness and
engagement with school activities.
Child Program:
• Reductions in conduct problems at home and school.
• Improved cognitive problem-solving strategies with peers.

15. Multisystemic Therapy
Description: Multidimensional home-based family therapy
Target: Chronic and violent offenders, ages 12-17
Cost: $4,500 per youth
Contact: Scott Henggeler, Ph.D.
Program Designer
Medical University of South Carolina
For training and implementation questions:
Keller Strother,
MST, Inc.,
P.O. Box 21269
Charleston, SC 29413
843-843-856-8226

16. Multisystemic Therapy
Evidence of Effect
• Reductions in rearrests 25-70%.
• Reductions in self-reported criminal
behavior.
• Decreased drug use.
• Improvements in family functioning.
• Decreased mental health problems.

17. Life Skills Training
Description: Drug prevention
Target: All middle/junior high school (grades 6/7) students
Cost: $625 per 30 students for 3 years (no training incl)
Contact: Gilbert Botvin, Ph.D., Director
Institute for Prevention Research
Cornell University Medical College
411 East 69th Street, KB-201
New York, NY, 10021
212-746-1270

18. Life Skills Training
Evidence of Effect
• Reduces tobacco, alcohol, and marijuana
use 50-75%.
• Effects maintained through grade 12.
• Cuts polydrug use up to 66%.
• Reduces pack-a-day smoking by 25%.
• Decreases use of inhalants, narcotics,
and hallucinogens.

19. Promising Programs
Programs Effective at Reducing
Delinquency/Crime, Violence & Drug Use
ATLAS
BASICS (Brief Alcohol Intervention with College Students)
CASASTART (formerly Children at Risk Program)
Behavioral Monitoring and Reinforcement Program
Brief Strategic Family Therapy
CTC (Communities that Care)
Guiding Good Choices
Iowa Strengthening Families Program
Project Northland
Preventive Treatment Program (Montreal)
Raising Healthy Children
Seattle Social Development Project
Strong African American Families

20. Promising Programs
Programs Effective at Reducing Childhood
Precursors of Crime (Predelinquent Aggression)
FAST Track
Good Behavior Game
I Can Problem Solve
Linking the Interests of Families and
Teachers (LIFT)
Perry Preschool
Triple P- Positive Parenting Program

21. Demonstrated Outcomes of EB
Prevention Programs
 Improved Grades
 Higher Rates of Next Grade Promotion
 Increased GPA
 Higher Graduation Rates
 Better Reading, Math and Writing Skills
 Higher Standardized Test Scores
 Increased Credits Earned
 Higher Child Development Levels (young
Children
CSAP Model Programs’ Academic Outcome Measures and Proven Outcomes,
2002

22. Demonstrated Outcomes of EB
Prevention Programs Con’td
 Lower Absenteeism
 Lower High School Dropout
 Higher Parent Involvement
 Fewer Suspensions
 Fewer Retentions
 Fewer Special Education Referrals
 Fewer School Behavioral Incidents
 Improved Social Competence
 Higher Participation in After School Activities
 CSAP Model’s Academic Outcome Measures and Proven Outcomes, 2002

23. Benefit-to-Cost Ratios:
Selected Blueprint Programs*
 Program Ratio Outcome
 BBBS $1.01 None
 FFT $13.25 Crime reduction
 LST $25.61 Drug reduction
 MPP $5.29 Drug reduction
 MST $2.64 Drug reduction
 MTFC $10.88 Crime reduction
 NFP $2.88 Crime reduction
*Washington Institute for Public Policy (2004)

24. Benefit-to-Cost Ratios:
Selected Other Programs*
 Program Ratio Outcome
 DARE $0.0 None
 Boot Camps $0.0 None
 Head Start $0.23 None
 Even Start $0.0 None
 Healthy Families $0.34 None
 Scared Straight $-203.51 Increases crime
*Washington Institute for Public Policy (2004)

25. Annual Taxpayer Costs & Benefits:
Forecast with Moderate Portfolio of
Evidence-Based Programs
$450
$400
$350
$300
$250 Benefits
$200 Cost
$150
$100
$50
$0
2008 2012 2016 2020 2024 2028 2030

26. Redirection Project*
State of Florida
 Felony conviction/adjudication rates, 1
year post completion: Redirect 19% vs
Comparison 25%
 Felony referral/arrest: Redirect 32% vs
Comparison 43%
 Cost savings: Redirection costs $28,500
less per offender
 Cost savings: Nearly $40 Million [2004
-2008].
* MST &FFT. Comparison involves current residential programs

27. Feasibility Example
 Cost to provide every student in U.S. a
model drug prevention program like LST is
$550 million per year
 Current national drug control spending is
approximately $40 billion per year
 This represents 1.5% of the current drug
control spending

28. “…both benefit-cost analyses
and meta analyses have proven
quite appealing in public policy:
they lead to simple , quantified
results of general application,
can be readily remembered, and
are not hindered by multiple
caveats”. Shadish et al., 1991

29. The Ideal Evidence-Based
Program*
 Addresses major risk/protection factors that are
manipulatable with substantively significant effect sizes
 Relatively easy to implement with fidelity
 Causal and change rationales and services/treatments
are consistent with the values of professionals who will
use it
 Keyed to easily identified problems
 Inexpensive or positive cost-benefit ratios
 Can influence many lives or have life-saving types of
effects on some lives
 *Adapted from Shadish, Cook and Leviton, 1991:445.

30. Summary: Benefits of Using
Evidence-Based Programs and
Practices
 Ethics-Avoiding Harmful Effects
 Stronger & More Consistent Positive
Outcomes
 Cost Savings to Taxpayers
 Improving the Well-being of Our
Children

31. THANK YOU
Center for the Study and Prevention
of Violence
Phone: 303-492-1032
Web Site: www.colorado.edu/cspv