"Assessing the Impacts of Citizen Security Projects in Latin America and the Caribbean" Regional Review Conference on the Geneva Declaration on Armed Violence and Development Antigua, Guatemala | 28-30 April 2014

1. Assessing the Impacts of Citizen Security
Projects in Latin America and the
Caribbean
Jennifer Peirce
Institutional Capacity of the State Division(ICS)
Institutions for DevelopmentDepartment (IFD)
GenevaDeclarationRegional ReviewConference
Antigua, Guatemala, April 29, 2014

2. Overview: The IDB & citizen security
• Mandate: Crime and violence are obstacles to social and
economic development.
• Approach: Institutional strengthening – for prevention
• Trend:  Scale and complexity of problems & demands
• Analysis & Strategy:
– 5 pillars: social prevention, situational prevention, police,
criminal justice system, and corrections
– Conceptual Framework: evidence-based models
– Emphasis on building data collection & analysis capacity

3. Protocol for Evidence-Based Crime Prevention
CRIME & VIOLENCE:
• Measured reliably and
precisely by well-audited
systems
• Classified in a way that
supports crime prevention
• Analyzed in multiple units,
or categories
• A power few units cause
most harm
PREVENTION PRACTICES:
• Based on priorities set according
to the degree of harm of the crime
• Theory: a logical sequence -
outputs outcomes + how & why
• Ethics: Not dependent on success,
but rather on respecting human
rights and balancing harms of
coercion vs. harms of crime
• Evidence from impact evaluations
+ local crime and context analyses
• Efficiency: concentration of resources based on predictions
derived from past behavior.
• Effectiveness: determined by field tests that compare programs.
Source: Lawrence Sherman, IDB 2012

4. Monitoring & Evaluation Systems
• Requirements for M&E systems for loan projects:
– Shift from output information to outcome information
– Shift from anecdotal to survey-based data
– Shift toward more quantifiable and comparable data
• Increasing emphasis on experimental & quasi-
experimental methods
…with debates about cost, feasibility, adaptability, innovation
…as complementary to non-experimental and process evaluations
• Investing resources in:
– Dialogue on political & policy implications of M&E + data access
– Administrative data systems
– Baseline data collection; data collection & management tools

5. Examples of Indicators & Data Sources
INDIVIDUAL LEVEL
• Risk factors (e.g. employment,
substanceabuse)
• Knowledge & Skills; Attitudes
& Behaviours
• Access & Use of services
GROUP OR COMMUNITY LEVEL
• Crime data
• Victimization data
• Perceived responsivenessof
institutions
• Fear of specific types of crime
• Attitudes & Behaviours
• Re-offending rate (?)
Data Sources
• Pre/post-activity
instruments
• Community-level
survey (with match)
• National survey
• Admin data (schools,
hospitals, police,
courts, prisons, social
services)
• Caseworker records
• Qualitative: Focus
groups & interviews
• Client mgmt system

6. Data and Information: Advances
• Mapping of details and trends in international funds
– www.seguridadciudadana-centroamerica.org
• Mapping of types of interventions and types of evaluation
and evidence
• More reliability in homicide data, including at city level
• More and better-quality victimization surveys
• Separate methods and modules on domestic violence
• More standardization, reconciliation & comparability of data
– www.seguridadyregion.com
• More resources for data systems, data collection tools and
methods, and inter-institutional coordination

7. IDB Study on Observatories
OBSERVATORIES
• 40%: no record of date,
place, weapon for crimes
• 60%: cannot determine the
time of a crime
• 50%: have data on drivers of
crime, occupations and
relations of victim and
offender
• 25%: do not produce
analysis reports
USE OF DATA
• 33% for policy-making
• 20% for information
dissemination
• 11% for decision-making
• 7% for citizen or public
oversight

8. M & E – Main Challenges
Harmonized indicators & quality data – incl for cities
Regular & systematic victimization surveys
Disaggregation: by community, age, sex, risk group
Individual & incident-specific characteristics
Access to admin data & methods, incl. non-traditional
Databases that can track individual beneficiaries
Common understanding of theory of change & eval implications

9. Lessons Learned
• Rigour vs. adaptability & practicality
• Overcomingpolitical cycles
Building Buy-in
• Balancing disbursement vs. quality
• Unintentional incentives in indicators
Incentives
• Mainstream M&E skills & processes
• Oversight: data collection & application
Capacity
• Process evaluation includes M&E itself
• Clarity on how data will affect decisions
Implementation

10. KEY MESSAGES
• Crime and Violence in LAC is an over-diagnosed
issue, yet we lack basic types of data, data systems,
analysis capacity, and evidence-based models/cases.
• We need to focus on a more granular picture:
individuals and communities, differentiating by
gender, risk group, etc.
• More robust evaluation methods require investment
not only in the evaluation itself, but also in building a
common understanding, trust & buy-in, and follow-
through with all stakeholders.

11. Technical Cooperation: Building & Applying Data
• IDB Citizen Security Initiative: data, evaluation, management
– Mapping: International Cooperation, C&V Observatories
– SES: Regional Standardized Indicators System on Citizen Security
– Violent Crime in LAC cities (with UNODC)
– Urban Citizen Security: data, mapping, evaluation, toolkit
– Caribbean Crime & Victimization Survey + CentAm methodology
– Standardization of Victimization Survey methodology
– Support to SICA Security Strategy + Corrections Model & Baseline
• Research Products:
– Methodology for a prison census
– Violence against Women: costs; incarceration; effects of CCTs
– Impact Evaluations of Citizen Security Policies (Bien Público Regional)
• Regional Policy Dialogue & Training Clinic
• SIN MIEDOS: blogs.iadb.org/sinmiedos & @BID_seguridad