SlideShare ist ein Scribd-Unternehmen logo
1 von 11
Downloaden Sie, um offline zu lesen
Economist, FAO
From Challenges to Solutions: A
Comprehensive Guide for Conducting
Impact Evaluations in Fragile Settings
Jeanne Pinay
Jeanne Pinay and Marco d’Errico
#2023 AGRODEP CONFERENCE
Motivation
• 2022: 274 million people require humanitarian assistance and protection
• 260 million people (95 percent) living in fragile areas (UN OCHA, 2022)
Fragile countries have attracted
increased attention from actors of
the humanitarian and
development world
• 2021: total international humanitarian assistance = USD 31.3 billion (GHA,
2022)
Enormous amounts of
humanitarian aid have been
injected into crisis countries
Policymakers need to identify
interventions that work best to
better assist people in need, and
more efficiently allocate funds
Rigorous IE in fragile settings is
necessary…
… because rigorous IE in fragile settings is challenging!
… but not well-developed
Why?
#2023 AGRODEP CONFERENCE
Contribution and target audience
• Based on the literature (esp. Puri et al., 2017; Bakrania et al., 2021) and new evidence
from FAO case studies
• Contributes to the literature on aid effectiveness and impact evaluation methods
• Target audience: International and local researchers, practitioners, and donors involved
in impact evaluations in fragile settings
Challenges
Implications on data
quality and validity of
impact evaluation
Solutions
Identify the challenges encountered
Determine appropriate solutions to maintain
design, data quality, and credibility of the IE
• Provide guidance to researchers
and local actors working on impact
evaluation in fragile contexts to:
#2023 AGRODEP CONFERENCE
Definition of fragile settings and FAO
case studies
• OECD (2022): Fragility is the
combination of exposure to risk and
insufficient coping capacity of the
state, system and/or communities to
manage, absorb or mitigate those risks
• Based on the assessment of the
respective balance between risk and
coping capacities, contexts are
categorized as extremely fragile,
fragile, or ‘rest of the world’
#2023 AGRODEP CONFERENCE
Challenges
Practical
• Complex settings or
unpredictability
• Burkina Faso, Mali,
Niger, Nigeria,
Somalia
• Need for speed or time
constraints
• Burkina Faso, Gaza
Strip
• Multiplicity of actors
• High co-variability or
invalid counterfactual
Ethical
• Do no harm
• Gaza Strip
• Randomization
and ethics
• Myanmar
Methodological
• Selection bias
• All FAO case studies
• Information bias
• South Sudan
• Contamination bias
• Somalia
• Attrition
• Somalia, Nigeria, …
Local capacity
• Misidentification of the
treated
• Pakistan
• Misunderstanding on IE
design
• Madagascar
• No HHID
• Gaza Strip
• No experience in tracking
HHs
• Remote trainings and lower
degree of supervision
#2023 AGRODEP CONFERENCE
Implications
Invalid
counterfactual
Measurement
error
Attrition
#2023 AGRODEP CONFERENCE
Solutions
Invalid
counterfactual
• Quasi-experimental methods
• IV, RDD, DD, Matching
• Combining primary with
secondary data
• Somalia, Madagascar
• Alternative designs for
randomization
• Phase-in randomization
• Factorial design
• Randomization at higher
levels (e.g., village)
Measurement error
• Before fieldwork
• Pilot testing
• Reduce questionnaire
• During fieldwork
• CAPI
• Mixed methods
• Gaza Strip
• After fieldwork
• Correct measurement
error
• Secondary data
• Nigeria
Attrition
• Before fieldwork
• Regular assessment of security
situation
• CATI
• Diversified C group
• Oversampling
• Clear communication
• During fieldwork
• Tracking techniques
• Nigeria
• After fieldwork
• Attrition tests
• Correct attrition (IPW, CiC, bounding)
#2023 AGRODEP CONFERENCE
Conclusions
• Rigorous impact evaluation in fragile settings is challenging but possible, and
necessary  cannot wait for the right setting to generate evidence!
• Strike a balance between
• Rigor of the evaluation vs. assistance of people in need
• Rigor of the evaluation vs. costs
• Collaboration among humanitarian agencies, donors and local actors
• Agree upon a common research agenda
• Flexible and adjustable budget
• Combine data collection efforts
Investing substantial public resources in programs
whose effectiveness is unknown is unethical!
(Gertler et al., 2016)
• Program adaptability and flexibility
• Focus time and resources on less
but more reliable evidence
#2023 AGRODEP CONFERENCE
Need to identify
innovative and low-
cost techniques
• research designs
• statistical methods
• survey methods
Way forward
Geospatial data
Satellite data
Remote-sensing data
#2023 AGRODEP CONFERENCE
References
• Bakrania, S., Balvin, N., Daidone, S. & De Hoop, J. (2021). Impact Evaluation in
Settings of Fragility and Humanitarian Emergency. Office of Research – Innocenti
Discussion Paper DP-2021-02 February 2021
• Global Humanitarian Assistance. (2022) GHA Report 2022.
https://devinit.org/documents/1193/GHA2022_Digital_v8_DknWCsU.pdf
• OECD. (2022). States of Fragility 2022. OECD Publishing.
https://doi.org/10.1787/c7fedf5e-en
• Puri, J., Aladysheva, A., Iversen, V., Ghorpade, Y. and Brück, T. (2017). Can rigorous
impact evaluations improve humanitarian assistance? Journal of Development
Effectiveness, 9:4, 519-542, DOI: 10.1080/19439342.2017.1388267
• UN OCHA. (2022). Global Humanitarian Overview 2022, United Nations Office for
the Coordination of Humanitarian Affairs (UN OCHA), New York,
https://gho.unocha.org/
THANK YOU

Weitere ähnliche Inhalte

Ähnlich wie Conducting Impact Evaluations in Fragile Settings

Strategic Coordination and Integration
Strategic Coordination and IntegrationStrategic Coordination and Integration
Strategic Coordination and IntegrationMEASURE Evaluation
 
OECS GIS Needs Assessment Summary - June 2016
OECS GIS Needs Assessment Summary - June 2016OECS GIS Needs Assessment Summary - June 2016
OECS GIS Needs Assessment Summary - June 2016Rhea Yaw Ching
 
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...The Transfer Project
 
3 karen sudmeier rieux pedrr workshop synthesis session 3
3 karen sudmeier rieux pedrr workshop synthesis session 33 karen sudmeier rieux pedrr workshop synthesis session 3
3 karen sudmeier rieux pedrr workshop synthesis session 3unuehs
 
How are we building weather and climate resilience for the next decade?
How are we building weather and climate resilience for the next decade?How are we building weather and climate resilience for the next decade?
How are we building weather and climate resilience for the next decade?Water, Land and Ecosystems (WLE)
 
Value Chain Development and The Poor
Value Chain Development and The Poor   Value Chain Development and The Poor
Value Chain Development and The Poor IFPRI-PIM
 
Graduation Model Programme Burundi SEEP
Graduation Model Programme Burundi SEEPGraduation Model Programme Burundi SEEP
Graduation Model Programme Burundi SEEPalesbin
 
Sunil k regmi
Sunil k regmiSunil k regmi
Sunil k regmiClimDev15
 
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge Gaps
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge GapsDeveloping Gender-Responsive NDCs: Meeting Capacity and Knowledge Gaps
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge GapsWorld Resources Institute (WRI)
 
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...AIDSTAROne
 
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...NAP Global Network
 
Improving Use of Evidence in Decision Making on Agroforestry Interventions
Improving Use of Evidence in Decision Making on Agroforestry InterventionsImproving Use of Evidence in Decision Making on Agroforestry Interventions
Improving Use of Evidence in Decision Making on Agroforestry InterventionsWorld Agroforestry (ICRAF)
 
Assaad Bridging Evidence & Policy Talk.pptx
Assaad Bridging Evidence & Policy Talk.pptxAssaad Bridging Evidence & Policy Talk.pptx
Assaad Bridging Evidence & Policy Talk.pptxAhmed Ali
 
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...Omobolanle3
 
Programme planning and evaluation in extension work
Programme planning and evaluation in extension workProgramme planning and evaluation in extension work
Programme planning and evaluation in extension workDegonto Islam
 
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...Global Globin 2020 Challenge: Developing capacity for gene variant data shari...
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...Human Variome Project
 

Ähnlich wie Conducting Impact Evaluations in Fragile Settings (20)

USING EQUIST FOR BOTTLENECK ANALYSIS
USING EQUIST FOR BOTTLENECK ANALYSIS�USING EQUIST FOR BOTTLENECK ANALYSIS�
USING EQUIST FOR BOTTLENECK ANALYSIS
 
Strategic Coordination and Integration
Strategic Coordination and IntegrationStrategic Coordination and Integration
Strategic Coordination and Integration
 
OECS GIS Needs Assessment Summary - June 2016
OECS GIS Needs Assessment Summary - June 2016OECS GIS Needs Assessment Summary - June 2016
OECS GIS Needs Assessment Summary - June 2016
 
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...
Evaluation of Mozambique’s Child Grant: A cash + care intervention to reduce ...
 
3 karen sudmeier rieux pedrr workshop synthesis session 3
3 karen sudmeier rieux pedrr workshop synthesis session 33 karen sudmeier rieux pedrr workshop synthesis session 3
3 karen sudmeier rieux pedrr workshop synthesis session 3
 
How are we building weather and climate resilience for the next decade?
How are we building weather and climate resilience for the next decade?How are we building weather and climate resilience for the next decade?
How are we building weather and climate resilience for the next decade?
 
Value Chain Development and The Poor
Value Chain Development and The Poor   Value Chain Development and The Poor
Value Chain Development and The Poor
 
Module 5: Consideration of social aspects for inclusive development
Module 5: Consideration of social aspects for inclusive developmentModule 5: Consideration of social aspects for inclusive development
Module 5: Consideration of social aspects for inclusive development
 
Graduation Model Programme Burundi SEEP
Graduation Model Programme Burundi SEEPGraduation Model Programme Burundi SEEP
Graduation Model Programme Burundi SEEP
 
The climate-smart village
The climate-smart villageThe climate-smart village
The climate-smart village
 
Sunil k regmi
Sunil k regmiSunil k regmi
Sunil k regmi
 
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge Gaps
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge GapsDeveloping Gender-Responsive NDCs: Meeting Capacity and Knowledge Gaps
Developing Gender-Responsive NDCs: Meeting Capacity and Knowledge Gaps
 
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...
Skills-Building Workshop: Key Findings for Guiding Programming For MARPs In M...
 
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...
Integrated Vulnerability Assessment (IVA): Status overview and role in M&E of...
 
Improving Use of Evidence in Decision Making on Agroforestry Interventions
Improving Use of Evidence in Decision Making on Agroforestry InterventionsImproving Use of Evidence in Decision Making on Agroforestry Interventions
Improving Use of Evidence in Decision Making on Agroforestry Interventions
 
Assaad Bridging Evidence & Policy Talk.pptx
Assaad Bridging Evidence & Policy Talk.pptxAssaad Bridging Evidence & Policy Talk.pptx
Assaad Bridging Evidence & Policy Talk.pptx
 
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...
Fragility, Conflict and Migration (FCM) & National Policies and Strategies (N...
 
Programme planning and evaluation in extension work
Programme planning and evaluation in extension workProgramme planning and evaluation in extension work
Programme planning and evaluation in extension work
 
Learning Alliances in Jordan
Learning Alliances in JordanLearning Alliances in Jordan
Learning Alliances in Jordan
 
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...Global Globin 2020 Challenge: Developing capacity for gene variant data shari...
Global Globin 2020 Challenge: Developing capacity for gene variant data shari...
 

Mehr von AKADEMIYA2063

AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56] - Read-Only (1).pptx
AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56]  -  Read-Only (1).pptxAKADEMIYA2063_SE-GH2forEnergyTransition_MD[56]  -  Read-Only (1).pptx
AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56] - Read-Only (1).pptxAKADEMIYA2063
 
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptx
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptxAugustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptx
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx
 Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx
Mr. Kenao Lao - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptx
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptxMr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptx
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx
 Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx
Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptx
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptxDr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptx
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptx
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptxDr. Getaw Tadesse - 2023 ReSAKSS Conference.pptx
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptx
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptxDr. Fatima Kareem - 2023 ReSAKSS Conference.pptx
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptx
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptxDr. Fleur Wouterse - 2023 ReSAKSS Conference.pptx
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptx
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptxDr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptx
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Agnes Quisumbing - 2023 ReSAKSS Conference
 Dr. Agnes Quisumbing - 2023 ReSAKSS Conference Dr. Agnes Quisumbing - 2023 ReSAKSS Conference
Dr. Agnes Quisumbing - 2023 ReSAKSS ConferenceAKADEMIYA2063
 
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptx
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptxLea Magne-Domgho - 2023 ReSAKSS Conference.pptx
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Prince Agyemang - 2023 ReSAKSS Conference.pptx
Prince Agyemang - 2023 ReSAKSS Conference.pptxPrince Agyemang - 2023 ReSAKSS Conference.pptx
Prince Agyemang - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptx
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptxDr. Paul Guthiga - 2023 ReSAKSS Conference.pptx
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptx
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptxDr. John Ulimwengu - 2023 ReSAKSS Conference.pptx
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Silver Nanema - 2023 ReSAKSS Conference.pptx
Silver Nanema - 2023 ReSAKSS Conference.pptxSilver Nanema - 2023 ReSAKSS Conference.pptx
Silver Nanema - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference AKADEMIYA2063
 
Ms. Julie Collins - 2023 ReSAKSS Conference.pptx
Ms. Julie Collins - 2023 ReSAKSS Conference.pptxMs. Julie Collins - 2023 ReSAKSS Conference.pptx
Ms. Julie Collins - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptx
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptxDr. Ousmane Badiane - 2023 ReSAKSS Conference.pptx
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptx
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptxDr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptx
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptxAKADEMIYA2063
 

Mehr von AKADEMIYA2063 (20)

AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56] - Read-Only (1).pptx
AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56]  -  Read-Only (1).pptxAKADEMIYA2063_SE-GH2forEnergyTransition_MD[56]  -  Read-Only (1).pptx
AKADEMIYA2063_SE-GH2forEnergyTransition_MD[56] - Read-Only (1).pptx
 
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptx
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptxAugustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptx
Augustin Wambo Yamdjeu - 2023 ReSAKSS Conference.pptx
 
Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx
 Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx
Mr. Kenao Lao - 2023 ReSAKSS Conference.pptx
 
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptx
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptxMr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptx
Mr. Wondwosen Tefera - 2023 ReSAKSS Conference.pptx
 
Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx
 Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx
Dr. Keith Fuglie - 2023 ReSAKSS Conference.pptx
 
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptx
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptxDr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptx
Dr. Greenwell Matchaya et al - 2023 ReSAKSS Conference.pptx
 
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptx
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptxDr. Getaw Tadesse - 2023 ReSAKSS Conference.pptx
Dr. Getaw Tadesse - 2023 ReSAKSS Conference.pptx
 
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptx
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptxDr. Fatima Kareem - 2023 ReSAKSS Conference.pptx
Dr. Fatima Kareem - 2023 ReSAKSS Conference.pptx
 
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptx
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptxDr. Fleur Wouterse - 2023 ReSAKSS Conference.pptx
Dr. Fleur Wouterse - 2023 ReSAKSS Conference.pptx
 
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptx
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptxDr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptx
Dr. Ebenezer M. Kwofie - 2023 ReSAKSS Conference.pptx
 
Dr. Agnes Quisumbing - 2023 ReSAKSS Conference
 Dr. Agnes Quisumbing - 2023 ReSAKSS Conference Dr. Agnes Quisumbing - 2023 ReSAKSS Conference
Dr. Agnes Quisumbing - 2023 ReSAKSS Conference
 
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptx
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptxLea Magne-Domgho - 2023 ReSAKSS Conference.pptx
Lea Magne-Domgho - 2023 ReSAKSS Conference.pptx
 
Prince Agyemang - 2023 ReSAKSS Conference.pptx
Prince Agyemang - 2023 ReSAKSS Conference.pptxPrince Agyemang - 2023 ReSAKSS Conference.pptx
Prince Agyemang - 2023 ReSAKSS Conference.pptx
 
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptx
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptxDr. Paul Guthiga - 2023 ReSAKSS Conference.pptx
Dr. Paul Guthiga - 2023 ReSAKSS Conference.pptx
 
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptx
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptxDr. John Ulimwengu - 2023 ReSAKSS Conference.pptx
Dr. John Ulimwengu - 2023 ReSAKSS Conference.pptx
 
Silver Nanema - 2023 ReSAKSS Conference.pptx
Silver Nanema - 2023 ReSAKSS Conference.pptxSilver Nanema - 2023 ReSAKSS Conference.pptx
Silver Nanema - 2023 ReSAKSS Conference.pptx
 
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference
Mrs. Tsitsi Makombe - 2023 ReSAKSS Conference
 
Ms. Julie Collins - 2023 ReSAKSS Conference.pptx
Ms. Julie Collins - 2023 ReSAKSS Conference.pptxMs. Julie Collins - 2023 ReSAKSS Conference.pptx
Ms. Julie Collins - 2023 ReSAKSS Conference.pptx
 
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptx
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptxDr. Ousmane Badiane - 2023 ReSAKSS Conference.pptx
Dr. Ousmane Badiane - 2023 ReSAKSS Conference.pptx
 
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptx
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptxDr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptx
Dr Jean Paul Latyr FAYE - 2023 ReSAKSS Conference.pptx
 

Kürzlich hochgeladen

2024: The FAR, Federal Acquisition Regulations - Part 22
2024: The FAR, Federal Acquisition Regulations - Part 222024: The FAR, Federal Acquisition Regulations - Part 22
2024: The FAR, Federal Acquisition Regulations - Part 22JSchaus & Associates
 
Item # 4&5 - 415 & 423 Evans Ave. Replat
Item # 4&5 - 415 & 423 Evans Ave. ReplatItem # 4&5 - 415 & 423 Evans Ave. Replat
Item # 4&5 - 415 & 423 Evans Ave. Replatahcitycouncil
 
Madison Cat Project Foster Training - Lesson 1
Madison Cat Project Foster Training - Lesson 1Madison Cat Project Foster Training - Lesson 1
Madison Cat Project Foster Training - Lesson 1KelleyWasmund
 
Everyone, everywhere has the right to good mental health.
Everyone, everywhere has the right to good mental health.Everyone, everywhere has the right to good mental health.
Everyone, everywhere has the right to good mental health.Christina Parmionova
 
Item # 6 - TBG Partners Landscape Architectural Design Services
Item # 6 - TBG Partners Landscape Architectural Design ServicesItem # 6 - TBG Partners Landscape Architectural Design Services
Item # 6 - TBG Partners Landscape Architectural Design Servicesahcitycouncil
 
PPT Item # 7 - Demolition & Replacement Structure Processes
PPT Item # 7 - Demolition & Replacement Structure ProcessesPPT Item # 7 - Demolition & Replacement Structure Processes
PPT Item # 7 - Demolition & Replacement Structure Processesahcitycouncil
 
2024 ECOSOC YOUTH FORUM -logistical information - United Nations Economic an...
2024 ECOSOC YOUTH FORUM -logistical information -  United Nations Economic an...2024 ECOSOC YOUTH FORUM -logistical information -  United Nations Economic an...
2024 ECOSOC YOUTH FORUM -logistical information - United Nations Economic an...Christina Parmionova
 
Pope Francis Teaching: Dignitas Infinita- On Human Dignity
Pope Francis Teaching: Dignitas Infinita- On Human DignityPope Francis Teaching: Dignitas Infinita- On Human Dignity
Pope Francis Teaching: Dignitas Infinita- On Human DignityEnergy for One World
 
Conscience. Love. Peace - International Day of Conscience 2024.
Conscience. Love. Peace  - International Day of Conscience 2024.Conscience. Love. Peace  - International Day of Conscience 2024.
Conscience. Love. Peace - International Day of Conscience 2024.Christina Parmionova
 
Actions to take at the global, national and local levels to realize the right...
Actions to take at the global, national and local levels to realize the right...Actions to take at the global, national and local levels to realize the right...
Actions to take at the global, national and local levels to realize the right...Christina Parmionova
 
PPT Item # 2 -- Announcements Powerpoint
PPT Item # 2 -- Announcements PowerpointPPT Item # 2 -- Announcements Powerpoint
PPT Item # 2 -- Announcements Powerpointahcitycouncil
 
2024: The FAR, Federal Acquisition Regulations - Part 23
2024: The FAR, Federal Acquisition Regulations - Part 232024: The FAR, Federal Acquisition Regulations - Part 23
2024: The FAR, Federal Acquisition Regulations - Part 23JSchaus & Associates
 
World Leader Summit of Love and Peace - 5th International day of conscience -...
World Leader Summit of Love and Peace - 5th International day of conscience -...World Leader Summit of Love and Peace - 5th International day of conscience -...
World Leader Summit of Love and Peace - 5th International day of conscience -...Christina Parmionova
 
NL-FR Partnership - Water management roundtable 20240403.pdf
NL-FR Partnership - Water management roundtable 20240403.pdfNL-FR Partnership - Water management roundtable 20240403.pdf
NL-FR Partnership - Water management roundtable 20240403.pdfBertrand Coppin
 
In War and conflict, health workers, facilities and supplies are off limits.
In War and conflict, health workers, facilities and supplies are off limits.In War and conflict, health workers, facilities and supplies are off limits.
In War and conflict, health workers, facilities and supplies are off limits.Christina Parmionova
 
World Health Day theme 2024 is 'My health, my right’.
World Health Day theme 2024 is 'My health, my right’.World Health Day theme 2024 is 'My health, my right’.
World Health Day theme 2024 is 'My health, my right’.Christina Parmionova
 
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...Christina Parmionova
 
Madison Cat Project - Foster Training: Lesson 1
Madison Cat Project - Foster Training: Lesson 1Madison Cat Project - Foster Training: Lesson 1
Madison Cat Project - Foster Training: Lesson 1KelleyWasmund
 
Digital Transformation of the Heritage Sector and its Practical Implications
Digital Transformation of the Heritage Sector and its Practical ImplicationsDigital Transformation of the Heritage Sector and its Practical Implications
Digital Transformation of the Heritage Sector and its Practical ImplicationsBeat Estermann
 
April 7th - World Health Day 2024 - My Health. My Right.
April 7th - World Health Day 2024 - My Health. My Right.April 7th - World Health Day 2024 - My Health. My Right.
April 7th - World Health Day 2024 - My Health. My Right.Christina Parmionova
 

Kürzlich hochgeladen (20)

2024: The FAR, Federal Acquisition Regulations - Part 22
2024: The FAR, Federal Acquisition Regulations - Part 222024: The FAR, Federal Acquisition Regulations - Part 22
2024: The FAR, Federal Acquisition Regulations - Part 22
 
Item # 4&5 - 415 & 423 Evans Ave. Replat
Item # 4&5 - 415 & 423 Evans Ave. ReplatItem # 4&5 - 415 & 423 Evans Ave. Replat
Item # 4&5 - 415 & 423 Evans Ave. Replat
 
Madison Cat Project Foster Training - Lesson 1
Madison Cat Project Foster Training - Lesson 1Madison Cat Project Foster Training - Lesson 1
Madison Cat Project Foster Training - Lesson 1
 
Everyone, everywhere has the right to good mental health.
Everyone, everywhere has the right to good mental health.Everyone, everywhere has the right to good mental health.
Everyone, everywhere has the right to good mental health.
 
Item # 6 - TBG Partners Landscape Architectural Design Services
Item # 6 - TBG Partners Landscape Architectural Design ServicesItem # 6 - TBG Partners Landscape Architectural Design Services
Item # 6 - TBG Partners Landscape Architectural Design Services
 
PPT Item # 7 - Demolition & Replacement Structure Processes
PPT Item # 7 - Demolition & Replacement Structure ProcessesPPT Item # 7 - Demolition & Replacement Structure Processes
PPT Item # 7 - Demolition & Replacement Structure Processes
 
2024 ECOSOC YOUTH FORUM -logistical information - United Nations Economic an...
2024 ECOSOC YOUTH FORUM -logistical information -  United Nations Economic an...2024 ECOSOC YOUTH FORUM -logistical information -  United Nations Economic an...
2024 ECOSOC YOUTH FORUM -logistical information - United Nations Economic an...
 
Pope Francis Teaching: Dignitas Infinita- On Human Dignity
Pope Francis Teaching: Dignitas Infinita- On Human DignityPope Francis Teaching: Dignitas Infinita- On Human Dignity
Pope Francis Teaching: Dignitas Infinita- On Human Dignity
 
Conscience. Love. Peace - International Day of Conscience 2024.
Conscience. Love. Peace  - International Day of Conscience 2024.Conscience. Love. Peace  - International Day of Conscience 2024.
Conscience. Love. Peace - International Day of Conscience 2024.
 
Actions to take at the global, national and local levels to realize the right...
Actions to take at the global, national and local levels to realize the right...Actions to take at the global, national and local levels to realize the right...
Actions to take at the global, national and local levels to realize the right...
 
PPT Item # 2 -- Announcements Powerpoint
PPT Item # 2 -- Announcements PowerpointPPT Item # 2 -- Announcements Powerpoint
PPT Item # 2 -- Announcements Powerpoint
 
2024: The FAR, Federal Acquisition Regulations - Part 23
2024: The FAR, Federal Acquisition Regulations - Part 232024: The FAR, Federal Acquisition Regulations - Part 23
2024: The FAR, Federal Acquisition Regulations - Part 23
 
World Leader Summit of Love and Peace - 5th International day of conscience -...
World Leader Summit of Love and Peace - 5th International day of conscience -...World Leader Summit of Love and Peace - 5th International day of conscience -...
World Leader Summit of Love and Peace - 5th International day of conscience -...
 
NL-FR Partnership - Water management roundtable 20240403.pdf
NL-FR Partnership - Water management roundtable 20240403.pdfNL-FR Partnership - Water management roundtable 20240403.pdf
NL-FR Partnership - Water management roundtable 20240403.pdf
 
In War and conflict, health workers, facilities and supplies are off limits.
In War and conflict, health workers, facilities and supplies are off limits.In War and conflict, health workers, facilities and supplies are off limits.
In War and conflict, health workers, facilities and supplies are off limits.
 
World Health Day theme 2024 is 'My health, my right’.
World Health Day theme 2024 is 'My health, my right’.World Health Day theme 2024 is 'My health, my right’.
World Health Day theme 2024 is 'My health, my right’.
 
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...
World Health Day 2024 - Zero Discrimination, Affordable treatments, Respectfu...
 
Madison Cat Project - Foster Training: Lesson 1
Madison Cat Project - Foster Training: Lesson 1Madison Cat Project - Foster Training: Lesson 1
Madison Cat Project - Foster Training: Lesson 1
 
Digital Transformation of the Heritage Sector and its Practical Implications
Digital Transformation of the Heritage Sector and its Practical ImplicationsDigital Transformation of the Heritage Sector and its Practical Implications
Digital Transformation of the Heritage Sector and its Practical Implications
 
April 7th - World Health Day 2024 - My Health. My Right.
April 7th - World Health Day 2024 - My Health. My Right.April 7th - World Health Day 2024 - My Health. My Right.
April 7th - World Health Day 2024 - My Health. My Right.
 

Conducting Impact Evaluations in Fragile Settings

  • 1. Economist, FAO From Challenges to Solutions: A Comprehensive Guide for Conducting Impact Evaluations in Fragile Settings Jeanne Pinay Jeanne Pinay and Marco d’Errico
  • 2. #2023 AGRODEP CONFERENCE Motivation • 2022: 274 million people require humanitarian assistance and protection • 260 million people (95 percent) living in fragile areas (UN OCHA, 2022) Fragile countries have attracted increased attention from actors of the humanitarian and development world • 2021: total international humanitarian assistance = USD 31.3 billion (GHA, 2022) Enormous amounts of humanitarian aid have been injected into crisis countries Policymakers need to identify interventions that work best to better assist people in need, and more efficiently allocate funds Rigorous IE in fragile settings is necessary… … because rigorous IE in fragile settings is challenging! … but not well-developed Why?
  • 3. #2023 AGRODEP CONFERENCE Contribution and target audience • Based on the literature (esp. Puri et al., 2017; Bakrania et al., 2021) and new evidence from FAO case studies • Contributes to the literature on aid effectiveness and impact evaluation methods • Target audience: International and local researchers, practitioners, and donors involved in impact evaluations in fragile settings Challenges Implications on data quality and validity of impact evaluation Solutions Identify the challenges encountered Determine appropriate solutions to maintain design, data quality, and credibility of the IE • Provide guidance to researchers and local actors working on impact evaluation in fragile contexts to:
  • 4. #2023 AGRODEP CONFERENCE Definition of fragile settings and FAO case studies • OECD (2022): Fragility is the combination of exposure to risk and insufficient coping capacity of the state, system and/or communities to manage, absorb or mitigate those risks • Based on the assessment of the respective balance between risk and coping capacities, contexts are categorized as extremely fragile, fragile, or ‘rest of the world’
  • 5. #2023 AGRODEP CONFERENCE Challenges Practical • Complex settings or unpredictability • Burkina Faso, Mali, Niger, Nigeria, Somalia • Need for speed or time constraints • Burkina Faso, Gaza Strip • Multiplicity of actors • High co-variability or invalid counterfactual Ethical • Do no harm • Gaza Strip • Randomization and ethics • Myanmar Methodological • Selection bias • All FAO case studies • Information bias • South Sudan • Contamination bias • Somalia • Attrition • Somalia, Nigeria, … Local capacity • Misidentification of the treated • Pakistan • Misunderstanding on IE design • Madagascar • No HHID • Gaza Strip • No experience in tracking HHs • Remote trainings and lower degree of supervision
  • 7. #2023 AGRODEP CONFERENCE Solutions Invalid counterfactual • Quasi-experimental methods • IV, RDD, DD, Matching • Combining primary with secondary data • Somalia, Madagascar • Alternative designs for randomization • Phase-in randomization • Factorial design • Randomization at higher levels (e.g., village) Measurement error • Before fieldwork • Pilot testing • Reduce questionnaire • During fieldwork • CAPI • Mixed methods • Gaza Strip • After fieldwork • Correct measurement error • Secondary data • Nigeria Attrition • Before fieldwork • Regular assessment of security situation • CATI • Diversified C group • Oversampling • Clear communication • During fieldwork • Tracking techniques • Nigeria • After fieldwork • Attrition tests • Correct attrition (IPW, CiC, bounding)
  • 8. #2023 AGRODEP CONFERENCE Conclusions • Rigorous impact evaluation in fragile settings is challenging but possible, and necessary  cannot wait for the right setting to generate evidence! • Strike a balance between • Rigor of the evaluation vs. assistance of people in need • Rigor of the evaluation vs. costs • Collaboration among humanitarian agencies, donors and local actors • Agree upon a common research agenda • Flexible and adjustable budget • Combine data collection efforts Investing substantial public resources in programs whose effectiveness is unknown is unethical! (Gertler et al., 2016) • Program adaptability and flexibility • Focus time and resources on less but more reliable evidence
  • 9. #2023 AGRODEP CONFERENCE Need to identify innovative and low- cost techniques • research designs • statistical methods • survey methods Way forward Geospatial data Satellite data Remote-sensing data
  • 10. #2023 AGRODEP CONFERENCE References • Bakrania, S., Balvin, N., Daidone, S. & De Hoop, J. (2021). Impact Evaluation in Settings of Fragility and Humanitarian Emergency. Office of Research – Innocenti Discussion Paper DP-2021-02 February 2021 • Global Humanitarian Assistance. (2022) GHA Report 2022. https://devinit.org/documents/1193/GHA2022_Digital_v8_DknWCsU.pdf • OECD. (2022). States of Fragility 2022. OECD Publishing. https://doi.org/10.1787/c7fedf5e-en • Puri, J., Aladysheva, A., Iversen, V., Ghorpade, Y. and Brück, T. (2017). Can rigorous impact evaluations improve humanitarian assistance? Journal of Development Effectiveness, 9:4, 519-542, DOI: 10.1080/19439342.2017.1388267 • UN OCHA. (2022). Global Humanitarian Overview 2022, United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA), New York, https://gho.unocha.org/