“Designing baseline surveys for impact analysis and evaluation of progress” presented by Kristin Penn, USAID/BFS at the ReSAKSS-Asia Conference, Nov 14-16, 2011, in Kathmandu, Nepal.

1. FTF Monitoring and Evaluation:
An Overview

2. The FTF Big Picture
• At the G-8 Summit in July 2009, the President pledged to
provide at least $3.5 billion over the next three years
(FY 2010 to FY 2012) to attack the root causes of global
hunger through accelerated agricultural development
and improved nutrition.
• The overall goal of the President’s Feed the Future
(FTF) initiative is to accelerate progress towards
achieving the first Millennium Development Goal (MDG
1) of halving by 2015 the proportion of people living
in extreme poverty and suffering from hunger.
• The USG commitment leveraged more than $18 billion
in support from other donors.

3. FTF on the Forefront of Implementing
• USAID FORWARD, changing the way the Agency does
business—with new partnerships, local capacity building, an
emphasis on innovation and a relentless focus on results as
a way to achieve high-impact development.
• USAID’s new evaluation policy that links systematic
monitoring of performance and evaluation of impact in all
program design, budgeting and strategy work.

4. Monitoring versus Evaluation
4
Monitoring Evaluation
Performance monitoring of
changes in performance
indicators reveals whether
desired results are
occurring and whether
implementation is on track.
In general, the results
measured are the direct and
near-term consequences of
project activities.
the systematic collection of
information about the
characteristics and
outcomes of Development
Objectives, projects, or
activities in order to make
judgments, improve
effectiveness, and/or inform
decisions about current and
future programming.

5. The Balancing Act
Utilization of
complex research
methods
Efficient M&E
management and
implementation

6. Gather Empirical
Evidence
Provide Accountability
(show results)
Improve Program
Design
FTF Monitoring and Evaluation Approach

7. M & E System Components
FTF Strategy &
Project
Development
- FTF MYS
- AADs
- PADs
Monitoring
- PMP or M&E Plan
- Indicators (Collection & Targets, CBA)
- Portfolio Reviews
- Reporting (FACTS Info/FTFMS)
Evaluation
- PMP or M&E Plan
- Hypotheses and Questions
- Performance Eval
- Impact Eval
- Learning Agenda
Knowledge
Management
- M&E Plan
- Dissemination Methods
(Reports, Presentations,
Workshops)

8. The M&E System
Generate
Development
Hypotheses and
Results Framework
Design
Interventions (and
evaluations)
Build Performance
Management Plan/
Select indicators
and set targets
Collect Baseline
Data for Monitoring
and Evaluation
Monitor, Evaluate,
and Report
Utilize M&E
findings in strategy
and project
(re)design

9. # Custom
Indicators
28 project level
8 WoG, required
if applicable
13 IR level,
required if
applicable
8 high-level
required
FTF Performance Monitoring
Indicators
Total of 57 FTF indicators
8 required
21 required if applicable (1)
13 – Intermediate Result
level
8 - whole of government
27 recommended standard
M&E Contractor will collect
data for 13 of the high-level
and IR-level indicators
(next session)
All other indicators collected
by Implementing Partners or
from secondary sources

11. Why Evaluate?
•Accountability (Show results): Measure and provide
findings related to project effectiveness, relevance, and
efficiency.
•Learning: Systematically generate knowledge about
project performance, outcomes, and impact to inform
project and strategy design and implementation.

12. Impact and Performance Evaluations address different questions:
Performance Evaluation - No Inherent Causality
– Is the program being implemented as designed?
– Could the operations be more efficient?
– Are the benefits getting to those intended?
Impact Evaluation - Causality
– What difference did the intervention make?
– Were the desired results achieved?
– Was it the intervention that caused these results?
Performance vs. Impact Evaluation

13. USAID Evaluation Standards…
Minimized Bias
Relevant to Future Decisions
Based on Best Methods
Reinforcement of Local Capacity
Commitment to Transparency
Integrated into Design
…are FTF Evaluation Standards
1
2
3
4
5
6

14. Building Local
Capacity through
Monitoring and
Evaluation

15. Local Capacity-Building Investments for Data
Collection and Use
• All countries require valid and reliable statistics to measures a
country’s status and welfare.
• The agriculture sector is at the center of critical global issues: food
security, feed-stocks for biofuels, climate change.
• In many FTF focus countries, responsible entities to collect and use
data to build effective policy responses to complex development
problems and to monitor progress toward economic and social
objectives is limited.
USG will be a strong advocate for investment in strengthening
national statistical systems/capacity in four ways.

16. Global Strategies to Improve Statistics
1. Support the GLOBAL STRATEGY to improve agriculture
statistics launched by United Nations Statistical Commission
based on common concerns, with following goals:
– Help countries develop and implement a strategic vision for
agriculture and rural statistics.
– Build on existing institution and groups - national & international
statistical agencies and data users.
– Promote a set of core data across countries, coordination across
national statistical system, and sustainability through statistical
capacity building

17. Promotion at Donor Platforms
2. USG is using multiple donor platforms to promote
strengthening national statistical systems/capacity
including CAADP, AFSI, G8/G20.
– Data collection and use to inform policy, budget allocations, and
program design for both agriculture and nutrition sectors;
– Sex- and age-disaggregated data collection in agriculture and
nutrition to better monitor and analyze gender dynamics, adjust
programming where necessary to improve positive impact across
family members;
– Promote common tools for data collection against performance
indicators, such as household surveys (e.g. LSMS) or population
sampling (e.g. LQAS) among local groups or host government
agencies.

18. Coordinating Support to Improve Ag.
Statistics at the National Level
3. Coordinating Support to Improve Ag. Statistics at the
National Level:
•Accurate measures of national food security and economic growth status and
trends;
•Relevant and timely agricultural and nutrition statistics are needed to analyze
implications for government policy related to trade, social safety nets, stockpiles;
•Timely and accurate information is also required for efficient functioning of
agricultural markets;
•LINK TO FTF - Strong national data systems are required to access progress
toward the higher level agriculture and health objectives in FTF.

19. Local Providers of M&E Services
4. Engage local firms and institutions to carry out FTF M&E services
and to formulate impact evaluation priorities
• FTF’s M&E strategy urges missions to contract with and
provide grants to more and varied local partners.
• Build metrics into M&E contractor deliverables to maximize
opportunities to sub-contract with local firms or direct
contract with local institutions to capitalize on local
knowledge and capabilities and build local capacity.

20. Way Forward and Best Practices
Next steps to consider:
1.USAID explore what is current situation in country with agricultural
statistics
2.If not adequate, initiate an outside detailed agricultural statistics capacity
assessment
 Typically a two week visit by two senior statisticians with
experience in operational statistical programs
 Recommendations on organizational, procedure, methodology, and
capacity building issues would be provided
3.Depending on assessment recommendations, consider a statistical
capacity improvement project

21. Thank You!
Kristin Penn, Feed the Future/USAID