The presentation provides an overview of what is meant by the term evidence based adaptation and the approaches for it. Cost-benefit analysis is outline as an important thematic in the NAP process. The presentation was made by Babatunde Abidoye, Economist Specialist for UNDP, on Day 2 of the Integrating Agriculture in National Adaptation Plans Workshop, 5-7 April 2016, Rome, Italy. © FAO: http://www.fao.org

1. Building an Evidence base:
Introduction to Economics of Adaptation and
Impact Assessment.
Babatunde Abidoye (PHD)

2. What is Evidence base for climate change
adaptation?
• Evidence-based adaptation argues that all stages of an
adaptation process should use state of the art research and
information to guide decisions at all levels.
– Building an evidence base advocates a more rational, rigorous and
systematic approach to decision making.
• The argument is that adaptation made based on this
framework produces better outcomes.

3. Approaches for Evidence Based adaptation
• Applying rigorous techniques to understanding Impacts and
vulnerabilities in agriculture – e.g. sectoral and national level
studies.
• Applying rigorous techniques to appraise adaptation options –
Cost Benefit Analysis (CBA)
• Applying Impact Evaluation techniques to understand the
benefit of a project
• Linking lessons learned and results to the global question of
climate change adaptation and mitigation.

4. Where Does CBA fit within the project?
• Will primarily contribute to Outcome 1 - Technical capacity and institution-building on NAPs
strengthened
– through training on CBA of Agriculture and other related sectors stakeholders such as Ministries of
Water and Ministries of Environment
• It will also contribute to Outcomes 2, 3 and 4
– Outcome 2: Integrated roadmaps for NAPs developed - Mainstreaming of climate change requires results
framework
– CBA will help with prioritizing actions for integrating climate change adaptation into development
policies and strategies at the national and sub-national level, starting with agriculture as the key sector.
– CBA will help the Ministries of Agriculture with access to finance from national planning.
• Outcome 3: Evidence-based results for NAPs improved.
– The activities under Outcome 3 involve the development of an impact assessment framework and conducting
sectoral and programme specific cost-benefit analyses of adaptation options in the agriculture sectors (in
collaboration with other ongoing UNDP/FAO initiatives).
• Outcome 4: Advocacy and knowledge sharing
– CBA will be featured in interactive good practice compilation of case studies.

5. The role of CBA in the NAP process
• The NAP process typically involves four elements:
 (A) Laying the groundwork and addressing gaps
 (B) Preparatory elements
 (C) Implementation strategy
 (D) Reporting, Monitoring and Review.
• CBA is an important tool in helping countries at (B) and (C) stages
• CBA is an important tool for project appraisal because it helps to identify and
value the costs and potential benefit of a project over the life of the project.
• The major difference between CBA and Impact assessment is the calculation of
benefits – IA does not make assumptions on this – it is based on observed
behavior of economic agents.

6. Economic tools for ranking and prioritization of
adaptation options
Source GIZ (2013)

7. Objective ways to know we are
maximizing benefits
• Evaluating distinct but equally plausible adaptation options (or measures)
requires a CBA of each alternative option
• This will allow an objective comparison to be made between the options.
• For adaptation options that have implications on entire sectors, a market
analysis is required to see how entire economic systems are affected. This
can be done at two levels – a sector by sector or an inter-sectoral approach

8. Objective ways to know we are
maximizing benefits
• Example of project analysis under uncertainty:
• Suppose a low dam costs $1 billion, a high dam costs $1.5 billion
and with current climate, a low dam has benefits of $1.2 billion
and high dam of $1.6 billion
low dam is better with $200 million net gain
• With future wetter climate, suppose benefit of low dam rises to
$1.3 billion and benefit of high dam rises to $2 billion
high dam now better option with $500 million net gain

9. Cost benefit analysis: intro - I
• When making a decision, esp. in the public sector, trade-off
analysis is inevitable. Costs and benefits of an action and / or
inaction need to be understood for policy decisions.
• Add up costs and add up benefits; if net benefits are positive:
do the project!
• Simple, right?
• Not quite! Issue of measurement of costs and
benefits.

10. Cost benefit analysis: intro - II
• Often times, costs are simpler – tend to be one-off and are
market transactions e.g. one time cost of constructing an
irrigation system and operating and maintenance cost.
• Benefits can be trickier e.g.: How do we calculate the market
(e.g. wealth) and non-market (e.g. health) benefits over time?

11. CBA: 8 suggested steps
1. Define the scope of analysis
2. Identify all potential impacts of the project
3. Quantify the predicted impacts
4. Monetize impacts
5. Discount rate to find present values of costs and benefits
6. Calculate the net present value (NPV)
7. Perform expected value and/or sensitivity analysis
8. Make recommendations

12. Time preference and discounting – I
• When determining the optimal allocation of resources over time, one
must deal with time preferences.
• CBA requires consideration of stream of benefits over time against costs
which incur largely today (e.g. install irrigation system that is expected to
be useful over x years).
• The problem is that the value of money does not stay constant over time.
• Also, most individuals prefer to receive benefits now as opposed to
receiving the same level of benefits in future (i.e. they have positive time
preferences).

13. Time preference and discounting – II
So, we need a system:
• The discount rate (value between 0 and 1) is a product of society’s
time value of money (composed of the pure rate of time preference
and the goods discount rate) [closer to 0: future is more important;
closer to 1: today is more important].
• The discount rate will vary by country and who is funding the project.
• National development project
• International Climate Finance Agencies:
– World Bank
– Green Climate Fund
– Regional Development Banks

14. Discounting: How do we do it?
• Use this formula to convert all future
values to present values:
• PV = FV(t) / (1+r)t
Where
• FV is expected future value
• r is the discount rate (value
between 0 and 1)
• t is time.
• Example: How much is $100000 in 30
years worth today? Try it using
different values for r!
$0
$10,000
$20,000
$30,000
$40,000
$50,000
$60,000
$70,000
$80,000
$90,000
$100,000
0 5 10 15 20 25 30
Value
Year
The impact of a discount rate on present
value estimates
15
12
%
9%
6%
3%
Source GIZ (2013)

15. Potentials within the NAP process
• CBA works best when it is possible to estimate both the market and
non-market values of all benefits that accrue.
• It is a systematically estimated analysis that provides a key piece of
information to facilitate a decision. It is not meant to be used as the
sole decision-criteria.
• If properly executed, CBA enables comparison, in equal terms, the
investments that have to be made today with benefits that accrue over
time.
• Within the NAP process, results from CBA can be used as one of the
ways of evaluating and ranking adaptation options after they have been
identified. Political, social and other considerations will also matter
along with economic considerations.

16. Limits of CBA
• One potential limitations of a CBA is that in some cases not all
direct costs and benefits can be quantified in monetary terms – in
many cases we don’t need to calculate all the costs and benefits.
• The discount-factor can matter in the final analysis.
• Need to also account for uncertainty over time. Can use sensitivity
analysis to capture some uncertainty.

17. Application in this project
• Activities under outcome 1, 2 and 3 can assist to build national
capacity for economic appraisal techniques and tools through
building capacity of departments of ministries involved in CBA
as well as other national institutions.
• Many countries and international agencies require CBA be
carried out during the design stage of a project.
• speaks the language of most ministry of finance and national
planning that can be useful to unlock funding for climate
change adaptation projects.

18. How can the programme implement these
activities?
• Country teams can make use of a guidance note on how to conduct
a CBA prepared by the global support team
• Provision of earmarked training activities under outcome 1, outcome
2 and outcome 3 for 2016-2018 to build the capacity of national
experts, line ministries and national institutions
• Technical and financial assistance to national experts
and/institutions to appraise a few selected adaptation options and
select them for a NAP agriculture roadmap (Outcome 2)
• Through documentation of case studies in outcome 3 and 4 of the
country work-plans as well as documentation by the global team.

19. Where Does IA fit within the project?
• Will primarily contribute to Outcome 3: Evidence-based results for NAPs
improved.
– The activities under Outcome 3 involve the development of an impact
assessment framework and the conducting of sectoral and programme specific
cost-benefit analyses of adaptation options in the agriculture sectors.
• It will also contribute to Outcomes 1 and 2
– Outcome 1 - Technical capacity and institution-building on NAPs strengthened
• This will include understanding the linkage between IA and Cost Benefit Analysis
– Outcome 2: Integrated roadmaps for NAPs developed - Mainstreaming of
climate change requires results framework
• IA will help in integrating climate change adaptation into development policies and
strategies at the national and sub-national level, starting with agriculture as the key sector.
• IA will enable Ministries of Agriculture to systematically learn about the effectiveness of
adaptation options that they implement

20. The role of Impact Assessment in the NAP
process
The NAP process typically involves four elements:
 (A) Laying the groundwork and addressing gaps
 (B) Preparatory elements
 (C) Implementation strategy
 (D) Reporting, Monitoring and Review.
• Impact Evaluation is an important tool in helping countries at (B) (C) and (D)
but traditionally categorized into D.
• Impact evaluation particularly will support investment appraisal and
prioritization of adaptation options

21. Why Impact Assessment?
• Development programs and investments are designed to change
outcomes of individual agents and economies.
– For example – we want to know by how much a change (measured in terms
of yields or other outcomes) brought about through investments in risk
management in the agriculture sector can be measured
• However, traditional M&E typically focuses on counting inputs rather
than results
– These inputs can include (1) Number of farmers trained, (2) number of
beneficiaries of the NAP project intervention, (3) percentage of extension
worked trained.
• Impact evaluation proposes a shift in focus from inputs to outcomes
and results

22. What is impact assessment?
• Causal impact is at the center of all impact
assessments- it is the difference in outcomes
that is caused by the program/investment
• How do beneficiaries of an investment perform
compared to how they would have fared if they
had not participated in the program?
Each farmer that adopts climate smart agriculture
practice will have a potential what-if profit level if
they did not adopt it and vice versa. This alternative
income/outcome level serves as the counterfactual.
This two states of potential income exists only in
theory!
• The central focus of impact evaluation
framework is finding a way to infer the
counterfactual from what happened to other
people or what happened to the participants of
the program before the start of the program.

23. Why Use Impact Assessment?
• IA is useful to generate evidence on questions on the impact of adopting specific
adaptation options or investments (policies/structures)
• For example: An adaptation practice involved reintroducing livestock varieties and
generating awareness of growing of indigenous crops among farmers in Mwingi
District of Kenya.
• Typical evaluation will ask:
– How many farmers are reintroduced to livestock varieties and aware of growing of indigenous
crops in the Mwingi District of Kenya?
• Impact evaluation however will ask:
– Did the project reintroducing livestock varieties and promoting the growing of indigenous crops
increase agricultural yield and profit of farmers in the Mwingi District of Kenya? And by how
much?

24. Different Approaches to Impact Evaluation
• The validity of any impact evaluation framework estimate depends
on the validity of the assumptions on the counterfactuals.
• once we intervene through a NAP project, we do not have a parallel
universe that we can observe what would have been without the
NAP project.
– So we can only observe the net revenue of a group of farmer that
participated in the project and a group of farmers that did not.
• How the two groups were selected and the kind of influence the
household have on getting the intervention information plays a
central role in the methodology.
• This provides a fundamental difference between experimental
design approach to evaluation and regression based approaches to
evaluation.

25. Experimental Design
• Experimental design is perfect for a reliable identification of program effects in
the face of complex and multiple channels of causality.
• Experiments make it possible to vary one factor of a program at a time and
therefore provide “internally” valid estimates of the causal effect.
– Will Savings and Fertilizer Initiative (SAFI) program in Kenya increase fertilizer usage?
• This research can produced a number of interesting results.
– In the first season, the basic SAFI increased usage by 14 percentage points, on a base of 24
percentage points.
– SAFI with ex ante Choice of Timing was also successful, increasing usage by 22 percentage
points.
• On the other hand, one might also get surprisingly results:
– the subsidy increased usage by 13 percentage points.
– A free delivery visit later in the season had no significant effect on usage.
• This suggests that it was the lack of commitment mechanism that was
preventing farmers from purchasing and using fertilizer.

26. So how does experimental design work?
• Experimental design is premised on the ability to randomize
treatment.
• The key feature of a randomized evaluation is that the people
who have access to the program or benefit from the policy are
selected randomly.
– This is called random assignment
– This ensures that there are no systematic differences between those
who receive the program and those who serve as the comparison
group.
• It is this random assignment that gives randomized evaluation
an advantage in measuring program impact.

27. Quasi-experimental design:
• Quasi-experiments are experimental designs with no
randomized treatment and ability to claim causality.
• Two major forms of quasi-experimental designs are the
nonequivalent groups design and regression-discontinuity
design:
1. Nonequivalent groups design does not control the assignment to
groups before the project as is typically done in the randomized
control treatments.
2. Regression Discontinuity approach on the other hand is used to
evaluate program that have eligible cutoff.

28. Regression Based Impact Evaluation
• Regression based approaches is one of the major ways
economists evaluate projects.
• Regression analysis typically relates a causal variable of
interest and an outcome variable of interest - the relationship
between them statistically is considered the impact of the
project.
• Individuals who received the program are compared with
those who did not, and other factors that might explain
differences in the outcomes are “controlled” for.

29. What is required?
Nonequivalent groups design Regression Discontinuity Regression Based
Data on outcomes
“variables for matching” both
participants and nonparticipants
Information on exactly how
program participation is decided
Data on “explanatory” or
“control” variables such as age,
income levels, education. (The
control variables need either to
be collected at baseline or after.)

30. Application in this project
• Activities under outcome 3 can assist to build national capacity for impact
assessment through building capacity of departments of ministries
involved in M&E as well as other national institutions.
– Can assist in building an evidence base for adoption of adaptation options
• Impact Assessment can help understand new programs/interventions –
especially with the application of experimental design
• It can also be used to understand if a project achieved the intended
outcomes – applied ex post
• With competing needs for funding – Impact evaluations provides
compelling evidence for funding from ministry of finance and national
planning for climate change adaptation projects.
– One of the major part of mainstreaming is ensuring evidence-based impact and
strategies.
• In the working group, we will discuss the different approaches with
application on how to evaluate current projects.

31. How can the programme implement these
activities?
• Develop and refine tool-kit/guidance applying impact assessment to
adaptation activities in the agriculture sector.
• Apply the toolkits/guidance through training to build the capacity of
government monitoring units and national institutions on impact
assessment applying the guidance to an existing agricultural project
• Through consultations, select 1-2 countries, and identify targeted
technical and financial support to ongoing efforts related to IA
• Through support and mentoring of national activities by impact
assessment experts in the global UNDP-FAO team
• Through documentation of case studies, learning, and knowledge
exchange (outcomes 3 and 4) related to IA