1. making an impact
Learning from ESPA research
September 2011
Do ecosystem
conservation
ABOUT THIS PROJECT
Name
projects work?
What types of investment can most
cost-effectively ensure ecosystem
service provision? A randomised program
evaluation
How can NGOs and donors measure the impact of work
Principal investigator
Nigel Asquith, Fundación Natura Bolivia on forest conservation and poverty?
Partners
Kelsey Jack, Sustainability Science
Program, Harvard University; Sampurno
Bruijnzeel, Vrije University Amsterdam
Time frame
July 2010 to June 2012
ESPA regions
Amazonia
ESPA themes
Biodiversity, Forests, Political Economy, Water
Objective
Using a controlled experimental design
modelled on the natural sciences, this
project is evaluating the impact of a
payments for ecosystem services (PES)
scheme on water quality, biodiversity, forest
cover and the socioeconomic welfare of the Imagine running an pharmaceutical trial in large-scale, controlled tests of a programme
poor in Bolivia’s farming communities. which you take a patient suffering from a fever that compensates local people for conserving
and give them an aspirin. If the patient gets forests in south-eastern Bolivia.
Summary better, you’ll conclude the aspirin worked. If the With funds from the European Union and
Despite the billions spent on programmes patient’s condition worsens or becomes fatal, other donors, Natura Bolivia has successfully
to conserve ecosystems and help poor you’ll conclude the aspirin was not the best piloted reciprocal watershed agreements
communities, there is rarely good evidence treatment. Of course, the patient might have — small-scale, locally-managed payments
that these projects have their intended improved without the drug, and if their condition for ecosystem services (PES) initiatives —
impacts. In the Santa Cruz valleys of had worsened you wouldn’t know if the aspirin since 2003. The NGO has worked with local
Bolivia, a unique large-scale experiment had slowed the decline — that’s why real drug governments and water cooperatives to explain
is underway, testing a forest conservation trials enrol many patients and use control to communities that cattle-grazing in forests
scheme across 130 villages — divided groups. But you have an ailing patient, and you along streams contaminates water supplies
randomly into groups who do or do not think a pill should help. It simply seems wrong and exacerbates droughts for downstream
receive payments for protecting forested not to try. farmers. Water users and providers then
watersheds. The experiment will show
Most monitoring and evaluation of conservation negotiate in-kind payments — such as
whether conditional in-kind payments for
programmes takes an analogous approach, beehives — so that landowners can preserve
conservation actually lead to environmental
argues Nigel Asquith — especially when the the upstream wooded watersheds while
and economic improvements, and will
aim is to protect ecological resources for the gaining steady income from them.
shed new light on the relationship between
benefit of the poor. Asquith, a conservation The Natura-led initiative has grown from 1,235
poverty and ecosystem service provision.
policy researcher at the Bolivian NGO Natura protected acres to 22,000 acres in 2010.
The results will provide useful feedback for Bolivia, is leading a project to introduce more In 2007, a new 1.8-million-acre protected
NGOs and governments rolling out similar rigour to the field. area was created to conserve the forested
schemes elsewhere. And it is hoped that the
For the past two years, Natura Bolivia and headlands of the Santa Cruz valleys, providing
project’s scientific approach will serve as a
colleagues from Harvard University and Vrije an opportunity for villagers within the protected
model for other action-research groups.
University Amsterdam have been preparing area to adopt their own watershed agreements.
Find out more: www.espa.ac.uk/projects/ne-i00436x-1

2. making an impact
Facing the opportunity for a huge scale- way to judge cost-effectiveness. As Paul
up, Natura wanted evidence that reciprocal Ferraro and Subhrendu Pattanayak — two New knowledge
agreements not only brought forest areas environmental economists who have long When NGOs and donors evaluate
l
under contract, but that the conservation of proposed the use of program evaluations in conservation programmes — in particular
these forests, watersheds, and the livelihoods the environmental field — note, “If we want efforts to protect ecological resources for
depending on them was both additional and to ensure that our limited resources make the benefit of the poor — they usually ask
effective. Experts at Harvard’s Sustainability a difference, we must accept that testing the wrong questions, measure outputs
Science programme designed an evidence- hypotheses about which policies protect rather than outcomes and exclude
based evaluation, including the complex biological diversity requires the same scientific appropriate ‘no-intervention’ controls.
statistical formulas needed to analyse and rigour and state-of-the-art methods that we This project explores how to rigorously
compare 130 villages across the protected area. invest in testing ecological hypotheses.” measure the impact of conservation and
In this way, ESPA funding has enabled The ESPA-supported project’s rigorous development programmes.
Natura Bolivia to borrow approaches from results will be equally useful to experts This study is perhaps the first to apply a
l
experimental economics to generate new and practitioners asking broader questions classic scientific approach to conservation
knowledge and evidence. This highly about whether poor communities gain when and development, testing a PES scheme
innovative project is one example of how ecosystem services such as watersheds are over a large sample of communities and
ESPA’s framework grants can pioneer new protected. comparing ‘experimental’ to ‘control’ sites.
approaches linking ecosystem services and There are side benefits to the ESPA-supported Project partners have gathered baseline
l
poverty alleviation. project. Asquith reports that in the process data on forest biodiversity, water quality
of designing a well-controlled evaluation — and socioeconomics in 130 communities,
A tale of two samples asking which key elements for impact should which will be useful to other researchers,
To establish a baseline, researchers be included in the ‘experimental’ group and left local NGOs and policymakers; discussed
surveyed the area’s 2,700 families on their out of the ‘control’ — his team identified several with communities how forest degradation
socioeconomic situation, perceptions about ways to improve the seemingly successful affects farmers downstream; and initiated
the environment and local institutions. In existing PES payment schemes . conservation payment contracts.
each community, Natura Bolivia’s staff also This led Natura Bolivia to try to improve the
measured water quality, vegetation cover and, incentives for conserving the most valuable
as a proxy for biodiversity, the abundance and parcels of forest in the programmes. If Natura
diversity of beetles, amphibians and aquatic Bolivia’s research on conservation incentives
macro-invertebrates. were a trial of aspirin, it would not only have
Using an ESPA grant, the NGO then hired created a comprehensive study that doctors
technicians and research managers to launch worldwide could rely on, but would have also
the PES scheme. All 130 communities have come up with epiphanies along the way about
received information about the threats to what’s likely to be the best dosage.
watersheds, better cattle-grazing practices, CREATING IMPACT
and alternative ways to live off forests. Half Next steps l The process of designing a well-controlled
the villages — chosen randomly in a public Landowners have begun signing up for PES experimental assessment focused
lottery — now have the chance to enrol in the contracts, and payments are being lined up attention on potential flaws in existing
payment scheme. with support from local municipalities and local PES schemes, prompting project
Follow-up data will show whether forest international donors. ESPA funds will back implementers to rethink and improve their
conservation is more effective where project implementation until June 2012, when programme design — despite its apparent
compensation is paid, as well as the effects of researchers return to the field to assess success in earlier pilot projects. This
payments on local livelihoods and outlooks. It’s preliminary results. With data in hand, the emphasises the value of involving non-
the comparison with the 65 villages receiving group will consider how to engage NGOs and researchers early in the research process.
no payments that makes the study uniquely donors with their findings and experimental l randomised programme evaluation is
A
robust; at best, environmental NGOs may approach. complex, time-consuming and requires
measure the situation before and after their Meanwhile, Bolivian officials have drafted a cutting-edge statistical work. Other
interventions, but donors virtually never require national strategy based on the project; Natura research groups can use the economic
them to assess an untouched site as a control. Bolivia is helping to pilot the plan, and is tools and surveys developed in this project
That’s a mistake, says Asquith — mainly seeking sites elsewhere in the world to further to get a head start, while innovating and
because including a control site is the best develop the randomised evaluation model. adapting the method to new purposes.
Incomplete and inappropriate evaluations
l
result in wasted money. With the spread
The Ecosystem Services for Poverty Alleviation (ESPA) research programme funds high- of this project’s approach, and growing
quality, cutting-edge research to improve our understanding of the way ecosystems function, data on which interventions really work
the services they provide and how they can contribute to poverty alleviation and enhanced to improve people’s environments and
wellbeing. This provides the evidence and tools to enable decision makers to manage livelihoods, donor investments can do
ecosystems sustainably and in a way that helps improve the lives of the world’s poorest people. more good.
Find out more: www.espa.ac.uk/projects/ne-i00436x-1 Photos: Nigel Asquith