Impact Evaluation of The Innovation for Agribusiness (InovAgro) Project in Northern Mozambique

Mulubrhan Amare1 and Helder Zavale2
1. International Food Policy Research Institute 2. University of Eduardo Mondlane
Innovation for Agribusiness (InovAgro) Project Impact Evaluation Results
Dissemination Workshop | October 28, 2021
Contact: Jenny Smart (j.smart@cgiar.org) or Mulubrhan Amare (m.amare@cgiar.org)
Impact Evaluation of The Innovation for
Agribusiness (InovAgro) Project
in Northern Mozambique
Financial support from
Disclaimer: The analysis presented in this slide deck are the team’s own and does not necessarily reflect the views of IFPRI, UEM or SDC.

Presentation Structure
• Background and theory of change of the project;
• Challenges in impact evaluation of the InovAgro project;
• Mitigative measures taken;
• Empirical strategy;
• The household level impacts of the InovAgro project;
• The market level impacts of the InovAgro project;
• Conclusions and implications.

Background
• Poverty is higher in rural
areas (50.1%) compared to
urban areas (37.4%)
52.8
48.2
55.0
51.7
46.8
53.8
46.1
37.4
50.1
0
20
40
60
Poverty
rate
(%)
2003 2008 2015
National Urban Rural
52.8 51.9
49.2
59.9
51.7
45.1
57.0
51.2
46.1
55.1
46.2
32.8
0
20
40
60
Poverty
rate
(%)
2003 2008 2015
National North Central South
• Poverty is more pronounced
in Northern and Central than
in Southern

Background
• Agriculture employing about 80% of the country’s labor
force.
• Agriculture is characterized by production systems
predominantly based on rain-fed coupled with low use of
modern inputs which lead to low agricultural productivity;
• However, agriculture has the largest poverty elasticity
estimated at -2.7%; more than threefold higher than that
of other economic sectors;
• This suggests that agriculture has the largest potential for
reducing poverty incidence;

Background
• The InovAgro project was designed aiming at
• Increasing incomes for poor smallholder farmers in Northern Mozambique
through
• improved agricultural productivity and participation in selected high-potential
value chains.
• The InovAgro project’s primary approach is to promote
the development of inclusive and sustainable market
systems;
• This approach is also known as the Market Systems
Development (MSD) approach;
• The objective of this impact evaluation is to assess the
impact of the MSD approach on HH welfare of
participating farmers.

Theory of change of the InovAgro project
• The InovAgro project consists of three phases
• InovAgro I: A three-year pilot phase from 2011 to 2013;
• InovAgro II: A four-year expansion phase from 2014-2017;
• InovAgro III: A three-year wrap-up phase from 2018-2020.
• Generic MSD theory of change:

• InovAgro has an articulated theory of change.

• InovAgro project sought to
1) Facilitate the use of field days and increased visits by
extension officers to
• Help raise demand for quality seed and other agricultural inputs;
• Support the promotion of good agricultural practices;
• Provide greater extension service delivery to smallholder farmers;
2) Increase smallholder farmers’ access to finance and
mechanization by
• Facilitating the relationships of financial institutions and the financial
institutions’ willingness to offer loans;
• Establishing savings groups to enable group purchases of inputs;
• Promoting the relationship between smallholder farmers and mechanization
service providers;

• InovAgro project sought to
3) Support the opening of new seed stores by private sector
seed companies
• Generate demand and promote the use of certified seed by establishing
demonstration plots on lead farmers’ own fields and organize field days;
• Training agro-dealers to work with the seed suppliers to retail the certified
product to local smallholder farmers;
4) A better enabling environment and services for certified
seed production and sales by:
• Means of approving the operational plan produced for the National
Seed Dialogue Platform (NSDP);

Challenges in InovAgro impact evaluation
• The initial design was to carry out a RCT
• RCT approach was not feasible from an operational perspective
mainly due to:
• Ethical issues: issues involved with exclusion of subjects for a control group
• Challenges associated with the project implementation
• Selection bias issues;
• Contamination effects: hard to contain treatment activities from control
groups;
• Historical phenomena/effects.
• Challenges associated with level of impact (scope)
• Beneficiary versus institutional/systemic level effects
• The longer the project period the more susceptible the impact of
evaluation design will be to challenges of spillover and contamination;
• MSD programs usually are designed to have systemic level effects that are
prone to unintended effects (positive or negative)
• Not accounting for the potential discrepancy between treatment and
intention to treat could understate the impact of the intervention.

Mitigative measures taken
• To account for such potential bias,
• three-wave HH panel dataset of intended beneficiary and non-beneficiary
HHs with “intention-to-treat” data are used as an instrument for
treatment (Abadie et al. 2002);
• Supplemented with a unique geo-reference census data of every value
chain interventions
• Define a median distance to these value chain interventions (60 minutes
in our case) to define catchment area of the intervention/treatment.
• Using the intention-to-treat definition for treatment (exposure to
intervention);
• Smallholder farmers located within a 60-minute walking distance buffer
of a value chain intervention are defined as beneficiary (treated or
exposed HHs);
• While those located outside the 60-minute walking distance are defined
as control (non-beneficiary HHs);

• Even after such careful approach for definition of
treatment, major challenges remain:
• Natural learning;
• Self selection, program targeting;
• The adaptive nature of MSD programs:
• Causes for concern in making causal inferences (or
differentiating contribution [correlation] versus
attribution [causality]);

• To isolate InovAgro effects and account for possible influence of
external factors, we utilized a before-and-after intervention data
and employ a difference-in-difference (DID) approach;
• To disentangle the potential effects of InovAgro from other
similar MSD programs, we analyze the number of years since
operation (using GIS census data).
• comparing InovAgro sponsored MSD value chain interventions with those
MSD programs that are not directly affiliated with InovAgro;
• Hence, the validity of our identification strategy (our ability to
claim causal inferences) depends on whether InovAgro has had
an overall systemic (crowding-in) effect;
• Propensity score matching (PSM) was used to produce
comparable sub-samples of beneficiary and non-beneficiary HHs;

Empirical strategy
• DID estimation technique to assess the impact of the exposure to
InovAgro on selected outcomes using the following regression
model:
0 1 2
ijt t j j t ijt
Y T C C T
β β β γ ε
= + + + × +
denotes the outcome variable of interest;
is a dummy variable equal to one if year equal to 2017 and
zero if year equal to 2015;
is a dummy variable equal to one if community was exposed
to InovAgro (treatment community) and zero otherwise
(control community);
is a random error with mean zero and constant variance;
ijt
Y
t
T
j
C
ijt
ε

Empirical strategy
• We assessed potential InovAgro project impacts on
(outcome variables):
• Adoption of modern farm practices;
• Access to agricultural (input and output) market information;
• Agricultural productivity and marketing;
• Income diversification and overall HH welfare; and
• Empowerment of women and other vulnerable groups (youth):
Unintended (positive or negative) effects;

Empirical strategy
• One of the key assumptions behind the DID approach is
that other covariates – rather than the InovAgro project –
do not change between the baseline and midline surveys;
• However, this assumption is violated in our case.
• We controlled for HH-level characteristics that could affect
the difference in trends between treatment and control
groups by modifying the above regress as follows:
represents a set of HH-level characteristics that could
influence outcome variables of interest;
0 1 2 3
ijt t j j t ijt ijt
Y T C C T
β β β γ ε
= + + + × + +
β X
ijt
X

Empirical strategy
• Estimating the average outcome variable among
beneficiary HHs, had they not benefited from the
InovAgro project suffers from the selection bias problem;
• We control for this selection bias by using the propensity
score matching (PSM) approach;

Data
• The treatment unit is the community (comunidade). Four
communities were selected in each district where
InovAgro’s intervention carried out;
• All selected treatment communities were located in the
same administrative post within each district;
• The control communities were selected from comparable
localities in a different administrative post from where the
treatment communities are located;

Data
• A different administrative post were selected for the
control communities to limit spillovers effects;
• The HH listing information, and the extent of soybean
and pigeon pea cultivation was used to select the final set
of both treatment and control areas and control
communities;
• The final sample was drawn from 16 communities in four
administrative posts in two districts.

Data
• Initial estimated sample size of 1,886 HHs, of which 937 from
the treatment communities and 949 from the control
communities (baseline survey);
• Sample size dropped to 1,733 HHs (880 from the
treatment communities and 853 from the control
communities) between the baseline survey (2015) and
endline survey (2019);
• Overall attrition rate of 8.1% between the baseline survey
(2015) and endline survey (2019);

Data
• Using a geographic boundary (community boundary) to
define treatment remains a major methodological
challenge;
• HH presumably residing in a control community could be located within close
proximity to an InovAgro sponsored intervention in one of the treatment
communities.
• Hence, geo-spatial data (collected during the midline &
endline survey) was used to define “physical accessibility”
as an identification strategy to define comparable
treatment and control HHs;

Data
• To assess the potential InovAgro effect on overall macro
(market) level effects, we combined primary, secondary and
geo-referenced data;
• HH presumably residing in a control community could be located within close
proximity to an InovAgro sponsored intervention in one of the treatment
communities.
• Complementary data was also acquired via key informant
interviews (KIIs) and focus group discussion (FGD with local
stakeholders;
• These are able to group all geo-referenced value chain
intervention into three groups:
1. MSD – InovAgro facilitated;
2. MSD – Non-InovAgro facilitated; and
3. Non-MSD value chain interventions.

Data
• Treatment HHs are further sub-grouped as MSD and non-MSD HHs
depending on whether they are exposed the MSD value chain
intervention or not;

Results: Household level InovAgro impacts
• We differentiate whether the channel of intervention
(agro-dealer, a lead farmer or a demonstration plot; and
exposure to the three value chain interventions
simultaneously) plays any role in dictating the magnitude
of program outcome;
• We defined treatment based on exposure to each of three
value chain interventions and compare the magnitude of
impact with those HHs that have benefited from exposure
to the three value chain interventions simultaneously;
• We also aim to differentiate potential differences on
short-term (a two-year gap) versus long-term (a 4-year
gap) impacts.

• InovAgro has a positive and significant impact on HHs’
likelihood of adopting (using) agro-chemicals like
pesticide, herbicide, etc
0
5
10
15
20
InovAgro
impact
on
agro-chemicals
adoption
Agro dealer Lead farmer Demo plot Complete package
Short term (2016/17) Long term (2018/19)

• The likelihood of fertilizer adoption within 2-years after
InovAgro exposure seems to depend on project
beneficiaries getting exposure to the complete package
0
5
10
15
InovAgro
impact
on
fertilizer
adoption

• The positive InovAgro effect is wiped out in the long-term
as the impact remains positive but not significant
0
5
10
15
InovAgro
impact
on
Modern
seed
adoption

• The positive and significant InovAgro effect on access to
agricultural input market information for beneficiary HHs
who are exposed to all three value chain interventions
0
5
10
15
InovAgro
impact
on
input
market
information

• The positive and significant InovAgro impact on access to
output market information by beneficiary HHs compared to
non-beneficiary households: Robustness to single value chain
intervention or complete package
0
5
10
15
20
InovAgro
impact
on
output
market
information

• A positive and significant InovAgro effect on not only in
boosting agricultural productivity of beneficiary HHs but also
their likelihood of agricultural output market participation as
well as the ratio of marketable surplus
0.0
0.2
0.4
0.6
InovAgro
impact
on
agricultural
productivity
Productivity
(output per ha)
Sale of
agricultural output
Ratio of
marketable surplus

• A positive and significant InovAgro effect on HH welfare is
only shown when beneficiary HHs are exposed to the
most intense (complete package) treatment
0.00
0.05
0.10
0.15
0.20
InovAgro
impact
on
household
wealth

• InovAgro impact on income generation from non-
agricultural is only positive and significant in the long
term
0.00
0.05
0.10
0.15
0.20
InovAgro
impact
on
participation
on
non-agriculture

• A positive and significant InovAgro effect on temporary
migration among members of beneficiary HHs in the
short term while no impact of such effect in the long-
term
0.00
0.05
0.10
0.15
0.20
0.25
InovAgro
impact
on
temporary
migration

• A positive and significant InovAgro effect on women land
access in the long term while negative and significant
impact the short term
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
InovAgro
impact
on
women
land
access

• A negative and significant InovAgro effect on youth land
access in the short term while no impact in the long term
.02
0
-.02
-.04
-.06
-.08
-.1
InovAgro
impact
on
youth
land
access

• A negative and significant InovAgro effect on women income
generation in non-agriculture in all value chain interventions except
lead farmer in the short term while no impact in the long term
except for those exposed to lead farmers. Similar findings for youth
-0.10
-0.05
0.00
0.05
0.10
InovAgro
impact
on
women
in
non-agriculture

Results: Macro (market) level InovAgro impact
The market level impact evaluation also attempts to
document the macro level effects, namely:
• Systemic (long-term) effects: could be either due to crowding-in or
copying of the InovAgro (MSD) approach;
• Sustainability effects;
• Large-scale (spillover or multiplier) effects; and
• Unintended (positive/negative) program effects.

Systemic long-term effects:
• To investigate the potential crowding-in effect of InovAgro,
• compare the number of InovAgro facilitated MSD versus Non-
InovAgro facilitated MSD value chain interventions before and after
the launch of InovAgro activities in 2015;
• also compare the average time lapsed since the program’s launch for
InovAgro facilitated versus non-InovAgro sponsored MSD value chain
interventions;
• Overall, results reveal that, comparing the time before and after
the launch of the InovAgro project, there is a significant
percentage increase in the number of non-InovAgro facilitated
value chain interventions;
• On average, non-InovAgro facilitated MSD value chain
interventions had significantly shorter time lapsed since the
program’s launch compared to InovAgro sponsored value chain
interventions

• Systemic long-term effects:
3
15
39
0
19
27
2
25
31
7
58
25
5
40
35
7
77
25
0
20
40
60
80
Number
of
value
chain
intervention
Number
of
months
Molumbo Alto Molocue Total sample
MSD Non MSD MSD Non MSD MSD Non MSD
Before 2015 After 2015
Number of months since launch

Sustainability effect
• To investigate the potential sustainability effect of InovAgro,
• use the three wave panel data and monitor the HH adoption history of modern
farming practices,
• comparing those who benefit from MSD with those only exposed to non-MSD
value chain interventions;
• To claim a sustainability effect of the InovAgro MSD program, the proportion
of MSD treated or exposed HHs under “continue adoption” to be significantly
larger than the proportion of HHs treated or exposed to non-MSD value
chain interventions in the case of sustainability effects

• Adoption trajectory
2018/2019 (endline) survey
Adoption of modern farm practice
YES NO
2016/2017
(midline)
survey
Adoption
of
modern
farm
practice
YES
I
(continue adoption)
III
(become non-adopters)
NO
II
(become new adopters)
IV
(continue non-adoption)

• Sustainability effect on input adoption: continuing adoption higher
among MSD-exposed HHs
0
10
20
30
40
%
HHs
who
continue
adopting
(2016/17-2018/19)
NPK Insecticide Herbicide Inoculant
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
MSD Non-MSD

• Sustainability effect on input adoption: continuing adoption higher
among MSD-exposed HHs
0
10
20
30
40
50
60
70
80
90
100
%
HHs
who
continue
adopting
improved
seed
(2016/17-2018/19)
Soyabeans Pigeon peas Maize
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
MSD Non-MSD
0
10
20
30
%
HHs
who
continue
adopting
fertilizer
(2016/17-2018/19) Soyabeans Pigeon peas Maize
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
MSD Non-MSD
• Overall: the results support our hypothesis that the
InovAgro MSD program is more sustainable than non-MSD
programs

• Spillover or multiplier effect
• We define spillover/multiplier effects to refer to wider changes
resulting from InovAgro by benefitting larger numbers of
smallholder farmers beyond InovAgro’s direct domain of
intervention;
• Using our unique spatial database, we were able to compare the
adoption practices of non-beneficiary HHs that are located in
close proximity to an MSD beneficiary HH (those within a 60-
minute buffer cutoff) versus non-beneficiary HHs who are not in
close proximity to an MSD beneficiary HH

• Spillover effects: Overall, the results support the hypothesis,
showing the potential effect of InovAgro in benefitting large
numbers of smallholder farmers beyond the program’s direct
sphere of influence and intended beneficiaries
0
5
10
15
%
HHs
using
agricultural
input
NPK Insecticide Herbicide Inoculant
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
MSD Control Pure control

• Of the 30 hypotheses tests conducted, 23 cases show that the
proportion of HHs who were new adopters of modern farm
practices was significantly for non-beneficiary HHs who resided
in close proximity to households treated or exposed to the
InovAgro MSD program compared to those who resided further
away (more pure controls)
0
5
10
15
20
25
30
%
HHs
using
fertilizer
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
0
5
10
15
%
HHs
using
improved
seed
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r
A
g
r
o
-
d
e
a
l
e
r
D
e
m
o
p
l
o
t
L
e
a
d
f
a
r
m
e
r

• Potential unintended effects:
• Results show that a negative unintended effect of both MSD & non-
MSD programs on HHs’ crop diversification
Change over time (2015 --> 2019)
Proxy variables MSD programs Non-MSD programs
CROP DIVERSIFICATION Total number of crops cultivated (Negative) (Negative)
INCOME
DIVERSIFICATION
Head in non-agricultural (Positive) (Negative)
Spouse in non-agricultural (Positive) (Negative)
Female in non-agricultural (Positive) (Negative)
Youth in non-agricultural (Positive) (Negative)
A household member in non-
agricultural
(Positive) (Negative)
MIGRATION
A temporary migrant member (Positive) (Negative)
A permanent migrant member (Positive) (negative)
INTRA-HOUSEHOLD
BARGAINING
Head has credit access
Female has credit access (Positive)
Youth has credit access
A family member has credit access (Positive)
LAND RIGHTS OF
VULNERABLE GROUPS
Spouse has access to/control over land (Positive) (Positive)
Female has access to/control over land (Positive) (Positive)
Youth has access to/control over land (Negative) (Positive)

Conclusions
• InovAgro interventions increase farmers’ use of yield-
enhancing agricultural inputs, productivity and welfare.
• InovAgro interventions improve in the number of non-
InovAgro facilitated or sponsored value chain interventions.
• The InovAgro MSD program has more sustainable impact
than non-MSD programs.
• The InovAgro project benefited large numbers of smallholder
farmers beyond the project’s direct sphere of influence and
intended beneficiaries.

Conclusions
• The InovAgro MSD program has a negative unintended effect of
both MSD and non-MSD programs on HHs’ crop diversification.
• The study provides evidence in support of the project’s having a
systemic market-level effect, as well as sustainable long-term
effects on HHs’ adoption of good agricultural practices and access
to market information,
• A more intense, combination approach of using agro-dealers, lead
farmers and demonstration plots appears to be necessary to
achieve long-term positive effects on the overall welfare of HHs.

Impact Evaluation of The Innovation for Agribusiness (InovAgro) Project in Northern Mozambique

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Impact Evaluation of The Innovation for Agribusiness (InovAgro) Project in Northern Mozambique

Ähnlich wie Impact Evaluation of The Innovation for Agribusiness (InovAgro) Project in Northern Mozambique (20)

Mehr von International Food Policy Research Institute (IFPRI)

Mehr von International Food Policy Research Institute (IFPRI) (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Impact Evaluation of The Innovation for Agribusiness (InovAgro) Project in Northern Mozambique