2. Proving and Improving Results
Proving
Results
Improving
Results
M&E
system
capable
of
credibly
demonstra5ng
programme
results
M&E
system
supports
improved
design
and
implementa5on
of
interven5ons
System
Requirements
ü Well
defined
indicators
ü Robust
measurement
tools
ü Robust
monitoring
plan
ü Baselines
ü Credible
a>ribu5on
strategy
ü Credible
aggrega5on
strategy
ü M&E
system
generates
5mely
and
relevant
informa5on
for
management
ü Knowledge
management
3. M&E in a Market Systems Context
Market
development
programmes
explicitly
acknowledge
that
systems
are
complex
• M&E
needs
to
integrate
complexity
into
the
way
it
funcEons,
including
accommodaEng
for:
•
•
•
•
FormulaEng
and
tesEng
assumpEons,
piloEng
intervenEons,
proceeding
by
trial
and
error
Scaling-‐up
intervenEons
that
work,
modifying
or
dropping
intervenEons
that
do
not
work
Providing
credible
evidence
on
sustainability,
systemic
change
and
poverty
reducEon
4. Implications
•
TradiEonal
approaches
to
M&E
are
insufficient
in
market
systems
context:
• Measure
results
only
at
the
middle
and
end
of
the
programme
• M&E
divorced
from
management
funcEon
• M&E
system
uses
LogFrame
only
• “EvaluaEon”
seen
as
an
external
acEvity
only
• Use
of
Randomised
Control
Trials
5. Impact Profiles
High-‐level
Impact
SCALE
SUSTAINABILITY
• Market
development
programmes
will
have
different
impact
profiles
• Can
take
1-‐3
years
to
deliver
high-‐level
impact
• Progress
can
be
bumpy!
Time
Project
LifeEme
Market
Development
Direct
Delivery
7. WHAT IS A THEORY OF CHANGE?
Theories
of
Change
posit
the
underlying
causes
that
are
driving
poor
performance
and
resul5ng
limited
benefits
accruing
to
the
poor.
Theories
of
Change
then
provide
an
overview
of
the
systemic
change
needed
to
improve
performance,
with
a
specific
focus
on
how
the
poor
will
benefit
from
improved
performance.
8. Monitoring & Results Measurement Cycle
Theory
of
Change
1.
Draw
Interven5on
Results
Chain
7.
Report
Results
8.
Knowledge
Management
6.
Data
Analysis
2.
Define
Indicators
3.
Establish
Baseline
5.
Data
Collec5on
4.
Predict
Results
9. Results Chains
Results
Chains
map
the
systemic
change
process
of
an
intervenEon
• RCs
are
more
granular
and
flexible
than
LogFrames
• RCs
are
a
key
design
and
monitoring
tool
•
Best
PracEce:
• RCs
developed
jointly
by
the
M&E
team
and
intervenEon
team
• Every
RC
includes
boxes
to
capture
systemic
change:
sustainability,
copying
/
crowding-‐in
10. Results Chains: Basic Structure
Impact
on
poor
men
and
women
• Increased
incomes;
more
secure
livelihoods
• Job
crea5on
• Economic
empowerment
Enterprise
performance
• Improved
produc5vity/compe55veness
• E.g.
improved
yields
for
small-‐holder
farmers
System-‐level
change
Ac5vi5es
• Improved
linkages
between
system
actors
• Improved
market
access
• Changes
in
formal/informal
rules
• Introduc5on
of
new/improved
interconnected
systems
(e.g.
policy,
inputs,
finance,
ICT)
• Programme
facilita5on
ac5vi5es
11. Small-‐holders
apply
new
prac5ces
Small-‐holders
access
new
services
Copying
by
non-‐project
supported
Millers
Millers
increase
incomes
Miller
delivers
embedded
services
to
small-‐holders
Miller
adopts
new
supply
chain
prac5ces
(incl
performance
incen5ves
and
transparent
pricing)
Project
facilitates
strategic
planning
process
on
new
supply
chain
prac5ces
Project
iden5fies
Millers
willing
to
adopt
new
supply
chain
strategy
IMPACT
ACTIVITIES
Targeted
small-‐holders
achieve
higher
yields
&
sales
Copying
by
non-‐targeted
Small-‐holders
ENTERPRISE
PERFORMANCE
Targeted
small-‐holders
increase
incomes
SYSTEM
CHANGE
SYSTEMIC
CHANGE
12. Establishing Baseline
To
assess
change,
you
need
to
establish
a
baseline
for
each
indicator
• Common
mispercepEon
is
to
conduct
one
big
baseline
study
of
the
target
populaEon
at
the
beginning
of
the
programme
•
“Treatment”
group
is
hard
to
define
at
start
AUribuEon
consideraEons
can
be
difficult
to
understand
at
start
• SomeEmes
necessary
to
conduct
retrospecEve
baselines
•
•
•
Best
pracEce:
targeted
baselines
conducted
for
each
intervenEon,
based
on
clear
understanding
of
change
process
13. Predicting Results
•
Predict
results
for
each
box
in
the
Results
Chain:
What
change
do
you
expect?
• By
when?
•
•
PredicEng
results
helps
with:
Planning
data
collecEon
• TesEng
the
underlying
logic
and
making
•
assumpEons
explicit
• Monitoring
progress
•
•
Best
pracEce:
predicted
impact
is
used
to
decide
whether
to
proceed
with
an
intervenEon
or
not
–
intervenEons
that
offer
low
predicted
Value
for
Money
(VFM)
are
dropped
or
modified
14. Predicting Results: Rice Example
Average
income
increase
by
20%
for
50
Smallholders
during
first
season
Average
yields
increase
by
20%
for
50
Smallholders
during
first
season
50
Smallholders
accept
service
condi5ons
Miller
offers
services
to
200
Smallholders
during
first
season
1
Miller
adopts
new
model
Project
supports
2
Millers
15. Applying Degrees of Evidence
MSE/Household
level:
May
be
able
to
aUribute
changes
to
project
intervenEon
through
staEsEcal
and
qualitaEve
research
Market
system
level:
Use
qualitaEve
techniques
to
determine
preponderance
of
evidence
on
effecEveness
at
market
level
Triangulate
evidence
from
all
sources
16. Summary Check-list
•
•
•
•
•
•
•
•
•
M&E
system
is
capable
of
“proving”
and
“improving”
results
Every
intervenEon
has
a
clear
results
chain
Specific
and
measureable
indicators
are
defined
for
each
box
in
the
results
chain
Baselines
are
established
for
each
indicator
Predicted
results
produced
for
each
intervenEon
Every
intervenEon
has
a
measurement
plan,
with
appropriate
aUribuEon
strategy
Data
analysis
is
Emely
and
accessible
Management
decisions
are
made
on
the
basis
of
evidence/
results
Results
are
aggregated
(controlling
for
overlap)
and
reported
in
an
accessible
way