1. Monitoring, evaluation and impact assessment C. Crissman Science Week Workshop 22 July 2011, Penang Photo: A. Gordon

7. The results chain Source: ADB, 2006

8. WorldFish Strategic Plan Research in Development Reduce poverty and vulnerability through fisheries and aquaculture. Increase food and nutrition security through fisheries and aquaculture. Our Strategic Results

9. Multiple dimensions of poverty Certain groups are systematically disadvantaged due to discrimination based on: gender, ethnicity, race, religion, caste, age, HIV status, migrant status People’s exposure to risks; sensitivity of livelihoods to risks; capacity to use assets and capabilities to cope and adapt Little access to means to make a decent standard of living A complex problem… Vulnerability Income and asset poverty Marginalization

10. Interventions to reduce poverty Organisational development, labour rights, migrant’s rights, gender equity Improve access to health services, secure land rights, aquatic property rights Diversification, microfinance, education & skills … requiring complex solutions Vulnerability Income and asset poverty Marginalization

11. Selecting paradigms for implementation AR4D

12. Strategic Plan – Implementation strategy

13. Outputs - change in knowledge - change in capacity - change in technology - change in materials - change in policy options - change in awareness/understanding Research - recognition/appreciation of research knowledge Outcomes - use of knowledge by partners - mobilisation of new capacity - extension of technology/materials - change in policy environment Impacts - change in problem - change in opportunities Development - change in actions/behaviour of stakeholders Outcomes - change in productivity - change in equity/empowerment - change in market conditions - change in investments - change in security of assets/habitats

14. Ex-post Impact Assessment Measuring ‘the change in the problem’ Reduce poverty and vulnerability through fisheries and aquaculture. Increase food and nutrition security through fisheries and aquaculture.

15. Did we make it to happen?

17. What should be monitored? SIMPLE COMPLICATED COMPLEX What works? What works in what contexts? (implementation environments and participant characteristics) What works here and now? What do we mean by ‘works’?

18. Types of interventions Simple intervention Complicated or complex intervention Single causal strand. Intervention is sufficient to produce the impacts Multiple simultaneous causal strands required to produce the impacts Universal mechanism Intervention is necessary to produce the impacts Different causal mechanisms operating in different contexts Linear causality, proportional impact Recursive, with feedback loops, leading to disproportionate impact at critical levels Pre-identified outcomes Emergent outcomes

19. Challenges for impact assessment Simple Complicated Complex Deciding impacts Likely to be agreed Likely to differ, reflecting different agendas May be emergent Describing impacts More likely to have standardised measures developed Evidence needed about multiple components Harder to plan for given emergence Analysing cause Likely to be clear counter-factual Causal packages and non-linearity Unique, highly contingent causality Reporting Clear messages Complicated message Uptake requires further adaptation

20. M&E system interaction with project implementation Monitoring captures what happened Evaluation explains why

21. M&E Implementation strategy

22. Evaluations and the R4D Results Chain Scale Pilot / Small Global Research Unit of Impact Analysis Project System Program Outcome evaluation studies that measure the effect size Outcome evaluation that measures the scale of output adoption/ uptake Time Input Output Outcome Impact Objectives Goals Program M&E, Impact pathway analysis, Adoption constraints analysis Ex post Impact Assessment as a function of (effect size * scale) Source: taken from a presentation by M. Maredia

24. Logic Models and Theory of Change Photo: World Bank

25. Conventional logic for achieving results Inspired by Jeff Conklin, cognexus.org Are we efficient? Are we effective? Source: cognexus.org ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS Time

26. Works well for outputs Inspired by Jeff Conklin, cognexus.org Workshops, training manuals, research and assessment reports, guidelines and action plans, strategies, and technical assistance packages, amongst others. Source: cognexus.org ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS Time

27. But not so well for outcomes and impact Source: cognexus.org Multiple pathways of changes in behavior of different, interacting actors; this is where we have the possibility of collecting and making sense of evidence that sustains the impact we are aiming to contribute to. Social change is long-term, complex and is the result of what many actors do (their actions and interactions) Vision and Mission Intentional design ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS Time

30. What is a logic model? Basic United Way format, 1996 Inputs Activities Outputs Inter-mediate Outcomes Long-term Outcomes

32. What is a Theory of Change? Long-term Outcome Necessary Pre- condition Necessary Pre- condition Necessary Pre- condition Necessary Pre- condition Necessary Pre- condition All outcomes that must be achieved BEFORE long-term Explain WHY here Show activities here also