In Nairobi on the 12th November 2015, Dr Andy Hall (CSIRO) lead a session as part of the Australian Award African Short Course with University of Sydney. Andy lead a discussion with approximately 25 participants from different academic institutions across Africa on different tools and approaches to MEL for designing and managing agriculture research for development projects.

1. Designing and managing projects for impact
at scale
Dr Andy Hall • Research Group Leader - Global Agriculture Innovation Dynamics
12 November 2015

2. Objectives of today
• Understand Project Design and MEL are part
of the same process
• Understand how to develop the activities,
output, outcome, impact logic of projects
• Understand the range of tools used for
different aspects of monitoring evaluation and
learning.
• Understand how to select relevant tools for
different MEL tasks
Designing and managing projects for impact at scale • Dr Andy Hall2 •

3. Recapping key terms
• Project purpose: The development challenge that the project is contributing to
resolving
• Project outcome: Changes in behavior of farmers, traders, policy makers and
other actors related to the project purpose that result from the activities and
outputs of the project
• Project output: New information, ideas, technologies created by the project.
• Project activity: Actions implemented by the project, including research.
Designing and managing projects for impact at scale • Dr Andy Hall5 •

4. Worked example from AAPP
• Growth and development of food systems in east Africa
• Researchers, farmers, private sector collaborate to develop and apply new
knowledge
Designing and managing projects for impact at scale • Dr Andy Hall3 •

5. AAPP ground rules
• All projects had to identify a development purpose that they were contributing
to: e.g. consumers have reduced exposure to aflatoxin.
• All projects had to identify 2 major types of outcome that would contribute to
the purpose (end of project deliverables)
• Adoption of technology/policy/standards
• Researchers, farmers, private sector, NGO, etc. working in new ways that deliver
solutions.
Designing and managing projects for impact at scale • Dr Andy Hall6 •

6. Design process
• Build on earlier research and research teams
• Extensive collaborator and stakeholder workshops and consultations
• Development of problem trees (identifying a interconnected set of problems)
• Development of project outcomes, outputs and activities needed to address those
problems.
• Development of a theory of change
• Development of an MEL systems to track progress on outcomes
Tip: Don’t start with research you would like to do. Start with the problem you
want to solve.
6 • Designing and managing projects for impact at scale • Dr Andy Hall

7. Identify the development challenge
7 •

8. Outputs & Outcomes to Achieve Impact
8 •

9. A Theory of Change
9 • Designing and managing projects for impact at scale • Dr Andy Hall

10. • Technology adoption and impacts:
– selecting indicators and data collection methods
• Tracking effectiveness of collaboration:
– Quality or quantity? Scope, Relevance?
• How stakeholders and systems respond:
– Unpredictable and unexpected. But project needs this information to keep on
track.
MEL challenges
10 • Designing and managing projects for impact at scale • Dr Andy Hall

11. 11 •
Morning coffee break
Designing and managing projects for impact at scale • Dr Andy Hall

12. 12 •
Current State: what are we
doing at the moment
Designing and managing projects for impact at scale • Dr Andy Hall

13. You have 5 minutes to think about the following questions. Use your note pads to capture your
thoughts. After your 5 minutes of individual reflection we are going to discuss what you currently
do in terms of MEL as a group.
Question 1: If you do MEL why do you do it?
Question 2: Do you have any particular approaches or tools you use?
Question 3: Have you found success with any particular approach or tool used, if so why?
Question 4: What do you use the information collected and analysed as part of your MEL for?
Question 5: Are there any limitations to these tools?
Exercise 1: What do you currently use?
13 | Designing and managing projects for impact at scale • Dr Andy Hall

14. 14 •
What is a MEL System
Designing and managing projects for impact at scale • Dr Andy Hall

15. What is the MEL System about?
15 •
What: Describe the change
Why: Explain the change
Is it good enough: Evaluative part
DATA
DIALOGUE
Designing and managing projects for impact at scale • Dr Andy Hall

16. Why M&E&?
• Project design: Contributing to the project design process by developing a logic that links project
activities to a stated project outcomes and purpose.
• Project learning performance management/ monitoring: Organises the collection of information
during the life of the project to make judgments about the progress, relevance and effectiveness
of the project activities towards the achievement of a project outcomes and purpose.
• Project accountability/ evaluation: Organises the collection of information about the levels of
outcomes and purpose (impact) achieved to report to donor and other stakeholders.
• Program learning: Organises the collection of information about generic lessons from the
implementation of projects/ clusters of projects that can inform project and program practice in
the future.
Designing and managing projects for impact at scale • Dr Andy Hall16 •

17. 17 •
Different MEL Tools & approaches
for different purposes
Designing and managing projects for impact at scale • Dr Andy Hall

18. Indicators
18 |
INDICATOR ASSUMPTIONS
MEANS OF
VERIFICATION
Designing and managing projects for impact at scale • Dr Andy Hall

19. Activity plans and milestones
19 • Designing and managing projects for impact at scale • Dr Andy Hall

20. Internal activity monitoring
Presentation title • Presenter name20 •

21. Technology adoption/ impact tracking
Presentation title • Presenter name21 •
• Base line surveys and end of project / post project resurveys
• Rapid appraisals and ad hoc studies – valuable for keeping an eye
on progress and getting feeds back on weather the project
activities are leading to expected results.

22. Rubrics
22 | Designing and managing projects for impact at scale • Dr Andy Hall

23. Outcome rating report
Presentation title • Presenter name23 •
Implementation and output performance
statements RATING
Commentary, evidence and prospects
Learning events are relevant, timely, appropriately
designed; involve Australian and/or in-country
partners and their networks; and are perceived as
worthwhile; GOOD
A series of training events on Gender and value chain development, DCED
standards and impact investing have been delivered and well received. An
ambitious series of events are planned that respond to DFAT priorities. See
www.foodsystemsinnovation.org.au/events
Reflection events with DFAT in country programmes have been slow to
materialise, although concrete plans are in place to conduct 2 events in the
next quarter
Knowledge products are relevant, practice-based
and practice-oriented, and aligned with current and
emerging (agriculture food, and nutrition)
international development thinking, practices, and
needs within Australia and overseas; collaboratively-
created, reader-friendly and audience-appropriate,
and produced and delivered in a timely manner;
ADEQUATE
A series reports synthesising international perspectives on aid for trade have
been well received and are frequently cited. Inputs have been made into DFAT
strategy and operational guidelines. A dossier on improving nutrition through
agriculture linkages has been published. A practice note series has been slow
to take off. A renewed effort to develop practice note collection addressing
FSI priorities is being progressed by bringing in additional expertise and
external commissioning. A recently commenced line of research on inclusive
agribusiness innovation will deliver high profile knowledge products by the
end of the project. See www.foodsystemsinnovation.org.au/resources
An expanded range of relevant experts and
international development practitioners who
actively contribute to FSI Australian and in-country
partners’ and their networks’ discussions, designs,
and practices in ways that are perceived as
collaborative, salient, credible and useful.
ADEQUATE
External expertise has been used in the areas of M&E, partnership
development and impact investing. Regional expertise will be used to service
research and networking activities associated with an inclusive agribusiness
innovation round table. This outcome has been slow to develop as tasks to
engage external expertise have only just started to be come apparent.
FSI creates opportunities for in-country partners to
participate in reflection and learning on food
systems innovation EXCELLENT
A good start was made with the FSI symposium. FSI has built strong learning
alliances with partner projects in East Africa, South Asia, Indonesia and Timor
Leste. A major initiative in the remainder of the project will support the Grow
Asia platform with an inclusive agribusiness innovation round table supported
with research and analysis.

24. Most Significant Change (MSC)
24 | Designing and managing projects for impact at scale • Dr Andy Hall

25. Institutional History (IH)
25 •
• A narrative that records key points about how institutional arrangements –
new ways of working – have evolved over time and have created and
contributed to more effective ways to achieve project or programme goals.
• An IH is generated and recorded in a collaborative way by scientists, farmers
and other stakeholders.
• A key intention behind institutional histories is to introduce institutional
factors into the legitimate narrative of success and failure in research
organizations.
Designing and managing projects for impact at scale • Dr Andy Hall

26. Reviews and external evaluations
26 |
Relevance: The extent to which the aid activity is suited to the priorities and policies of the
target group, recipient and donor.
Effectiveness: A measure of the extent to which an aid activity attains its objectives.
Efficiency: Efficiency measures the outputs - qualitative and quantitative - in relation to the
inputs. This generally requires comparing alternative approaches to achieving the same
outputs, to see whether the most efficient process has been adopted.
Impact: The positive and negative changes produced by a development intervention, directly
or indirectly, intended or unintended. This involves the main impacts and effects resulting
from the activity on the local social, economic, environmental and other development
indicators.
Sustainability: Sustainability is concerned with measuring whether the benefits of an
activity are likely to continue after donor funding has been withdrawn.
Designing and managing projects for impact at scale • Dr Andy Hall

27. So in summary what do we know …
27 |
There is no silver bullet
and
design and MEL are not
separate functions … they
inform each other
Designing and managing projects for impact at scale • Dr Andy Hall

28. 28 •
Finding your fit for purpose
approach
Designing and managing projects for impact at scale • Dr Andy Hall

29. Resources
• Online resource portals for MEL practitioners
• Better Evaluation http://betterevaluation.org
• Online community of practices
• Eval Partners http://www.mymande.org/evalpartners
• S.Stone-Jovichi (2015) The journey to fit for purpose: The development of a
MEL system for a complex project. Practice Note Series by Food Systems
Innovation
• www.foodsysteminnovation.org.au
29 | Designing and managing projects for impact at scale • Dr Andy Hall

30. 30 •
Lunch break
Designing and managing projects for impact at scale • Dr Andy Hall

31. 31 •
Applying MEL concepts and
lessons to an example
Designing and managing projects for impact at scale • Dr Andy Hall

32. GOAL:
Technologies
and services by
public and
private sectors
help
Pastoralists
have SRL
Companies
crowd in and
adopt
technology and
practice
innovations
(Private Sector)
Improved policy
& public
investments
targeting
pastoralists
(National Public
Sector)
Incubation & refinement
through experimentation
/ problem solving
Technical
backstopping
Market & community
intelligence
Improved
development
investments
targeting
pastoralists
(International public
sector)
SYSTEM & BEHAVIOURAL CHANGESPROVING CONCEPT
Strengthened
organisations for
capacity,
facilitation &
advisory services
Filling market gap
Capacity building
Feedback
Feedback
Feedback
Science Capability & Networks :
- Science of innovation - International researchers
- Economic and finance - ILRI researchers
- Remote sensing programing - etc
- Impact Assessments (ex-anti & ex-post)
- Systems diagnosis
- Tracking system changes
Implementation Capability & Networks :
- Practice of implementation innovation - private sector
- Community consultations - Government
- Capacity development capability - NGOs
- Qualitative / Ad hoc studies - Donors
- Quality Assurance / trusted advisor - etc
- Partnership management & brokering
Feedback
Feedback
Panel data collection &
impact assessment
Feedback
Insurance Example

33. 33 • Presentation title • Presenter name
Plans for scaling the theory of change / operations model

34. Thinking about the example you just saw break into 4 small groups and discuss the
following questions.
Question 1: What sort of change needs to be tracked?
Question 2: What sort of data or evidence will be needed?
Question 3: What sort of tools will they need?
Question 4: When should they do it?
Question 5: who should do it?
Each group will then report back to the larger group.
Exercise 1: What do you currently use?
34 | Designing and managing projects for impact at scale • Dr Andy Hall

35. 35 •
Questions
Designing and managing projects for impact at scale • Dr Andy Hall

36. Level 1 Ecosystem Sciences Black Mountain Laboratories, Acton ACT 2602 Australia
TEL. +61 2 6246 4771
Mob +61 477 735 348
Email: andrew.hall@csiro.au
www.foodsystemsinnovation.org.au
• ADD BUSINESS UNIT/FLAGSHIP NAME
Thank you