1. Date: 15 Nov 2010
What difference did it
make?
Penelope Beynon

3. Evaluating an Open API
Who should benefit from an Open API?
How will they benefit? Ie what will be different
if the Open API is successful?
How will you know if those benefits are
realised?

4. How are we evaluating MK4D?
Monitoring outputs – what did we do?
– webstats, finance, publications, reach figures
– Event reports
Evaluating outcomes – what difference did it make?
– Principles of outcome mapping
– Interviews, Surveys, ‘tests’, data analysis
Improvement - what lessons did we learn?
– Event reports
– Learning labs

5. Principles of outcome mapping
OM was developed by International Development Research Centre
(IDRC) in the late 1990s – an online user group is run by Overseas
Development Institute (ODI) http://www.outcomemapping.ca/
OM is a methodology for planning, monitoring and evaluation.
– OM focuses on outcomes rather than outputs
– OM acknowledges the limit to your influence
– OM is people-focussed
– OM looks for contribution rather than attribution
In IDS we are exploring use of OM for understanding the research
uptake process, and evaluating research uptake work

6. Intentional Design
1. What are you trying to achieve? (vision)
2. How are you going to do it? (mission)
3. Who do you need to influence? (boundary partners)
4. How will other people behave if you have been successful?
(outcome challenge)
5. What changes do you hope to see in other people along
the way? (progress markers)
6. What will you do to achieve the mission? (strategy maps)
7. How will you behave if you are working well?
(organisational practices)
Outcome and Performance
Monitoring
8. How will you make monitoring
manageable? (monitoring priorities)
9. How will you record changes in other
people? (outcome journals)
10. How will you record changes in what
you are doing? (strategy journal)
11. How will you record changes in your
own behaviour? (performance journal)
Evaluation Planning
12. What will you explore in-depth?
(evaluation plan)
(adapted from Earl et. al. 2001)
Three stages of outcome mapping

7. Project team
Outputs:
Inputs, activities
Partners/actors
Outcomes:
Changes in
attitudes/awaren
ess/behavior
Beneficiaries
Impact:
Changes in
economic,
physical and
social
wellbeing
Sphere of
control
Sphere of
influence
Sphere of interest
Context Context Context
Spheres of control, influence, interest

8. Evaluating an Open API
Who should benefit from an Open API?
How will they benefit? Ie what will be different
if the Open API is successful?
How will you know if those benefits are
realised?