1. Introduction to the Improve It Framework -
Effectiveness:
Improving and Proving It Conference
Edinburgh November 2012
Ben Jackson Bjackson@bond.org.uk
Chief Executive, Bond
bond.org.uk

2. Context

3. Consortium Effectiveness Programme
…supports UK NGOs in improving how they measure, manage and
demonstrate their contribution to social change
1. Developing agreement and supporting implementation of a sector wide
framework of common outcome indicators and data collection
tools (Improve it Framework); and online organisational self-
assessment and resource portal (Organisational Health Check;
NIDOS Effectiveness Tool); and agreed quality standards regarding
NGO evidence.
2. Building knowledge and skills to support members to measure and
manage effectiveness and value for money
3. Creating an enabling environment that encourages and supports
organisations to deliver improvements in their effectiveness
bond.org.uk 3

4. Consortium Effectiveness Programme:
Key elements
• NIDOSEffectiveness Tool /
Bond Organisational Health Check
• Value for Money
• NGO Evidence Principles
• Improve it Framework
bond.org.uk 4

5. Contributors to the programme
Consortium partners:
bond.org.uk 5

6. How the strands of programme fit together
Development
effectiveness
Robust, Appropriate
practical approaches to
approaches to managing cost
measuring efficiencies &
effectiveness effectiveness
Strong systems, culture, leadership etc
Bond Organisational Health Check/
NIDOS Effectiveness Tool
bond.org.uk 6

7. Improve It
Framework

8. The Improve it Framework
What is it?
• A tool for improving how UK NGOs measure,
learn from and demonstrate outcomes and
impact
• A platform for sharing information on
outcomes, indicators and data collection tools
Why are we developing it?
• Developing robust methods for measuring
effectiveness is a collective challenge for UK
NGOs
• UK NGOs need to start telling a more robust story at sector level as well as
at organisational level about their added value
bond.org.uk 8

9. Improve it Framework: what it is and is not
It is not… It is…
• …about creating a single way of • …about providing a collective resource
assessing effectiveness –it’s about that UK NGOs can draw on when
encouraging greater harmonisation developing their own context specific
and consistency where appropriate M&E frameworks
• …going to offer an off the shelf • …about developing shared
answer to measuring effectiveness, approaches where we can
but it will provide a common starting • ….about providing organisations with
point the tools to be able to tell a more
• …going to produce an encyclopaedia robust story of how they are
of indicators and tools. There will be contributing to change
an element of prioritisation.
• …going to be practical & simple
• …an organisation specific tool;
• …going to be dynamic & will continue
seeing 60% of your organisation in it
to evolve
is good enough
bond.org.uk 9

10. Where the Improve it Framework can help in
your work
Focus of Improve it Framework
1 2 3 4 5 6
Identify who Identify the Collect,
Identify the
Articulate you will work Identify what evidence analyze and
problem &
your vision with & you want to that you will draw
underlying
of change influence & change collect and conclusions
causes
how how for the data
Outcomes Indicators & Principles
7 8 tools
Reflect and
learn from Communicate
the data results
bond.org.uk

11. In practice

12. Improve it
Framework
Capacity
development

13. Domains of Change Framework for Education
Outcomes
a) Laws, policies and practices b) Adequate resources are allocated c) Performance in
are in place that support quality and adequate teachers recruited and education is monitored
education that is accessible for trained for the provision of quality at a national and local
all children education for all level
Outcomes
Outcomes 2
a) School management
 Mothers, fathers and other
a) Governments and committees exist and manage
caregivers support all their other mandated bodies schools effectively
children to access education
ensure all children b) School managers access,
b) Communities actively support access quality manage, and deploy resources
and participate in children’s education in an effective and transparent
education way
6 3
c) CSOs support and advocate for c) Teachers, resources and
improved education for all infrastructure are in place to
Communities actively Schools are well support education
support the education managed, safe and well
of all children d) Schools are safe and
resourced
healthy environments for girls
1) All children and boys
access quality
education where
Outcomes
they learn useful
knowledge and a) Teaching is high quality,
skills child-centred and child-
Outcomes friendly
a) Children participate in a b) Teachers deliver a full
meaningful way in decisions that 5 4 allocation of lessons
affect their education c) Teachers are trained and
b) Children participate in their All children participate All children receive supported to deliver high
own and others’ learning in their education and their full allocation of quality teaching
in school life high quality teaching d) The curriculum is relevant
c) Children have a positive
and appropriate
attitude towards education
e) Teaching supports equality
and empowerment

14. Domains of Change Framework for Education
Outcomes
a) Laws, policies and practices b) Adequate resources are allocated c) Performance in
are in place that support quality and adequate teachers recruited and education is monitored
education that is accessible for trained for the provision of quality at a national and local
all children education for all level
Outcomes
Outcomes 2
a) School management
 Mothers, fathers and other
a) Governments and committees exist and manage
caregivers support all their other mandated bodies schools effectively
children to access education
ensure all children b) School managers access,
b) Communities actively support access quality manage, and deploy resources
and participate in children’s education in an effective and transparent
education way
6 3
c) CSOs support and advocate for c) Teachers, resources and
improved education for all infrastructure are in place to
Communities actively Schools are well support education
support the education managed, safe and well
of all children d) Schools are safe and
resourced
healthy environments for girls
1) All children and boys
access quality
education where
Outcomes
they learn useful
knowledge and a) Teaching is high quality,
skills child-centred and child-
Outcomes friendly
a) Children participate in a b) Teachers deliver a full
meaningful way in decisions that 5 4 allocation of lessons
affect their education c) Teachers are trained and
b) Children participate in their All children participate All children receive supported to deliver high
own and others’ learning in their education and their full allocation of quality teaching
in school life high quality teaching d) The curriculum is relevant
c) Children have a positive
and appropriate
attitude towards education
e) Teaching supports equality
and empowerment

15. Domains of Change Framework for Education
Outcomes
a) Laws, policies and practices b) Adequate resources are allocated c) Performance in
are in place that support quality and adequate teachers recruited and education is monitored
education that is accessible for trained for the provision of quality at a national and local
all children education for all level
Outcomes
Outcomes 2
a) School management
 Mothers, fathers and other
a) Governments and committees exist and manage
caregivers support all their other mandated bodies schools effectively
children to access education
ensure all children b) School managers access,
b) Communities actively support access quality manage, and deploy resources
and participate in children’s education in an effective and transparent
education way
6 3
c) CSOs support and advocate for c) Teachers, resources and
improved education for all infrastructure are in place to
Communities actively Schools are well support education
support the education managed, safe and well
of all children d) Schools are safe and
resourced
healthy environments for girls
1) All children and boys
access quality
education where
Outcomes
they learn useful
knowledge and a) Teaching is high quality,
skills child-centred and child-
Outcomes friendly
a) Children participate in a b) Teachers deliver a full
meaningful way in decisions that 5 4 allocation of lessons
affect their education c) Teachers are trained and
b) Children participate in their All children participate All children receive supported to deliver high
own and others’ learning in their education and their full allocation of quality teaching
in school life high quality teaching d) The curriculum is relevant
c) Children have a positive
and appropriate
attitude towards education
e) Teaching supports equality
and empowerment

16. School management committees exist and manage schools effectively
Indicators Tools
• # and % of schools with a Parent Teacher The Right to Education indicators and form
Association (PTA) for section 9: right to transparent and
accountable schools.
• Level of activity of the Parent Teacher
The Plan UK child friendly tool Section 1:
Association (PTA)
School management
• # and % of schools with a School The VSO quality scale for education services
management committee (SMC) quality areas on education management
and community engagement
• Level of activity of the School
management committee (SMC)
bond.org.uk 16

21. Timelines

22. What to look forward to
• Organisational Health Check pilot starts (Summer 2012)
• Launch of pilot Evidence Principles (October 2012)
• Beta version of the Improve it Framework (December 2012)
• Comic Relief, Big Lottery Fund, DFID, Bond research into the cost
of measuring effectiveness (Spring 2013)
• Guide to demystifying theories of change (Spring 2013)
bond.org.uk 22

23. Thank you
Contact:
Ben Jackson
Chief Executive
bjackson@bond.org.uk
Regent’s Wharf • 8 All Saints Street • London • N1 9RL • United Kingdom
+44 (0)20 7837 8344 • bond@bond.org.uk • bond.org.uk bond.org.uk 23

24. Organisational Health Check
What is it?
• An online self-assessment that:
• Identifies strengths and weaknesses in
organisational capacity / functionality
• Facilitates benchmarking with peers
• Sign posts to other initiatives, tools, training and
support
Why are we developing it?
• Strong systems, leadership etc are key building
blocks for delivering good development outcomes
• A common framework can help facilitate structured peer learning and the
sharing of best practice
• An online self-assessment enables bond to provide more tailored
capacity support
bond.org.uk 24

25. Value for Money (VfM)
What are we doing?
• Providing common tools and frameworks
for NGOs to navigate the VfM agenda
• Bringing NGOs together to share emerging
experience
• Ensuring constructive dialogue with DfID
Why are we doing it?
• At the heart of the VfM agenda are important questions about how UK NGOs
use their resources
• VfM is a part of the wider results agenda
• Want to ensure VfM remains appropriate and proportionate to NGO
operations and programmes
bond.org.uk 25

26. NGO Evidence Principles (pilot to May 2013)
1. Voice and Inclusion: We listen to beneficiaries’ perspectives on the
effects of the intervention, and can identify who has been affected and
how.
2. Appropriateness: We use methods for data collection and analysis
that are justifiable given the nature of the intervention and purpose of the
assessment
3. Triangulation: We make conclusions about the intervention’s effects by
using a mix of methods, data sources, and perspectives
4. Contribution: We show how change happened, explore a range of
explanations for why it happened, test our theory of how we thought it
would happen, and explain how we contributed
5. Transparency: We are open about the data sources and methods
used, the results achieved, and the strengths and limitations of the
evidence
bond.org.uk 26

27. NGO Evidence Principles – the checklist
A NGO checklist for assessing the quality of evidence (DRAFT)
Piece of evidence being assessed: Name of assessor:
Principle Criteria 1 2 3 4 NA Comments / evidence
1a. Are the perspectives of beneficiaries included in the
1) Voice and Inclusion evidence?
We listen to 1b. Are the perspectives of the most excluded and
beneficiaries’ marginalised groups included in the evidence?
perspectives on the
effects of the 1c. Are the findings disaggregated according to sex,
intervention, and can disability and other relevant social differences?
identify who has been
affected and how 1d. Do beneficiaries play an active role in designing the
assessment process and/or analysing data?
Score for voice and inclusion (average of criteria scores):
2a. Are the methods used to collect data appropriate given
2) Appropriateness the nature of the intervention and the purpose of the
assessment?
We use methods that 2b. Is the size and composition of the group from which
are justifiable given data is collected appropriate given the nature of the
the nature of the intervention and purpose of the assessment?
intervention and 2c. Is the composition of the team collecting and analysing
purpose of the the data appropriate given the nature of the intervention
assessment and purpose of the assessment?
2d. Are the methods used to analyse the data appropriate
given the nature of the intervention and purpose of the
assessment?
Score for voice and inclusion (average of criteria scores):
bond.org.uk

28. NGO Evidence Principles – the results
• A quality assurance
framework Voice & Inclusion Appropriateness
13/16 7/16
• A kite mark / seal of
approval Transparency Triangula on
9/16 12/16
Contribu on
• A tool for identifying skill 4/16
gaps and building capacity
bond.org.uk 28