SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere Nutzervereinbarung und die Datenschutzrichtlinie.
SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere unsere Datenschutzrichtlinie und die Nutzervereinbarung.
Scribd wird den Betrieb von SlideShare ab 1. Dezember 2020 übernehmen.Ab diesem Zeitpunkt liegt die Verwaltung Ihres SlideShare-Kontos sowie jeglicher Ihrer Inhalte auf SlideShare bei Scribd. Von diesem Datum an gelten die allgemeinen Nutzungsbedingungen und die Datenschutzrichtlinie von Scribd. Wenn Sie dies nicht wünschen, schließen Sie bitte Ihr SlideShare-Konto. Mehr erfahren
Evaluation – both progress against end goal and process (i.e. Did it work, did it work as we thought it would?)
Theory of Change (ToC) approaches to evaluation of community-based change initiatives also emerged in the mid-1990s. Notable early exponents include the Aspen Institute (Weiss, 1995) and The Annie E. Casey Foundation (1997). Although the focus of these early discussions was evaluation the argument that misinformed or conflicting theories of change could reduce the impact of a given initiative had direct implications for programme planning (Weiss, 1995, pp67, 71-2).
History (h/t Mark Cabaj http://www.naturalstep.ca/mark-cabaj): Evaluators in the 1960s concluded the policies and programs they were asked to evaluate did not come with clear enough goals/objectives and designs to develop an evaluation. Evaluators created “evaluability” assessment to help surface these issues early in the planning process. Evaluators began to get involved in helping policy makers and program designers to conceptualize the “intervention”.
Michael Quinn Patton – Essentials of Utilization Focused Evaluation pgs 3-4
Evaluation leads to action
Assumption that something needs to be changed i.e. things could be better than they are? There is a job to be done.
Defined as three key questions (‘What are you trying to change?’, ‘How will you affect change?’, and ‘Why do you believe your actions will affect this change?’) the ToC approach can provide a useful structure to frame important conversations.
Short statement (40-60secs) that contains all three elements. Clear, memorable, and makes sense to a stranger. If you can’t summarise your ToC then it’s probably too complicated, and unclear.
W. K. Kellogg Foundation. (2004). Logic Model Development Guide. Battle Creek, MI: W. K. Kellogg Foundation. 2004. Available: http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx
Common in planning practice to use a reverese logic model process i.e. Start with end result/impact and work backwards.
Funnell, S. C. and Rogers, P. J. (2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. Jossey-Bass
Black box – “In science, computing and engineering a black box is a device, system or object, which can be viewed in terms of its input, output and transfer characteristics without any knowledge of its internal workings.” Wikipedia
If there’s no explanation or prediction of how and why change will occur then it is very hard to evaluate a project’s claims/effectiveness
Scrutiny, challenges assumptions – esp. around causation. Will explain in further slides.
One of the key features of the ToC approach is the emphasis on scrutinising the causal linkages that exist between an organisation’s activities and its intended goals i.e. asking whether an organisation’s activities will bring about the changes they intend in the way they describe.
Important, but can be difficult. Requires time and access to both information, expertise and partnerships
Internal factors (e.g. peoples, systems, and resources) External factors (e.g. changes in government policy, economy, or population)
Mayne, J (2208) Contribution Analysis: An approach to exploring cause and effect, ILAC, http://www.cgiar-ilac.org/files/ILAC_Brief16_Contribution_Analysis_0.pdf
Also, stuff happens - wider implication for ToC approach as it demonstrates problems of predicting future events (how do you factoring unexpected events into your outcomes chain).
Weiss, C. (1995) ‘Nothings as practical as a good theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families’. In Connell, J. P. et al. (eds) Approaches to Evaluating Community Initiatives: Concepts, Methods and Contexts. Washington; Aspen Institute. Available: http://www.eric.ed.gov:80/PDFS/ED383817.pdf
What are the assumptions? E.g. first ‘?’ resources, need, design, etc.
Can also add risks and measures.
Risks can be rated similar to risk register exercise: Score 1-10 how important is this assumption / Score 1-10 How confident are we the assumption is correct?
What are the main assumptions underlying each step of outcomes chain?
Also what are the main risks? What are the indicators or measures i.e. how will we know that each step, each change, has occurred in the way that we predicted?
Plausible aka the sniff test e.g. This app will eradicate malaria
Can be a challenge as well i.e. Radical innovations may not seem plausible to entrenched or embedded practice
Confidential – please do not share without permission. Nominet Trust
Fixing a clear (and measurable) end goal/destination is crucial, but can be a time-consuming, complex and sensitive process
Joel Orosz? – complementary question to TOC Particularly relevant to funders as often faced with challenges/opportunities that vastly outsize their resources – where can we make the most different with what we have?
Reis, E The Lean Startup: How Constant Innovation Creates Radically Successful Businesses (2011)
Snowden, D. and Boone, M (2007) ‘A Leader's Framework for Decision Making.ʼ Harvard Business Review. November 2007 85 (11) p68-76
Human beings (both individual and especially collectively) are not machines, difficult to crank a handle and get predictable results. Connection between cause and effect often unclear, inconsistent and disproportionate.
Perhaps impossible to cover every possibility and predict sequence of outcomes in advance. However ToC could be seen as a very useful, open process, for constanttly scrutinising and testing out assumptions in order to improve what we do.
Nominet trust projects theory of change presentation 2016
Theory of Change & Measuring
Theory of Change – Background and Use
• Communication (verbal and visual)
Evaluation – 3 questions
What? So What? Now What?
Michael Quinn Patton
What is my project/organisation trying to
(Team Discussion – 15mins)
Theory of Change - Definition
“No matter how it is defined, at its heart, a theory
of change lays out what specific changes the
group wants to see in the world, and how and
why a group expects its actions to lead to
those changes. “
(Guthrie et al. 2005 – emphasis by Lamb, 2011, p6)
Theory of Change – Elevator Pitch (Groups
1. What are we (i.e. my project/organisation) trying to
2. What are we doing to bring about that change?
3. How and why do we believe our activities will bring
about the change we want to see?
Tangible results of the your
The changes that result (new
behaviours, changes in well being,
increased skills etc...
Outputs vs Outcomes
Theory of Change approach – In a word
Questioning Causality (aka ‘exploding the arrow’)
i.e. asking whether an organisation’s
activities will bring about the changes
they intend in the way they describe.
Causality ‘Exploding the Arrow’ (1)
1) Evidence – what
evidence is there the
intervention will lead
to the desired
2) Single or multiple
steps – will the
directly to the desired
*....or are there a number of micro steps along the way? Is there a chain of connected
outcomes? (aka: ‘so that’ chains)
(Funnell and Rogers, 2011)
Causality ‘Exploding the Arrow’ (2)
3) Linear or
multifaceted – is the
outcomes chain a
simple linear process
or is it multifaceted?*
4) Internal and External
factors – does the
outcomes chain take
into account the
influence of internal
and external factors?
*Do separate outcomes chains need to run in parallel in order to combine towards the final,
ultimate outcome? Necessary vs. Sufficient.
(Funnell and Rogers, 2011)
Causality ‘Exploding the Arrow’ (3)
5) Plausibility and coherency – ultimately any plan
that details a causal link between an
intervention and an outcome needs to be
scrutinised, ideally by those who have
knowledge relevant to the intended change
(e.g. practitioners, academics and intended
beneficiaries). (Funnell and Rogers, 2011)
Making the implicit explicit
“..takes for granted that social programs are based on
explicit or implicit theories about how and why the
programme will work. The evaluation should surface
those theories and lay them out in as fine detail as
possible, identifying all the assumptions and sub-
assumptions built into the program. … The aims is to
examine the extent to which the program theories
hold.” (Weiss, 1995, pp66-7)
What makes a good theory of change? (1)
1) Plausible - stakeholders
believe the logic of the
model is correct: if we do
these things, we will get
the results we want and
2) Doable - human, political
and economic resources
are seen as sufficient to
implement the action
strategies in the theory
What makes a good theory of change? (2)
3) Testable - stakeholders
believe there are credible
ways to discover
whether the results are
4) Meaningful -
stakeholders see the
outcomes as important
and the magnitude of
change in these
outcomes being pursued
as worth the effort
Theories of Change – Some Examples
One Project – Several theories of change
Some projects use a limited number of activities to serve multiple
purposes. For example, a project involving the construction of a water
well in a village in Burundi could be linked to the following theories of
• By providing access to a closer water source, we will improve safety
(less danger of assault or rape, as is the case in many refugee camps)
and health (improved sanitation, no contamination in drinking water,
ability to cook with water).
• If there is closer access to water, then children can attend school
instead of having to travel for hours to provide water for the family.
• If we teach locals how to build water wells, then they can repeat the
process in neighbouring villages, thus spreading access to water
throughout the region.
Practical Approaches to Theories of Change in Conflict, Security & Justice Programmes.
Department for International Development
Defining success – the ToC approach:
1) Offers guidance on
how to get to your
ultimate goal, it will
not tell you what that
goal should be
2) Is dependent on
having a clearly
goal, but defining,
agreeing and sticking
to such a clear can
be a challenge
Theory of Change Toolbox
ToC approach contains different elements
and levels of detail that will appeal
variously to organisations depending on
their interests, values, and size.
Theory of Leverage
“Give me a place to stand and with a lever I will move the whole
Lean Start Up and
Theories of Change:
Interesting parallels e.g.
(ToC) and validating
Criticisms and Challenges
• Limited research available – especially for
non-US contexts and smaller charities
• Most documentation produced by ToC
• ToC still an emerging approach (no
Criticisms and Challenges
Complexity Science and Complex Adaptive Systems
1. Complex systems can be defined as having various
characteristics including a large number of
interacting elements and being dynamic
compositions, where the whole is larger than sum
of parts (Snowden & Boone, 2007, p71).
2. Cause and effect pathways often unclear, and
visible only in hindsight
The more complex (and less linear) the social change the more difficult it is to develop a ToC
ToC approach about unearthing assumptions, but also values,
beliefs and paradigms
Tension - evaluators like stable theories and models, whilst
innovators happier working with still evolving models
Half-life/strategic decay. Also applies to ToC, which need to be
refreshed/reviewed. ToC change. The three main reasons are:
1) context changes
2) new players/change makers, and
3) new evidence.
Complex change = variety (Gladwell’s pasta sauces), i.e. not
'what works', but 'what works for who, in what context and why?'
ToC Approach – better as a thinking
tool than a planning tool?
Papers & Books
• Brest, P. (2010). The Power of Theories of Change. Available: http://www.kipp.org/news/stanford-social-innovation-review-
• Clark, H and Anderson, A. (November 2004) Theories of Change and Logic Models: Telling Them Apart. Presentation at
American Evaluation Association, Atlanta, Georgia. http://www.theoryofchange.org/wp-
• Funnell, S. C. and Rogers, P. J. (2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models.
• Keystone Accountability. (2008). Developing a theory of change: A guide to developing a theory of change as a framework
for inclusive dialogue, learning and accountability for social impact. Available: http://www.researchtoaction.org/theory-of-
• Lamb, B. (February 2011) Campaigning for Change: Learning from the United States. Campaigning Effectiveness, NCVO.
• Mayne, J (2208) Contribution Analysis: An approach to exploring cause and effect, ILAC, http://www.cgiar-
• Stachowiak, S. (2009) Pathways for Change: 6 Theories About How Policy Change Happens. Seattle, Organizational Research
• Weiss, C. (1995) ‘Nothings as practical as a good theory: Exploring Theory-Based Evaluation for Comprehensive Community
Initiatives for Children and Families’. In Connell, J. P. et al. (eds) Approaches to Evaluating Community Initiatives: Concepts,
Methods and Contexts. Washington; Aspen Institute. Available: http://www.eric.ed.gov:80/PDFS/ED383817.pdf
For a critical view:
• Reusga, A. (2011). Philanthropy’s Albatross: Debunking Theories of Change.