2. SROI - Just another attempt to
turn social programs into
commercial transactions?
3.
4.
5.
6.
7. “Dear Mr Ghandi,
We regret we cannot
fund your project
because the link
between spinning
cloth and the fall of
the British empire
was not clear to us”
19. Presentation outline
1. What is SROI?
2. Contexts
3. How is SROI Implemented
• Stages
4. Uses (inappropriate)
5. Weaknesses
6. Strengths
20. 1. What is SROI?
• Approach to program, project and policy evaluation that
aims to account for non-financial outcomes using monetary
values to represent them.
• A way of reporting on value creation measuring
social, environmental and economic results.
• Includes a consistent approach with standard steps.
• Strong emphasis on involving stakeholders.
(Nicholls, Lawlor, Neitzert, & Goodspeed, 2012)
21. SROI measures the value of social benefits created by
an organisation, in relation to the relative cost of
achieving those benefits, expressed in a SROI ratio:
SROI ratio = present value
value of inputs
(Rotheroe & Richards, 2007)
22. Two types of SROI
Evaluative
• Conducted
retrospectively
• Based on outcomes that
have already taken
place.
• Preferred use in ongoing
evaluation and not as a
final outcome measure.
Forecast
• Conducted before hand.
• Predicts how much social
value will be created if the
activities meet their
intended outcomes
(Nicholls, Lawlor, Neitzert, & Goodspeed, 2012)
23. Brief history and context
• In 1997, REDF (Roberts Enterprise Development Fund, USA) launched an
initiative to asses impact of non for profits. In 2000 SROI was first
documented.
• A Network of practitioners was formed in 2006: SROI Network (UK and USA).
• New Economics Foundation in the UK edited a DIY Guide to Social Return on
investment in 2007.
• Office for the Third Sector (UK) developed a Measuring Social Value project
in 2008, aiming to develop SROI.
(Flockhart, 2005)
(Lingane & Olsen, 2004)
(Arvidson, Lyon, Mc Kay & Moro, 2010)
24. Purpose
Its fundamental purpose is to provide a model for allocating
monetary expression of the value of outcomes for which no
agreed market value exists.
It can de used for a range of evaluation purposes:
• Assess projects. (Forecast)
• Demonstrate achievements (Evaluative).
• Help improve organisational operations.
(Nicholls, Lawlor, Neitzert, & Goodspeed, 2012)
25. 7 Principles of SROI
Involve stakeholders.
Understand what changes.
Value the things that matter.
Only include what is material.
Do not over-claim.
Be transparent.
Verify the result.
33. How is it different to Cost Benefit Analysis?
SROI
• Used by managers to inform the
practical decision-making
optimizing their social and
environmental impacts.
• Strong explicit emphasis on
stakeholders and the types of
involvement they can have.
• Comparison is not recommended,
unless certain precautions are
taken.
CBA
• Used by funders outside an
organization to determine whether
their investment or grant is
economically efficient.
• Does not necessarily include
stakeholders.
• Aimed at comparison.
• (Arvidson, Lyon, Mc Kay & Moro, 2010)
34. 2. Context
• Designed originally to be used among NGOs and not for profits
(“Third Sector”).
• Growing interest in social value measures in the contexts of
increased outsourcing of the delivery of public services, and the
increased need of funders to secure real value for money.
(Wood & Leighton. 2010)
• Can be used by:
• Private businesses.
• Non for profit and social organizations.
• Government departments (Public Service Commissioners)
• Funders.
(Nicholls, Lawlor, Neitzert, & Goodspeed, 2012)
35. 3. How is SROI Implemented
Reporting, using and embedding
Calculating the SROI
Establishing impact
Evidencing outcomes and giving them a value
Mapping outcomes
Establishing scope and identifying key stakeholders
36.
37. 4. Inappropriate uses
It is NOT appropriate to compare the social
return on investment ratios alone.
Also NOT appropriate when:
• A strategic planning process has already been undertaken and is already
being implemented and there is no chance of modifying;
• Stakeholders are not interested in the results;
• It is being undertaken only to prove the value of a service and there is no
opportunity for changing the way things are done as a result of the
analysis;
• Resources are scarce.
38. 5. Weaknesses
• Social Impact can be a personal or political measurement. (Lingane & Olsen.
2004)
• Needs considerable resources to be implemented. (Flockhart, 2005)
• SROI “readiness” mainly involves being able to identify and measure
organisational outcomes adequately in a quantitative way. (Wood &
Leighton, 2010)
• Can easily be misused focusing solely on SROI Ratio (Arvidson, Lyon, Mc Kay
& Moro, 2010)
• The “if it can not be measured it can not be managed” trap.
(Arvidson, Lyon, Mc Kay & Moro, 2010)
• Quantifying inputs can be very tricky. (Arvidson, Lyon, Mc Kay & Moro, 2010)
39. “ An SROI analysis is only as good as the data that
is put in. In addition to properly resourcing
organisations to collect outcomes data, SROI
analyses can be strengthened by shared research
on outcomes, proxies, and indicators”
(New Economics foundation, 2008. in Wood, Leighton. 2010. p 28)
40. 6. Strengths
• Fosters a commitment towards transparency and accountability (Rotheroe &
Richards, 2007)
• Promotes better communication and engagement between different stakeholders
• Expected to foster improvement of quality data
• Evaluative process promoted by SROI includes making organisations aware of
their own values
(Arvidson, Lyon, Mc Kay & Moro, 2010)
• SROI principles have widespread approval, provide a benchmark for
organizations to set their goals and review their activities. ( Wood &
Leighton, 2010)
• Method includes specific guidelines that refer both to technical aspects as social
interaction / political aspects, allowing relatively consistent procedures.
42. 6 steps to implement SROI
Reporting, using and embedding
Calculating the SROI
Establishing impact
Evidencing outcomes and giving them a value
Mapping outcomes
Establishing scope and identifying key stakeholders
43. • Funded by WVA
• Project implemented
in 9 villages
• Project implemented
over 3 years
Project
Objectives
44. Project Objectives
• Project goal: To improve the livelihoods of the people of
Kalomo area.
• Project Outcomes: Farmers adopt sound natural
resource management practices
45. Project interventions
1. Community mobilisation around FMNR
2. Intensively training of community members
on FMNR practices
3. Promotion of complementary natural
resource management (NRM) techniques
4. Strengthening of community structures
46. Rationale
• 1. To find out what
project outcomes
impacted on key
stakeholders
48. All as a way of
interpreting the
project‟s value as a
result of the
investment made
49. Data collection methods
•Focus Group discussions of primary
stakeholders
•Key Informant interviews of selected
•Household survey stakeholders
•Visual data of geographical area
•Shadow pricing
50. SROI stage 1: Scope and stakeholder
•Measurement over the 3 yr
implementation period
•Primary stakeholders – Lead
farmers
•Neighbouring farmer households
•Comparison group
51. • Stakeholders validated and identified the following
outcomes
1. Increased household and communal assets in
the form of trees and livestock
2. Increased household consumables sourced from natural
resources
3. Increased household income
4. Improved health
5. Psychological Benefits – increased hope, aesthetics
6. Economic assets
7. Environmental Benefits
SROI stage 2: Map Outcomes
52.
53. SROI stage 3:
Evidencing and Valuing Outcomes
• Step 1: Develop outcome indicators:
• Remember outcome: Increased household and communal
assets in the form of trees and livestock. Indicator is
• Nº of Households reported increased availability of
and accessibility to the resources (rafters for
re/construction, firewood for cooking, thatch for
roofing, and herbal medicines for basic treatment)
• Amount of trees in the area
54. • Step 2: Collect outcomes data
• How many experienced this change? 52 Households
• How many trees in the area had regenerated? 1 000 000
• Step 3: Establish how long the outcomes last
• 6 years (2 years of project + 4 years post-project)
• Step 4: Put a value on the outcome.
• Market value of rafters, firewood, thatch, and herbal medicines (not
for trade)
• Communities thought about how they used to collect it before and the
risks involved
• They valued this particular outcome at $100,000
56. • Impact is only what is a result of the
intervention
• Deadweight
What would have happened anyway?
• 9% of those not accessing FMNR said access to more wild
resources had increased
• Displacement
• 0% displacement
SROI stage 4: Establishing Impact
57. • Attribution
How much of the outcome is because of other
organisations or interventions.
10% - One community was already partly organised
around tree protection
• Drop off
0% drop off - Community commitment unlikely to drop in
the short period of analysis due to the extent of
benefits, therefore trees will continue to be there or
even increase in number
58. To calculate the impact of this outcome:
= (Financial proxy x qty of outcome) minus dead weight
minus attribution
(100,000 x 52) - 9%
5,200,000 – 468,000
4,732,000 - 10%
4,732,000 - 473,200
= $4,258,800 in that year
59. • Five steps involved:
1. Projecting into the future – drop off rate.
2. Calculating the net present value – discount rate (time
value of money)
3. Calculating the SROI ratio = Present value/value of
inputs
4. Sensitivity analysis – Which assumptions have the
greatest effect on your model?
5. Payback period- At what point does return value >
investment.
SROI stage 5: Calculating the SROI
60. • Communicate meaningfully
• Short Report
• Transparent and concise
• Consistent
SROI stage 6:
Reporting Using and Embedding
61. Challenges
• SROI methodology is silent on whether to define „value‟ in
terms of money funds‟ origins or the recipient community
. E.g. a benefit in Choolwe, worth $500, is the equivalent of 50% of
the average per capita income in Australia.
Should it be expressed like that? Or should it be expressed as the
equivalent in the financier economy?
• SROI literature is weak on providing guidance on how to
facilitate stakeholder identification of meaningful values
for non-marketable benefits.
62. Challenges (continued)
• Evaluators found it difficult and time-consuming to explain
to stakeholders groups the notion of proxy financial values
for social, environmental and cultural returns.
• Interviews and focus groups took a lot longer or covered
fewer topics in the allotted time due to the cultural
disconnect of trying to elicit proxy market/financial values
from people who have an almost entirely non-economic
culture, livelihood and value system.
63. References
• Arvidon, M., Lyon, F., McKay, S., & Moro, D. (2010). The ambitions and challenges of SROI . UK: Third Sector Research Centre, University of Birmingham.
Retrieved from http://www.tsrc.ac.uk/LinkClick.aspx?fileticket=QwHhaC%2br88Y%3d&tabid=500
• Davidson, J. (2005) Evaluation Methodology Basics – The nuts and bolts of sound evaluation Thousand Oaks California: Sage Publications
• Flockhart, A. (2005). Raising the profile of social enterprises: The use of social return on investment (SROI) and investment ready tools (IRT) to bridge the
financial credibility gap. Social Enterprise Journal, 1(1), 29.
• Lingane, A., & Olsen, S. (2004). Guidelines for social return on investment. California Management Review, 46(3), 116-135.
• London Business School, New Economics Foundation and Small Business Foundation (2004). Measuring social impact: the foundation of social return on
investment (SROI). Retrieved from http://sroi.london.edu/Measuring-Social-Impact.pdf
• New Economics Foundation (2008) Investing for Social Value: Measuring social return on investment for the Adventure Capital Fund. London, UK: NEF.
•
• Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A guide to social return on investment (2nd ed.). UK: The SROI Network. Retrieved from
http://www.thesroinetwork.org/publications/doc_details/241-a-guide-to-social-return-on-investment-2012
• Rotheroe, N., & Richards, A. (2007). Social return on investment and social enterprise: Transparent accountability for sustainable development. Social
Enterprise Journal, 3(1), 31.
• SROI Network. (2011). The seven principles of SROI The SROI Network. Retrieved from http://www.thesroinetwork.org/publications/doc_details/140-the-
seven-principles-of-sroi
• Shergold, P., (2012) The Social Return on Universities Retrieved from http://www.onlineopinion.com.au/view.asp?article=13605
• Wood, C., & Leighton, D. (2010). Measuring social value. London, UK: Demos
• World Vision Australia. (2012). Social Return on Investment. unpublished manuscript
• Zappala, G. (2011). CSI Briefing Paper no. 5. Solving social problems & demonstrating impact. A tale of two typologies. Centre for Social Impact, University
of New South Wales, Sydney, Australia. Retrieved on 20/04/13 from http://www.csi.edu.au/assets/assetdoc/145e2b8d68c4a0b6/CSI_Briefing_5_Paper_-
_Solving_Social_Problems_and_Demonstrating_Impact_2011.pdf
Editor's Notes
The aim of our presentation is to explore evaluation approaches to determining outcomes in monetary values, with a particular emphasis on Social Return On Investment, a relatively new approach to cost benefit analysis. In the first part of the presentation, Helen will give a brief overview of the broader spectrum of cost benefit analysis models, and explain why we felt that Social Return on Investment is an evaluation approach that every evaluator should have in their tool box. Fran will outline the essence of SROI, including its purpose, principles and design. Cynthia will then demonstrate its application to evaluation practice by presenting a case study in which SROI was used to answer questions of value and impact in an evaluation of an agricultural and environmental aid programs in Africa. We finish with an exercise for you and our on line colleagues that we hope will enhance our collective appreciation of the value of this tool for evaluators.In acknowledgement of Amy’s edict to be creative in this assignment, our choice of communication tool for the first part is an Ignite presentation. Ignite presentations are the equivalent of a power point peptide or caffeine hit. It is a five minute presentation with 20 slides in which the slides automatically turn over every 15 minutes. So could you please hold any questions til the end of this part of the presentation. You won’t have to wait long!
There are many evaluation theorists and practitioners who put the conversion of benefits into dollar figures in the too hard or too dodgy basket. Julian King, when he presented to our class in session 3 was clear that many intangibles can’t or shouldn’t be valued in dollars: Zappala asserts that ‘many NFPs are falling prey to a ‘Wall Street Syndrome’ when it comes to measuring program effectiveness by applying complicated, time consuming and expensive measurement approaches to straightforward programs (Zapala: 2011) – yes there is risk and I will have more to say on that later, but the real danger is not so much that evaluators seek to monetise the value of non financial program outcomes but that in practicethe framework of evaluation is generally conceived far too narrowly. (Shergold:2012)
The question of how evaluators should value programs that cost funders money but create financial value AND at the same time build social benefit is in fact a noble quest, but where is our fellowship of evaluation cost-benefit theory ? Where is the fierce and comprehensive debate by evaluation theorists and academics on this topic? Even more simply, where is the extensive and considered analysis of the merit of using a SROI approach? Our search of literature found very little critical research on the technological and methodological aspects of SROI, and of what we found, hardly any came from evaluation theorists.
Attributing cost to intangible benefits is not tilting at windmills. Remember, governments and funders appear to ‘lose money’ on most programs in a financial sense, so there is a critical imperative for evaluators to get on with this, and do it well. It is about staying true to the essential evaluation tasks of assessing value and merit (Davidson:2005)
Judgment about the value of a program is an explicit feature of almost every impact or outcome evaluation. Regardless of whether the evaluation specifications have a requirement to do cost benefit analysis, as an evaluator, you won’t be able to reliably ascertain impact if you ignore the elephant in the room - the evaluation cost and benefit questions of economic impact, value and merit.
This slide says it all. Evaluation commissioners often feel short changed when evaluation of efficiency and effectiveness does not extend to consideration of cost effectiveness or cost benefit analysis.. Cartoon sourced from http://www.econosseur.com/economic-jokes.html
You can interpret this cartoon (sourced from http://www.econosseur.com/economic-jokes.html) in many ways, but to me its intention is to highlight how your ability to explain the link between a specific action and its outcome has long been a fundamental requirement for gaining support.
Judgment on the merit of a program or intervention means understanding cost and benefit related to the program. The evaluator will need skills and tools to capture cost and identify benefit. This is where, in general, the literature becomes very thin. If you are an evaluator looking for the right tools to allow you to provide ‘explicitly evaluative conclusions’ (Davidson:2005) and identify and place a value on all relevant outcomes , there has actually been very little development of the cost benefit models beyond those developed by and for economists. Fran and Cynthia will explain more on how SROI captures cost
Cost benefit approaches vary significantly, but in essence all attempt to define in monetary terms the relationship between cost and benefit for a given initiative. Of the two aspects of cost benefit analysis, cost is usually considered to be more easily measured particularly for program outcomes rather than products. But there are risks if only costs with an agreed market value, the diamonds, are measured. An evaluator needs to move beyond ‘lip service’ in defining benefits for the purpose of establishing thefull costs of investment in the program.
There are many forms and models for evaluating cost and benefit. Economists have donated these. Can you see any that were developed by evaluation theorists? This enormous gap in evaluation theory Cost benefit analysis often refers to determining cost and benefit to the whole of society; cost effectiveness (insert definitions for each term in the slide)
As you can see Galileo is the godfather of SROI. He was not only a renaissance man, he was a SROI man. Unfortunately none of his drawings on cost benefit models survived so the task of working out how you ‘make measurable what is not so’ has been left to contemporary evaluation theorists and practitioners. In a way, SROI attempts to pick up where Galileo left off
One reason we wanted to explore Social Return on Investment was because of its orientation towards measuring value from the bottom up and according to the perspective of different stakeholders. If you did the on line survey you will have some idea of how stakeholders might be engaged in defining, and putting a value on, outcomes.
Brief outline of why SROI is a significant tool for evaluators who use participatory approaches and/or qualitative methods, and its close relationship to program logic development.
A frequent dihlemma for an evaluator is that no market value exists for measurement of a program’s most significant outcomes. A defining aspect of SROI is that it provides a framework for measuring and accounting for both tangible and intangible values, and uses monetary values to represent these values. This is used to prepare ratios of benefits to costs, again expressed in dollar figures.
Hand over to Fran to explain what happens in Step 2 – the ‘miracle’ of SROI
Economist joke to finish A man walking along a road in the countryside comes across a shepherd and a huge flock of sheep. He tells the shepherd, "I will bet you $100 against one of your sheep that I can tell you the exact number in this flock." The shepherd thinks it over. It's a large flock, so he accepts the bet. "There are 973 sheep," says the man. The shepherd is astonished, because the man is exactly right. "O.K., I'm a man of my word. Take one." The man picks one up and begins to walk away. "Wait," cries the shepherd. "Let me have a chance to get even. Double or nothing that I can guess your occupation." The man agrees. "You are an economist for a government think tank," says the shepherd. "Amazing!" responds the man. "You are exactly right! But tell me, how did you deduce that?" "Well," says the shepherd, "Put down my dog and I'll tell you.”Joke sourced from http://www.econosseur.com/economic-jokes.html
What IS SROI?Brief history and contextPurposeSeven PrinciplesDifferences with CBA
Please note that all names and figures in this example are false. Also this case study is exclusively for the RARE class of 2013 only and must not be shared beyond this class. These are the conditions from the organisation to use this case study as the study is not yet finalised and one to the public.
