Pro bono economics

2© Pro Bono Economics
PBE aims to match professional economists with
charities;
providing pro bono help to measure performance
and understand charity impact, and
fostering a culture of volunteering within the
economics profession

History
2009 - incorporated and registered with Charity Commission
11 projects – no management – none completed
2010 – 2 members of staff from September
Managing 24 projects – 1 completed
2012 – + part-time economist
Managing ± 40 projects – 41 completed to date

People
Supported by over 260 volunteers from the public sector
(Government Economic Service, Bank of England and
others), private consultancy firms (Charles Rivers Associates,
Europe Economics, Frontier Economics, FTI Consulting, and
Oxera), academics and individual economists
Trustees Martin Brookes (Chair), Andy Haldane (Treasurer),
Kate Barker,Lynne Berry, Lucy Heady,Nicola Pollock, Dave
Ramsden
Patrons include Gus O‟Donnell and Adair Turner

Projects
160 charities have registered 168 projects
Where: 127 London & South East, 25 North & East, 5 in
Scotland,1 Northern Ireland and 2 Wales
Who for: 13 with homeless people,13 with ex-offenders, 9 with
disable people, 4 with the elderly, 8 with other charities
What: 12 employment support, 15 using arts or sport, 20
looking for analysis to support campaigning
88 proposals have been withdrawn or rejected
79 projects have had 162 economists allocated to them
41 reports completed to date
16 projects in pipeline, 14 being scoped, 9 being worked on

© Pro Bono Economics
Vfm of interventions UK analysis
Data collection
advice
Other economic
advice
Some charities PBE has helped

Types of evaluation
Process: how and how well is the project working?
Impact: what change occurred as a result of the project?
Economic: were the benefits worth the costs?
requires an impact evaluation

Measuring outcomes
Measure a characteristic before the intervention
e.g. number of people with a cold
Deliver the intervention
e.g. cold and flu medicine
Measure the same characteristic after the intervention
Difference = impact?
Most people get better from a cold with or without medicine

Measuring outcomes
Measure a characteristic before the intervention
e.g. school attendance
Deliver the intervention
e.g. mentoring scheme to improve attendance
Measure the same characteristic after the intervention
Difference = impact?
What other things might be going on at the same time?
New breakfast club
New attendance officer

What would have happened
anyway?
i.e. without the intervention
N.B. the intervention did take place, so we can never know for
certain
Can we measure changes in outcomes for a credible
comparison group, who didn‟t get the intervention?
credible = no reason to think that there may be differences
(bias) which affect the outcomes
e.g. employment programme for women cf. national
employment outcomes – unbiased?

Control Groups
National average
PBE: Foundation Training Company
Before
PBE: MEAM pilots – adults with multiple needs
Matched records in admin dataset
Peterborough Social Impact Bond/MoJ datalab
Natural experiment in charity data
PBE: Barnardos
RCT
Education Endowment Foundation/Social Research Unit

Estimating impact
Measure outcomes
Best estimate of what would have happened anyway (the
counterfactual)
Outcomes minus counterfactual = impact
A measurement minus an estimate = an estimate

Discuss
What are you using/might you use as a control group?

Data 1: project admin
Can provide data on activities and outputs (monitoring)
Rarely useful for outcomes (evaluation)
Tends to be less reliable over time at an individual level
Often essential for identifiers to link to large data sets
need to ensure this is gathered from the start

Data 2: survey
Produce your own survey
Pros:
can tailor to your outcomes
Cons:
time consuming (design, trialing, collection)
usually need specialist input
biases
data collection from control groups
Sometimes possible to link to other large surveys

Data 3: large data sets
Educational attainment
National Pupil Database (NPD)
Re-offending
PNC, prison, probation, MoJ Data lab
Pros:
good measures of specific outcomes
can „match‟ / control – „synthetic‟ counterfactual
Cons:
access (security)
use….needs v high level of expertise
bias (unrecorded variable bias)

Data 4: RCTs
Randomised control trials
Randomly allocating participants between „treatment / project‟
and „control‟ groups & compare outcomes.
Produces the best counterfactual with least bias → best
estimation of impact

Issues 1: bias
Response bias
especially to surveys
Unrecorded variable bias
self selection
attitude
personality
RCTs are the best way to overcome bias

Issues 2: sample size
Only relevant once bias is eliminated or known
Needs to be large enough to be representative, but too large
is wasteful
What is large enough?
Ask a statistician

Issues 3: timing
Outcomes happen as a result of activity
Therefore can only be measured after the activity not during
it!
Can be difficult for funders who want reports at a project‟s
close

Discuss
Sample evaluations:
What level of evidence?
Strengths and weaknesses?

Levels of Evidence

Definitions
Social value
monetary worth of something to society or
Social value act
“how what is proposed to be procured might improve the
economic, social and environmental well-being of the relevant
area”
Social cost benefit analysis
CBA including wider benefits
Social return on investment
particular methodology for social CBA emphasising role and
input of stakeholders

Whose perspective?
Value to society
won‟t include transfer payments e.g. tax and benefits
Value to individuals
change in income including benefits (+)
Value to Exchequer
Increase in tax receipts (+) or increase in benefit payments (-)
Potential costs avoided
Cashable savings

Which benefits?
Hard outcomes
Qualifications → employment → lifetime earnings
Costs avoided by public services
Soft outcomes
Early stages of work in this area
Related to income, so only for those who are earning
But still important so
No numbers without stories, no stories without numbers

Which costs and values?
Input costs
Programme costs
Volunteer input
Other activities
Benefit values
Average costs
Marginal cost

Discuss
What type of valuation might be most relevant to you?

Pro bono economics

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (9)

Andere mochten auch

Andere mochten auch (7)

Ähnlich wie Pro bono economics

Ähnlich wie Pro bono economics (20)

Mehr von SWF

Mehr von SWF (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Pro bono economics