Evaluation made exciting, interesting and simple.

1. The f Word
Naomi Jones
Angela Paine
Amanda Prosser
Natalie Low
Stephen M K
St h McKay

2. What difference do we make?
Evaluating v and v projects

3. Overview
• Who are we?
• Why bother with evaluation?
• Wh t diff
What difference do you
d
make?
• How do we all go about
proving it?

4. Who we are
• A consortium of four organisations
• U d t ki
Undertaking an independent
i d d t
evaluation for v
• Working with v over the next two
years
• Using a variety of evaluation
methods
• Constantly feeding back findings
y g g
• Shaping the future development of
v

5. Turn to the person on your right
and introduce yourself and say
two words to sum up your view
of evaluation
f l ti

6. Why bother with evaluation?
• For accountability and learning
• Evidence funders - what difference their
money makes
y
• Evidence for trustee - strategic decision
making
• Evidence for volunteers - what
difference their time makes
• Attracting new volunteers – what they
will get out of it
• Organisational learning - what works;
what can b iimproved
h t be d

7. Less about
about…

8. More about
about…

9. What does it all mean?
• Outputs: The countable direct products of a
countable,
project or organisation’s activities (e.g. number of
training courses delivered)
• Outcomes: Changes, benefits, learning or other
g , , g
effects that happen as a result of your project (e.g.
increased skills and confidence among service
users)
• Impact: Effect of a project at a higher or broader
level in the longer term after a range of outcomes
has been achieved (e.g. reduce poverty)
• Monitoring: Collecting information on a project or
an activity to help report on a project. Needs to be
planned and managed.
• Evaluation: Using monitoring and other
g g
iinformation to make judgments about effectiveness
f ti t k j d t b t ff ti
of a project and to make changes and
improvements
(adapted from Charities Evaluation Service, 2007, Your Project and its Outcomes, CES: London)

10. Overall
aim
What you ultimately hope to Impact
achieve.
Should be brief and focused.
Specific aims
Outcomes
The changes you hope to achieve as a result of your work
Use words like ‘improve’, ‘increase’, ‘reduce’, ‘develop’
Outputs
O t t
Activities
The activities you undertake and the services you offer to
bring these changes about
Inputs Inputs Inputs Inputs
Adapted from Charities Evaluation Planning Triangle

11. What difference does your
project make?
• Your project will make a difference
to lots of different people:
– Volunteers
– Service users
– Your organisation as a whole
Your organisation as a whole
– The community in which you work

12. What difference does
y
your p j
project make?
• It will make a difference in a number of
ways:
• Things [aka ‘physical capital’]:
– The things produced by volunteering
• Number of training courses delivered or people
Number of training courses delivered or people
mentored
– Need to consider not just quantity but also quality
• People [
[aka ‘human capital’]
]
– Personal development
• Changes in levels of confidence, self‐esteem,
wellbeing
– Gaining new skills or enhancing existing ones
• Soft skills – e.g. team work, communication, inter‐
personal
• Hard skills – e.g. IT skills, construction, horticulture

13. • Money [aka ‘economic capital’]:
economic capital ]:
– The monetary value of volunteering and its
outcomes on all stakeholders, e.g. value of
training received, changes in employability,
training received changes in employability
value of work done by volunteers
• Relationships [
[aka ‘social capital’]:
]
– Relationships, networks, bonds of trust
between people and reciprocity
between people and reciprocity
• Attitudes [aka ‘cultural capital’]:
[ p ]
– Shared sense of cultural and religious identity
– Awareness and understanding of other peoples
cultures

14. How do we go about
proving it?
• You can collect evidence to evaluate
your project in a number of ways:
– Monitoring information
– Surveys of volunteers or staff members
– Focus groups of volunteers or service users
– Interviews with service users and key
stakeholders
– Photograph projects before, during and after
– Video diaries by volunteers or service users

16. How do we go about
proving it?
• You don’t have to do it all yourself:
– The evaluation website will include guidance
– You can get others involved: volunteers or students from
a local university might be able to help
a local university might be able to help
– You could make use of the questions within IVR’s
Volunteering Impact Assessment Toolkit

17. Case study: Children’s Hospice Assoc.
• Volunteers trained to undertake interviews and
surveys
y
• Volunteers reported positive experience:
– 84% had built friendships and networks
– But, 18% felt their skills weren’t being utilised
But, 18% felt their skills weren t being utilised
• Staff valued the role of volunteers:
– 83% felt volunteers helped create open and diverse culture
– But 18% felt they were over reliant on volunteers
But, 18% felt they were over reliant on volunteers
• Families reported a significant impact of volunteers:
– 73% said volunteers led to new friendships and networks
– But some concerns about vetting and role appropriateness
But, some concerns about vetting and role appropriateness
• Recommendations were built into a new strategy:
– Reviewing how volunteer skills are utilised to full effect
– Raising awareness of the role and volunteer status of the board
Raising awareness of the role and volunteer status of the board
– Exploring the possibility of extending volunteer roles
– Developing a leaflet for families about volunteers

18. On your table work to
complete the planning
and evaluation triangle
g

19. How will we go about
evaluating v?
g
• Analysing national data from the Citizenship
Survey
• Analysing monitoring data collected by v
• Speaking to key national stakeholders
• Undertaking a survey of young people
• D i
Doing telephone iinterviews with grant
t l h t i ith t
recipients
• Undertaking a series of case studies

20. How can you get
involved?
• Answer our phone calls and surveys!
• Become a case study
• Participate in pilot initiatives
• Provide good quality monitoring information
• Visit our website
– Launching in July
Launching in July
– Housed on vinspired.com/evaluation
– Purpose is to:
• Share any insights we’re picking up
Share any insights we re picking up
• Enable you to share experiences and best practice
• Enable you to feed into evaluation
• Provide guidance on self evaluation
d d lf l