1. Evidencing Digital Inclusion in UK
Dr Alice Mathers, Online Centres Foundation
5th Social Digital Research Symposium

2. The key issue…
‘there is little incentive for digital inclusion
practitioners to critically evaluate their
practices in a meaningful way’
(Seale and Dutton, 2012)

3. Evidence from practice
Top down interventions and national programmes
Bottom up community initiatives and innovation
Differing approaches to
measuring impact
on and of
Digital Inclusion

4. Existing evidence of impact
Digital Unite
• Providers of free learning content
• Online community forum.
• Home visit tuition through network of local Digital Unite Tutors
• Training for organisations through community learning
programmes, staff training programmes and digitally-orientated
customer care services.
• Structured skills development through Digital Champion ITQ (DC
ITQ)
• Advocacy

5. Existing evidence of impact
Digital Intervention:
• Get Digital, Digital Unite (2010-2012)
Programme focus:
• Digital inclusion through
comprehensive, structured learning
programmefor residents in sheltered
housing, working with staff, landlords and
wider community.
Evaluation measurement:
• Mixed method: quantitative (online and
paper
surveys, baseline, progression, outcome),
plus qualitative data from 12 case study

6. Existing evidence of impact
Impact:
• Skills and confidence: 57% of staff have increased confidence in
use of ICT, 83% of residents have a more positive attitude towards
computers and the internet,88% likely to use a computer/the
internet for email, phone calls, search for information, access
public services and use social networking sites.
• Reduced isolation and loneliness: 50% find it easier to keep in
touch, 42% easier to meet new people, 42% contact with family
and friends
• Enhanced health and wellbeing: Almost 20% of residents now look
after themselves better and know more about their health.
Evaluators: NIACE (2012)

7. Existing evidence of impact
UK online centres
• National coverage through a network of 5,000 centres
• Longitudinal daily DI data collection + daily surveys
• Products, support, advocacy, research
• 4 specialist networks:
– Disabled people
– Carers
– Older people
– Into work
1m
people
get online
430,000
shift
(43%)
1.634m
contacts
shifted/mt
h
(3.8 per
person per
month)
£157m
saved in a year
based on £8
saved per
contact shifted
online
(Image source: Helen Milner, Nov 2012)1,089,402 people as of 24th April 2013

8. Existing evidence of impact
Digital Intervention:
Social Impact Demonstrator Project (2007-8)
Programme focus:
• Partnership working through UK online
centres to reach socially disadvantaged
people and engage them in ICT activities.
Evaluation measurement:
• 3 stages of data collection:
baseline, progression and outcomes.
• 4 focus groups and 8 individual semi-
structured interviews
• 20 individual project evaluations

9. Existing evidence of impact
Impact:
• Cost efficiency: 12,234 people were engaged at a cost of
£163 per person.
• Skills & confidence: 60% now happy using computers and
internet, 70% now confident.
• Employability: 40% progressed in terms of training and
employment etc.
• Social capital: 39% met up with new friends and 32%
volunteered.
Evaluators: Ipsos MORI (2011)

10. Existing evidence of impact
Age UK
• Digital Inclusion Network of over 200 members
• The Network is a UK-wide membership programme managed and
delivered by Age UK in partnership with Age Scotland, Age Cymru
and Age NI.
• Best practice: developing and share best practice approaches.
• Advocacy: working with key stakeholders to gather
intelligence, provide a channel for discussion on digital inclusion
issues at a local, regional and national level.
• Communications: regarding funding‚ information &
advice, publications, networking opportunities, events and resources.

11. Existing evidence of impact
Digital Intervention:
• Reach for IT, Age UK (2010-2011)
Programme focus:
• Tackling the digital exclusion of older
people in residential care homes
through partnership working with five
community-based organisations
Evaluation measurement:
• Qualitative practice reporting
• Social Return on Investment
(intended)

12. Existing evidence of impact
Impact (organisational and individual):
• Engagement: 20 care homes, 34 volunteers delivering IT
training sessions to 234 residents.
• Communicating DI benefits: offering a new services
increases profile, reputation and attractiveness to potential
residents and their families.
• Meeting consumer demand: for personal development of IT
skills in IT-friendly environments.
• Quality of life: creation of ‘happier homes’ and better working
environments for staff.

13. Academic Evaluation
• Sus-IT: Loughborough University
– Participative research into older people in the
digital world
• University of London, Imperial
College, evaluation of NHS Choices
– Measurement of financial benefits to
Government of digital interventions

14. Practical concerns
• The limited extent to which impact of digital practice
is currently being evidenced.
• That evidencing impact is not considered a priority
(due to lack of political and financial support).
• That smaller organisations delivering digital inclusion
on the ground do not have the capacity or expertise
or money to carry out robust evaluation of impact.
• Funders pay lip service to project evaluation – no
support up front for projects
• That approaches and outcomes of evaluation do not
always ‘empower’ the most digitally excluded
individuals.

15. Current and New Approaches
1. Social Impact evaluation of Community
Capacity Builders
• Online Centres Foundation
2. Evaluation of ‘Get Connected’
• Digital Unite
3. Regular online reporting of ‘Get IT
together’
• Citizens Online

16. Valuing Evaluation
What Works Centres
• Launched: March 2013.
• Aim: to improve the evidence used in decision making
across a number of key policy areas.
• Approach: creation of specialist independent research
centres
• Partnership working and accountability: with the ‘Alliance
for Useful Evidence’, a network of over 1,000 members
who champion evidence, the opening up of government
data for interrogation and use, alongside the
sophistication in research methods and their
applications.

17. Suggested areas for development
1. Robust evidence regarding the social and economic
value of digital inclusion.
2. Methodologies that may be utilised by non-academic
organisations to capture and communicate impact.
3. Political support behind the importance of evaluation
as a means to develop effective practice.
4. Legitimising communication of ‘failure’ in approaches
as a learning tool: agile development and iteration
5. LONG TERM GOAL? Creation of a single digital
inclusion framework ..…

18. Thank you
Contact: alice.mathers@ukonlinecentres.com