Delhi Russian Call Girls In Connaught Place ➡️9999965857 India's Finest Model...
BIG Assist programme - OPM Evaluation
1. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Evaluation of the Assist Investment in
National Infrastructure Programme
Final Report
Report to NCVO and Big Lottery Fund
July 2015
2. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Client NCVO and Big Lottery Fund
Company OPM
Title Evaluation of the Assist Investment in
National Infrastructure Programme
Subtitle Final Report
Dates Last revised 23 Mar. 2016
Status Version: Final draft
Classification Restricted Internal
Project Code 8933
Author(s) Karen Naya, Oliver Ritchie, Sophie
Wilson
Quality Assurance by Dr. Heather Heathfield
Main point of contact Karen Naya
Telephone 020 7239 7813
Email Knaya@opm.co.uk
If you would like a large text version of this
document, please contact us.
OPM
252B Gray’s Inn Road 0845 055 3900
London www.opm.co.uk
WC1X 8XG info@opm.co.uk
3. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Contents
Summary ....................................................................................................................1
The evaluation ........................................................................................................1
Outputs and experience: infrastructure organisations ...........................................2
Outputs and experience: suppliers.........................................................................2
Peer to peer experience: face to face and online ..................................................3
Impacts on IOs........................................................................................................4
How well has BIG Assist met the outcomes for evaluation?..................................4
The future and recommendations ..........................................................................6
Introduction................................................................................................................7
The context for infrastructure .................................................................................7
What is BIG assist and how does the programme work? ......................................8
The evaluation of BIG Assist..................................................................................9
Findings....................................................................................................................12
How to navigate this section.................................................................................12
The Marketplace ......................................................................................................13
Customer outputs and experience .......................................................................13
Finding out about the programme ........................................................................13
Motivations for engaging with BIG Assist and key outputs..................................14
Use of the online platform to apply for vouchers..................................................17
Accessing a supplier.............................................................................................18
Receiving support.................................................................................................19
Making the most of vouchers ...............................................................................21
The customer experience: conclusions................................................................23
Suppliers: outputs, experience and impacts .......................................................25
Motivations for engaging with BIG Assist and uptake/outputs.............................25
Suppliers: Reflections on the programme design ................................................28
Impact for suppliers ..............................................................................................30
The Supplier experience, outputs and impacts: conclusions...............................30
Peer to peer offers: Outputs, experience and impact .........................................32
Motivations for engaging and uptake/ outputs .....................................................33
The peer to peer experience: conclusions ...........................................................36
4. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Impacts on infrastructure organisations ..............................................................38
Extra help and added impact................................................................................39
Greater efficiency .................................................................................................45
Other and longer term changes............................................................................47
Factors influencing the level of impact on infrastructure organisations: drivers and barriers 48
Conclusions: Impacts on infrastructure organisations .........................................52
Overall conclusions and recommendations ........................................................53
Summary ..............................................................................................................53
Conclusions: Has BIG Assist met the outcomes for evaluation?.........................53
Recommendations................................................................................................55
Appendices ..............................................................................................................57
Appendix 1: Pathways to Outcomes Model .........................................................57
Appendix 2: Evaluation activity and programme milestones ...............................58
Appendix 3: Sampling for customer and supplier interviews ...............................59
Appendix 4: Summary of output data used in this evaluation..............................61
Collation of output data used in this report...........................................................61
Appendix 5: Interview Topic Guides.....................................................................64
5. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 1 of 87
Summary
BIG Assist
BIG Assist is a £6million programme funded by the BIG Lottery Fund and delivered by the
National Council for Voluntary Organisations (NCVO). The programme has piloted new ways
of offering a targeted support programme to help voluntary sector infrastructure organisations
(IOs) be more efficient, effective and sustainable. Support is offered across the following
areas:
- Strategy, planning and managing change.
- Financial sustainability.
- Innovation, new products and ways of working.
- Marketing and building strategic relationships.
- Supporting and developing people and organisational change
Infrastructure organisations access relevant support via an online marketplace of
approved suppliers, and pay for the support using vouchers supplied by the BIG Assist
programme.
A key element of the programme is also to promote peer to peer learning and the
exchange of ideas between IOs.
The evaluation
OPM conducted an independent evaluation of the BIG Assist programme between
December 2012 and April 2015. The evaluation focuses on the impact of the BIG Assist
Programme and the extent to which the following four programme outcomes have been met:
1. Infrastructure organisations perceive and can evidence that they provide higher
quality support to customer and frontline VCSE organisations.
2. National VCS and private sector support providers develop better and more
sustainable models of providing support services to infrastructure organisations.
3. IOs value and feel they benefit from the opportunities for peer to peer learning and
support.
4. BIG Assist develops and shares learning about how demand-led models of
national support services could work in a local and national context and in a more
market oriented way.
The evaluation drew upon qualitative interviews with IOs, suppliers and the BIG Assist
team, as well as routinely collected programme output data which are largely quantitative.
6. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 2 of 87
Outputs and experience: infrastructure organisations
BIG Assist support to IOs has been extensive - the programme has issued 846 vouchers,
with a total value of £3,545,9501. Over 700 IOs have completed diagnostic review to access
voucher support and 576 IOs have been awarded vouchers2. The average voucher award
was £8,040.70.
The IOs that engaged in BA were motivated and wanted to make changes that would secure
their survival in a challenging world. Programme changes especially the increase in value of
the voucher awards encouraged more to engage.
Planning for a more sustainable future was their key focus. IOs wanted to develop new
and efficient ways of working. They looked to BIG assist to identify new sources of funding
and be more effective at generating income.
The Marketplace experience has worked well for IOs in terms of:
- The online platform to apply for vouchers – Many found the experience of
applying to BIG Assist a simple and positive experience. Review calls provided an
important opportunity for them to reflect on their needs as an organisation.
- Accessing a supplier - IOs were largely able to access a choice of supplier,
although it could sometimes be hard to differentiate between supplier offers. Some
IOs in rural areas and with specialist needs experienced issues with supplier
availability.
- Receiving support – IOs were very satisfied with the support they received. The vast
majority gave their supplier the highest rating available. They particularly appreciated
their expertise and understanding of IO issues and wider context.
- Making the most of vouchers – Larger vouchers also helped. IOs generally
reflected that larger vouchers had enabled them to conduct more comprehensive, far
reaching projects. Many benefitted from the opportunity to apply again for support
and vouchers. A few smaller customers pooled their vouchers, using flexibility within
the programme, to get more value from support.
- Advantages to a voucher system - It is less time consuming for customers to
administer and because vouchers were for specific, ring fenced work, funds for
support were secure and could not be diverted for other purposes.
Outputs and experience: suppliers
There are 223 approved suppliers in the Marketplace and they are a diverse group of
organisation type including companies (106), VCS organisations (69), sole traders (34) and
others (14).
1 Source of all output data: BIG Assist programme data. Reported 22nd May 2015
2 These figures include IOs resubmitting to the programme for further support.
7. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 3 of 87
There is variation in the volume of projects they have undertaken. One supplier has
completed 37 projects, to a value of £196,000. 124 Suppliers, 53% have completed at least 1
completed project.
Suppliers were motivated to join BIG assist for a variety of reasons. It was an important
new source of funding their work; it helped them reach new clients who needed their support;
and many hoped that being an approved supplier to Big Assist would raise their profile
across the sector. Some also felt that approved supplier status was a ‘badge of recognition’
for their experience and quality of support.
The self-assessment and review call meant that customers had a good knowledge of
their support needs before work started. Some suppliers however still had to work with
customers to identify what could be realistically delivered for the voucher value.
Suppliers feel the Marketplace is an efficient model for delivering support. They
understand that sustainability is an issue for BIG Assist. Without ongoing funding the
Marketplace would soon disappear, although they hope that the new contacts they’ve made
may endure.
Peer to peer experience: face to face and online
Peer to peer opportunities have developed along with the programme. Awareness and
involvement have increased over time, as the numbers involved in BIG Assist have grown
and feedback and information has spread.
ConnectSpace, offering sponsored visits (and mentoring, now suspended) has seen
significant uptake. 229 visitors have participated in supported visits.
IOs are positive about the visit experience. They have made new contacts, gained
confidence and practical knowledge about the areas they need to develop and in some
cases, have found new collaborators for projects.
ShareSpace, offering online discussion forms around key topics, has seen very
considerable volumes of traffic. Some live discussions have had over 10,000 views. IOs
access the forum and are interested in the topics raised. We have little evidence about how
active participation is and how IOs are using ShareSpace to build their own capacity.
The BIG Assist Library has built up over 600 resources for IOs. We have little evidence
about how active participation is and how IOs are using the library to build their own capacity.
The library resources are viewed on average 26 times per month, with the following pages
being the most popular:
1. Six challenges for infrastructure organisations
2. Changing role not just for volunteer centres
3. Five ways to stay afloat without more revenue
4. The perils of ignoring infrastructure
5. Small charities big impact
8. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 4 of 87
Impacts on IOs
Extra help and added impact. BIG Assist is a valued source of funded support in a time of
great need for IOs. Important change projects were unlikely to have happened without BIG
Assist funded resources and expertise from suppliers, or support would have been of lesser
quality and/ or delayed, limiting impact.
New sources of funding. IOs have won new sources of grant funding or contracts to deliver
work. This has sometimes been because BIG Assist has helped them identify new funding
sources to bid for and/ or because IOs can demonstrate they operate more effectively, which
has been important to funders.
Improved opportunities for consortia bidding. This is an emerging impact of the
programme. Using vouchers to set up consortia has allowed IOs to identify and work more
effectively with partners and they can now bid effectively as a group.
New ideas for generating income. This includes IOs developing new chargeable services
or products.
Increased ability or capacity to adapt to change. BIG Assist has been a catalyst for
necessary change in many organisations. BIG Assist projects have helped to secure staff
and Trustee engagement and buy-in for change. Having independent, expert support has
built staff confidence in the solutions put forward.
Clearer focus on impactful and/or sustainable activities. IOs feel it is likely that BIG
Assist support will make a long term difference to them. Some benefits won’t be felt for some
time however, and sustainability over the longer term is subject to many variables, including
the tough economic environment.
How well has BIG Assist met the outcomes for evaluation?
1. Many infrastructure organisations perceive that they can provide higher quality
support to customer and frontline VCSE organisations, or will be in able to in
time.
This programme has had extensive reach. Many IOs have gained new knowledge and
skills that are of direct relevance to supporting frontline organisations. By improving their
offer and range of support to the frontline, IOs hope to become more sustainable. In
addition, BIG Assist has helped IOs extend their networks and find collaborators for new
projects.
Some organisations made rapid changes to the way they work and provide services, or
saw immediate results from applying new skills. Others feel that the impact of some
changes and projects are likely to be manifested over the medium to longer term.
The wider context remains very challenging and may be limiting the capacity of
infrastructure organisations to maximise the full potential of their BIG Assist support.
9. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 5 of 87
However without BIG Assist support, projects are unlikely to have happened in many
cases, despite being of high importance to the organisation.
2. We are unable conclude that National VCS and private sector suppliers have
developed better and more sustainable models of providing support services to
infrastructure organisations. However, we have found much positive evidence
that:
This model is effective. Suppliers have delivered high volumes of quality support
through the programme, which matches their own areas of expertise and IOs’ needs.
The Marketplace has worked well for those who use BIG Assist as a means to deliver
support both to existing clients and networks, and new clients. Suppliers have
successfully extended their reach through the programme.
IOs are exercising choice from a selection of quality suppliers but there is room for
suppliers to improve the ways they respond to customers, market their expertise and raise
their profiles in the Marketplace.
3. IOs do value and feel they benefit from opportunities for peer to peer learning
and support.
Peer to peer offers have grown with the programme. There are some very high levels
of interest and participation. Sponsored visits are of great benefit to visitors and hosts,
with positive feedback and emerging case studies suggesting immediate and medium
term impacts.
IOs are choosing to view ShareSpace live forum discussions in very large numbers.
The potential for knowledge exchange is considerable given the interest, although
there is limited evidence of how IOs are using their participation to support change in
their own organisations.
IOs value opportunities to exchange knowledge with others in similar positions to
themselves, who are perhaps working in different ways. Time constraints on IO staff
appear to be a limiting factor to more active participation.
4. Big Assist develops and shares learning about how demand-led models of
national support services could work in a local and national context and in a
more market oriented way, although there are opportunities to do more.
BIG Assist has established a range of channels for developing and sharing learning about
this pilot. Events, both face to face and online, are reaching a large audience.
Since its launch, BIG Assist has used feedback from stakeholders, self-reflection and
learning from evaluation to develop and refine the programme offer.
There are opportunities to engage a wider group of stakeholders, connecting with
infrastructure organisations that do not actively participate in BIG Assist offers. There is
also potential to share learning with funders and others who influence VSC policy.
10. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 6 of 87
The future and recommendations
The BIG Assist 3 year contract has been extended for a further year in order to develop four
work areas:
1. A further £1 million investment in the infrastructure through the BIG Assist platform.
2. An extension of the evaluation.
3. A programme of outreach and consultation with funders and other stakeholders to
secure additional investment to support the longer term sustainability of the BIG
Assist platform.
4. A programme of engagement with leading infrastructure organisations to learn
from and share their vision for the future.
These work areas are strongly supported by the findings and conclusions of this
evaluation.
This evaluation makes the following recommendations for the
programme:
1. Continue to offer demand led support for IOs through a marketplace model,
retaining the features of self-assessment and range of topic areas for support.
2. Revisit opportunities to strengthen the provision of support for follow-up and
implementation of projects, through approved suppliers and peer infrastructure
organisations.
3. Explore options for making the programme sustainable over the long term by
opening up the BIG Assist platform to partnership with other funders.
4. Work to engage infrastructure organisations in BIG Assist who have not yet
participated in order to (i) better understand obstacles to access and (ii) broaden
impact across the sector and demonstrate the potential reach, or limitations, of this
demand-led model.
5. Maintain the existing high levels of programme flexibility and responsiveness to
feedback from infrastructure organisations in order to sustain change and to
ensure that the programme stays closely aligned to the needs of the sector.
6. Track impact within individual infrastructure organisations over an extended
timescale in order to (i) provide evidence of longer term programme impact and (ii)
better understand the impact of wider contextual barriers and levers for change.
11. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 7 of 87
Introduction
The context for infrastructure
Changes in the voluntary sector environment have meant that more infrastructure
organisations (IOs) than ever are seeking extra support and advice to help become more
efficient, effective and sustainable.
Austerity measures in the national budget have led to significant cuts across local
government.3 As a result, there has been a large reduction in the amount of money that
councils are able to make available to support infrastructure organisations through grant
funding. Competition for the remaining grants has become increasingly tough, and
infrastructure organisations are being forced to find new ways to fund themselves, either by
working with local government on a contract basis, by finding external funding sources, or by
raising money themselves.4
The combined effects of the economic downturn and reduced spending on welfare and public
services have also led to some front line charities experiencing increased demand for their
services, at a time when some charities are also struggling to maintain their funding streams.
In some cases, this has placed additional pressure on infrastructure organisations because
the charities that they work with are asking for increased support, or because their support
needs are changing.
As a result of these pressures, a number of infrastructure organisations have downsized or
been forced to close down entirely over the past few years.
As part of the BIG Assist programme, voluntary sector infrastructure organisations were
asked to mark themselves on a map in order to build a picture of the infrastructure sector.
898 infrastructure organisations were identified by August 2015. These organisations had an
income of around £811 million in 2013/14. This was a drop of around £113 million in cash
terms from 2010/11.5
3 Local Government Association. (April 2014). Under Pressure: How councils are planning for future cuts.
Available http://www.local.gov.uk/documents/10180/5854661/Under+pressure.pdf/0c864f60-8e34-442a-8ed7-
9037b9c59b46 (accessed 17/04/2015)
4 Justin Davis-Smith. (2013). The Future of Infrastructure. Speech at 3rd
July 2013 BIG Assist Conference.
Available http://blogs.ncvo.org.uk/2013/07/03/six-challenges-for-infrastructure-organisations/ (accessed
17/04/2015)
5 NCVO Almanac. Available http://data.ncvo.org.uk/data/voluntary-sector-infrastructure/
12. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 8 of 87
What is BIG assist and how does the programme work?
The £6 million programme BIG Assist, aims to provide support that meets sector needs in
these challenging times. Since 2012 NCVO has delivered the BIG Assist programme under
contract to the BIG Lottery Fund. BIG Assist is testing new ways of delivering support to
infrastructure organisations in the voluntary sector to adapt and change how they work to
meet the challenges of a much changed operating environment. The programme has been
piloting a ‘demand led’ model of support by awarding a voucher that the organisation uses to
select support of their choice through an on line market place of approved suppliers of
support.
The BIG Assist programme offers a wide range of support for infrastructure organisations in
addition to awarding vouchers, through a large peer to peer programme.
BIG Assist opportunities include:
Marketplace: where infrastructure organisations can browse and get in
touch with BIG Assist approved suppliers.
ShareSpace: an online discussion forum where infrastructure
organisations are encouraged to engage with each other by sharing thoughts,
ideas and other information.
ConnectSpace: opportunities to get involved in sponsored visits and BIG
Assist events (both face to face events around the country and online
participation).
The BIG Assist Library, giving access to handpicked resources that are
relevant to infrastructure. This includes opportunities to proactively edit and add
content.
The aim of BIG Assist is to help infrastructure organisations be more effective, sustainable
and better able to adapt to change’. As such BIG Assist offers organisational support in the
following topic areas:
Strategy, planning and managing change.
Financial Sustainability.
Innovation, new products and ways of working.
Marketing and building strategic relationships.
Supporting and developing people and organisational culture.
To apply for BIG Assist vouchers, IOs, or customers, organisations that access support
through BIG Assist go through a 3 step process: firstly, they answer a set of on-line pre-
qualifying questions to determine they are eligible for an award. Next, they must submit an
on-line self-assessment which is a more detailed set of questions about the organisation.
Finally, they have a review call with a BIG Assist customer consultant, who will have read
through the on line self- assessment and undertaken desk research about the organisation.
13. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 9 of 87
The review call is used to identify areas of need and assess the organisations ability to
implement change. Support priorities and clear outcomes for the support are agreed. Review
calls take approximately 1 hour.
Once the focus of support and voucher value of support is agreed between the customer and
BIG Assist, customers go to the on line Marketplace to review suppliers and select a supplier
of their choice. Customers make contact with suppliers by email or telephone and once a
project is agreed, BIG assist is notified and work can commence. On completion of the
project, the supplier submits their invoice for payment and customers are asked to rate and
comment on the support provided by their supplier. Reviews are available online for other
customers to see.
Suppliers apply online. The supplier applications are assessed by independent assessors to
determine if they are approved as a BIG Assist supplier to deliver support though the
programme. Approval includes assessment of relevant experience, prior work and
references.
Infrastructure organisations can be both customers and suppliers through BIG Assist.
Support from the BIG Assist team is available to both customers and suppliers at each stage,
over the telephone, on-line and through guidance documents and information.
The evaluation of BIG Assist
Aims and objectives
OPM has been asked to conduct an independent evaluation of the Assist programme. The
evaluation will focus on the impact of Assist: the extent to which the following outcomes have
been met:
1. Infrastructure organisations perceive and can evidence that they provide higher
quality support to customer and frontline VCSE organisations because through
Assist, they:
Gain knowledge and skills to implement change and new ways of working.
Over the medium to longer term, have implemented changes to the way
they work and provide service.
Feel increased confidence that they will be more sustainable in the future.
2. National VCS and private sector support providers develop better and more
sustainable models of providing support services to infrastructure organisations
because they:
Supply support which matches their own areas of quality practice and IOs
needs.
Learn from, and can respond to, IOs exercising choice from a selection of
quality suppliers.
14. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 10 of 87
Learn from and make changes to the support they provide in response to
IO feedback.
3. IOs value and feel they benefit from the opportunities for peer to peer learning and
support.
4. Assist develops and shares learning about how demand-led models of national
support services could work in a local and national context and in a more market
oriented way.
Evaluation activities and methods
The evaluation was conducted between December 2012 and April 2015. Appendix 2
shows how evaluation activities aligned with key milestones in the programme.
Evaluation employed a range of activities consisting of:
A ‘pathways to outcomes’ model: developed to understand the
programme and provide a guiding framework for the evaluation priorities and
questions.
In depth qualitative interviews with 50 customers which engaged with
the programme. These were grouped into 2 sets of informants. Each set was
interviewed initially after they have been involved in the programme for 6 months
to 1 year (first interview) and again when they exited the programme or support
has been delivered (follow up interview).
In depth qualitative interviews with 5 IOs who are eligible for the
programme but who have not engaged with it;
- In depth qualitative interviews with 47 approved suppliers. These were
grouped into in 2 sets of informants. The first group were interviewed at the start
of the programme and the second group as projects were being delivered,
towards the end of the programme
- Three group interviews with the BIG Assist team: developed to understand
the learning from voucher awards and peer to peer support opportunities
IOs were sampled to cover a range of support needs, regions, and focus for their own work.
Suppliers were sampled from all regions of England, and include sole traders, companies
and VCS suppliers. Details of the sampling framework and interviews conducted are
provided in Appendix 3
Interview data have been analysed thematically, i.e. all interview notes were reviewed and
categorized, to identify patterns and developing themes. This evaluation also draws upon
BIG Assist reports and output data:
- These data are sourced largely from the BIG Assist online administration
platform.
15. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 11 of 87
- Data are submitted directly by customers and other infrastructure
organisations, or are created automatically by the on-line system.
- BIG Assist supplied additional quantitative data, which it routinely collects, to
OPM.
- Programme data have been selected and interpreted by OPM and verified by
BIG Assist.
We employed descriptive statistics to understand these data, describing the main features of
the data and providing summaries where appropriate, i.e. frequencies and some basic
measures of central tendency. A table collating the quantitiive programme data used in this
report is provided in Appendix 4.
Considerations around the evaluation
It is useful to note the following considerations:
Whilst the evaluation has focused around the impacts of BIG Assist, this is
a largely formative evaluation. BIG Assist started from scratch and we should
recognise that the experiences of customers and suppliers have changed, as the
programme developed.
Not all offers were fully in place from the start of the programme (notably
peer to peer support) and important changes have been implemented along the
journey, for example phasing from the pilot Beta online platform to full online
functionality, and increasing the maximum voucher value. We acknowledge the
effect on interviewee experiences and when interpreting the programme data.
Impacts of the programme are still emerging and it is not possible to fully
understand the extent of some changes within the scope of the current evaluation.
BIG Assist is being delivered in a very challenging environment for the
VCS. IOs are experiencing high demand on staff time and in many cases there
has been significant staff turnover. This has influenced the experience of the
programme and its impact, but has also meant that it has not always been
possible to conduct evaluation interviews with the target informant, or someone
who can discuss the ‘big picture’ of the BIG Assist context and journey for an IO.
16. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 12 of 87
Findings
How to navigate this section
In this section we discuss the experiences and outputs of IOs and suppliers at various stages
in their journey with BIG Assist. We also present findings around the impacts of BIG assist on
IOs. This is a complex programme. To present findings and conclusions systematically, we
have structured them as follows:
We start with the Marketplace:
Customers: We first present the customer experience of the Marketplace
and voucher system, with key customer outputs and impact. This is prefaced with
an overview of these findings. We offer brief conclusions at the end of this
subsection.
Suppliers: We then present the supplier experience of the Marketplace
and key supplier outputs. This is prefaced with an overview of these findings. We
offer brief conclusions at the end of this subsection.
We move on to peer to peer offers: online and face to face:
Here we present the experience and outputs of the various peer to peer
offers. This is prefaced with an overview of these findings. We offer brief
conclusions at the end of this subsection.
We then present impacts for IOs:
Impacts on IOs, attributable to various offers of the BIG Assist
programme, are identified and discussed. This is prefaced with an overview of
these findings. We offer brief conclusions at the end of this subsection.
17. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 13 of 87
The Marketplace
Customer outputs and experience
Overview of outputs and experience
IOs were motivated to engage with BIG Assist to make changes that
would secure their survival in a challenging world. Financial sustainability was
their key focus. IOs wanted to develop new and efficient ways of working. They
looked to BIG assist to identify new sources of funding and be more effective at
generating income.
BIG Assist support has been extensive: the programme has issued 846
vouchers, with a total value of £3,545,9506. The maximum number of vouchers
awarded to an IO is 5. The average is 1.91 per IO, with an average total value
per IO of £8,040.70.
The process of applying for vouchers in itself added value for some IOs,
because it gave them an opportunity to discuss and clearly identify their needs
through the diagnostic review.
Customers were able to access a choice of supplier and the range of
suppliers was good, for most customers.
Customers were generally very satisfied with the support they received.
72% gave their supplier the highest rating possible. Suppliers are
knowledgeable and experienced. They also demonstrated commitment to
helping IOs and offered good value.
They report that larger vouchers allowed more comprehensive work to
take place. Many had received larger vouchers and had also resubmitted to the
programme, for further voucher support. This was often a way of embedding the
impact of the initial project.
Finding out about the programme
BIG Assist was promoted widely through a series of face to face engagement events. Nearly
1,400 people attended events held between October 2012 and September 2014. 15 events
took place in Year 1 of the programme, 11 in Year 2 and 3 in Year 3. The BIG assist National
Summer Conference in 2013 was attended by 162 delegates.
6 Source for all output data: BIG Assist. Data reports extracted 22nd
May 2015.
18. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 14 of 87
The majority of IOs that we spoke to found out about BIG Assist through email alerts,
particularly through the NCVO and NAVCA mailing lists. Additional ways in which IOs found
out about the programme include networking events or colleague contacts.
Motivations for engaging with BIG Assist and key outputs
IOs were seeking support to secure their survival
A number of organisations applied to the BIG Assist programme because they recognised a
fundamental need for support which would help them to survive in a challenging world.
In this context, a number of infrastructure organisations have hailed the BIG Assist
programme as a source of much needed support for a struggling sector.
“There were no other sources of support around. Even realising that things were
not right you put everything down to the external environment – you are a product
of your environment, but it is you that can change it, react to it, take yourself out
of it. That is hard to do when you are so engrossed in that. […] BA were able to
do that in partnership with us. We know we had to do something, but were not
sure what. […] It has really helped.” IO, round 1, first interview
7557 IOs completed on line diagnostic the second stage of applying for voucher support.
Key outputs showing response to demand
We will describe IO motivations for applying in more detail below, but here it is important to
understand the scale and type of support BIG Assist has provided, in response to IOs:
Overall, BIG Assist has issued 846 vouchers to support infrastructure, with a value of
£3,545,950.
576 IOs received voucher awards8.
The table below shows the allocation of BIG Assist vouchers for support, by topic area and
sub topic.
7 This figure includes re-submissions and applications since withdrawn.
8 This figure includes re-submissions
19. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 15 of 87
Topic area Sub-topic Number
issued
Total value
Financial sustainability Cost efficiency and
savings
1 £3,000
Financial management 8 £30,000
Income strategy and new
business models
153 £620,750
Managing your assets 6 £28,000
Innovation, new products and ways of
working
Developing new products,
services and ways of
working
130 £560,500
Innovating culture 2 £11,000
Marketing and strategic relationships Collaboration and
partnerships
27 £113,000
Marketing and
communications
131 £533,500
Strengthening and
creating effective
relationships
23 £91,000
Strategy, planning and managing
change
Business planning 103 £398,000
Leading change 17 £84,200
Organisational strategy 80 £330,500
Planning, assessing and
communicating impact
52 £217,500
Supporting and developing people
and organisational culture
Developing skills and
adapting to change
75 £363,000
New structures and ways
of working
34 £146,000
Organisational culture 4 £16,000
TOTAL 846 £3,545,950
We can see that the biggest single sub topic area for both number and value of voucher
awards was around income strategy and developing new business models, in the topic area
of Financial Sustainability, Over 150 vouchers were issued for this sub-topic with a total
value of £620,750. Marketing and strategic communications, and Developing new products,
20. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 16 of 87
services and ways of working were the next greatest sub-categories in terms of number of
vouchers issued.
Planning for a more sustainable future
Seeking support to adapt to the tough economic landscape was a major driver for IOs to
apply to BIG Assist for support. Many organisations saw assistance from the BIG Assist
programme as a means of planning for a more sustainable financial future. Several
organisations were reaching the end of a grant cycle and applied to use the voucher to
secure a future beyond this funding:
“Where are we going to get money from? We looked around…and out of the
blue popped up the Big Assist Offer which we grabbed with both hands”. Round
2, first interview with IO
Infrastructure organisations commented that support for voluntary and community
infrastructure organisations has become increasingly intermittent, and mentioned that other
CVSs have disappeared. As a result, many IOs recognised the need to ‘come up with a
different approach’, and used the programme to help them to implement this.
“It is a great resource, any money you can get as an infrastructure organisation,
it’s always more difficult to get money as a second-tier organisation – so I think
it’s important to know about”. Round 2, first interview with IO
Developing more efficient ways of working
Many IOs hoped that BIG Assist support would help them to streamline their ways of
working, in order to use available resources more efficiently
Some wanted to use the vouchers to increase the scope and reach of their organisation, so
that they could provide more support, or reach a greater number of frontline organisations:
“This will benefit the community and voluntary organisations because if we are bigger
they are better”. Round 2, first interview with IO
Many IOs also talked about being motivated to apply to BIG Assist to become more
streamlined and accessible in order to serve their present members in an increasingly
efficient and timely manner. In several cases, this involved developing more comprehensive
online materials, or modifying the services they deliver to ensure they meet the needs of their
user groups. IOs also reflected that increasing their own efficiency would enable them to
minimise costs for services to front line organisations, or even allow them to deliver some
support at no charge.
21. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 17 of 87
Use of the online platform to apply for vouchers
The application process could be a positive experience and provide
space to focus on the future
BIG Assist required the creation of an entirely new online platform. Many customers found
the experience of applying to BIG Assist a simple and a positive experience, This was either
because they viewed it as being less burdensome than other applications that they had taken
part in, or because it was actively helpful, providing support to structure their thinking about
what they need to do as an organisation.
For a number of IOs, the application process was an important opportunity to take time to
focus clearly on their needs and future. Finding this time and making it a priority was
sometimes difficult to do in a busy organisation, driven by immediate demands.
Review calls add value for infrastructure organisations
We predominantly gathered feedback on customer review calls in the baseline round 1 and
baseline round 2 interviews. Customers were particularly positive about their review calls,
which took place as part of the diagnostic review process. Often customers felt the interview
helped them to prioritise issues and understand the needs of their organisation better, by
providing a space for constructive reflection. Many commented that the interview process
added value in itself, and indeed one applicant identified the interview and subsequent report
as the most valuable part of the BIG Assist programme:
“One of the best interviews I’ve ever had; informative and constructive”. Round 2,
first interview with IO
“The guidance was brilliant, they were professional, they asked lots of questions,
really took time to understand our organisation”. Round 2, first interview with IO
Many interviewees added that the interviewer had been able to provide them with helpful
suggestions and advice, informed by a thorough understanding of the needs of the sector.
Some interviewees described how the interviews could be further improved. This included
having more information in advance about what information would be needed during the
interviews so that they could prepare, and there being more clarity about whether the
interview would affect whether they got vouchers, or whether it was purely diagnostic of their
needs.
It should be noted that a minority of interviewees, particularly those in early rounds of
interviews, felt that the online application process was time-consuming and placed a
significant demand on the IO to complete. This theme lessened as the initial online pilot site,
was phased out and replaced from January 2013 onwards. Based on user feedback the BIG
Assist team modified and improved the on line diagnostic including amending the wording of
questions and providing guidance information such as Question and Answer guides, to help
22. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 18 of 87
IOs through the process as smoothly as possible. This suggests that ongoing work to
improve the website and application process has been successful
To understand how the adoption of an online system might affect the impacts of BIG Assist,
and to identify potential areas for programme improvement, in the second round of interviews
with customers we asked participants about their self-assessed IT ability. Whilst this requires
the respondent to judge, it is perhaps helpful context about their general levels of capacity
and ‘comfort’ around using online systems. Responses to this question showed that there are
a number of individuals in the sector who find IT difficult in general. This may provide one
explanation for the issues reported by some organisations.
A consistent theme throughout the evaluation found that where people had difficulties using
the online platform, the BIG Assist team were able to support people effectively and remedy
difficulties quickly.
“I do have to say that my human contact and IT based contact are two very
different experiences”. Round 2, first interview with IO
The BIG Assist team are responsive and able to provide solutions
A strong theme was that interviewees provided positive feedback about the responsiveness
of the BIG Assist team. When issues had arisen (for a minority), phone or email contact had
minimised or negated the impact of any difficulties they had been experiencing. Such
comments were made in all rounds of interviews, especially the later rounds, possibly
indicating that contact from the team has been maintained throughout the customer’s BIG
Assist journey and did not tail off after the initial application process.
Accessing a supplier
Customers were able to exercise choice of supplier
Feedback on the availability of suppliers was generally positive. Most customers reported
that they were able to find a suitable supplier.
Approaches to identifying a supplier varied greatly between different infrastructure
organisations. While many customers browsed the information on the online marketplace to
identify a supplier, others looked for organisations they were familiar with, or even
encouraged their preferred supplier to sign up to the BIG Assist programme if they were not
already on the approved supplier list.
With over 220 suppliers available (across all topics for support, nationally) customers
frequently described how it could be difficult to differentiate between suppliers on the online
marketplace. It was felt this was because many suppliers had put up similar information
about themselves, or had ticked the same range of boxes as to what types of support that
23. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 19 of 87
they could deliver. Ratings systems and online feedback were valued as a source of
additional information to help make a decision, and this information built up over the course
of the programme. Some customers nevertheless reported that more advanced sorting and
search functions would have been helpful to refine their selection.
Many customers initially approached a range of suppliers (those we interviewed typically
contacted approximately 3-5) before making a decision about which to select. These
approaches varied considerably in formality from sending out a quick email or ringing up
possible providers for a discussion, to producing ‘invitation to tender’ documents asking
suppliers to produce bids for the work. Customers told us they did not always receive
responses from all of their contacts to suppliers, which some felt had limited their options.
Not all customers used the on line market place to find suppliers. A few applied to the BIG
Assist programme already knowing which supplier they planned to use, perhaps because
they had worked with them before. In some cases, they encouraged a supplier to register on
the programme so that they could use them. The BIG Assist Marketplace was therefore used
by some as a vehicle for working with established contacts. Additionally, many customers
who were involved in more than one round of support chose to use the same suppliers again
a second time, rather than go back to the marketplace and make a new selection. Some said
that this was because they knew that they would be satisfied with the support that they
received, while others said it was because it meant that the supplier would already know their
background context, making the support delivery more efficient.
A few customers expressed a preference for working with VCS suppliers, feeling that where
possible ‘money’ should be kept within the sector. The marketplace enabled these customers
to choose VCS suppliers for their support.
However a few organisations with specific needs reported some difficulties. Additionally,
some customers in more rural locations felt that many of the suppliers seemed to be based in
cities. One customer based in a very remote area said that after discussions with suppliers,
some had declined working with them due to the distances involved, which they felt had
limited the options available to them.
Despite this minority experience, BIG Assist have an understanding gained from interactions
with customers and suppliers that suppliers are travelling and extending their range to deliver
projects. Evaluation interviews with suppliers also suggest this happens in some cases.
Receiving support
Customers were very satisfied with the support they received
Customers had the opportunity to rate the support that they had received from 1 to 5, as part
of a feedback process in the online systems. Levels of satisfaction with the supplier were
very high overall. 72% (487 / 676) of customers gave a 5 rating – the maximum possible - for
the support that they received as part of the programme. 162/ 676 customers awarded a
rating of 4, indicating good satisfaction.
24. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 20 of 87
Interviewees and online reviews show that organisations giving high scores to suppliers often
highlighted the supplier’s excellent knowledge of the operating context for IOs and their
responsiveness to the specific issues they faced.
“Very experienced and knowledgeable consultant who quickly grasped our
operating context, culture and challenges. ” IO, online feedback
Several IOs reported that they are already working with the same supplier on another
project, or hope to do so if they can obtain more funding because of their high levels of
satisfaction with the support that they received.
“We couldn’t have afforded it [without BIG Assist funding] even though it was only
a small amount – we’re only a small organisation, and the work wouldn’t have got
done at all. […] This piece of work was invaluable.” Round 1, follow up interview
with IO
IOs as new clients for a consultancy model of support
In fact, some IOs have changed their views of using external consultancy support as a result
of their positive experiences of BIG Assist. A comment made by a few infrastructure
organisations was that prior to the BIG Assist programme, they had not seriously considered
the possibility of bringing consultancy support to help them. The programme had therefore
given them experience of a new way of working. Some of these organisations said that if
they could afford to do so they would consider engaging a consultant again in the future as a
result of their experiences.
One organisation told us about a bad experience that they had had with consultants in the
past, which had made them initially a little wary of the support model. However, the positive
support that they received through the BIG Assist programme made them more open to the
possibility of using a consultant again in the future.
Some respondents who awarded a lower score of 3/5 said that this was due to the need for
additional internal capacity of their own to take the support forward and achieve outcomes,
rather than because the support had not been high quality. For this small group, it is hard to
conclude if the supplier may have made unrealistic assumptions or demands around the
customer’s capacity, which might explain the lower score and would in effect, be associated
with to the quality of supplier support.
Only a tiny minority (5 ratings, less than 1%) awarded low scores of 2 or 1 to their supplier.
Where customers gave a low score, they commonly explained that this was due to how well
the supplier understood their local context, or how easy it was to communicate with the
supplier, for example how easy it was to organise dates for training.
“[They] had no knowledge of the local issues in [the local area]. Their advice was
of a standard nature and the research they intended to undertake was already
known to us.” IO, online feedback
25. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 21 of 87
Commitment and value from suppliers
During interviews, some organisations also noted that their supplier went ‘above and beyond’
the amount of work they would normally do for that amount of money. This dedication was
greatly appreciated by the IOs. However a few also noted that due to the small size of each
voucher, they might not have been able to fulfil their aims if suppliers hadn’t worked over and
above in this way.
“We got more from the consultant than expected. The extra value they added
was going above and beyond the call of duty; they did more than we paid them
for. […] If [the consultant] had only done what they’d been paid to do I don’t think
we’d have got the kind of results we got.” Round 1, follow up interview with IO
Making the most of vouchers
For high numbers of interviewees, the use of a voucher-based system ran smoothly and had
not presented any particular issues.
Larger vouchers allowed more comprehensive work to take place
In May 2014, towards the end of the programme, the maximum voucher size available to
infrastructure organisations increased. In the final rounds of interviews we spoke to a number
of organisations which had been awarded larger vouchers (up to approximately £9k each, in
our sample) It was felt that these larger vouchers had allowed more comprehensive work to
take place, meaning that the level of input from suppliers has enabled customers to take
forward new ideas into practice, rather than just delivering information about them. There is
anecdotal evidence that this has done a good deal to increase the impact of the support, as
some organisations had previously struggled to find the resources to make full use of the
strategic guidance that they had received from their support.
Use of multiple vouchers
Customers were able to receive more than one voucher, either through one application and
award, or by re-applying to the programme for additional support. A voucher is awarded for
each area of support. It is common for customers to need support in a number of different
areas.
Customers received on average 2 voucher awards. 162 organisations reapplied for voucher
awards. The average award over the whole programme was £6,258. One organisation
received 5 voucher awards to the value of £18,000, this was the highest award.
Where two vouchers were awarded at one time, the same supplier was generally, although
not always, used for both vouchers.
26. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 22 of 87
Resubmitting for additional support
Many organisations we spoke to in follow up interviews said they had applied to the
programme a second time and received additional support. Organisations had done this as a
way to further embed the impacts of the initial support. For example, organisations initially
applied for support developing a new strategy, and then later applied for a second voucher to
help implement this. Others applied for additional vouchers in order to meet support needs
that had been identified through the initial support.
Pooling vouchers with other organisations
The opportunity to pool vouchers with other organisations was a flexibility offered by the
programme. Pooling arrangements were intended to allow IOs to get added value from
combined support. Some customers were supported by the BIG Assist team to do this.
Others sorted this out themselves.
“Working with another organisation worked well- a lot of shared learning can
happen across that. A £2,500 grant doesn’t pay for much consultant time- 5 or 6
days. By working together and combining projects, it feels like we got better
value.” Round 1, second interview with IO
The programme has details of the situations where the team have supported the pooling of
vouchers but it has not been able to record all situations where organisations have
themselves shared vouchers. So we cannot quantify the uptake of this opportunity.
Interviewees described several reasons for pooling vouchers. Some did this because they
were trying to find ways to work more efficiently with one another or wished to set up a
consortia. Others did it because they realised that they had similar needs, and wanted to be
able to afford a bit more support. In such situations, one piece of feedback that we received
during round 1 interviews was that while BIG Assist processes did not seem to have been
explicitly designed to accommodate voucher pooling, the BIG Assist team were flexible about
customers’ pooling arrangements and were helpful in making them a possibility.
Vouchers can benefit IOs not directly involved in BIG Assist
Some organisations spent their voucher in a way that was beneficial to another organisation,
even if that other organisation had not also applied to the BIG Assist programme. In one
such example, two CVSs were considering merging, and one of them used support from the
BIG Assist programme to help consult stakeholder views on this.
Advantages to using a voucher system
Some IOs, especially smaller ones or those with less back office support, welcomed the fact
that they did not have to process payments to suppliers themselves, as this can use up
27. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 23 of 87
valuable admin time. It also made budgeting easier for some because the money did not
have to pass through their systems at all.
Some interviewees thought that receiving the support as a voucher for a specific element of
their work meant it had a greater impact than if they had simply been given the money. One
reason for this is that the programme helped them to identify what their needs were, and then
provided them with funding that was ring fenced to address these. Some interviewees said
that if they had been given cash, it might have been harder to justify using the money to
support longer term investment in this way, rather than simply using it to help balance to
books in the short or medium term.
We also received feedback that the use of vouchers could lead to a better working
relationship between customers and suppliers because payment was assured through the
BIG Assist process. Just one customer who we spoke to in the first round of interviews had a
different experience: a delay in drawing down the voucher meant that their supplier was paid
late, and they were worried that this might have affected their relationship.
Challenges around the use of vouchers: timescales
A minority of infrastructure organisations suggested that deadlines for customers to choose
and engage suppliers were not long enough. This observation is supported by the fact that in
the round one follow up interviews, over half of interviewees reported having asked for an
extension to finish spending their voucher.
Suppliers also commented on the tight timescales, with vouchers expiring after 4 months,
explaining that the time left to complete the work after being approached and contracted was
sometimes insufficient to do a really good job.
However, one interviewee from an infrastructure organisation said that they had found the
deadlines extremely helpful in keeping her on task. She explained that without them, it would
have been harder to prioritise organising the support, and it would have taken her much
longer to engage a provider.
The BIG Assist team have reflected on this learning. They state their experience is that the
longer timescales given at the early stages of the programme led to a lack of momentum
around selecting suppliers and starting the work. These delays in starting were perceived by
the BIG assist team as a limiting factor on impact and the reduced timescales introduced
during the programme encouraged customers to get on with the work and take action.
The customer experience: conclusions
BIG Assist has delivered important support to help IOs survive in a very
challenging landscape.
Outputs are very significant: there are great numbers of IO applying for,
and receiving support. Whilst we are unable to make definitive conclusions about
28. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 24 of 87
the reach of BIG Assist, due to uncertainties of mapping the VCS infrastructure
sector.
Feedback and programme data around the allocation of vouchers
suggests that BIG Assist projects are closely aligned to the needs of IOs.
Most IOs are highly satisfied with the support they have received.
Applying to BIG Assist has given IOs an opportunity to consider their future
and identify needs in a focused way, aided by review calls with the BIG Assist
team
They have exercised their choice of supplier.
Customers can manage the online process and come back to the
programme for more support.
The BIG Assist team have developed much insight into the needs of IOs.
We note that a process using cash based transactions could also identify
needs, determine the focus of support and set time limits for undertaking the
project in the same way. Apart from any efficiencies around reduced IO
administration through a voucher as opposed to managing a cash award, use of a
voucher process for support may be a ‘psychological’ benefit over a cash based
process.
29. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 25 of 87
Suppliers: outputs, experience and impacts
Overview of outputs and experience
Suppliers were quick to engage with BIG Assist. There are 223 approved
suppliers in the Marketplace.
Suppliers are a diverse group, the majority are companies (106), followed by
VCS organisations (69), Sole traders (34) and others (14).
There is variation in the volume of projects they have undertaken. One supplier
has completed 37 projects, to a value of £196,000. The 124 Suppliers with at least
1 completed project have delivered an average of 5.7 projects each.
Suppliers were motivated to join BIG assist because it was a new source of
funding their work with infrastructure. They expected BIG Assist to be the vehicle
for delivering the bulk of infrastructure support and felt they needed to be on board.
They hoped that being an approved supplier would raise their profile across the
sector and introduce them to new clients. They wanted BIG Assist to help them to
reach clients who needed their support.
Suppliers think the voucher system is an efficient way of working although
would sometimes like more time to complete projects, or to deliver an extended
period of support to help embed change.
Finding out about the programme
Suppliers heard about BIG assist through their own networks of contacts, including news
updates from key VCS organisations and through promotional material produced by BIG
Assist. In some cases, infrastructure organisations told us that they had asked suppliers to
sign up to the programme so that they could get support from them.
Motivations for engaging with BIG Assist and uptake/outputs
A new source of funding their work with infrastructure
Many suppliers saw the BIG Assist programme as the means by which they could access
funding for infrastructure support work within the wider context of austerity. They were
motivated by being part of a programme that enables them to exchange support and share
expertise. Key outputs regarding suppliers are shown below:
30. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 26 of 87
The Marketplace attracted a large number of suppliers
223 approved suppliers are currently in the Marketplace. Suppliers were quick to engage
with the programme. Around 200 applied to be approved suppliers in the first application
round when the programme opened.
Approved suppliers are a diverse group. A breakdown from programme data shows:
Companies: 106.
Other: 14.
Sole traders: 34.
Voluntary or community organisations 69.
The amount of support delivered by various suppliers differs greatly.
There is variation in the volume of projects completed by suppliers. An average of 3.15
projects have been undertaken per supplier (if all approved suppliers are included). However,
the 124 suppliers with completed projects9 have delivered an average of 5.7 projects each.
Some suppliers have undertaken a great deal of projects - the greatest number of completed
projects by any supplier was 37, with a total voucher value of £196,000.
This diversity makes variation in suppliers’ experiences of the programme inevitable.
However, themes did emerge from the interviews conducted throughout the programme.
These are reported below:
Big expectations of BIG Assist
When we spoke to suppliers in December 2012 – January 2013, some thought that most, or
all, infrastructure support might end up being delivered through the BIG Assist Programme. A
number therefore saw it as essential to register on the programme in order to ‘stay in the
game’. A very large number signed up at the first opportunity when the programme opened,
with over 200 applications received.
This view of BIG Assist as the main source of work for delivering infrastructure support was
not discussed to the same extent in the second round of interviews, conducted after
suppliers had started to deliver support from June 2014.
Uncertainty about the volume of new work
Although suppliers appreciated the scale of BIG Assist early on, they were not always sure
how much work they would end up delivering through it. Expectations were particularly
unclear to suppliers in early interviews. In the round two interviews we found that
9 The BIG assist online system shows projects as completed when supplier invoicing and other requirements
have been met, so this would not include suppliers with their first projects pending or underway
31. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 27 of 87
experiences had been quite varied, with some organisations managing to deliver a good deal
more than anticipated and others a little less.
Opportunity to work with new contacts and/ or work in new ways
Suppliers saw BIG Assist as an opportunity to reach out to new clients. Suppliers hoped the
programme would expand their networks of contacts and clients and we heard from many
that they had achieved this aim.
In some cases, BIG Assist projects brought suppliers into contact with clients from slightly
different backgrounds from their usual client base. This was more of a theme for suppliers
who were companies, who despite demonstrating relevant experience, were in some cases
used to working predominantly with slightly larger organisations than BIG Assist customers.
Raising their profile
Across both sets of interviews, suppliers were attracted by the potential for the programme to
raise their profile. Several suppliers said they felt ‘proud’ to be on the scheme and felt they
would benefit from being associated with the BIG Lottery and the NCVO brands.
Suppliers felt that involvement in the programme would ensure their visibility to IOs in the
sector. For example, one stated reason for getting involved was:
‘So people who need me, can find me’ (Round 1 supplier interviews).
Some suppliers also thought that becoming an approved supplier would act as a mark or
standard of their quality and so would help them be accredited for the work they’d done
previously. Going forward, they hoped that Approved Supplier status would help them to win
work outside of BIG Assist.
Feedback from some suppliers was that they were not always clear on how they could
market themselves most effectively through the online marketplace, especially if they did not
come up on the first few search pages, or had become involved in the programme later.
Suppliers joining the programme later told us they felt marketing themselves was more
difficult because other suppliers already had a good deal of positive reviews from customers,
which made it harder for them to get their first piece of work through the programme.
Others said that they had not tried very hard to market themselves, either because they were
getting as much work through the programme as they wanted, or because they did not have
the time to do so due to other commitments.
The BIG Assist team have encouraged suppliers to improve their marketing and presence on
the online platform, and have issued information to support them to do this more effectively.
32. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 28 of 87
Suppliers: Reflections on the programme design
Some suppliers also changed their practice by expanding their geographical working area, or
stated that they were available to work across a wider area. We do not have data on how
many of those who wished to do so may not have been selected because customers prefer a
local supplier.
Knowledge of customer’s support needs
A positive piece of feedback from suppliers was that because infrastructure organisations
went through a rigorous application process, customers often had a clear idea of what
support they wanted, making it easier to deliver what was needed. However not all suppliers
had this experience: some said that they often had to do some work initially to help
customers work out what could reasonably be achieved with the budget.
Timescales for delivery
Like customers, some suppliers reported challenges around programme timescales.
Suppliers articulated their issues as sometimes finding it quite demanding to finish work by
the deadline set for completion. This was particularly an issue where customers had taken a
long time deciding which supplier to use. However, in such situations solutions were often
found. In some cases suppliers report that it was possible to get an extension from the Big
Assist team, in others they report treating the deadlines slightly informally, for example by
conducting some unpaid follow up work after being paid on the official deadline.
Timing of payments
A number of smaller suppliers (especially sole traders) said that the voucher payment system
could cause them cash flow difficulties because the money is only released after the project
is complete.
Follow-up support to maximise impact
Like some customers, suppliers reported the need for follow up support after the work had
been delivered, in order to ensure maximum impact. One individual reported that in order to
fund an extended period of support, they charged slightly higher rates for the period of
‘official’ support, on the assumption that they would then provide additional input for no
further charge over subsequent months.
BIG Assist is an efficient way to deliver support
A few suppliers also commented on the efficiency of the BIG Assist model. One commented
that a marketplace is an effective way for them to spend money because it allows them to
33. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 29 of 87
only pay for the exact support that is delivered: if they were to use in house employees to
deliver the support there might be additional overhead costs involved.
“From Assist’s point of view it’s a low risk model, they’re not employing me, if
people don’t choose to work with me there’s no consequence re cost, there’s no
management with regard to me on BIG Assists part, they only pay me for
something I deliver. […] It’s a new way of working, a leaner way […] so this
means the money stretches further and the frontline services can continue.”
Supplier interview, 2013
In addition, some suppliers conceptualised the support that they were providing as being part
of a cascade model: they would help to upskill infrastructure organisations, which in turn
could pass that learning on to the organisations that they support. One supplier described
this as ‘developing 10 birds with one stone’ (supplier, round 1 interview.)
Potential of an open marketplace
A minority of third sector organisations (VCS suppliers and BIG Assist customers) felt
strongly that where possible, money for infrastructure support should be kept ‘within the
sector’. Some suppliers with this view said that they hoped that customers would choose
them over commercial suppliers for this reason.
Some suppliers (and some customers) noted that certain VCS organisations could be quite
‘territorial’, only asking for or delivering services to organisations on their patch. A number of
suppliers said that they thought the online marketplace might help to encourage IOs to look
at a wider range of possible options to meet their support needs. There is evidence from BIG
Assist team learning that while geography remained an important consideration for many
customers, some did report engaging in suppliers who they wouldn’t have considered
otherwise as a result of the programme.
Sustainability of BIG Assist
Suppliers (and customers) reflected that the BIG Assist programme had created a somewhat
artificial market for support services because while infrastructure organisations have a good
deal of need for support, they do not typically have the resource to buy it in themselves. It
was therefore observed that the while a useful market had been created, if the financial
injection provided by the BIG Assist programme was to cease, then the market would quickly
collapse. This observation was not typically meant as a criticism: it was simply an
observation that positive influences of the programme on the market may not last long
beyond the end of the programme
34. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 30 of 87
Impact for suppliers
New markets and new work for suppliers
Taking part in the Assist programme has had a number of positive impacts on suppliers,
including helping some to access new markets, and providing them with a source of income.
We spoke to suppliers at two different stages of the programme. In the first round of
interviews, they had provided little or no support as yet, while the second cohort of suppliers
that we spoke to had all delivered some.
As discussed previously, suppliers’ experiences of the programme were varied, with some
delivering much larger amounts of support than others. There was consequently a degree of
variation in the types of impact that suppliers experienced.
When we spoke to suppliers in the round 1 interview, a number said that they were hoping to
use the programme to become involved in new markets, either in terms of the type of work
that they were doing, the type of organisation they were supporting or the geographical
region in which they were working. During the second round of interviews, many suppliers
said that they had indeed managed to access new markets in this way.
“For me it has expanded my geography. When I started working on BA most of
my clients were in one area, and I have now had them all over the country.”
Round two, supplier interview
For some suppliers, the support in accessing new markets, or increasing their profile in
existing ones was considered a greater impact than any profits that they made as a result of
the work. Indeed, some suppliers told us that they delivered work through BIG Assist for very
little or no profit, as a way to expand their client base. There is some supporting evidence for
this from customers, who said that their supplier had delivered work ‘over and above’ the
value of the voucher.
The Supplier experience, outputs and impacts: conclusions
BIG Assist has successfully created a Marketplace of suppliers.
Suppliers recognise the scale and importance of BIG Assist to the sector.
They are using the opportunity to reach out to old and new clients,
although there is room for some to improve their visibility in the Marketplace.
Some suppliers have found BIG Assist to be a strong source of work,
delivering multiple projects through voucher support.
Suppliers identify the challenges IOs face around implementing support
over the longer term. They see a need to offer longer periods of support to IOs, or
additional capacity to maximise the impact of BIG Assist projects.
35. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 31 of 87
There is limited evidence that suppliers have changed their ways of
working as a result of BIG Assist. We cannot assess the extent to which suppliers’
expressed willingness to work in different ways (for example in new geographical
areas or with new types of client) has translated into practice. There is some
evidence that a number of suppliers have varied their geographical range of
operation.
Despite the huge volumes of support work carried out through BIG Assist,
we cannot know the extent to which BIG Assist has become the main vehicle
through which IO support is delivered, as some suppliers anticipated. Projects
provided additional work that suppliers would not have otherwise undertaken.
36. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 32 of 87
Peer to peer offers: Outputs, experience and
impact
A key element of the programme is around facilitating peer to peer learning and exchange
between IOs.
This section discusses the peer to peer support provided by BIG Assist, beyond the
Marketplace and voucher system.
It is important to note that much of this support was offered and promoted to infrastructure
organisations that had not received a voucher to engage a supplier, as well as those who
had received vouchers.
The evidence reported here draws upon programme data (output data around on-line and
other peer to peer activity) and a secondary analysis of the reports submitted by those
participating in Sponsored visits. We also report themes emerging from interviews with
voucher recipients.
Overview of findings: outputs, experience and impact
Peer to peer opportunities have developed along with the programme.
Awareness and involvement have increased over time, as the numbers
engaged in BIG Assist has grown and feedback and information has spread.
ConnectSpace, offering face to face and online events, sponsored visits
(and mentoring, now suspended) has seen significant uptake.
229 visits have taken place, by 129 organisations.
IOs are positive about their visit experience.
Impacts are emerging. IOs have made new contacts, gained confidence
and practical knowledge about the areas they need to develop. Some have
found new collaborators for projects. IOs feel positive impacts will emerge from
visits.
ShareSpace, offering online discussion forms around key topics, has
attracted considerable volumes of ‘traffic’. Some live discussions have had over
10,000 views.
There have been 29 events over the 3 years of the programme, the
largest with 162 delegates.
37. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 33 of 87
Motivations for engaging and uptake/ outputs
ConnectSpace: Face to face and online events, sponsored visits and
mentoring
The programme team has promoted opportunities for sponsored visits and, until spring 2014,
mentoring opportunities.
In all, programme metrics show that 229 visitors participated.
There has been significant uptake of sponsored visits as the programme has run, probably
due to word of mouth reporting through networks, feedback about visits online and efforts
from the BIG Assist team to promote sponsored visits. The BIG Assist team has
communicated about visits actively via a bi-monthly newsletter, social media and through
case studies demonstrating the benefits of visits, which may have also increased interest.
Sponsored visits have been welcomed by IOs and feedback is highly positive. They report
the main benefits of visits are:
Time out to focus
IOs valued ‘time out’ to exchange knowledge in a focused way. Some felt that time for fact-
finding and networking visits with a clear purpose can be easier to justify, in a context where
resource constraints can make it difficult to take time away from day to day delivery.
Value of connecting with others in a similar position
Many interviewees reported finding it particularly useful to speak to organisations which
worked in a similar context to them (e.g. do they have a membership model in a local area,
or provide a particular service nationally). This was more important for many interviewees
than visiting a geographically close organisation, or one in a similar stage of development.
For example, visitors were keen to have the chance to speak to others who had already
initiated a similar charging process for their services. They reported finding it incredibly
helpful in informing the design of their own processes. However it was commented on by
many that meeting organisations who were approaching things a bit differently to them was
one of the more useful aspects of the visits.
Value of seeing new ways of working
Visitors state that they had benefited by seeing new ways of working in practice. They
find it helpful to use alternative approaches being adopted by organisations in a similar
position to themselves.
“We came away with a very clear picture about elements we need to work on in
order to move to charging for development support, including determining what
38. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 34 of 87
services to charge for, pricing, publicity and managing relationships with donors
and service users. We will share our learning with the rest of our staff team and
trustees. We will produce a menu of services and publicity about the new
approach. We will examine our systems (admin / finance) and update where
necessary to support this new approach.” IO, online feedback about a sponsored
visit to an organisation, focusing on developing a model for charging for support.
Value of developing networks
In addition, IOs report how visits have developed their networks of contacts. They have
identified potential for future collaborations on projects and some collaboration is emerging at
this stage.
“This has supported the group to establish a more coherent regional offer, share
practice and investigate how we can share resources and a more national offer in
the future. We have identified a number of areas for joint delivery [with the host
organisation] and the format for a business plan. Additionally we will be
facilitating more specific sessions to follow up this work.” IO, online feedback
Potential for more IOs to become engaged
Many in the round two follow up interviews said that lack of time had held them back from
making greater use of the ConnectSpace offer. Some planned to make greater use of it in
the future.
“One of the things that we haven’t tapped into is the site visits and the stuff on the
website. You can only use so much stuff at once. I think it is just time.” Round 2,
second interview with IO
The positive experience of visiting encouraged one host to engage further in BIG Assist, by
applying both for a voucher for support themselves, and as a potential supplier of support to
others.
ShareSpace: Online discussion forums
Programme data show the traffic on ShareSpace, the BIG Assist platform for online
discussion forums.
ShareSpace: number of topics and posts per forum
Forum Topics Posts
Innovation
One live discussion achieved over 11k views.
24 550
39. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 35 of 87
People and culture 20 41
Financial sustainability 16 137
Marketing and relationships
3 live discussions with over 7k to over 10k views each
22 504
Strategy and change
One live discussion with nearly 5k views
Live Q&A with nearly 8k views
26 237
General 47 96
We can see that certain forums, notably Innovation and Marketing and relationships have
over 500 posts, so have found an audience and there are active contributions being made.
However the greatest levels of engagement came from live on line discussions. Many of
these discussions have a very high number of views (with 2 over 10 thousand). Live
discussions with the greatest number of views reflect IO interest in developing innovative
ways of working and reaching new audiences/ markets, or working in more effective ways
with the market they have.
This is an extensive level of interest in live discussions. Audiences are clearly able to access
the content and topics selected for discussion are resonating with those involved, in their
thousands, in many cases.
However some caution must be taken interpreting the data, as we cannot know how many
views are by single individuals, or by people viewing in groups. Some individuals may be
logging in and out repeatedly to discussions, rather than joining in continuously.
Far fewer organisations actively post or contribute and it is therefore difficult to infer what
impact participation has at this stage. It may be worth exploring the impact of forums more, if
the programme is to continue offering this opportunity.
Of course, in the early rounds of evaluation interviews, the peer to peer offer and supporting
platforms were still in development. We explored interviewee knowledge and use of
ShareSpace in more detail in the round two follow up interviews.
While awareness of the online forums was high at that point, many interviewees said that
they had not yet made much use of them. This finding is difficult to reconcile with the high
viewing rates for online discussions. Our interviewees gave reasons for why they hadn’t yet
made use of the opportunity. They described how they were too busy or had not yet had a
chance to engage with Sharespace [the online forum]. Many interviewees in the round two
follow up interviews commented that receiving and implementing support from the BIG Assist
programme had increased their workloads, and that once the support was over and they had
a bit more free time they would be more likely to engage online. However, a couple of
40. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 36 of 87
individuals were unsure how long they would still have access to these resources, either
because they were no longer actively receiving support from the programme, or because of
uncertainty over how long the programme would continue to exist.
BIG Assist Library: resources including articles and wikis
The BIG Assist Library contains over 600 articles and wikis, including case studies to share
experience and learning10 .This extensive Library and its resources were mentioned rarely in
interviews. When prompted, there was limited awareness of the offer amongst our
interviewees.
The BIG Assist team have worked to drive up ‘traffic’ and engagement in the Library by
promoting it widely and ensuring content is updated and responds to current issues for
infrastructure.
From online analytics, the library resources are viewed on average 26 times per month, with
the following pages being the most popular:
1. Six challenges for infrastructure organisations
2. Changing role not just for volunteer centres
3. Five ways to stay afloat without more revenue
4. The perils of ignoring infrastructure
5. Small charities big impact
Events for shared learning
BIG Assist has delivered a range of events as an opportunity for stakeholders to find out
more about BIG Assist and share learning about how it can support them. Events were held
between October 2012 and March 2015. 15 events took place in year 1, 11 in year 2 and 3 in
year 3. The BIG Assist National Summer Conference in 2013 had the greatest attendance
with 162 delegates.
IOs told us that they value events as an opportunity to network face to face. People also
came to hear about innovative practice and practical ideas for taking them forward. A few
reported it was easier, or felt more justifiable, for them to take time out for learning at a
designated event than for other forums for knowledge exchange.
The peer to peer experience: conclusions
Peer to peer offers are a way for very large numbers of IOs to access
support that is designed with their needs in mind.
IOs are eager to exchange knowledge with their peers, particularly those in
similar circumstances who are implementing new ways of working. Interest and
10 Data extracted from online platform on 22nd May 2015
41. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 37 of 87
engagement in some offers, e.g. events, have been apparent from the start of the
programme.
Sponsored visits are very well received and are showing promising
impacts for IOs. Increased knowledge, confidence and partnerships are leading to
new opportunities for income generation and/ or improving services offered to the
frontline. There is evidence from post-visit reviews and interviews that we can
expect other changes to emerge as a result of active participation in visits.
ShareSpace, offering live discussions, is attracting huge volumes of traffic
for some outputs. This indicates that a great many IOs are able to access
materials and find the topics of interest. This level of views suggests enormous
potential for shared learning.
However, we cannot currently determine how active ShareSpace
participation is and how IOs are using ShareSpace to build their own capacity.
Despite huge viewing figures, conversations with voucher recipients (who
made limited use of on-line peer to peer resources) suggest that there is
further room for increased uptake of the on-line peer to peer offer.
At this point, it is difficult to conclude what the impacts of Share Space will
be.
At this point, there is little evidence for us to draw conclusions about the
BIG Assist Library, in terms of the use experience or possible impacts.
It can be difficult for IOs to make time for reflection and learning in such
times of constraint and demand, despite an awareness of the need to do so.
Funding time for knowledge exchange appears to be an enabling factor but having
a clear focus for how that time will be used, for example through a dedicated
event, with practical outcomes, may give IOs the ‘permission’ they need to step
back and take a wider view.
42. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 38 of 87
Impacts on infrastructure organisations
Summary of findings around impact
BIG Assist has been a catalyst for necessary change in many
organisations. Change projects and the resulting impacts were unlikely to have
happened without BIG Assist funded resources and expertise from suppliers, or
support would have been of lesser quality, and/ or would have taken IOs longer
to get around to.
Impacts are emerging in critical areas for IOs.
The support received through BIG Assist has helped a number of IOs to
win new sources of grant funding or contracts to deliver work.
IOs are identifying new ideas for generating income. This includes the
development of chargeable services and/ or products.
Improved consortia bidding is an emerging impact of the programme.
Using vouchers to set up consortia has allowed IOs to identify and work more
effectively with partners and they can now bid effectively as a group.
IOs report that the BIG Assist project has helped to secure staff and
Trustee engagement and buy-in for change, due to the external nature of the
support.
Some IOs feel that the main impacts of the programme are of a longer
term nature and are likely to emerge in the future.
Enablers of change include:
Voucher value (although impact is seen with smaller voucher awards)
Relevant, tailored support
Role of external, independent support
Barriers to change include:
Organisational capacity and timing
Wider context
In this section, we describe the key impacts of BIG Assist on IOs.
We also use examples from specific organisations to illustrate these impacts.
43. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 39 of 87
Extra help and added impact
A source of funded support in a time of great need
Many organisations said that they would not have been able to get support via other means,
so the impacts seen through the programme would have not taken place. They lacked
capacity internally and could not have found resources to bring in a consultant. Some said
that they would have tried to do similar work of their own accord, but this would not have
been done as quickly, efficiently, or to the same quality.
Very few organisations said that they would have managed to get the same quality of
support, or the same outcomes that they described in the absence of the BIG Assist
programme.
New sources of funding
Improved fundraising abilities
Organisations described how BIG Assist projects helped them to demonstrate the quality of
their work, and hence strengthen their ability to win bids.
Projects which improved core internal processes, such as financial management and staff
development, also helped customers to secure funding,
More awareness of funding options and new sources of funding
Case study: new sources of funding
One organisation has accessed multiple sources of new funding following support from the
BIG Assist programme. Their first vouchers award focused on income generation and
innovative ways of working encouraged them to apply for workplace development funding
from the Department for Business, Innovation and Skills. This has provided the IO with
steady funding for the next three years.
“The biggest impact has been the new funding stream. Without this I think we
would be having a very different conversation if we didn’t have this contract.
We would be a different second tier organisation by this point.”
This contract has had a significant impact on the organisation as a whole, allowing them to
develop expertise in education and training:
Growth in the number of staff members from 1 to 17.
New organisational structure including an operations manager.
Two new dedicated spaces in London and the North.
Accredited existing training.
44. OPM Evaluation of the Assist Investment in National Infrastructure Programme
Page 40 of 87
Developed 2 new training qualifications.
The original vouchers also encouraged the IO to look at a donation model for increasing
funding; they have since raised £60,000 in donations. The IO has since reapplied and
received more vouchers from the BIG Assist looking at sustainability in response to the IO
having to frontload costs to deliver the programme. These new vouchers have supported
the IO to manage their growth ‘mindfully’, without over extending beyond their means: “we
are not expanding or creating an operational cost we can’t sustain”.
The IO felt the BIG Assist programme had met their needs “100%”.
“It’s enabled us to develop and grow, and I think in this current climate that is
quite unusual.”
Improved consortia bidding
Some organisations also used the vouchers to support the organisation to set up consortia or
other, more informal, arrangements to help multiple smaller providers to work together to
deliver work. This has allowed the voluntary and community sector to compete more
effectively with private organisations and larger national providers. Individuals who we
interviewed were often positive that the new arrangements would help the organisations
involved to win funding. One organisation described how they had used the support to help
them to improve their processes of working with other organisations, which allowed them to
bid more effectively as a consortium. They said that a direct impact from this was that they
had been able to win much larger grants and contacts from local funders:
“It helped to secure £1m from the local authority over two years, and an
additional £42k a year from the CCG. Helped join up our work and bring us more
closely together. The council have always told us there would be one contract-
the BIG Assist helped us to get there. It has given us the database to do the work
together and do the monitoring and provide the evidence that funders need.”
Round 2 second interview with IO
Case study: Training for front line organisations
One London based IO used BIG Assist to develop training in partnership with a supplier
organisation. This was designed to support front line organisations to work with the health
and social care sector. The training sessions revolved around:
Procurement and commissioning skills.
Managing a voluntary consortium.
The training and events have helped to bring voluntary organisations together into
consortia, allowing them to bid for £700,000 new funding from the local CCG. In addition to