Dr Tony Moody - Business & Science Group - BEIS - 14 october 2016

1. PRODUCTIVITY,
FIRM-LEVEL INTERVENTION &
HIGH-QUALITY EVALUATION
Dr Tony Moody | Business & Science Group | BEIS
October 2016
Tony.Moody@beis.gov.uk
1

2. Belief: UK Productivity is being held back
by deficiencies in management and
leadership capabilities.
• Productivity decompositions attribute much of the
productivity gap against G7 competitors to TFP, rather
than labour quality or capital investment. Management &
leadership capability would appear in TFP.
• International comparisons place the UK in mid-table for
quality of its management and leadership.
• Deficiencies in UK management skills are a key driver of
the productivity gap
• Bloom et al (2014) estimate that about a quarter of
the UK’s productivity gap with the United States
could be down to poor management
• Recent research with the ERC has expanded analysis to
SMEs in the UK and finds shows similar links between
skills, practices and performance (lower diagram), and a
‘long tail’ of firms with room for improvement
• UK Government invests extensively in improving
workforce skills (schools, University, vocational quals).
Likewise in improving capital markets, R&D, innovation.
These are the ingredients of productivity.
• But investment in how the ingredients are combined –
the recipe and the culinary skills – in UK is low,
suggesting potential for intervention to valuably improve
how people and resources are used more effectively.
How management skills and practices lead to improved outcomes
Source: Constraints on Developing UK Management Practices, BIS 2011
Source: BIS (2015) Leadership and management skills in small and medium-sized businesses.
Average management and leadership scores by country,
ranked from low to high
2

3. Hypothesis: Business-Level intervention can
Improve Whole Economy Productivity
• Adding fertiliser to a plant to improve the ecology of a habitat.
• …Yet, so many processes between intervention and impact,
and spanning big differences in scale
?
3

4. Where to start?!
Remains extremely difficult to pick future HGFs
• Predicting future high-growth prone to Type I & Type II errors
• Only know the characteristics that improve the chances of
future high growth:
• E.g. capabilities of owner/managers, level of ambition,
market opportunities.
• …but often unobservable and even then hardly high precision.
• If hard to pick winning businesses, even harder to pick proven
winning policy interventions!
4

5. Logic Model with Huge Black Box
Economy-level impacts
Displacement
(factor market
and/or product
market)?
Firm-level impacts
Perceptions or
business metrics
Proxies for
productivity.
Intervention
Selection effects? Purity of treatment
Firm to
receive
intervention
Productivity
Impact?
5

6. There is scarce robust evidence to measure
impacts or improve design
• Evaluations typically focus on the beneficiaries’ assessment of benefits and a
single policy design
• This approach does not provide a robust assessment
• The UK What Works Centre conducted a review of impact evaluations from
across the OECD
• Just 1% provided credible evidence of growth impacts
Area
Evaluations
Reviewed
Robust approach
Growth impacts
identified
Business Advice 730 23 14
Access to Finance 1450 27 17
Innovation 1700 63 9
6

7. Randomised Control Trials (RCTs) of
business advice in BEIS
• Using RCTs to address specific
questions in our hypothetical
causal chain.
• If one link in the circuit doesn’t
work, then whole black box
won’t work.
• Four RCTs on Business Advice:
- Main Growth Vouchers Programme (GVP)
- Growth Vouchers Additional Target group
- Business Schools RCT
- Growth Impact Pilot RCT
7

8. Growth Vouchers Programme - An
example of a large scale trial
• UK launched the Growth
Vouchers Programme as a
research trial testing the
benefits of supporting
businesses to use external
strategic advice
• Between Jan 2014 and
March 2015:
• 38,000 applications
• 28,000 diagnostics
• 20,000 vouchers issued,
worth up to £39.4 million
• 6,400 vouchers used,
£11m claimed
• RCT implemented
successfully
8

9. Growth Vouchers Programme – Early Evidence
• Subsidising cost of Business Advice led to increased use of
advice in treatment group (no evidence of substitution).
• Positive initial perception evidence from treatment group in
terms of future business actions needed for growth.
• But too early for business performance metrics (and take a long
time to measure in admin data because of reporting lags).
• There might be an adverse impact on self-assessment of
business skills within treatment as they are exposed to what
they don’t know, and consider focus (tackling ‘how’, not ‘what’).
9

10. Emerging lessons from wider BEIS evaluation
• Can’t pick likely future winners from data. We cannot identify future
HGFs from standard business data on current and past characteristics
(e.g. growth trajectories, size, age, sector, geography).
• …yet businesses receiving intervention grow quicker. Businesses
participating on GrowthAccelerator (part of Business Growth Service)
grew 3x faster than peers – so something going on – either successful
selection based on richer data (inc. soft data), or an indicator of
intervention impacts.
• Perception data from managers may not align to business
performance.
• How businesses act on advice varies. We don’t know what changed
behaviour will produce the most improvement.
• Time lags. Our wider evaluation evidence suggests that impacts on
business metrics might take 3-5 years to reach full effect – just as
many evaluations stop listening.
10

11. Many Questions remain unanswered
We don’t know:
• Improvement in SME-level productivity. [We measure turnover/jobs,
not GVA]
• Improvement in wider economy productivity.
But suspect:
• Rather than measure productivity directly, past interventions seek to
minimise substitution or displacement through eligibility and targeting.
• Return of investment unlikely to be scalable to offer impacts
measureable at economy-wide level.
• …but they are likely to be a helpful investment with a speedy return
because they engage currently economically active and ambitious
businesses.
11

12. PRODUCTIVITY,
FIRM-LEVEL INTERVENTION &
HIGH-QUALITY EVALUATION
Dr Tony Moody | Business & Science Group | BEIS
October 2016
Tony.Moody@beis.gov.uk
Our research, evaluations and surveys are published here:
https://www.gov.uk/government/collections/enterprise-analysis-research
12