An analytical impact assessment was conducted on government R&D subsidies for private firms in Korea using different firm-level data sources. Various methods, including propensity score matching and difference-in-differences, found generally positive impacts of subsidies. Subsidies increased total R&D investment by 98% on average over 4 years but decreased firms' own R&D funding by 11% in the benefit year. Sales growth increased 12% over 4 years on average while profit growth was not significantly impacted. Capital investment and employment each increased by around 12% over 4 years. Subsidies also positively impacted firm survival rates.

1. An analytical impact assessment of government R&D subsidy for private firms in Korea was con-
ducted using different kinds of firm level data beyond CIS (community innovation survey) by the
Oslo manual.
Korean government built a data management system called NTIS (National Science and Technol-
ogy Information Service) in which all of detailed information of each public programs and projects
including R&D subsidies for firms has been collected for decades. We used the datasets from
NTIS and Frascati manual, combining them with an external DB of financial statements, in order
to assess the impacts of R&D subsidy on the additionality and performance of the firms that bene-
fited.
Various methods including PSM (Propensity Score Matching), DID (Difference in Differences) and
other econometric models were utilized in order to obtain results as robust as possible.
The impact of R&D subsidy on total R&D investment of firms is clearly positive for beneficiaries
with 98% additional accumulative increase after 4 years in average. Regarding own funding of
R&D investment, however, the result shows negative impact of R&D subsidy by 11% less of an in-
vestment compared to non-supported firms in the benefit year.
Impact of R&D subsidy on sales growth is positive by 12% additional increase after 4 years accu-
mulation on average; however, the result does not show any statistically significant impact on
profit growth for beneficiaries
Impact on capital investment and employment are positive by 12% and 11% additional increase
respectively after 4 years accumulation. The impact on firms’ survival is obviously positive by 57%
decrease in hazard rate for R&D subsidy beneficiaries.
Impact Assessment of R&D Subsidy on Input Additionality and
Firms’ Performance Using Firm Level Data: the Korean Case
Seogwon Hwang*, Seung-Hwan Oh, Hee-Jong Kang, Justine Kim
OECD BLUE SKY III, 19-21 SEPTEMBER, GHENT, BELGIUM
* hsw100@stepi.re.kr; STEPI, Sejong National Research Complex, 370 Sicheong-daero, Sejong-si, 30147 Korea stepi.re.kr
ABSTRACT
STEPI developed a framework that allows systematic assessment of STI policies’
economic and societal impact and also proposed an appropriate process in doing
such assessments in the future.
Overall framework consists of economic impact and societal impact, where eco-
nomic impact is then divided into microscopic and macroscopic perspectives.
Among them, we conducted the microscopic economic impact assessment as a
pilot analysis. The results are thoroughly described in the following section.
IMPACT ASSESSMENT PROCESS
Microscopic Economic Impact of R&D Subsidy
An analytical impact assessment of government R&D subsidy for private firms in Korea was
conducted using different kinds of firm level data beyond CIS (community innovation survey)
by the Oslo manual.
After the Blue-sky conference 2006 held in Ottawa, ‘OECD Innovation Microdata Project’ was
launched and produced many implications for evidence based STI policies using firm level mi-
cro-data. The micro data used for the project mainly came from CIS, which includes some indi-
cators for innovation outputs. Detailed information of public support for firms in terms of direct
R&D subsidy, which just come from the governmental accounting system, is another very con-
crete data source. Korean government built a data management system called NTIS (National
Science and Technology Information Service) in which all of detailed information of each public
programs and projects including R&D subsidies for firms has been collected for decades. We
used the datasets from NTIS and Frascati manual, combining them with an external DB of
financial statements, in order to assess the impacts of R&D subsidy on the additionality and
performance of the firms that benefited.
The analysis of R&D investment additionality includes the behavior of so called ‘zombie com-
panies’ which has recently emerged as a big problem in the Korean economy. It is expected
that combining public data sources such as NTIS and survey data by Frascati manual with ex-
ternal sources such as financial statements DB would give much opportunity for impact as-
sessment in the future, contributing to the evidence base of STI policies.
Various methods including PSM (Propensity Score Matching), DID (Difference in Differences)
and other econometric models were utilized in order to obtain results as robust as possible.
The impact of R&D subsidy on total R&D investment of firms is clearly positive with 98% addi-
tional increase accumulatively after 4 years for beneficiaries on average (Hwang et al., 2016).
Regarding own funding of R&D investment, however, the results show negative impact of R&S
subsidy by 11% less investment compared to non-supported firms in the benefit year (Hwang
et al., 2016). The impact of R&D subsidy on so called ‘zombie companies’, which are defined
by low profitable companies with TIE (Times Interest Earned) below 1 for 3 consecutive years,
is very noticeable. Total R&D investment containing both of own funding and subsidy does not
increase for very severe ‘zombie companies’ with TIE below 1 for 5 consecutive years, which
means that those firms misuse R&D subsidy for other purposes than R&D activity (Kang and
Hwang, 2016).
Regarding other input additionality of beneficiary firms, the result shows positive impact on
capital investment and employment by 12% and 11% additional increase respectively after 4
years accumulation (Hwang et al., 2016). The impact on firms’ survival is obviously positive by
57% decrease in hazard rate for R&D subsidy beneficiaries (Kim and Hwang, 2016).
One of the most important impact goals might be the performance of benefited firms, so we
analyzed two representative indicators of growth and profitability. Impact of R&D subsidy on
sales growth is positive by 12% additional increase after 4 years accumulation on average;
however, the result does not show any statistically significant impact on profit growth for bene-
ficiaries (Hwang et al., 2016).
PILOT ANALYSIS
Why is the impact assessment of STI (Science and Technology Innovation) policies important?
Since the global financial crisis of 2008, world economy has been suffering from uncertainties
and low-growth. Several countries including Korea are facing budget limitation for R&D invest-
ment, whereas people expect to benefit more economically and socially from STI investments
of their countries. In order to meet these people’s expectations, countries around the world are
endeavoring in designing novel STI policies that would allow STI investments to yield greater
economic and societal impacts. In doing so, ‘impact assessment’ is an essential part of the
policy infrastructure.
In order for the STI impact assessment to be properly executed, an extensive number of data
(big-data level) is required, and methodologies must be improved as well. It is also suggested
that the cooperation of many countries, centering on OCED countries, is needed. Such issue
was discussed as one of the main agenda in OCED World Science & Technology Forum that
was held in Daejeon, South Korea in 2015.
FIGURE 1
source: modified by the authors based on EC(1997)
INTRODUCTION STI Policy Demand
Efficiency Evaluation
Effectiveness Evaluation
Impact Assessment or Impact Evaluation
Evaluation
Societal Needs,
Government Initiatives
Result
- initial impact
- achievement of clearly
described goals
Outcomes
Economic, Societal and
Academic Longer-Term
Impact
Objectives
Input
- R&D funds
- Human Capital
- Time
Planning and
Implementation
Output
- Papers
- Patents
- Prototype
- Design (Theory)
- Development
(Experiments)
- Testing
- Project Management
R&D Process
- Commercialization
- Technology Transfer
Application
Process
Data Collection / DB
Microscopic Economic Impact
Assessment Process
Macroscopic Economic Impact
1st stage
Linked with enterprise DB
R&D project Information
- funding size, etc.
R&D stock
Physical capital stock
Employment
Trade (incl. technology trade)
Education investment
Output data (NTIS)
- academic papers
- patents, etc.
Linked with enterprise DB
Innovation policy measures
Information
- financial support for SMEs
- tax benefits
- public procurement
2nd stage
Indicators and Statistics
Areas of Societal Impacts
1st stage
Collecting indicators and
statistics related to societal
impacts
Income / Wealth
Work
Health
Relationship
Political / Societal
activities
Transaction
Education
Food / Clothing and Housing
Safety
Government Service
Environment / Energy
Leisure
Locomotion
Social Problems in
real and virtual spaces
- employment
- entrepreneurship
- start-ups
- scale-up
- zombie companies
- reunification economy
- creative economy, etc.
Survey for industrial leaders
and/or the public on societal
impacts from STI policies
2nd stage
Planning for Impact Assessment
Assessing Economic/Societal Impacts
Economic impact analysis based on econometirc models
Societal impact assessment by the expert panel
Suggestions for improving policy implementation processes
Suggestions for policy measures to enhance economic / societal
impacts
Reporting Process
Implementing New Policy Measures
New measures to improve policy design, implementation, evalua-
tion processes
New policy measures to enhance economic and societal impacts of
STI policies
Reporting to NSTC (National S&T Committee) in January next year
NSTC decision for measures to improve policy processes
NSTC decision for policy measures to enhance economic and
societal impacts
To start in January for preparing next year’s budget decision making
Time schedule for impact assessment in detail
Main issues at that time
Selection of programs
Impact Assessment
of STI Policies
Classification
Growth
Sales Growth Rate
ATT (Average Treatment Effect on the Treated)
A Year
Later
Two Years
Later
Three Years
Later
Four Years
Later
Asset Growth Rate
Debt Growth Rate
Growth Rate of Employee
R&D Growth Rate
Growth in R&D per Sale
ROA Growth
ROE Growth
Profit Growth
Labor Productivity Growth
Growth Rate in R&D per personInnovativeness
Profitability
STI Policy
Evaluation
Subject
Post
Management
Methodology
Pre vs Post
Evaluation
Tone of
Policies
Evaluation
Category
Background
Focus
R&D programs Technology
R&D Management S&T Regulation
Technological Risk
Regulation Set-up
Qualitative Analysis
Budget Allocation
Post < PrePost ~ PrePost > Pre
ReflectiveManagerialProactive
STI Promotion
Impacts of STI policy
Economic / Societal
Positive Impact
Betterment of existing
policies and establishment
of new policies
Level1: Quantitative or
Econometric Analysis
Level2: Qualitative Analysis
Excellence of
R&D Outputs
Peer review and index-
based evaluation
Outputs of
R&D Programs
Social / Environmental
Negative Impact
Usual R&D
Evaluation
Technology
Assessment
TABLE 1
How Is Impact Assessment Different from
Usual R&D Evaluation or Technology Assessment
FIGURE 2
Impact Assessment Process Suggested
REFERENCES
We believe the results of pilot analysis on impact assessment of R&D
subsidies in Korea using combined datasets from NTIS, surveys by
Frascati manual and a commercial DB of financial statements would con-
tribute to the issues of 2016 Blue-sky conference such as ‘key lessons
after 10 years of science for science and innovation policy initiatives’, ‘in-
teraction and impact of STI policies’, ‘new data infrastructures for the
analysis of science and innovation’ and ‘the integration of STI statistics
and other statistical frameworks’.
There are many issues on firm level impact assessment not yet fully ana-
lyzed in our study [Figure 3]: The impact of R&D subsidy on productivity
and the relationship between productivity and profitability, knowledge
flow from universities and GRIs which might imply the impact of public re-
search investment on firms’ performance, impact on scale-up and over-
seas market performance of benefited firms, etc., all of which are possibly
future topics for our ongoing impact assessment study.
Although not mature enough, impact assessment of STI policies should
cover the impacts not only on economic growth and industrial competi-
tiveness but also on our life, jobs and sustainability in the future. Data
coverage should be extended wide into various STI policy areas including
not only R&D subsidies, but also tax benefits for firms, supporting pro-
grams by government procurement, loans and/or equity investments for
innovation from public financial institutes and so on.
(1)
(2)
(3)
(4)
(5)
EC, Evaluation EU Expenditure Programmes: A
Guide. Ex post and Intermediate Evaluation,
1997
Hwang S., Kang H. J., Oh S. H., Lee S. Y., Kim Y.
H., Jung J. H., Lee S. W. and Kim K. H., 2016, A
study on the construction of economic and soci-
etal impact assessment framework of national
R&D investment, policy report supported by
MSIP, STEPI (in Korean)
Kang H. J. and Hwang S., 2016, Impact of R&D
Subsidy on R&D Investment additionality of
Firms, STEPI working paper (in Korean)
Kim K. H. and Hwang S., 2016, Impact of Devel-
opment Subsidy on Firms’ Survival, STEPI work-
ing paper (in Korean)
OECD, 2009, Innovation in Firms: A microeco-
nomic perspective
CONCLUSION
Research Objective:
Analyzing the impact of R&D subsidy on the performance of private firms
Methodology:
Propensity Score Matching (PSM)
Results:
- Beneficiary firms showed greater performance in growth and funding ability 4 years
after receiving the subsidy, compared to those that did not receive.
- Whereas, no clear evidence of profitability growth of beneficiary firms 4 years later.
PSM APPROACH
Research Objective:
The impact of government R&D subsidy on the firms’ own R&D activities/investment was analyzed.
Models:
The dependent variable refers to log-value of R&D expense of firm in year . The dummy variable refers to whether or not
firm received government R&D subsidy in the given year (if yes = 1). Variable refers to the size of firm in year , where big-size=1,
mid-small size=2, and venture firm=3. Variable refers to the type of business (manufacturing=1, service=2, construction=3, etc=4).
Results:
Firms that received the government R&D subsidy in year showed smaller self-investment in R&D in year than those that did not receive the subsidy.
- As the result of Model 1, the result through DID analysis method showed that the firms that received the government R&D subsidy in year showed
smaller self-investment in R&D in year than those that did not receive the subsidy.
- Firms that received government R&D subsidy in year invested a greater amount in R&D in year than those that did not receive the subsidy.
ECONOMETRIC APPROACH
, 0 1 , 1 2 , 1 3 , 1 ,
4 , , ,
ln ln tititititi
i t i t i t
SelfRD SelfRD DumGRD DumGRD DumGRDβ β β β −−−= + + + g
nlnlnl titititititi −+
t t
t
t
t 1t +
,ln i tSelfRD i t DumGRD
i ,i tSize i t
,i tSector
(2-1)
(2-2)
Coefficient
Dependent
Variable
Major
Independent
Variable
Model 1
Model 2-1
Model 2-2
-0.1220 *
: Negative **
: Positive **
TABLE 3
,ln i tSelfRD
,ln i tSelfRD
TABLE 2
1.88
2.83
2.19
3.36
35.7
21.1
-1.46
0.83
17.6
7.62
0.007
**
***
***
***
***
***
**
4.18
6.15
5.63
8.06
58.2
32.3
2.58
2.45
119.1
4.26
-0.16
***
***
***
***
***
***
***
**
*
6.57
6.78
7.61
9.22
81.8
42.7
-4.19
-0.75
0.46
5.74
-0.001
***
***
***
***
***
***
12.1
10.75
11.9
14.37
97.9
35.4
-21.2
-0.32
-2.7
10.24
0.005
***
***
***
***
***
***
*
**
Pubic R&D
Investment
Impact on
Input Additionally
Impact on
Performance
Public Research Institutes,
Universities
3.7% annual growth(quantitative: no. of SCI papers)
6.0% annual growth(qualitative: average Impact factor)
11.9% additional increase
after 4 years
10.8% additional increase
after 4 years
12.1% additional increase
after 4 years
impact on total R&D investment of firms
- 97.9% additional increase after 4 years
- elasticity 0.21
impact on self funding of R&D investment of firms
- 11% less investment compared to non supported firms in that year
Impact of subsidy
to development cost:
decrease in hazard rate 57.7%
R&D Subsidy for firms
Academic Research Impacts
R&D Investment
Capital Investment
Employment
Pilot Results Positive
Knowledge Capital
Stock
Physical Capital
Stock
Labor Force
Survival
in the Market
Impacts on Human Capital
Productivity
Production
Growth
=
Quantitative
Growth
Profit
Growth
=
Qualitative
Growth
Innovation Activity
Scale-up
Going to Global
Market
Not PositiveFuture Research
FIGURE 3
Impact(effects on society)