Presentation by Christiane Arndt, Programme Co-ordinator Measuring Regulatory Performance, OECD, at the 6th Expert Meeting on Measuring Regulatory Performance: Evaluating Stakeholder Engagement in Regulatory Policy, Introduction, The Hague, 16-18 June 2014. Further information is available at http://www.oecd.org/gov/regulatory-policy/

1. Evaluating stakeholder
engagement:
Practices in OECD countries
Laura Seiffert, Junior Policy Analyst, Regulatory Policy
Division, OECD
Breakout session 1
6th Expert Meeting on Measuring Regulatory Performance “Evaluating
Stakeholder Engagement in Regulatory Policy ”

2. - Stakeholder engagement is essential to
enhance regulatory quality, compliance
with regulation, and trust in government
- Evaluation helps to:
- Highlight successes and areas for
improvement
- Guide policy makers in their efforts to
improve stakeholder engagement
Why evaluate?

3. OECD Framework for Regulatory Policy Evaluation –
applied to stakeholder engagement
Have the good
practices in been
implemented?
 E.g. % of
consultations that
comply with formal
requirements, such as
minimum periods
Are requirements for
good practices in
stakeholder
engagement in place?
 E.g. requirements to
consult on draft
regulation, requirement
to make relevant
material of regulatory
dossier available to
inform consultation
(impact assessment,
explanation of reasons
to regulate ect.)
What resources are
committed to
stakeholder
engagement?
 E.g. budget, staffing,
expertise
Have good practices helped to
improve quality regulation?
 E.g. % of those involved in
the regulatory process that
think stakeholder engagement
has improved the quality of
regulation as opposed to being
a tick-the box-exercise;
comparison of cost/benefit of
initial regulatory proposals with
those of regulations that were
actually passed
Have strategic objectives for
stakeholder engagement been
achieved?
 Improved compliance rate
 Reduced enforcement costs
 Trust in government
 Inclusive growth
 Economic: Net benefits (possibly in
comparison with counterfactual and
alternatives)
 Reduced compliance costs and
burdens
 Improved perception of regulatory
quality in general
 Transparency and accessibility
 Objective of regulation achieved
easy hard
• Regulatory system
• Regulatory impacts
• Other factors that influence
outcomeFeedback loop
Input Process Output Intermediate outcome Outcome
Set strategic
objectives
for
stakeholder
engagement
Design Implementation Strategic outcome
I II III IV V

4. • Public consultation is seldomly evaluated compared to
other regulatory management tools
Current country practices
8
6
20
14
21 20
0
5
10
15
20
25
Reports Indicators
Numberofcountries
Number of countries publishing performance reports or indicators
on consultation, RIA or administrative burden reduction
programmes
Consultation on draft
regulation
Regulatory Impact Analysis
(RIA) on draft regulation
Administartive burden
reduction programmes
Source: OECD survey on Regulatory Policy Evaluation (2012-13)

5. Current country practices
• Countries publishing performance reports:
– Czech Republic, European Union, Japan, Mexico, the
Netherlands, Slovenia, Switzerland, Turkey
• Countries publishing performance
indicators:
– Estonia, European Union, Japan, Mexico, Slovenia, Turkey

6. • Who publishes reports and indicators?
– Independent or central government agency/department
– Research institutes
• For what purpose are indicators used?
– To adjust the design and implementation of stakeholder
engagement processes
– To monitor quality and compliance with formal
requirements
– To communicate with internal or external stakeholders
Current country practices

7. • Output indicators
– Percentage of consultations that comply with
minimum periods (e.g. EU, Japan, Slovenia)
– Number of different stakeholders participating in
consultation (e.g. Mexico, Estonia)
– Number of languages in which consultation
documents are published (EU)
– Total numbers of consultations (e.g. EU, Switzerland,
Turkey)
Type of performance indicators used

8. OECD Practitioner‘ s Guide to
Perception Surveys (2012)

9. • European Union: Review of the Commission
Consultation Policy (2012)
• Switzerland: Evaluation of practices with regard
to hearing and consultation procedures (2011) –
in French
Examples of performance reports

10. • There is consensus on the importance of
effective stakeholder engagement
• Few countries evaluate how consultation is
functioning in practice
• Systematic evaluation is within reach for all
OECD countries
• OECD work provides guidance and many
practical examples on how to evaluate and
improve stakeholder engagement
Key messages

11. 1. What are key good practices in evaluating government
systems to engage with stakeholders in the regulatory policy
cycle?
2. How can countries assess :
 The inclusiveness of consultation?
 The transparency of the consultation process?
 Whether stakeholders’ views are adequately considered and
integrated into the decision-making process?
3. What are key challenges in assessing the engagement with
stakeholders and how can they be overcome?
Questions for discussion

12.  OECD (2014), OECD Framework for Regulatory Policy
Evaluation
• Radaelli, Claudio and Oliver Fritsch (2012), Measuring
Regulatory Performance - Evaluating Regulatory
Management Tools and Programmes, Expert Paper, No. 2,
Paris.
• OECD (2012), Measuring Regulatory Performance: A
Practitioner’s Guide to Perception Surveys
• OECD (2005), Evaluating Public Participation in Policy
Making
• Coglianese, Cary (2003), Is satisfaction success? Evaluating
public participation in regulatory policy-making
Relevant material

13. • Christiane Arndt, Programme Co-ordinator Measuring
Regulatory Performance, E-mail:
christiane.arndt@oecd.org, Tel. +33 (0) 1 45 24 76 56
• Daniel Trnka, Policy Analyst, E-mail:
daniel.trnka@oecd.org, Tel.: +33 (0) 1 45 24 18 68
• Antonia Custance Baker, Economist/Policy Analyst, E-
mail: antonia.custancebaker@oecd.org, Tel.: +33 (0) 1
45 24 93 22
• Laura Seiffert, Junior Policy Analyst, E-Mail:
laura.seiffert@oecd.org, Tel.: +33 (0) 1 45 24 13 37
Contacts