Do you know how data-driven approaches can influence the policy cycle and the benefits derived from this? Have you ever participated in a policy-lab, collaborating with other stakeholders to develop and test a policy? In this session, Anne Fleur van Veenstra from TNO will delve into current practices, insights and lessons learnt from current policy-lab projects, followed by Francesco Mureddu, from the Lisbon Council, who will look ahead and identify the main challenges and opportunities by presenting and discussing a roadmap for Future Research Directions in data-driven Policy Making.

1. DATA-DRIVEN POLICY MAKING:
METHODOLOGY AND EXPERIMENT
Anne Fleur van Veenstra
TNO Strategy & Policy

2. THERE IS A NEED TO RENEW THE LEGITIMACY OF
PUBLIC POLICY-MAKING, ESPECIALLY THROUGH
GREATER CITIZEN’S INVOLVEMENT AND OF
DELIVERING BETTER PUBLIC SERVICES FOR ALL
(ECOM 4614, 2016).
1. Data driven policy making
2. Policy Lab approach
3. Experiment: Rotterdam youth care
Data-driven policy making: methodology and experiment

3. DATA DRIVEN POLICY MAKING
Data such as (real-time) sensor data may
provide new insights for (‘evidence-based’)
policy making
This may also require new methodologies, e.g.
will machine learning enhance policy models?
Co-creation: stakeholder involvement in
different phases of the policy cycle
Challenges include organizational readiness
and policy makers’ willingness for using data
and data-driven methods for policy making
Data-driven policy making: methodology and experiment

4. THE ROLE OF EXPERIMENTS
Legal restrictions, such as the GDPR, require a
safe environment and methodology for
experimenting:
to explore the impact of new technologies and
methodologies on public policy;
to develop or enhance an evidence-base for
policy;
to involve policy makers and their organizations
to investigate the opportunities and challenges
that arise for scaling; and
to involve citizens (and other stakeholders) in
policy making (‘co-creation’)
Data-driven policy making: methodology and experiment

5. POLICY LAB
APPROACH
1. Identify new data sources and
technologies that impact public policy.
2. Design experiments to test new
technologies, methodologies and policy
models.
3. Implement and monitor policy;
develop opportunities for scaling.
Data-driven policy making: methodology and experiment

6. CASE: YOUTH POLICY IN ROTTERDAM
Data on social-emotional skills of
youngsters may enhance the current
policy model
Develop a (double) hybrid policy model
using theory and data and machine
learning and ‘traditional’ statistics
Data-driven policy making: methodology and experiment

7. Exploring new data sources and technologies and
their impact on policy
1. Analyse the current theory-based policy model
2. Identify data sources that may enhance the policy
model
3. Develop DPIA / data processing agreement
4. Gather and clean the data
5. Train the model (machine learning)
6. Perform statistical analyses
7. Analyse outcomes; explainability
8. Develop hybrid policy model
PREDICT
Data-driven policy making: methodology and experiment

8. MODEL FOR YOUNGSTERS’ SOCIAL EMOTIONAL
CAPABILITY AND BEHAVIOUR
1. Based on literature, the municipality developed
a conceptual model covering many factors that
are of influence on the social-emotional
capabilities of youngsters
2. For every aspect related to this factor, data
sources were identified and a data
collaboration was set up between multiple
organizations supplying these data sets
3. Data analyses were carried out, both using
machine learning and statistical methods
4. A hybrid policy model that is used to inform
interventions was developed
Data-driven policy making: methodology and experiment

9. OUTCOMES
Not one variable could be found that would have the largest influence on factors related to social-
emotional capabilities
House value as a proxy for income was found to predict social emotional capabilities best
Youngsters with higher attributed social emotional capabilities generally showed better behavior
It was hard to obtain the necessary data
Pre-process data was difficult and required iteration
Multi-disciplinarity takes a lot of time!
Data-driven policy making: methodology and experiment

10. REQUIREMENTS FOR SCALING
Joint development of a data processing
agreement is important for establishing trust
‘Top management support’ is essential
Many aspects of the GDRP remain unclear
and need to refined
Differentiation between experiment and
application in practice is important
Legal grounds for experimentation differ
between domains
Application of findings to new data may
include biases
Multidisciplinarity is a requirement for useful
outcomes
Understanding of the domain, the model and
the data is necessary
An ‘agile’ way of working may support this
Explainability of machine learning is a
challenge
Training the model is dependent on the data
available
Importance of a hybrid approach in which
statistics test the machine learning precitions
Data-driven policy making: methodology and experiment

11. IMPLICATIONS FOR POLICY
Data is often gathered within a different context
dan in which is it reused
Data landscape often grew organically
Technical interoperability; semantics en
contextual understanding are all necessary
for reuse of data
New dependencies emerge: data is not always
gathered within the same organization
Cooperation in networks is required
Organizations that were formerly a part of
government now become data providers
It is expected that the policy cycle will
accelerate
Different policy phases follow up more quickly
and new links between phases emerge
This also offers opportunities for rapid
responses in case of undesired outcomes
Collaboration in networks and co-creation require
a large degree of openness
In which way governments should become
transparent: data, algorithm, via validation?
Outcomes of analyses under increased
scrutiny
Data-driven policy making: methodology and experiment

12. DR.IR. ANNE FLEUR VAN VEENSTRA
ANNEFLEUR.VANVEENSTRA@TNO.NL