EGOV / ePart 2015 - Policy Compass Workshop Presentation

1. POLICY COMPASS
OPEN DATA DRIVEN POLICYANALYSIS AND IMPACT
EVALUATION
Dr. Sotiris Koussouris
Decision Support Systems Laboratory - DSSLab
National Technical University of Athens - NTUA
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

2. Evaluating Policy Making ....
• Which metrics?
• For which kind of applications?
• How do we exactly interpret concepts like
• Quality of life
• Prosperity
• Performance measurement
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
?

3. The Social Indicators Zoo
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Aggregate
(summary)
indicators
• e.g. GDP
Composite
Indicators
Headline
Indicators
• Sustainability
Indicators
International,
National,
Regional and
Local
indicators

4. Issues on the use of indicators
• Institutionalization: setup routine procedures and practices to ensure
the continuing existence of an indicator and to legitimize the method
and concept of the measure. (Drawbacks identified)
• Avoiding bias: indicators should be produced by professional
statistical agencies that have a strong awareness of policy issues,
without having responsibility for them.
• Maintaining the sensitive balance between (global) standardization
and local democracy (identifying the relevant measure to the specific
local circumstances)
31 August 2015
eGov Conference 2015 - Thessaloniki, Greece

5. A methodological framework of indicator development
[Coombes, Wong, 1994]
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Step 1: Conceptual consolidation
Clarifying the concept to be represented by the analysis
Step 2: Analytical Structuring
Providing an analytical framework within which key factors will be analysed
Step 3: Identification of Indicators
Translation of key factors identified in step 2 into specific measurable indicators
Step 4: Synthesis of indicator values
Synthesizing the identified indicators into composite index/indices or into analytical summary

6. How can Policy Compass contribute?
• "Open data" movement and the "social computing" capabilities of
today open a fantastic observatory of experimentation with (and
for) Social Indicators
• Assess the limits of “causal analysis” in policy making, combining
indicators with causal models
• Assess the feasibility of assisting policy design and decision-
making with quantitative handles (Social Indicators) at the global &
local level
• Evaluate the ramifications and the consequences of mixing the
previous items with eGov techniques, ePart tools & Social Media.
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

7. The Problem
• Internet has evolved into a rich source
for information but also to an
instrument of spreading misinformation
and propaganda
• Lack of consensus about a suitable
metric for measuring progress
• Difficulty of objectively assessing the
impacts of government policies
The Proposed Approach
 Open Public Data
 Prosperity Indicators
 Fuzzy Cognitive Maps
 Argumentation Technology
 Deliberation Platforms and Social
Media
The Context
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Improve the quality and
transparency of the policy
analysis and evaluation phases
of the policy cycle for a variety
of stakeholders, ranging from
citizens to policy makers
Agenda
Setting
Analysis
Adoption
Implementation/
Monitoring
Policy
cycle

8. Policy Compass: Overview
A research prototype of an easy-to-use, highly visual and intuitive tool for:
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Metrics
• Constructing
policy metrics
from open data
sources.
Charts
• Generating
graphs and
charts
visualizing
metrics
Annotat
ions
• Annotating
graphs and
charts with
events.
FCM
• Constructing
causal models
as Fuzzy
Cognitive Maps
(FCM).
Web2.0
• Sharing and
debating in
Web2.0
Argume
ntation
• Summarizing and
visualizing the
debates in
argument maps

9. Overview of Main Concepts
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Policy
Compass
Online
Deliberation
and
Argument
Mapping
Policy
Analysis
Policy
Monitoring
and
Evaluation

10. Pillar I: Prosperity Indexes
• Prosperity metrics capture the level of welfare and quality of life in a
given region or society.
• Prosperity is a vague and subjective concept with essential
psychological, social and economic aspects.
• There is no consensus about how to objectively measure prosperity
• Indicators of economic growth:
• Gross Domestic Product (GDP)
• Genuine Progress Index (GPI)
• Index of Sustainable Economic Welfare (ISEW)
• GINI Index
• Alternatives:
• Human Development Index (HDI)
• Legatum Prosperity Index
• “Healthy life years statistics” by Eurostat
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

11. Pillar II: Open Data
• Open and unrestricted access to large scale data sets is essential for
political engagement and scientific research
• Available large scale data sets have nowadays their own self-
contained existence rules.
• Micro-data can be used to construct new indicators of multifaceted
nature.
• Sources of micro-data:
• Eurobarometer surveys
• European Union Statistics on Income and Living Conditions (EU-SILC) by Eurostat
• Urban Audit (the European cities Eurostat)
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

12. Pillar III: Fuzzy Cognitive Maps
• FCMs provide a well-founded, general-purpose and intuitive method for
modelling and simulating relationships between variables.
• An FCM is a fuzzy directed graph of nodes and edges, where nodes
represent fuzzy concepts, describing behavioral characteristics of a system
that occur to some degree, and directed edges represent the causal
relationships among these concepts.
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
• The graph edges are weighted by a real
value from the interval [-1, 1], which
expresses the strength of the relation
between two concepts.
• FCMs have been widely used to model
and simulate policies and their effects.

13. Pillar IV: Argumentation Technology
• Argumentation support systems are computer software for helping people
participate in various kinds of goal-directed dialogues in which arguments are
exchanged.
• The idea of using argumentation support systems for e-Participation can be
traced back at least to Horst Rittel’s pioneering work in the early 1970s who
used visual maps of arguments, to help people collaborate and find solutions
to what he called “wicked problems”.
• “Wicked problems” have no algorithmic, scientific or objectively optimal
solutions for a variety of reasons, including the lack of consensus among
stakeholders about utilities and values.
• Typically, e-Participation projects make use of generic groupware systems
(e.g. discussion fora, online surveys, etc.) not providing though specific
technical support for argumentation.
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

14. Pillar V: Deliberation Platforms and Social Media
• Deliberation platforms incarnate the efforts taken by government
agencies, to increase citizens’ engagement in their decision and
policy making processes.
• The first wave of deliberation platforms has witnessed extensive
information on government activities, decisions, plans and policies,
the proliferation of e-voting and e-consultation spaces, along with
various types of e-fora.
• Still, the first generation of deliberation platforms did not meet the
original expectations.
• The advent of Web 2.0 tools has created a more vivid environment
and the popularity of the social media has set a new battlefield for the
concept of e-Participation.
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

15. Use Case Scenarios @ The Policy Compass “workflow”
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
Policy
Monitoring
and
Evaluation
Data
Discovery and
Processing
Metrics
Definition
Metrics
Calculation
and
Visualization
Graphs
Annotation
Seeking
further
explanation of
policy impacts
Sharing
Knowledge
and Results
Policy
Analysis
Discovery of
Open Data
related to
Policies
Creating or
Refining
Causal
Networks
Turning
Causal
Networks to
FCMs
FCMs
Simulation
and Impacts
Visualization
Sharing
Knowledge
and Results
Online
Deliberation
and Argument
Mapping
Initiation of
/Participation
in
Deliberations
Transforming
Structured
Discussions
into Argument
Maps
Navigation
through
Argument
Maps

16. Example of an FCM regarding Drug Phenomenon
10 February 2015 Policy Compass Plenary Meeting, Berlin, Germany 16

17. Deaths from Drugs vs. Drug Law Offences
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece
http://alpha.policycompass.eu
Drug Mortalities Drug Law Offences

18. Lessons Learnt
• There exists a growing concern on the limits of mesuring societal welfare
with GDP-like metrics. GDP although very successful, is not able to
derive deep results on many aspects of everyday life.
• It makes perfect sense to attempt constructing aggregate, headline or
composite indicators, to measure important social phenomena at the
regional /local level.
• There do not exist ‘objective’ or ‘neutral’ indicators of any kind. One
should not focus on constructing objective indicators, but rather useful
ones.
• Open Data calls for a broad experimentation on the calculation and
exploitation of indicators, and is definitely interesting scientifically and will
provide useful "social" feedback.
31 August 2015 eGov Conference 2015 - Thessaloniki, Greece

19. www.PolicyCompass.eu
www.twitter.com/PolicyCompassEU
www.facebook.com/PolicyCompass
PolicyCompass
Thank you!
Dr. Sotiris Koussouris
Decision Support Systems Laboratory - DSSLab
National Technical University of Athens - NTUA
skous@me.com