2. Meet the team
Lefkothea Spiliotopoulou, Dr. Yannis Charalabidis, Dr. Euripidis Loukis, Dr. Dimitrios Damopoulos
2
3. Introduction
• Ideal democracy allows citizens to directly be involved in the
decision making process
• Traditional direct policy making mechanisms (e.g.
referendums, elections, gallops, polls) cannot be utilized
frequently due to high cost
• Innovation in ICT & social media can be the key to facilitate
collaborative governance with citizens & online participation
Aim:
• Governmental policy making mechanism can utilize public’s
stance to empower direct democracy through text mining
exploitation
3
4. Today we focus on…
• How a stance classification system can be utilized in digital
government for direct democracy empowerment
• Building our model through the use of text mining techniques
(topic modeling, sentiment analysis, stance classification)
• Empirical studies on:
• Public’s stance on 4 EU Referendums
• Political candidates' opinions during
2016 U.S. Presidential Elections
• 2016 U.S. President’s stance towards
societal issues & their affects
4
5. Policy Making Process
• Decisions taken by governments solving problems & improving
citizens’ quality of life
• Stages of policy cycle
• Agenda Setting
• Policy Formulation
• Decision Making
• Policy Implementation
• Policy Evaluation
5
6. Text Mining Techniques
• Topic modeling
• statistical modeling to discover abstract "topics" that occur in a
collection of documents
• Sentiment analysis
• natural language processing to determine positive, negative or
neutral opinions expressed in a text
• Stance classification
• categorize an author’s personal position towards a topic of
discussion as for or against
6
8. • Experiments based on 10-fold cross validation
• Data analysis on the antsle one pro server with a 2.40GHz Intel
8 Core, 32GB ECC DDR3, 12TB internal storage
8
System Evaluation
10. Portrait of EU Referendums
• Utilization of online news sites with articles’ topics based on
the 4 EU Referendums
• Sequence of daily political occurrences (events)
• events with high rate of articles & users’ posts critical
• We extract from each article:
• event’s title
• article
• user’s comments
• timestamp
10
11. Policy making cycle
for end of policy
cycleagainst 11
• Visually represent citizens' stance
• Political parties can learn from real-
time feedback during the process
• Relate positive or negative stance
towards a specific discussion topic
of the agenda
• Re-evaluate policies before decision
making stage
12. Duration of a policy making cycle
12
• Not all policy making cycles are the same when it comes to their
duration
• Represent an event macroscopic or microscopic over the years
• Still something that needs further analysis
• -
for end of policy cycleagainst
13. Early awareness leads to a better
policy
for end of policy cycleagainst 13
• Continues evaluation of policy making process guides to happier
citizens
• Mitigating issues of the policy in early stages can be a route to
successful policy evaluation (or not)
Agenda Setting
15. 2016 U.S. Presidential Elections
• Track public’s stance for Donald Trump & Hilary Clinton
• 2016 U.S. presidential elections public feeling
• online user’s stance towards Donald Trump & Hilary Clinton
• Compare Donald Trump’ & Hilary Clinton’s Stance on societal
issues
• Utilization of Twitter API & online news sites for data
collection
• Events Timeline:
• starting point: Spring of the year before an election (12/04/2015)
• ending point: Inauguration Day (20/01/2017)
15
16. U.S. Presidential Elections Public
Feeling
• Determine public’s sentiment for Donald Trump & Hilary
Clinton in each stage of policy life cycle
• Timeline: 2016 U.S. Presidential elections
16
17. Trump vs Clinton Twitter Stance
• Classify the overall stance for Donald Trump & Hilary Clinton in
Twitter
17
19. Trump vs Clinton Twitter Topic
Stance
• Compare the overall stance of
Donald Trump & Hilary Clinton
in societal issues
• Selection of topics with the
highest popularity for the US
Community
Topics
Guns
Clinton Trump
Abortion
Taxes
Immigration
Health
For
Against
19
20. U.S. President Trump’s Feeling &
Stock Market
• U.S. President Donald Trump’s tweets stance towards policies
formulation (e.g. immigration law)
• Stock market value measurement: correlation of political
presidential statements with market capitalization
20
Marketcap
tech companies
21. Conclusions
• Investigate the societal impact of strong political events
• Introduce a stance classification architecture with
linguistic features
• Analyze citizens’ opinions and their stance towards critical
political topics through our 3 empirical studies
• Online news sites require users’ subscription
• APIs from Twitter or third party services frequently change
or upgrade
• Need to constantly update a platform, to maintain its
functionalities, is a challenging task 21
Summary
Limitations
22. Future Steps
• Explore additional features containing tolerance or irony
• Examine more sophisticated classifiers (e.g. Deep Learning) to
discover hidden features
• Analyze similar political events to gain further insights
• Need of an ideal framework for real-time policy making to
empower users’ participation in the modern era of Digital
Direct Democracy
• Policy making is not just a mechanism. Is a viable system
22
26. Back-End Module
• Twitter & Online News
Sites
• Collect event title,
article, users’
comments, username &
timestamp
• News Sites selection
based on high
popularity measured by
Alexa
26
27. Policy Making Module
• Statistical Analysis to
distinguish days with
high rate of posts &
comments in articles
• Tokenization
• POS-tagging
• Sentence Splitting
• Stanford Parser to
calculate shortest
distance between words
• Porter Stemming
Algorithm to find root of
each word
• NLTK for n-gram
generation
• Create uni-grams & bi-
grams utilizing nouns,
adjectives & verbs
• Utilize them as features
for stance classification
27
28. Policy Making Module
• Mallet to extract topics
with top-words
• SentenceLDA algo with
input posts' sentence &
output topics in each
sentence with top-
words
• Diffchecker to find &
compare whether topics
& top-words of each
posts' sentence are the
same with topic
modeling results
• if yes, sentence is
relative
• Split dataset in training
& testing sets
• Training dataset is by
learning from the 20%
of the daily topics of
each study containing
top-words
• Manual annotation to
label training dataset
• 2 human annotators, via
Mechanical Turk, label
each post's sentence
stance towards topics as
for or against
• Annotate each sentence
based on the topic to
which it was most
related (topic
classification)
• Annotate post ’s overall
position towards the
topic (stance
classification)
28
29. Policy Making Module
• Keep top-words from
topic modeling that are
same with those
identified as uni- or bi-
grams in a sentence
• Calculate top-word score
with tf—idf assigning the
score at the specific POS-
tag as a weight
• MPQA subjectivity
lexicon to assign to POS--
tags sentiment polarities
• Classification Features:
uni- & bi-grams with
weights & sentiment
polarities
• Weka library to build
Stance Classification
module
• Selection of Random
Forest classifier as
engine after cross-
evaluation
• Topic Stance as
predicted class with
values for or against
• Classify topics' stance of
each sentence in each
datasets as for or
against
• Identify overall topic
stance of each post of
the datasets by
summing up the for-
stances and the against-
stances
• Summarization to
determine the overall
stance across all posts of
each empirical study
29
