anny suggestions and comments will be apreciated

1. By
Muhammad Tahir Orakzai
NATIONAL INSTITUTE OF MANAGEMENT, PESHAWAR
Analyzing Public Sentiments
Using
Twitter Feeds
23rd SMC
Current Issue Presentation
Faculty Advisor
Akbar Ali Khan 2nd May 2018

2. • Introduction
• Acronyms
• What is Analysis?
• What is Sentiment Analysis?
• Social Media & Twitter Feeds
• Importance of Public Sentiment / opinion
• Sentiment Analysis
• Evolution of WWW
• Data Mining: KDD
• Sentiment Analysis Process
• Case Studies: Pros & Cons
• Conclusion
• Recommendations
2
Sequence of Presentation

3. Glossary of Terms
3
@ : at the rate of
# : Hashtag Preceding a Word
AI : Artificial Intelligence
Bot: Web Robot
Com : Commercial
DM : Direct Message
EDU : Education
HTTP : Hyper Text Transfer
Protocol
ICT : Information
Communication Technology
IM : Instant Message
KDD : Knowledge Discovery of
Data
NET : Network
NLP : Natural Language
Processing
OS : Operating System
PC : Personal Computer
PEW : Family Name not and acronym
PPT : Power Point Presentation
Pros & Cons : Pro et Contra (Latin Word, for and
Against)
RAND : Research & Development
SA : Sentiment Analysis
SMS : Sort Message Send
SWOT : Strengths, Weaknesses, Opportunities, Threats
UGC : User Generated Content
URL : Uniform Resource Locator
Web 2.0 : Social Web
WWW : World Wide Web

4. Heard of these terms?
• Analysis
• Public Sentiment Analysis
• Twitter Feeds: Social Media
• User Generated Content (UGC)
• WWW: Web 2.0
• Data Mining: KDD
• Metadata
• Artificial Intelligence (AI)
4
Introduction

5. “The beginning of
wisdom is the
definition of terms.”
Socrates (470–399 B.C.)1
1 https://knowledgism.com/glossary-2/the-importance-of-the-definition/?v=47e5dceea252 5
Introduction

6. • A systematic examination and
evaluation of data or information, by
breaking it into its component parts
to uncover their interrelationships.
• Opposite of synthesis.2
2 http://www.businessdictionary.com/definition/analysis.html 6
Analysis

7. • The process of computationally identifying
and categorizing opinions expressed in a
piece of text, especially in order to
determine whether the writer's attitude
towards a particular topic, product, etc. is
positive, negative, or neutral.3
3 https://en.oxforddictionaries.com/definition/sentiment_analysis 7
Sentiment Analysis

8. Sentiment analysis has many other names 4
• Opinion mining
• Opinion extraction
• Sentiment mining
• Subjectivity analysis
4 https://web.stanford.edu/class/cs124/lec/sentiment.pptx 8
Sentiment Analysis: Synonyms

9. • the collective opinion of many people on
some issue, problem, etc., especially as
a guide to action, decision, or the like 5
5 http://www.dictionary.com/browse/public-opinion 9
Public Opinion

10. Social
Media
Social
Networking
Blogs/
Microblogs
Wiki/
Forums
Content
Sharing
10
Social Media

11. • Online social networking service
• 140 (280) character - “tweets”
• Registered users can read & post
tweets (UGC)
• Communicate and stay connected
• Tweets - words, photos, videos and
links 6
6 https://en.wikipedia.org/wiki/Twitter 11
Twitter

12. • Created & Launched in 2006
• 330+ million active users
• 500 million tweets are generated
everyday
• Microblogs: SMS of the Internet
• Audience common man - celebrities
• Discussion on current affairs and share
personal views on various subjects
7 https://en.wikipedia.org/wiki/Twitter 12
Twitter7

13. • Retweet: a tweet that you forward to your followers
that always retains original attribution
• @: the @ sign is used to call out usernames in tweets
• #: any word or phrase immediately preceded by the #
symbol. By clicking a hashtag you can see other tweets
containing the same keyword or topic
• Follower: another person who receives your tweets on
their Home stream
• Direct Messages: private messages sent from one
Twitter user to another.
• Home: timeline displaying a stream of tweets from
accounts you have chosen to follow
8 https://blog.hubspot.com/marketing/34-twitter-terms-defined-list 13
Twitter: Terminology 8

14. • Democratic governments rest on the
consent of the governed
• Hence, major shifts in public opinion should trigger
a shift in public policy
• This attention to public opinion has
created an industry in public opinion
polling and survey research.
14
Public Opinion Importance

15. • Opinions are the influencers of our
behaviors
• Before making a decision, we usually seek
opinion of others:
• Buy a product, rent a car, reserve a hotel room,
looking for a good restaurant …
• Individuals
• Organizations
• Governments 15
Public Opinion Importance

16. • General: is this review positive or negative?
• Product: what do people think about the new
product?
• Public sentiment: how is consumer confidence?
is despair? or increasing?
• Politics: what do people think about this
candidate or issue?
• Prediction: predict the outcomes or assess trends
from sentiments
16
Importance of Twitter SA

17. 17
Buzz Tweet

18. 9 https://medium.com/@vivekmadurai/web-evolution-from-1-0-to-3-0-e84f2c06739 18
Evolution of WWW : Rise of
Social Media 9

19. 10 https://apinyaphanumphai.wordpress.com/2013/02/18/the-web-evolution/ 19
Evolution of WWW 10

20. 11 https://twitter.com/wef/status/930131155248209920 20
Scale of Internet Today 11

21. • Twitter “feed” is any list of tweets
that constantly updates when new
tweets that fit the specified criteria
pop up.
• An ongoing stream of Twitter
messages (tweets)
12 https://blog.hubspot.com/marketing/34-twitter-terms-defined-list 21
Twitter Feeds 12

22. • Opinion Mining, Text Mining
• Sentiment and Subjectivity
Analysis
• Artificial Intelligence
• Natural Language Processing
• Computational Linguistics
• Etc.
22
Knowledge Discovery from Data

23. • Sociologists
• Statisticians
• Researchers
• Linguists
• Computer Experts
23
Back End Professionals

24. 13 http://www.academicstuff.co.in/Web-Sentiment-Analysis-for-Scoring-Positive.html
Opinion of
People
Feature
Extraction
Sentence
Extraction
Sentiment
Classification
Sentence
Analysis
Data Mining Techniques
(Supervised +
Unsupervised)
Results
Overall
Review
Score
Calculation
24
Process 13

25. • Simplest task:
• Is the attitude of this text positive or
negative?
• More complex:
• Rank the attitude of this text from 1 to 5
• Advanced:
• Detect the target, source, or complex
attitude types
14 https://web.stanford.edu/~jurafsky/slp3/slides/7_Sent.pdf 25
SA : Tasks

26. • Easy: search the Web and find a Sentiment
Analysis tool and URLs
• http://www.twitalyzer.com/index.asp
• http://twendz.waggeneredstrom.com/
• http://www.sentiment140.com/
• http://www.blogmeter.it
• http://twitrratr.com/
• http://www.socialmention.com
• http://www.lovewillconquer.co.uk/
• Many more….
• And professional sites for companies
• www.radian6.com
• www.sysomos.com
26
Automated Analysis

27. • People express opinions in complex ways
• In opinion texts, lexical content alone can be misleading
• Intra-textual and sub-sentential reversals, negation, topic
change common
• Rhetorical devices/modes such as sarcasm, irony,
implication, etc.
• Unstructured and also non-grammatical
• Lexical Variation
• Out of Vocabulary Words
• Extensive usage of acronyms like asap, lol, afaik
15 https://web.stanford.edu/~jurafsky/slp3/slides/7_Sent.pdf 27
Text (Data) Mining Challenges

28. 28
At present there are two different
schools of thoughts on using social media
for analyzing public opinion:
• The Pro School– RAND Corporation
• The contra-group – PEW Research
Center
Pros & Cons of SA

29. Potential policy usage of computer based analysis
of social media:
• Make informed assessments of public opinion
• Forecast events, such as large-scale protests
• Assess the impact of political events or actions
on public opinion
• Inform outreach efforts to foreign populations
• Pinpoint intelligence gaps
16 https://www.rand.org/pubs/research_briefs/RB9685.html 29
Case Study – 1: A Case Study of Iranian
Public Opinion After the 2009 Elections 16

30. • The reaction on Twitter often differs a great
deal from public opinion as measured by
surveys
• Twitter May Be Different at Times
• Often it is the overall negativity that stands
out
• Reliability?
17 http://www.pewresearch.org/2013/03/04/twitter-reaction-to-events-often-at-odds-with-
overall-public-opinion/ 30
Case Study – 2: Twitter Reaction to Events
Often at Odds with Overall Public Opinion 17

31. Advantages
• Huge data is available for analysis
• Low cost than traditional surveys & polls
• Faster way of getting insight from user generated data
• New area for researchers: evolving in Pakistan
• Can be used in:
• Disaster Management
• Public health
• Assessing efficiency of services and infrastructure
• Measuring degree of citizens satisfaction
31
Conclusion

32. • Useful tool for SWOT Analysis of organizations and
Governments
• 80% of all data consists of words: the Sentiment Engine is
an essential tool for making sense of it all
• More accurate and insightful perceptions and feedback
32
Conclusion
Advantages

33. • Irony/sarcasm
• Slang
• Languages
• Geographical variations
• Text Analysis Problem
• Inference Problem
• Supervised Data Mining
33
Conclusion
Disadvantages

34. • How many people do have access to computer
• How many of them are using twitter
• Are we targeting a segment of society
• Is the sample representative of the society
• Often public opinion is contradictory
• Lower Taxes, More Government Action
• Also, the public may be uninformed or misinformed on
many topics
• Osama being involved in the 9/11 attacks
• Weapons of mass destruction in Iraq
34
Conclusion
Reliability

35. • Misinterpretation – If tweets are misunderstood and
then spread virally, this can create long-lasting negative
effects
• Hacks – Twitter feed can be hacked and used to spread
misinformation
• Use of BoTs - a social networking account powered by
artificial intelligence can change the results
• Opinion vary by demographics, social groups, sentiment,
topics, time
35
Conclusion
Reliability

36. • Develop a culture in the government to capitalize on ICT
systems
• Knowledge updation of public sector organizations
• Public policy formulation: Feedback
• Effective use for intelligence gathering
• can be a great vehicle to get closer to the citizens: Arab
Spring phenomenon
• Sentiment analysis as an alternative research technique
for collecting and analyzing textual data on the internet
especially opinion polls.
36
Recomendations

37. 1. P. Anderson, What is Web 2.0? Ideas, technologies and implications for education. JISC Technology and
Standards Watch (2007)
2. A. Bifet, E. Frank, Sentiment discovery in Twitter streaming data (2010)
3. A. Bifet, G. Holmes, R. Kirkby, B. Pfahringer, MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–
1604 (2010). http://moa.cs.waikato.ac.nz
4. Z. Bodnar, Generating small business customers with social media marketing: small business case studies
(2014), Hubspot http://www.HubSpot.com.
5. C. Burke, Twitter search tools (2011), http://charleneburke.com/2011/02/12-twitter-search-tools/
6. M. Cha, H. Haddadi, F. Benevenuto, K.P. Gummadi, Measuring user influence in Twitter: the million
follower fallacy, in Proceedings of the 4th International AAAI Conference on Weblogs and Social Media,
2010.
7. D. Dasgupta, R. Dasgupta, Social networks using Web 2.0. IBM Developer Works (2009),
http://www.ibm.com/deveoperWorks/
8. https://www.rand.org/pubs/research_briefs/RB9685.html
9. http://www.pewresearch.org/2013/03/04/twitter-reaction-to-events-often-at-odds-with-overall-public-
opinion
10. https://www.researchgate.net/publication/286841798_Sentiment_Analysis_for_Government_An_Opti
mized_Approach
11. N. Agarwal and H. Liu. Modeling and Data Mining in Blogosphere, vol- ume 1 of Synthesis Lectures on
Data Mining and Knowledge Discovery. Morgan and Claypool, 2009.
12. A. Anagnostopoulos, R. Kumar, and M. Mahdian. InÀuence and corre- lation in social networks. In
Proceeding of the 14th ACM SIGKDD in- ternational conference on Knowledge discovery and data
mining, pages 7–15, New York, NY, USA, 2008. ACM.
13. A. Go and L. Huang. Twitter sentiment analysis using supervision, in Proceedings of the Workshop on
Languages in Social Media 2011; 3-38.
14. https://www.sciencedirect.com/science/article/pii/S1877050916320130
15. Meena Rambocas, João Gama Marketing Research: The Role of Sentiment Analysis
References