Talk given at the ESSENCE International Conference on Computational Approaches to Diversity in Interaction and Meaning, October 2017, San Servolo

1. Adapting NLP tools to diverse data:
challenges and solutions
Dr. Diana Maynard
University of Sheffield, UK
October 2017, San Servolo, Italy

2. Twitter Fun Facts
• 500 million tweets sent per day
• 24% of all internet male users use Twitter (vs 21% of
women)
• 37% of Twitter users are 18-29
• 25% of Twitter users are 30-49

3. Which country has the most Twitter
users?

4. Twitter Users per country
• US: 67 million
• Brazil: 27.7 million
• Japan: 25.9 million
• Mexico: 23.5 million
....
• UK: 13 million

5. Which country has the highest
penetration of Twitter users?

7. 1/3 of all
internet
users there
are on
Twitter

8. Who do we follow on Twitter?

9. Top 10 most followed Twitter users
2017 2015 2013
Katy Perry Katy Perry Katy Perry
Justin Bieber Justin Bieber Justin Bieber
Barack Obama Taylor Swift Lady Gaga
Taylor Swift Barack Obama Barack Obama
Rihanna Youtube Taylor Swift
Ellen de Generes Lady Gaga YouTube
Lady Gaga Rihanna Britney Spears
Youtube Ellen de Generes Rihanna
Justin Timberlake Twitter Instagram
Twitter Justin Timberlake Justin Timberlake

10. Social media: a valuable source of information
l business insights
l sharing and receiving important news
l campaigns
l all kinds of collective intelligence
l an alternative to traditional polls
l and much more

11. Uses of social media during disasters
• Broadcasting info about the disaster
• Requesting info from local people and
eyewitnesses
• Requesting and offering help and
support
• Disaster mapping
• Mobilising the crowd to support
initiatives

12. 1
2
Ecuador, 7.8 earthquake , April 2017, ~700 people
die
Droughts, affecting 60 million in 34 countries
Maxwell, California, Feb 2017
Portugal, forest fires, 64 confirmed deaths, Jun 2017
Manchester, May 2017, 22 dead
Haiti, Hurricane Matthew, Oct 2016,
~500 people died, farming devastated

13. 1
3

14. Harnessing the Crowd
• Using citizen
reporters, and digital
responders for
mapping crises
• Ushahidi deployed
over 50k times
• Free and open source
• Working with us on
the COMRADES
project

16. Why is social media interesting for NLP?
l Fast-growing, highly dynamic and high volume
source of data
l Reflects language used in today's society
l Reflects current views of society
l It's a great source of material for opinion mining
tools
l Challenging research area due to specialised use of
language

17. Gartner 3V definition of Big Data
• Volume
• Velocity
• Variety
• High volume & velocity of messages:
• 500 million tweets per day
• Massive variety:
• Stock markets
• Earthquakes
• Social arrangements

18. Big Data is not new!
Staff sorting 4M used tickets from #London Underground
to analyse line use in 1939

19. Problems and solutions for NLP
• Volume and velocity can largely be dealt with using
modern processing methods
• GATE social media toolkit allows (almost) real-time
collection, analysis and visualisation of a twitter stream,
e.g. studies of Brexit tweets, UK elections
• But still issues with data collection (e.g. Twitter rate
limits)
• Variety and veracity are rather more complex.

20. Social Media sites are not all the same
l Twitter has varied uptake per country:
• Low in China (often censored, local competitor – Weibo)
• Low in Denmark, Germany (Facebook is preferred)
• Medium in UK, though often complementary to Facebook
• High in USA
l Networks have common themes:
• Individuals as nodes in a common graph
• Relations between people
• Sharing and privacy restrictions
• No curation of content
• Multimedia posting and re-posting

21. Challenges of social media
l Strongly temporal and dynamic:
l Temporal information (e.g. post timestamp) can be combined
with opinion mining, to examine the volatility of attitudes
towards topics over time (e.g. gay marriage)
l Exploiting social context: who is the user connected to? How
frequently do they interact?
l Derive automatically semantic models of social networks,
measure user authority, cluster similar users into groups, as well
as model trust and strength of connection
l Implicit information about the user: research on recognising
gender, location, and age of Twitter users.
l Helpful for generating opinion summaries by user demographics

22. Linguistic challenges of social media
• Language
• Problem: typically exhibits very different language style
• Solution: train specific language processing components
• Relevance
• Problem: topics and comments can rapidly diverge.
• Solution: train a classifier or use clustering techniques
• Lack of context
• Problem: hard to disambiguate entities
• Solution: data aggregation, metadata, entity linking

23. People don’t write “properly”
l Grundman:politics makes #climatechange scientific issue,people
don’t like knowitall rational voice tellin em wat 2do
l Want to solve the problem of #ClimateChange? Just #vote for a
#politician! Poof! Problem gone! #sarcasm #TVP #99%
l Human Caused #ClimateChange is a Monumental Scam!
http://www.youtube.com/watch?v=LiX792kNQeE … F**k yes!!
Lying to us like MOFO's Tax The Air We Breath! F**k Them!
l The last people I will listen2 about guns r those that know
nothing about them&politicians who live in states w/strictest
gun laws #cali #ny

24. NLP Pre-Processing Pipeline
Language ID
TokenisationPOS tagging
Text

25. Named Entity Recognition and Linking
NER (Professor Plum)
dbpedia.org/resource/.....
Michael_Jackson
Michael_Jackson_(writer)
Entity Linking

26. Pipelines for tweets
Errors have a cumulative effect
Good performance is important at each stage
Per-stage
Overall

27. Language Identification
LADY GAGA IS BETTER THE 5th TIME OH BABY(:
je bent Jacques cousteau niet die een nieuwe soort heeft
ontdekt, het is duidelijk, ze bedekken hun gezicht. Get
over it
I'm at 地铁望京站 Subway Wangjing (Beijing)
http://t.co/KxHzYm00
The Jan. 21 show started with the unveiling of an
impressive three-story castle from which Gaga emerges.
The band members were in various portals, separated
from each other for most of the show. For the next 2
hours and 15 minutes, Lady Gaga repeatedly stormed
the moveable castle, turning it into her own gothic
Barbie Dreamhouse .
Newswire
Twitter

28. Tokenisation is only 80% accurate on tweets
Improper grammar, e.g. apostrophe usage:
doesn't → does n't
doesnt → doesn’t
Smileys and emoticons: loss of information (e.g. sentiment)
I <3 you
This piece ;,,( so emotional
Punctuation for emphasis
*HUGS YOU**KISSES YOU* → * HUGS YOU**KISSES YOU *
Words run together / skip
I wonde rif Tsubasa is okay..

29. We need tools for hashtag analysis
l Hashtags need unravelling and disambiguating:
l #nowthatcherisdead
l #powergenitalia
l #lesbocages
l #molestationnursery
l #teacherstalking
l #therapist

30. Test your social media skills
What do these hashtags mean?
l #kktny
l #fomo
l #jomo
l #ootd
l #wcw

31. Hashtag Hijacking

32. Have sex to save the planet!

34. Tweet Normalisation
l “RT @Bthompson WRITEZ: @libbyabrego honored?! Everybody
knows the libster is nice with it...lol...(thankkkks a bunch;))”
l OMG! I’m so guilty!!! Sprained biibii’s leg! ARGHHHHHH!!!!!!
l For some components to work well (POS tagger, parser), we need
normalisation
l BUT uppercasing, and repetition often convey strong sentiment
l Other forms of “misspelling” might indicate information about the
author
l First challenge: separate out-of-vocabulary and in-vocabulary
l Second challenge: fix mis-spelled IV words (e.g. Levenshtein edit
distance)
l The ZOMG phenomenon

35. Part-of-speech tagging
• Similar issues as for normalisation – we don’t have big datasets
to train on
• Label unlabelled data with multiple taggers and accept tweets
where tagger votes never conflict
• Model words using Brown clustering and word representations
(Turian 2010)
2m, 2ma, 2mar, 2mara, 2maro, 2marrow, 2mor, 2mora, 2moro, 2morow,
2morr, 2morro, 2morrow, 2moz, 2mr, 2mro, 2mrrw, 2mrw, 2mw, tmmrw, tmo,
tmoro, tmorrow, tmoz, tmr, tmro, tmrow, tmrrow, tmrrw, tmrw, tmrww, tmw,
tomaro, tomarow, tomarro, tomarrow, tomm, tommarow, tommarrow,
tommoro, tommorow, tommorrow, tommorw, tommrow, tomo, tomolo,
tomoro, tomorow, tomorro, tomorrw, tomoz, tomrw, tomz

36. NER on Tweets
l NER on Tweets much harder than on longer text
l Very short, so ambiguous terms hard to interpret
l Poor grammar and spelling, use of abbreviations,
shorthands
l Twitter-specific features: hashtags, @mentions, etc.
l Tools designed for longer texts do very badly on
Twitter
l As low as 40% accuracy of traditional NER taggers

37. Lack of context causes ambiguity
Branching out from Lincoln park after dark ... Hello
Russian Navy, it's like the same thing but with glitter!
??

40. It’s all about the Named Entities
Branching out from Lincoln park after dark ... Hello
Russian Navy, it's like the same thing but with glitter!

41. Some cool applications with GATE

42. Real-time Opinion Monitoring
42
vs replies

43. Climate change, ISIS and Trump
43

44. Querying election data with MIMIR
• Dataset: every
tweet by MP /
Candidate / Party,
plus all
replies/retweets
• Find all tweets
where a
Conservative MP
talked about the
economy

45. Parties / themes co-occurrence
UK economy
Europe
Tax and revenue
NHS
Borders and Immigration
Scotland
Employment
Community and society
Public health
Media and communications
LabourPartyCandidate
LabourPartyMP
ConservativePartyCandidate
ConservativePartyMP
UKIPCandidate
OtherMP
SNPCandidate
GreenPartyCandidate
LiberalDemocratsCandidate
SNPOther

46. Behaviour Analysis
• Based on the assumption that users in different behavioural
stages communicate differently (different emotions,
directives, etc.)
Pajarito @lindopajarito . 2h
Our building needs 40% of all energy consumed in Switzerland! L
DJPajarito @DJPajaritoGenial . 12h
I'm so proud when I remember to save energy and I know
however small it's helping.
Desirability: Negative sentiment (expressing personal
frustration- anger/sadness)
Buzz: Positive sentiment (happiness/joy). I/we + present tense
HotelPajarito @HotelPajarito . 18h
Join us today today to switch of a light for EH! J
Invitation: Positive sentiment (happy) + use of vocatives

47. Hate speech towards MPs on Twitter
20172015
http://greenwoodma.servehttp.com/data/buzzfeed/sunburst.html

48. Some useful links
• GATE http://gate.ac.uk
• GateCloud https://cloud.gate.ac.uk
• Come on a GATE training course!
• Brexit study blog post from NESTA http://www.nesta.org.uk/blog/network-
analysis-top-eu-referendum-tweeters
• Brexit study blog posts from Sheffield
http://gate4ugc.blogspot.co.uk/search/label/Brexit
• UK elections monitor http://gate.ac.uk/projects/pft
• Blog post on abuse of MPs
https://www.buzzfeed.com/tomphillips/twitter-abuse-of-mps-during-the-
election-doubled-after-the
• COMRADES project on social media during disasters
http://gate.ac.uk/projects/comrades

49. Publications
• Diana Maynard, Ian Roberts, Mark A. Greenwood, Dominic Rout and Kalina
Bontcheva. A Framework for Real-time Semantic Social Media Analysis. Web
Semantics: Science, Services and Agents on the World Wide Web, 2017.
• K. Bontcheva, L. Derczynski, A. Funk, M.A. Greenwood, D. Maynard, N. Aswani.
TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text.
Proceedings of the International Conference on Recent Advances in Natural
Language Processing (RANLP 2013).
• D. Maynard and K. Bontcheva. Understanding climate change tweets: an open
source toolkit for social media analysis. In Proc. of EnviroInfo 2015,
Copenhagen, Sep. 2015
• Diana Maynard, Kalina Bontcheva, Isabelle Augenstein. Natural Language
Processing for the Semantic Web. Morgan and Claypool, December 2016. ISBN:
9781627059091 (contains a chapter on social media analysis)
• M. Fernandez, L. Piccolo, D. Maynard, M. Wippoo, C. Meili, H. Alani. Pro-
Environmental Campaigns via Social Media: Analysing Awareness and
Behaviour Patterns. To appear in Journal of Web Science 2017.
• Available (and more) at https://gate.ac.uk/gate/doc/papers.html

50. Acknowledgements
This work supported by:
• the European Union/EU under the Information and
Communication Technologies (ICT) theme of the 7th
Framework and H2020 Programmes for R&D
• DecarboNet (610829) http://www.decarbonet.eu
● SoBigData (654024) http://www.sobigdata.eu
● COMRADES (687847) http://www.comrades-project.eu
● Nesta http://nesta.org.uk