1. Dissertation proposal defense Xiaoju Zheng June 9, 2010 Life Cycle of #hashtags: “words” in the #Twittertopia 1

2. Road Map of the Presentation Background of research questions Research Questions Overview of the data Follow-up Experiments Diffusion models Challenges 2

3. The Laws of Imitation “why, given one hundred different innovations conceived of at the same time – innovations in the form of words, in mythological ideas, in industrial processes, etc. – ten will spread abroad, while ninety will be forgotten.” --- Gabriel Tarde (1903) “The Laws of Imitation” Merit of its own? Or something else? 3

4. The Laws of Imitation “……..we see that the incessant struggle between minor linguistic inventions which always ends in the imitation of one of them, and in the abortion of the others, finally comes to transform a language in such a way as to adapt it, more or less rapidly and completely, according to the spirit of the community, to external realities and to the social purposes of language. ..” --- Gabriel Tarde (1903) “The Laws of Imitation” 4

5. The Laws of Imitation Merit is not the only catalyst of the spread of an idea. In situations where “the poorest innovations, from the point of view of logic, are selected because of their place, or even date of birth.”, Tarde attributes these irrational occurrences to “extra-logical influences” 5

6. Social Network Analysis Current research in social network analysis asserts that these “extra-logical” influences can be explained by examining the dynamics of the network through which influence is transmitted between individuals. In other words, if we view individuals as nodes in a social network, where a directed edge indicates that one node influences another, then some graph configurations make it more likely that an innovation will be widely adopted than others. 6

7. Basic Research Questions How is a word created? What makes a newly-created word better than others? How is a newly-created word picked up by users at large? How does a word gain popularity among the population? In a word, the life cycle of a word. 7

8. Research Question --- Data Word creation in EnglishIn spoken English, it can take decades – even centuries – for new words to emerge, become part of common parlance, and then fade into disuse. Word creation on Twitter,a word in the form of #hashtags can live the entire lifecycle in very short period of time, e.g. a couple of days A news story breaks, and competing hashtags vie for dominance. Then a few influential people adopt the same one. Suddenly the conversation coalesces around it, the term trends, the spammers start using it, and then the conversation peters out as we move on to the next topic. (only one possibility) Is that the pattern? And how closely does it map onto the ways that words and phrases emerge in spoken language? #hashtag – word on twitter 8

9. Twitter Twitter.com:Twitter is a social networking and micro-blogging service that enables its users to send and read messages known as tweets. Tweets are text-based posts of up to 140 characters displayed on the author's profile page and delivered to the author's subscribers who are known as followers. 9

10. Twitter: some conventions 10 @mentions - following word is the name of a twitter user and as such this tweet refers to that user, e.g. ”@dave thanks for the help” or ”Talking with @paul about twitter”. (can be used to spot smaller network) Retweets -”RT” means ”I am retweeting (copying) something from elsewhere”, e.g. ”RT@john I just saw Madonna” means that I am retweeting theoriginal message from John (can be used to spot smaller network) #hashtags –give contextual relevance to a tweet or identified as a keyword, e.g. ”Like this demo #acita09” or ”Why does #ms-word keep crashing”

11. Top Influential People on Twitter the Edinburgh Twitter Corpus (around 2 billion tokens) 11 Six Singers

12. Top 10 trending topic from the Edinburgh Twitter Corpus (around 2 billion tokens) 12

13. 13

14. Research Question Word creation and its propsperitywhat count as criteria for a newly-coined “word” to be accepted as a good #hashtag and how a good #hashtag gain popularity among groups of people. Logical: linguistic groundings of a good #hashtagLinguistic analysis of the #hashtags and behavioral studies Extra-logical: social groundings of a popular #hashtage.g. network structure and dynamics 14

15. #Hashtag 15

16. I. Linguistic Analysis of #hashtagsmore at https://docs.google.com/Doc?docid=0AWbvIzcQLhQXZGdoY256cDJfMTExZjZjbjNxbTM&hl=en Linking words into a sentence: e.g. whatsyourbackground, tweetwhatyoueatLetsMakeATrendingTopic,goodluckjustin Part of word + existing word: e.g. animtip,appstore Compounding: noun + noun e.g. sundayhug, pubquiz, waikikilunch Compounding: verb + noun e.g. hashtagme, pickon,killcapscop Compounding: adv + verb e.g. currentlycrushing Compounding: adj + noun e.g. digitalbritan, GoodTimes, morningsickness Splinter: e.g. socialmem (SocialCamp Memphis), Acronym & Initials: e.g. smlb (St Michael Le Belfrey church), emr (electronic medical records), #eu (European Union), #cah (Crimes against humanity) Neologism (splinter involved): e.g. twacker(twitter users who lose user account), tweetie (Twitter client for Mac and iPhone), twitvorce(to divorce yourself from a Twitter member by unfollowing them), twittertopia, twendsetter MISC: omgfact, #tcot (top conversation on twitter) 16

17. Preliminary Analysis Public timeline: 20 tweets per minute 20 days of non-stop crawling Total tweets = 567,091 Total words = 8,495,323 Average words per tweet = 14.98 NPS Chat Corpus: 45010 tokens/6,066 types Webtext corpus in NLTK: 396,736 tokens/21,537 types 17

18. Top 10 Frequent words 18

19. Top 10 Frequent words 19

20. Top 10 Frequent words 20

21. Twitter presents a different genre of texts Self expression: "I" is the top-ranking word that tweets begin with. Stats update: "Watching", "trying", "listening", "reading" and "eating" are all in the Top 100 first words, revealing just how often people use Twitter to report on whatever they are experiencing at the time. News broadcast: The abbreviation "RT" (retweet) is extremely common 21

22. Twitter presents a different genre of text popular web addresses (e.g. URL shortening service) among the top 500: "tinyurl.com", "twitpic.com", "ff.im", "twurl.nl". These all appear because they offer services useful to twitterers. Tech vocabulary: among top 500: "Google”, “Faceobok”:, “internet”, “website”, “blog”, “Mac”, and “app”. popular web addresses (e.g. URL shortening service) among the top 500: "tinyurl.com", "twitpic.com", "ff.im", "twurl.nl". These all appear because they offer services useful to twitterers. Tech vocabulary: among top 500: "Google”, “Faceobok”:, “internet”, “website”, “blog”, “Mac”, and “app”. 22

23. Research Question 23 linguistic groundings of a good #hashtagLinguistic analysis of the #hashtags and behavioral studies social groundings of a popular #hashtage.g. network structure and dynamics Would linguistically equally good #hashtags have different degrees of popularity? Is it because of the different network structure? Behavioral studies to get quantitative measurement about linguistic goodness of #hashtags.

24. Linguistic Grounding 24 Question 1: Does the tag length distribution of adopted #hashtag demonstrate a different distribution from words? Does it conform to a power law distribution or a lognormal distribution? Do #hashtags of different length receive different goodness judgement (e.g. are extremely short tags better than extremely short words?)

25. Linguistic Grounding 25 Question 2: What are the linguistics processes of creating a #hashtag? What count as a good #hashtag (morphologically, phonotactically, and semantically)? A more qualitative analysis of the #hashtags needs to be done to design a metrics of analysis: e.g. compounding, splinter (of what kind)

26. Linguistic Grounding – behavioral experiements 26 Word vs. Nonword: Subjects will be presented with #hashtags collected from twitter.com, and asked to label them as either word or nonword. Come up with specific criterion for word vs. nonword Morpheme identification: based on the results obtained from the Word vs. Nonword experiment, #hashtags will be presented for subjects to divide them into morphemes and identify meaningful subparts.

27. Linguistic Grounding – behavioral experiements 27 Semantic transparency:word association game: for hashtags like “twitvorce”, subjects will be asked to provide free word associations. For instance, subjects are likely to provide “twitter” and “divorce” for the “twitvorce”.

28. 28 Goodness rating: general: for both #hashtags, that are “nonwords”, subjects will provide subjective goodness ratings, e.g. on a scale from 1 to 7. phonotactic: subjects rate the pronouncability, e.g. for acronyms and initials.For instance, some acronyms are just strings of consonants without vowels, some are strings of vowels, and others are mixture of consonants and vowels. Would more pronouncable #hashtags be perceived as better #hashtags?

29. Linguistic Grounding – statistical parser 29 Phonotactic likelihood: Develop a statistical parser (e.g. finite state machine) for #hashtags and words, and compare the phonotactic probability. Also compare the statistical parser with e.g. Vitevich (2004) model.

30. Social Grounding 30 Based on the realistic data from twitter, diffusion models can be tested. Diffusion models:Linear Threshold ModelCascade Model

31. The Threshold model Threshold Model.It says that people adopt a new behavior because a sufficiently large proportion of their friends have adopted that behavior. E.g. Early adopters have a very low threshold, say 5% or 10%, while late adopters would have a much higher threshold. Every person, however, has their own individual threshold. The key variable here is the initial distribution of thresholds across a social network, which describes in totality the final extent of the behavior. But this model says nothing about how people initially adopt behavior. That is, it says nothing about innovators or the things that are being invented, only about the spread of innovation through a social network. 31

32. The Threshold model 32 In the threshold model every person u has a threshold :and each of their neighbors v is weighted according to: W u,v.If then the person u adopts the behavior. The set of thresholds, weights, and initial adopters determines the extent of the behavior in the social network.

33. The Cascade Model Cascade Modelevery person has a chance of adopting a new behavior whenever one of their neighbors adopts it. The probability that a person adopts the new behavior is the conversion rate for the notification. This probability is both a function of the sender and the recipient, so more influential people are more likely to convince others to adopt a behavior. 33

34. The Cascade Model 34 In the cascade model, for every person u and neighbor v there is a random variable X u,v which describes the likelihood of u adopting the behavior if v has adopted it.

35. Diffusion Model 35 Threshold model: neighborhood densityadopt if enough friends do so.  Cascade Model: function of the sender and receiverpeople have a chance of doing something if one of their friends is doing it.

36. Several Challenges at this step 36 Design a metrics for #hashtag classification ( e.g. p. 16): position of #hashtag, functions, word structure. Different #hashtag may have different adoption patterns and diffusion patterns. Quantitative measurement of “success” of a #hashtag: by frequency of mentioning, logevity (within a short or long time frame) Design a way to find competing, equally good #hashtags Representative sample

37. Twitter Network: spot the right network Despite having large networks, a smaller circle is maintained: for users with a high number of followers, they actually only still communicate with a smaller subset of users. Where’s the value? Within the hidden network: find out the true influence model of who people really trust above all other users by looking at actual “@” behavior and follow behavior. 37

38. 38 http://www.hpl.hp.com/research/scl/papers/socialmedia/socialmedia.pdf

39. 39