SXSWi '11: Beyond Wordclouds: Analyzing Trends with Social Media APIs
1. Beyond Wordclouds Chris Busse Partner, Fahrenheit Technology Emerging Media Group Richmond, VA #RVA Work: FahrenheitTechnology.com Blog: ChrisBusse.com Twitter: @busse Analyzing Trends with Social Media APIs SXSW 2011 #beyondwc SXSW 2011 #beyondwc
2. This talk will cover: Some things I’ve learned from building systems to monitor online communications for brands An analysis of 100K Tweets from 48 hours of monitoring “coffee” on Twitter on Thursday and Friday of SXSW Q & A SXSW 2011 #beyondwc
3. API: Application Programming Interface An application programming interface (API) is a particular set of rules and specifications that a software program can follow to access and make use of the services and resources provided by another particular software program that implements that API. It serves as an interface between different software programs and facilitates their interaction, similar to the way the user interface facilitates interaction between humans and computers. http://en.wikipedia.org/wiki/Application_programming_interface SXSW 2011 #beyondwc
5. <entry> <id>tag:search.twitter.com,2005:47520097484550144</id> <published>2011-03-15T04:50:48Z</published> <link type="text/html" href="http://twitter.com/wingztw33t/statuses/47520097484550144" rel="alternate"/> <title>I'm at Sin Hoi How Coffee Shop (JalanTun H S Lee, Kuala Lumpur) http://4sq.com/ebaDo2</title> <content type="html">I&apos;m at Sin Hoi How <b>Coffee</b> Shop (JalanTun H S Lee, Kuala Lumpur) <ahref="http://4sq.com/ebaDo2">http://4sq.com/ebaDo2</a></content> <updated>2011-03-15T04:50:48Z</updated> <link type="image/png" href="http://a1.twimg.com/profile_images/658397775/zoom_resize_normal.jpg" rel="image"/> <twitter:geo> </twitter:geo> <twitter:metadata> <twitter:result_type>recent</twitter:result_type> </twitter:metadata> <twitter:source><ahref="http://foursquare.com" rel="nofollow">foursquare</a></twitter:source> <twitter:lang>id</twitter:lang> <author> <name>wingztw33t (TeikWingChoo)</name> <uri>http://twitter.com/wingztw33t</uri> </author> </entry> SXSW 2011 #beyondwc
6. APIs are the Bridge Between Systems API Customer Service API SXSW 2011 #beyondwc
7. APIs are the Bridge Between Systems API SXSW 2011 #beyondwc
8. A Problem I Faced Need to monitor a large volume of SM data around a brand, its products, and competitors Need to identifytrends in that data that might not be readily apparent Need to manage the communications activities of a team of online “Digital Advocates” Need to link this data to other internal software/ database systems (order processing, support center) SXSW 2011 #beyondwc
9. The Starting Point Wanted custom reporting, but commercially available tools didn’t seem like a good fit UI/UX frustrations Cost for large volume of data Extensibility / deep customization Felt there were limitations in the Twitter Search API Historical depth Query flexibility Needed to mix in more data (corporate mashups) I wanted control (choosing the wrong vendor partner jeopardizes my client relationships) I wanted to work with the development tools I’m familiar with SXSW 2011 #beyondwc
10. Buy vs Build?: Reasons to Build Standard question for “the Enterprise” Get data in to other sources, no silos Custom workflows Social CRM Start an archive now, you never know when you’ll need it Business Intelligence (BI) staff have a favored toolset Enterprise data security: SaaS might be a NoNo Enterprise platform standards: .NET, Java, LAMP, PowerPoint SXSW 2011 #beyondwc
11. Build to be minimally relianton platform APIs Get the data out and move on
12. <entry> <id>tag:search.twitter.com,2005:47520097484550144</id> <published>2011-03-15T04:50:48Z</published> <link type="text/html" href="http://twitter.com/wingztw33t/statuses/47520097484550144" rel="alternate"/> <title>I'm at Sin Hoi How Coffee Shop (JalanTun H S Lee, Kuala Lumpur) http://4sq.com/ebaDo2</title> <content type="html">I&apos;m at Sin Hoi How <b>Coffee</b> Shop (JalanTun H S Lee, Kuala Lumpur) <ahref="http://4sq.com/ebaDo2">http://4sq.com/ebaDo2</a></content> <updated>2011-03-15T04:50:48Z</updated> <link type="image/png" href="http://a1.twimg.com/profile_images/658397775/zoom_resize_normal.jpg" rel="image"/> <twitter:geo> </twitter:geo> <twitter:metadata> <twitter:result_type>recent</twitter:result_type> </twitter:metadata> <twitter:source><ahref="http://foursquare.com" rel="nofollow">foursquare</a></twitter:source> <twitter:lang>id</twitter:lang> <author> <name>wingztw33t (TeikWingChoo)</name> <uri>http://twitter.com/wingztw33t</uri> </author> </entry> SXSW 2011 #beyondwc
13. Data Fields Id Published Link (to Tweet) Title Content Updated Link (to Profile Image) Twitter:Geo Twitter:Metadata Twitter:Result Type Twitter:Source Twitter:Lang Author Name URI SXSW 2011 #beyondwc
34. What’s in a word? 140,295 words in 100K tweets
35. How to Turn Tweets in to Words Loop through all your collected Tweets Split() them on “ ‘ Strip out , . ! ? ( ) … and also “: “ Escape the single quotes Strip out/exclude common words like “the”, “for”, “you”, … Record the Word back to a database table Perhaps add counts to that word, and segment by a time span such as day or hour SXSW 2011 #beyondwc
36. General Word Clouds Lack Relevance Source:http://tweetcloud.com/search/term/coffee Source: custom script/db via Twitter Search API SXSW 2011 #beyondwc
42. Look at More Data(there’s an API for this too) SXSW 2011 #beyondwc
43. Other things to explore Better location-based analysis Words over time span, compared to previous time span Add the people who tweeted about this back in to the search strings, capture their tweets for analysis Find even more data to add in to the mix Begin to identify real influencers in the context of your strategic goals and objectives – this might not be as simple as a “score” SXSW 2011 #beyondwc
44.
45. Platform commitmentTraditional software package vendors working with platform/service APIs intelligently Smart people solving this problem for the Enterprise (and smaller orgs) in a way that fits strategy, not forcing strategy to fit a tool SXSW 2011 #beyondwc
46. Q & A Contact Info: Chris Busse @busse cbusse@fahrenheittechnology.com http://FahrenheitTechnology.com http://ChrisBusse.com Thank You: @jboogie& @mr_sterling @mavnet & @VCU_Brandcenter @cdharrison & @krusk & @akuzemchak @willowtree84 @IconBuffet The entire @FahrenheitTech / @FahrenheitEM Team All who helped promote this talk & all the SXSW staff & volunteers! SXSW 2011 #beyondwc