Learn how to work with public, conversational, real-time data. This workshop will provide some perspective on data collection strategies using Twitter’s public APIs as the starting point (REST and Streaming).

1. Realtime Social Data
workshop @ startupfest
Sylvain Carle
Senior Developer Advocate
@froginthevalley

2. 2
Twitter is the
pulse of the planet

3. @Sylvain: from Startup to Twitter
3
• Startup CTO (a few times around)
• REST + Streaming API building Needium
• Didn’t work so well (that’s another story)
• The tech stack was pretty awesome!
• Modern version of our 2011 idea today
• Colin @ElasticSearch, Sylvain @Twitter
• In the end, not so bad!

4. Twitter Data APIs
4
Twitter APIs deal both with
the moment and the past

5. The pulse of the planet
5
• More than 241 million monthly active users
• 500 million Tweets are sent per day, or 1 billion every ~2 days
• 300+ billion Tweets have been sent since founding in 2006
• 76% of our active users are mobile users
• 40% of our active users simply consume content on Twitter
• Over 400 million monthly unique visitors to twitter.com
• Twitter supports 35 different languages
• 77% of Twitter accounts are outside the U.S.

6. Get started
6
!
Create you app: easy
t.co/apps
Do not re-invent the wheel: libs
t.co/libs
!

7. Create your App
7

8. Twitter Data APIs
8
The Past: REST API
t.co/rest
Optimize your calls, don’t get 429-ied
!
The Moment: Streaming API
t.co/streaming
1% Sample or Track Keywords
!

9. Getting started
9

10. Twurl
10
!
t.co/twurl

11. Twurl
11
!
t.co/console

12. Hosebird (HBC)
12
!
t.co/hbc

13. Getting started
13

14. Think with Data
14
• Focus on 2nd order effects
• Derive value from the content/network
• Catch emerging patterns: trends
• Listen to the chatter: conversations
• Flip it inside out: content from users
• Broader perspective: analytics
• Understand users: personalization

15. Think with Data
15
• Find one top level object to get started
• Users, Tweets, Timelines
• Map this concept to your domain knowledge
• Start with a measurable experience
• Moebius virtuous circle: fed and feed data
• Add your own layer of data for max value
• Analyze collections + context, not single tweets

16. Building you taxonomy
16
• Human Insights (domain knowledge)
• Data Insights
• Discovery mode (keyword inventory / baseline)
• Machine Learning (co-occurrences, clustering)
• Historical Indices
• Noise cancellation
• Detecting emerging trends

17. Taxonomy: Needium Locksmiths
17
• Human Insights (existing taxo + research)
• http://en.wikipedia.org/wiki/Lock
• Data Insights
• Discovery mode: swearing, types of issues
• Machine Learning: “locked up” “locked out”
• Historical Indices: evaluate opportunity by geo
• Noise cancellation: “lockin” “boat” “clock”
• Detecting emerging trends: seasonal, time of day

18. Other Examples
18
• Dataminr - finding emerging news
• Lithium - managing social at scale
• Sprinklr - social experience management
• Wayin - best brand stories by customers
• Inside - most important stories, for you
• More at t.co/partners

19. REST API 1.1 Resources
19
!
t.co/1.1

20. Streaming API Resources
20
!
t.co/1.1

21. Ramp up
21
• Start small, grow organically
• Get your core data in and out (+ management)
• Build your MVP Twitter integration
• Measure, experiment, learn
• Accelerate: rewrite, optimize, elevate
• Pedal to the metal: promoted, hoses

22. Twitter Data APIs
22
!
Ask us @TwitterDev
Ask in the forums
t.co/discuss
!

23. Twitter Data APIs
Streaming API t.co/streaming
REST API t.co/rest
Create App t.co/apps
Twurl t.co/twurl
Open Source Libs t.co/libs
Discussion forums t.co/discuss
Ask us @TwitterDev