1. Instagram Big Data
Analytics
Creative Chaos
Amir Imani

2. TRANSFORM
DATA
COLLECT
STORE
INGEST
INSIGHT

3. DATA

4. Kat
900K follower
4-5 photo/day
20K likes/ 100 cmnts
Kevin
62K follower
1-2 photo/day
500 likes/ 5 cmnts
Scott
132K follower
4-5 photo/day
2000 likes/ 100 cmnts
Creative Chaos
2K follower
1-2 photo/week
65 likes/ 5 cmnts

5. Kat
850K follower
4-5 photo/day
20K likes/ 100 cmnts
Kevin
57K follower
1-2 photo/day
500 likes/ 5 cmnts
Scott
113K follower
4-5 photo/day
2000 likes/ 100 cmnts
Creative Chaos
2K follower
1-2 photo/week
60 likes/ 5 cmnts
# &

6. COLLECT

7. • Extract
• Filter
• Integrate
• Interactions: like & cmnts
• Augmentation: Gender/Language

8. {"hash":"da3b25df71bf16a847491533e468d359","hash_type":"stream","id":"25bbf43253ed4
576ed832d0038c316b4","delivered_at":"Tue, 03 May 2016 04:03:03 +0000","interactions":
[{"demographic":{"gender":"female"},"instagram":{"from":
{"id":"1765062192","username":"fatemehkavandi","full_name":"Fatemeh
Kavandi","profile_picture":"https://scontent.cdninstagram.com/t51.2885-19/
s150x150/12501620_208060992908600_18549704_a.jpg"},"type":"like","media":
{"id":"1241673596494888825_21515974","created_time":
1462239049,"username":"kat_in_nyc","link":"https://www.instagram.com/p/
BE7T46uDz95/","type":"image","caption":"my love @sashakichigina
❤ ","filter":"Normal","image":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/
sh0.08/e35/13098947_539207262947852_1918000948_n.jpg?
ig_cache_key=MTI0MTY3MzU5NjQ5NDg4ODgyNQ%3D%3D.2.l"}},"interaction":
{"id":"1e610ceaa6daaa80e019000040de3cfb","type":"instagram","subtype":"like","content":"f
atemehkavandi likes kat_in_nyc's image","author":{"avatar":"https://
scontent.cdninstagram.com/t51.2885-19/
s150x150/12501620_208060992908600_18549704_a.jpg","id":"1765062192","link":"http://
instagram.com/fatemehkavandi","name":"Fatemeh
Kavandi","username":"fatemehkavandi"},"link":"https://www.instagram.com/p/
BE7T46uDz95/","created_at":"Tue, 03 May 2016 03:58:07 +0000","received_at":
1462247887.3020000},"language":{"tag":"ms","tag_extended":"ms","confidence":
97},"source":{"id":"c9b92d83ee02431bb3e3c263efeef962"}},{"demographic":
{"gender":"male"},"instagram":{"from":
{"id":"472927476","username":"ildefonsod","full_name":"Ildefonso
Domínguez","profile_picture":"https://scontent.cdninstagram.com/
t51.2885-19/10540331_303893719790855_791725810_a.jpg"},"type":"like","media":
{"id":"1241539081550118008_21515974","created_time":
1462223013,"username":"kat_in_nyc","link":"https://www.instagram.com/p/
BE61Td6jzx4/","type":"image","caption":"@ksyander ❤ ","filter":"Normal","image":"https://
scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/
e35/13108840_1590136574634198_742652310_n.jpg?
ig_cache_key=MTI0MTUzOTA4MTU1MDExODAwOA%3D%3D.2.l"}},

9. STORE

10. Scalable Cloud-Storage
Full lifecycle support
Reliable and CheapS3
{
{JSON}

11. Scalable Cloud-Storage
Full lifecycle support
Reliable and CheapS3
{
{JSON}

12. INGEST

13. Fast and general engine for big data processing
Open source cluster computing framework
built-in modules for streaming, SQL
{

14. Fast and general engine for big data processing
Open source cluster computing framework
built-in modules for streaming, SQL
{

15. TRANSFORM

16. Python 3.5
Widely used high-level, dynamic language
Open source with variety of libraries
Online support community
{

20. INSIGHT

21. Build a dashboard for:
Followers analysis (top likers, etc)
Content analysis (top liked, etc)
Historic data comparison
{

22. Gender breakdown Gender breakdown/user
Filters
Overview of likes for each photo

23. overall Life span
Average Life span Average interaction

24. Filters
Followers rankingLanguage

25. • DataSift lacks: follower counts, accurate gender/language, accurate
interaction time
• It is expensive (at least $20,000 per year just for the infrastructure)
• It’s new API sucks!
{"hash":"5765b1d967610b0b5089a399a81bef59","hash_type":"stream","id":"8b53e742335c
0dabc9dcb6ad5c840798","delivered_at":"Wed, 29 Jun 2016 14:45:51
+0000","interactions":[{"instagram":{"count":39,"type":"comment_count","delta":
0,"media":{"username":"kat_in_nyc","filter":"Normal","type":"image","from":
{"fullName":"kat irlin","username":"kat_in_nyc","profile_picture":"https://
scontent.cdninstagram.com/
t51.2885-19/11906380_1642296532707452_1269120372_a.jpg","id":"21515974"},"link":"h
ttps://www.instagram.com/p/BHOGzD9DCTk/","content":"brooklyn fog with
@sashakichigina â¤ï¸â¤ï¸â¤ï¸","created_time":"Wed, 29 Jun 2016 01:44:46
+0000","caption":"brooklyn fog with @sashakichigina â¤ï¸â¤ï¸â
¤ï¸","image":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/
e35/13534471_286775051671508_916687424_n.jpg?
ig_cache_key=MTI4Mjk5Mjg0MTA2NDk4MTczMg%3D%3D.2","tags":
[],"id":"1282992841064981732_21515974"}}