Talk at IBM's Transparent Text Forum about the Berkman research project MediaCloud

4. #pman
32,107 posts
1979 authors
high RT #s
influencers

5. flickr photo by mhartford, cc

8. Nutritional Information: New York Times
Ingredients: International coverage 42%
(includes 8% Iraq, 5% Afghanistan, minimum
weekly 5% China, and no less than 2%
Africa), Washington coverage 28% (includes
7% Obama, 6% Congress, and trace
amounts of Limbaugh), New York State/
Albany coverage 14%, New York City
coverage 10%, and less than 6% domestic
US coverage.
Warning: contains less than 40% of sports
coverage of the leading competitor, the
New York Post, and 50% of business
coverage of the Wall Street Journal. May
contain less than your recommended daily
allowance of Latin America News.

9. You have not read any
international news today.

10. magazines
radio (npr, talk)
blogs (int’l, domestic, left, right)
?
newspapers
twitter
television (network, cable)
facebook
email forwards
websites

11. hand-coding
link analysis
automated content analysis

12. PEJ News coverage index

13. mediatenor.com

14. Hand-coding: traditional media monitoring
Upsides:
- High accuracy
- Flexibility
- Low startup cost
Downsides:
- Small data sets, problems extrapolating
- Time consuming, no real-time data
- Intercoder reliability, difficulty of coder setup

15. Adamic and Glance

17. Link analysis: Leveraging web architectures
Upsides:
- Highly automatable
- Large data sets
- Leverage existing tools for network research
Downsides:
- Only consider link structure, not content
- Danger of conflating linking with social structure
- Need for hand-coding to make sense of clusters
- Good for blogs, bad for MSM

18. newsmap.jp

21. Content analysis: Just becoming possible
Upsides:
- Can work with unstructured text, blogs and MSM
- Large data sets, highly automatable
- Easy linkage with visualization platforms
Downsides:
- Inaccuracy
- Language constraints
- Major programming investments

23. What We Have Done
1 3
get create
news term
stories list
2 4
extract allow
story rich
text queries

24. Terming
Lexicon-based simple matching More complex term extraction
Archer Daniels Midland Company
Archer, Bill
Archer, Dennis W.
Archer, Jeffrey
Archery
Archibold, Randal C.
Architecture
Architecure and Design
Archives and Records
Archon Corp.
Archstone-Smith Trust
ArcSight Inc.
Arctic Cat Inc.
Arctic Monkeys

25. - 9.25 million stories
- 900G of database + downloaded content
- 162 million story / tag associations
- 1,500 sources
- 10,000 feeds
- roughly 20,000 stories per day

26. Topic focus

27. Pivoting on “republican”

28. Global Attention and Power Laws

29. You say “stimulus” and I say “bailout”

34. what’s been hard:
topic clustering
replicating across languages
legal concerns
dark matter