A review of Eysenbach, G., 2011. Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), p.e12

1. Digital Enterprise Research Institute www.deri.ie
Twitter and research impact
Marie Boran
Copyright 2011 Digital Enterprise Research Institute. All rights reserved.
Enabling networked knowledge

2. Digital Enterprise Research Institute www.deri.ie
 A review of: Eysenbach, G., 2011. Can Tweets Predict
Citations? Metrics of Social Impact Based on Twitter and
Correlation with Traditional Metrics of Scientific Impact.
Journal of Medical Internet Research, 13(4), p.e12.
Enabling networked knowledge
2

3. A little background…
Digital Enterprise Research Institute www.deri.ie
Impact Factor as a
measure of scientific
impact:
The Good, the Bad and the
Ugly.
Enabling networked knowledge
3

4. Sick of Impact Factor?
Digital Enterprise Research Institute www.deri.ie
 Imperial College London researcher Stephen Curry: „the
stupid, it burns.”
 http://occamstypewriter.org/scurry/2012/08/13/sick-of-
impact-factors/
 “dependency on a valuation system that is
grounded in falsity.”
 “we need to find ways to attach to each piece of work the
value that the scientific community places on it though
use and citation.”
Enabling networked knowledge
4

5. What are altmetrics?
Digital Enterprise Research Institute www.deri.ie
 Alternative web-based social metrics
 Scientometrics from online social activity centred around
scholar‟s work
 Self-publishing: blogging, uploading, tweeting, sharing
 Impact measured via: articles viewed, shared, downloaded,
„retweeted‟, „liked‟, etc.
“Scholars are moving their everyday work to the web.
Online reference managers Zotero and Mendeley each
claim to store over 40 million articles (making them
substantially larger than PubMed); as many as a third of
scholars are on Twitter, and a growing number tend
scholarly blogs. These new forms reflect and transmit
scholarly impact […] That hallway conversation about a
recent finding has moved to blogs and social networks–
now, we can listen in.
- Altmetrics.org manifesto A ”
From: altmetrics.org/manifesto
Enabling networked knowledge
5

6. Eysenbach (2011)
Digital Enterprise Research Institute www.deri.ie
 Study objectives:
 Feasibility of measuring social
impact/public attention to scholarly
articles through social media
 Relation between dynamics, timing
of tweets about a scholarly article
(aka tweetations) and journal
citations
 Evaluating accuracy of resulting
metrics in predicting highly cited
articles
Journal of Medical Internet Research top articles, ranked by
tweets
Enabling networked knowledge
6

7. Methods
Digital Enterprise Research Institute www.deri.ie
 Journal of Medical Internet Research
 Highly-cited, open access journal
 Articles published between issues 3/2009 and 2/2010
 Thomson Reuters 3-year impact factor of 4.7
 Citation counts (SCOPUS and Google Scholar)
 Twitter citations or „tweetation” – must mention journal article
URL
 Only tweets with URLs linking directly to the journal article are
captured. Does not count links to blogs or newspaper articles
mentioning research.
Note: Eysenbach is the editor-in-chief and publisher of JMIR
Enabling networked knowledge
7

8. Methods (cont‟d)
Digital Enterprise Research Institute www.deri.ie
 Tweets captured: all sent and archived
by JMIR between July 24, 2008 and
November 20, 2011
 Classification: “highly-cited” articles -
top 25th percentile of each issue (by
citation counts)
 “highly-tweeted” - top 25th percentile
(ranked by tweetations)
 Adjusted for increasing popularity of
Twitter over time & older articles have
higher citations.
Enabling networked knowledge
8

9. Results
Digital Enterprise Research Institute www.deri.ie
 55 articles
 4208 tweetations
 Average 14 tweetations per article
 Majority of tweets published on or day
after article published (see graph)
 First 30 days: “network propagation
phase”
 30+: “sporadic tweetation phase”
 Observed 80/20 rule (Pareto principle)
 Highly tweeted articles 11 times more
likely to be highly cited than less-tweeted
articles
 75% of highly tweeted articles were
highly cited in comparison to 7% of less-
tweeted articles
Enabling networked knowledge
9

10. Results (cont‟d)
Digital Enterprise Research Institute www.deri.ie
 Citation and tweetation patterns
 Scopus and Google Scholar citations tested for agreement
 Eysenbach observed “distribution […] typically observed for citations”
Enabling networked knowledge
10

11. Findings
Digital Enterprise Research Institute www.deri.ie
 First systematic, prospective, longitudinal article and journal-level
investigation of how mention (citations or tweetations) of scholarly
articles in social media accumulate over time
 First study correlating altmetrics to citations
 Online buzz around articles is measurable
 Tweets are “surprisingly accurate” predictors of future journal
citations
Enabling networked knowledge
11

12. Limitations
Digital Enterprise Research Institute www.deri.ie
Via Scienceblogs.com
 Complementary, *not* a replacement for Impact Factor
 “Tweetations” as buzz, attentiveness, social impact
Enabling networked knowledge
12

13. Conclusions
Digital Enterprise Research Institute www.deri.ie
 Proposes “twimpact factor” (twn) as metric of impact in social media, where
n is cumulative number of tweetations within n days after publication
 “The cumulative number of tweetations by day 7 (perhaps as early as day
3), could be used as a diagnostic test to predict highly cited articles.”
 Tweetations as proxies for social impact of scientific research
 Can be applied to other social media and non-scholarly articles to measure
issue impact on social media user population
+ =
Twitter + metrics = wider perspective on research impact
Enabling networked knowledge
13

14. Related research
Digital Enterprise Research Institute www.deri.ie
• Priem, J. & Costello, K.L., 2010. How and
why scholars cite on Twitter. Proceedings
of the 73rd ASIST Annual Meeting, 47(1),
p.1-4.
• Priem, J. & Hemminger, B.M., 2010.
Scientometrics 2.0: Toward new metrics of
scholarly impact on the social Web. First
Monday, 15(7)
Enabling networked knowledge
14

15. Happy Christmas!
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge
15