About one-fifth of current scientific papers are being shared on Twitter. With 230 million active users and 24 percent of the U.S. online population using the microblogging platform, hopes are high that tweets mentioning scientific articles reflect some type of interest by the general public and might even be able to measure the societal impact of research. However, early studies show that most of the engagement with scientific papers on Twitter takes place among members of academia and thus reflects visibility within the scientific community rather than impact on society. At the same time, some tweets do not involve any human engagement but rather are generated automatically by Twitter bots. This talk focuses on identifying audiences on Twitter and teaches participants how to collect, analyze, visualize, and interpret diffusion patterns of scientific articles on Twitter. The course provides an overview of Altmetrics research and present the challenges – including methods and first results – of classifying Twitter user groups, with a particular focus on identifying members of the general public and measuring societal impact. The course will provide hands-on exercises and instructions on how to analyze by whom, when, and how scientific papers are shared on Twitter. Speaker: Stefanie Haustein, Ph.D., Assistant Professor, School of Information Studies, University of Ottawa

1. Digital Scholar
Webinar
1st April 2020
Hosted by the Southern California Clinical and Translational Science Institute (SC
CTSI), University of Southern California (USC) and Children’s Hospital Los Angeles
(CHLA)

2. Eric Pedersen, Ph.D.
Associate Professor of
Psychiatry and the
Behavioral Sciences
Director of Digital Mental
Health
@ericRpedersen
@SoCalCTSI
About Today’s Session
Career building

3. Questions: Please use the Q&A Feature
1. Click on the tab here to
access Q&A
2. Ask and post question here
1
2

4. Today’s Topic
Disseminating Scientific Papers via Twitter:
Practical Insights and Research Evidence
Speaker:
Stefanie Haustein, Ph.D.

5. About the Speaker
Stefanie Haustein, Ph.D.
• Assistant Professor at the University of Ottawa’s
School of Information Studies
• She teaches research methods and evaluation,
knowledge organization, information
visualization, social network analysis and
information literacy.
• Her research focuses on scholarly communication,
bibliometrics, altmetrics, and open science.
• Co-director of the ScholCommLab, a research
group based in Ottawa and Vancouver that
analyzes all aspects of scholarly communication in
the digital age

6. Disseminating scientific papers via Twitter
Practical insights and research evidence
Stefanie Haustein | @stefhaustein

7. SC CTSI Webinar
7
Outline
→Social media in academia
→Twitter
→Scholarly metrics
→Altmetrics
→Metrics literacies

8. Social media
How Twitter & Co. are used in academia

9. Social media in academia
9
Social media use
→ ResearchGate most popular
among scientists and
engineers, less popular in
humanities and social sciences
→ Facebook known by >90% but
used regularly by <40%
→ Twitter among most known
(85%) but least used (>10%)
platforms
→ Hype medium
Van Noorden, R.. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126-129. https://doi.org/10.1038/512126a

10. Social media in academia
10
Social media use
→ 10-15% of academics use Twitter
→ Used for informational needs:
→ Post work content
→ Follow discussions
→ Discover papers
→ Used in social context:
→ Discover peers
→ Criticism and reluctance:
→ Shallow medium
“pointless babble”
→ Blurred boundaries
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for
Information Science and Technology. https://doi.org/10.1002/asi.23833
Van Noorden, R.. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126-129. https://doi.org/10.1038/512126a

11. Social media in academia
11
Twitter users per country
Number of users 2018 (in million) Percentage of population
https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/

12. Social media in academia
12
Geolocation of tweets linking to journal articles
Tweets with geotags captured by Altmetric 01/2012 to 06/2016

13. Social media in academia
13
Academics on Twitter
scholcommlab.ca

14. Social media in academia
14
Journal articles shared on Twitter
https://doi.org//10.1148/radiol.2020200490 https://dimensions.altmetric.com/details/76472854/twitter

15. Social media in academia
15
Journal articles shared on Twitter
https://monthlyreview.org/2020/04/01/covid-19-and-circuits-of-capital/ https://twitter.com/nosoyyo7/status/1244271157420986369

16. Scholary metrics
How quantitative indicators are used in
research evaluation.

17. Scholarly metrics
17
Impact
👩🏽🔬
👨👩👦👦
👫🏽
👭🏽
👬🏽
🏽
👨👨👧👦
👪
👨🏽🎓
👩🔬
👨🏽🎓
👨🏽🔬
👩🏫
🙎♂️
🙎♀️

18. Scholarly metrics
18
Impact
👩🏽🔬
Policy
pull
Data
push

19. Scholarly metrics
19
Types of metrics
→ Bibliometrics
→ Publications
→ Peer-reviewed journal articles
→ Books
→ Conference proceedings
→ Datasets
→ Citations
→ In peer-reviewed journals
→ In books
→ In conference proceedings
→ In policy documents
→ Collaboration
→ Usage metrics
→ Views
→ Downloads
→ Reuse
→ Social media metrics
→ Social networking
→ Facebook
→ ResearchGate
→ Social reference management
→ Mendeley
→ Zotero
→ Social data sharing
→ Github
→ Zenodo
→ Blogging
→ Microblogging
→ Twitter
→ Weibo
→ Wikis
→ Social recommending and reviewing
→ Reddit
→ F1000Prime

20. Scholarly metrics
20
Types of metrics
Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423. https://doi.org/10.1007/s11192-
016-1910-9
informetrics
bibliometrics
cybermetrics
webometrics altmetrics
scientometrics
data metrics
scholarly metrics

21. Scholarly metrics
21
Webometrics and altmetrics
→Webometrics
“Polymorphous mentioning is likely to become a defining feature of Web-based
scholarly communication.”
“There will soon be a critical mass of web-based digital objects and usage
statistics on which to model scholars’ communication behaviors […] and with
which to track their scholarly influence and impact, broadly conceived and
broadly felt.”
→PLOS article level metrics (ALM)
→Altmetrics
“study and use of scholarly impact measures based on activity in online tools and
environments”
“a good idea but a bad name”
Priem (2014)
Cronin, Snyder, Rosenbaum, Martinson & Callahan (1998)
Cronin (2005)
Rousseau & Ye (2013)

22. Scholarly metrics
22
Twitter-based metrics
Twitter API
Documentidentifier
Twitter
Facebook
Blogs
News
Wikipedia
…
DOI
Metadata
Citations

23. Scholarly metrics
23
Twitter-based metrics: Disciplines
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

24. Scholarly metrics
24
Twitter-based metrics: Journals
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

25. Scholarly metrics
25
Twitter-based metrics: Journals
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

26. Scholarly metrics
26
Twitter-based metrics: Users (>30k tweets)
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

27. Scholarly metrics
27
Twitter-based metrics: Users (>1k tweets)
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

28. Scholarly metrics
28
Twitter-based metrics: Hashtags
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

29. Scholarly metrics
29
Twitter-based metrics: Time patterns
Haustein, S. (2019). Scholarly Twitter metrics. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research
(pp. 729-760). Cham: Springer International Publishing. https://doi.org/ 10.1007/978-3-030-02511-3_28 | https://arxiv.org/abs/1806.02201

30. Scholarly metrics
30
Twitter-based metrics
→ No or only weak correlations with citations
Tweets are not a substitute for citation-based indicators.
→ Higher for more recent publications
Need to account for age differences by week or month or publication.
→ Need field-normalization (e.g., percentiles)
Articles in biomed or social sciences are tweeted more often than those in STEM.
→ Not all tweet activity is impact
Many tweets to articles are sent automatically or for promotional purposes.
→ Do not measure societal impact
The majority of users sharing links to journal articles are academics.
→ Tweet content is more interesting than tweet counts
Instead of using tweets for rankings, engage with interested users and tell a story.

31. Metrics literacies
Improving understanding and use of
scholarly metrics in academia.

32. Metrics literacies
32
We need to improve the way in
which metrics are used in academia.
Scholars are already overwhelmed
by too much to read.
Why not provide audio-visual
OER instead of text?

33. Metrics literacies
33
Definition
Haustein, S. et al. (2020). SSHRC Insight Development Grant Application.
Metrics literacies are an integrated set of competencies, dispositions and
knowledge that empower individuals to recognize, interpret, critically assess
and effectively and ethically use scholarly metrics.
Objectives
→ Develop open educational resources (OER) to support metrics literacies using
audiovisual material.
→ Experimentally test and identify the most effective types of OER for helping
academics and research administrators to appropriately apply and understand
research metrics.
→ Disseminate the most effective resources under CC-BY licences via knowledge
mobilization plan including social media and community partnerships.

34. Metrics literacies
34
Project team
→ Project management
→PI Stefanie Haustein
uOttawa, School of Information Studies
→Co-PI Michelle Schira Hagerman
uOttawa, Faculty of Education
→RA Alyssa Jeffrey
uOttawa, School of Information Studies
→ Bibliometric expertise
→Robin Champieux
OHSU, Librarian
→Carey Ming-Li Chen
Science & Technology Policy Research and
Information Center, Taiwan
→Isabelle Dorsch
Heinrich Heine University Düsseldorf
→Elizabeth Gadd
Loughborough University, Research Policy Manager
→ Media production
→Marie-Josée Archambault
Concordia, Mel Hoppenheim School of Cinema
→Peter Musser
Librarian and Youtuber
→Alli Torban
Podcast host and data visualization expert
→Community outreach
→Michelle Riedlinger
University of the Fraser Valley, Communication
→Germana Barata
State University of Campinas, SciComm
→Fiona Smith Hale
Ingenium, Chief Knowledge Officer
→Robin Champieux
Metrics Toolkit

35. Thank you!
doi.org/10.5281/zenodo.3736768
Stefanie Haustein | @stefhaustein