Presentation from a University of York Library workshop on bibliometrics. The session covers how published research outputs are measured at the article, author and journal level; with discussion of the limitations of a bibliometric approach.

1. Who’s counting?
An introduction to
bibliometrics
Thom Blake and Lindsey Myers
Research Support Librarians, Library Research Support Team
Academic Year
2018-19
AcademicYear
2019-20

2. What are bibliometrics?
Oxford English Dictionary:
The statistical analysis of books, articles, or
other publications.
…in order to evaluate research quality by
measuring the related scholarly output.
indicate
evaluate

3. Today’s workshop
 Mathematical definitions 
 Articles; Authors; Journals 
 Author rankings 
 Alternative metrics 
 Uses for bibliometrics 
 Further help 

4. Sources
 Google Scholar
 Scopus
 Web of Science
 Altmetric.com
Opportunities for hands-on practice
Questions welcome!

5. So: what counts?
“The impact of a piece of research is the
degree to which it has been useful to other
researchers and users” (Shadbolt et al. 2006,
p. 202).
How many times has it been cited?
More recently: page views, downloads, social
media mentions etc.

6. Pitfalls
 Not all citations are the same
 ‘Attention’ has a social context
 Disciplinary cultures vary
 Vulnerable to gaming?
 But subjective judgment isn’t perfect
either…

7. Exercise 1
Monitoring the number of times an
author has been cited is a reliable and
fair approach to evaluating the quality
of their research.
Discuss this statement with your neighbours.
(5 minutes)

8. Responsible metrics

9. Article-level indicators

10. Field weighting
 Mean citations
accrued by 2017 for
an article published
in this year
 Note the variation

11. Field-Weighted Citation Index
FWCI score of 1.00 =
mean citation rate for
 discipline
 year of publication
 type of article

12. Exercise 2
Using Scopus
 Carry out a keyword search for articles related to
your research.
 Identify the most highly cited articles and explore
the metrics. Are there any surprises?
 Note the details of a highly-cited article, for use
later in this workshop.
(10 minutes)

13. Alternative metrics
 Social media
 Mainstream media
 Reader behaviour

14. www.altmetric.com/products/free-tools/bookmarklet

15. Exercise 3
http://eprints.whiterose.ac.uk
Browse or search White Rose Research Online
for articles authored by members of your
department.
Explore the Altmetric data available.
(10 minutes)

16. Author-level indicators
Uses
 On a CV
 To report on a team or
department
 For management decisions
 To rank individuals 
Limitations
 Need to take account of
career length and pattern
 Very little data available for
books
 Disciplinary variation
 Incomplete data
Collated data for a researcher over the course of their
career

17. H-Index

18. Google Scholar profiles
 Create your own profile, by selecting your articles from
Google Scholar results.
 Register for ORCID so that your name is unique.

19. Exercise 4
Google Scholar profiles
Browse Google Scholar for York authors, or
search for prominent authors in your discipline.
Identify an author with a Scholar profile, and
explore their personal metrics.
(10 minutes)

20. Journal-level indicators
 Calculated by aggregating citation scores for all
the articles published in a specific journal,
e.g. Journal Impact Factor
 Ranked lists for each discipline
 Interpret with caution…

21. Limitations
Acta Crystallographica Series A
 a single highly-cited article from 2008 grossly distorted
the JIF

22. University policy
Policy for research evaluation using quantitative
data
http://bit.ly/PolicyResearchEvaluation

23. Where to publish?
Factors that might influence where you decide to
publish:
 appropriateness of the journal for your research
 status of the journal amongst peers
 likelihood of being accepted for the journal
 quality of the editorial process
 speed of publication
 openness of the journal ... etc.
Think.Check.Submit
https://thinkchecksubmit.org

24. In conclusion
1. Always compare like with like
 discipline
 career pattern
2. Be mindful of publishing
culture
 books or journal articles?
 turnaround
 social media attention?
3. Don’t rely on a single
bibliometric tool
 underlying metrics differ
4. Put the data in context
 to support subjective
judgment

25. A final word from you
Monitoring the number of times an
author has been cited is a reliable and
fair approach to evaluating the quality
of their research.
Have your views changed?

26. Further help
 Bibliometrics: a practical guide
subjectguides.york.ac.uk/bibliometrics
 Citation analysis and bibliometrics
www.york.ac.uk/library/info-for/researchers/citation
 Research Support Team
lib-research-support@york.ac.uk
 For the strategic use of indicators at institutional or
department level contact the Research Strategy and
Policy Office
www.york.ac.uk/staff/research/about-re/contacts-rsp