This document provides information about various tools for assessing research impact, including journal impact factors, citation indexes, the h-index, and alternative metrics. It discusses how to find impact factors for journals, citations to your own work, and how to quantify your research impact. It also covers controversies around citation metrics and tips to increase the visibility of your research.

1. IMPACT FACTORS
Part 3
Scholarly Communication Workshops
CONTACT INFORMATION
•UTSC | Sarah Forbes | sforbes@utsc.utoronto.ca
•Gerstein Science Information Centre |
Gail Nichol | gail.nichol@utoronto.ca or
Allison Bell | allison.bell@utoronto.ca
•UTM | Pam King | pam.king@utoronto.ca

2. How to Assess Your Research Impact
Learning Outcomes:
• Finding impact factors for journals
– JIF, Eigenfactor, SNIP, SJR
• How to find citations to your own work
– Science citation lookup
– Social science citation lookup
• How to quantify your research impact
– H index
• AltMetrics

3. How to Assess Your Research Impact
• Major controversies surrounding the value of
citation searching
• Tips to increase the visibility of your
research

4. 3 Citation Searching Tools
Web of Science Scopus Google Scholar
Access Commercial Commercial Free
# of titles >10,000 >23,600 ?
Customized Yes Yes Not yet
searches?
Contains citation 1945 (some back 1996 ?
content back to to 1900)
Notable features Contains older Wide international Includes books,
material, journal coverage, 100% of conferences,
articles and Medline content, records from
proceedings proceedings Open WorldCat
included, patents Project, anything
and theses on Open Web
Adapted from Sigma Xi International Newsletter, volume 3, number 11/12
November/December 2004, with additions

5. What can I say about the
journals I publish in?
Journal Citation Reports
• Journal Impact Factor
• Top journals in a field
• Eigenfactor
Scopus journal analyzer

6. Journal Impact Factor (JIF)
• A measure of the citations to a journal
– (NOT individual articles or authors)
• One indicator of a journal’s importance
• Calculated annually by Thomson Scientific
• Published in Journal Citation Reports
– (now an online database)

7. Journal Impact Factor
Calculated based on a two-year period – e.g. 2009
A = the number of times articles published in 2007-2008
were cited in indexed journals during 2009
B = the number of "citable items" (usually articles, reviews,
proceedings or notes; not editorials and letters-to-the-
editor) published in 2007-2008
2009 impact factor = A/B
(note that the 2009 impact factor was actually published in the
2010 Journal Citation Reports)

8. To access the Journal Citation Reports:
•Start at: http://gerstein.library.utoronto.ca/
•Click on Journal Citation Reports

9. There are two editions: Science and
Social Sciences
A group of journals can be
viewed by : Subject Category,
Publisher or Country

10. Depending on your discipline you may need to look at
both databases.
In the Science version there is a category called
EDUCATION SCIENTIFIC DISCIPLINES – 27 titles

11. Social Sciences version for Education….
Choose one or more
categories to see the journals
in that area.
Choose type of data to
sort by.

12. Click here to change
the data sorted by and
then sort again.
What do the IF numbers tell us?
A single number without context is hard to interpret. Review of Educational
Research has an IF of 3.326 (BUT for example in the subject of psychology
the Annual Review of Psychology has an IF of 22.750).
More meaningful? RER is the number one ranked journal in the area of
Education and Educational Research OR 1st of 139 journals.

13. Click here to change
the data sorted by and
then sort again.
Data sorted by Impact Factor (2-year IF) – note how sorting by 5-
year IF or by Eigfenfactor may change the order of journals

14. Eigenfactor Metrics
• Like the IF, essentially a ratio of number of
citations to total number of articles and is
a measure of a journal compared to other
journals.
• Eigenfactor Score
• Article Influence Score
• “better rank the importance of scientific
journals or papers by viewing them in the
context of the full citation network.”

15. Eigenfactor Score
• Articles from past 5 years
• How many times cited in the JCR year
• Consider which journals have cited –
highly-cited journals have more influence
• Journal self-citations are removed

16. Article Influence Score
• Roughly analogous to the JIF
• Mean AIS is 1.00
• A score greater than 1.00 means that
each article in the journal has above-
average influence
• Metrics Eigenfactor Project was conducted by Carl
Bergstrom and his laboratory at University of
Washington

18. Scopus Journal Analytics
• Select up to 10 journals
• Graphical display
• See how journals are performing relative
to each other
• More info:
http://www.info.sciverse.com/scopus/scopus-in

19. SNIP
• Source-Normalized Impact per Paper
• Measures contextual citation impact by
weighting citations based on the total
number of citations in a subject field.
• The impact of a single citation is given
higher value in subject areas where
citations are less likely, and vice versa.

20. SJR (SCImago Journal Rank)
• SJR is a measure of scientific influence of
scholarly journals that accounts for both
the number of citations received by a
journal and the importance or prestige of
the journals where such citations come
from.
• http://www.journalmetrics.com/sjr.php
• http://www.scimagojr.com/

21. The top ten journals ordered by
number of articles on the topic of a
Scopus search: buddhis*

22. Enter the titles of the top journals (up to 10) in the Journal Analyzer.
E.g. Japanese Journal of Religious Studies, Religion, Journal of Indian Philosophy,
Asian Philosophy, and Journal of the American Academy of Religion.

23. SCImago Journal Rank (SJR) , is a measure of the scientific prestige of
scholarly sources: value of weighted citations per document.
E.g. same 5 journals

24. How to quantify your research
impact
• H index
– Both Web of Science and Scopus will
calculate an H index based on content in their
database
– Only looking at citations in journals
• Other sources
– Google Scholar will also retrieve citations in
online sources like theses, government
publications, white papers, course reading
lists, books included in Google Books.

25. h index
• Developed by J.E. Hirsch
• Designed to “improve upon” simpler
measures such as total number of
citations or publications
• "A scientist has index h if h of his/her Np
papers have at least h citations each,
and the other (Np-h) papers have no more
than h citations each."

26. i.e. A scholar with an index of h has published h
papers, each of which has been cited by others
at least h times.

27. Finding Who Cites You
Scopus

28. SCOPUS
• Best for the Sciences but now has some
Social Sciences and Humanities coverage
– Life Sciences >4,300 titles
– Health Sciences >6,800 titles (all of Medline)
– Physical Sciences >7,200 titles
– Social Sciences & Humanities >5,300 titles
• References for items published 1996+
• Journals, conference proceedings, trade
publications & book series

29. From main library portal at http://www.library.utoronto.ca
click on Popular Databases and choose Scopus from list

30. Author Search
Last name: vicente, Initial or first name: k
For broadest results, leave Affiliation blank and all Subjects checked.

31. Note: A link to a details
page is only shown for
authors with more than
one document in
Scopus. Documents with
Click on the hyperlinked name for a details insufficient data may not
page about that author. be matched, this can
lead to more than one
entry in the results list for
the same author.

32. To see if there are any potential author matches

33. Select one or more authors and click
View grouped with author.
Note spelling of names:
Kim is spelled K i r n
Vicente is spelled Vincente
If you are the author you can ask
Scopus to merge the records.

34. Now we can see the
numbers for the main author
entry and also for the added
authors.
h index = 19

35. Remember that this graph and the h index
of 19 only reflect citations in journals or
conference proceedings. And in Scopus
only citations from 1996 to the present are
used.

36. A search in Google Scholar shows that
his book, The human factor, has been
cited at least 214 times.

37. If you have a common name, you may need to
search for the title of each article you have
written, one at a time.

38. link to your other identifiers
(such as Scopus or
ResearcherID or LinkedIn).
ORCID=Open Researcher and Contributor ID
More info about Scopus and ORCID collaboration:
http://www.info.sciverse.com/scopus/scopus-in-detail/orcid

39. Researcher ID is associated with Web of Science.

40. Finding Who Cites You
Web of Science

41. From main library portal at http://www.library.utoronto.ca
click on Popular Databases and choose Web of Science
from list

42. AUTHOR SEARCH
Use this for:
•citation analysis
•h-index auto-generation
•Author publishes under
>1 name

44. Option to limit by
topical area (or not)

45. 107 articles found.
Who cites them?
Click “Create Citation
Report”

46. 107 articles by Vicente cited
1675 times in WOS…..

47. What’s indexed in WOS?
• List of journals here:
– http://science.thomsonreuters.com/mjl/
• Conferences 1990+
– List of conference proceedings here:
• http://thomsonreuters.com/products_services/science/science_prod
ucts/a-z/conf_proceedings_citation_index
….. Next up: Scopus

48. Google Scholar

50. My Citations: author needs to setup a profile
and add their own publications
Metrics: h-index of a publication. GS displays
the h5-index (i.e. 5 years)
Alerts: create an alert on any query
Settings: this is where you will be able to set
your Library Links

53. How do the numbers compare
for Geoffrey Hinton?
Web of Scopus Google
Science Scholar
# of docs 203 146 337
Times cited 9,930 9,138 60,231
H-index 37 23 84

54. Journal Metrics on GS

55. How do the numbers compare?
5 year Impact Factor Eigenfactor Google Scholar
Cancer J of Clinicians Nature Nature
NEJM PNAS NEJM
Rev of Modern Physics Science Science
Annual Rev of Immunol Physical Review Letters The Lancet
Nat Rev Mol Cell Bio J Am Chem Soc Cell
Chem Rev Phys Rev B PNAS
Nat Rev Cancer J Biol Chem J of Clinical Oncology
Nat Materials Appl Phys Letters Nature Genetics
Nature NEJM JAMA
Physiological Reviews Cell Physical Review Letters

56. Google Scholar provides journal metrics for
Humanities publications.

57. Google Scholar Facts
• Estimated 40% journals, 34%
conferences, rest is “other”
• > coverage of non-English citations,
international coverage compared to other
databases
• Many unique citations in GS
• Expect duplication within GS search
results

58. Qualitative Measure of Impact
• Intended to balance quantitative approaches
• Becker Model (Bernard Becker Medical Library,
Washington University School of Medicine)
– https://becker.wustl.edu/impact-assessment
– Biomedical, but many measures can be applied
• Patent applications; licensing; cited in curricula;
material transfer agreements generated by a study;
requests for reprints; citations in a major review
article; hits on research study’s web site; results
factored into standards…

60. AltMetrics
• No one can read everything
• Rely on filters
• Traditional filters: peer review, citation
counting and JIF
• Are there others worth looking at?

61. Article-level metrics
• PLOS ONE - >2000 articles/month
• Metrics as a discovery tool – a filter for
you to find what interests you
• Data-driven stories about the post-pub
reception of an article
• More info at:
http://article-level-metrics.plos.org/

63. Scroll down to see
different metrics.

69. The Citation Debate

70. General Debate
• No one source is comprehensive
• Only some formats are searchable
• Publication dates can affect results
• Citation counts provide a limited
indication of impact

71. Journal Impact Factor (JIF) Debate
• JIFs vary across disciplines
– Variants in time-to-publication
• Review journals have higher JIFs
– Fewer articles per journal, cited more often
• JIF calculations are easily skewed
– Heavy editorial/letter/opinion content; high/low
percentage of review articles
• Open-access journals = higher JIFs

72. H-Index (H-i) Debate
• H-i does not consider the context of citations
– Fundamental to the argument, or just a passing ref?
• Two scientists, same h=30
– A has 20 papers cited more than 1000 times
– B has none
• H-i can only be compared fairly within a
discipline, not between disciplines
• H-i likes a long publishing career
– Greater chances for higher number of citations to
same articles

74. Increase Your Research Visibility
1. Do NOT automatically relinquish publishing rights
– SPARC Canadian authors addendum: preserve certain
rights for your use
– http://www.carl-abrc.ca/projects/author/author-e.html
– Find publisher archiving and publishing policies at
SHERPA http://www.sherpa.ac.uk
1. Publish in open access journals
• Find O/A journals at SHERPA
1. Self-archive your articles
• https://tspace.library.utoronto.ca/
• Find more archives at OpenDOAR:
http://www.opendoar.org/

75. Increase Your Research Visibility
• Strategies for enhancing the impact of
research: https://becker.wustl.edu/impact-
assessment/strategies
– Use the same name variation
– Standardized affiliation, no abbreviations
– And many other suggestions about
dissemination and keeping track of your
research….