A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013.
Presented by: Jon Ritterbush of the Calvin T. Ryan Library at the University of Nebraska-Kearney.
Visit to a blind student's school🧑🦯🧑🦯(community medicine)
Quantifying the impacts of scholarship
1. Quantifying the impacts of scholarship
by Jon Ritterbush
E-Resources & Serials Librarian
Calvin T. Ryan Library
University of Nebraska-Kearney
Mountain Plains Management Conference
October 18, 2013
2. A few housekeeping items
• A link to these slides will be posted at:
guides.library.unk.edu/scholarship
• Q & A period following today’s presentation
• Tweet a question during today’s presentation to
#MPMCimpact
3. What are Impact Factors (IFs)?
• Quantitative values assigned to academic journals –
not to authors’ individual articles – based on
citations to a journal over a specified time period
• Idea originated with Dr. Eugene Garfield in 1955
• IFs are published annually in Journal Citation
Reports (JCR)
- a Thomson Reuters database ($$$ - check your library)
- Provides IFs for 8,200+ science and 2,900+ social science journals
- Includes 2-year and 5-year IF values, with other journal metrics
4. Why do IFs matter?
• Used by scholars to help decide where to publish an
article for maximum exposure
• Used by librarians to evaluate journal titles for
acquisition or cancellation
• May be used by some accreditation boards to
evaluate a program’s scholarly productivity
• May be used in faculty evaluations for
promotion, tenure, or grants
5. How are IFs calculated?
The 2011 2-year impact factor for Academy of Management Review
is calculated by:
# of times AMR articles published in 2009 and 2010
were cited in indexed journals1 during 2011
# of “citable” 2 AMR articles published in 2009 and
2010
1. Only journals indexed by JCR are counted. Book chapters, proceedings, and
journals not indexed by JCR are not counted in this numerator.
2. Only research articles and reviews are counted as citable works. Letters to the
editor or other commentaries are not counted in this denominator.
6. How are IFs calculated?
The 2011 2-year impact factor for Academy of Management Review
is calculated by:
Note: a journal’s IF
will vary each year!
7. Who else provides IF data?
Eigenfactor (free)
• Its “article influence” (AI) score measures the average influence of
each article in a journal during 5 years following publication
- The AI score is somewhat comparable to JCR’s impact factors
- AI scores are weighted, so that journals are ranked as more influential if
cited by other influential journals
• Limited to/Ranks the same journals indexed in JCR
• Unlike JCR, journals in Eigenfactor are assigned to only one
subject category
• “Self-citations” to the same journal are removed!
• Freely available at eigenfactor.org
8. Who else provides IF data?
SCImago Journal Rank - SJR (free)
• Based on citations in Elsevier’s Scopus database to articles
published over a 3-year period
• Impact factors are weighted based on rankings
• Self-citations to the same journal are removed
• Freely available at scimagojr.com
- Note: SJR results use European/international numerical notations for
decimals and the thousands separator.
9. Who else provides IF data?
• Google Scholar Metrics (free)
- Indexes journal and conference papers, theses and
dissertations, academic books and other content from
publishers, societies, and open repositories. Court opinions and
patents are also included.
- Uses h5-index for journal rankings: Equal to h articles published in
last 5 years, which have been cited elsewhere at least h times.
- See bit.ly/14pgXmL for business journal rankings
• Publishers’ descriptions of journals (free)
- Google the name of the journal to find publisher’s website listing
subscription info, author instructions, and sometimes the journal’s IF
10. Criticisms of IFs in general
• Some disciplines take longer to build upon and cite
new research
• IFs vary between subject disciplines -- an IF for a
chemistry journal is not comparable to the IF of a
business journal.
• Publication changes can affect IF values
• Not all scholarly journals receive IF values; new
journals are at some disadvantage
• The quality and impact of an author’s work cannot
be truly assessed by a journal’s IF
11. Individual measures of scholarship
Two common methods:
• Counting citations to one’s published research
• h-index
Other new methods:
• Views/downloads of articles
- Rarely available for public access from publishers
- Some publishers may make this information available to authors (e.g.
Nature, Springer)
• Altmetrics
12. Who provides citation counts?
• Some academic library databases
- Social Science Citation Index
- Scopus
- EBSCO’s Academic Search Premier, Business Source
Premier, Communication & Mass Media Complete, PsycINFO
- ProQuest’s ABI/Inform among others
• Google Scholar (free)
14. Citation counts in Google Scholar
Start at http://scholar.google.com
Search by an article title, or
use Advanced Scholar Search
to search by author
15. Citation Count Comparison
Number of Articles Indexed / Number of Citations to those Articles
Social Science
Citation Index
Dr. Sample
EBSCO Business
Source Premier
Google
Scholar
7 / 24
11 / 28
21 / 75
16. h-index
h-index: Equal to h papers which have been cited elsewhere
at least h times.
Paper
Cited by
My first paper
14
My second paper 6
My third paper
4
My fourth paper
2
My fifth paper
0
h-index = 3
17. Tracking your citations in Google Scholar
Google Scholar’s “My Citations”
• Free with Google
account
• Add, edit, or ignore
citations within your profile
• More details at Google:
see bit.ly/YzJ2BY
18. “Publish or Perish” software
• Freeware for analyzing GS citations, available for Windows and
MacOS
• See http://www.harzing.com/pop.htm
19. Additional readings at bit.ly/11wjLiK
• Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals.
College & Research Libraries News, 68(5), 314–316. Retrieved from
http://crln.acrl.org/content/68/5/314.full.pdf+html
• Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 9093. Retrieved from http://garfield.library.upenn.edu/papers/jamajif2006.pdf
• Levine-Clark, M. & Gil, E. L. (2009). A comparative citation analysis of Web of Science,
Scopus, and Google Scholar. Journal of Business & Finance Librarianship, 14(1), 32-46.
doi:10.1080/08963560802176348
• Monastersky, R. (2005). The number that’s devouring science. Chronicle Of Higher
Education, 52(8), A12-A17.
• Rossner, M., Epps, H. V., & Hill, E. (2007). Show me the data. The Journal of Cell Biology,
179(6), 1091–1092. Retrieved from
http://jcb.rupress.org/content/179/6/1091
• Testa, J. (2012). The Thomson Reuters journal selection process. Retrieved from
http://thomsonreuters.com/products_services/science/free/essays/
journal_selection_process/
20. Questions? #MPMCimpact
For a brief slidecast and more links on
scholarly communication, go to:
guides.library.unk.edu/scholarship
Thank you!
Jon Ritterbush
ritterbushjr@unk.edu
Skype & Twitter: @loperlibrarian
Hinweis der Redaktion
A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013. Presented by: Jon Ritterbush of the University of Nebraska-Kearney’s Calvin T. Ryan Library.
Dr. Garfield set out to identify source journals for inclusion in what would eventually become the print Science Citation Index.JCR published since 1975. JCR is updated once a year – usually in June/July.
In England, hiring panels routinely consider impact factorsAccording to Spanish law, researchers are rewarded for publishing in journals ranking in the upper third of IF listings.In some Chinese universities, graduate students in physics must place at least two articles in journals with a combined IF of 4 to earn their PhDs
The next two slides offer an example of how a 2-year Impact Factor are calculated by Journal Citation Reports.Bear in mind: Not all journals are indexed in JCR! And other scholarly sources are not either! Titles are added, and sometimes removed from JCR based on selection criteria.
Remember the IF will vary each year, and it is not cumulative!“Of 38 million items cited from 1900-2005, only 0.5% were cited more than 200 times. Half were not cited at all...” according to Dr. Garfield in an article on the “History and Meaning of the Journal Impact Factor” published in JAMA 2006.
Article Influence scores are normalized so that the mean article in the entire Thomson Journal Citation Reports (JCR) database has an article influence of 1.00. There are some key difference with JCR, such as the inclusion of subject categories and removal of self-citations. “Self-citations” are articles citing other articles from the same journal. Some scholars have alleged that self-citation practices lead to over-inflated journals IF values.
Scopus indexes more titles than (about 21,000 titles versus 11,000 in JCR).SCImago uses a 3-year period, instead of 2- or 5-years.They weight their IF values based on rankings, similar to Eigenfactor, and also removed self-citations in the calculations.
Because Google Scholar is crawling so many sources automatically, it can cover a wide body of literatureExample: In GS – Academy of Management Journal currently has an h5-index of 69 which means in the last 5 complete years (2008-2012), 69 articles have at least 69 citations each...Talk more about h-index values in a few minutes, as it relates to individual productivityGS also provides business journal rankingsJournal of Management – is an example of publisher-listed IF.
Title changes: After a title change, two JCR years must pass before the new title fully replaces the previous title in JCR. This would also impact Eigenfactor. And changes can even influence Google Scholar indexing and hence its metrics! Likewise, if a journal cuts back from publishing 6 times a year to 4 times a year, the impact factor may rise temporarily.As mentioned before, new journal titles may take awhile to be included in JCR, if ever!
Keep in mind that EBSCO databases are looking at specific slices of scholarly literature – typically scholarly journals and selected trade publications within a particular subject discipline. These citation counts will often be lower than what is retrieved using Google Scholar!
It is imperative if you search the SSCI database that you use the “Cited Reference Search” option!!! Using this, I found more citations for Dean Burkink (7 items with 24 citations, instead of 4 and 17 using the “Author search” option.The differences are linked to what is in each source’s database. SSCI is the smallest of the three sources in this table, so it’s not a terrible surprise that they came up with the lowest numbers.
In this example, an author has three papers that have been cited at least three times, yielding an h-index of 3.
Creating a GS “My Citations” profile requires a Google accountWorth noting that an article with 18 citations was published in the Journal of Web Librarianship. This peer-reviewed journal has been published by Taylor & Francis since 2007, but still isn’t indexed by JCR! While that journal doesn’t have an impact factor, it doesn’t mean that my article isn’t having an impact!Meanwhile, my article published in the Journal of Academic Librarianship has only been cited once. The journal is in JCR, and is regarded among academic librarian as a leading publication in the field.Publishing in a high-impact factor journal may not always translate into more views/downloads, nor into more citations.
This didn’t work so great for my name. It pulled up all of the database/website reviews I had written (none of which had been cited). It also pulled in results for lots of other Ritterbushes with different first initials – searching on “J* Ritterbush” It also appeared to treat my title and place of employment as articles(?!) and it pulled in some webpage research guides I’ve created too.