Citation metrics tools can be used to measure the impact of scholarly work. Established tools like Scopus, Web of Science, and Google Scholar provide citation counts and metrics like an author's h-index. Emerging altmetrics tools measure impact through social media mentions, saves, downloads, and other non-traditional sources. While still new, altmetrics provide a broader view of impact beyond citations alone. Librarians should be aware of both established and emerging citation metrics tools to help faculty evaluate research.
2. Citation metrics -
Tools to gauge scholarly impact
• Quantity
– Publications (tweets, blog posts, mentions…)
• Quality
– Citations (saves, downloads, likes…)
• Funding
– Research funds, collaborations
• Technology transfer
– Patents, start-ups, etc.
Focus today is on individual author metrics
3. Why should librarians care?
• Build relationships with faculty
• Learn about faculty research interests
• Assist in evaluation of departments, programs,
and faculty
• Accreditation efforts
• Marketing for academic programs
4. Scholars care because…
• Citation metrics are used in promotion and
tenure decisions
• Prestige and influence
– Within a field
– Within an institution
– Comparison to peer and aspirational institutions
• Vanity
– Individual and institutional
6. Consider Before Beginning:
• Cost of subscription databases
• Ease of use
• Time frames
– Citations to past year’s work (2011),
– Citations to author’s work in past 5 years (07-11)
– Citations to author’s work in past 10 years (02-11)
• Skewed towards STEM fields
• Don’t compare across databases!
7. Citation Metrics for Individual
Faculty Members
Typically, assess scholarly impact by looking at:
• Works published
• Citations to works published
• Publication influence (h-index, journal prestige,
etc. )
8. Scopus or Web of Science?
Scopus Web of Science
SciVerse Scopus is the world’s largest abstract Web of Science consists of nine databases
and citation database of peer-reviewed containing information gathered from
thousands of scholarly journals, books, book
literature. series, reports, conferences, and more.
• Contains 47 million records, 70% with
abstracts • It fully covers over 12,000 major journals.
• Nearly 19,500 titles from 5,000 publishers • Create a visual representation of citation
worldwide relationships with Citation Mapping
• Capture citation activity and trends
• Includes over 4.9 million conference graphically with Citation Report
papers • Use the Analyze Tool to identify trends
• Interoperability with Engineering Village & and patterns
Reaxys
• Provides 100% Medline coverage SU subscription:
• Science Citation Index Expanded (1899-
Subscription includes: present)
• 26 million records with references back to • Social Sciences Citation Index (1898-
1996 (of which 78% include references). present)
• 21 million records pre-1996 which go back • Arts & Humanities Citation Index (1975-
present)
as far as 1823.
As of 10/2012
9. Cited Reference Search
in Scopus
Tips:
Create an account to save your work
Check within each author group for publications
Merge author profiles, if necessary
10.
11. Cited Reference Search
in Web of Science
Tips:
Create an account to save your work
If you have a very prolific author, ask for
assistance with article identification
By default, citation counts are for All Years – you
must modify for your chosen parameters
Read the ‘Cited Reference Search’ how-to and
follow the directions closely
14. Citation Mapping in Web of Science
Visually Demonstrate Author/Article Influence
15. Google Scholar Citations
Tips:
Public profiles are available in Google
Can search for an author from within
your own profile page
16. Cited Reference Search
in Google Scholar Citations
• Author needs to set up their profile (using their
Google account); Google Scholar will harvest
related data.
• Authors can add articles, groups of articles, edit
entries, etc.
• Profiles are private unless author elects to make
public
• Quality control: “To be eligible for inclusion in
Google Scholar search results, your profile needs
to be public and needs to have a verified email
address at your university”
17. Comparison
Times cited H-Index
Scopus 135 7
Web of Science 85 11
Google Scholar 279 10
Times cited = number of documents published from 2007-current*
that have cited this author's work
H index = Number of author's articles that have been cited at least
this many times (during time span indicated)
*Normally, would choose the past 5 years, i.e. 2007-2011. However, to compare more
equitably to Google Scholar, used 2007- current (25 Nov 2012)
18. Author Disambiguation
Identification Schemes ORCID, about.orcid.org
• Scopus – Scopus Author • Not for profit
Identifier • Create registry of
• Web of Science – unique identifiers for
ResearcherID individual researchers
• Google Scholar Profiles • Open and transparent
linking between ORCID
• Institutional ID and other ID schemes
• Many other identifiers… • Many vendors,
institutions are
members
19. altmetrics is the creation and study
of new metrics based on the Social
Web for analyzing, and informing
scholarship
Altmetrics: A manifesto
http://altmetrics.org/manifesto/
20. What can be measured?
“Evidence of Use” – http://impactstory.org
• # of Tweets
• # of “Saves” in online reference managers
• Scholarly (and popular) blog interest and
activity
• Activity in social networking platforms, tools
• Etc!!
21. Tracking Mentions (or uploads/downloads, etc.)
in Social Media
citeulike is a free service for
Mendeley is a combination of a desktop
managing and discovering
application and a website which helps you
scholarly references
manage, share and discover both content and
6,402,349 articles - 3,030 added
contacts in research.
today.
ImpactStory is a website that makes it quick
and easy to view the impact of a wide range of Zotero is a free, easy-to-use tool to help
research output. It goes beyond traditional you collect, organize, cite, and share
measurements of research output -- citations your research sources. It lives right
to papers -- to embrace a much broader where you do your work—in the web
evidence of use across a wide range of browser itself.
scholarly output types.
23. Altmetric Limitations
• New
• Time frame – some new tools cannot search
old mentions, tweets, etc.
• Rely on user generated metadata
• Should social media mentions be given the
same weight as scholarly article citations?
• Can these tools be easily manipulated to raise
significance of an article?
24. Citation Metrics:
Established and emerging tools
• Established tools – Web of Science and Scopus
– Widely recognized and trusted
– Pricey
– Require some level of expertise to use
– Google Scholar is becoming an established tool
• Emerging tools – Altmetrics
– New, many promising platforms and services
– Librarians & information professionals should be aware
– When a few tools are established, trusted and verified
==>> acceptance
25. Thank you!
More information:
http://researchguides.library.syr.edu/citationmetrics
References
1. Eysenbach, G. (2011). Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and
Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4),
e123. doi:10.2196/jmir.2012
2. Google. (n.d.). Google Scholar Citations Help. Retrieved November 19, 2012, from
http://scholar.google.com/intl/en/scholar/citations.html#overview
3. ImpactStory. (n.d.). ImpactStory: faq. Retrieved November 19, 2012, from http://impactstory.org/faq
4. Galloway,L. & Rauh, A. (2012). Subject Guides. Citation Metrics. Web of Science. Retrieved November
16, 2012, from http://researchguides.library.syr.edu/citationmetrics
5. ORCID Inc. (2012). ORCID. Retrieved November 6, 2012, from http://about.orcid.org/
6. Priem, J., Groth, P., & Taraborelli, D. (2012). The Altmetrics Collection. (C. A. Ouzounis, Ed.)PLoS ONE,
7(11), e48753. doi:10.1371/journal.pone.0048753
7. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2011, September 28). altmetrics: a manifesto –
altmetrics.org. Retrieved November 16, 2012, from http://altmetrics.org/manifesto/
Hinweis der Redaktion
HI, I’m really glad to be here today and excited to talk about citation metrics. Using these tools and teaching others how to calculate individual author metrics has been a very interesting part of my job and has helped me connect with the faculty I serve.So today, as I mentioned, we are going to be talking about individual author or researcher citation metrics.
Scientists’ contributions to their field are reflected in their publications; for many scholars, publishing is important for peer recognition, promotion, and tenure. At this point in time, scholarly communication via academic journals remains the most widely accepted method of disseminating information. And quantifying the scholarly output from journals remains the time-honored method of determining productivity.So traditionally, when we look at citation metrics we evaluate the quality and quantity of publications, funding received, research collaborations and technology transfer. In purple, next to the traditional metrics, are the more ephemeral types of scholarly output ranging from tweets to mentions in social media to saves in online reference managers.And again, while there are many uses for citation metrics, today we are focusing on individual author metrics.
I’m a subject specialist librarian and fairly new to Syracuse University. One of the most effective ways that I have to get to know my constituents is by reviewing their publications – it helps me to get a good sense of the type of research they do.
Faculty and researchers care about this because this data is used in promotion and tenure decisions. And this also applies to librarians, of course, some of whom are faculty but if we publish, we want to see that our work is useful.Scholars may also like to gauge their impact on a particular field by using citation metrics. These data can also be used to compare various departments within an institution, although I would caution anyone who compares the citation patterns of different disciplines. Practitioners in some fields publish and cite much more than others. Metrics can also be used to compare to peer and aspirationalinstituions.And some people just like to see how much they are cited.
The established tools that I’ll briefly talk about today are Scopus, Web of Science and Google Scholar, which of course is a newer tool but uses primarily conventional data to calculate metrics. A note of caution: the data is skewed toward STEM (science, technology, engineering, mathematics) fieldsAnd, Books and book chapters not well covered by these tools
Don’t compare across tools – content coverage is different, metrics different and/or calculated differently. Be consistent in the work that you do with your faculty.Useful data can be found in each tool but direct comparisions across databases are problematic. These resources use different pools of data, date ranges and may interpret citations differently. Correct attribution of authorship can also cause reporting errors.
Traditional Citation Metrics measure:Number of published worksCitations to published worksPublication influencePublishing in prestigious journalsH-index – number of an author’s publications (h) that have at least (h) citations (so if an author has an h-7, they have 7 articles with at least 7 citations each) Other indicies
Citation analyses in WoS require more expertise and more time, in my experience. In my last position, I ran individual author citation metrics on about 130 faculty members. This work takes a LONG time in WoS. I would suggest working closely with authors if you are using WoS.
Need to look through all the citations to an author in order to make sure they are correctly attributed, and then ‘Finish Search’Want to know how many citations have SE Parks’ articles received in the past 5 years (2007-2011).For Parks, se = had to look through 51 citations to articles , however WoS did index some conference proceedings not found in Scopus.
Check/modify date range – make sure you know what date ranges mean (a little confusing in WoS)For example, ‘Cited Years’ in main search fields refers to the years that SE Parks’ articles were authored.But in our case, we are interested in citations to SE Parks’ work during the time frames we specified.
Citation mapping feature in WoS is very useful and provides excellent bibliometric data and a nice visual representation of an article’s (by extension, authors’) influence.
Google Scholar citationsCitations to articles publishedH-indexI10-index - i10-index is the number of publications with at least 10 citations. The second column has the "recent" version of this metric which is the number of publications that have received at least 10 new citations in the last 5 years.An author chooses to make their profile public - can find then in GoogleCan search for an author from within your own profile page
Will always find differences – function of data/publications included, date ranges and other variables – some that you can control and some that you can’t. Correct attribution of scholarly works is a concern, particularly with common names or Asian names. I’ll talk about this a bit more in the next slide.In S.E. Parks’ case – missing citations to two 2007 book chapters – not indexed in Scopus, but some conference proceedings are indexed in WoSMuch more content – and types of content – is included in Google Scholar. Also, in GS you “get what you get” – can’t change date ranges or work with the citation metrics. Perhaps, though, GS captures a more complete picture of a scholar’s output.
ORCID - Open Researcher ID - is an initiative to provide researchers and scholars with a persistent, unique identifier. This will enable individuals to get recognized for all their scholarly output, in both established and emerging media. With broad-based support from publishers, academic institutions, and funders, ORCID registration and services are free to individuals. Sign up here: http://about.orcid.org/.
Supplement to traditional peer-reviewed metrics“Crowdsourced peer-review”Many tools currently available:Google Scholar Profile/citations, Mendeley, Total-Impact, ReaderMeter, Plus, many more!
What can Altmetrics measure?“Evidence of Use” – Impactstory.org Rapidly gauge use and influence of an article – for example# of Tweets (has been found to predict citation rates)# of saves in online reference manager (Mendeley, Zotero) librariesScholarly blog interest and activity
At this point, there are many tools and resources to explore. Some of the most popular are on this slide. I personally like Mendeley as sort of a “readers guide” site – on the next slide you’ll see what I mean. Zotero is a fabulous citation management tool with awesome sharing and collaborative functions. And then there is ImpactStory (formerly total-impact) that allows users to create a report of impacts from many resources including articles, blog posts, datasets and much more. Altmetric toolsEmerging area with many emerging tools and platformsDescribing some of the most well known and more established tools
Here’s an example of crowdsourced peer-review – I’ve looked at the Marine Biology group in Mendeley, there are almost 90,000 articles in this group and the most popular papers are displayed on the home page of this discipline.
AnnieLogistical limitationsNew tools – still under developmentSome new tools can’t look at older tweets that happened before the tool was developedRely on user generated metadata – indexes get it wrong too but have some QC, for something to “count” when you tweet about something, how often do you have the Orcid ID, DOI or PubMed ID ?Larger questions to grapple with?Should social media mentions be given the same weight as scholarly article citations?Can these tools be easily manipulated to raise significance of an article?
ConclusionIndividual researcher metrics are moving beyond H-indices and Times CitedAltmetric tools can and should complement traditional metric toolsField is changing with technology and adoption of social networking for scientists It is a really interesting time to be involved with citation metrics. As we grapple with the importance and impact of social media on scholarly communication, and as adoption of different forms of communication becomes more widespread, the measurement of influence will need to change. I think it’s a great time for librarians to become familiar with traditional tools and help their faculty to begin to embrace altmetrics which allow for much more rapid measure of scholarly influence.