Panel organized by Michael Habib (Scopus) New Possibilities in Evaluation Metrics: Authors + Altmetrics = ?

1. AUTHOR IDENTIFIERS &
RESEARCH IMPACT:
A ROLE FOR LIBRARIES
Kristi Holmes, PhD
Becker Medical Library
Washington University ICTS
VIVO
ORCID: 0000-0001-8420-5254
ALA Midwinter
January 26, 2014
New Possibilities in Evaluation Metrics:
Authors + Altmetrics = ?

2. So many great library-based projects
Projects that depend on
good identifiers
Tools in our toolbox
• Research impact
• Library-based services,
reports, analyses
• ORCID iDs
• Students and postdocs,
research groups, others on
campus
• Research networking and
• Scopus & other data
discovery, research
information systems
• Identity management, data
acquisition and processing,
analyses and visualizations,
liaison-type services
sources
• Analysis and visualization
software (Sci2, etc.)
Leverage the skills, resources,
strengths, and expertise of the library

3. Research Impact
Services
• Reports from many sources
• Analyses (pub data and more)
• Visualizations
• Other support services,
workshops

4. Why do we need to think about research
impact?
• Quantify and document
•
•
•
•
•
•
research impact
Justify future requests for
funding
Quantify return on research
investment
Discover how research findings
are being used
Identify similar research projects
Identify possible collaborators
Determine if research findings
are duplicated, confirmed,
corrected, improved or
repudiated
https://becker.wustl.edu/impact-assessment/model
• Determine if research findings
•
•
•
•
•
•
were extended (different human
populations, different animal
models/species, etc.)
Confirm that research findings
were properly attributed/credited
Demonstrate that research
findings are resulting in
meaningful health outcomes
Discover community benefit as a
result of research findings
Progress reports
Tenure
Promotion dossiers

5. Research Impact
“It is no longer enough to measure what
we can – we need to measure what
matters.”
How do we measure what
matters?
Wells R, Whitworth A. 2007. Assessing outcomes
of health and medical research: do we measure
what counts or count what we can measure?
Australia and New Zealand Health Policy, 4:14

6. Leveraging the Becker Model
• Would like to scale the Becker
Model for use by others.
• Adaptations in disciplines such as
agriculture, archeology,
nanotechnology, and more
https://becker.wustl.edu/impact-assessment

7. The Becker Model
• Provides a supplement to publication analysis to provide a
more robust and comprehensive perspective of biomedical
research impact.
• reporting templates, glossary of resources and terms, examples of
relevant indicators of impact across the research process, and readings
• Straightforward framework for tracking diffusion of research
outputs and activities to locate indicators that demonstrate
evidence of biomedical research impact
• individual, core, and institutional-level; modify for different disciplines
• Guidance for quantifying and documenting research impact as
well as resources for locating evidence of impact.
• Strategies for enhancing the impact of research
• Preparing for Publication, Dissemination, and Keeping Track of Your Research.

8. Meaningful impact
• New diagnostic criteria
• New standard of care
• Curriculum guidelines
Likely familiar…
• Measurement instruments
• Continuing education materials
• Clinical/practice guidelines
• Reviews
• Quality measure guidelines
• New funding awarded
Pathways
• Private healthcare benefit plans
 Advancement of
• New research studies
• Cost-effective intervention
Knowledge
• Invited lectures, new focus
• Consensus development
 Clinical Implementation
areas at conferences
 on committees
• MembershipLegislation and Policy
Enactment
• Awards
conferences American Medical
Association Current Procedural
Terminology (CPT) codes
• Change in delivery of
healthcare services
 Economic Benefit
 Community Benefit

9. Ok…so what do we DO with this?
How can we continue to make use of
this information?
(we don’t want it getting lost in a pile
of paper on somebody’s desk) ;)

10. Research networking and discovery,
Research information systems
• Showcase achievements and expertise
• Facilitate diffusion of research products
• Better understanding of the research enterprise
• Peer comparisons
• Strategic planning
• …and so on
• Many products (VIVO, SciVal Experts, Profiles, etc.)

11. What is VIVO?
1. An open source
semantic web
application
2. An information model
3. An open community*
* A big, welcoming OS community! Let us know if you have
questions or need information, connections, or materials:
http://vivoweb.org/contact

12. VIVO
An open-source semantic web application that
enables the discovery of research and scholarship
across disciplines in an institution.
VIVO harvests data from verified sources and
offers detailed profiles of faculty and researchers.
Public, structured linked data about investigators
interests, activities and accomplishments, and
tools to use that data to advance science.
VIVO enjoys a robust open community space to
support implementation, adoption, & development
efforts around the world.
See http://wiki.duraspace.org/display/VIVO

14. Acknowledgements
Collaborators:
• Cathy Sarli, MLS, AHIP
• Washington University ICTS
• Becker Medical Library
Questions/Follow-up:
• holmeskr@wustl.edu
• @kristiholmes
Funding:
• Washington University Institute of Clinical and
Translational Sciences, NIH award UL1
RR024992
• VIVO - DuraSpace