dans.knaw.nl
DANS is an institute of KNAW en NWO
Bibliometrics, Webometrics, Altmetrics,
Alternative metrics
A possible Ze...
NARCIS - http://www.narcis.nl/
EASY: https://easy.dans.knaw.nl/ui/home
Motivation
PhD on math
models of science
dynamics –
measurement –
scientometrics
(e.g., # researcher
in a field; # PhD
stu...
Growth of science and indicator systems –
How metrics came about?
Growth of science and indicator systems –
How metrics came about?
1950 1960 1970 1980 1990 2000 2010
NSF (1950)
https://ns...
Box model of research
Output
journal articles; citation
impact; patents
Input
Human capital: authors; ….
?students?
Expend...
Tailored metrics or all-in metrics?
Perhaps counter-intuitively, when it comes to metrics more is not necessarily always b...
Metrics - What for?
Questions
To better understand science dynamics
To better monitor science dynamics
How have discipline...
Blind spots – infrastructure and new fields
Start of large scale
digitization projects at the
Royal Library
Start of the "...
Get inspiration
Evidence Analytics & Information Systems
But be aware
Local (geo, topic, institutional)
science measurement
Global, cross-domain,
long-term
ResearchInformation
Sys...
Take away
Understanding Monitoring
Combine qualitative and quantitative research
Make sure to refer to standard data model...
References
Godin, B. (2005). Measurement and statistics on science and technology: 1920 to the present. London: Routledge....
dans.knaw.nl
DANS is an institute of KNAW en NWO
Thanks for your attention!
Andrea.scharnhorst@dans.knaw.nl
@ScharnhorstA ...
Nächste SlideShare
Wird geladen in …5
×

Bibliometrics, Webometrics, Altmetrics, Alternative metrics.

133 Aufrufe

Veröffentlicht am

A. Scharnhorst (2016) Bibliometrics, Webometrics, Altmetrics, Alternative metrics. Presentation given at the COST Action TD1210 Knowescape Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw

Veröffentlicht in: Bildung
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Bibliometrics, Webometrics, Altmetrics, Alternative metrics.

  1. 1. dans.knaw.nl DANS is an institute of KNAW en NWO Bibliometrics, Webometrics, Altmetrics, Alternative metrics A possible Zeno effect for science metrics, and why we nevertheless look for metrics? Andrea Scharnhorst www.knowescape.org Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw
  2. 2. NARCIS - http://www.narcis.nl/
  3. 3. EASY: https://easy.dans.knaw.nl/ui/home
  4. 4. Motivation PhD on math models of science dynamics – measurement – scientometrics (e.g., # researcher in a field; # PhD students in a field) Use of metrics in science policy – EastEurope in the mirror of bibliometrics – Matthew effect of countries (Bonitz) New practices, new metrics Web indicators for scientific, technological and innovation research – WISER 2002-5 Academic Careers Understood through Measurement and Norms - ACUMEN 2011-14 Impact-EV - Evaluation of SSH 2013-17 Visualisation of structure and evolution of science Visualising NARCIS Mapping Digital Humanities Digital Observatory for DH (Pilot) Semantic web technologies - Open Data CEDAR Dutch Historic Census New practices Research Data - FAIR
  5. 5. Growth of science and indicator systems – How metrics came about?
  6. 6. Growth of science and indicator systems – How metrics came about? 1950 1960 1970 1980 1990 2000 2010 NSF (1950) https://nsf.gov/statistics/ i.e., PhDs per field OECD (1961) Frascati Manual 63 EuroCRIS (2002) CERIF Standard Data Model VIVITI (1952) RZH ISI (1960) WoK, Citation indexing Altmetrics.com (2011) VIVO Open source software/ontology for scholarship wikipedia Google Scholar (2004) CASRAI (2006) Open standards RI, CA
  7. 7. Box model of research Output journal articles; citation impact; patents Input Human capital: authors; …. ?students? Expenditures: projects; ...?infrastructures? Process
  8. 8. Tailored metrics or all-in metrics? Perhaps counter-intuitively, when it comes to metrics more is not necessarily always better. When deciding what to record, you should picture yourself at operationally significant periods within the year like year-end, budget submission time, and month end, imagining the information you would ideally like to report upwards or use to make operational decisions for your department. For example a handy technique is to design your ideal annual departmental report and then work backwards asking whether at present you have the necessary data to produce the report. The annual report should talk to your firm’s strategic goals if it is to be effective and well received. Of course you won’t collect metrics solely for upward reporting to management, you’ll also collect metrics to help run your department better. Differentiate between external and internal metrics – those meant to help you and your team run things better, and those meant to communicate your value externally within the firm. Peter Borchers, Managing Director http://priorysolutions.com/articles/law-firm-library-metrics-aall-session-summary/
  9. 9. Metrics - What for? Questions To better understand science dynamics To better monitor science dynamics How have disciplines developed over centuries? Do innovation, institutionalisation, education operate on different time scales? What is the dynamic of the academic job market? How much ‘small fields’ does an university need? How adequate are national portfolios to team science? Impact of large scale infrastructure investment Who does re-use research data?
  10. 10. Blind spots – infrastructure and new fields Start of large scale digitization projects at the Royal Library Start of the "Cultural memory of The Netherlands" Start of Staten-Generaal Digitaal - Parlamentary Debates LifeCoursesInContext NWO - Mega RIS - Digital Databank for Newspapers (DDD) PoliticalMashup CEDAR - Dutch Historic Census EliteNetworkShifts DELPHER - Portal to digital sources of the KB ExPoSe Digging into Linked Parliamentary Data 1-Jan-99 31-Dec-00 31-Dec-02 30-Dec-04 30-Dec-06 29-Dec-08 29-Dec-10 28-Dec-12 28-Dec-14 27-Dec-16 From Digitization to Digital Humanities
  11. 11. Get inspiration
  12. 12. Evidence Analytics & Information Systems
  13. 13. But be aware Local (geo, topic, institutional) science measurement Global, cross-domain, long-term ResearchInformation Systems Not all measurement should be pursuit on all levels of granularity and all time! Up-scaling comes with a price!
  14. 14. Take away Understanding Monitoring Combine qualitative and quantitative research Make sure to refer to standard data models – re-use ontologies RI data are ‘just’ data – use the FAIR principles (findable, accessible, interoperable, re-usable) When experimenting with new Research Information Systems communicate where they are located (local-global; incidental-long- time;….) Communicate about error margin’s, uncertainty and ambiguity – visualise!
  15. 15. References Godin, B. (2005). Measurement and statistics on science and technology: 1920 to the present. London: Routledge. Godin, B. (2001). The Emergence of Science and Technology Indicators: Why Did Governments Supplement Statistics With Indicators? (No. 8). Montreal. Retrieved from http://www.csiic.ca/PDF/Godin_8.pdf - (annex: NSF indicators (scores/feasibility), considered by not recommended) Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Alt-metrics: a manifesto. October. Retrieved from http://altmetrics.org/manifesto/ Diana Hicks, & Wouters, P. (2015). The Leiden Manifesto for research metrics. Use these ten principles to guide research evaluation... Nature, 520(7548), 9–11. doi:10.1038/520429a Borgman, C. L. (2015). Big data, little data, no data: Scholarship in the networked world. Cambridge, Mass: MIT Press Börner, K. (2010). Atlas of science: Visualizing what we know. Cambridge, Mass: MIT Press. Börner, K. (2015). Atlas of knowledge: Anyone can map. Cambridge, Mass: MIT Press. Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., ... Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Nature, 3, 160018. DOI: doi:10.1038/sdata.2016.18
  16. 16. dans.knaw.nl DANS is an institute of KNAW en NWO Thanks for your attention! Andrea.scharnhorst@dans.knaw.nl @ScharnhorstA @knowescape Dans.knaw.nl

×