Presentation at the iReg forum June 11th, 2015

1. Impact of clinical research as a
confounder for medical school
rankings
JENS PETER ANDERSEN, PHD
MEDICAL LIBRARY, AALBORG UNIVERSITY HOSPITAL

2. Clinical and basic research
 Definitions, as terminology varies:
 Basic research: experimental, basic medical research (e.g. medical
genetics)
 Pre-clinical research: experimental, medical research usually not
involving human subjects (e.g. animal studies)
 Clinical research: experimental, medical research focused on patients

3. Medical schools and teaching
hospitals
 Generally, basic and pre-clinical research is carried out at
universities, while clinical research is conducted at hospitals.
 Medical industry may be involved in all stages.

4. Important differences
 Generally:
 Distance from healthcare and –policy
 Distance from public
 For bibliometrics:
 Author-factors
 Citation densities & speed

5. Example of field differences
Basic:
Anatomy & morphology
 Average cited half-life: 7.6 years
 Average JIF (top10): 3.3
 Top JIF: 9.8
Clinical:
Clinical neurology
 Average cited half-life: 6.3 years
 Average JIF (top10): 11.7
 Top JIF: 21.8

6. Hospital ranking
 Leiden ranking: hospitals included as parts of universities
 ARWU ranking: undocumented, but likely
 Times Higher Education ranking: undocumented, but likely
 U-Multirank: hospitals are included – also with regards to beds per
student for teaching purposes

7. Consequences for ranking and
assessment
 Proper normalisation is always necessary (field, age etc.)
 Impossible to discern between effects from universities and hospitals
 Ranking tables may be misleading, e.g. for recruitment and student
attraction, if research impact reflects hospital research

8. Clinical evidence
and citations
From: Andersen, J. P., & Schneider, J.
W. (2011). Influence of study design
on the citation patterns of Danish,
medical research. In Proceedings of
the ISSI 2011 Conference (pp. 46–53).
Distribution of national and international publications
for university and university hospitals
Research institutions
Univ Hosp national pub
Univ Hosp international pub
Univ national pub
Univ international pub
Relativecitationimpact
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0

9. Clinical evidence
and citations
From: Andersen, J. P., & Schneider, J.
W. (2011). Influence of study design
on the citation patterns of Danish,
medical research. In Proceedings of
the ISSI 2011 Conference (pp. 46–53).
Meta-
analysis
RCT
Clinical trialSystematic
review
Nonsystematic
review
Cohort
Case-
control
Case-
report
0
0.5
1
1.5
2
2.5
3
0 200 400 600 800 1000 1200 1400
Meannormalizedcitationimpact
Publications
Danish
International

10. Clinical evidence
and social media
From: Andersen, J. P., & Haustein, S.
(2015). Influence of study type on
Twitter activity for medical research
papers. In Proceedings of ISSI 2015 -
15th International Society for
Scientometrics and Informetrics
Conference (p. in press). Istanbul,
Turkey.

11. Impact of universities
and hospitals:
Denmark as a case
Unpublished data.
Source: Web of Science,
AD=Denmark AND PY=(2011-2013)
AND DT=(Article OR Review)
Limited to medical research

12. Impact of universities
and hospitals:
Denmark as a case
Unpublished data.
Source: Web of Science,
AD=Denmark AND PY=(2011-2013)
AND DT=(Article OR Review)
Limited to medical research

13. Research profiles
 Hospitals, teaching hospitals and universities are all dissimilar.
 Profiling can illustrate varying types of institutions

14. Research profile I: King’s College
London and King’s College Hospital
King’s College London King’s College Hospital Collaboration
N c mc N c mc N c cc Coll-adv Hosp-adv
Obstetrics Gynecology 88 708 8,05 60 527 8,78 37 464 12,54 1,56 1,43
Surgery 171 721 4,22 57 225 3,95 28 127 4,54 1,08 1,15
Clinical neurology 241 1999 8,29 56 457 8,16 25 229 9,16 1,10 1,12
Gastroenterology hepatology 62 561 9,05 53 504 9,51 34 254 7,47 0,83 0,79
Hematology 89 982 11,03 35 348 9,94 16 170 10,63 0,96 1,07
Psychiatry 663 4572 6,90 12 85 7,08 9 77 8,56 1,24 1,21
Neurosciences 336 3034 9,03 25 188 7,52 15 86 5,73 0,63 0,76
Multidisciplinary sciences 255 2387 9,36 11 46 4,18 7 39 5,57 0,60 1,33
Clinical neurology 241 1999 8,29 56 457 8,16 25 229 9,16 1,10 1,12
Biochemistry molecular
biology
191 1963 10,28 9 43 4,78 5 20 4,00 0,39 0,84

15. Research Profile II: Harvard University
and Dana-Farber Cancer Institute*
Dana-Farber Cancer Institute Harvard University Collaboration
N c mc N c mc N c mc Dana-adv Univ-adv
Oncology 704 8525 12,11 1458 15532 10,65 704 8525 12,11 1,00 1,14
Hematology 172 1660 9,65 447 4020 8,99 172 1660 9,65 1,00 1,07
Multidisciplinary sciences 144 3664 25,44 1518 26541 17,48 144 3664 25,44 1,00 1,46
Cell biology 137 3641 26,58 888 17056 19,21 137 3641 26,58 1,00 1,38
Biochemistry molecular biology 106 2816 26,57 976 16012 16,41 106 2816 26,57 1,00 1,62
Multidisciplinary sciences 144 3664 25,44 1518 26541 17,48 144 3664 25,44 1,00 1,46
Oncology 704 8525 12,11 1458 15532 10,65 704 8525 12,11 1,00 1,14
Surgery 36 181 5,03 1122 5193 4,63 36 181 5,03 1,00 1,09
Neurosciences 11 119 10,82 1020 8954 8,78 11 119 10,82 1,00 1,23
Biochemistry molecular biology 106 2816 26,57 976 16012 16,41 106 2816 26,57 1,00 1,62
*Unlike many other medical schools, Harvard does not own their affiliated hospitals.

16. Implications
 Call for more complete details in rankings:
 Are hospitals counted?
 Where does the impact come from?
 Identification of basic and clinical research
 Should we normalise based on study types? Or report profiles of
study types?

17. Discussion points
 Why does medical research require special attention?
1. It doesn’t – technical research institutes potentially provide similar
problems
2. They differ from other university research areas, because of e.g. funding
sources, employment types, research cultures, research goals – and
there are vast geographical differences
 What about translational research?

18. Thank you for your attention
In picture: Aalborg University Hospital

19. Jens Peter Andersen
Email: jepea@rn.dk
Slides: slideshare.net/ipoga