Presented at the 14th International Society for Scientometrics and Informetrics Conference held in Vienna, July 16, 2013. The paper is available at: arXiv:1305.1216

1. The role of national university
rankings in an international context
The case of the I-UGR Rankings for Spanish universities
Authors: N Robinson-Garcia, JG Moreno-Torres,
D Torres-Salinas, E Delgado López-Cózar, F Herrera

2. 0. Rationale
University rankings have gained an undisputable position in
the research managers ‘toolkit’ for measuring the state of
health of higher education institutions
… however, they present many shortcomings that prevent
against a careless use…

3. Main limitations
I. Technical errors and difficulties on the recollection of reliable
and standardized data
II. Criteria for ranking universities not scientifically supported
III. Multidimensional nature of universities
IV. Neglecting the disciplinary focus of universities
V. International rankings do not reflect national university
systems
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

4. Main limitations
I. Technical errors and difficulties on the recollection of reliable
and standardized data
II. Criteria for ranking universities not scientifically supported
III. Multidimensional nature of universities
IV. Neglecting the disciplinary focus of universities
V. International rankings do not reflect national university
systems
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

5. International rankings either include only world-class
universities, not covering national systems…
or
… ignore the muldisciplinary focus
of universities
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

6. 1. Rankings I-UGR
Why a national ranking
Spain is poorly represented in rankings
-> 11 univs in the Shanghai Ranking – 16 in the Leiden Ranking <-
Existing rankings at the moment were unsuitable
-> Lack of continuity – disregard for disciplinary focus - … <-

7. ~ First edition dates from 2010
~ Measures the research performance of universities -> Web
of Science -> Main database considered for research
policy
~ Offers 12 broad fields and 37 disciplines based on
aggregates from the TS subject categories
~ Offers a five years-publication window and a ten years-
publication window
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Characteristics

8. Quantitative dimension
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
The indicator -> IFQ2A- Index
Qualitative dimension
NDOC - Publications
NCIT – Citations
H – H-Index
%Q1 - Ratio of Q1 papers
ACIT – Avg cit/paper
TOPCIT– Ratio of 10% Highly
Cited Papers
𝑸𝑵𝑰𝑭 = 𝑵𝑫𝑶𝑪 × 𝑵𝑪𝑰𝑻 × 𝑯 𝑸𝑳𝑰𝑭 = %𝑸𝟏 × 𝑨𝑪𝑰𝑻 × 𝑻𝑶𝑷𝑪𝑰𝑻

9. The indicator -> IFQ2A- Index
0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

10. Can we combine national and international
rankings to offer a global picture of a
university system’s positioning?
2. Comparing rankings

11. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
World-Class universities
INTERNATIONAL RANKINGS
NATIONAL RANKINGS The rest of the universities

12. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Selection criteria
 Based on the research dimension
 Based on a reliable or consensed data source
 Must offer rankings by fields

13. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Selected international rankings
Shanghai Ranking
QS Ranking
NTU Ranking

14. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Matching fields between rankings
SHANGHAI RANKING RANKINGS I-UGR RHO A
Natural Sci & Math Math ; Physics ; Chemistry * ; -0.5 ; 0.5 0/3 ; 3/3 ; 2/3
QS RANKING RANKINGS I-UGR RHO A
Chemical Engineering Chemical Engineering 0.84 4/7
NTU RANKING RANKINGS I-UGR RHO A
Computer Science Computer Science 0.75 8/13

15. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Matching fields between rankings
The Shanghai Ranking is the less consistent with the I-UGR
Rankings as only two fields have high correlations…
… while the NTU Ranking and the QS Ranking match better
with the I-UGR Rankings.

16. 3. Conclusions
Can national rankings complement international
rankings?
National rankings are needed in state-oriented university
systems, as they provide an accurate picture on the state of
universities…
… international rankings are interesting for research policy
makers as they put universities into a global context…

17. 0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions
Main conclusions and demands
I. There are discrepancies between rankings due to:
a. Methodology
b. Construction of fields
TRANSPARENCY AND CLARITY ON THE METHODOLOGY AND
CONSTRUCTION OF FIELDS
II. Despite methodological differences, it is possible to use both
and combine them in a research policy context

18. Thank you!
The role of university rankings in an international context:
The case of the I-UGR Rankings for Spanish universities
Authors: N Robinson-Garcia, JG Moreno-Torres,
D Torres-Salinas, E Delgado López-Cózar, F Herrera
Nicolás Robinson García
elrobin@ugr.es
http://www.ugr.es/~elrobin
@nrobinsongarcia