International university rankings are increasingly used by various stakeholders to compare higher education institutions, though they are also subject to criticism. This document provides an overview of some of the most prominent international rankings, including the THE, QS, Shanghai, and Leiden rankings. It discusses the methodology, indicators, and criticisms of each. The document also examines the positioning of Dutch universities like TU Delft in these various rankings and notes some new initiatives to improve rankings, such as U-Multirank, that aim to be more multi-dimensional and personalized.
University rankings; an overview for the municipality of Delft July 2013
1. 1Challenge the future
International Rankings of Universities
An overview for the municipality of Delft
Kim Huijpen, Corporate Policy Affairs | 04/07/13
2. 2Challenge the future
About me
Kim Huijpen
• Policy Advisor TU Delft
• Strategic Development / Corporate Policy Affairs
• Member of the Delft city council
• Committee Society and Housing
• Member of D66
3. 3Challenge the future
International university rankings
1. Context
2. Criticism
3. Overview
4. The position of the TU Delft in important rankings
5. New initiatives to improve rankings
6. How do we use international rankings?
And how do you use international university rankings?
4. 4Challenge the future
Context
Rankings fill in a need
• Stakeholders – students, parents, governments, accreditation
councils, industry (inter)national organizations – want to
know the differences between HEI’s and how they perform
Rankings are more and more used (directly or indirectly
via reputation)
• By the media
• By governmental institutions (reallocation of funds)
• By students (Asia)
• By HEI‟s themselves! For marketing purposes or to select
partners for cooperation
• By local governments?
5. 5Challenge the future
International rankings, criticism
and new developments
Most important international rankings in 2013
• QS-, THE-, Shanghai-, Leiden-ranking
• Not 4 rankings, but much more (also subject & reputation rankings)
Criticism
• Content: bias for big & old universities, focus on research, bias for
natural & medical sciences, language bias, comparison of whole HEI‟s
• Methodology: adding up all kind of indicators, numbering, dubious
weighting, intransparency, institutions deliver data, methodological
changes
6. 6Challenge the future
Criticism
Conceptual
1. Some universities have an advantage: Anglo-Saxon, beta- and
medical disciplines, focus on research, big, old, general
2. You can’t compare whole universities
3. You can’t add up all the indicators
Methodology
1. Underpinning of the weight factors
2. Sensitivity for outliers: best HEI=100 (z-scores are better)
3. Methodological changes in time
Data
1. Limited or no insight in the raw data
2. Data provided by HEI’s themselves: mistakes, manipulation
7. 7Challenge the future
Overview: similarities and differences
Ranking Focus Indicators Data Time Type
THE Research
Education
Internat.
Income
Subjective
Objective
Own Research
Dbase (WoS)
Data HEI’s
Present General
Field
QS Research
Education
Internat.
Subjective
Objective
Own Research
Dbase (Scopus)
Data HEI’s
Present General
Field
Shanghai Research Objective Dbases (e.g. WoS,
Nobel-prize.org)
Past
Present
General
Field
Subject
Leiden Research Objective Dbase (WoS) Present General
HEEACT Research Objective Dbase (WoS/ESI) Present General
Field
Subject
8. 8Challenge the future
TU Delft in rankings ‘12 & spring ‘13
World
University
Rankings
Engineering/
Technology
Rankings
Other Rankings
Subject
Rankings
Top 10 UIRC Scoreboard
(3)
QS Civil & Struct. Eng. (4)
QS Chemical Eng. (10)
Top 50 QS (18)
QS Environmental
Sciences (17)
THE (32) QS Materials Science (32)
QS Mechanical Eng. (18)
QS Electrical Eng. (42)
Top 100 THE (77) Shanghai
(76-100)
THE Reputation
(51-60)
4 QS Subject Rankings
Top 200 QS (103) 2 QS Subject Rankings
Leiden (164)
Top 300 Shanghai
(201-300)
Taiwan (276)
9. 9Challenge the future
THE-ranking (with Thomson Reuters)
Fields:
•‘Engineering &
Technology’
•‘Life Sciences’
•‘Clinical, pre-clinical
& Health’
•‘Physical Science’
•‘Social Sciences’
•‘Arts & Humanities’
Ranking by field:
based on same 13
indicators with
slightly different
weights
10. 10Challenge the future
What is citation impact?
• Citation impact is one of the key indicators in most rankings
• With a citation an author acknowledges the original author,
year, title, and source of an idea in a new publication
• Citations are measures of the impact of the cited work
• Citation „cultures‟ differ between disciplines therefore we
calculate citation impact normalized for field differences
11. 11Challenge the future
Position of 3TU, LDE & IDEA
League in THE ranking
University World
University
Rankings 2012-
13 (2011)
Engineering &
Technology
Ranking 2012-
13 (2011)
TU Delft 77 (104) 32 (22)
TU Eindhoven 114 (115) -
Universiteit Twente 187 (200) -
Universiteit Leiden 64 (79) -
Erasmus Universiteit Rotterdam 72 (157) -
Imperial College London 8 (8) 10 (10)
ETH Zürich 12 (15) 8 (9)
Ecole Polytechnique* 62 (63) 29 (29)
Aachen RWTH 154 (168) -
* ParisTech exists of eleven „Grandes Ecoles Paris‟ of which Ecole Polytechnique is the most well known.
12. 12Challenge the future
THE Engineering and Technology
Universities 2012
Position THE Engineering & Technology
ranking 2012-13 (2011)
1 Caltech, US (2)
2 Princeton University, US (3)
2 MIT, US (1)
4 University of California, Berkeley, US (4)
5 University of Cambridge, VK (6) /
Stanford University, US (5)
Highest non UK/US nr. 8 ETH Zürich (Switzerland)
Highest European (non UK/US) nr. 8 ETH Zürich (Switzerland)
13. 13Challenge the future
Indicators QS ranking
Fields:
• Arts & Humanities
• Engineering & Technology
• Life Sciences & Medicine
• Natural Sciences
• Social Sciences & Management
Ranking by field:
• Based on same
indicators
• Weightings are
different
14. 14Challenge the future
Position of 3TU, LDE & IDEA
League in QS ranking
* ParisTech exists of eleven „Grandes Ecoles Paris‟ of which Ecole Polytechnique is the most well known.
University General
ranking
(2011)
Engineering
and
Technology
ranking
(2011)
Natural
Sciences
ranking
(2011)
TU Delft 103 (104) 18 (18) 91 (79)
TU Eindhoven 158 (146) 67 (61) 186 (177)
Universiteit Twente 224 (226) 101 (116) 267 (229)
Universiteit Leiden 75 (88) - 89 (80)
Erasmus Universiteit Rotterdam 99 (103) - -
Imperial College London 6 (6) 6 (6) 11 (11)
ETH Zürich 13 (18) 8 (8) 10 (10)
Ecole Polytechnique* 41 (36) 36 (36) 43 (40)
Aachen RWTH 150 (140) 30 (35) 83 (82)
16. 16Challenge the future
Indicators Shanghai-ranking (since '03)
Focus Indicators Weighting
Quality of
education
Alumni winning Nobel prizes
and fields medals
Alumni 10%
Quality of faculty Staff winning Nobel prizes and
fields medals
Award 20%
Highly cited researchers HiCI 20%
Research output Articles and papers in Nature
and Science
N&S 20%
Articles and papers in SCI and
SSCI
PUB 20%
17. 17Challenge the future
Position of 3TU, LDE & IDEA
League in Shanghai ranking
* ParisTech exists of eleven „Grandes Ecoles Paris‟ of which Ecole Polytechnique is the most well known.
University Academic Ranking of
World Universities -
2012 (2011)
TU Delft 201-300 (151-200)
Universiteit Twente 301-400 (301-400)
TU Eindhoven 301-400 (301-400)
Universiteit Leiden 73 (65)
Erasmus Universiteit Rotterdam 151-200 (151-200)
ETH Zürich 23 (23)
Imperial College London 24 (24)
Ecole Polytechnique* 301-400 (301-400)
RWTH Aachen University 201-300 (201-300)
18. 18Challenge the future
Shanghai-ranking calculated trend
242
224 234
191 194 197 193 185 197
214
0
50
100
150
200
250
300
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Calculated positions TU Delft on Shanghai ranking
These calculated scores are based on calculations per indicator by University of Groningen
20. 20Challenge the future
Position TU Delft on impact indicators
Leiden Ranking 2010-2013
Ranking 2010 2011
2011* 2013
2013*
Leiden-ranking
(Worldwide top 500)
Old ‘Crown Indicator'
(CPP/FCSm)
123
‘Alternative Crown Indicator'
(MNCS)
176 189
99 219
168
Proportion top 10% publications
(PPtop 10%)
179
115 203
164
* Size-independent indicators, impact indicators using fractional counting & exclude publications in special types of journals
21. 21Challenge the future
Position Dutch universities Industry
Research Connections 2013
World NL Uni PP(UI collab)
1 1 Eindhoven University of Technology 15,6%
2 2 Delft University of Technology 14,0%
18 3 Wageningen University and Research Centre 10,1%
24 4 University of Twente 9,8%
137 5 Erasmus University Rotterdam 7,4%
147 6 Leiden University 7,3%
153 7 Maastricht University 7,2%
160 8 Utrecht University 7,1%
186 9 University of Groningen 6,9%
214 10 VU University Amsterdam 6,5%
253 11 Radboud University Nijmegen 6,1%
257 12 University of Amsterdam 6,1%
Or #3 with different counting
22. 22Challenge the future
New developments
Improvement of existing rankings
• More attention for education, finance and field (QS, THE)
• More representative survey on reputation (QS, THE)
• Rankings per field and subject (Shanghai, QS)
New rankings and classifications (education and third mission,
fields and subjects, ranking per indicator, no numbering)
• CHE university ranking: BSc-students
• CHE excellence ranking: MSc/PhD-students
• U-Map (CHEPS): types/profiles
• U-Multirank (CHERPA/CHE): institutional and field* rankings
* e.g. engineering
24. 24Challenge the future
What is U-Multirank?
2
Multi-dimensional
• Performance comparison not only on research but also on
education, exploitation, international orientation and regional involvement.
Multi-level
• Performance profiles based on a broad set of indicators. The performance profiles
are available at two levels: institution as a whole and underlying disciplinary fields.
Multi-stakeholder
• Designed in close consultation with stakeholders and intended for - students,
administrators, policy makers, employers, etc. - to meet their needs.
Multi-ranking
• Users can decide which areas of performance to include in the comparison of the
selected group of universities; in this way U-Multirank produces personalised
rankings.
26. 26Challenge the future
QS Best student cities
Methodology
• “Two pre-requisites have been established to identify the
cities evaluated in this exercise. The first is that each city
must have a population of over 250,000, the second that it
must be home to at least two ranked institutions. Current
calculations suggest that 98 cities in the world qualify on this
basis.”
1: Paris, 2: London, 3: Boston, 4: Melbourne , 5: Vienna
and 36: Amsterdam
27. 27Challenge the future
Indicators of QS Best student cities
• Student mix
• Student Population: as a proportion of the city‟s population
• International Volume: number of international students in the city
International Ratio: number of international students as a proportion
of all students
• Quality of living
• Mercer Quality of Living Survey 2011
• Employer activity
• Domestic Employer Popularity: Number of domestic employers who
identified one institution in the city as producing excellent graduates
• International Employer Popularity [x2]
• Affordability
• Tuition Fees [x2], Big Mac Index & Mercer Cost of Living Index
28. 28Challenge the future
Mercer's Quality of Living ranking
The Quality-of-living index
encompasses 39 different factors
within the following 10 categories:
• Political and social environment
• Economic environment
• Socio-cultural environment
• Medical and health considerations
• Schools and education
• Public services and transport
• Recreation
• Consumer goods
• Housing
• Natural environment
29. 29Challenge the future
How do we use international rankings?
Until now
• Participation in
rankings
• Internal memos
for the Executive
Board
• Annual report
• Roadmap 2020
• Website‘facts and
figures’
• Marketing and PR
30. 30Challenge the future
Messages
• More and more international rankings (need)
• Are used by several stakeholders and affect your reputation
• Are biased and have methodological drawbacks
• However, methodologies are improving
• Nevertheless, important to be in the rankings
• It is difficult for specialized universities to reach a high position
in general rankings (TU Delft: technology/engineering)
• However, field normalization is improving
• New initiatives to improve international rankings:
• U Multirank
31. 31Challenge the future
Questions and discussion:
Which rankings are relevant for the municipality of Delft ?
Which rankings do you choose for marketing purposes?
• More information:
• www.3tu.nl/uploads/media/Rankings_en_3TU.pdf
• Thanks to Johan Verweij & Jan Salden
• I elaborated on Johans presentations & used U-Multirank sheets of Jan
Kim Huijpen, Policy Advisor, TU Delft / Corporate Policy Affairs
T +31 (0)15 27 85296 | E K.Huijpen@tudelft.nl | @KimHuijpen
UIRC Scoreboard: TU Delft has even a second positionwith default “Exclude publications in special types of journals” in the Leiden ranking.UIRC 2013 scoreboard on university-industry research connections and cooperation (copublicationswith industry)4 QS Subject Rankings in top 100: Computer Science & Info Systems,QS Chemistry,QS Earth & Marine Sciences andQS Physics & Astronomy (51-100)2 QS Subject Rankings in top 200:Mathematics (101-150) & QS Education (151-200)
IndicatorenIndustry income: innovation (worth 2.5 per cent)Research income from industry (per academic staff)Research: volume, income and reputation (worth 30 per cent)Reputational survey – researchResearch income (scaled)Papers per academic and research staffCitations: research influence (worth 30 per cent)Citation impact (normalised avarage citations per paper)International outlook: staff, students and research (worth 7.5 per cent)Ratio of international to domestic staffRatio of international to domestic studentsProportion of internationally co-authored research papersTeaching: the learning environment (worth 30 per cent)Reputational survey – teachingPhD awards per academicUndergraduatesadmitted per academicIncome per academicPhD awards / bachelor awards
This ensures that institutional comparisons are “like with like” and not “apples and oranges”.Waarom: StudentenGeïnformeerde studiekeuzeKennisinstellingenPositionering, zichtbaarheid, strategische vergelijkingenBeleidsmakersInzicht in diversiteit en performanceBedrijvenPartners voor samenwerking