This document provides an overview of quality measurement in healthcare from a European perspective. It discusses conceptual frameworks for quality indicators, validity and reliability of quality measures, strengthening health information infrastructure, and reporting and using quality measures in practice. Key points include different quality dimensions, functions of healthcare systems, data availability across countries, country variations in data linkage capabilities, conceptual framework for OECD quality indicators project, and various European organizations involved in reporting healthcare quality. The future research agenda discussed includes improving conceptual robustness of measures, validity and actionability of indicators, information infrastructure, and evaluation of reporting and quality improvement impact.

1. A EUROPEAN
PERSPECTIVE
Niek Klazinga
OECD, Paris
AMC/UvA, Amsterdam

2. The R&D agenda on quality measures
• Conceptual Frameworks
• Validity and reliability of quality indicators
• Strengthening the information
infrastructure for measuring quality
• Reporting and use of quality measures in
practice

3. Conceptual considerations
• Quality dimensions: effectiveness, safety,
patient centeredness
• Functions of the health (care) system
• Scope of services and health problems
• Population based system perspective
• Quality alongside access, appropriateness,
efficiency and equity

4. Conceptual Framework for OECD
Health Care Quality Indicator
(HCQI) Project.
(shaded area represents the current
focus of the HCQI Project)
Source: Arah OA, et al. A conceptual framework for the
OECD Health Care Quality Indicators Project. International
Journal Quality Health Care. 2006; Sep 18; Suppl.1:5-13.
4

5. Data availability
•
•
•
•
•
Mortality statistics
Registries
Administrative data-bases
Electronic Health Records
Surveys
• Importance data linkage, use UPI, secondary
diagnoses coding, present at admission
codes, quality coding, completeness data-sets
……….

6. Country variation
National health data linkage
projects conducted on a
regular basis…
Country
With many national databases
Australia, Belgium,
Denmark, Finland, France,
Israel, Republic of Korea, Sweden, United
Kingdom and Australia
Canada,
Malta,
Norway and Switzerland
With several national databases
With 2 national databases
Cyprus,
Portugal,
Singapore
None
Japan,
Poland,
Germany
Source: OECD HCQI Country Survey, 2011/12
6

7. Too few countries are harnessing value from their
data for performance monitoring
Hosp. Prim- Can- Rx
inary
cer
patient care reg.
Mortality
LT
care
Mental
hosp. inpatient
National dataset 20
available…
Contains a UPI 15
16
18
14
20
16
17
12
14
12
15
11
12
Contains other 15
identifiers
Used for data
14
linkage studies
Used regularly
12
for data linkage
studies to
monitor health
care quality
12
17
12
17
12
15
10
14
12
16
11
8
4
12
7
12
4
5
Table 1: Number of countries reporting linkable data and reporting data use
Source: OECD HCQI Country Survey, 2011/12
7

8. 21 countries use clinical terminology
standards for some elements
8

9. For more information
Health policy brief and
report (2013):
http://www.oecd.org/els/healthsystems/strengtheninghealthinform
ationinfrastructure.htm
9

10. Asthma hospital admission in adults, 2006 and
2011 (or nearest year)
Rates per 100 000 population
2006
2011
160
120
80
Source: OECD Health Statistics 2013 , http://dx.doi.org/10.1787/health-data-en.
n.a.
151
122
117
119
103
76
76
126
74
63
69
72
68
96
68
90
61
80
61
55
51
55
46
29
41
42
40
n.a.
39
52
38
n.a.
37
53
37
44
37
n.a.
36
60
35
26
32
32
30
42
28
26
24
21
22
23
21
21
20
18
16
n.a.
15
18
14
0
17
11
40

11. 0
Source: OECD Health Statistics 2013 , http://dx.doi.org/10.1787/health-data-en.
13.9
12.2
10.8
8.9
8.9
8.8
12.6
12.1
12.5
15.8
15.2
14.8
12.9
11.1
10.8
9.9
11.6
15.4
15.7
2006
8.5
8.4
10.3
9.4
12.4
11.8
10.3
9.1
8.1
13.3
12.6
11.2
10.1
8.6
9.3
9.3
8.8
9.2
7.9
7.4
6.9
9.1
9.3
8.2
8.1
7.2
6.4
11.1
9.4
8.4
7.6
6.7
6.0
6.3
8.8
8.4
2001
7.8
7.7
7.6
7.6
7.1
7.0
6.8
6.8
6.8
7.9
6.2
5.9
5.8
5.7
5.7
5.5
5.2
4.8
4.5
4.5
6.0
5.1
5
4.5
3.0
Reduction in case-fatality within 30 days after
admission for AMI, 2001-2011 (or nearest year)
Rates per 100 admissions
2011
20
15
10

12. Postoperative sepsis in adults,
2011 (or nearest year)
Abdominal surgeries
All surgeries
452
98
Israel
Switzerland
Poland ¹
366
Belgium
998
350
413
United States
1135
1207
1226
527
Italy ¹
1791
603
1651
Canada
615
Finland ¹
627
Germany
787
657
Denmark
696
1182
1380
United Kingdom
758
716
Norway
732
OECD (20)
1831
1647
784
Slovenia
2356
946
n.a.
France
964
Spain
3027
1026
Australia
1052
New Zealand
1539
1068
Portugal
2763
2749
1121
1460
1427
Sweden
Ireland
2995
1667
0
500
1000
1500
2000
2500
3000
Rates per 100 000 hospital discharges
Source: OECD Health Statistics 2013 , http://dx.doi.org/10.1787/health-data-en.
3500

13. Breast cancer five-year relative survival, 20012006 and 2006-2011 (or nearest period)
2006-2011
2001-2006
89.3
United States ²
87.7
Canada ²
87.5
87.7
Australia ¹
86.5
Iceland ²*
87.5
87.4
87.3
Japan ²
86.4
New Zealand ¹
84.4
Sweden ²
85.7
86.3
86.2
Israel ¹
86.1
Norway ¹
83.7
Netherlands ²
84.9
Finland ¹
84.7
Slovenia ¹
83.1
86.1
85.9
85.9
85.6
85.2
Korea ¹
82.6
85.0
Germany ¹
85.0
Belgium ¹
84.8
84.5
Austria ²
81.8
OECD (18)
82.9
84.4
82.6
Portugal ²
82.3
Denmark ²
81.8
82.0
80.7
United Kingdom ¹
80.7
Czech Rep. ²
76.4
Ireland ¹
77.3
80.5
73.6
Poland ²
70.2
0
20
Source: OECD Health Statistics 2013 , http://dx.doi.org/10.1787/health-data-en.
40
60
80
100
Age-standardised rates (%)

14. Regular doctor spending enough time with patient
in consultation, 2010 (or nearest year)
Czech Rep. ¹
97.2
Luxembourg ¹
95.5
New Zealand ¹
92.7
Germany
92.5
Netherlands
91.5
Switzerland
91.3
United Kingdom
88.6
OECD (14)
87.1
Australia ¹
86.6
France
85.4
Israel
84.1
United States
81.4
Canada
80.5
Norway
78.0
Sweden
74.0
0
20
40
60
80
Age-standardised rates per 100 patients
Source: OECD Health Statistics 2013 , http://dx.doi.org/10.1787/health-data-en.
100

15. Reporting and Using Quality Measures
• Internal/formative and
external/summative functions
• Evidence on the impact of public reporting
• Need to make indicator sets fit for purpose
• Indicators as part of and embedded in
PDCA cycles (global validity versus local
actionability)

16. European organizations involved in
reporting on quality of health care services
• Aqua Institut (Germany)
• HAS (France)
• Kwaliteitsinstituut and RIVM (The
Netherlands)
• KCE (Belgium)
• Agenas (Italy)
• Socialstyrelsen (Sweden)
• Danish National Indicator Project (Denmark)

17. EU Funded related research projects
• ECHO (linking administrative databases
in six countries and looking at practice
variation and hospital outcomes)
• EuroHOPE (linking registries from seven
countries and looking at outcome
indicators)
• DuQue (looking at quality improvement in
European hospitals in eight countries on
process and outcome level)

18. The future R&D agenda
• Conceptual robustness
• Validity, reliability, relevance and action
ability of quality measures
• Strengthening the (national) information
infrastructures
• Evaluation research on reporting and use
of quality indicators (from quality watch to
quality action)

19. •
Health at a Glance 2013 (OECD) forthcoming
•
Health system performance comparison: New directions in research and policy.
•
Papanicolas I, Kringos D, Klazinga NS, Smith PC.
•
Health Policy. 2013 Aug 12.
•
International comparisons of health system performance among OECD countries: Opportunities and data privacy protection
challenges.
•
Oderkirk J, Ronchi E, Klazinga N.
•
Health Policy. 2013 Jul 16
•
Influences of hospital information systems, indicator data collection and computation on reported Dutch hospital performance
indicator scores.
•
Anema HA, Kievit J, Fischer C, Steyerberg EW, Klazinga NS.
•
BMC Health Serv Res. 2013 Jun 12;13:212
•
Influences of definition ambiguity on hospital performance indicator scores: examples from The Netherlands.
•
Anema HA, van der Veer SN, Kievit J, Krol-Warmerdam E, Fischer C, Steyerberg E, Dongelmans DA, Reidinga AC, Klazinga NS, de
Keizer NF.
•
Eur J Public Health. 2013 Apr 18.
•
Measuring and reporting on quality of care and patient safety in Canada: focusing on what matters.
•
Veillard J, Gula C, Huynh T, Klazinga N.
•
Healthc Pap. 2012;12(1):32-7; discussion 50-7
•
The validity of indicators for assessing quality of care: a review of the European literature on hospital readmission rate.
•
Fischer C, Anema HA, Klazinga NS.
•
Eur J Public Health. 2012 Aug;22(4):484-91. doi: 10.1093/eurpub/ckr165. Epub 2011 Dec 2. Review