Pre conference workshop Economic Evaluations of Public Health Interventions
Amsterdam, EUPHA 2010
Public health economics was one of the themes of a pre conference at the 3rd European Public Health Conference in Amsterdam that took place from 10-13 November of 2010. Around 40 people participated at this pre conference. In four presentations the main topics in Public Health economics were introduced and illustrated. Economics is concerned with allocation of scarce resources in society over alternative uses. Some different types of evaluations were shown. The preference (utility) based health measure QALY (Quality Adjusted Life Years) was explained and discussed. In general methods for economic evaluations can be applied for evaluation of Public Health interventions. This was illustrated by a presentations on the economic impact of prevention strategies in tackling obesity. This study showed some good results in improving population health and decreasing health expenditure. However in many Public Health areas the effectiveness of public health interventions is still limited and should be assessed carefully concerning assumptions, costs calculated and models used.
More attention should be paid to inter-sectoral effects, equity considerations and a societal perspective in performing economic evaluations. Finally the involvement of relevant stakeholders is key to the success of Prevention.
The chair of this meeting concluded that Public Health and Economics could make a good couple. However for a longstanding relationship, we should put more effort in the evidence base of Public Health interventions. It is important that Public Health interventions demonstrate value for money!
The Health of the Portuguese. Portugal National Health Plan, Revision and Ext...
Eupha 2.lessons learned from published economic evaluations by_mathijsvandenberg
1. 1 11 November 2010
Lessons learned from
published economic
evaluations in public
health
Matthijs van den Berg
Johan Polder
Ardine de Wit
2. 2 Lessons learned from published economic evaluations in
public health | 11 November 2010
Content
1. Introduction
2. RIVM project
3. Cost-effectiveness of public
health interventions
4. Critical issues in public
health economic evaluations
3. Lessons learned from published economic evaluations in public
health | 11 November 2010
3
Cost-effectiveness is quite the thing
● Science
– Exponential increase in cost-effectiveness studies
– Both absolute and relative
● Practice
● Policy
4. Lessons learned from published economic evaluations in public
health | 11 November 2010
4
Exponential increase cost-effectiveness studies
0
10000
20000
30000
40000
50000
1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
5. Lessons learned from published economic evaluations in public
health | 11 November 2010
5
Exponential increase proportion c-e studies
0,0
0,1
0,1
0,2
0,2
0,3
0,3
1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
6. Lessons learned from published economic evaluations in public
health | 11 November 2010
6
Cost-effectiveness is quite the thing
● Science
● Practice
– Dutch manual for evidence-based guideline development:
› Cost-effectiveness should be one of the ‘other considerations’
that should be taken into account before coming to
recommendations (CBO, 2007)
– Guidelines more frequently use cost-effectiveness considerations
› Cardiovascular risk management
● Policy
7. Lessons learned from published economic evaluations in public
health | 11 November 2010
7
Dutch guideline ‘Cardiovascular risk management’
● Guideline:
– CVD 10 year mortality risk > 5%: lifestyle advice
– CVD 10 year mortality risk >10%: pharmacological treatment
● Recommendations are based on analyses of cost-effectiveness and
budget impact
● The threshold for cost-effectiveness is set on € 20.000 per QALY
gained, which corresponds with the value of ƒ 40.000 in the 1998-
guideline, corrected for inflation
8. Lessons learned from published economic evaluations in public
health | 11 November 2010
8
Cost-effectiveness is quite the thing
● Science
● Practice
● Policy
– Dutch policy document on public health: ‘Being healthy, and
staying healthy; a vision of health and prevention’
– Health care insurance board: judgement framework for
reimbursement decisions on the basic health care package
9. Lessons learned from published economic evaluations in public
health | 11 November 2010
9
● One of the main principles in public health policy making
– “Forms of preventive intervention that do not work will be
stopped,…
– and promising forms of intervention will be investigated to
establish how (cost-)effective they are....
– Forms of intervention that are known to be (cost-)effective
should, in principle, be funded through the appropriate insurance
systems, subject to the usual budgetary constraints.”
10. Lessons learned from published economic evaluations in public
health | 11 November 2010
10
● Dutch Health Insurance Board uses a judgement framework for
which interventions to cover, consisting of 4 criteria:
– Necessity
– Effectiveness
– Cost-effectiveness
– Feasibility
● Health Council of the Netherlands uses frameworks for decisions on
new vaccinations and screenings
– Cost-effectiveness is a main assessment criterion in both
frameworks
11. Lessons learned from published economic evaluations in public
health | 11 November 2010
11
Take-home message
● Cost-effectiveness is hot, especially in public health, but don’t take
the result of a cost-effectiveness analysis (i.e. an ICER) for granted
● Read cost-effectiveness analyses of preventive interventions very
carefully and ask yourself questions like:
– Are the assumptions on effectiveness well-founded and realistic?
– Which costs are included and which are ignored?
– What are critical choices in the model that was used?
12. Lessons learned from published economic evaluations in public
health | 11 November 2010
12
RIVM project ‘Cost-effectiveness of prevention’
● Aim of the project
– To identify potentially cost-effective preventive interventions that
have not yet been systematically implemented in the
Netherlands.
– i.e. to find opportunities for public health policy
● Basic idea of the project:
– Systematic and frequent literature search
– Read and appraise the cost-effectiveness analyses
– Write a report or a paper
13. Lessons learned from published economic evaluations in public
health | 11 November 2010
13
14. Lessons learned from published economic evaluations in public
health | 11 November 2010
14
15. Lessons learned from published economic evaluations in public
health | 11 November 2010
15
Number and type of prevention-CEAs
● In 2008 we found over 230 papers studying the cost-effectiveness
of a preventive intervention or program
● About half of the included studies used QALY’s of DALY’s as
outcome measure
16. Lessons learned from published economic evaluations in public
health | 11 November 2010
16
What diseases do prevention-CEAs focus on?
– Infectious diseases: 32%
– Cancers: 21%
– Cardiovascular diseases: 10%
– Mental- and behaviour diseases: 7%
– Musculoskeletal system: 7%
– Rest 23%
● Vaccination gets disproportionally much attention in cost-
effectiveness analyses relative to the burden of infectious diseases
● Mental- and behavioural diseases cause large burden of disease,
while they were the subject of only 7% of included studies
17. Lessons learned from published economic evaluations in public
health | 11 November 2010
17
How cost-effective were the studied interventions?
>80% lower than 50,000/(QA)LY
Median: €12,000/(QA)LY
● What do you conclude from this chart?
18. Lessons learned from published economic evaluations in public
health | 11 November 2010
18
How cost-effective were the studied interventions?
● Almost all preventive interventions have favourable cost-
effectiveness ratios
– Is prevention always cost-effective?
– Is there a publication bias?
– Do researchers work up to favourable ICERs?
19. Lessons learned from published economic evaluations in public
health | 11 November 2010
19
Some critical comments
● Based on observations in economic evaluations of public health
interventions
● Three categories
– Costs (numerator of the ICER)
– Effects (denominator of the ICER)
– Models
20. Lessons learned from published economic evaluations in public
health | 11 November 2010
20
Costs
● Which costs are included?
– Health care perspective (only health care costs and health benefits)
or societal perspective (all costs and all benefits)
– The societal perspective is preferred
– Public health measures have impact outside the healthcare sector
– Most CEA’s on prevention use the health care perspective, ignoring
very relevant cost categories
21. Lessons learned from published economic evaluations in public
health | 11 November 2010
21
Effects
● Is the intervention scenario realistic?
– What is the reach of the intervention? What participation rate is
assumed?
– Case: colorectal cancer screening using colonoscopy
22. Lessons learned from published economic evaluations in public
health | 11 November 2010
22
•Unpleasant bowel
preparation
•Uncomfortable
proceduce
•Risk of bowel
perforation
23. Lessons learned from published economic evaluations in public
health | 11 November 2010
23
Participation rate
● Is the intervention scenario realistic?
– What is the reach of the intervention? What participation rate is
assumed?
– Case: colorectal cancer screening using colonoscopy
– How many people would participate in such a screening?
24. Lessons learned from published economic evaluations in public
health | 11 November 2010
24
25. Lessons learned from published economic evaluations in public
health | 11 November 2010
25
Participation rate
● Systematic review of cost-effectiveness analyses of colonoscopy
screening
– Model assumptions on uptake: ~85%
– Real life data for participation: ~30%
● Conclusion:
– In many CEAs model assumptions on screening uptake were
more positive than real life data suggest
26. Lessons learned from published economic evaluations in public
health | 11 November 2010
26
Adherence
27. Lessons learned from published economic evaluations in public
health | 11 November 2010
27
Adherence
● Systematic review of medication CEAs found that many studies
used adherence rates from clinical trials which notoriously
overestimate adherence found in the general population
● However in a small subset of studies that varied adherence in
sensitivity analyses, the impact on the cost-effectiveness ratio was
substantial
● With policymakers increasingly turning to these studies for
guidance, failure to account for nonadherence may lead to
suboptimal resource allocation strategies
28. Lessons learned from published economic evaluations in public
health | 11 November 2010
28
Uncertainty in effectiveness
29. Lessons learned from published economic evaluations in public
health | 11 November 2010
29
Uncertainty in effectiveness
● “The aim of this paper was to determine the cost-effectiveness of a
Dutch school-based smoking education program.
● The incremental cost-effectiveness ratio of the school program was
estimated at €19,900 per quality adjusted life year gained.
● Main problem in estimating the cost-effectiveness was the lack of
proper effectiveness data on daily smokers among adolescents.”
30. Lessons learned from published economic evaluations in public
health | 11 November 2010
30
31. Lessons learned from published economic evaluations in public
health | 11 November 2010
31
Long term effects
32. Lessons learned from published economic evaluations in public
health | 11 November 2010
32
Long term effects
33. Lessons learned from published economic evaluations in public
health | 11 November 2010
33
34. Lessons learned from published economic evaluations in public
health | 11 November 2010
34
● “The range of effectiveness of the vaccine was considered between
30% en 90% with a positive baseline at 80%.”
● “The costs of the vaccine was assumed to be $450 for three doses
and ranged from $300 to $2000.”
● “Vaccination was found to be potentially cost-effective with an ICER
of -$2384 relative to standard treatment.”
› However, such a vaccine is not available at the moment
› This is a purely hypothetical excercise
35. Lessons learned from published economic evaluations in public
health | 11 November 2010
35
Models
●Time horizon
–Many modelling studies on the cost-effectiveness of preventive
interventions have very long time horizons (E.g. 80 yrs, 100
yrs, lifetime)
–Necessary to capture all health benefits (e.g. stop smoking
interventions and the prevention of lung cancer)
–Shorter time horizons would make the ICERs of many
preventive interventions very unfavourable
36. Lessons learned from published economic evaluations in public
health | 11 November 2010
36
Models
37. Lessons learned from published economic evaluations in public
health | 11 November 2010
37
Models
●Discount rates
– Most international CEAs use same discount rate for both costs
and effects (e.g. 4% or even 6%)
– Dutch guidelines prescribe different discount rates for costs
and effects (4% and 1.5%)
– This has a large influence on the ICER, making prevention a
lot more cost-effective
38. Lessons learned from published economic evaluations in public
health | 11 November 2010
38
●Case: cost-effectiveness of HPV vaccination
– Dozens of recently published CEAs
– Including several Dutch model studies
– Most of these conclude favourable cost-effectiveness
– However, de Kok et al., 2009 does not
39. Lessons learned from published economic evaluations in public
health | 11 November 2010
39
De Kok et al.: “In conclusion, many uncertainties still exist
about the effects of HPV vaccination on HPV-related diseases.
Our cost-effectiveness analysis shows that in the Netherlands,
a country with low cervical cancer incidence and mortality,
HPV vaccination is not cost-effective […].”
40. Lessons learned from published economic evaluations in public
health | 11 November 2010
40
Coupé et al.: “In a sensitivity analysis, de Kok et al. reported
an ICER of € 19 700 per QALY at discount rates of 4% for costs
and 1.5% for effects. The difference in reported ICERs between
the two studies can thus be largely explained by the discount
rates that were used.
41. Lessons learned from published economic evaluations in public
health | 11 November 2010
41
Postma: “I conclude that the conclusion by de Kok et al. is
misleading, should be re-visited and would probably better be
formulated as “In the Netherlands, HPV vaccination is likely to be
cost-effective if compared with screening alone”, […]”
42. Lessons learned from published economic evaluations in public
health | 11 November 2010
42
●Moreover
– Assumptions concerning participation rate in HPV vaccination
were about 85%
– While the real participation rate in the Netherlands appeared
to be about 50%
– In what way would that influence the cost-effectiveness ratio?
43. Lessons learned from published economic evaluations in public
health | 11 November 2010
43
Another example: rotavirus vaccination
44. Lessons learned from published economic evaluations in public
health | 11 November 2010
44
45. Lessons learned from published economic evaluations in public
health | 11 November 2010
45
46. Lessons learned from published economic evaluations in public
health | 11 November 2010
46
47. Lessons learned from published economic evaluations in public
health | 11 November 2010
47
● “The intervention had a gross incremental cost-effectiveness ratio of
AUD$ 3.70 (95% uncertainty interval (UI) $2.40, $7.70) per DALY.”
● “When the present value of potential savings in future health-care
costs was considered, the intervention was ‘dominant’ […].”
● “Restricting televised advertisements targeting children could
potentially be one of the most cost-effective population-based
obesity prevention interventions available to governments
today.The ICER was most sensitive to effectiveness assumptions.”
● “Limited evidence of the effectiveness of this intervention was a key
concern”
48. Lessons learned from published economic evaluations in public
health | 11 November 2010
48
● “The most relevant study to assess the effectiveness of a reduction
in advertising was a randomized controlled trial that compared the
impact of exposure to different food advertisements on food and
beverage selections in 288 5- to 8-year old children on holiday
camp for a period of 2 weeks in Quebec in 1982.
● The children were exposed to sweet commercials, no commercials,
fruit commercials or nutritional public service announcements.
Children who viewed sweet commercials chose (and ate)
significantly more sweets over fruit as snacks compared with
children in the other three groups (75 versus 67%, 64 and 65%,
respectively).”
● “Whether te reduction in BMI would be maintained over the lifetime
of the child is unknown and difficult to predict.”
49. Lessons learned from published economic evaluations in public
health | 11 November 2010
49
Conclusions
●Don’t take the result of a cost-effectiveness analysis (i.e. an ICER)
for granted
●Please read cost-effectiveness analyses of preventive interventions
very carefully
– Are the effect assumptions realistic?
– Are all relevant costs included?
– What are critical model choices?