In this ISPOR session Chuck Phelps and Adrian Towse debated the case for and against using MCDA to support HTA decision making, as compared to weighting or augmenting a QALY based ICER approach. Chuck Phelps argued for use of MCDA, Adrian Towse for weighting the QALY. Nancy Devlin set the scene and moderated. Author(s) and affiliation(s): Nancy Devlin, Director, Centre for Health Policy, University of Melbourne, Australia Adrian Towse, Emeritus Director, Office of Health Economics, London, UK Chuck Phelps, University of Rochester, Rochester, NY USA Event: ISPOR 2019 Location: New Orleans, USA Date: 21/05/2019

1. Augmented CEA or
MCDA?
Prof. Nancy Devlin
Director, Centre for Health Policy
University of Melbourne
IP11, ISPOR New Orleans, May 2019

2. Our starting point
• HEOR%methods%and%evidence%generation%for%HTA%principally%focused%on%
CEA%and%ICERs
• But%even%in%cases%where%HTA%is%primarily%concerned%with%cost%
effectiveness,%it%is%not%the%only%consideration
• There’s%a%push%toward%more,%not%fewer,%criteria%eg. US%value%frameworks
• Our%focus%in%this%session%is%how%HEOR%can/should%go%beyond%generating%
evidence,%to%methods%to%assist%decision%makers%to%weigh%up%evidence%on%
multiple%factors
2

3. 3
Why not rely on deliberation alone?
• errors,'biases'&'heuristics'
that'affect'individuals’'
judgements'when'confronted''
with'complex'decision'
problems
• combined,'in'committee'
deliberations,'with'group'
dynamics,'differences'in'
chairing'styles
• can'materially'affect'the'
decision'that'results'from'
considering'a'given'body'of'
evidence'

4. The role of judgement in HTA
• We#need#judgements#to#be#made#(scientific;*social*value*judgements)*in*order*to*make*
decisions*– there’s*no*way*around*that.**That’s*why*we*have*HTA*committees.*
• But*we*also*want*a*process*where*those*judgements#are#made#in#a#consistent,#
transparent#manner.
• Accountability*to*taxpayers
• Clear*‘signaling’*to*the*suppliers*of*health*technologies
Saying*that*‘HTA*should*be*transparent’*is*a*value*judgement*– but*surely*a*fairly*weak*
one?*
4

5. Time to move the debate along…
(1)$Multiple$criteria$are$and$will$continue$to$be$used$in$HTA
(2)$There$will$always$be$a$need$for$judgements$and$deliberation$in$HTA
(3)$There$are$ways$of$improving$the$way$deliberation$and$judgements$are$
reflected$in$decision$making
Two$principal$options:
• ‘augment’$the$standard$CEA$approach
• adopt$some$form$of$structured$decision$making$(MCDA)
What$are$the$relative$merits$of$each?$
5

6. What do we mean by augmented CEA?
• Reflecting*considerations*other*than*health*care*costs*and*QALYs
ICER*=*(∆ Costs/∆ QALY+)*
6
In*the*denominator:*
weighting*the*QALYs*to*
reflect*characteristics*of*
those*experiencing*them
In*the*numerator:*
include*wider*costs*and*
offBsetting*monetary*
benefits*not*captured*in*
the*QALYs*
The*ratio:*Vary*the*
threshold*used*to*judge*
the*ICER*to*reflect*factors*
not*captured*in*the*
numerator*or*denominator

7. What do we mean by MCDA?
Resources:
ISPOR-taskforce-reports-on-MCDA:--Marsh-et-al--(2016)-and-Thokala-et-al-(2016)
Hansen-&-Devlin-(2019)-MCDA-in-health-care-decision-making-Oxford-Encyclopedia-of-Economics-and-Finance
7
• MCDA-formally-incorporates-multiple-dimensions-of-value.-
• DecisionOmakers-select-relevant-dimensions-of-value-and-use-their-judgement-to-
specify-their-relative-importance-(“weights”).-
• MCDA-models-combine-each-option’s-performance-on-chosen-dimensions-of-value-
into-comprehensive-scores,-used-to-rank-options.-
• Many-different-MCDA-approaches;--supported-by-software
• Processes-to-elicit-decisionOmakers’-weights-vary-considerably-across-different-
models.-

8. Our protagonists
Adrian'Towse
Arguing'the'case'for'augmented'ICERs
Charles'Phelps
Arguing'the'case'for'MCDA'
&'plenty'of'time'for'discussion'and'debate'
from'the'floor
8

9. 9
Thanks!
email:
nancy.devlin@unimelb.edu.au
twitter:
@HEU_unimelb
@nancydevlin1

10. Multi&Criteria+Decision+Analysis+(MCDA)
vs
Cost&Effectiveness+Analysis+(CEA)
Charles(E(Phelps,(PhD
University(of(Rochester
Rochester,(NY(USA(

11. What’s'good'about'CEA?
• It#flows#directly#from#the#theory#of#maximizing#expected#utility#for#a#
single#(representative)#individual.#
• If#properly#used,#it#leads#to#efficient#use#of#resources#to#improve#
health.
• It#is#widely#used#and#generally#well=understood.#

12. What%Can’t%CEA%Do?
• Deal%with%issues%of%equity,%fairness
• Income%and%wealth%disparities
• Racial%disparities
• Regional%issues%(e.g.,%Northern%Territories%in%Canada)
• Incorporate%things%that%can’t%be%measured%in%QALYS%or%$
• Fear%factor%from%dread%disease
• Fit%with%local%health%system%or%beliefs

13. What%Does%CEA%Do%Poorly,%If%at%All?
• Contagious*diseases
• Herd*immunity
• Microbial*resistance*to*antibiotics*(AMR)
• Other*Externalities*
• Scientific*spillovers
• People*with*multiple*chronic*conditions
• Add*or*multiply*QALY*adjustments?
• The*“disabled”*issue*looms*large*in*US*policy

14. MCDA%Models%in%a%Nutshell
• MCDA%formally%incorporates%multiple%dimensions%of%value.%
• Decision8makers%select%relevant%dimensions%of%value%and%specify%their%relative%
importance%(“weights”).%
• Processes%to%elicit%decision8makers’%weights%vary%considerably%across%different%
models.%
• They%are%not%yet%perfected!%
• MCDA%models%combine%each%candidate’s%performance%along%chosen%dimensions%
of%value%into%comprehensive%scores%used%to%rank%candidates.%
• Despite%differences%in%intellectual%heritage,%these%value%metrics%often%use%simple%
linear%combinations%(using%decision8makers’%weights)%of%each%candidate’s%
performance%(perhaps%mathematically%transformed)%on%each%value%dimension%

15. In#the#simplest#form
Multi&attribute+Utility+Index+(MAUI)+for+candidate+j,+where+!" =+weight+
placed+by+decision+maker+on+attribute+i and+#"$ =+normalized+score+for+
candidate+j+on+attribute+i:+
%&'($ = Σ"!"#"$

16. HOW$MCDA$Can$Help
• It#formally#brings#“other#issues”#into#the#model
• Explicit,#transparent#to#others
• Usable#at#different#levels#of#decision=making
• System=wide#adoption#of#technologies
• Coverage#in#health#insurance#plans
• Decisions#by#individual#patients
• Which#health#plan
• Which#among#available#treatments

17. Virtues(of(MCDA
• Transparency."How"“other"issues”"are"incorporated"into"decisions"is"wholly"transparent,"specified"by"
the"weights"assigned"in"MCDA"models"to"each"attribute"in"the"decision"model."
• “Flight1simulator”1testing."Allows"people"to"test"perceived"value"of"alternative"combinations"of"
attributes"before"construction"or"purchase,"potentially"focusing"R&D"in"earlier"development"stages."
• Guiding1data1improvement."Data"imperfections"always"exists."MCDA"models"help"focus"dataC
improvement"efforts"on"those"data"that"mostCaffect"decisions,"thus"conserving"resources."
• Decision1convergence.1In"some"(but"not"all)"settings,"MCDA’s"formal"structure"can"assist"decision"
convergence,"since"goals"and"preferences"of"all"participants"are"clearly"visible"as"MCDA"weights."
Obviously,"in"some"settings,"some"parties"may"prefer"otherwise,"but"MCDA"can"have"value"in"others."
• Avoiding1cognitive1bias."The"emerging"field"of"“behavioral"economics”"has"cataloged"over"two"dozen"
common"errors"in"human"decision"making,"many"involving"steps"necessary"to"make"“intuitive”"
choices."MCDA’s"formal"structure"helps"to"bypass"most"of"these"cognitive"errors."

18. Barriers'to'Use
• “MCDA&requires&too&much&data.”&
• Response:(Yes,%MCDA%requires%more%data.%%But%the%problems,%not%the%MCDA%models,%
create%this%complexity.
• “It’s&too&easy&to&‘game’&the&results.”&
• Response:(You%can%“get%any%outcome%you%want”%by%changing%the%weights%in%the%
MCDA%model,%but%MCDA%models%actually%make%it%more%difficult%to%do%this,%not%less%
so,%since%the%decision%structures%(weights%on%attributes)%are%wholly%visible.
• “The&meaning&of&the&resulting&‘index’&is&unclear.”&
• Response:(Each%MDCA%model%has%its%own%scoring%system,%since%users%determine%their%
own%weights.%Thus%“my”%index%is%not%comparable%to%“your”%index,%but%the%weights%
are%visible,%so%the%meanings%can%be%easily%inferred.%
• “MCDA&models&are&too&complicated&to&use.”&
• Response:((Different%models%differ%hugely%in%demands%on%users.%%Working%to%
maximize%usability%is%key%to%further%expansion%of%MCDA%use.%
• “You&can’t&use&MCDA&in&situations&with&a&budget&constraint.”&
• Response:&&No%longer%true.%%Several%approaches%allow%calculating%cost/value%ratios%
just%as%done%with%ICERs%in%CEA.%

19. Next%steps%to%expand%use
• Reduce&&user&complexity
• How&many&decisions&must&“decision&makers”&make?
• Models&differ&by&factors&of&K&or&N
• Decision&making&in&groups
• Voting&methods
• Increased&data&availability
• Increased&pool&of&skilled&“users”
• Familiarity&breeds&acceptability

20. Who$can$(and$should)$do$what?
• Consensus'on'key'attributes
• Differs'by'disease
• Patient'viewpoint'essential
• Medical'specialty'groups?''Patient'advocacy'groups?'…..'????'
• Expanded'data'gathering
• FDA'rules
• Build'from'Expanded'and'Augmented'CEA
• Improved'software'for'groups'and'individuals'(ease'of'use)
• Trained'personnel'
• MPH,'Public'Policy,'Systems'Engineering,'Business,'…
• Build'into'routine'clinical'settings'where'appropriate
• Must'be'simple'patientKcentric'decision'tools

21. Improving*CEA*also*has*value
• MCDA%models%“must”%include%QALYS%
• or%other%health%outcome%measures
• In%most%cases,%should%be%first%ranked%attribute
• In%many%cases,%probably%a%significant%proportion%of%the%weight
• Opportunities%exist%to%expand%QALY%concept
• Incorporating%uncertainty%of%outcomes%(variance,%skewness,…)
• Ongoing%work%by%Lakdawalla and%Phelps%addresses%these%issues%
• Possibly%also%“scientific%spillovers)
• Requires%subjective%estimates%of%probabilities%
• Others?

22. CONCLUSION:))The)Future)is)MCDA
“On$the$plains$of$hesitation
Bleach$the$bones$of$countless$millions$
Who,$at$the$dawn$of$victory,$sat$down$to$wait,$
And$waiting….$died.”$
(George$W.$Cecil.)$

23. Thank&you&for&your&attention!

24. ohe.org
PRESENTATOINISPOR*NEW*ORLEANS
The case for Weighted /
Augmented CEA
MCDA*OR*WEIGHTED*CEA*BASED*ON*
THE*QALY?*WHICH*IS*THE*FUTURE*
FOR*HTA*DECISION*MAKING?
Adrian*Towse
Emeritus*Director*and*Senior*Research*Fellow
Visting*Professor,*London*School*of*Economics*
21ST
MAY(2019

25. Issues
ISPOR&NEW&ORLEANS IP11
25
21ST MAY(2019
●The centrality of health gain
●Additional attributes of value
●The challenge of opportunity cost
●Who is the decision maker? Welfarist versus extra-welfarist
approaches
●The UK experience of failed implementation of Value-
based pricing
●Deliberation and weighted CEA
●Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?

26. The centrality of health gain
ISPOR&NEW&ORLEANS IP11
26
21ST
MAY(2019
● Assume(it(is(measured(in(QALYs
● A(core(challenge(– can(we(have(things(that(the(health(system(or(an(intervention(provides(that(are(not(to(do(
with(health?(
● The(answer(is(“yes”,(for(example(reassurance,(or(indeed(information(that(is(not(reassuring(but(enables(life(
choices(to(be(made(– the(“value(of(knowing.”(
● But(I(would(argue(it(is(a(key(challenge.(If(it(is(not(an(attribute(that(can(be(expressed(as(a(multiple(of(health(
gain(then(think(carefully.
● This(links(to(a(related(issue.(Even(if(this(“non(health”(attribute(exists(should(the(health(system(be(paying(for(
it?([This(may(also(apply(to(attributes(that(can be(expressed(as(multiples(of(health(gain]
● Again(the(answer(can(be(“yes”(providing(we(are(clear(that(this(is(part(of(the(objectives(of(the(health(
system(to(be(paid(for(in(premiums(P be(they(funded(via(taxes,(social(insurance(or(private(insurance.(
● If(we(have(these(elements(that(are(not(a(function(of(health(gain(and(so(cannot(be(in(a(weighted(QALY(then(
we(can(have(an(estimate(of(Net(Monetary(Benefit((NMB)(or(Net(Health(Effects((NHE),(i.e.(we(can(convert(
them(into($$(or(QALYs(if(we(have(the(relevant(“rate(of(exchange”,(MRS,(or(ʎ

27. Additional attributes of value
ISPOR&NEW&ORLEANS IP11
27
21ST MAY(2019
Value
Quality(
adjusted-
life(years--
(QALYs)-
gained-
Net-health-
system-
costs
Productivity
Adherence
(improving-
factors
Value-of-
knowing
Fear-of-
contagion
Insurance-
value
Severity-of-
disease
Value-of-
hope
Real-
option(
value
Equity
Scientific-
spillovers
• Do(we(want(to(
include(them?
• Can(we(
measure?
• Can(we(avoid(
double(
counting?
• How(do(we(
aggregate?

28. The challenge of opportunity cost
ISPOR&NEW&ORLEANS IP11
28
21ST MAY(2019
●We(are(looking(at(value8for8money(or(cost8effectiveness,(so(what(are(we(comparing(our(value(to?(
What(is(our(ʎ?
●If(we(start(from(a(Garber(and(Phelps(individual(utility(function1,(then(we(have(marginal(rates(of(
substitution((MRS)(as(between(the(different(elements(of(expenditure(on(health((measured(in(QALYs)(
and(other(goods(and(services.
● Additional(elements(of(treatment(value(can(be(measured(and(we(can(construct(a(NMB(or(NHE((latter(
using(the(MRS(as(between(consumption(and(health).
●However,(we(have(third(party(payers(acting(on(behalf(of(their(enrollees /(citizens.(Two(issues:
●Heterogeneity(of(preferences.(However,(a(payer(could,(for(example,(take(the(preferences(of(their(
median(enrollee /(citizen
●The(payer(may(not(wish(to,(or(regard(it(as(efficient,(to(adopt(the(preferences(of(their(enrollees /(
citizens(for(two(reasons:
" (i)(They(have(a(budget(constraint(and/or((ii)(they(have(a(different(social(welfare(function
1(Garber(and(Phelps,((Economic(foundations(of(cost8effectiveness(analysis.(J(Health(Econ.(1997(FebZ16(1):1831.

29. Who is the decision maker? Welfarist versus extra-welfarist
approaches
ISPOR&NEW&ORLEANS IP11
29
21ST MAY(2019
●Are(we(constructing(a(CBA(for(a(“welfarist”(decision(maker?(We(are(using(a(“demand(side”(
threshold
●Or(do(we(have(an(“extraFwelfarist”(health(plan((public(or(private)(decision(maker(s)(with(a(
budget(constraint(that(does(not(allow(them(to(efficiently(adopt(all(treatments(with(a(positive(
CBA.(We(are(using(a(“supply(side”(threshold.(
●By(implication,(if(ʎ(is(less(than(the(MRS((or(citizen(WTP(for(a(QALY),(then(there(is(a(
divergence(as(between(short(and(long(run(preferences.(
●The(“extraFwelfarist”(health(plan(decision(maker(s)(can(use(one(or(more(of((i)(weights(based(
on(citizen(preferences((ii)(weights(applied(to(∆QALY(effects((iii)(an(MCDA(or(deliberative(
decision(making(process.(
●So(an(extraFwelfarist can((a)(weight(the(∆QALY(with(attributes((a)((that(seeks(to(take(
account(of(opportunity(cost((oc)((wa/woc)(and((b)(express(nonFQALY(attributes(in(money(and(
then(convert(into(QALYs(using(an(exchange(rate(of((ʎ(/(MRS)

30. The UK experience of failed implementation of Value-
based pricing (VBP)
ISPOR&NEW&ORLEANS IP11
30
21ST MAY(2019
●In(2013(the(UK(Health(Minister(sought(to(introduce(VBP.((This(included:
●A(disease(severity(adjustment(to(the(∆QALY,(with(weights(anchored(around(the(average(
disease(severity(of(patients(treated(in(the(health(plan
●A(calculation(of(the(net(social(impact(of(a(treatment,(converted(into(QALYs(using(an(
exchange(rate(of((ʎ(/(MRS)
●Conversion(into(a(price(using(the(threshold(ʎ
●The(Minister(retreated(and(handed(it(the(“hot(potato”(of(VBP(to(NICE
●NICE(replaced((i)(the(disease(severity(adjustment(with(a(proportional(shortfall(adjustment(
and((ii)(the(net(social(impact(calculation(with(an(absolute(shortfall(adjustment.(The(adjusted(
ICER(would(be(compared(with(the(threshold(range(of(£20K(to(£30K.(
●Both(were(weighted(CEA(approaches(that(could(be(operationalised.
●They(were(abandoned(because(of(equity(concerns(and(loss(of(deliberation.(

31. Deliberation and weighted CEA
ISPOR&NEW&ORLEANS IP11
31
21ST MAY(2019
●The(weights(do(not(need(to(be(pre8set.(It(can(be(left(to(the(decision8makers(in(
Committee(to(give(implicit(or(explicit(weights(to(the(factors(they(think(are(
relevant.(
●This(can(be(applied(to(the(attributes(converted(into(QALYs(using(an(exchange(
rate(of((ʎ(/(MRS).(They(can(be(downrated(or(uprated.
●The(decision8makers(in(Committee(act(as(extra8welfarists making(a(decision(in(
line(with(their(social(welfare(function((SWF).
●It(may(be(that(the(SWF(is(formed in(deliberation.(Preferences(are(not(pre8set(
but(related(to(both(the(nature(of(the(items(under(discussion(and(the(views(
formed(in(the(deliberative(process.

32. Which is the general case? Is weighted CEA a specific
example of MCDA, or vice versa?
ISPOR&NEW&ORLEANS IP11
32
21ST
MAY(2019
●Weighted(CEA(is(a(specific(example(of(MCDA.(It(is,(arguably,(“clean”.(It(focuses(on(
health(gain(and(enables(an(element(of(deliberation(in(setting(weights(for(decision(
making.(
●Or(as(the(UK(VBP(example(shows,(it(can(be(algorithmic(and(set(a(drug(price.(
●The(challenge(for(Chuck(is,(therefore,(what(do(other(forms(of(MCDA(bring(to(the(
party?

33. To#keep#up#with#the#latest#news#and#research,#subscribe#to#our#blog.#
OHE’s#publications#may#be#downloaded#free#of#charge#from#our#website.
ohe.org
OHE
Southside
105#Victoria#Street##
London#SW1E 6QT##
United Kingdom
Telephone
+44#(0)20#7747 8850
FOLLOW%US
To#enquire#about#additional#information#and#analyses,##
please contact:
Adrian%Towse
Emeritus%Director%and%Senior%Research%Fellow,%OHE
Visting%Professor,%London%School%of%Economics
atowse@ohe.org