1. Journal of Evidence-Based Medicine ISSN 1756-5391
METHODOLOGY
A methodological framework of preparing economic
evidence for selection of medicines in the Chinese setting
Xin Sun1 and Thomas Alured Faunce2
1
Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, China
2
Associate Professor and Australian Research Council Future Fellow, College of Medicine, Biology and the Environment and College of Law
(joint appointment), Australian National University, Australia
Correspondence
Abstract
Xin Sun, Chinese Evidence-Based Medicine
Center, West China Hospital, Sichuan Medicines are becoming a major component of health expenditure in China. Se-
University, No. 37 GuoXueXiang, Chengdu lection of effective and cost-effective medicines represents an important effort to
610041, China.
improve medicines use. A guideline on cost-effectiveness studies has been avail-
Tel: +86-28-85423458
able in China. This guideline, however, fails to be a practical tool to prepare and
Fax: +86-28-85422253
Email: sunyzmy@hotmail.com critically appraise economic evidence. This article discusses, in the Chinese con-
text, the approach to integrating economic component into the medicines selection,
and elaborates the methods of producing economic evidence, including conducing
Received 22 June 2010; accepted for economic reviews and primary economic studies.
publication 5 July 2010.
doi: 10.1111/j.1756-5391.2010.01087.x
Medicines are an increasingly significant component of tical tools for preparing and using economic evidence for
health expenditure in China. The medicines expenditure selection of medicines. This article offers a methodologi-
of hospitalized patients, for example, usually accounts for cal framework to address issues about economic evidence in
around 45% of their total medical bill (1). Rapid increases the context of medicines selection for a Chinese Essential
in price are rendering medicines unaffordable to many Medicines List.
Chinese patients. The demand for more efficient, quality use
of medicines has become increasingly strident, and the im- Integrating economic evidence into
plementation of an Essential Medicines List in patient care the selection of medicines
represents one of the efforts to improve medicines use.
Selection of effective and cost-effective medicines is the Integrating economic evidence in the process of medicines
key to implementing an efficient Essential Medicines List. selection for an Essential Medicines List typically starts with
In the last decade, extensive methodological work has been formulation of a focused economic question that reflects pol-
undertaken in China to improve the assessment of effective- icy maker’s perspective. It is followed with a systematic re-
ness of medicines (2, 3). However, little was done to explore view of existing studies that assesses the strength of eco-
best approaches to addressing cost-effectiveness. Despite nomic evidence. The subsequent decisions will then be made
increasing publication of cost-effectiveness studies in the on the basis of strength of economic evidence, which can be
Chinese medical literature (4, 5). the methodological quality categorized into three levels.
varies substantially. r Strong evidence without uncertainty. The existing evi-
In an effort to improve the conduct of cost-effectiveness dence provides precise and unbiased estimates of compar-
studies, China has developed a guideline on economic evalua- ative cost effectiveness, and the study settings—including
tion of medicines (6). This endeavor—which follows a stream patients, interventions, outcomes, and clinical practice
of similar attempts in other countries (7–11)—offers method- patterns—are applicable to the question being evaluated. In
ological standards of conducting economic studies. However, such case, a decision about acceptance or refutation is able
it fails to provide policy makers and researchers with prac- to be made.
156 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University

2. X Sun and TA Faunce Economic evidence for selection of medicines
Develop the research question Corresponding to the research questions, eligibility criteria
Target population, interventions, and outcomes
should be established. Eligibility criteria may be wide or
Search the economic studies Foreign and Chinese medical databases, narrow, however, should be reflected with policy makers’
Specialized database, health economics
databases perspective. That is, the patients and interventions defined,
Screen the economic studies and outcomes of interest should be relevant to the policy
Primary inclusion, and justify the exclusion
making. For example, the finding that a new drug gives a
Assess the methodological quality
Adequate – secondary inclusion, and move to
patient increased quality adjusted life years (QALYs) may
the next step;
Inadequate – exclusion
be more important than the finding it alters physiological
Assess the transferability
parameters (like blood pressure).
Acceptable – final inclusion In defining the criteria, the systematic review should in-
Poor – exclusion
Extract and adjust data clude all types of study designs (eg, modeling, trial-based
Adjustable – results applicable for drug studies, database study, and observational cohort), and all
decision-making
Analyze and summarize data Not adjustable – justify the results forms of evaluation (eg, cost-effectiveness analyses, cost-
utility analyses, cost-consequence analyses, and cost analy-
Figure 1 The flowchart of systematic review of economic evidence. ses), unless otherwise indicated by policy makers.
Methodological quality
r Moderate evidence suggesting cost-effectiveness pro-
Methodological quality, which provides the extent to which
files. The estimates of comparative cost-effectiveness are
the results are biased, is subject to study design, data collec-
either imprecise, involve moderate likelihood of bias, or
tion, and data analysis, and reporting (15–17). Tools address-
the study settings differ in part of components affecting
ing methodological quality of economic studies differ (18–
transferability of results. In such case, a decision will in-
20), and two checklists—Drummond’s checklist (28). and
volve uncertainty, and conduct of new economic studies is
Chiou’s checklist (21)—are commonly used, both consisting
desirable.
r Week evidence involving substantial uncertainty. The of similar items. Chiou’s instrument also weighs items, and
is validated externally (22). A modified Drummond’s check-
estimates of results involve significant uncertainty (either the
list incorporating three set of weights (23). demonstrated its
estimates is imprecise, the methodological quality is very
ability in rating methodological quality, while not validated.
low, or both), and/or the study settings are substantially dif-
We recommend using both checklists to examine consis-
ferent from question being evaluated. New economic studies
tency of results; in the presence of discrepancy, one may
are required in such circumstances.
consider exploring details. In presenting the risk of bias of
estimates, the overall methodological quality may be rated
Systematic review of economic and categorized into three levels (ie, high, moderate, and
evidence low).
The framework for conducting systematic review of
economic evidence on a medicine’s comparative cost- Transferability
effectiveness is presented in Figure 1. Empirical studies are
undertaken elsewhere (12–14). We discuss points with par- The extent to which the study is applicable (or transferrable)
ticular interest to the selection of medicines for subsidy under to the question of interest depends on a number of potential
a Chinese Essential Medicines List. factors (24–27). Currently, neither consensus nor instrument
is available to rate the impact of on the transferability of
results. Here, we suggest key factors and propose methods
Develop the research question and
of assessment (Table 1).
eligibility criteria
Methodological characteristics are likely to be adjustable,
A single most important step of an economic review is to de- including currency, base year, and discount rate, Methods are
velop the research question. Typically, this question has three available for adjusting these characteristics, and the adjust-
minimal components, including study population, interven- ment requires availability of sufficient primary data. The im-
tion strategies, and health outcomes of interest. Collaboration pact of patient characteristics on transferability is subtle, and
between researchers and policy makers is crucial in devel- empirical evidence manifesting this does not exist. Health
oping a well-defined research question. An example might preference is among the major factor influencing transfer-
be a study comparing the cost-effectiveness of a new heart ability. In assessing the potential impact, patients character-
failure medication against an existing marketed product at istics should be listed, and the judgment about the impact
the standard dose and price in rural or urban populations. should be justified.
JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 157

3. Economic evidence for selection of medicines X Sun and TA Faunce
Table 1 Factors influencing transferability of economic evidence
Factors Appraisal guide
Methodological characteristics
Perspective Applicable if conducted from the healthcare sector perspective, or if disaggregated data were available
from the societal perspective.
Possibly inapplicable if applying a narrower perspective, or if aggregated data are used.
Clinical setting Desirable if including study population of diverse socio-economic conditions
Potentially problem for a single healthcare centre study and international study.
Discount rate Applicable if using the same discount rate or similar range.
Costing methods Questionable if micro-costing methods are not used.
Patient characteristics
Demographics Minor impact on the transferability in the presence of age or sex differences only.
Case mix May significantly influence transferability of results. Potentially inapplicable in the presence of differential
case mix groups across countries/regions.
Comorbidity Small to moderate effect on transferability, particularly when its contribution to total costs and effects
are minor.
Health preferences Its impact on transferability varies. Sensitivity analyses may be considered to examine its impact.
Compliance Particular relevance to trial-based studies. Its impact depends on the extent of compliance. Small to
moderate influence on transferability.
Healthcare system characteristics
Unit costs/prices Large impact on transferability. Adequate adjustment might be appropriate to transfer the published
results to the Chinese setting.
Resources use Its impact depends on clinical practice variations and patient characteristics. Questionable transferability
if resources use pattern differ across regions. Adjustment is usually not sufficient. Transferable if only
a small part of resources use differs and have minimal impact on the total costs.
Clinical practice variations Unlikely transferable if clinical practice pattern differs substantially.
Unit costs, resources use, and clinical practice patterns Audiences and perspective
represent the key components affecting transferability. Unit
Typically, policy makers form the primary audience of eco-
costs usually differ across regions, and may be adjustable.
nomic studies, and health care sector perspective may be
Clinical practice patterns, which also underlie resources use,
used as the primary perspective of study, supplemented with
have the great potential of influencing transferability, and
an analysis with the societal perspective (37).
may be least adjustable. In the presence of differences of
clinical practice patterns, a summary detailing the differences
and potential impact is desirable. Indications
Ideally, the study population should cover all that are pre-
scribed with major and minor indications. Most often,
Data Analysis and Presentation economics studies focus on major registered drug indica-
Economic studies are often heterogeneous, and may differ tions. Nevertheless, the selection of indications needs to be
qualitatively (28). Four approaches are available to ana- justified.
lyze results in systematic reviews (19, 29–31), and narra-
tive summaries and permutation matrix represent two most Comparators
widely used methods. However, all these methods have dif-
ficulties with handling discrepant results that equally spread In the context of competing medications, three treatment
in the matrix. A possible solution is to use the single best strategies—including most widely existed practice, mini-
evidence. mal practice/or doing-nothing practice, and the drugs it
might replace—may be included as comparators. In the pres-
ence of large number of competing medicines, compara-
tors should at least include a reference drug listed in the
Conducting primary studies
formulary, and the most widely used alternatives (includ-
In the context of medicines selection, the absence of reliable ing “do nothing” or “minimal practice”). The decision and
and transferable economic studies necessitates the conduct use of reference drug is usually based on decision-makers’
of new studies. perspective.
158 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University

4. X Sun and TA Faunce Economic evidence for selection of medicines
Time horizon Four approaches are used to measure resources use, the
level of measurement ranging from primary data to expert
The choice of time horizon is typically made on the basis of
opinions. The choice of approach is based on study per-
question of interest. A desirable time horizon is able to cap-
spective, contribution to total costs, availability of data, and
ture important health outcomes and costs. Surrogate markers
balance between internal and external (40). We recommend
are often used in clinical studies, in which case modeling
that that resources use of significant contributions to the total
represents an alternative to acquire outcomes important to
costs should be measured. Expert opinions are the last resort
patients.
of measuring resources use.
Unit costs can be obtained through national healthcare
Study design price reports, charges or tariffs, published data, and calcu-
lation of unit costs. National Healthcare Price Report rep-
Five study designs can inform cost-effectiveness studies resents a common source of unit costs, and may be good
(33). Pragmatic trials provide reliable elements for assess- representation of price from societal perspective. Charge
ing economic efficiency of medicines, and offer advantages and tariff is potentially useful to value price from health-
of achieving desirable internal and external validity, while it care sector perspective. Naive charge is likely to over rep-
is associated with substantial costs and time. Piggy-backed resent unit costs, and an adjustment for charge-to-price
studies enjoy the high internal validity, but are susceptible to ratio may be desirable. Published data, although readily
low external validity. Probably, multicenter trial offers an al- accessible, may be invalid. In any case, the choice of
ternative to addressing the drawback. Observational studies unit costs is associated with measurement of resources
often yield “real-world” data, but are more prone to bias. Such use (41).
design is time and cost consuming. Retrospective database
analysis provides a relatively inexpensive and expedient ap-
proach. Its major advantages include a wide and large pop- Discounting
ulation over long time, and coverage of current practice. Its Most analysts agree both the health outcomes and costs be
key limitation is low internal validity. Modeling technique discounted at the same rate. A discount rate of 5% can be
offers a flexible and timely framework. However, integra- used as the reference case, and a range of 3%–6% used for
tion of data from various studies and populations mandates sensitivity analysis.
sophisticated expertise, and may render biased results.
Dealing with uncertainty
Health outcomes
The approaches to handling uncertainty depends on the na-
Study outcomes of cost-effectiveness studies include natural ture of data. A CI around cost-effectiveness ratios may be
clinical outcomes and health-related adjusted quality of life. estimated, using one of the following methods including the
The single best natural outcomes may be based on a synthe- Fiedler’s theorem method, the Taylor series or delta method,
sis of multicenter trials, secondary to observational studies nonparametric bootstrapping estimation techniques. A sen-
and expert opinions (34). In the case of cost-utility analyses, sitivity analysis is desirable, and the choice of types of sen-
the approaches to valuing outcomes differ (35, 36). Standard sitivity analyses depends. One rule of thumb is to conduct
gamble and time trade-off may be desirable through a deliv- sensitivity analysis on those variables that have important
ery of patient survey. Quality of life instruments may also be influence on results. The development of cost-effectiveness
used for valuing utility (37,38). ratios comparing cost with equality of life years gained will
become an important factor in government price negotiation
over what it will pay manufacturers whose drugs are listed
Valuing costs
on the Essential Medicines List.
The approaches to valuing costs differ (39), and can be con-
ducted in four steps, including identification of cost compo-
Data analysis and presentation
nents, measurement of resources use, identification of unit
costs, and valuing resources use. Identification of cost com- Information about resources use, unit costs, total costs, and
ponents is usually associated with the choice of perspective. health outcomes should be fully reported. In the presence of
In the context of health care sector, direct medical costs are a dominant strategy, there is no need to report incremental
measured, and resources use is collected associated with dis- ratio. The incremental ratio is reported, otherwise. The results
ease management. The impact of each cost-generating event of analyses should be presented in disaggregated manner, so
on the total costs differs. We suggest documenting details of that the results could be recalculated and examined when
those events that have significant contributions. adapting to different perspectives.
JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 159

5. Economic evidence for selection of medicines X Sun and TA Faunce
Conclusion 11. Canadian Coordinating Office for Health Technology
Assessment. A guidance document for the costing process
The development of a Chinese Essential Medicines List in (Version 1.0), Canadian Coordinating Office for Health
which the cost of listed medicines is subsidized is an impor- Technology Assessment, Ottawa, 1996.
tant public health initiative. Value for government expendi- 12. Brady BK, McGahan L, Skidmore B. Systematic review of
ture on such a list will be enhanced if pharmaco-economic economic evidence on stroke rehabilitation services.
experts recommend listing after rigorous cost-effectiveness International Journal of Technology Assessment in Health Care
analysis of new medications against existing products and 2005; 21(1): 15–21.
initiate a related price negotiation with the manufacturer. 13. Holloway RG, Benesch CG, Rahilly CR, Courtright CE. A
systematic review of cost-effectiveness research of stroke
Acknowledgment evaluation and treatment. Stroke 1999; 30(7): 1340–9.
14. Siebert U, Bornschein B, Walbert T, Dodel RC. Systematic
This study is funded by NSFC (70703025). assessment of decision models in Parkinson’s disease. Value
Health 2004; 7(5): 610–26.
References 15. Jefferson T, Demicheli V. Quality of economic evaluations in
1. Health statistics from the Ministry of Health, China. Available health care: it is time for action to ensure higher methodological
from: URL: http://61.49.18.68/statistics/digest04/s34.htm and quality. British Medical Journal 2002; 324(7333): 313–4.
http://61.49.18.68/statistics/digest04/s35.htm (accessed 20 16. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM.
December 2005). Cost-effectiveness and cost-benefit analyses in the medical
2. Li SL, Cao LY, Xiao AL,Yan L, Lan F. Evidence-based literature. Are methods being used correctly?. Annals of Internal
evaluation of essential medicines. Chinese Pharmaceutical Medicine 1992; 116(3): 238–44.
Issues 2003; 17(6): 337–9 (in Chinese). 17. Jefferson T, Demicheli V. Is vaccination against hepatitis B
3. The Centre for Drug Re-evaluation, State Food and Drug efficient? A review of world literature. Health Economics 1994;
Administration. The evaluation process of the essential 3(1): 25–37.
medicines. Available from: URL: 18. Drummond MF, Jefferson TO. Guidelines for authors and peer
http://www.cdr.gov.cn/work-prog/1.htm (accessed 11 March reviewers of economic submissions to the BMJ. British Medical
2006). Journal 1996; 313(7052): 275–83.
4. Sun X, Cao LY, Chen XZ, Lan F, Li X, Xiao AL, et al. Four 19. Jefferson T, Demicheli V, Vale L. Quality of systematic reviews
non-sedating antihistamines for allergic diseases: health of economic evaluations in health care. Journal of the American
technology assessment (protocol). Chinese Journal of Medical Association 2002; 287: 2809–12.
Evidence-based Medicine 2003; 3(1): 35–40. 20. Evers S, Goossens M, de Vet H, van Tulder M, Ament A.
5. Li YP, Sun X. An evidence-based approach to evaluation and Criteria list for assessment of methodological quality of
selection of post-marketing medicines. In:Zheng XY, Liao QJ, economic evaluations: consensus on Health Economic Criteria.
eds. Annual Advance of Pharmacy. Beijing: Chinese Medical International Journal of Technology Assessment in Health Care
Science and Technology Press, 2004. 2005; 21(2): 240–5.
6. ISPOR Chinese Medical Doctor Association 21. Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ,
Pharmacoeconomics Chapter (ISPOR Chinese MDA-PE Sullivan SD, et al. Development and validation of a grading
Chapter). China Pharmacoeconomic Evaluation Guidelines (the system for the quality of cost-effectiveness studies. Medical
Solicitation Draft), Beijing, China, 2006. Care 2003; 41(1): 32–44.
7. Commonwealth Department of Human Services and Health. 22. Spiegel BM, Targownik LE, Kanwal F, Derosa V, Dulai GS,
Guidelines for the Pharmaceutical Industry on Preparation of Gralnek IM, et al. The quality of published health economic
Submissions to the Pharmaceutical Benefits Advisory analyses in digestive diseases: a systematic review and
Committee. Commonwealth Department of Human Services and quantitative appraisal. Gastroenterology 2004; 127(2):
Health, Canberra, 1995. 403–11.
8. Garattini L, Grilli R, Scopelliti D, Mantovani L. A proposal for 23. Gonzalez-Perez JG. Developing a scoring system to quality
Italian guidelines in pharmacoeconomics. Pharmacoeconomics assess economic evaluations. European Journal of Health
1995; 7(1): 1–6. Economics 2002; 3(2): 131–6.
9. Canadian Coordinating Office for Health Technology 24. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S,
Assessment. Guidelines for Economic Evaluation of Urdahl H, et al. Generalisability in economic evaluation studies
Pharmaceuticals, Canadian Coordinating Office for Health in healthcare: a review and case studies. Health Technology
Technology Assessment, Ottawa, 1997. Assessment 2004; 8(49): iii–iv, 1–192.
10. Oostenbrink JB, KoopmanschapMA, Rutten FFH. 25. Welte R, Feenstra T, Jager H, Leidl R. A decision chart for
Standardization of Costs: The Dutch Manual for Costing in assessing and improving the transferability of economic
Economic Evaluations. Pharmacoeconomics 2002; 20(7): evaluation results between countries. Pharmacoeconomics 2004;
443–54. 22(13): 857–76.
160 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University

6. X Sun and TA Faunce Economic evidence for selection of medicines
26. Barbieri M, Drummond M, Willke R, Chancellor J, Jolain B, effectiveness in health and medicine. New York: Oxford
Towse A. Variability of cost-effectiveness estimates for University Press, 1996.
pharmaceuticals in Western Europe: lessons for inferring 34. Levels of Evidence – March 2002. Evidence-Based on Call.
generalizability. Value Health 2005; 8(1): 10–23. Available from: URL: http://www.eboncall.org/content/
27. Greiner W, Schofski O, Graf v. d., Schulenburg JM, et al. The levels.html
transferability of international economic health-economic results 35. Arnesen T, Trommald M. Are QALYs based on time trade-off
to national study questions. HEPAC 2000; 1: 94–102. comparable? A systematic review of TTO methodologies.
28. Nixon J, Khan KS, Kleijnen J. Summarising economic Health Economics 2005; 14(1): 39–53
evaluations in systematic reviews: a new approach. British 36. Dolan P. Valuing health-related quality of life: issues and
Medical Journal 2001; 322(7302): 1596–8. controversies. Pharmacoeconomics 1999; 15(2): 119–27.
29. NHS Centre for Reviews and Dissemination. Undertaking 37. Bala MV, Wood LL, Zarkin GA, Norton EC, Gafni A, O’Brien
systematic reviews of research on effectiveness. CRD’s B. Valuing outcomes in health care: a comparison of willingness
Guidance for Carrying Out or Commissioning Reviews. 2nd ed. to pay and quality-adjusted Life-years. Journal of Clinical
York: NHS Centre for Reviews and Dissemination, 2000. Epidemiology 1998; 51(8): 667–76.
30. Black WC. The CE plane: a graphic representation of 38. Perez DJ, McGee R, Campbell AV, Christensen EA, Williams S.
costeffectiveness. Medical Decision Making 1990; 10(3): A comparison of time trade-off and quality of life measures in
212–4. patients with advanced cancer. Quality of Life Research 1997;
31. Birch S, Gaffni A. Cost-effectiveness and cost utility analyses: 6(2): 133–8.
methods for the non-economic evaluation of healthcare 39. Jacobs P, Ohinmaa A, Brady B. Providing systematic guidance
programs and how we can do better. In:Geisler E, Heller O, eds. in pharmacoeconomic guidelines for analysing costs.
Managing Technology in Healthcare. Norwell, MA: Kluwer Pharmacoeconomics 2005; 23(2): 143–53.
Academic, 1996. 40. Jacobs P, Baladi J-F. Biases in cost measurement for economic
32. Drummond MF, O’Brien B, Stoddart GL. Methods for the evaluation studies in health care. Health Economics 1996; 5(6):
economic evaluation of health care programs. 2nd ed. Oxford: 525–9.
Oxford University Press, 1997. 41. Horngren CT. Cost accounting. 5th ed. Englewood Cliffs, NJ:
33. Gold M, Siegel J, Russell L, Weinstein M, editors. Cost- Prentice-Hall, 1987.
JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 161