Core Outcome Measures in Effectiveness Trials

1. Core Outcome Measures in Effectiveness Trials
www.comet-initiative.org
Dr Aoife Waters
MRC North West Hub for Trials Methodology Research
University of Liverpool, UK

2. Outline
• Background
– What is the problem?
– Why do we need a more scientific approach to
outcomes?
• Core Outcome Sets (COS)
• The COMET Initiative

3. Why are outcomes important?
• Interventions are compared in RCTs by measuring
differences in patient outcomes between the groups
• Selection of appropriate outcomes is therefore crucial in
order to assess the effectiveness of an intervention
• “Clinical trials are only as credible as their outcomes”
(Tugwell 1993)

4. What is the problem?
• Several tens of thousands of research studies are underway and 500+ are
published every week
• Identifying effectiveness of interventions can be challenging because studies in
the same clinical area or of comparable interventions describing findings in
different ways
• It’s also difficult for trials to affect policy and healthcare decision making unless
they are designed in ways that reveal meaningful answers for patients
• Measurement and reporting of outcomes needs to be tidied up if research is to
achieve its aim of helping practitioners and patients to improve health care and
health

5. Cosmetic Outcomes Systematic Review
Aspects of cosmesis assessed
Potter et al. Assessment of Cosmesis After Breast Reconstruction Surgery: a Systematic Review. Ann Surg Oncol (2011) 18:813–823

6. Problems with outcomes in six
important Cochrane reviews
5 most accessed in 2009*:
• Preventing childhood obesity
• Treating childhood obesity
• Promoting activity in children
• Preventing falls in the elderly
• Preventing Type 2 DM
Most cited in 2009*:
• Nicotine replacement therapy
*Tovey D. Impact of Cochrane Reviews [editorial]. The Cochrane Library 2010 (7 July)

7. “The studies … varied greatly in intervention design, outcome
measurements and methodological quality.” (Preventing childhood obesity)
“Appropriate short- and long-term outcomes need to be defined for
children and youth at various weight levels, rather than using conventional
or adult-oriented outcomes.” (Treating childhood obesity)
“The studies were heterogeneous in terms of study design, quality, target
population, theoretical underpinning, and outcome measures, making it
impossible to combine study findings using statistical methods..”
(Promoting activity in children)
“No study reported relevant data on diabetes and cardiovascular related
morbidity, mortality and quality of life..” (Preventing Type 2 DM)
“We sought data for rate of falls, number of people falling, and number of
people sustaining a fracture. However, few studies provided fracture data.”
(Preventing falls in the elderly)
“Definitions of abstinence varied considerably ... In five studies it was unclear
exactly how abstinence was defined.” (Nicotine
replacement therapy)

8. Outcome reporting bias
-the selection of a subset of the original recorded outcomes in a study,
selected on the basis of the results, for inclusion in publication
• ORB suspected in at least one trial in 34% of 283 reviews (Kirkham et al,
BMJ 2010)
•42 significant meta-analyses
– 8 (19%) would not have remained significant
– 11 (26%) would have overestimated the treatment effect by > 20%
• Outcomes that are statistically significant are more likely to be fully
reported
OR 2.2 to 4.7 (Dwan et al, PLoS ONE 2008)

9. Core outcome sets
– an agreed standardised set of outcomes that should be
measured and reported, as a minimum, in all clinical
trials in specific areas of health or health care
•Disease/condition specific
•All treatment types or a particular intervention
•Should consider both benefits and harms
•The minimum – expect others to be collected
•Focus of effectiveness trials
•Involve stakeholders in their development
•Relevant within routine clinical practice

10. Advantages of COS
•Increases consistency across trials
• Maximise potential for trials to contribute to meta-
analyses of these key outcomes
•Reduction in selective reporting
• Much more likely to measure appropriate outcomes due
to early involvement of stakeholders
•More likely to reach a conclusion about an interventions
effectiveness more quickly

11. The COMET (Core Outcome Measures
in Effectiveness Trials) Initiative
• Brings together people interested in the development of COS
• Facilitate and promote development and application of COS
• Liverpool, 2010; Bristol, 2011
- Trialists, systematic reviewers, health service users, clinical
teams, journal editors, trial funders, policy makers, regulators

12. The COMET (Core Outcome Measures
in Effectiveness Trials) Initiative
• ‘What’ to measure
– Guidance on methods for developing core outcome sets,
including patient involvement and funding applications
– Providing a reporting standard for COS
• ‘How’ to measure (validity, reliability, feasibility)
– PROMIS
– COSMIN
– TREAT-NMD

13. COMET website and database
• COMET website makes it easier to find existing research,
helping
• funders who may wish to fund work in this area avoid
duplication
• researchers who want to know what developments
are underway
www.comet-
initiative.org/studies/search

14. COMET website and database
• Work is ongoing to identify, collate and maintain
relevant resources in an online searchable database
• 130 completed projects in various areas of health
• COS development is planned or ongoing in 25 clinical
areas, with a further 32 in discussion
www.comet-
initiative.org/studies/search

15. Search results

16. Impact
• In the months since the launch of the COMET website and
database (August 2011)
– 1335 searches have been undertaken
– 3874 individuals visited (6679 visits, 27791 page views)
– 91 countries visiting the site

17. Development of core outcome sets:
issues to consider (submitted)
• Scope
• Identifying existing knowledge
• Stakeholder involvement
• Consensus methods
• Achieving global consensus
• Implementation
• Regular review, feedback, updating

18. What next?
• SR and survey
– Identify COS (and other relevant work)
– Describe methods used for COS development
– Survey COS developers
• Cochrane CRG survey
• Funding applications
• COMET III
• PPI meeting

19. Stakeholder buy-in: NIHR HTA
‘Where established Core Outcomes exist they should be
included amongst the list of outcomes unless there is
good reason to do otherwise. Please see The COMET
Initiative website at www.comet-initiative.org to
identify whether Core Outcomes have been
established.'

20. Professor Hywel Williams, Chair of the NIHR HTA
Commissioning Board: ‘Patients and professionals making
decisions about health care need access to reliable
evidence. The new COMET database will help researchers
across the NIHR family and beyond when choosing the
outcomes to include in the studies that will establish this
evidence base'.

21. In conclusion
• Currently little consistency
– Unacceptable waste of data
– Need to make things better
• It is vital to collect important outcomes in all trials
– Especially outcomes important to patients
– There should be a more scientific approach to outcomes
• Growing activity in development of core outcomes and
support for COMET
• Improving the quality of evidence to support informed
healthcare decision and inform policy

22. www.comet-initiative.org
Elizabeth Gargon
e.gargon@liv.ac.uk
Twitter: @COMETinitiative
Acknowledgments
 COMET Management Group: Doug Altman, Jane Blazeby, Elizabeth
Gargon, Mike Clarke, Paula Williamson
 Funding: MRC HTMR Network, now MRC
 Collaborators: Peter Tugwell, Maarten Boers, Caroline Terwee, Holger
Schunemann, Michael Rose, Sunita Vohra, Roberto D’Amico, Lorenzo
Moja