1. COHG Editors’ &
Clinical Advisors’
Meeting
23 October 2013
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

2. Agenda
COHG Editors & Clinical Advisors Meeting
23 October 2013
1. Welcome
2. Role of Editor
3. Prioritisation of topics
4. Title registration – discussion of proposed titles
5. Introduction to MECIR
6. Screening of reviews/ Tips for good practice
7. Worked example – flossing review:
a. Abstract
b. Risk of Bias
c. Outcomes
d. Summary of findings
e. Consistency across review

3. The role of a
Cochrane OHG
Editor
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

4. COHG’s Editorial team









Co-ordinating Editors
Managing Editor/Deputy Managing Editor
Trials Search Co-ordinator (Feedback Editor)
Consumer Co-ordinator
Editors
Clinical Advisors
Referees
Research/Editorial Support Co-ordinator
Administrative Assistant

5. COHG’s membership
To support review teams:
805 registered-COHG authors

6. Aim of session
 To explore the role of Editors
 Minimum requirements
 Input into editorial process
 Role of Contact Editor
 Open discussion

7. Editors
 Minimum requirements:
 Completed a review prior becoming an editor
 Attend at least two Cochrane/COHG events a year
(e.g. editorial meetings, review proposal
meetings, regional meetings, colloquia, training days…)
 Input into editorial process
 Act as Contact Editor

8. Editorial process input
 Selecting and shaping the Cochrane Reviews
prepared by COHG
 new review proposals
 priorities / emerging areas of research
 Providing comments on review drafts

9. Act as Contact Editor
 Allocated reviews within specialty;
 To provide ad-hoc methodological and/or content advice to
review teams throughout entire editorial process;
 Judge whether the review is ready for peer review and help
identify appropriate referees;
 Look over submitted comments; outline expected revisions;
advise authors on response to referee comments;
 Examine revised reviews and confirm whether review should
go forward for signing off/publication.

10. Discussion
 What aspects of the Editor’s role could be made
easier?
 List of assigned reviews?
 Improved checklist form?
 More/less time for comments?
 Sufficient support from Editorial base?
 Unmet training needs?
 Ways to keep up-to-date with developments in
Cochrane or COHG?

11. Prioritisation of
Cochrane OHG Reviews
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

12. Funding providers
 Base-line salaries (NIHR)
 Global Alliance (donations from dental bodies)
 British Orthodontic Society, UK
 Royal College of Surgeons of Edinburgh, UK
 New York University College of Dentistry, USA
 British Society of Paediatric Dentistry, UK
 National Center for Dental Hygiene Research &
Practice, USA
 Canadian Dental Hygienists Association, Canada
 British Society of Periodontology, UK
 British Society of Oral Surgeons, UK
 Mayo Clinic, USA

13. Prioritisation of reviews
 Importance of our reviews asking relevant
questions
 Important reviews are kept up-to-date
 Two international prioritisation exercises so
far:
 Orthodontics and paediatric dentistry

14. Paediatric prioritisation
exercise process
1. Invitation to all COHG paediatric authors;
2. Invitation to international expert panel;
3. Two teleconferences;
4. Final list of reviews

15. Expert panel members:
paediatric dentistry











Paul Ashley, UK
Joel Berg, USA
Michael Casas, Canada
Urshla Chaudhry (BSPD), UK
Ben Cole, UK
Michael Cranfield, UK
Ivana Čukovid-Bagid, Croatia
Peter Day, UK
Chris Deery, UK
Monty Duggal, UK
Mike Harrison, UK











Marie-Therese Hosey, UK
Milton Houpt, USA
Norbert Krämer, Germany
Jessica Lee, USA
Nick Lygidakis, Greece
David Manton, Australia
Jeanette Mooney, UK
Linda Rosenberg, USA
Svante Twetman, Denmark
Jaap Veerkamp, Netherlands
Richard Welbury, Scotland

16. Results of paediatric
prioritisation exercise
 Prevention
 Fluoride
varnishes, gels, mouthrinses, toothpastes
(8)
 Sealants (2)

17. Results of paediatric
prioritisation exercise
 Treatment
 Different methods for treating and
managing decay in primary and permanent
dentition (6)
 Pulp treatment (1)
 Preoperative analgesics (1)

18. Results of paediatric
prioritisation exercise
 Additional topics (with reviews)
 Primary school-based behavioural
interventions for preventing caries
 Hypnosis for children undergoing dental
treatment
 Sedation of children undergoing dental
treatment

19. Results of paediatric
prioritisation exercise
 Additional topics (no reviews)
 Complex interventions for preventing caries
 Diagnosis of caries
 Management of early caries lesions in
children

20. Expert panel discussion:
paediatric guidelines
 Paediatric experts wanted to collaborate over
guidelines
 Not what we are here for, but we thought of
an idea to help:
 Guideline repository

21. Guideline repository
 The purpose of the repository is to:
 help identify priority review topics that could
inform guideline development;
 identify areas of duplication/overlap, where
evidence tables could be shared between
Guideline Development Groups (GDGs);
 increase stakeholder involvement in guideline
development by widening dissemination.

23. Guideline implementation
 We already liaise with NICE and SIGN to ensure we
are prioritising the reviews they need
 Fluoride varnish review
 Initial review published in 2002 (9 trials)
 Update published July 2013 (22 trials)

24. Prioritisation of reviews
 Prioritisation exercises need to:
 include all stakeholders
 cover whole of oral health

25. Introduction
to MECIR
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

26. MECIR
Methodological Expectations of Cochrane
Intervention Reviews
 Methodological standards for the conduct of new Cochrane
Intervention Reviews
 December 2012
 Standards for the reporting of new Cochrane Intervention
Reviews
 December 2012
 Standards for the reporting of Plain Language Summaries in
new Cochrane Intervention Reviews (PLEACS)
 February 2013

27. Standards for conduct

28. Standards for conduct

29. Standards for reporting

30. Standards for reporting

31. Reporting of PLS (PLEACS)

32. Reporting of PLS (PLEACS)

33. Aim
 To highlight some of the key features of a
Cochrane Review and apply MECIR standards
where appropriate:





Abstract
Risk of Bias
Outcomes
Summary of Findings
Consistency across the review

34. Worksheet
 Sambunjak D, Nickerson JW, Poklepović T, Johnson
TM, Imai P, Tugwell P, Worthington HV.
Flossing for the management of periodontal
diseases and dental caries in adults.
Cochrane Database of Systematic Reviews
2011, Issue 12. Art. No.: CD008829.
DOI: 10.1002/14651858.CD008829.pub2.

35. Abstract
 You’ve been provided with:
 A set of MECIR standards for the reporting of
abstracts
 An abstract for the flossing review
 Read through the abstract and consider
whether the reporting of the results and
conclusion in the abstract complies with the
MECIR standards.

36. Abstract
 Do the conclusions follow on from the
results?
 What changes, if any, would you make?

37. Risk of Bias
 Chapter 8 of the Cochrane Handbook
 It’s about validity of findings
 It’s not about ‘quality’
 A main component of the Summary of Findings

38. Example RoB table
Bias
Random sequence generation
(selection bias)
Allocation concealment
(selection bias)
Blinding of participants and
personnel (performance bias)
Blinding of outcome assessment
(detection bias)
Incomplete outcome data
(attrition bias)
Selective reporting
(reporting bias)
Other bias
Authors’
Support for judgement
judgement

39. Further examples
 Children were sequentially allocated. The first
child who came for examination to the first
group, second child to the second group
Random sequence generation:
HIGH - systematic method, not true randomisation

40. Further examples
 The study was conducted in a single blind
fashion
One of the blinding domains, but we don’t know
which one:
UNCLEAR – who is blinded?

41. Further examples
 An envelope was opened, in which it indicated
whether the patient should receive the bioadhesive gel or not
Allocation concealment:
UNCLEAR – is it opaque, sealed and sequentially
numbered, and were they centrally/remotely
assigned???

42. Further examples
 22 participants had amalgam restorations
whilst 23 had composite restorations. 8/47
teeth had restoration failures in the amalgam
group compared to 6/52 in the composite
group.
Selective reporting:
HIGH - reported at tooth level rather than at
patient level, ignoring the paired nature of the
data and clustering of teeth in the mouth

43. Further examples
 In a trial of drug X versus placebo to treat oral
mucositis, 13/35 were withdrawn from the
drug X group, 7 of these due to lack of
efficacy.
25/34 were withdrawn from the placebo
group, 17 due to lack of efficacy.
An intention-to-treat (ITT) analysis was carried
out including those remaining.
Incomplete outcome data:
HIGH – only 9 of the 34 in placebo group analysed

44. Summarising RoB

45. Which outcomes?
 Include ‘all outcomes that are likely to be
meaningful to clinicians, patients
(consumers), the general
public, administrators and policy makers….’
 may not have considered/evaluated all in trial
 unlikely to be reported in same publication
 useful to acknowledge even if not recorded/
reported

46. Prioritisation
 What are the main outcomes of interest for decision
making?
 Importance of outcomes from clinicians, consumer
advisory groups, guidelines, scoping search
 COMET initiative
 ≤3 primary outcomes (beneficial and adverse)
 Info for SoF table – key information
 ‘There should be no more than seven main outcomes, which
should generally not include surrogate or interim outcomes.’
 Directly relevant to review question, not ‘what the
review found’

47. Outcomes should…
 be well-defined measuring a single concept
 or where an outcome is complex, this should be clearly specified
 specify the timing of measurement
 clinically important period of measurement
 multiple observations introduce unit of analysis issues
 specify the nature of measurement
 subjective or objective outcomes, validated scale of
measurement, consistent with other reviews in same clinical area?
 be directly measured wherever possible, avoiding surrogate
outcome measures
 have relevance for future updates of the review
 The outcomes of a review should evaluate both the beneficial and
negative effects of the interventions.

48. Problem illustration

"Placement of a preformed metal crown in one tooth compared with caries
removal followed by placement of a filling material or no treatment“

The main outcome measures for children and carers were long-term freedom from
the main symptom of dental decay - pain.


Primary outcome measures
freedom from clinical or radiographic signs or symptoms of pulp pathology
including pain/pulp infection/discharging sinus/swelling Single or multiple outcomes
time until filling or crown needs to be replaced or requires further intervention
proportion of filled or crowned teeth retained until appropriate age of shedding



49. Problem illustration

"Placement of a preformed metal crown in one tooth compared with caries
removal followed by placement of a filling material or no treatment“

Other measures of success recorded and analysed
 absence of clinical or radiographic evidence of secondary caries




Different outcomes (surrogate?)
other clinical signs of pathology (fracture of tooth or filling, wear of
crown, inflammation of gingival (gum) tissue) or multiple outcomes
Single
patient satisfaction
costs to patient and provider
adverse events Any adverse events or those likely to be related to the intervention?

50. Possible solutions
Primary outcome measures :
 Pain from dental decay (after the procedure) at time points x, y and z
 Treatment success at x year (defined as freedom from clinical or radiographic signs or
symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling)
 Time until filling or crown needs to be replaced or requires further intervention
 Proportion of filled or crowned teeth retained until appropriate age of shedding
Other measures of success recorded and analysed:
 Absence of clinical evidence of secondary caries (add definition how clinical evidence
of secondary caries are to be measured). Where studies do not report clinical
outcomes for secondary caries, radiographic evidence will be reported if available.*
 Inflammation of gingival (gum) tissue
 Peri-operative pain - Pain (as rated by children) during the procedure (adverse event)
 Bleeding ( adverse events)
 Patient satisfaction
 Costs to patient and provider
*This means that only one of these will be reported if both have data are available in the studies.
If only radiographic is available, then this will be included as a surrogate for the real outcome of interest.

51. Comparable review
Primary outcomes
The main outcome of interest was success or failure of the replacement or
repair restoration and associated tooth as assessed by clinical examination.
The primary outcome measures were therefore the clinical acceptability or
unacceptability of each restoration, defined by the USPHS criteria, Ryge criteria
or modifications of these scales, and assessed by clinical examination.
It was anticipated that this would be recorded as success or failure of the
restoration, and/or that further repair or replacement of the restoration was
necessary.
 Other outcome measures indicating the failure of a replacement or repaired
amalgam restoration included the following (occurring in relation to the
repaired/replaced amalgam):
 placement of an additional restoration e.g. crown or inlay;
 root filling;
 clinical symptoms e.g. pain, swelling, diagnosis of pulpitis, abscess
formation, and extraction of the tooth.
Studies should have determined success or failure according to the same
criteria used in the decision to replace or repair the restoration.

52. Comparable review
Secondary outcomes
In addition to the main outcome measure, any outcomes reported
perioperatively (e.g. pain/discomfort) and postoperatively (i.e. within 48 hours;
e.g. pain/discomfort) were to be recorded.
Where any other outcomes were presented e.g. related to patient experience
or aesthetics these were also to be recorded.
Timing of outcome assessment
The decision on which outcome period to use for the review was to be based
on the most commonly reported period(s) of assessment amongst studies
meeting the inclusion criteria.
Outcome data from all periods of follow-up were to be included, but where
the period of follow-up differed between studies, this was to be categorised as
medium-term (less than 5 years) or long-term (5 years and above).
Time-to-event (survival data) was to be collected and analysed where
available.

53. Study eligibility
Remember!
Reporting of outcomes should rarely determine the
eligibility of studies for a review.
 Studies shouldn’t be excluded because they provide
no 'usable' data.
Even in absence of any information for a review
outcome there will still be an entry into the SoF
table.
 Look for ‘expected’ key outcomes in the clinical area
are routinely measured

54. Summary of Findings
Based on work presented by
Nancy Santesso, Quebec 2013
The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

55. Summary of Findings
Developed PICO
Located studies
Abstracted data from studies
Assessed the risk of bias in the studies
Synthesised the data for each outcome
Assessed heterogeneity and publication bias
Assess the evidence and draw conclusions

56. Summary of Findings
How do we:
 interpret results and draw conclusions?
 GRADE
 present results to reader/users?
 Summary of Findings tables

57. Summary of Findings
 Two main concepts
when interpreting
results:
 magnitude of
effect
 confidence in
that effect

58. Summary of Findings

59. Summary of Findings

60. Summary of Findings

61. Determinants of quality of evidence/
confidence in effect
Consider 5 factors to evaluate the quality of the
evidence:
 Risk of bias
 Inconsistency (or heterogeneity)
 Indirectness (PICO and applicability)
 Imprecision
(number of events and confidence intervals)
 Publication bias
See also Cochrane Handbook, Chapter 12

62. Pulling it together
Quality of Evidence varies from:
HIGH

MODERATE



LOW
VERY LOW
Confidence

63. Pulling it together
Quality
Interpretation
We are very confident that the true effect lies close to that of the
estimate of the effect.
We are moderately confident in the effect estimate:
Moderate The true effect is likely to be close to the estimate of the effect,
but there is a possibility that it is substantially different.
Our confidence in the effect estimate is limited:
The true effect may be substantially different from the estimate
Low
of the effect.
We have very little confidence in the effect estimate:
Very low The true effect is likely to be substantially different from the
estimate of effect.
High

64. Systematic review and clinical question information:
Participants, interventions and comparisons

65. Outcomes

66. Outcomes
Present important outcomes:
 Select from primary outcomes identified a priori
in protocol or those important to making
recommendations
 Choose patient-important outcomes
 Include outcomes, whether you have results or not
 Describe the outcomes
(e.g. Provide scale range and indicate a high score)

67. Results: Absolute effects
 What happens to people with, and without, intervention?

68. Results: Relative effects

69. Results: Number of
participants/studies
 Only some studies contributed information about an outcome

70. Results:
Quality of the evidence

71. Results: Comments

72. Results: Footnotes
 Clarification
 Judgements
 Transparency

73. Conclusions
Considerations when making conclusions
 Do I believe the results from these studies?
Risk of bias
 Are the results consistent across studies?
Inconsistency
 Are these all of the studies?
Publication bias
 Is this effect size precise?
Imprecision
 How do these results apply?
Applicability, directness

74. Flossing review: SoF
 You are provided with a SoF table for the
flossing review.
 Two forest plots showing the results for
gingivitis and plaque at 1 month.
 Examine the SoF and the Forest Plots for
discussion

75. Internal consistency
 You are provided with:
 MECIR standards that relate to the conclusions
from the main text of the review
 the conclusions from the flossing review
 First consider the information presented in
the conclusion alone, and then compare it to
that in the abstract and summary of findings

76. Internal consistency
• What changes, if any, do you think are necessary
in the conclusion?
• How does the information in the conclusion
compare to that in the abstract and summary of
findings?
• Are there any additional changes you would make
and why?

77. Discussion
 What aspects of the Editor’s role could be made
easier?
 List of assigned reviews?
 Improved checklist form?
 More/less time for comments?
 Sufficient support from Editorial base?
 Unmet training needs?
 Ways to keep up-to-date with developments in
Cochrane or COHG?