The ABC of Evidence-Base Medicine

Dr Max Mongelli 2016
http://drmaxmongelli.weebly.com

What is EBM?
““The conscientious, explicit and judiciousThe conscientious, explicit and judicious
use ofuse of current best evidencecurrent best evidence inin
making decisions about the care ofmaking decisions about the care of
individual patients”individual patients”
Prof. David L. Sackett, 1997Prof. David L. Sackett, 1997

Primum non nocerePrimum non nocere
““First do no harm”First do no harm”
Hippocrates,Hippocrates, EpidemicsEpidemics

Sources of Evidence in Medicine
• Traditional TeachingTraditional Teaching
• TextbooksTextbooks
• Basic sciencesBasic sciences
• Observational studiesObservational studies
• Computer simulationComputer simulation
• Decision AnalysisDecision Analysis
• Case-Control StudiesCase-Control Studies
• Randomised Controlled Trials (RCT)Randomised Controlled Trials (RCT)
• Meta-analysesMeta-analyses

RCOG Classification of Evidence LevelsRCOG Classification of Evidence Levels
 1++ High quality meta-analyses of RCT's1++ High quality meta-analyses of RCT's
 1+ Meta-a. Or RCT's at low risk of bias1+ Meta-a. Or RCT's at low risk of bias
 1- Meta-a. Or RCT's at high risk of bias1- Meta-a. Or RCT's at high risk of bias
 2++ High quality meta-analyses of CC's2++ High quality meta-analyses of CC's
 2+ Well-conducted cc or cohort studies2+ Well-conducted cc or cohort studies
 2- Case-control or cohort studies with ?2- Case-control or cohort studies with ?
biasbias
 3 Case reports3 Case reports
 4 Expert opinion4 Expert opinion Dr Max Mongelli 2016

 ““Effectiveness andEffectiveness and
Efficiency: RandomEfficiency: Random
Reflections of HealthReflections of Health
ServicesServices “, 1971“, 1971
Archie Cochrane (1909-88)Archie Cochrane (1909-88)

Randomized Controlled TrialsRandomized Controlled Trials
 ““Gold standard” in evaluating newGold standard” in evaluating new
therapies or surgical techniquestherapies or surgical techniques
 May also be applied to new diagnosticMay also be applied to new diagnostic
teststests

Objectives of RCT:Objectives of RCT:
 Minimize bias by randomisationMinimize bias by randomisation
 Achieve statistical power throughAchieve statistical power through
adequate sample sizeadequate sample size
 ““Blinding “ - single or doubleBlinding “ - single or double
 Analysis by intention to treatAnalysis by intention to treat

RandomisationRandomisation
 Several techniques availableSeveral techniques available
 Computer software linked to centralComputer software linked to central
monitoring stationmonitoring station
 ““Block “ randomisationBlock “ randomisation
 Sealed envelope methodSealed envelope method

What about observational studies?What about observational studies?

RCT’s and Observational StudiesRCT’s and Observational Studies
• Two studies published in the NEJM in 2000 suggested that
RCTs and observational studies overall produced similar
results
• JAMA 2001: “discrepancies beyond chance do occur and
differences in estimated magnitude of treatment effect are
very common”
• RCTs may be unnecessary for treatments that have
dramatic and rapid effects relative to the expected

RCT’s and Industry FundingRCT’s and Industry Funding
•RCT’s funded by industry are significantly more likely to
report positive results
•Possibly due to publication bias
•RCTs may be unnecessary for treatments that have
dramatic and rapid effects relative to the expected

RCT’s and Statistical ErrorRCT’s and Statistical Error
• Type I error – “false positive”
• Type II error – “false negative”
• Sample size calculations often inaccurate

Diagnostic TestsDiagnostic Tests

2 X 2 Table2 X 2 Table
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Sensitivity = TP =Sensitivity = TP =
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Sensitivity = TP = a/(a+c)Sensitivity = TP = a/(a+c)
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 False Positive Rate = FP =False Positive Rate = FP =
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 FP = b/(d+b)FP = b/(d+b)
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Specificity = 1 - FP =Specificity = 1 - FP =
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Specificity = 1 - FP = d/(d+b)Specificity = 1 - FP = d/(d+b)
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Positive predictive value (PPV) =Positive predictive value (PPV) =
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 PPV = a/(a+b)PPV = a/(a+b)
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 Negative predictive value (NPV)Negative predictive value (NPV)
==
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

 NPV = d/(c+d)NPV = d/(c+d)
Disease
present
Disease
absent
Test
Positive
a b
Test
Negative
c d

PPV and PrevalencePPV and Prevalence
 Steep drop in positive predictiveSteep drop in positive predictive
value as disease prevalencevalue as disease prevalence
decreasesdecreases

PPV and PrevalencePPV and Prevalence
 PPV =PPV = (sens x prev)(sens x prev)
(sens x prev +(1 - spec)x(1 - prev))(sens x prev +(1 - spec)x(1 - prev))

The Likelihood RatioThe Likelihood Ratio
 Single value to indicate theSingle value to indicate the
clinical utility of a testclinical utility of a test
 Independent of prevalenceIndependent of prevalence
 LR = Sensitivity/(1- Spec.)LR = Sensitivity/(1- Spec.)
 LR >8 : tests usuallyLR >8 : tests usually
clinically usefulclinically useful

The Likelihood RatioThe Likelihood Ratio
 LR is an odds modifier:LR is an odds modifier:
 Posterior odds =Posterior odds =
prior odds x LRprior odds x LR

Odds and ProbabilityOdds and Probability
 Inter-convertible:Inter-convertible:
 Odds = p/(1-p)Odds = p/(1-p)

Can tests be combined ?Can tests be combined ?
 Rare conditions: high rates ofRare conditions: high rates of
false positivesfalse positives
 Lead to excessive unnecessaryLead to excessive unnecessary
interventionintervention
 Can be reduced by combiningCan be reduced by combining
tests e.g. intrapartumtests e.g. intrapartum
monitoringmonitoring

Impact of new diagnosticImpact of new diagnostic
test on clinical outcomes:test on clinical outcomes:
 RCTRCT
 Cohort studyCohort study
 Case-control studyCase-control study
 Before and after studyBefore and after study

SYSTEMATIC REVIEWSSYSTEMATIC REVIEWS

"It is surely a great criticism of our"It is surely a great criticism of our
profession that we have not organisedprofession that we have not organised
a critical summary, by specialty ora critical summary, by specialty or
subspecialty, adapted periodically, ofsubspecialty, adapted periodically, of
all relevant randomized controlledall relevant randomized controlled
trials."trials."
Archie Cochrane, 1972Archie Cochrane, 1972

Role of systematic reviewsRole of systematic reviews
 Before commencing a new project: to determineBefore commencing a new project: to determine
whether further studies are reallywhether further studies are really indicated: ‘state-indicated: ‘state-
of-the-art’ literature review.of-the-art’ literature review.
 Gain in statistical power for average estimates.Gain in statistical power for average estimates.
 'Cumulative' meta-analysis can determine when'Cumulative' meta-analysis can determine when
further studies are nofurther studies are no longer indicated.longer indicated.
 Design of subsequent studies.Design of subsequent studies.
 Setting policy for treatment and health care –Setting policy for treatment and health care –
making the best use of themaking the best use of the data available.data available.

Can Studies be Combined?Can Studies be Combined?
 Identification of optimal inclusion criteria can beIdentification of optimal inclusion criteria can be
difficult.difficult.
 The most critical step is choosing the appropriateThe most critical step is choosing the appropriate
research question.research question.
 A fairly general question is more preferable to a veryA fairly general question is more preferable to a very
specific one.specific one.
 Tukey : "...far better an approximate answer to theTukey : "...far better an approximate answer to the
right question, than an exact answer to the wrongright question, than an exact answer to the wrong
question.."question.."

Publication BiasPublication Bias
 Entire research studies may fail to reach publicationEntire research studies may fail to reach publication
because of the nature of the results.because of the nature of the results.
 Identification of unpublished trials can be veryIdentification of unpublished trials can be very
difficult - in one study it accounted for 22% of thedifficult - in one study it accounted for 22% of the
papers included in the meta-analysis.papers included in the meta-analysis.
 Failure to publish rests with the investigators ratherFailure to publish rests with the investigators rather
than editors.than editors.

 Comparison of the meta-analyses of smaller studiesComparison of the meta-analyses of smaller studies
with the corresponding result of the largest study.with the corresponding result of the largest study.
 30 meta-analyses including a total of 185 randomised30 meta-analyses including a total of 185 randomised
controlled studies (RCT) obtained from the Cochranecontrolled studies (RCT) obtained from the Cochrane
pregnancy and childbirth database. The meta-pregnancy and childbirth database. The meta-
analyses were only included if they had at least oneanalyses were only included if they had at least one
trial with a total sample size of over 1000.trial with a total sample size of over 1000.
 Calculations differ from the Cochrane database inCalculations differ from the Cochrane database in
that the largest trial was excluded, this being used asthat the largest trial was excluded, this being used as
the 'gold standard' for outcomethe 'gold standard' for outcome
PREDICTIVE ABILITY OF META-ANALYSESPREDICTIVE ABILITY OF META-ANALYSES
Villar et al, Lancet 1995Villar et al, Lancet 1995

 There was total agreement between the meta-
analysis and the largest study in 18/30 (60%
[C.I. 42-78]) of comparisons.
 There was partial agreement between the
meta-analysis and the largest study in 6/30
(20% ) of comparisons.
 There was disagreement in 6/30 (20% [C.I. 6-
34] ) of comparisons.
PREDICTIVE ABILITY OF META-ANALYSESPREDICTIVE ABILITY OF META-ANALYSES
Villar et al, Lancet 1995Villar et al, Lancet 1995

The Cochrane CollaborationThe Cochrane Collaboration
• Established in 1993 by Sir Iain Chalmers
• International: 100 countries
• Independent
• Not-for-profit
• Over 27000 contributors

RANZCOG and EBMRANZCOG and EBM
“RANZCOG endorses the principles of Evidence-
based Medicine and recognizes the NHMRC levels
of evidence and grades of recommendations”
College Statement C-Gen 15, Nov. 2009

…a scientific idea can never be proven true,
because no matter how many observations seem to
agree with it, it may still be wrong. On the other
hand, a single contrary experiment can prove a
theory forever false…
Sir Karl Popper

The ABC of Evidence-Base Medicine

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (19)

Ähnlich wie The ABC of Evidence-Base Medicine

Ähnlich wie The ABC of Evidence-Base Medicine (20)

Mehr von Dr Max Mongelli

Mehr von Dr Max Mongelli (14)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The ABC of Evidence-Base Medicine