1. Principles of Diagnostic Testing
Statistics for Research
William F. Auffermann, MD/PhD
Department of Radiology and Imaging Sciences
Emory University School of Medicine
2. Learning Objectives
• Provide an overview of the basic statistical
concepts needed to critically appraise and
perform research
3. Diagnostic Testing
• Diagnostic tests are designed to answer
specific medical questions.
• When there is concern for a medical
disease, appropriate diagnostic testing can
be used to better risk stratify patients
• The probability of a disease after testing is a
function of both pre-test probability and the
results of the test.
4. Diagnostic Testing
• Diagnostic testing may be thought of as a
way of refining the estimate for the
probability of a patient having a particular
disease.
• Understanding the principles of diagnostic
testing requires an understanding of
probability and statistics.
5. Probability and Statistics
Two Sides of the Same Coin
• Probability: assumes you know the
underlying laws of a process, and can be
used to predict outcomes
• Statistics: used to compare data with
theory/model and look at how well they
agree
7. Hypothesis
• A proposed explanation for a phenomenon‡
• A key aspect of diagnostic testing and
statistics is formulation of a good
hypothesis
‡ http://en.wikipedia.org/wiki/Hypothesis
Accessed 2014-11-13
8. Hypothesis
• Hypothesis are often paired with their
logical opposite
• The null hypothesis (H0) is considered the
default hypothesis
• The alternative hypothesis (HA) its logical
complement
9. Hypothesis
• H0: the medication does not reduce blood
pressure
• HA: the medication does reduce blood
pressure
10. Hypothesis
• Hypotheses should address the question of
interest and be testable
• Clear statement of the hypothesis is critical
for appropriate statistical testing
11. Hypothesis
• H0: mean blood pressure in treatment group
the same as control group (MBP2 = MBP1)
• HA: mean blood pressure in treatment
group lower than the control group (MBP2
< MBP1)
13. Probability
• Probability relates to the likelihood of a
particular event occurring
• There is an assumption we know the laws
governing the behavior of the process being
examined
• For example if we have a fair coin where
the probability of heads/tails are both 0.5
(equal), then we can estimate the probability
of flipping a coin and obtaining: HHTH
14. Pre/Post Test Probability
• Diagnostic testing is useful as it effects the
post test probability of a diagnosis.
• Diagnostic testing which does not
significantly effect the post test probability
may not be clinically useful
15. Pre/Post Test Probability
• Let ‘p’ represent the probability of a disease
and ‘t’ the results of a diagnostic test
p2 = LR(t) * p1
• Where p1 and p2 are the pre and post test
probabilities respectively, and LR(t) is the
likelihood ratio for the test.
• LR(t) gives probability values for both
positive and negative results.
16. Pre/Post Test Probability
p2 = LR(t) * p1
Fagan nomogram
http://http://mcmasterevidence.wordpress.c
om/2013/02/20/what-are-pre-test-
probability-post-test-probability-and-
likelihood-ratios/
Accessed 2014-11-13
17.
18. V/Q Scan
• Consider a patient with symptoms
concerning for pulmonary embolism.
• Based on the patients clinical symptoms, we
can risk stratify them for probability of
pulmonary embolism, corresponding to the
pretest probability (p1)
19. V/Q Scan
• A V/Q test is performed to better risk
stratify the patient.
• The various patterns of findings on V/Q
scan correlate with the probability of
pulmonary embolism
20. V/Q Scan
• The post-test probability is derived from
both the pretest probability and the results
of the test.
27. Probability Distributions
• There are several different probability
distributions
• Different physical and biological
phenomena can be modeled using different
distributions
• One of the most common naturally
occurring distribution is the normal
(Gaussian) distribution
29. Probability Distributions
• Based on the knowledge of a probability
distribution, it is possible to estimate the
probability of observing a range of values
31. Probability Distributions
• When performing or evaluation research it
is very important that the data being
modeled can actually be represented by the
proposed distribution
• Graphical displays of data can be helpful to
confirm this is true (frequency polygon,
histogram)
34. Statistics
The science that deals with the collection, classification,
analysis, and interpretation of numerical facts or data,
and that, by use of mathematical theories of probability,
imposes order and regularity on aggregates of more or
less disparate elements.
http://dictionary.reference.com/
Accessed 2014-11-13
35. Why Does Statistics Matter?
• Statistics provides a means of summarizing
a data set and making inferential statements
• Appropriate application can highlight
important aspects of the data
• Incorrect application can be confusing at
best, and misleading at worst
• Statistics do not ‘lie’, but they may be
misleading
36. Statistic
• A mathematical summary of a data set
• Examples include the mean (-), median (-),
mode (-), standard deviation
37. Statistic
0 5 10 15 20
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
mean (-), median (-), mode (-)
Gama(2,3)
Frequency
38. Statistic
• The selection of a statistic for representing
data should be based on the nature of the
process underlying the observations
• The statistic should be based on the model
which best represents the data
39. Statistics
• Qualitative: specific summary measures of
the data (statistics) may provide greater
clarity than the data set as a whole.
• Quantitative: Based on the underlying
theory of the process being measured,
inferential statements may be made
regarding whether the data and theory agree
42. Quantitative
• Based on known properties of the statistical
test in question and the distribution of the
data, it is possible to make statements of the
significance a result
44. P-values
• A p-value is the probability that a value
from the proposed distribution is the same
as or farther from the expected value than
the observed value.
46. P-values
• The lower the p-value, the less likely that
the observed statistical value can be
explained by the model under H0
47. P-values
• Assume you want to know if a coin is a fair
coin (equal probability of H/T after
flipping)
• You flip the coin 100 times and get H 60
times. Is the coin fair?
48. P-values
0 10 20 30 40 50 60 70 80 90 100
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
pdf
Observed
Value
Area under curve = p-value = 0.0176
49. P-values
• By convention, p-values less than or equal
to 0.05 are generally considered statistically
significant
• Note that other thresholds can and are used
• Type I error (often denoted by α ) is the
probability of rejecting the null hypothesis
based on the result of a test if H0 is in fact
true.
51. Multiple Comparisons
• P-values give the probability of an value at
least as extreme as the one observed for a
single test.
• What happens if there are multiple tests?
• Does this affect our decision to consider p-
values less than 0.05 statistically
significant?
52. Multiple Comparisons
• Consider we are looking at a set of anti-
hypertensive medications for effect on
blood pressure
• A p-value of 0.05 corresponds to a 1/20
probability
53. Multiple Comparisons
• If we examine 20 medications, we would
expect 1 to have a p-value of 0.05 or lower
by chance alone even if there were no
therapeutic effect
56. Multiple Comparisons
• It is possible to correct for multiple
comparisons
• There are several ways to perform this
correction
• Several are dependant on knowledge of the
correlation between variables
57. Bonferroni Correction
• A conservative correction assuming each
test is independent
• The threshold for significance if changed to
the overall desired significance (often 0.05)
/ number of comparisons
• New threshold = 0.05/20 = 0.0025
58. Bonferroni Correction
• This correction adjusts the type I error such
that there is α overall probability of a
positive result for any test if H0 is true
(across all tests).
61. Diagnostic Testing
• Diagnostic tests are designed to answer
specific medical questions.
• When there is concern for a medical
disease, appropriate diagnostic testing can
be used to better risk stratify patients
• Recognize that diagnostic tests are not
perfect, and even the best may misclassify
patients.
64. Confusion Table Derivations
• Sensitivity = the probability of a positive case
being marked positive
• Specificity = the probability of a negative case
being marked negative
• PPV = The probability of a positive test result
being positive
• NPV = The probability of a negative test result
being negative
65. Confusion Table Derivations
• Sensitivity
• Specificity
• PPV
• NPV
Not effected by prevalence
of disease in a population
Effected by prevalence of
disease in a population
66. Sensitivity and Specificity
• Diagnostic Testing is a compromise
between sensitivity and specificity
• Most tests offer a compromise between
these two measures
• Very often two or more tests may
complement each other (one may be high
sensitivity, the other may be high
specificity)
68. Sensitivity and Specificity
• Sensitive tests: useful for screening, test
usually negative if disease is absent
• Specific tests: useful for confirming a
diagnosis, test usually positive if disease is
present
69. Diagnostic Testing
• It is important to note that there are
instances where diagnostic testing will not
significantly alter the posttest probability
relative to the pretest probability.
70. Diagnostic Testing
• Diagnostic testing may be less useful in
instances of very low or very high
probability.
• Diagnostic tests may be thought of as most
useful in instances of intermediate
probability.
