Dreaming Music Video Treatment _ Project & Portfolio III
Clinical Trial Statstics 2016
1. EBM: Clinical Trial Statistics
Stefan Tigges MD MSCR
Department of Radiology, Emory
University 1
2. 2
External Industry RelationshipsExternal Industry Relationships ** Company Name(s)Company Name(s) RoleRole
Equity, stock, or options in biomedicalEquity, stock, or options in biomedical
industry companies or publishersindustry companies or publishers****
General Electric and Microsoft Stockholder
Board of Directors or officerBoard of Directors or officer None
Royalties from Emory or from externalRoyalties from Emory or from external
entityentity
None
Industry funds to Emory for my researchIndustry funds to Emory for my research None
OtherOther None
*Consulting, scientific advisory board, industry-sponsored CME, expert witness for company, FDA
representative for company, publishing contract, etc.
**Does not include stock in publicly-traded companies in retirement funds and other pooled
investment accounts managed by others.
Stefan Tigges,
Personal/Professional Financial Relationships with Industry within the past year
3. Lecture/Reading Goals and Objectives
• Define: probability, distribution, variability and
central tendency.
• Explain how a sample may be biased and the
difference between bias and random sampling error.
• Explain how and why hypothesis testing and
statistical inference are used in clinical trial analysis.
• Define: confidence intervals, statistical significance,
type I error, type II error, power, p-value, alpha and
beta.
• Describe the effect of increasing sample size on type I
and type II error.
• Describe the effect of sample size, effect variability,
level of alpha, and effect size on power.
3
7. Antihypertensive Trial: Result is Positive
7
No Effect−10−20−30 +10 +20 +30
Explanations:
1)Real Effect: HA true
2)FP: Random Error (α)
3)FP: Bias
New
Drug
Old
Drug
9. Antihypertensive Trial: Result is Negative
9
No Effect−10−20−30 +10 +20 +30
Explanations:
1)Real Effect: H0 true
2)FN: Random Error (β)
3)FN: Bias
New
Drug
Old
Drug
10. FN Beta error, effect exists, not detected
10
12. Clinical Research, 2x2 Table
12
Trial
Result
HA
True
HA
False
Positive TP FP (α)
Type I
→ PPV
Negative FN (β)
Type II
TN → NPV
↓
Power
↓
*p value
Total
13. Randomized Clinical Trial Steps
13
Population
of interest
Sample
Drug
A
Drug
B
Time
Drug A
∆ BP
Drug B
∆ BP
Time Compare,
publish,
accept
Nobel
prize
H0: A=B
HA: A≠B
Bias vs.
random
error
18. Types of Statistics
• Descriptive
– Summarize/display data
– Mean, median, mode, σ etc.
• Inferential
– Use sample to make
conclusions about population
– Example: Hypertension
• Test population: all w/ ↑ BP
– Definitive, descriptive stats only
• Test sample
– Hypothesis testing
– P(Observed results given H0)
13%
17%
57%
13%
1st Qtr
2nd Qtr
3rd Qtr
4th Qtr
18
Population:
all w/↑ BP
Sample
19. Hypothesis Testing: Is H0 plausible?
EUSM vs. NBA Mean Height
19
H0
: Expected
Trial: Observed
When you stare into the abyss [of statistics], the abyss stares back into you.
Statistics:
P(O given H0)
p<.05
reject H0
p≥.05, cannot
reject H0190
H0:μEUSM=μNBA(190)
HA:μEUSM≠μNBA (190)
170
20. Determining p value: Normal Distribution
20
0 1 2−2 −1
Central Tendency:
Mean, median and mode
Dispersion:
Standard deviation
√Σ(x-μ)2
/N
68%
95%
25. Example 1: EUSM M-1s vs. NBA Heights
• Is mean height of EUSM M-1s different than mean
height of NBA players?
• H0:μEUSM=μNBA(190 cm) with σ=10 cm
• HA:μEUSM≠μNBA(190 cm) with σ=10 cm
• 25 M-1 heights, mean=170 cm, ∆=20 cm
• SEM= σ/√n=10/√ 25=2
• 20/2= 10 σ, p<.0001
• Reject H0at α of .05
• α predetermined for
H0rejection 25
O
bserve: 170
Expect: 190
28. Example 2: EUSM M-1s vs. Brand X M-1s
• Is mean height of EUSM M-1s different than
mean height of M-1s at Brand X medical school?
• H0:μEUSM=μBrandX(170 cm) with σ=10 cm
• HA:μEUSM≠μBrandX(170 cm) with σ=10 cm
• 25 M-1 heights, mean=170 cm, ∆=0 cm
• SEM= σ/√n=10/√ 25=2
• 0/2= 0 σ, p=1
• Don’t reject H0
28
O
bserve: 170
Expect: 170
33. Example 3: EUSM M-1s vs. Jockeys
• Is mean height of EUSM M-1s different than
mean height of Jockeys?
• H0:μEUSM=μJockey(160 cm) with σ=10 cm
• HA:μEUSM≠μJockey(160 cm) with σ=10 cm
• 25 M-1 heights, mean=170 cm, ∆=10 cm
• SEM= σ/√n=10/√ 25=2
• 10/2= 5 σ, p=.0062
• Reject H0at α of .05
33
Observe:170
Expect: 160
37. Meaning of P Value
• P value tells us about plausibility of H0,(A=B)
– Assumes H0 is true, what is probability of observed given
expected
– Example: Hypertension trial, Drug A>Drug B, p=.031, reject H0
– Example: Coin toss, 5 heads in a row chance
37
.500.500 .250.250 .125.125 .063.063 .031.031
40. P value: Effect Size & SNR (variability)
• Example: Weight loss pills vs. placebo:
• Precise pill: 2 lb loss w/ sem of .9 lbs, p value < .05, reject H0
• Noisy pill: 10 lb loss w/ sem of 6 lbs, p value > .05, don’t
reject H0
• Which pill is more effective?
40
0lbs
2 lbs 10 lbs
41. Confidence limits vs. p values
• P value says nothing about effect size or variability
• 95% confidence limits: sample mean±2(sem)
• Estimate of effect size and precision (variability)
• 95%CI≠95% chance μ is w/in CI, more complex
• CI does not include bias
• Can be used for significance testing
4112 lbs10 lbs8 lbs0 lbs
95% CI
42. What effects β Error/Power?
• Power is P(Detecting real effect) Sensitivity
• β is P(Missing a real effect) FN, random
effects
• Power=1-β
• Power effects:
–Level of α
– Effect size
– Sample variability
– Sample size
42
43. Clinical Research, 2x2 Table
43
Trial
Result
HA
True
HA
False
Positive TP FP (α)
Type I
→ PPV
Negative FN (β)
Type II
TN → NPV
↓
Power
↓
*p value
Total
50. Clinical Research, 2x2 Table
50
Trial
Result
HA
True
HA
False
Positive TP FP (α)
Type I
→ PPV
Negative FN (β)
Type II
TN → NPV
↓
Power
↓
*p value
Total
