Weitere ähnliche Inhalte
Ähnlich wie Randomization: Too Important to Gamble with. (20)
Mehr von Dennis Sweitzer (12)
Randomization: Too Important to Gamble with.
- 1. Randomization:
Too Important to Gamble with
A Presentation for the Delaware Chapter of the ASA
Oct 18, 2012
Dennis Sweitzer, Ph.D., Principal Biostatistician
Medidata Randomization Center of Excellence
Optimizing Clinical Trials:
Concept to Conclusion™
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 1
- 2. Outline
Randomized Controlled Trials
• Basics
• Balance
Randomization methods
• Complete Randomization
• Strict Minimization
• Permuted Block
• Dynamic Allocation (Covariate-adaptive, not Response-Adaptive)
Randomization Metrics
• Balance
• Predictability
• Loss of Power /Loss of Efficiency
• Secondary Imbalance: drop-outs
Simulations comparing methods
• Confounding site & treatment effects (small sites)
• Overall performance
• Discontinuing patients
• Weighting stratification factors
Meta-Balance
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 2
- 3. Why randomize
anyway?
Some basic principles
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 3
- 4. Why Gold Standard?
Randomized Controlled Trial
• Trial:Prospective & Specific
• Controlled:
• Comparison with Control group
• (placebo or active)
• Controlled procedures ⇒ Only Test Treatment Varies
• Randomization: Minimizes biases
• Allocation bias
• Selection bias
• Permits blinding
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 4
- 5. Eliminating Bias
¿ The Fact of bias ?
• (conscious, unconscious, or instinctive)
¿ The Question of bias ?
• Always 2nd guessing
• Critics will think of unanticipated things
¡ Solution !
• Treat it as a game
• 1 statistician vs N clinicians
• Statistician generates a random sequence
• Clinicians sequential guess at each assignment
• Statistician wins if clinician guesses are no better than chance
(NB: 75% wrong is just as bad as 75% right)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 5
- 6. Randomization Metrics
What do we want in a randomization sequence or
system?
Randomness ó Unpredictable
⟶ Reduce Allocation Bias (All studies)
⟶ Reduce Selection Bias (All studies)
⟶ Reduce placebo effects (Blinded studies)
Balance ó “Loss of Efficiency”
⟶ Maximizes statistical power
⟶ Minimize Confounding
⟶ Enhance Credibility (Face Validity)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 6
- 8. Balanced Study
Equal allocation
between
treatment arms
• Maximizes
Statistical
Power
Control Test
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 8
- 9. Imbalanced
Statistical power limited by
smallest arm
• 36 subject simulation with
Complete Randomization
⟶
average loss ≈ 1 subject
10% lose ≥2 subject
• Can add 2 to compensate
• BUT only large imbalances
have much effect
on statistical power
Resulting in light weight results….
Severe Imbalances are rare in large studies
Pr{worse than 60:40 split} for:
• n=25 ⟶ <42% n=100 ⟶ <4.4% n=400 ⟶ 0.006%
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 9
- 10. (NB: Planned Imbalance)
1:1 randomization maximizes power per patient
But there are other considerations
• Utility:
• Need 100 patients on drug to monitor safety
• Study only requires 60 (30/arm)
• 2:1 randomization ⟶ 100 Test & 50 Placebo
• Motivation:
• Better enrollment if 75% chance of Test drug (3:1)
• Ethics:
• 85 Placebo + 255 Test vs. 125 Placebo + 125 Test
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 10
- 11. Imbalance
• Overall balance
• Only an issue for small studies
• Subgroup Balance
• Fixed size studies can have variable sized subgroups
⟶ Increased risk of underpowered subgroups
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 11
- 12. Effective Loss of Sample Size
Effective Loss = Reduction of Power
Females Male as Reduction in Sample Size
s
Test
Simulations of:
Pla
• 36 and 18 subjects,
• males as strata at 33% of population,
Test Con • randomized 1:1
• (complete randomization)
N=36 N=18
Overall Females Males Overall Females Males
Effectively Lost
Mean ± SD 1.0 ±1.4 0.9 ±1.3 1.0 ±1.4 1.0 ±1.4 1.0 ±1.4 1.0 ±1.3
≥2 pts 12% 14% 18% 23% 16% 17%
≥4 pts 6% 4% 5% 3% 4% 5%
>=100% 0.0% 0.0% 0.4% 0.0% 0.5% 7.9%
Q1 0.11 0.15 0.09 0.22 0.09 0.14
Median 0.44 0.43 0.47 0.22 0.40 0.50
Q3 1.00 1.19 1.33 0.89 1.33 1.29
Imbalance (% of N)
Mean ± SD 13% ±10% 16% ±12% 25% ±19% 18% ±15% 23% ±18% 35% ±28%
>=50% 0.5% 1.6% 12.8% 3.1% 10.0% 27.9%
Q1 6% 8% 9% 11% 9% 14%
Median 11% 14% 20% 11% 20% 33%
Q3 17% 22% 33% 22% 33% 50%
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 12
- 13. Bad Imbalance!
Males
Females Treatment
Imbalances
Pla
Test within factors
⟶ spurious
Test Pla findings…..
Leads to conversations like:
ANCOVA
Higher estrogen showed no
levels in patients Credibility…..
differences in
on Test estrogen
Treatment ??
levels due to
Hmm… treatment
?
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 13
- 14. ?"
Randomization
!!!
! !!
!
Methods
(See Animated Powerpoint Slides…)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 14
- 15. Randomization
4 methods
• Complete Randomization (classic approach)
• Strict Minimization
• Permuted Block (frequently used)
• Dynamic Allocation (gaining in popularity)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 15
- 16. Complete Randomization
Every assignment
• Same probability for each assignment
• Ignore Treatment Imbalances
• No restrictions on treatment assignments
Advantages:
• Simple
• Robust against selection & accidental bias
• Maximum Unpredictability
Disadvantage
• High likelihood of imbalances (smaller samples)
.
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 16
- 17. Minimization
Strict Minimization
randomizes to the
imbalanced arm
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 17
- 18. Minimization
Strict Minimization
rebalances the Arms
• BUT at a cost in
predictability
• Random only when
treatments are currently
balanced
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 18
- 19. Permuted Block
Blocks of Patients
(1, 2, or 3 per treatment)
Here: 2:2 Allocation
T P
P ?
T P
P T
T P (Unless Incomplete
P * Blocks:
More strata
⟶ More incomplete)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 19
- 20. Dynamic Allocation
Biases Randomization to
the imbalanced arm
• Unpredictable
• Almost Balanced
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 20
- 21. Dynamic Allocation
Complete Randomization
• Optimizes Unpredictability
• Ignores Balance
Strict Minimization
• Optimizes Balance
• Ignores Predictability
Dynamic Allocation
2nd Best Probability Parameter
Controls
Balance vs. Predictability
Tradeoff
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 21
- 22. Dynamic Allocation Flexibility
2nd Best Probability= 0
⟶ Strict Minimization
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 22
- 23. Dynamic Allocation Flexibility
2nd Best Probability= 0.5
⟶ Complete
Randomization
(for 2 treatment arms)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 23
- 25. Stratification Factors
Over both sexes Factors
Males Females
≣ Main Effects
18-35 yo Pla
Strata
Pla
Pla Test Test Test ≣ 1st Order
Interactions
35-65 yo Test Test
Pla
Pla Test Pla ce Randomizing a
al Balan
Marg in 25 yo Male:
>65 yo To PLA
Test Pla
Test Pla Test ⟶ Worsens Male
Pla
balance
lance
Marg inal Ba To Test
Over all ⟶ Worsens
Ages: Test Pla Pla Test 18-35yo balance
lance
Pla Test O verall Ba
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 25
- 26. Permuted Block Stratified Randomization
Over both sexes • Only balances
Males Females within strata
T P P * Pla • Most strata will
18-35 yo
P T * *
Test have incomplete
blocks
T P P T
35-65 yo T * * * Test
Pla • Imbalances
accumulate at
margins
>65 yo T T P *
P * * *
Pla Test
Over all
Ages:
Test Pla
Pla Test
Pla Test
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 26
- 27. Minimization & Marginal Balance
* Only balances on
margins
Over both sexes * Useful if too many
Males Females
strata, e.g.:
18-35 yo Pla Pla
Pla Test Test Test
N
# Strata >
blocksize
35-65 yo Test Test
Pla Test
Pla Pla nce
inal Bala * Appropriate for a
Marg
main effects analysis
>65 yo
Test Pla (ie, no interactions)
Pla Test Pla Test
Balance
Marg inal
Over all
Ages: Test Pla Test
Pla
Pla alance
Test Overall B
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 27
- 28. Stratification & Dynamic Allocation
Over both sexes DA: uses weighted
Males Females combination
of
18-35 yo Pla Pla • Overall balance
Pla Test Test Test
• Marginal
balances
35-65 yo Test Test
Pla Test • Strata balance
Pla
Pla lance
inal Ba
Marg ⇒ Flexible
>65 yo
Test Pla
Pla Test Pla Test
Balance
Marg inal
Over all
Ages:
Test Pla
Pla Test
Pla alance
Test Overall B
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 28
- 29. Site as a Special
Subgroup
(Max 2 lines, 35 characters)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 29
- 30. Imbalance
• Overall balance
• Only an issue for small studies
• Subgroup Balance
• Fixed size studies can have variable sized subgroups
⟶ Increased risk of underpowered subgroups
• Site as special case of subgroup
• Small sites ⟶ Increased risk of "monotherapy” at site
⟶ Confounding site & treatment effects
⟶ Effectively non-informative/”lost” patients
• Actual vs Assumed distribution of site size
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 30
- 31. Enrollment per Center (Densities)
Data Sample
• 13 Studies
• 7.7 mo Average Enrollment period
• 3953 Obs.Pts
• 460 Listed Sites
• 372 Active.Sites
Size Categories:
{0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49,
50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 31
- 32. Enrollment per Site (#Sites)
Data Sample
• 13 Studies
• 7.7 mo Average Enrollment period
• 3953 Obs.Pts
• 460 Listed Sites
• 372 Active.Sites
# Sites per Size Category {0, 1, 2, 3, 4-7, 8-11, 12-15,
16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, 100-149, 150-199, ≥200 }
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 32
- 33. Site Enrollment Simulation
Simulation based on Observations
• 4 mo Enrollment Period
• Enrollment ~ Poisson distribution
µ = Obs. Pts/mo (active sites)
or
µ ≈ 0.5 / Enrollment period (non-active sites)
• Randomize using CR, PB(2:2), or DA(0.15).
• Confounded Pts ≣ Patients at centers with only one
treatment
⇒ treatment & center effects are confounded
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 33
- 34. Results
mean ±SD
(80% C.I.)
• Affected studies had many sites with low enrollment
• Studies with fewer sites (and more pts at each) were rarely affected
• Dynamic Allocation reduced confounding slightly more effectively than
permuted block
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 34
- 35. Performance Comparison
(for two treatments)
s of Efficiency (Atkinson, 1999)
E (Y ) z X
Treatment difference A constant term and k
2
Randomization
prognostic factors
Metrics
Var ( )
z T z z T X ( XT X ) 1 X T z
Loss Ln zT X (X T X) 1 X T z
atients and k factors;
a n k design matrix)
5
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 35
- 36. Randomization Metrics
How do we measure “badness” of a randomization
sequence or system?
• Predictability
• Goal: an observer can guess no better than chance
⟶ Score based on Blackwell-Hodges guessing rule
• Easily calculated
• Imbalance
Imbalance ⟶ reduced statistical power
⟶ “Loss of Efficiency”
• Measure as effective loss in number of subjects
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 36
- 37. Blackwell-Hodges
Use Blackwell-Hodges guessing rule
• Directly corresponds to game interpretation
• Investigator always guesses the most probable treatment
assignment, based on past assignments
• “ bias factor F”
F ≣ abs(# Correct – Expected # Correct by chance alone)
• Measures potential for selection bias
• Modifications:
• Limits on knowledge of investigator (eg, can only know
prior treatment allocation on own site)
• Score as percentage
e.g., Score ≣ abs(% Correct – 50%)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 37
- 38. Blackwell-Hodges Scoring (1)
For treatment sequence “TCCC”
Initial guess ⟶ Expectation = ½
“T” ⟶ Imbalance =+1 ⟶ Guess C ⟶ Correct
“TC” ⟶ Imbalance=0 ⟶ Guess either
⟶ Expectation=½
“TCC” ⟶ Imbalance=-1 ⟶ Guess T ⟶ Wrong
“TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2
Score = #Correct - 2 = 2-2 = 0
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 38
- 39. Blackwell-Hodges Scoring (2)
For treatment sequence “TCCC”
“TCCC” ⟶ # Correct= ½ + 1+ ½ +0 =2
Complete Randomization ⇒ Pr{“TCCC”} = 1/16
Dynamic Allocation (p=0.15)
⇒ Pr{“TCCC”}= 0.5 *0.85 * 0.5 * 0.15 = 0.031875
Permuted Block (length≤4) ⇒ PR{“TCCC”} = 0
Strict Minimization ⇒ Pr{“TCCC”}=0
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 39
- 40. Blackwell-Hodges Scoring (3)
Sequence “TCCT”
# Correct= ½ + 1 + ½ + 1 = 3
Score = 3 – 2 = 1
• Complete Randomization ⇒ Pr{TCCT}= 1/16
• Strict Minimization ⇒ Pr{TCCT} = ½*1*½*1 = ¼
• Permuted Block ⇒ Pr{TCCT} = 1/6
(NB: 6 permutations of TTCC)
• Dynamic Allocation (2nd best prob.=0.15)
⇒ Pr{TCCT} = 0.5 * 0.85* 0.5 * 0.85 = 0.180625
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 40
- 41. Warning!
Blackwell-Hodges
• Assesses potential selection bias
― Given known imbalance!
¿¿ But which imbalance(s)??
(Overall imbalance? Within strata? Within Factors?)
• Henceforth: only use imbalance within strata
• Proxy for center
• Assume observer only knows
imbalance within “his center” Local
• Simple & unambiguous Predictability
M Requires some caution ONLY
in interpretation
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 41
- 42. Loss of EfficiencyComparison
Performance
(for two treatments)
Loss of Efficiency (Atkinson, 1999)
Inference in Covariate-Adaptive
E (Y ) z X allocation
Treatment difference Elsa Valdés Márquez & Nick Fieller
A constant term and k
prognostic factors EFSPI Adaptive Randomisation
Meeting
2 Brussels, 7 December 2006
Var ( )
z T z z T X ( XT X ) 1 X T z
Loss Ln zT X (X T X) 1 X T z
(for n patients and k factors;
X a n k design matrix)
• Loss can be expressed as equivalent # Patients
5
• In a 100 patient study:
Loss of Efficiency= 5
⇒ A perfectly designed study would require only 95
http://www.efspi.org/PDF/activities/international/adaptive-rando-docs/2ValdesMarquez.pdf
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 42
- 43. 2
Var ( )
RCT vs DOE z T z z T X ( XT X ) 1 X T z
Loss Ln zT X (X T X) 1 X T z
(for n patients and k factors;
X a n k design matrix)
X ≣ design matrix: 5
⟶n rows, 1 per pt
⟶K columns,
1 per covariate
z ≣ Treatment
assignments
Designed Experiment (DOE):
⟶ Select z and covariate values to minimize Ln
RCT ⟶ Select only z (No control of covariates)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 43
- 44. Loss of Efficiency (Máquez & Fieller)
Performance Comparison
Performance Comparison
(for two treatments)
Loss of efficiency of various methods
Loss of Efficiency (Atkinson, 1999)
CR: Complete Randomization
E (Y ) z X
TV: Minimization (Taves,1974)
Dynamic
PS:Minimization
Treatment difference A constant term and k
Allocation
(Pocock & Simon, 1975)
prognostic factors
Ds: Ds-Optimum Design
(Begg&Iglewicz, 1980)
2
Var ( ) Biased Coin Design 1 Sequentially
DA: DA-Optimum
zT z zT X( XT X ) Xassign Z
(Atkinson,1982)
T
z
to minimize
Loss Ln zT X (X T X) 1 X T z
(for n patients and k factors;
THE BEST
(without random elements) Simulated data:-
X a n k design matrix)
100 subjects, 5 prognostic factors
6
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 44
- 45. Loss of Efficiency (Máquez & Fieller)
Different factors and samples
Covariate adaptive methods always more
efficient than complete randomisation
method with random element (PS)
only efficient for larger sample sizes
1,000 group of patients
7
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 45
- 46. 25%#
Dynamic)Alloca&on:)Readjus&ng)balance)for)
discon&nuing)pa&ents)
Randomization
20%# PB(2:2)#
PB(2:2)#
αδϕυστ( PB(2:2),#25%DC#
Performance
DA(0.15),#Eq.Wts#
Poten&al)Selec&on)Bias)
15%#
δισχοντινυ( DA(0.15),#Eq.Wts,#25%DC#
DA(0.15),EqWts,Adj.25%DC#
10%#
Simulations
DA(0.15),#Margins#
DA(0.15),#Margins,#25%#DC#
DA(0.15),#Margins,Adj.25%Dc#
5%# CR#
CR(25%DC)#
νωο Δισχ.(
CR#
0%#
0%# 5%# 10%# 15%# 20%# 25%#
%)Loss)of)Efficiency)))
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 46
- 47. Simulation Set up
3 methods: 4 Measures:
• Complete Randomization • Loss of Efficiency
• Permuted Block • B-H Score (“Within Strata”)
• Dynamic Allocation • Overall Imbalance
• Relative Loss of Efficiency vs CR
Each simulated patient • % Loss of Efficiency (of #pts)
randomized w/ each method
6 Strata (Factors: Sex, Age) • 48 subjects Total
• 33% or 50% Males • With random 25% Dropout
• 1:1:1, 1:1:2, 1:2:3
(Young : Middle : Old)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 47
- 48. Note on Figures
Simula&on)results)as)80%)Confidence)Intervals)
25%#
Plot B-H score
vs
20%#
DA(0),#Margin#Balance#
Loss of Efficiency
PB(1:1)#
15%#
DA(0),#Margin#Balance# Median
Poten&al)Selec&on)Bias)
PB(1:1)# +
80% C.I.
10%# ⇒
10% lower
& 10% higher
5%#
0%#
0# 1# 2# 3# 4# 5# 6#
Loss)of)Efficiency)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 48
- 49. Simulation Results(1)
Predictability %Imbalance Efficiency Loss ⟵Averages
DA(0.00) 22% 0.6% 0.87 of Metrics
DA(0.15) 16% 1.6% 1.45
DA(0.25) 13% 2.8% 1.99
But for
DA(0.33) 8% 4.3% 2.64 managing
DA(0.50) 4% 11.3% 4.99 risk, need
CR 4% 11.4% 5.03 Worst Case
PB(8:8) 7% 7.1% 3.00
PB(4:4) 13% 4.9% 1.52
PB(3:3) 16% 4.2% 1.13 80% ⟶
PB(2:2) 19% 3.5% 0.79 Confidence
Intervals
PB(1:1) 23% 2.6% 0.47
Both DA & PB are stratified.
Simulation: 48 subjects, 2 stratification factors, 6 strata, uneven sizes
(DA) Dynamic Allocation (PB) Permuted Block (CR) Completely Random
DA( 2nd Best Probability ), PB( Allocation Ratio )
Simulated subjects were randomized by all 3 methods
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 49
- 50. Randomizations Plotted by Metrics
25%# 25%#
PB(1:1)# PB(1:1), PB(2:2)#
PB(2:2),
20%#
DA(0.00),#Wt(3:3:3)#
DA(0) 20%#
DA(0.15),#Wt(3:3:3)#
DA(0.15)
(Essentially Strict
Poten&al)Selec&on)Bias)
Poten&al)Selec&on)Bias)
15%# 15%#
Minimization)
10%# 10%#
5%#
25%# 5%#
25%#
PB(4:4)# PB(8:8)#
DA(0.33),#Wt(3:3:3)# DA(0.50),#Wt(3:3:3)#
0%#
20%# CR# 0%# CR#
20%#
DA(0.5) ≣ CR
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency) Loss)of)Efficiency)
PB(4:4) PB⟶CR
Poten&al)Selec&on)Bias)
Poten&al)Selec&on)Bias)
15%# 15%#
PB(8:8)
10%# 10%#
DA(0.33) DA(0.5)
5%# 5%#
CR
0%#
CR
0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency)
Loss)of)Efficiency)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 50
- 51. Correlation of Metrics
Correla'ons*of*Predictability*and*Loss*of*Efficiency*
0.40%
0.20%
0.00%
%
%
%
%
DA CR%
DA 0)%
DA 5)%
DA 5)%
DA 3)%
PB )%
PB )%
PB )%
PB )%
PB )%
)%
CR
CR
CR
CR
0
:1
:2
:3
:4
:8
.0
.1
.2
.3
.5
(1
(2
(3
(4
(8
(0
(0
(0
(0
(0
!0.20%
!0.40%
!0.60%
!0.80%
!1.00%
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 51
- 52. Backup scatterplots
25%#
PB(4:4)#
DA(0.33),#Wt(3:3:3)#
20%# CR#
25%#
PB(8:8)#
DA(0.50),#Wt(3:3:3)#
Poten&al)Selec&on)Bias)
15%#
20%# CR#
25%#
PB(8:8) 10%#
PB(3:3)#
DA(0.25),#Wt(3:3:3)#
Poten&al)Selec&on)Bias)
15%#
20%# CR#
10%#
DA(0.5), 5%#
CR
Poten&al)Selec&on)Bias)
15%#
5%#
0%#
0.000# 1.000# 2.000# 3.000#
PB(3:3),
4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
10%#
DA(0.25)
Loss)of)Efficiency)
0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency) 5%#
0%#
0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# 10.000#
Loss)of)Efficiency)
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 52
- 53. Simulated Comparison
25%#
Predictability,vs,Loss,of,Efficiency,
20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#
Dynamic#{0%,#15%,#25%,33%,#50%}#
DA(0.25)
Predictability,Score,
Complete#RandomizaGon#
15%# PB(3:3)
• 1,000 simulations per case
* 48 subjects each
10%#
* 6 Strata, 2 factor,
Variety of proportions
5%#
0%#
0.0# 1.0# 2.0# 3.0# 4.0# 5.0# 6.0# 7.0# 8.0# 9.0#
Optimizing Clinical Trials: Concept to Conclusion™ Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 53
- 54. Simulated Comparison
25%#
Predictability,vs,%,Loss,of,Efficiency,
20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#
Dynamic#{0%,#15%,#25%,33%,#50%}#
DA(0.25)
Predictability,Score,
Complete#RandomizaDon#
15%# PB(3:3)
10%#
Loss of Efficiency
%Loss of Efficiency =
Sample Size
5%#
0%#
0%# 2%# 4%# 6%# 8%# 10%# 12%# 14%# 16%# 18%# 20%#
Optimizing Clinical Trials: Concept to Conclusion™ %Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 54
- 55. Relative Loss of Efficiency
25%#
Predictability,vs,Rela0ve,Loss,of,Efficiency,,
•
20%#
Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}#
DA(0.25)
Predictability,Score,
Dynamic#{0%,#15%,#25%,33%,#50%}#
15%#
PB(3:3)
10%#
5%#
0%#
0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00#
Optimizing Clinical Trials: Concept to Conclusion™ Rela0ve,Loss,of,Efficiency, © 2012 Medidata Solutions, Inc. § 55
- 56. Local
Predictability
ONLY Special Topics
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 56
- 57. Dynamic Allocation Weighting
Dynamic)Alloca&on)Weights) Dynamic)Alloca&on)Weights)
25%# 25%#
Balancing)on){Strata,)Margin,)Overall}) versus)Permuted)Block,)Complete)Randomiza&on)
PB(1:1)#
PB(1:1)#
20%# 20%#
DA(0),#Strata#Balance# DA(0),#Strata#Balance#
DA(0),#Margin#Balance# DA(0),#Margin#Balance#
DA(0),#Overall#Balance#
DA(0),#Overall#Balance#
Poten&al)Selec&on)Bias)
Poten&al)Selec&on)Bias)
15%# CR# 15%#
CR#
10%# 10%#
5%# 5%#
0%#
0%#
0# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10#
0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00#
Loss)of)Efficiency)
Loss)of)Efficiency)
DA(0) balanced only within strata ó Approximates PB(1:1) Local
Predictability
DA(0) equal weighting ó Approximates PB(1:1)
ONLY
DA(0) balanced on margins ó Intermediate properties
DA(0) balanced only overall ó Approximates CR (large N)
NB: Predictability is limited to imbalance within a stratum!
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 57
- 58. Dynamic Allocation Weighting
Dynamic)Alloca&on) Weighting:
25%#
Various)Weigh&ngs) (Strata, Margins, Overall)
DA(0),#Strata#Balance# DA(0) Equal Weighting (1,1,1)
20%# DA(0),#Equal#WeighCng# ó Strata Balance Dominates
DA(0),#Margin&Strata# ó Approximates PB(1:1)
DA(0),#Unequal#WeighCng#
Poten&al)Selec&on)Bias)
15%# DA(0),#Margin#Balance# DA(0) Margin & Strata (1:9:0)
DA(0),#Overall#Balance# ó Separates from PB(1:1)
10%#
DA(0) Unequal Weighting (1,6,20)
DA(0) Margin Balance (0,1,0)
5%#
DA(0) Overall Balance (0,0,1)
ó Approx. CR
0%# Local
0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00# Predictability
Loss)of)Efficiency)
ONLY
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 58
- 59. addition way of usingstudy and factor imbalances. Furthermore, because of the importance of main-
A to the overall a random element to prevent determinism and avoid potential bias.
taining site balance and the fact that the International Conference on Harmonisation (ICH) guidelines
DA Algorithm
emphasizeswe introduce a new a multicenter trial should be stratified by study sites (ICH E9, 1998) is hig
Here, that randomization in generalized multidimensional dynamic allocation method that
[12], the method here specifically singlesrandomizationsite imbalance in the scoring formula.
flexible and can be applied to most out the overall scenarios.
In this generalized MDA method, when a new subject c needs to be assigned to a study arm Ai , we
calculate the weighted sum of the distance measure factor imbalances.
2.1. Marginal imbalance as study, site, strata and
Distance function ≣ Weighted Sum of Imbalances
IMB.c; Ai / D is a key rIMB.Study.c/; Ai // C .wSTRATUM rIMB.St ratum.c/; Ai //
Distance measure.wSTUDY component in DA methods. A number of distance measures have been p
posed, including range, standard deviation and variance [3, 7]. In this paper we use the marginal bala
C .wSITE rIMB.Site.c/; Ai //
X
function as another measure of imbalance. For a actor.v; c/; Alevel, marginal balance has been descri
C .wFACTOR .v/ rIMB.F given factor i // (2)
as evaluating the overall balance of treatment allocation [10], and here the marginal imbalance func
16v6K
is defined as:
wSTUDY ; wSTRATUM ; Imbalance:
• Relative wSITE are the weights assigned to the study, stratum, and site imbalance respec-
ˇ ˇ
X ˇ ˇ X Av t h C ı.i; j /D 1; : : : ; K. Similarly
tively. Similarly, wFACTOR .v/ is the imbalance weight assigned the j factor, v ˇ
ˇ
S t udy.c/ is the set of all subjects randomized before c ˇinto the study, S i t e.c/ is rj ˇ set of subjects
rIMB.X; Ai / D the
ˇ .kX k C 1/ ˇ
randomized before c at c’s site, S t rat um.c/ is the subset of those that belong to the same site and share
16j 6N
the same factor levels as c across all factors, and F act or.v; c/ is the set of all the already randomized
where X share as Union of Strata already been randomized, kX k is the cardinality of
• Factor subset of the subjects c on the v factor.
subjects thatis any the same level or state as that haveth⇒
set X , N is the number of arms in the study, for i D 1; : : : ; N , Ai is the set of subjects already assig
P
2.3. arm Ai , ri assignment using the arm weight, (or ratio) for arm Ai1(so
to Treatment is the normalized generalized method , ri 1 1/, and ı.i; j / is
D
X= X ⇒ X ≥ X ⇒
As expected of a DAk method, arms that provide the least imbalance are collected into the
Kronecker delta.
first-choice set:X ∈X
k
X +1
≤
16i 6N
X +1
rIMB.X; Ai / provides a measure of the imbalance that would result from randomizing a new m
k k
ber of X into arm AiC.c/ Dmeasure is general, it does;:::;AN g IMB.c; Aj /gnumber of arms, and can han
F . This fAi W IMB.c; Ai / D minfA1 not depend on the (3)
⇒ and uneven arm ratios. This dominate Distance functionthe new class of multi-
both even Strata Imbalances feature makes it particularly useful for
To keep the study balanced, it is also that unlike other distance measures, the any one of the arms
adaptive clinical trials. Note preferable that the subject c will be assigned to measure here is inversely p
in F C.c/. to the size of X . This ensures that an imbalance of n > 0 subjects on a small group will ‘cou
portional
more than the method allows for the incorporation of a random element, a ‘Second Best Probability’
However,
an n subject imbalance on a larger group.
parameter that sets the Conclusion
Optimizing Clinical Trials: Concept to probability that even when there is just one best minimizing arm, 2012 Medidata Solutions, Inc. § 59
™ © that arm will
- 60. Weighting
Over both sexes
Males Females
18-35 yo
Pla Test
Pla
Test
Pla
Test • Stratified Randomization weights
35-65 yo
Pla
Test
Pla Test
Pla
Test on strata, not margins or overall Over both sexes
Males Females
• Imbalances within strata tend to
>65 yo
Test Pla
18-35 yo Pla Pla
Pla Test Pla Test
Pla Test Test Test
Over all
Ages:
Test Pla
dominate in DA 35-65 yo Test
Pla Test Test
Pla Test
Pla Pla Pla
Test
>65 yo
• Minimization weights on margins, not strata.
Test Pla
Pla Test Pla Test
• DA can weight exclusively on margins
Over all
Ages:
Test Pla
Pla Test
Pla
Test
Over both sexes
Males Females
18-35 yo
Pla Test
Pla
Test
Pla
Test • If a Strata is balanced, the next assignment
35-65 yo
Test
Pla Test
Pla
Test attempts to balance the margins.
Pla
>65 yo
Pla
Test Pla
Test Pla Test
• Since small groups are more likely to have
Over all
Ages:
Test Pla
imbalances which reduce efficiency, balancing
strata 1st is appropriate
Pla Test
Pla
Test
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 60
- 61. Hierarchical Balancing
• While Imbalances within strata tends to dominate in DA,
if a Strata is balanced, the next assignment attempts to balance the
margins
• Since small group imbalances tend to dominate, balancing tends to be
sequential
Males Females Over both sexes
⟵ This example:
18-35 yo
Pla Test
Pla
Test
Pla
Test (1) Balance within strata
35-65 yo Test
Pla Pla Test Pla
Test
(2) If balanced within the strata, balance by age group
>65 yo
Pla
Test Pla
Test Pla Test
(since age groups tend to be smaller than sex groups)
Over all
(3) If balanced within age group, balance within sex group
Ages: Test Pla
Pla Test
Pla Test
(4) If balanced within sex group, balance overall
However: cumulative imbalances may change this order
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 61
- 62. ?" Replacement
Randomization
!!!
! !!
!
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 62
- 63. Dynamically Adapting to Dropouts
Patients discontinue
25%#
Effect)of)Drop9outs)on)Permuted)Block)and)Dynamic) ⟶ Imbalances
Alloca&on) ⟶ Reduced efficiency
20%# PB(2:2)#
25% DC PB(2:2)#
“Tight” randomizations
(PB with small blocks,
PB(2:2),#25%DC# DA with small 2nd best Prob.)
⟶ Lose more
Poten&al)Selec&on)Bias)
15%#
DA(0.15),#Eq.Wts#
efficiency
DA(0.15),#Eq.Wts,#25%DC#
10%# DA(0.15),#Margins#
“Loose”
randomizations
DA(0.15),#Margins,#25%#DC# (CR, PB with large blocks,
DA with large 2nd best Prob.)
CR#
5%#
⟶ Lose less efficiency
CR(25%DC)# ⟶ Little or no change
CR#
No DC
0%#
0%# 5%# 10%# 15%# 20%# 25%#
%)Loss)of)Efficiency)))
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 63
- 64. Dynamically Adapting to Dropouts
Dynamic Allocation: Can
Effect)of)Drop9outs)&)Rerandomiza&on)) allocate new patients to
24%$
on)Permuted)Block)and)Dynamic)Alloca&on)
restore balance
PB(2:2)$
PB(2:2)$
22%$
PB(2:2),$25%DC$
DA(0.15),$Eq.Wts$
DA(0.15),$Eq.Wts,$25%DC$
DA(0.15),EqWts,Adj.25%DC$
20%$
Poten&al)Selec&on)Bias)
18%$
25% DC
16%$
14%$
DA Adj.
No DC
12%$
0%$ 1%$ 2%$ 3%$ 4%$ 5%$ 6%$ 7%$ 8%$
%)Loss)of)Efficiency)))
Optimizing Clinical Trials: Concept to Conclusion™ © 2012 Medidata Solutions, Inc. § 64