The Screen Design and Assay Technology SIG meeting at SLAS2014, January 22 in San Diego, featured a presentation by Wayne J. Levin of Predictum titled Design of Experiment Challenges & Opportunities. The slides and supplementary material are provided here.

1. DOE challenges &
opportunities
agenda
WAYNE J LEVIN
PREDICTUM INC.
LEVIN@PREDICTUM.COM
!
WWW.PREDICTUM.COM
About Predictum
Use the right methods (and there are more of them)
Build a system
Assume nothing
PREDICTUM INC. WWW.PREDICTUM.COM ©2014
1
About Predictum
we increase productivity
more exploitable insights in less time, effort, cost
less frustration
3
2
DOE purpose
look at numerous effects comprehensively yet isolate each
effect’s influence independently of the other effects
in less time, effort and cost
4

2. use latest & greatest methods
Definitive Screening Designs
latest in computer-generated optimal designs
Split-plot designs
hard to change factors
definitive screening
more independence of effects
5
Optimal Design -20 runs
6
Definitive screening - 21 runs
Optimal Design
Definitive screening
Sacrifice some D-efficiency, but not much
Zero correlation among main
effects and 2-way interactions
Some correlation among main
effects and 2-way interactions
7
8

3. split-plot designs
The concept of hard to
change is broader than you
might think.
!
Here multiple pipetting
constitutes hard to change
where materials are the same
across wells
it’s a multivariate universe
9
10
11
12
Scenario
The tool is typical of HTS, in that there is limited
ways a chemical can be varied within a plate.

4. generating errors
Under REML
highlighted items
show type I
errors if analyzed
traditionally
generating errors
DOE can be a lot of work
take the time, make the effort
longer more complex experiment designed right often yields
more correct insights than a series of small experiments
if not done correctly, DOEs will generate
type I and type II errors
confusion and frustration
!
Under
Traditional
highlighted items
show type II
errors if analyzed
traditionally
13
14
R&D and Improvement Initiatives are launched and completed in isolation.
build a system
time
Copyright © 2013 Predictum Inc.
15
Confidential
16

5. This is typically what happens.
Insufficient institutional memory.
Each project is isolated.
What if they were connected? What if
they started with all relevant
information previously acquired and
paid for?
!
Slippage on retaining acquired, prior insights. Must
pay to re-acquire what was previously known.
time
Copyright © 2013 Predictum Inc.
Confidential
17
assume nothing
be methodical
19
time
Copyright © 2013 Predictum Inc.
Confidential
18

6. Utilizing Design of Experiment Statistical
Models to Improve Assay Development
in High Throughput Biology
Diana Ballard of Predictum Inc., Samuel Hasson of NIH, Wayne Levin of Predictum Inc.
Overview
Step 1
All 6 plates
Nozzle Buffer
Type
1
HEPES
2
Tris
3
Tris
4
HEPES
5
Tris
6
HEPES
7
Tris
8
HEPES
similar tests assume that the entire experiment
pH PK Enzyme
concentration
8
100 uM
7
10 uM
8
10 uM
7
10 uM
8
100 uM
8
10 uM
7
100 uM
7
100 uM
Nozzle
1
2
3
4
5
6
7
8
REML
ANOVA
Buffer Type
<.0001
<.0001
PK Enzyme concentration*Buffer Type*Nested Packaging
of assay
<.0001
<.0001
pH
Buffer Type*pH *PK Substrate [Nested Packaging of assay]
Buffer Type*PK Enzyme concentration
pH*PK Substrate[Nested Packaging of assay]
PK Enzyme concentration
PK Enzyme concentration*Timing*Nested Packaging of assay
Magnesium ion concentration*pH
pH*Buffer Type
pH*Timing*Nested Packaging of assay
Buffer Type*Magnesium ion concentration*PK Substrate
[Nested Packaging of assay
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
0.0004
0.0006
<.0001
0.0186
0.0111
0.2577
0.0377
0.0432
0.0651
0.0669
0.1765
0.2379
Factor / Effect
PK Enzyme
concentration
Step 2
Error Term in the model
PK Enzyme concentration
* Buffer Type * pH *
Iteration
Source (Partial List of 85)
Buffer Type
Column within plate
Experimental unit (EU)
The wells on all plates that had unique bottle setups for PK Enzyme.
Step 1 had 8 unique bottles, repeating step 1 in iteration 2. Across
both iterations, there are 16 EU.
The wells on all plates that were treated by one of the 16 bottles
made for step 1.
PK Enzyme concentration
* Buffer Type * pH *
Iteration
Magnesium concentration
* Iteration
Magnesium ion
concentration
Nested packaging
of assay
Magnesium
solution
5 uM
water
1 2 3 4 5 6 7 8
Introduction
Step 3
Changed on a per plate basis. Any plate could have gotten either
setting. Across both iterations, there are 12 EU.
Plate * Iteration
PK Enzyme
concentration *
Magnesium ion
concentration
Buffer Type *
Magnesium ion
concentration
The intersection of the EU for PK Enzyme and the EU for
Magnesium ion concentration. 16 x 4 = 64 EU across
both iterations.
Buffer Type *
Nested packaging
of assay
All 6 plates
The wells on all plates that had unique bottle setups for Step 2 and
Magnesium concentration. Across both iterations, there are 4 EU.
The intersection of the EU for Buffer Type and the EU for
Nested packaging of assay. 16 EU x 12 EU = 192 EU
across both iterations.
PK Enzyme concentration
* Buffer Type * pH *
Magnesium concentration
* Iteration
PK Enzyme concentration
* Buffer Type * pH *
Magnesium concentration
* Iteration
PK Enzyme concentration
* Buffer Type * pH * plate *
Iteration
The intersection of the EU for Buffer Type and the EU for
Magnesium ion concentration. 16 x 4 = 64 EU across
both iterations
The inclusion of two iterations is critical
Prob > F
FIG 1
Visualize
Buffer Type *
Magnesium *
PK Substrate
[Nested
Packaging
of Assay]
All 6 plates
handled in classical full and fractional factorial designs leading them to provide
BSA
0.2%
concentration water
solution
1 2 3 4 5 6 7 8
Prob > F
instead of the experimental unit to
Plate
Methods
OBJECTIVES:
Step 4
1
2
3
the experimental unit for this model
FIG 2
2800
PK Substrate
400nM
40nM
PK Substrate 400nM
combined with 40nM
Kinase Glo
4
5
6
2750
2700
PK Enzyme
2650
10
100
2600
2550
FACTORS: All eight of the following factors are difficult to vary on the Thermo Multi Drop
Plate
Step 5
1
2800
2
3
Tris
2750
2700
Buffer Type
2650
Timing
5 min
10 min
4
5
6
Conclusions
2600
HEPES
2550
10
20
30
40
50
60
70
80
90
100 110
Tris
HEPES
Plate
Step 6
RESPONSE:
Kinase Glo
4 uL
nothing
1
2
3
4
5
6
DESIGN NOTES:
Step 7
Read all 6 plates
Step 8
Wait Several Days
Do steps 1-7 again
All 6 plates
Copyright © 2013 Predictum Inc.
experimental design and we found a straightforward method to implement its use in DOE

7. Copyright © 2008 Predictum Inc.

8. ACT
•what was learned,
that if proven valid,
can be implemented?
•new questions/
insights sought?
•what next?
STUDY
•what happened that
was expected?
•what did not happen
that was expected?
•what happened that
was not expected?
PLAN
•insights sought
(specify model: main
effects, interactions,
quadratics)
•responses & goals
(maximize, minimize,
match target)
•factors & levels
•identify difficult to
change factors
•constraints on factors
& levels
•design experiment
(evaluate properties)
•detail expectations
•operational
definitions of run
changes and response
measurement
•list all assumptions

9. knowledge
time
Copyright © 2008 Predictum Inc.