1. Experimental Design
Andrew Rebeiro-Hargrave (PhD)

2. Experimental Design
• Experimental design is a planned interference in the natural order of events by the
researcher
– A selected condition or a change (treatment) is introduced
– Measurements are planned to see the effect of any change in conditions
• Experimental design is also the quest for inference about causes or relationships
– Researchers want to make inferences about what produced, contributed to, or caused
events and rule out alternative causes
• Experimental design entails:
– selecting or assigning subjects to experimental units
– selecting or assigning units for specific treatments or conditions of the experiment
(experimental manipulation)
– specifying the order or arrangement of the treatment or treatments
– specifying the sequence of observations or measurements to be taken

3. Considerations in Design Selection
• The selection of a specific type of design depends primarily on both
the nature and the extent of the information we want to obtain
• Experimental Design is the task of extracting the exact information
needed to solve the research problem
• Two ways of checking potential designs:
1. What questions will this design answer? We should specify
questions the design won't answer as well ones it will answer.
2. What is the relative information gain/cost picture? The major point
here is that the researcher must take a close look at the probable
cost before selecting a design.

4. Experimental Design Terminology
• Treatment group in an experiment receives the specified treatment
• Control Group serves as a baseline against which to measure the effect of the full treatment on the
treatment group
• A variable refers to almost anything (purchasing power, employment, health, education, housing, gender..)
There are only two kinds of stuff in the world for researchers: variables and constants
• Extraneous variables (external to the experiment) are variables that may influence or affect the results of
the treatment on the subject (decline in external remittances with increasing poverty)
• A variable of specific experimental interest is sometimes referred to as a factor.
– Factor is used when an experiment involves more than one variable (poverty variables)
– Level refers to the degree or intensity of a factor (education, gender)
• Randomness refers to the property of completely chance events that are not predictable. If they are truly
random, examining past instances of occurrence should give the researcher no clues as to future
occurrences
• Random assignment of subjects to groups tends to spread out differences between subjects in
unsystematic (random) ways so that there is no tendency to give an edge to any group

5. Experimental Design Terminology
• Ex post facto refers to causal inferences drawn "after the fact” - the causal event of interest has already
happened
• Variance refers to the variability of any event. If one uses a fine enough measuring device, one can find
differences between any two objects or events
• The inside logic of an experiment is referred to as internal validity. Primarily, it asks the question: Does it
seem reasonable to assume that the treatment has really produced the measured effect?
• External validity refers to the proposed interpretation of the results of the study. If asks the question:
With what other groups could we reasonably expect to get the same results if we used the same
treatment?
• Blocks usually refers to categories of subjects with a treatment group (low income block, middle income
block..)
• Interaction refers to variables in the treatment which may interact with each other. It may make a
difference whether a variable is used by itself, with another, or with different levels or degrees of another.

6. Poverty Experimental Design
Poverty Factors
Income security
Recurring Income
Increase asset value
V
a
r
i
a
b
l
e
s
:
+
/
-
E
d
u
c
a
V
a
r
i
a
b
l
e
s
:
+
/
-
H
o
u
s
e
V
a
r
i
a

7. Village Business Experimental Design
Village Business
Model
Income security
Recurring Income
Increase asset value
V
a
r
i
a
b
l
e
s
:
+
/
-
P
r
o
d
u
V
a
r
i
a
b
l
e
s
:
+
/
-
O
n
l
i
n
V
a
r
i
a

8. Classes of information
There are six major classes of information with
which an experimental designer must cope:
1. post -treatment behavior or physical measurement
2. pre-treatment behavior or physical measurement
3. internal threats to validity
4. comparable groups
5. experiment errors
6. relationship to treatment

9. Post-Treatment Behavior
Usually only immediate or short-range results are obtained
Five categories of post-treatment behavior or physical measurement can be
identified:
1. behavior or measurement immediately after treatment
2. a comparison of post-treatment behavior between experimental and
control groups
3. a comparison of the post-treatment behavior between experimental
groups or blocks
4. long-term effects with continuing treatment and periodic
observations
5. long-term effects without continuing treatment but with
observation(s)

10. Pre-Treatment Behavior
Information concerning pre-treatment behavior or condition requires
observation, a test, or measurement, to be administered before the
experimental manipulation.
Several classes of pre-treatment information can be acquired:
1. behavior or measurement immediately before treatment
2. comparing pre-treatment to post-treatment behavior or
measurement
3. a comparison of pre-treatment behavior or measurement
between different pairs of subjects
4. a comparison of the differences between pre-treatment and post-
treatment behavior among groups of subjects
5. the effect of the pre-treatment observation or measurement on
subsequent behavior or measurement of the subject

11. Internal Threats to Validity
This class of information refers to some rival hypothesis that threatens
clear interpretation of the experiment. Typically, the rival hypothesis
asserts that something outside of the experiment proper produced the
behavior or measurement of interest.
Typically, internal threats to validity include:
1. the subjects exhibited behavior because of some event other than
the treatment
2. some other drug or process caused the change
3. the subject changed naturally (just improved)
4. the subject had a massive change in attitude or emotion
5. some other physical change occurred
6. the subject could or would perform the behavior, or would have
exhibited the measurement without the treatment

12. Comparable Groups
This class of information concerns subjects in the different units were
about the same in relevant attributes before the treatment, and during
the treatment, except for the treatment condition itself.
There are two types of comparability information:
1. were the groups (either experimental or control) comparable before the
treatment?
2. did the groups receive a comparable degree of experiences during the
time of the study (except for differences in treatment?)

13. Experimental Errors
Experiment error refers to some unwanted side effect of the experiment itself
which may be producing effect rather than the treatment.
Two types of strategies exist to deal with the Hawthorne effect:
1. provide for a placebo treatment group which gets the attention, but not
the "real" treatment and use blind and double blind strategies as needed
2. continue the treatment over a longer period of time; research shows
that the Hawthorne effect tends to be short-lived

14. Relationship to Treatment
This class of information deals with the possible interaction of the treatment effects
with: different kinds of subjects, other treatments, different factors within a
complicated treatment, different degrees of intensity, repeated applications or
continuation of the treatment, and different sequences or orders of the treatment or
several treatments.
Typically, information of this type is acquired from blocking, from factorial designs, and
various repeated measures designs.
1. did the treatment interact with subject characteristics so that subjects with
different characteristics behaved or reacted differently?
2. how does the treatment interact when combined with other sorts of treatment?
3. does the treatment contain different factors which may operate differentially on
the subjects?
4. what is the effect of different levels or degrees of the treatment?
5. what is the effect of different orders or sequences of various treatments?

15. Basic Experimental Designs
Eleven commonly used experimental designs:
1. One-Shot
2. One-Group, Pre-Post
3. Static Group
4. Random Group
5. Pre-Post Randomized Group
6. Solomon Four Group
7. Randomized Block
8. Factorial
9. One-Shot Repeated Measures
10. Randomized Groups Repeated Measures
11. Latin Square

16. Discussion topics when setting up an
experimental Design
An experimental design or randomized clinical trial requires careful consideration of several factors
before actually doing the experiment. An experimental design is the laying out of a detailed
experimental plan in advance of doing the experiment.
1. How many factors does the design have? and are the levels of these factors fixed or random?
2. Are control conditions needed, and what should they be?
3. Manipulation checks; did the manipulation really work?
4. what are the background variables?
5. What is the sample size. How many units must be collected for the experiment to be generalisable and have
enough power?
6. What is the relevance of interactions between factors?
7. What is the influence of delayed effects of substantive factors on outcomes?
8. How do response shifts affect self-report measures?
9. How feasible is repeated administration of the same measurement instruments to the same units at different
occasions, with a post-test and follow-up tests?
10. What about using a proxy pretest?
11. Are there locking variables?
12. Should the client/patient, researcher or even the analyst of the data be blind to conditions?
13. What is the feasibility of subsequent application of different conditions to the same units?
14. How many of each control and noise factors should be taken into account?