Experimental research

1
Experimental Design
What is an Experiment?
• Research method in which
– conditions are controlled
– so that 1 or more independent variables
– can be manipulated to test a hypothesis
– about a dependent variable.
• Allows
– evaluation of causal relationships among variables
– while all other variables are eliminated or controlled.
Compiled by:Dr.V.Singh
E-Content- Module 11- Research Methods and Statistics
SXCE, Patna

2
Symbolism for Diagramming
Experimental Designs
X = exposure of a group to an experimental treatment
O = observation or measurement of the dependent variable
If multiple observations or measurements are taken,
subscripts indicate temporal order – I.e., O1, O2, etc.
= random assignment of test units;
individuals selected as subjects for the experiment
are randomly assigned to the experimental groups
R
Types of Experimental
Research
True-
Experimental
Design
Quasi-
Experimental
design
Pre-
Experimental
Design
Pretest/posttest control group design
Posttest only control group design
Solomon four group design
Non-equivalent control group
Time series
Multiple time series
One-short case study
One-group, pretest/posttest
Static group comparison
Single variable
designs
Factorial designs
SXCE, Patna

3
Pre-experimental Design
Lacking in several areas of the true-experimental criteria.
No random selection in most of the cases.
Employment of just single group that receives treatment, no control
group.
The advantages are:
Very practical
Set the stage for further research
Disadvantages:
Lower validity
Pre-experimental Design
The one-shot
case study
One group Pretest
Posttest study
The static group
comparison study
SXCE, Patna

4
Pre-experimental designs
Classified depending on whether there is an involvement of one or two groups, and
whether the groups are posttested only, or both are pretested and posttested:
-One-shot case studies
-One-group pretest-posttest design:
-Static-group comparison design:
One-shot case studies:
One group is exposed to the treatment, and only a posttest is given
to observe or measure the effect of the treatment on the dependent
variable within the experimental group. Since it is applied on a single
group, there is no control group involved in this design.
First of all, the chosen group is exposed to the treatment , and
then it is tested only once for the purpose of measuring the
degree of change on the dependent variable after the treatment
.
SXCE, Patna

5
One-Shot Design
• A single measure is recorded after the treatment
is administered
• Study lacks any comparison or control of
extraneous influences
• No measure of test units not exposed to the
experimental treatment
• May be the only viable choice in taste tests
• Diagrammed as: X O1
Suppose a researcher wants to
study the effect of a reading
program on reading achievement.
She might implement the reading program with a
group of students at the beginning of the school year
X
and measure their achievement at the end of
the year.
O
This simple design is known as a one-shot case
study design.
SXCE, Patna

6
One-Group Pretest-Posttest Design
• Subjects in the experimental group are measured
before and after the treatment is administered.
• No control group
• Offers comparison of the same individuals before
and after the treatment (e.g., training)
• If time between 1st & 2nd measurements is
extended, may suffer maturation
• Can also suffer from history, mortality, and
testing effects
• Diagrammed as O1 X O2
One-group pretest-posttest design:
One group is pretested and exposed to the treatment, and
then posttested. This is called a one-group pretest-posttest
design because the two tests are administered to the same
group.
The first one is administered at the beginning of the
treatment and the second one at the end.
SXCE, Patna

7
X O
Unfortunately, the students’ end of year reading scores
could be influenced by other instruction in school, the
students’ maturation, or the treatment.
We also do not know whether the students’ reading skills
actually changed from the start to end of the school year.
We could improve on this design by giving a pretest at the
start of the study.
O
This is known as a one-group pretest-posttest design.
Static-group comparison design
At least two groups are involved. After one group
receives the treatment, all groups are posttested.
This design has better control over most of the
variables
SXCE, Patna

8
Static Group Design
• Experimental group is measured after being exposed to
the experimental treatment
• Control group is measured without having been exposed
to the experimental treatment
• No pre-measure is taken
• Major weakness is lack of assurance that the groups
were equal on variables of interest prior to the treatment
• Diagrammed as: Experimental Group X O1
Control Group O2
X O
O
Unfortunately, the students’ end of year reading scores still
could be influenced by other instruction in school, the
students’ maturation, or the treatment.
O
O
Our researcher may wish to have a comparison group.
This is a static-group pretest-posttest design.
SXCE, Patna

9
If our researcher believes that the pretest has an impact
on the results of the study, she might not include it.
X O
O
O
O
This is a static-group comparison design.
Advantages of pretest design
Equivalency of groups
Can measure extent of change
Determine inclusion
Assess reasons for and effects of
mortality
Disadvantages of pretest design
Time-consuming
Sensitization to pre-test
SXCE, Patna

10
True Experimental group
Design
Advantages of the true-experimental design include:
Greater internal validity
Causal claims can be investigated
Disadvantages:
Less external validity (not like real world conditions)
Not very practical
True Experimental Designs
Experimental designs are considered true experiments when they employ randomization
in the selection of their samples and control for extraneous influences of variation on the
dependent variable. The three designs we will consider in this section are the best choices
for an experimental dissertation. These are the pretest-posttest
control group design, the Posttest Only Control Group design, and the Solomon Four
Group design.
True Experimental Design
The posttest only control
group design.
The pretest posttest
control group design.
The Solomon four group
control design.
SXCE, Patna

11
True experimental designs
Have the highest level of control among the three
single-variable experimental designs because the
subjects within the groups are randomly assigned
for each group. When subjects are randomly
assigned, there is higher control of the internal
validity as well as the external validity. Moreover,
there is always a control group to compare the
results of the subjects in the experiment with other
subjects of similar status that have not been exposed
to the treatment.
True experimental research
may be designed with or without a pretest on at least
two groups of randomly assigned subjects. The
classification of true experimental designs is made
accordingly :
1. The posttest-only control group design
2 .The pretest-posttest control group design
3. Solomon four-group design
SXCE, Patna

12
Posttest Only Control Group
Subjects are randomly selected and assigned to two
groups. Due to randomization, the two groups are
statistically equal. No pretest is given. One group
receives the Treatment
R X O 1
R O2
Example. Third graders are randomly assigned
to two groups. Then one group
receives a special study on the life of Iqball. Both are
tested on their knowledge of Iqball at the conclusion
of the study.
Analysis. The difference between group means
(O1 and O2) can be computed by an independent
groups t-test.
Posttest-Only Control Group Design
• True experimental design
• Experimental group tested after treatment exposure
• Control group tested at same time without exposure to
experimental treatment
• Includes random assignment to groups
• Effect of all extraneous variables assumed to be the
same on both groups
• Do not run the risk of a testing effect
• Use in situations when cannot pretest
SXCE, Patna

13
Posttest-Only Control Group Design
• Diagrammed as
– Experimental Group: X O1
– Control Group: O2
• Effect of the experimental treatment equals
(O2 – O1)
• Example
– Assume you manufacture an athlete’s foot remedy
– Want to demonstrate your product is better than the
competition
– Can’t really pretest the effectiveness of the remedy
R
R
SXCE, Patna

14
Pretest-Posttest Control Group
Two randomly selected groups are measured
before (O1 and O3) and after (O2 and
O4) one of the groups receives a treatment
(X).
R O 1 X O 2
R O3 O4
Example. Third graders are randomly
assigned to two groups and tested for
knowledge of Arithmetic. Then one group
gets a special study on Arithmetic. Both are
then tested again.
Pretest-Posttest Control Group Design
• Experimental group tested before and after
treatment exposure
• Control group tested at same two times without
exposure to experimental treatment
• Includes random assignment to groups
• Effect of all extraneous variables assumed to be
the same on both groups
• Do run the risk of a testing effect
SXCE, Patna

15
Pretest-Posttest Control Group Design
• Diagrammed as
– Experimental Group: O1 X O2
– Control Group: O3 O4
• Effect of the experimental treatment equals
(O2 – O1) -- (O4 – O3)
• Example
– 20% brand awareness among subjects before an
advertising treatment
– 35% in experimental group & 22% in control group
after the treatment
– Treatment effect equals (0.35 – 0.20) – (0.22 – 0.20)
= 13%
R
R
SXCE, Patna

16
Solomon Four-Group Design
• Combines pretest-posttest with control group
design and the posttest-only with control group
design
• Provides means for controlling the interactive
testing effect and other sources of extraneous
variation
• Does include random assignment
Solomon Four-Group Design
• Diagrammed as
– Experimental Group 1: O1 X O2
– Control Group 1: O3 O4
– Experimental Group 2: X O5
– Control Group 2: O6
• Effect of independent variable (O2 – O4) & (O5 – O6)
• Effect of pretesting (O4 – O6)
• Effect of pretesting & measuring (O2 – O5)
• Effect of random assignment (O1 – O3)
R
R
R
R
SXCE, Patna

17
Solomon four-group design
Takes the effect of pretest and posttest
sensitivization into consideration.
It is the combination of pretest-posttest
control group (G1 and G2) and posttest only
control group (G3 and G4) designs. In this
case, subjects are randomly selected and
placed into four groups;
SXCE, Patna

18
Example. Third graders are randomly assigned to 1 of
4 groups. The “knowledge of language” is measured in
groups 1 and 2. Groups 1 and 3 are given a special study on
the language learning. When the special study is over, all
four groups are tested.
Analysis. One-way ANOVA can be used to test the
differences in the four posttest mean scores (O2, O4,
O5, O6). The effects of the pretest can be analyzed by
applying a t-test to the means of O4 (pretest but no
treatment) and O6 (neither pretest or treatment). The
effects of the treatment can be analyzed by applying a t-
test to the means of O5 (treatment but no pretest) and
O6 (neither pretest or treatment). Subject maturation can
be analyzed by comparing the combined means of O1 and
O3 against O6. Compiled by:Dr.V.Singh
- the first and the second groups are retested;
- the first and the third groups are exposed to
the treatment, and the second and the fourth
groups are taken as control groups;
- all four groups are posttested.
This design provides the best result but it
requires a large sample so that enough
subjects could be assigned to four groups.
When the sample is large, administering the
tests becomes difficult, time and energy
consuming.
SXCE, Patna

19
Quasi-Experimental Designs
• More realistic than true experiments
• Researchers lacks full control over the
scheduling of experimental treatments or
• They are unable to randomize
• Includes
– Time Series Design
– Multiple Time Series Design
• Same as Time Series Design except that a control group is
added
Quasi-experimental Design
Without proper randomization
Lack of rigorous statistical scrutiny
Some advantages of the quasi-experimental design include:
Greater external validity (more like real world conditions)
Much more feasible given time and logistical constraints Disadvantage:
Not as many variables controlled (less causal claims)
SXCE, Patna

20
Quasi-experimental
Design
Nonequivalent Control
Group Design
Time Series
Multiple Time Series
Analysis. The t-test for independent
samples (Chapter 20) can be used to
determine if there is a significant difference
between the average scores of the groups (O2
and O4). You can also compute gain scores (O2
- O1 and O4 - O3) and test the significance of
the average gain scores with the matched
samples t-test.
This design’s only weakness is pre-test
sensitization and the possible interaction
between pretest and treatment.
SXCE, Patna

21
Quasi-experimental Designs
The term quasi- (pronounced kwahz-eye) means almost, near,
partial, pseudo, or somewhat. Quasi-experimental designs are
used when true experiments cannot be done. A common
problem in educational research is the unwillingness of
educational administrators to allow the random selection of
students out of classes for experimental samples. Without
randomization, there are no true experiments. So, several
designs have been developed for these situations that are
“almost true experiments,”
or quasi-experimental designs.
We’ll look at three:
the time series,
the nonequivalentcontrol group design,
and the counterbalanced design.
Nonequivalent Control Group Design
Subjects are tested in existing or “intact” groups
rather than being randomly selected. The dotted
line in the diagram represents “non-equivalent”
groups. Both groups are measured before and after
treatment. Only one group receives the treatment.
O 1 X O 2
---------------------
O3 O4
Comments. This design should be used only
when random assignment is impossible.
It does not control for
selection-maturation interaction
statistical regression.
pretest sensitization.
SXCE, Patna

22
Time Series
Establish a baseline measure of subjects by
administering a series of tests over time (O1 through
O4 in this case). Expose the group to the treatment
and then measure the subjects with another series of
tests (e.g., O5 through O8).
O1 O2 O3 O4 X O5 O6 O7 O8
Comments. Since there is no control group, one
cannot determine the effects of
history on the test scores.
Instrumentation may also be a problem (Are the
tests
equivalent?)
the reactive effects of repeated testing of subjects is a
source of external invalidity.
Counterbalanced Design
Subjects are not randomly selected, but are used in
intact groups. Group 1 receives treatment 1 and test
1. Then at a later time, they receive treatment 2 and
test 2. Group 2 receives treatment 2 first and then
treatment one.
Time
1 2
Group1 X1 O X2 O
Group2 X2 O X1 O
Example. Two third grade classes receive two special studies on
language: one in classroom and the other on a computer. Class 1
does the classroom work first, followed by the computer; class 2 does
the computer work first. Both groups are tested after both treatments
.
SXCE, Patna

23
Analysis. Use the Latin
Squares analysis (beyond the
scope of this text).
Comments. Since
randomization is not used in
this design, selection-
maturation
interaction may be a problem.
Multiple treatment effect is a
possible source of external
invalidity.
Time Series Design
• Involves periodic measurements on the
dependent variable for a group of test units
• After multiple measurements, experimental
treatment is administered (or occurs naturally)
• After the treatment, periodic measurements are
continued in order to determine the treatment
effect
• Diagrammed as:
O1 O2 O3 O4 X O5 O6 O7 O8
SXCE, Patna

24
Statistical Designs
• Multiple experiments are conducted
simultaneously to permit extraneous variables to
be statistically controlled and
• Effects of multiple independent variables to be
measured
• Advantages
– Can measure the effects of more than one
independent variable
– Can statistically control specific extraneous variables
– Economical designs can be formulated when each
subject is measured more than once.
Completely Randomized Design
• Involves randomly assigning treatments to group
members
– Allows control over all extraneous treatments while
manipulating the treatment variable
– Simple to administer, but should NOT be used unless
test members are similar, and they are also alike
regarding a particular extraneous variable
– Different forms of the independent variable are called
“levels.”
SXCE, Patna

25
Example
• Grocery store chain trying to motivate consumers
to shop in their stores
• 3 possible sales promotional efforts
X1 = offer discount of 5% off total shopping bill
X2 = offer taste sample of selected foods
X3 = control group, no sales promotional effort
applied
Example
SALES PROMOTION TECHNIQUE
LEVELS 5% discount Taste samples No sales
promotion
Sales, store 3 Sales, store 5 Sales, store 9
STORES Sales, store 1 Sales, store 8 Sales, store 7
Sales, store 6 Sales, store 4 Sales, store 2
Average sales Average sales Average sales
SXCE, Patna

26
Randomized Block Design
• Randomly assigns treatments to experimental &
control groups
• Test units broken into similar blocks (or groups)
according to an extraneous variable
– I.e., location, age, gender, income, education, etc.
• Particularly useful when small sample sizes are
necessary
Randomized Design
Example
• Grocery store chain trying to motivate consumers
to shop in their stores
• 3 possible sales promotional efforts
X1 = offer discount of 5% off total shopping bill
X2 = offer taste sample of selected foods
X3 = control group, no sales promotional effort
applied
Blocks = time stores have been in operation
SXCE, Patna

27
Latin Square Design
• Allows control or elimination of the effect of two
extraneous variables
• Systematically blocks in 2 directions by grouping
test units according to 2 extraneous variables
• Includes random assignment of treatments to
each cell in the design
• Used for comparing t treatment levels in t rows
and t columns
– I.e., if we have 3 treatment levels, we must have 3
rows and 3 columns
Latin Square Design
Extraneous Variable 2
A B C
Extraneous
Variable 1
B C A
C A B
where A, B, & C are all treatments
SXCE, Patna

28
Latin Square Design
Example
PER CAPITA INCOME
TIME IN
OPERATION
High Medium Low
< 5 years X1 X2 X3
5 – 10 years X2 X3 X1
> 10 years X3 X1 X2
Factorial Design
• Used to examine the effects that the manipulation
of at least 2 independent variables
(simultaneously at different levels) has upon the
dependent variable
• The impact that each independent variable has on
the dependent variable is referred to as the main
effect
• Dependent variable may also be impacted by the
interaction of the independent variables. This is
called the interaction effect
SXCE, Patna

29
Factorial Design Example
• Grocery store chain wants to use 12 of its stores to
examine whether sales would change at 3 different
hours of operation and 2 different types of sales
promotions
• Dependent variable is change in sales
• Independent variables
– Store open 6 am to 6 pm
– Store open 6 am to midnight
– Store open 24 hours/day
– Sales promotion: samples for a free gift
– Sales promotion: food samples
• Called a 3 x 2 factorial design
• Need 6 experimental groups (3 x 2 = 6)
Factorial Design Example
HOURS OF OPERATION
SALES
PROMOTION
6 am – 6 pm 5 am – midnight 24 hours
Gift stamps
Food samples
SXCE, Patna

30
Test Marketing
• Controlled experiment conducted on a small segment of
the target market
• Major objectives
– Determine how well products will be accepted in the
marketplace
– Determine how changes in marketing mix will likely affect
product success
• Major reason for test marketing is risk reduction
– Lose $ 1 million in test market or $ 50 million on product
failure?
• Problems
– Expense
– Time
– Competitors can disrupt
Factors to Consider
• Population size
• Demographic composition
• Lifestyle considerations
• Competitive situation
• Media coverage & efficiency
• Media isolation
• Self-contained trading area
• Overused test markets
• Loss of secrecy
SXCE, Patna

Experimental research

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (19)

Ähnlich wie Experimental research

Ähnlich wie Experimental research (20)

Mehr von Vikramjit Singh

Mehr von Vikramjit Singh (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Experimental research