A great presentation from a well versed friend in research and EBM, Dr Yaser Faden.
This is a simple introduction to study design with an accompanying workshop to simplify the different types of research study designs.
2. Observational and
Experimental Study Designs
Dr. Yaser Faden
Asst. Professor, Dept. of OB/GYN
Consultant, Maternal-Fetal Medicine
Director, RTP in OB/GYN
3. What is the basis of a good
research study?
AN APPROPRIATE STUDY
DESIGN
4. Learning Objectives
By the end of this session, you will be able to:
ïDistinguish between observational and experimental studies
ïDescribe the key characteristics of experimental, cohort,
case-control, cross-sectional, and ecologic studies
ïList the advantages and disadvantages of each type of study
design
ïIdentify the design of a particular study by reading an
abstract
ïDiscuss the factors that determine when a particular design
is indicated
5. Diagnostic
Puzzler
You are a practicing physician in the 1970âs.
Your patient is a very ill 24 year old woman
who is hospitalized for fever, low blood
pressure, and a rash, including peeling of the
hands. Your review of symptoms is positive
only for menstruation. You treat her in the
intensive care unit. You feel fortunate that she
survived, but are uneasy because you never
really knew what was wrong with her.
6. Categories of
Epidemiologic Studies
Epidemiologic studies
Observational Studies Experimental Studies
Descriptive Studies Analytic Studies RCTs
Case report Case control
Case Series Cohort
Cross sectional studies
Ecological Studies
Retrospective Cohort Prospective Cohort
7. Categories of Epidemiologic
Studies
ï Observational Studies
Investigators collect, record and analyze
data on subjects as they naturally divide
themselves by potentially significant
variables ( i.e. case-control, cohort)
ï Experimental Studies
Involve some sort of control by the
investigators (i.e. RCTâs)
8. Epidemiologic Study Designs
ï Type of observational studies based on:
â Type of sampling from population
Based on exposure and/or disease
â Temporal sequence of observation
One time point, forward, backwards
9. Exposure and Outcome
ï Exposure
â Refers to the potential risk factor
âą Can be exposure such as tobacco smoke
âą Can be behavior (e.g.. sedentary lifestyle)
âą Can be attribute ( e.g.. SES)
ï Outcome
â Is the disease or other health related problem
which is being studied
10. Descriptive Studies
ï Are a class of epidemiologic studies which focus
on characterizing morbidity or mortality of
populations by person, place or time variable and
have no a priori hypotheses.
ï Examples Include
â Case Report
â Case Series
â Some cross-sectional studies
â Some ecologic studies
11. Case Reports
ï Detailed presentation of a single case
ï Generally report a new or unique finding
â Previous undescribed disease
â Unexpected link between diseases
â Unexpected new therapeutic effect
â Adverse events
12. Case series
ï Experience of a group of patients with a
similar diagnosis
ï Cases may be identified from a single or
multiple sources
ï Generally report on a new/unique condition
ï May be the only realistic design for rare
disorders
13. Case reports and series
ï Case report: describes an observation in a
single patient.
â âI had a patient with a cold who drank lots of
orange juice and got better. Therefore, orange
juice may cure colds.â
ï Case series: same thing as a case report,
only with more people in it.
â âI had 10 patients with a cold who drank orange
juiceâŠ.â
14. Case Reports / Case Series
ï Pros
â Useful for hypothesis generation
â Informative for very rare diseases with few
established risk factors
â Easy to understand
â Can be written up in short period of time
ï Cons
â Cannot study cause and effect relationship
â Cannot assess disease frequency
15.
16.
17. Cross-Sectional Studies
ï Assess both exposure and outcome at
the same time âsnapshotâ
ï These are generally surveys or
interviews
ï Used to determine the prevalence of a
condition (prevalence study)
ï Used to identify possible causative
factors in disease
19. Cross-Sectional Studies
ïStrengths
ïŒOne stop, one time (snapshot)
ïŒRelatively easy, quick, and inexpensive
ïŒEstimates disease prevalence
ïŒUseful for planning services
ïŒGood design for hypothesis generation
ïŒRely on questionnaires and no follow-
ups are required
20. Cross-Sectional Studies
ï Weaknesses
âąOnly representative of participants
âąImpractical if disease is rare
âąMay not be possible to establish temporal
relationship
âąNot a useful study for establishing causal
relationships
21.
22. Ecologic Studies
ï Unit of analysis
ï In most epidemiological studies, this is the
individual; but in ecologic studies, this is
the group.
ï The group or the ecological unit represents
an aggregate of individuals such as
countries, provinces, cities, hospitals
23. Why do an ecologic study? HYPOTHESIS
BUILDING!
The data is easy to obtain, no follow up or
individual contact is needed.
An ecologic study can suggest avenues of
research that may cast light on an etiologic
relationship between exposure and disease
HOWEVER
An ecologic study does not itself
demonstrate that a causal relationship
exists
24.
25. Analytic Studies
ï Unlike descriptive studies, analytic studies are
designed to test hypotheses about an exposure of
interest and a particular outcome
?
Exposure Outcome
27. Observational Study Designs
ï Case-control
Groups determined by outcomes
ï Cohort Studies
Groups determined by risk factors
28. Back to our diagnostic puzzler
ï You make an inquiry to the CDC about patients
with these types of symptoms
ï Yes, they have collected a few other cases like
this. All were menstruating women.
ï You have a keen interest in this new syndrome
and work with the CDC and other doctors to
publish a case series.
ï You notice that one common characteristic of all
of the affected women is tampon use. Is this just
chance, or could it be related?
29. Case-control studies
ï Attempt to make inference from existing
observations (retrospective)
ï Compares patients with outcome/disease
with those without and attempts to
identify factors that influenced that
outcome (or caused that disease)
ï Important concept: start with the result
(disease) and work backwards for the
cause
31. Back to our diagnostic puzzler
ï How would you design a case-control
study to test the theory that
menstruation (or perhaps tampon use)
is somehow connected with this new
illness, which some people have started
to call âtoxic shock syndromeâ?
33. Strengths of case control studies
ï Rare diseases
ï Several exposures
ï Rapidity
ï Low cost
ï Small sample size
ï Available data
ï No ethical problem
34. Limitations of
Case-Control Studies
ï Cannot compute directly relative risk
ï Not suitable for rare exposure
ï Temporal relationship exposure-disease difficult
to establish
ï Biases +++
â control selection
â recall biases when collecting data
ï Loss of precision due to sampling
35. Cohort studies
ï Studies whether exposure to a ârisk
factorâ is associated with a subsequent
âoutcomeâ
ï Select two populations who seem the same
except for the hypothesized risk factor
ï Follow them ahead in time and see how
many have the outcome or disease
ï Important concept: Start with the risk,
then look for the outcome
39. Back to our diagnostic puzzler
ï How would you design a prospective
cohort study to test the theory that
tampon use by menstruating women
is somehow connected with âtoxic
shock syndromeâ?
40. Cohort study design
(Prospective)
Exposure Observer Disease
?
Tampon use or not TSS?
41. Cohort Studies
ï Prospective cohort studies start with the
exposure, then follow patients over time
ï Retrospective (or historical) cohort studies
start with an exposure that happened some
time ago, then look at the outcomes today
ï Important point: Even though this is
retrospective, it starts with the exposure or
risk and then measures the outcome
42. Strengths of cohort studies
ï Can directly measure
â incidence in exposed and unexposed groups
â true relative risk
ï Well suited for rare exposure
ï Temporal relationship exposure-disease is clear
ï Less subject to selection biases
43. Weaknesses of cohort studies
ï Large sample size
ï Lost to follow
ï Exposure can change
ï Multiple exposure = difficult
ï Ethical considerations
ï Cost
ï Time consuming
45. Epidemiologic Study Designs
Descriptive studies
Examine patterns of disease
Analytical studies
Studies of suspected causes of diseases
Experimental studies
Compare treatment modalities
46. Randomized Control Trial
(RCT)
ï Gold standard of all studies
ï Prospective
ï Two or more groups assigned by randomization
ï Baseline measurements on all groups
ï Give different treatments
ï Measure outcome
47. Types of Clinical Trials
ï Treatment trials test experimental treatments,
behavioral therapies, new combinations of drugs,
or new approaches to surgery or radiation
therapy
ï Prevention trials look for better ways to prevent
disease in people who have never had the disease
or to prevent a disease from returning
â These approaches may include medicines, vitamins,
vaccines, minerals, or lifestyle changes
48. Randomization
ï Assigned to groups by method similar to
âflipping a coinâ
ï If randomization works, groups will be the
same/comparable
ï The larger the sample, the greater the likelihood
of equal groups
ï Results should show that the demographic
characteristics between groups are similar
ï If groups are similar, do not need to control for
extraneous variables
49. Randomization
ï Sometimes we cannot randomize people (e.g., cross-
contamination or âsystemâ interventions)
ï Can randomize hospitals, or units instead
â For example, testing clinical reminder systems
ï Once randomized, always randomized
ï Subjects are treated as part of that group, even if they
die, are lost to follow up, or withdraw
50. Randomization
ï Randomization
ï Two kinds of randomization:
â Random sampling
âą Every person in a population must have
an equal chance of getting into the sample
â Random assignment
âą Each person in a sample must have an
equal chance of getting into the
experimental and control group
âą That is, they are randomly placed in one
of the groups
51. Randomization
ï Researcher must actually go through some
randomization process
â For example, number each potential subject, and then pull
numbers from a box or use a random table to determine
assignment to a group
ï Randomization is a very strong and positive control
method
ï Randomization can always strengthen a study
52. Homogenous Sampling
ï Trying to make your sample as much alike is
helpful in studies
â because it can minimize the possibility extraneous
variables have affected the results
ï However, it also has limitations
â Because it makes generalization more difficult since
the study population is smaller and applies to fewer
people
â It can also make it more difficult to get enough people
in the study
ï So, homogenous sampling increases internal
validity, but decreases external
53. Blinding
ï Un-blinded: Everyone knows treatment
ï Single Blinded: Researcher or patient does not know
treatment
ï Double Blinded: Neither researcher or patient knows
treatment
ï Why blinding?
â Many people believe they feel better if they are given
something
â This is the placebo effect
54. Double Blind Example
ï Patient:
â Patient agrees that he will be randomized to one of 4
smoking cessation treatments
â None of these 4 smoking cessation treatments are
known to be better than the other
ï Provider:
â Providers do not know that patients are assigned to
groups
â Hire different people to run each group and do not
tell them about the study
55. Intervention/Treatment
ï Treatment versus placebo
ï Treatment versus standard of care
ï Treatments should be made to be
as same as possible
â For example, new drug versus sugar pill
56. Phases of Clinical Trials
ï Phase I trials (a pilot study): Researchers test an
experimental drug or treatment in a small group of
people (5-60 subjects) for the first time to
â Evaluate its safety
â Determine a safe dosage range
â Identify side effects
ï Phase II trials (a larger pilot study): The experimental
study drug or treatment is given to a larger group of
people (100 subjects) to see if it is effective and to further
evaluate its safety
57. Phases of Clinical Trials
ï Phase III trials (RCT): The experimental study drug or
treatment is given to large groups of people (200-3,000
subjects) to
â Confirm its effectiveness
â Monitor side effects
â Compare it to commonly used treatments
â Collect information that will allow the experimental drug or
treatment to be used safely
ï Phase IV trials (implementation research):
â Post marketing studies
â Delineate additional information, including: the drug's risks,
benefits, and optimal use
58. Non-Randomized Comparison
Group
ï Next best thing to RCT
ï Used when we cannot randomize our
subjects
â For example, due to cross-contamination, or facility-
or community-level interventions
ï Make sure groups are as similar as
possible
60. RCT Advantages
â The âgold standardâ of research designs.
They thus provide the most convincing
evidence of relationship between exposure and
effect.
â Example:
âą trials of hormone replacement therapy in
menopausal women found no protection for
heart disease, contradicting findings of
prior observational studies
61. RCT Advantages
ï Best evidence study design
ï No inclusion bias (using blinding)
ï Controlling for possible confounders
ï Comparable Groups (using randomization)
62. RCT Disadvantages
ï Large trials (may affect statistical
power)
ï Long term follow-up (possible losses)
ï Compliance
ï Expensive
ï Possible ethical questions
63. Epidemiologic study designs
What type of study to choose depends on:
ï What is the research question/ objective
ï Time available for study
ï Resources available for the study
ï Common/rare disease
ï Type of outcome of interest
ï Quality of data from various sources
ï Often there are multiple approaches which will all work
ï Choosing an established design gives you a huge head start
in design, analysis and eliminating biases
66. Study Design
Examples
2. A study examines 200 women with
cervical cancer and 200 controls. They
determine that there is an increased
risk of cervical cancer with smoking
Groups by Disease
Case Control
67. Study Design
2. A study started in 1990 and followed 1000
consecutive women who smoked in
pregnancy and 2000 consecutive non
smoking pregnant women. The study was
completed five years after inception. They
determined that there is an increase in
stillbirth in smokers.
Groups determined by Risk factors ie smokers
Prospective Cohort
68. Study Design
3. A study compared the incidence of PET in
women who had IUI with their partners
semen compared to donor semen. Records
for the last 10 years were reviewed. It was
found that there was an increase in PET in
women who had donor semen.
Groups determined by Risk factors ie husband vs donor
Retrospective Cohort
69. Categories of
Epidemiologic Studies
THANK
Epidemiologic studies
Observational Studies Experimental Studies
YOU
Descriptive Studies Analytic Studies RCTs
Case report Case control
Case Series Cohort
Cross sectional studies
Ecological Studies
Retrospective Cohort Prospective Cohort
Hinweis der Redaktion
Not all prospective trials are placebo-controlled, however. A non-controlled trial might identify potential subjects, give them all a treatment, and then see how they do. Such open-label single arm trials cannot control for placebo effects or experimenter biases, and again results should be considered preliminary. Open or uncontrolled trials are not useless, however. The outcome of subjects in such trials can be compared to historical controls, and if a significant result is apparent (along with safety) can be used to justify a larger and more rigorous trial. Controlled trials have one or more comparison groups in the trial itself â different groups of subjects receive different treatments or no treatment. All subjects can be followed in same manner. Control groups allow the experimenter to make sure that all the subjects have the same disease or symptoms, that they receive known treatments, and many variables (such as other treatments they may be receiving, severity at inclusion, age, sex, race, etc.) can be accounted for.