2. Important DefinitionsImportant Definitions
PopulationPopulation
– The complete set of individuals or objects that theThe complete set of individuals or objects that the
investigator is interested in studyinginvestigator is interested in studying
SampleSample
– A subset of the population that is actually being studiedA subset of the population that is actually being studied
VariableVariable
– A characteristic of an individual or object that can haveA characteristic of an individual or object that can have
different values (as opposed to a constant)different values (as opposed to a constant)
Independent variableIndependent variable
– The variable that is systematically manipulated orThe variable that is systematically manipulated or
measured by the investigator to determine its impact onmeasured by the investigator to determine its impact on
the outcome.the outcome.
3. Important DefinitionsImportant Definitions
Dependent variableDependent variable
– The outcome variable of interestThe outcome variable of interest
DataData
– The measurements that are collected by theThe measurements that are collected by the
investigatorinvestigator
StatisticStatistic
– Summary measure of a sampleSummary measure of a sample
ParameterParameter
– Summary measure of a populationSummary measure of a population
4. Two branches of theTwo branches of the
science of statisticsscience of statistics
Descriptive StatisticsDescriptive Statistics
Inferential StatisticsInferential Statistics
5. Descriptive StatisticsDescriptive Statistics
Concerned with describing orConcerned with describing or
characterizing the obtained samplecharacterizing the obtained sample
datadata
Use of summary measures—typicallyUse of summary measures—typically
measures of central tendency andmeasures of central tendency and
spreadspread
6. Descriptive StatisticsDescriptive Statistics
Measures of central tendency includeMeasures of central tendency include
the mean, median, and modethe mean, median, and mode
Measures of spread include the range,Measures of spread include the range,
variance, and standard deviation.variance, and standard deviation.
These summary measures of obtainedThese summary measures of obtained
sample data are called statisticssample data are called statistics
7. Inferential StatisticsInferential Statistics
Involves using obtained sampleInvolves using obtained sample
statistics to estimate thestatistics to estimate the
corresponding population parameterscorresponding population parameters
Most common inference is using aMost common inference is using a
sample mean to estimate a populationsample mean to estimate a population
mean (surveys, opinion polls)mean (surveys, opinion polls)
8. Planning a studyPlanning a study
Suppose you were interested inSuppose you were interested in
determining whether treatment X hasdetermining whether treatment X has
an effect on outcome Y—there arean effect on outcome Y—there are
several issues that need to beseveral issues that need to be
addressed so that a sound inferenceaddressed so that a sound inference
can be made from the study resultcan be made from the study result
9. Planning a studyPlanning a study
What is the population?What is the population?
How will you select a sample that isHow will you select a sample that is
representative of that population?representative of that population?
– There are many ways to produce aThere are many ways to produce a
sample, but not all of them will lead tosample, but not all of them will lead to
sound inferencesound inference
10. Sampling StrategiesSampling Strategies
Probability samples—result whenProbability samples—result when
subjects have a known probability ofsubjects have a known probability of
entering the sampleentering the sample
– Simple random samplingSimple random sampling
– Stratified samplingStratified sampling
– Cluster samplingCluster sampling
11. Sampling StrategiesSampling Strategies
Non-probability samples—result whenNon-probability samples—result when
subjects do not have a knownsubjects do not have a known
probability of entering the sampleprobability of entering the sample
– Quota samplingQuota sampling
– Convenience samplingConvenience sampling
12. Sampling StrategiesSampling Strategies
Probability samples can be made toProbability samples can be made to
be representative of a populationbe representative of a population
Non-probability samples may or mayNon-probability samples may or may
not be representative of a populationnot be representative of a population
—it may be difficult to convince—it may be difficult to convince
someone that the sample results applysomeone that the sample results apply
to any larger populationto any larger population
13. Planning a study—Planning a study—
Validity IssuesValidity Issues
Internal validityInternal validity
– The extent to which the observed effectThe extent to which the observed effect
on the dependent variable is actuallyon the dependent variable is actually
caused by the independent variablecaused by the independent variable
– Depends on carefully controlling otherDepends on carefully controlling other
potential causes of an effectpotential causes of an effect
– Excessive control may result in artificialExcessive control may result in artificial
circumstancescircumstances
14. Planning a study—Planning a study—
Validity IssuesValidity Issues
External validityExternal validity
– The extent to which one would expect theThe extent to which one would expect the
results from a study to be duplicated inresults from a study to be duplicated in
the real world—in the larger populationthe real world—in the larger population
– Depends on the representativeness of theDepends on the representativeness of the
samplesample
– Also depends on artificiality of the studyAlso depends on artificiality of the study
15. Planning a study—Planning a study—
Validity IssuesValidity Issues
Always a tension between maximizingAlways a tension between maximizing
internal vs. external validityinternal vs. external validity
Efficacy studiesEfficacy studies
– Studies designed to determine the maximumStudies designed to determine the maximum
effectiveness of a treatment under idealeffectiveness of a treatment under ideal
conditions—internal validityconditions—internal validity
Effectiveness studiesEffectiveness studies
– Studies designed to determine the likely effect ofStudies designed to determine the likely effect of
a treatment in the real world—external validitya treatment in the real world—external validity
16. Planning a studyPlanning a study
Clinical trials are generally designed toClinical trials are generally designed to
be efficacy trials—highly controlledbe efficacy trials—highly controlled
situations that maximize internalsituations that maximize internal
validityvalidity
We want to design a study to test theWe want to design a study to test the
effect of treatment X on outcome Y,effect of treatment X on outcome Y,
and try to make sure that anyand try to make sure that any
difference in Y is due to Xdifference in Y is due to X
17. Planning a studyPlanning a study
The simplest design would involve two groups—anThe simplest design would involve two groups—an
experimental group and a control group—that areexperimental group and a control group—that are
created through random assignment. In addition,created through random assignment. In addition,
neither the subjects nor the experimenter knows theneither the subjects nor the experimenter knows the
group assignment (double blind)group assignment (double blind)
Two groups to address the possibility of change inTwo groups to address the possibility of change in
Y occurring regardless of treatment XY occurring regardless of treatment X
Random assignment to address the possibility thatRandom assignment to address the possibility that
the two groups were different to begin withthe two groups were different to begin with
Blinding to address the possibility that patient orBlinding to address the possibility that patient or
experimenter expectations play a role in theexperimenter expectations play a role in the
outcomeoutcome
18. Planning a studyPlanning a study
At the end of this study you observe aAt the end of this study you observe a
difference in outcome Y between thedifference in outcome Y between the
experimental group and the control group.experimental group and the control group.
All of the effort in designing the study withAll of the effort in designing the study with
strict control is for one reason—at the end ofstrict control is for one reason—at the end of
the study you want only two plausiblethe study you want only two plausible
explanations for the observed outcomeexplanations for the observed outcome
– ChanceChance
– Real effect of treatment XReal effect of treatment X
19. Planning a studyPlanning a study
The reason you want only these two explanations isThe reason you want only these two explanations is
because if you can rule out chance, you canbecause if you can rule out chance, you can
conclude that treatment X must have been theconclude that treatment X must have been the
reason for the difference in outcome Yreason for the difference in outcome Y
All inferential statistical tests are used to estimateAll inferential statistical tests are used to estimate
the probability of the observed outcome assumingthe probability of the observed outcome assuming
chance alone is the reason for the difference.chance alone is the reason for the difference.
If there are multiple competing explanations for theIf there are multiple competing explanations for the
observed result, then ruling out chance offers littleobserved result, then ruling out chance offers little
information about the effectiveness of treatment Xinformation about the effectiveness of treatment X
20. Two ways of usingTwo ways of using
Inferential statisticsInferential statistics
Hypothesis testing—answering theHypothesis testing—answering the
question of whether or not treatmentquestion of whether or not treatment
X may have no effect on outcome YX may have no effect on outcome Y
Point estimation—determining whatPoint estimation—determining what
the likely effect of treatment X is onthe likely effect of treatment X is on
outcome Youtcome Y
21. Hypothesis TestingHypothesis Testing
The goal of hypothesis testing isThe goal of hypothesis testing is
somewhat twisted—it is to disprovesomewhat twisted—it is to disprove
something you don’t believesomething you don’t believe
In this case you are trying to disproveIn this case you are trying to disprove
that treatment X has no effect onthat treatment X has no effect on
outcome Youtcome Y
You start out with two hypothesesYou start out with two hypotheses
22. Hypothesis TestingHypothesis Testing
Null Hypothesis (HNull Hypothesis (HOO))
– Treatment X has no effect on outcome YTreatment X has no effect on outcome Y
Alternative Hypothesis (HAlternative Hypothesis (HAA))
– Treatment X has an effect on outcome YTreatment X has an effect on outcome Y
23. Hypothesis TestingHypothesis Testing
If the trial has been carefully controlled,If the trial has been carefully controlled,
there are only two explanations for athere are only two explanations for a
difference between treatment groups—difference between treatment groups—
efficacy of X, and chanceefficacy of X, and chance
Assuming that the null hypothesis is correct,Assuming that the null hypothesis is correct,
we can use a statistical test to calculate thatwe can use a statistical test to calculate that
the observed difference would havethe observed difference would have
occurred. This is known as the significanceoccurred. This is known as the significance
level, or p-value of the test.level, or p-value of the test.
24. Hypothesis TestingHypothesis Testing
P-valueP-value
– The probability of the observed outcome,The probability of the observed outcome,
assuming that chance alone was involvedassuming that chance alone was involved
in creating the outcome. In other words,in creating the outcome. In other words,
assuming the null hypothesis is correct,assuming the null hypothesis is correct,
what is the probability that we would havewhat is the probability that we would have
seen the observed outcome.seen the observed outcome.
– This is only meaningful if chance is theThis is only meaningful if chance is the
only competing plausible explanation.only competing plausible explanation.
25. Hypothesis TestingHypothesis Testing
If the p-value is small, meaning theIf the p-value is small, meaning the
observed outcome would have beenobserved outcome would have been
unlikely, we will reject that chanceunlikely, we will reject that chance
played the only role in the observedplayed the only role in the observed
difference between groups anddifference between groups and
conclude that treatment X does in factconclude that treatment X does in fact
have an effect on outcome Yhave an effect on outcome Y
How small is small?How small is small?
26. Hypothesis TestingHypothesis Testing
Reality ->Reality ->
DecisionDecision
HHOO is trueis true HHOO is falseis false
Retain HRetain HOO CorrectCorrect
DecisionDecision
Type II ErrorType II Error
((ββ))
(.2, .1)(.2, .1)
Reject HReject HOO Type I ErrorType I Error
((αα))
(.05, .01)(.05, .01)
CorrectCorrect
DecisionDecision
27. Hypothesis TestingHypothesis Testing
Rules of thumb for effect sizes:Rules of thumb for effect sizes:
– Small=.2Small=.2
– Medium=.5Medium=.5
– Large=.8Large=.8
So, if you want 80% chance of detecting aSo, if you want 80% chance of detecting a
medium effect, using a .05medium effect, using a .05 αα value,value,
N= 4(1.96+.84)N= 4(1.96+.84)22
/.5/.52 =2 =
about 126, or 63 inabout 126, or 63 in
each groupeach group
28. Point EstimationPoint Estimation
Hypothesis testing can only tell you whetherHypothesis testing can only tell you whether
or not the effect of X is zero, it does not tellor not the effect of X is zero, it does not tell
you how large or small the effect is.you how large or small the effect is.
Important—a p-value is not an indication ofImportant—a p-value is not an indication of
the size of an effect, it depends greatly onthe size of an effect, it depends greatly on
sample sizesample size
If you want an estimate of the actual effect,If you want an estimate of the actual effect,
you need confidence intervalsyou need confidence intervals
29. Point EstimationPoint Estimation
Confidence intervals give you an idea ofConfidence intervals give you an idea of
what the actual effect is likely to be in thewhat the actual effect is likely to be in the
population of interestpopulation of interest
The most common confidence interval isThe most common confidence interval is
95% and gives an upper and lower bound95% and gives an upper and lower bound
on what the effect is likely to be.on what the effect is likely to be.
The size of the interval depends on theThe size of the interval depends on the
sample size, variability of the measure, andsample size, variability of the measure, and
the degree of confidence you want that thethe degree of confidence you want that the
interval contains the true effect.interval contains the true effect.
30. Point EstimationPoint Estimation
Many people prefer confidence intervals toMany people prefer confidence intervals to
hypothesis testing, because confidencehypothesis testing, because confidence
intervals contain more informationintervals contain more information
Not only can you tell whether the effectNot only can you tell whether the effect
could be zero (is zero contained in thecould be zero (is zero contained in the
interval of possible effect values?) but youinterval of possible effect values?) but you
also have the entire range of possiblealso have the entire range of possible
values the effect could bevalues the effect could be
So, a confidence interval gives you all theSo, a confidence interval gives you all the
information of a hypothesis test and a wholeinformation of a hypothesis test and a whole
lot more.lot more.
31. Choosing the right testChoosing the right test
Typically one is interested inTypically one is interested in
comparing group means.comparing group means.
If the outcome is continuous, and oneIf the outcome is continuous, and one
independent variable:independent variable:
– Two groups—t-testTwo groups—t-test
– Three or more groups--ANOVAThree or more groups--ANOVA
32. Choosing the right testChoosing the right test
If the outcome is continuous and thereIf the outcome is continuous and there
is more than one independentis more than one independent
variable:variable:
– ANOVA, if all independent variables areANOVA, if all independent variables are
categoricalcategorical
– ANCOVA or multiple linear regression, ifANCOVA or multiple linear regression, if
some independent variables aresome independent variables are
continuouscontinuous
33. Choosing the right testChoosing the right test
If the outcome is binary:If the outcome is binary:
– Logistic regressionLogistic regression
If outcome is time until a specifiedIf outcome is time until a specified
outcome:outcome:
– Survival analysis—Cox proportionalSurvival analysis—Cox proportional
hazards regressionhazards regression
34. Parametric vs. Non-Parametric vs. Non-
parametric testsparametric tests
Parametric tests are tests that use a knownParametric tests are tests that use a known
probability distribution to assess the p-value of theprobability distribution to assess the p-value of the
outcome.outcome.
Most outcomes do fairly closely follow a knownMost outcomes do fairly closely follow a known
probability distribution, and many tests are robust toprobability distribution, and many tests are robust to
violations of distributional assumptions, so theviolations of distributional assumptions, so the
assigned p-value will be fairly accurate in manyassigned p-value will be fairly accurate in many
situationssituations
For unique situations, such as specializedFor unique situations, such as specialized
outcomes or very skewed distributions, one canoutcomes or very skewed distributions, one can
generate their own probability distribution togenerate their own probability distribution to
calculate a p-value. (jacknife, bootstrap, etc.)calculate a p-value. (jacknife, bootstrap, etc.)
Computers make these techniques very fast andComputers make these techniques very fast and
easyeasy