Breast cancer -ONCO IN MEDICAL AND SURGICAL NURSING.pptx
Fixed-effect and random-effects models in meta-analysis
1. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Fixed-effect vs. Random-effects
models in meta-analysis
Dr. S. A. Rizwan M.D.,
Public Health Specialist & Lecturer,
Saudi Board of Preventive Medicine – Riyadh,
Ministry of Health, KSA
25.11.2019 1
With thanks to Michael Borenstein
2. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Outline
• Provide a description of fixed and of random effects models
• Outline the underlying assumptions of these two models in order to
clarify the choices a reviewer has in a meta-analysis
• Discuss how to estimate key parameters in the model
25.11.2019 2
3. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Our choice between the two depends on
• Our assumption about how the effect sizes vary in our meta-analysis
• The two models are based on different assumptions about the nature
of the variation among effect sizes in our research synthesis
25.11.2019 3
4. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Sampling assumptions
• Fixed-effect
• Sampling takes place at one level only
• Any between-study variance will be ignored when assigning weights
• Random-effects
• Sampling takes place at two levels
• Any between-study variance will be used when assigning weights
25.11.2019 4
5. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Fixed-effect
• When there is reason to believe that all the studies are functionally identical
• When our goal is to compute the common effect size, for the studies in the
analysis
• Example of drug company has run five studies to assess the effect of a drug
• Note that the effect size from each study estimate a single common mean –
the fixed-effect
• We know that each study will give us a different effect size, but each effect
size is an estimate of a common mean, designated in the prior picture as θ
25.11.2019 5
6. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 6
7. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random-effects
• We assume two components of variation:
• – Sampling variation as in our fixed-effect model assumption
• – Random variation because the effect sizes themselves are sampled from
population of effect sizes
• In a random effects model, we know that our effect sizes will differ
because they are sampled from an unknown distribution
• Our goal in the analysis will be to estimate the mean and variance of
the underlying population of effect sizes
• All the studies are functionally equivalent
• Example of studies culled from publications
25.11.2019 7
8. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random-effects
• We see in the picture that each distribution has its own mean that is
sampled from the underlying population distribution of effect sizes
• That underlying population distribution also has its own variance, τ2,
commonly called the variance component
• Thus, each effect size has two components of variation, one due to
sampling error, and one from the underlying distribution
25.11.2019 8
9. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 9
10. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Definition of combined effect
• Fixed effect model
• There is one true effect
• Summary effect is estimate of this value
• Random effects model
• There is a distribution of effects
• Summary effect is mean of distribution
25.11.2019 10
11. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
How weights are assigned?
25.11.2019 11
12. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
W = 1/ (V1)
Fixed-effects model
25.11.2019 12
13. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random-effects model
25.11.2019 13
1W = 1/ (V +T 2
)
14. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
190
How weights shift?
• If within-study variance only, W=1/V
• If between-study variance only, W=1/T2
• If both, W=1/(V+T2)
25.11.2019 14
15. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 15
16. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 16
17. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random vs. Fixed
25.11.2019 17
18. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random vs. Fixed
25.11.2019 18
RE weights are more balanced
19. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Random vs. Fixed
25.11.2019 19
RE confidence interval is wider
20. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Large study has less impact under RE
25.11.2019 20
21. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Small study has more impact under RE
25.11.2019 21
22. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
FE
25.11.2019 22
23. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Summary effect in Fixed model
25.11.2019 23
244.215
101.171
= 0.414FE
24. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Variance of summary effect in fixed model
25.11.2019 24
1
244.215
= 0.004FE
25. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
RE
25.11.2019 25
26. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Summary effect in random model
25.11.2019 26
90.284
32.342
= 0.358RE
27. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Variance of summary effect in random model
25.11.2019 27
1
90.284
= 0.011RE
28. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Why does it matter?
• One matches the sampling
• One does not match the sampling
• Wrong model yields incorrect weights
• Estimate of mean is wrong
• Estimate of CI is wrong
• MUST choose based on sampling model
• The meaning of the ES is different
• Relative weights are closer under RE (effect size will shift)
• Absolute weights are smaller under RE (CI will become wider)
• p-value will change (less significant in long run but can go either way)
25.11.2019 28
29. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Test of null
25.11.2019 29
M
M
Z =
SE
M*
M*
Z* =
SE
Fixed
Random
Two
sources
of error
One
source
of error
30. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Implications of the choice
• If you expect that the effect size from each study arises from a different
population then use random effects
• Random effects model is more likely to give non-significant estimates
• Recall that all our analyses in a meta-analysis are weighted by the inverse of
the variance of the effect size, i.e., by the precision of the effect size estimate
• Because we have more variation assumed in a random effects model, our
weights for each study will be more equal to one another
• In other words, in a fixed effect model, we will more heavily weight larger
studies. In a random effects model, the larger studies will not be weighted as
heavily
25.11.2019 30
31. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Question!
• Suppose we had four studies, each with
• N =1,000,000, and a true (mean) effect size of 50.
• Under the two models,
• What would the forest plot look like?
• What would the diamond look like?
25.11.2019 31
32. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 32
33. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 33
34. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Which model should we use?
• Base decision on the model that matched the way the data were
collected
• Not on test of homogeneity
25.11.2019 34
35. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
What you may hear
• Fixed-effect is simple model
• Random-effects is more complicated
25.11.2019 35
36. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Actually
• Fixed-effect is more restricted model
• Random-effects makes less assumptions
25.11.2019 36
37. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
An alternate view
• Random-effects model only makes sense if we have a clear picture of
the sampling frame
• Otherwise, we should report the mean and CI for the studies in our
sample without attempt to generalize to a larger universe
25.11.2019 37
38. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Goals in a fixed effect model
• In a fixed effect model, we will be most interested in estimating our
common effect, θ, and its standard error
• We will also want to know if there is heterogeneity present by
computing Q
• HOWEVER, WE WILL NOT decide on the basis of a significant Q that
we should really do a random effects model
• WE WILL make a decision about fixed effect versus random effects
models because of our substantive knowledge of the area of the
systematic review
25.11.2019 38
39. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Goals in a random effects model
• We also want to estimate a mean effect size, but now this is the mean
effect size from the underlying population, μ
• We also want to estimate the variance of the underlying effect size, τ2
• We will test heterogeneity by testing whether τ2 is different from 0.
• Biggest difficulty: how to estimate τ2
25.11.2019 39
40. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Estimating the variance component, τ2
• There are two main methods for computing τ2
• Variously called the method of moments, the DerSimonian/Laird estimate
• Restricted maximum likelihood
• The method of moments estimator is easy to compute and is based
on the value of Q, the homogeneity statistic
• Restricted maximum likelihood requires an iterative solution
25.11.2019 40
41. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Method of moments estimator
25.11.2019 41
42. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Restricted maximum likelihood estimator
• Many statisticians do not like the method of moments estimator
• Can estimate τ2 using HLM, SAS Proc Mixed, or R
25.11.2019 42
43. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Differences between the two
models
25.11.2019 43
44. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
ES assumptions
Fixed effect model
• We assume that the treatment
effect is the same in all trials.
• We use only the sampling
variation within the trials.
Random effects model
• We assume that the treatment
effect is the not same in all trials.
• The trials are a sample from a
population of possible of trials
where the treatment effect varies.
• We use the sampling variation
within the trials and the sampling
variation between trials.
25.11.2019 44
45. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Sampling assumptions
Fixed effect model
• If the treatment effect is the
same in all trials, it is more
powerful and easier.
• No assumption about
representativeness.
Random effects model
• Less powerful because P values are
larger and confidence intervals are
wider.
• The trials are a sample from a
population of possible of trials
where the treatment effect varies.
• They must be a representative or
random sample.
• Very strong assumption.
25.11.2019 45
46. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Variance
Fixed effect model
• Variance of treatment effect in
trial = standard error squared.
• Weight = 1/variance = 1/SE2
Random effects model
• Variance of treatment effect in trial
= standard error squared plus inter-
trial variance
• Weight = 1/variance.
• 1
= --------------------------------
SE2 + inter-trial variance
• Inter-trial variance has degrees of
freedom given by number of trials
minus one.
• Typically small.
25.11.2019 46
47. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
When heterogeneity exists
Fixed effect model
• When heterogeneity exists we
get:
• a pooled estimate which may give
too much weight to large studies,
• a confidence interval which is too
narrow,
• a P-value which is too small.
Random effects model
• When heterogeneity exists we
get:
• possibly a different pooled
estimate with a different
interpretation,
• a wider confidence interval,
• a larger P-value.
25.11.2019 47
48. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
When heterogeneity doesn’t exist
Fixed effect model
• When heterogeneity does not
exists:
• a pooled estimate which is
correct,
• a confidence interval which is
correct,
• a P-value which is correct.
Random effects model
• When heterogeneity does not
exist:
• a pooled estimate which is
correct,
• a confidence interval which is too
wide,
• a P-value which is too large.
25.11.2019 48
49. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 25.11.2019 49
50. Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 06/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Thank you
Kindly email your queries to sarizwan1986@outlook.com
25.11.2019 50