Vanderbilt Biostatistics Departmental Seminar

1. Clarifying and Contextualizing
Sensitivity to Unmeasured
Confounding Tipping Point
Analyses
Lucy D’Agostino McGowan
Robert Greevy, Jr

2. https://ryan-j-apps.shinyapps.io/
seminarSurvival/

3. motivation

5. review of 90 observational
studies 2015

6. review of 90 observational
studies 2015
45.6% mentioned unmeasured
confounding as a limitation

7. review of 90 observational
studies 2015
45.6% mentioned unmeasured
confounding as a limitation
4.4% included a quantitative
sensitivity analysis 😑

8. "In a study with binary
outcomes and binary exposures
the relative risk may be off
by a factor of 2, but unlikely
to be off more than that."
van Belle, G. (2011). Statistical Rules of Thumb. Wiley.

10. Stampfer, M. J., & Colditz, G. A. (1991). Estrogen replacement therapy and coronary heart disease: A
quantitative assessment of the epidemiologic evidence. Preventive Medicine, 20(1), 47–63. http://doi.org/
10.1016/0091-7435(91)90006-P

11. Stampfer, M. J., & Colditz, G. A. (1991). Estrogen replacement therapy and coronary heart disease: A
quantitative assessment of the epidemiologic evidence. Preventive Medicine, 20(1), 47–63. http://doi.org/
10.1016/0091-7435(91)90006-P
RR 0.5
(0.43,0.56)

12. Stampfer, M. J., & Colditz, G. A. (1991). Estrogen replacement therapy and coronary heart disease: A
quantitative assessment of the epidemiologic evidence. Preventive Medicine, 20(1), 47–63. http://doi.org/
10.1016/0091-7435(91)90006-P

15. Roumie, C. L., Min, J. Y., D’Agostino McGowan, L., Presley, C., Grijalva, C. G., Hackstadt, A. J., et al.
(2017). Comparative Safety of Sulfonylurea and Metformin Monotherapy on the Risk of Heart Failure: A Cohort
Study. Journal of the American Heart Association, 6(4), e005379. http://doi.org/10.1161/JAHA.116.005379

16. Roumie, C. L., Min, J. Y., D’Agostino McGowan, L., Presley, C., Grijalva, C. G., Hackstadt, A. J., et al.
(2017). Comparative Safety of Sulfonylurea and Metformin Monotherapy on the Risk of Heart Failure: A Cohort
Study. Journal of the American Heart Association, 6(4), e005379. http://doi.org/10.1161/JAHA.116.005379
HR 1.32
(1.21,1.43)

17. background

18. Cornfield 1959

19. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

20. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

21. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

22. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

23. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

24. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

25. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

26. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

27. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

28. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

29. Cornfield 1959
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

30. Cornfield 1959
9
R1
R0
=
p1RU + (1 p1)R¯U
p0RU + (1 p0)R¯U

31. Cornfield 1959
p1
p0
=
R1
R0
+
1
p0
R¯U
RU

(1 p0)
R1
R0
(1 p1)

32. Cornfield 1959
p1
p0
=
R1
R0
+
1
p0
R¯U
RU

(1 p0)
R1
R0
(1 p1)
+

33. Cornfield 1959
p1
p0
>
R1
R0

34. Cornfield 1959
p1
p0
> 9

35. Cornfield 1959
the proportion of hormone-X-
producers among cigarette smokers
must be at least 9 times greater
than that of nonsmokers...

36. Cornfield 1959
Bross 1966

37. Cornfield 1959
Bross 1966
Schlesselman 1978

38. Cornfield 1959
Bross 1966
Schlesselman 1978
RRobs = RRadj
p1 + (1 p1)
p0 + (1 p0)

39. Cornfield 1959
Bross 1966
Schlesselman 1978
RRobs = RRadj
p1 + (1 p1)
p0 + (1 p0)
R1
R0

40. Cornfield 1959
Bross 1966
Schlesselman 1978
RRobs = RRadj
p1 + (1 p1)
p0 + (1 p0)

41. Cornfield 1959
Bross 1966
Schlesselman 1978
RRobs = RRadj
p1 + (1 p1)
p0 + (1 p0)
RU
R¯U

42. Cornfield 1959
Bross 1966
Schlesselman 1978
RRobs = RRadj
p1 + (1 p1)
p0 + (1 p0)

43. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983

44. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998

45. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)

46. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)

47. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)

48. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)
HRadj HRobs

49. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)
HRadj HRobs

50. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)
HRadj HRobs

51. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we fit
P(Y = 1|X, Z) =
exp{↵⇤
+ ⇤
X + ✓⇤0
Z}

52. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we wish we fit
exp{↵ + X + ✓0
Z}(exp{ X }pX,Z + (1 pX,Z))

53. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we wish we fit
exp{↵ + X + ✓0
Z}(exp{ X }pX,Z + (1 pX,Z))

54. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we wish we fit
exp{↵ + X + ✓0
Z}(exp{ X }pX,Z + (1 pX,Z))PX PX

55. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we wish we fit
exp
n
↵ + log{e 0
p0 + (1 p0)}
+
✓
+ log
e 1
p1 + (1 p1)
e 0 p0 + (1 p0)
◆
X + ✓0
Z
o

56. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
what we wish we fit
exp
n
↵ + log{e 0
p0 + (1 p0)}
+
✓
+ log
e 1
p1 + (1 p1)
e 0 p0 + (1 p0)
◆
X + ✓0
Z
o

57. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
solving for
what we wish we fit
= ⇤
log
e 1
p1 + (1 p1)
e 0 p0 + (1 p0)

58. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)

59. Cornfield 1959
Bross 1966
Schlesselman 1978
Rosenbaum &
Rubin 1983
Lin 1998
Ding &
VanderWeele
2016

60. tipping point

62. what will tip our
confidence bound to cross 1

63. what will tip our
confidence bound to cross 1
assuming the sensitivity
parameters are fixed and
known, we can extend these
methods to confidence
bounds

64. RRadj = RRobs
p0 + (1 p0)
p1 + (1 p1)

65. LBadj = LBobs
p0 + (1 p0)
p1 + (1 p1)

66. 1
LBadj = LBobs
p0 + (1 p0)
p1 + (1 p1)

67. 1
LBadj = LBobs
p0 + (1 p0)
p1 + (1 p1)

69. (LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

70. (LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

71. "in order for our association
to no longer be significant,
there would need to exist an
unmeasured confounder of size
that is prevalent in p1 of the
exposed population and p0 of
the unexposed population”

72. "in order for our association
to no longer be significant,
there would need to exist an
unmeasured confounder of size
that is prevalent in p1 of the
exposed population and p0 of
the unexposed population”

73. (LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

74. (LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

78. p1 = 0.096
p0 = 0.038

79. p1 = 0.096
p0 = 0.038
= 2.34

80. calculate it

81. LB = 1.21
p1 = 0.096
p0 = 0.038
(LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

82. LB = 1.21
p1 = 0.096
p0 = 0.038
0.1
(LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

83. LB = 1.21
p1 = 0.096
p0 = 0.038
0.1
0
(LBobs, p0, p1) =
LBobs(1 p0) (1 p1)
p1 LBobsp0

85. > devtools::install_github(“LucyMcGowan/tipr”)
> library(‘tipr’)
> tip(p1 = 0.1,
p0 = 0,
lb = 1.21,
ub = 1.43)
[1] 3.1

87. > tip(p1 = 0.1, p0 = 0, lb = 1.21, ub = 1.43,
explanation = TRUE)
[1] "An unmeasured confounder of size 3.1
with a prevalence of 0.1 in the exposed
population and 0 in the unexposed population
would tip your (1.21,1.43) result to
nonsignificance."

88. lucy.shinyapps.io/tipr

89. anchor it

91. p1 = 0.096
p0 = 0.038

92. p1 = 0.096
p0 = 0.038
= 2.34

93. A hypothetical unobserved
binary confounder with a 10%
prevalence difference between
the therapies would need to
have an association with heart
failure of HR=3.1 to tip this
analysis to nonsignificance at
a 5% level.

94. For a comparison from the observed
confounders, baseline heart failure
history had a prevalence difference
of 5.8% in the pre-matching cohort
and an association with CHF of
HR=2.34; thus an unmeasured
confounder of this magnitude would
be insufficient to tip this analysis
to statistical insignificance.

95. write up

97. 1. state primary analysis
result

98. 1. state primary analysis
result

99. 1. state primary analysis
result
2. calculate the size &
prevalence of a hypothetical
tipping point confounder

100. 1. state primary analysis
result
2. calculate the size &
prevalence of a hypothetical
tipping point confounder

101. 1. state primary analysis
result
2. calculate the size &
prevalence of a hypothetical
tipping point confounder
3. anchor this in an example
from your data

102. conclusion

104. we have presented a useful, easily
implemented, and intuitively
understood approach to allow
researchers to assess the potential
impact of unmeasured confounders in
observational research

105. we have presented a useful, easily
implemented, and intuitively
understood approach to allow
researchers to assess the potential
impact of unmeasured confounders in
observational research
can be applied to both past and
future research, allowing readers to
understand the sensitivity of studies
that do not include such an analysis,
and allowing future investigators to
readily include such an analysis.

106. On the other hand, it took us embarrassingly long to clue in to the lung
cancer/cigarette thing, so I guess the real lesson is "figuring out which ideas
are true is hard."
https://xkcd.com/1592

107. tid bits

108. what about multiple
unmeasured
confounders?

109. bias factors

110. design
sensitivity

111. next steps

113. tutorial

114. tutorial
bias factors

115. tutorial
bias factors
design sensitivity

116. tutorial
bias factors
design sensitivity
puppets