2. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
3. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
5. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
6. Identify the primary research
question
Where to find the research question?
– Title of the study
– The objective(s)
– The conclusion(s)
If more than one, find the primary aim.
Try to make the question “quantifiable”
7. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
8. Identify the primary study
outcome
It is the “primary” dependence variable
It is the main finding that was used as the basis
for the conclusion of the study
It is the target of the statistical inference
It is the basis for sample size calculation
It resided in the :
–
–
–
–
–
–
Title
Research question
Objective
Sample size calculation
Main finding in the RESULTS section of the report
Conclusion
9. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
10. Type of the study outcome: Key for
selecting appropriate statistical methods
Study outcome
– Dependent variable or response variable
– Focus on primary study outcome if there are
more
Type of the study outcome
– Continuous
– Categorical (dichotomous, polytomous, ordinal)
– Numerical (Poisson) count
– Even-free duration
11. Continuous outcome
Parameters:
– Mean (SD)
– Median (Min:Max)
– Correlation coefficient: r and ICC
Modeling:
– Linear regression
The model coefficient = Mean difference
– Quantile regression
The model coefficient = Median difference
Example:
– Outcome = Weight, BP, score of ?, level of ?, etc.
– RQ: Factors affecting birth weight
12. Categorical outcome
Parameters:
– Proportion or Risk
Modeling:
– Logistic regression
The model coefficient = Odds ratio (OR)
Example:
– Outcome = Disease (y/n), Dead(y/n),
cured(y/n), etc.
– RQ: Factors affecting low birth weight
13. Numerical (Poisson) count outcome
Parameters:
– Incidence rate (e.g., rate per person time)
Modeling:
– Poisson regression
The model coefficient = Incidence rate ratio (IRR)
Example:
– Outcome =
Total number of falls
Total time at risk of falling
– RQ: Factors affecting tooth elderly fall
14. Event-free duration outcome
Parameters:
– Median survival time
Modeling:
– Cox regression
The model coefficient = Hazard ratio (HR)
Example:
– Outcome = Overall survival, disease-free
survival, progression-free survival, etc.
– RQ: Factors affecting survival
15. The outcome determine statistics
Continuous
Mean
Median
Categorical
Proportion
(Prevalence
Or
Risk)
Linear Reg.
Count
Survival
Rate per “space”
Median survival
Risk of events at T(t)
Logistic Reg. Poisson Reg.
Cox Reg.
19. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
20. Types of Research
Quantitative
Qualitative
Phenomenology
Grounded Theory
Ethnography
Description
Observational
Experimental
Quasi-experimental
Descriptive
Analytical
Clinical trial
Field trial
Community intervention trial
Cross-sectional descriptive
Prevalence survey
Poll
Cross-sectional
Case-control
Prevalence case-control
Nested case-control
Case-cohort case-control
Randomized-controlled
Cohort
Parallel or Cross-over or factorial
Fixed length or group sequential
With or without baseline
Prospective cohort
Retrospective cohort
Ambi-spective cohort
Systematic review
Meta-analysis
23. Caution about biases
Selection bias (SB)
Information bias (IB)
Confounding bias (CB)
If data available:
7
SB & IB can be assessed
CB can be adjusted using
multivariable analysis
24. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
25. Generate a data set that
would've been used
General format of the data layout
id
1
2
3
4
5
…
n
y
x1
x2
X3
26. Generate a data set that
would've been used
Continuous outcome example
id
1
2
3
4
5
…
n
y
2
2
0
2
14
x1
1
0
1
0
1
x2
21
12
4
89
0
X3
22
19
20
21
18
6
0
45
21
Mean (SD)
27. Generate a data set that
would've been used
Continuous outcome example
id
1
2
3
4
5
…
n
y
1
1
0
0
0
x1
1
0
1
0
1
x2
21
12
4
89
0
X3
22
19
20
21
18
0
0
45
21
n, percentage
28. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
29. Common types of the statistical goals
Single measurements (no comparison)
Difference (compared by subtraction)
Ratio (compared by division)
Prediction (diagnostic test or predictive
model)
Correlation (examine a joint distribution)
Agreement (examine concordance or
similarity between pairs of observations)
30. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
31.
32.
33. Dependency of the study outcome required
special statistical methods to handle it
Example of dependency or correlated data:
–
–
–
–
Before-after or Pre-post design
Measuring paired organs i.e., ears, eyes, arms, etc.
Longitudinal data, repeated measurement
Clustered data, many observation unit within a cluster
Choices of approaches:
– Ignore it => use ordinary analysis as independency not save
– Simplify it => use summary measure then analyze the
data as it is independent – not efficient
– Handle it => Mixed model, multilevel modeling, GEE recommended
34. Dependency of the study outcome required
special statistical methods to handle it
Continuous
Mean
Median
Categorical
Proportion
(Prevalence
Or
Risk)
Linear Reg.
Count
Survival
Rate per “space”
Median survival
Risk of events at T(t)
Logistic Reg. Poisson Reg.
Mixed model, multilevel model, GEE
Cox Reg.
35. Back to the conclusion
Continuous
Categorical
Count
Survival
Appropriate statistical methods
Mean
Median
Proportion
(Prevalence or Risk)
Rate
per “space”
Median survival
Risk of events at T(t)
Magnitude of effect
95% CI
Answer the research question
based on lower or upper limit of the CI
P-value
36. Always report the magnitude of
effect and its confidence interval
Absolute effects:
– Mean, Mean difference
– Proportion or prevalence, Rate or risk, Rate or Risk difference
– Median survival time
Relative effects:
– Relative risk, Rate ratio, Hazard ratio
– Odds ratio
Other magnitude of effects:
–
–
–
–
Correlation coefficient (r), Intra-class correlation (ICC)
Kappa
Diagnostic performance
Etc.
37. Touch the variability (uncertainty)
to understand statistical inference
id
A
1
2
2
2
3
4
0
2
5
14
20
Sum (Σ)
Mean(X)
SD
Median
4
(x-X) (x- X ) 2
-2
4
-2
4
-4
16
-2
10
4
100
0
0
128
32.0
2+2+0+2+14 = 20
2+2+0+2+14 = 20 = 4
5
5
0
2
2
2
14
Variance = SD2
5.66
2
Standard deviation = SD
38. Touch the variability (uncertainty)
to understand statistical inference
id
A
1
2
2
2
3
4
0
2
5
14
20
Sum (Σ)
Mean(X)
SD
Median
4
(x-X) (x- X ) 2
-2
4
-2
4
-4
16
-2
10
4
100
0
0
128
32.0
5.66
2
Measure of
central tendency
Measure of
variation
39. Standard deviation (SD) = The average distant between
each data item to their mean
(X − X)
∑
SD =
n −1
2
Degree of freedom
40. Same mean BUT different variation
id
A
id
B
1
2
2
2
1
2
4
3
3
4
0
2
3
4
5
4
5
14
20
5
4
20
Sum (Σ)
Mean
SD
Median
4
5.66
Sum (Σ)
Mean
SD
4
0.71
2
Median
4
Heterogeneous data
Homogeneous data
41. Facts about Variation
Because of variability, repeated samples will
NOT obtain the same statistic such as mean or
proportion:
– Statistics varies from study to study because of the
role of chance
– Hard to believe that the statistic is the parameter
– Thus we need statistical inference to estimate the
parameter based on the statistics obtained from a
study
Data varied widely = heterogeneous data
Heterogeneous data requires large sample size
to achieve a conclusive finding
47. Central Limit Theorem
Distribution of
the raw data
X1
XX
µ
Xn
Distribution of
the sampling mean
Large sample
(Theoretical) Normal Distribution
48. Central Limit Theorem
Many X, X , SD
X1
XX
Xn
µ
Standard deviation of the sampling mean
Standard error (SE)
Estimated by
SE =
SD
√n
Many X , XX , SE
Large sample
Standardized for whatever n,
Mean = 0, Standard deviation = 1
57. Report and interpret p-value appropriately
Example of over reliance on p-value:
– Real results: n=5900; ORDrug A vs Drug B = 1.02
(P<0.001)
– Inappropriate: Quote p-value as < 0.05 or put *
or **** (star) to indicate significant results
– Wrong: Drug A is highly significantly better
than Drug B (P<0.001)
– What if 95%CI: 1.001 to 1.300?
– This is no clinical meaningful at all….!
58. Report and interpret p-value appropriately
Example of over reliance on p-value:
– Real results: n=30; ORDrug A vs Drug B = 9.2 (P=0.715)
– Inappropriate: Quote p-value as > 0.05
– Wrong: There is no statistical significant
difference of the treatment effect (P<0.05).
Thus Drug A is as effective as Drug B
– What if 95%CI: 0.99 to 28.97?
– This is study indicated a low power, NOT
suggested an equivalence…!
– Correct: There was no sufficient information to
concluded that . . . => inconclusive findings
59. P-value is the magnitude of chance
NOT magnitude of effect
P-value < 0.05 = Significant findings
Small chance of being wrong in rejecting the null
hypothesis
If in fact there is no [effect], it is unlikely to get the
[effect] = [magnitude of effect] or more extreme
Significance DOES NOT MEAN importance
Any extra-large studies can give a very small Pvalue even if the [magnitude of effect] is very
small
60. P-value is the magnitude of chance
NOT magnitude of effect
P-value > 0.05 = Non-significant findings
High chance of being wrong in rejecting the null
hypothesis
If in fact there is no [effect], the [effect] =
[magnitude of effect] or more extreme can be
occurred chance.
Non-significance DOES NOT MEAN no
difference, equal, or no association
Any small studies can give a very large P-value
even if the [magnitude of effect] is very large
61. P-value vs. 95%CI (1)
An example of a study with dichotomous outcome
A study compared cure rate between Drug A and Drug B
Setting:
Drug A = Alternative treatment
Drug B = Conventional treatment
Results:
Drug A: n1 = 50, Pa = 80%
Drug B: n2 = 50, Pb = 50%
Pa-Pb
= 30% (95%CI: 26% to 34%; P=0.001)
62. P-value vs. 95%CI (2)
Pa > Pb
Pb > Pa
Pa-Pb = 30% (95%CI: 26% to 34%; P< 0.05)
63. P-value vs. 95%CI (3)
Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99
64. Tips #6 (b)
P-value vs. 95%CI (4)
Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99
There were statistically
significant different
between the two groups.
65. Tips #6 (b)
P-value vs. 95%CI (5)
Adapted from: Armitage, P. and Berry, G. Statistical methods in medical research. 3rd edition. Blackwell Scientific Publications, Oxford. 1994. page 99
There were no
statistically significant
different between the
two groups.
66. P-value vs. 95%CI (4)
Save tips:
– Always report 95%CI with p-value, NOT report
solely p-value
– Always interpret based on the lower or upper
limit of the confidence interval, p-value can be
an optional
– Never interpret p-value > 0.05 as an indication
of no difference or no association, only the CI
can provide this message.
67. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
68. The outcome determine statistics
Continuous
Mean
Median
Categorical
Proportion
(Prevalence
Or
Risk)
Linear Reg.
Count
Survival
Rate per “space”
Median survival
Risk of events at T(t)
Logistic Reg. Poisson Reg.
Cox Reg.
69. Dependency of the study outcome required
special statistical methods to handle it
Continuous
Mean
Median
Categorical
Proportion
(Prevalence
Or
Risk)
Linear Reg.
Count
Survival
Rate per “space”
Median survival
Risk of events at T(t)
Logistic Reg. Poisson Reg.
Mixed model, multilevel model, GEE
Cox Reg.
70. Back to the conclusion
Continuous
Categorical
Count
Survival
Appropriate statistical methods
Mean
Median
Proportion
(Prevalence or Risk)
Rate
per “space”
Median survival
Risk of events at T(t)
Magnitude of effect
95% CI
Answer the research question
based on lower or upper limit of the CI
P-value
71. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
72. Perform the data analysis using
a software
Use the data being generated similar to
what should have been used in the
research article
Analyze the way that the article did
Analyze the way it should be based on
your own judgments
Try to understand the computer output
Always ask yourself – does the output
answer the research question?
73. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
74. Report and interpret the
results from the outputs
Interpret the results based on the
confidence interval rather than the p-value
75. Retrospective Statistical Practices
(Read a published research article and do a critical appraisal)
Begin at the conclusion
Identify the primary research question
Identify the primary study outcome
Identify type of the study outcome
Identify type of the study design
Generate a data set that would've been used
Identify type of the main statistical goal
List choices of the statistical methods
Select the most appropriate statistical method
Perform the data analysis using a software
Report and interpret the results from the outputs
Summarize problems faced and lessons learned
76. Summarize problems faced and
lessons learned
Write in your own words
Evaluate what you wrote by the new round
of steps stated previously
Keep on doing this way
Eventually you will find statistics a logical
and intuitive tools for you