1. Calculating Effect Size
Power Analysis
Issues in Null Hypothesis
Significance Testing
Carlo Magno, PhD.
De La Salle University, Manila

2. A researcher wanted to look at the effect of behavior
modification technique on the aggression of clients. A group
of participants in the experimental group were given
behavior modification technique and no treatment in the
control. The aggression of the two groups were measured
after.
n
30
60
100
500
1000
df
28
58
98
498
998
X1
X2
4.6
4.6
4.6
4.6
4.6
4.1
4.1
4.1
4.1
4.1
SD1
SD2
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
2.3
t
0.60
0.84
1.09
2.43
3.44
p
value
0.56
0.4
0.28
0.02*
0.00*

4. Criticism on NHST
• 1. NHST does not provide the information which the
researcher wants to obtain
• 2. Logical problems derived from the probabilistic
nature of NHST.
• 3. NHST does not enable psychological theories to
be tested.
• 4. The fallacy of replication.
• 5. NHST fails to provide useful information because
H0 is always false.
• 6. Problems associated with the dichotomous
decision to reject/not reject the H0.
• 7. NHST impedes the advance of knowledge.

5. Alternatives to NHST
• Effect size
• Confidence levels
• Power analysis

6. Effect Size
• Cohen (1988) defines the effect size as the
extent to which the phenomenon is found within
the population or, in the context of statistical
significance testing, the degree to which the H0
is false.
• Snyder and Lawson (1993) argue that the effect
size indicates the extent to which the dependent
variable can be controlled, predicted and
explained by the independent variable(s).

7. Effect Size Measures
• Effect size measures of Two In/dependent
Groups
– Cohen’s d
– Hedges g
– Glass Delta
• Correlation Measure of Effect Size
–r
– χ2 ►Φ;
t ► r;
F ► r;
d►r
• Effect size for Analysis of Variance
– Eta Squared
– Omega Square Index of Strength
– Intercalss correlation

8. M1 − M2
d= 2
s 1 + s 22
2
Cohen’s d Formula
M1 − M2
t= 2
s 1 + s 22
n1 n2
t-test for independent Means
Formula

9. Computation
A research compared students who engaged
in group and individual sports on their
passion on the sport. Passion was
measured using the Passion Scale by
Vallerand with tow factors, harmonious
and obsessive passion. The t-test for two
independent samples was used to
determine the significant difference
between the students in the group and
individual sports on the two factors of
passion. The following statistical output
was obtained:

10. Statistical Results
M1
M2
t-value df
p
N1 N2 SD1 SD2
HP
5.51
5.68
-1.01 58 0.315 30 30 0.70 0.61
OP
4.91
5.36
-1.40 58 0.167 30 30 1.51 0.87
Compute for the effect size
http://www.uccs.edu/~lbecker/
http://effect-size-generator.software.informer.com/download/

11. Cohen's Standard
LARGE
MEDIUM
SMALL
Effect Size
Percentile Standing
2
1.9
1.8
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
97.7
97.1
96.4
95.5
94.5
93.3
91.9
90
88
86
84
82
79
76
73
69
66
62
58
54
50
Percent of
Nonoverlap
81.10%
79.40%
77.40%
75.40%
73.10%
70.70%
68.10%
65.30%
62.20%
58.90%
55.40%
51.60%
47.40%
43.00%
38.20%
33.00%
27.40%
21.30%
14.70%
7.70%
0%

12. Cohen's Standard
LARGE
MEDIUM
SMALL
d
2
1.9
1.8
1.7
1.6
1.5
1.4
1.3
1.2
1.1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
r
0.707
0.689
0.669
0.648
0.625
0.6
0.573
0.545
0.514
0.482
0.447
0.41
0.371
0.33
0.287
0.243
0.196
0.148
0.1
0.05
0
r2
0.5
0.474
0.448
0.419
0.39
0.36
0.329
0.297
0.265
0.232
0.2
0.168
0.138
0.109
0.083
0.059
0.038
0.022
0.01
0.002
0

13. Statistical Power
Reject H0
No real effect Real Effect
Type 1 error
α (.01, .05)
Ho not
rejected
Slim chance of concluding
that the treatment is
effective, despite the fact
that it is
Type 2 error
β (small as
possible)
1-β
Statistical
power

14. Statistical Power
• β=.20 (the error of rejecting a true Ho is 4x
more serious than the error of not
rejecting a false Ho)
• .80=acceptable power

15. Statistical Power
• The probability of rejecting a false null
hypothesis.
• The likelihood that a study will detect an
effect when there is an effect to be
detected.
• If statistical power is high, the probability
of making a Type II error, (or concluding
there is no effect when, in fact, there is
one) goes down.

16. Statistical Power
• The power of any test of statistical
significance will be affected by four main
parameters:
– the effect size
– the sample size (N)
– the alpha significance criterion (α)
– statistical power, or the chosen or implied
beta (β)

17. Statistical Power
Statistics Power small
r
.80
26
t
.80
29
medium Large
63
393
85
781
http://danielsoper.com/statcalc3/default.aspx
http://www.statisticalsolutions.net/pss_calc.php
https://www.dssresearch.com/KnowledgeCenter/toolkitcalculators/statisticalpow
ercalculators.aspx
http://homepage.stat.uiowa.edu/~rlenth/Power/

18. Influence of Effect Size on Power
High school n=65
College n=153

19. Influence of Effect Size on Power

20. Influence of Effect Size on Power
N=82 Taiwanese in Taiwan
N=98 Taiwanese in the Philippines

21. Influence of Effect Size on Power

22. • What inference can be gained between
effect size and power with fixed sample
size and alpha level?

23. Influence of Significance Level on
Power
• Study of De Frias, Dixon, and Strauss
(2006)
• N=418
• r=.14 (not significant)
• α=.01 (power=.23)
α=.05
Power=.45
α=.10
Power=.58
α=.15
Power=.65
α=.20
Power=.71

24. • What inference can be gained between
level of significance and power with fixed
sample size?

25. Influence of Sample size on
Power
Magno (2005)
Monitoring and metacognition
N=280
r=.14
Power=.65
Magno, Mamauag, & Parinas
(2007)
Independence and self-esteem
N=373
r=.14
Power=.78
Chemers, Hu, & Garcia (2001)
N=381
Challenge-threat and self-efficacy
r=.15
Power=.83

26. Influence of Sample size on
Power

27. • What inference can be gained between
sample size and power with fixed effect
size and significance level?