1. Linear Regression
with
2: Model selection
2012-12-10 @HSPH
Kazuki Yoshida, M.D.
MPH-CLE student
FREEDOM
TO
KNOW

2. Group Website is at:
http://rpubs.com/kaz_yos/useR_at_HSPH

3. Previously in this group
n Introduction n Graphics
n Reading Data into R (1) n Groupwise, continuous
n Reading Data into R (2) n Linear regression
n Descriptive, continuous
n Descriptive, categorical
n Deducer

4. Menu
n Linear regression: Model selection

5. Ingredients
Statistics Programming
n Selection methods n step()
n drop1()
n add1()
n leaps::regsubsets()

6. Open
R Studio

7. Open the saved
script that we
created last time.
See also Linear Regression with R 1 slides

8. Create full & null models
lm.full <- lm(bwt ~ age + lwt + smoke + ht + ui + ftv.cat +
race.cat + preterm, data = lbw)
lm.null <- lm(bwt ~ 1, data = lbw)
Intercept-only

9. Compare two models
anova(lm.full, lm.null)
Model 1 Model 2

10. Models
Partial F-test
Difference in residual SS
Residual sum of squares
Residual degree of freedom Significant

11. Backward elimination
Specify full model
lm.step.bw <- step(lm.full, direction = "backward")
Final model object

12. Initial
AIC Removing ftv.cat
for full makes AIC smallest
model
Removing age
makes AIC smallest
Doing nothing
makes AIC smallest

13. Forward selection
Final model object
Specify null model
lm.step.fw <- step(lm.null,
scope = ~ age + lwt + smoke + ht + ui + ftv.cat +
race.cat + preterm,
direction = "forward")
formula for possible
variables

14. Initial
AIC
for Adding ui
null makes AIC smallest
model
Adding race.cat
makes AIC smallest
Adding smoke
makes AIC smallest
Still goes on ...

15. Stepwise selection/elimination
Final model object
Specify null model
lm.step.both <- step(lm.null,
scope = ~ age + lwt + smoke + ht + ui + ftv.cat +
race.cat + preterm,
direction = "both")
formula for possible
variables

16. Initial
AIC Adding ui
for makes AIC smallest
null
model
Adding race.cat
Removing is makes AIC smallest
also considered
Adding smoke
Removing is makes AIC smallest
also considered
Still goes on ...

17. F-test using drop1()
## age is the least significant by partial F test
drop1(lm.full, test = "F")
## After elimination, ftv.cat is the least significant
drop1(update(lm.full, ~ . -age), test = "F")
## After elimination, preterm is least significat at p = 0.12.
drop1(update(lm.full, ~ . -age -ftv.cat), test = "F")
## After elimination, all variables are significant at p < 0.1
drop1(update(lm.full, ~ . -age -ftv.cat -preterm), test = "F")
## Show summary for final model
summary(update(lm.full, ~ . -age -ftv.cat -preterm))

18. Updating models
## Remove age from full model
lm.age.less <- update(lm.full, ~ . -age)
all variables(.) minus age
## Adding ui to null model
lm.ui.only <- update(lm.null, ~ . +ui)
all variables (.) plus ui

19. test full model
age least significant
F-test comparing age-in
model to age-out model
remove age, and test
ftv.cat least significant
remove age, ftv.cat

20. F-test using add1()
## ui is the most significant variable
add1(lm.null, scope = ~ age + lwt + race.cat + smoke + preterm +
+ ui + ftv.cat, test = "F")
## After inclusion, race.cat is the most significant
add1(update(lm.null, ~ . +ui), scope = ~ age + lwt + race.cat +
smoke + preterm + ht + ui + ftv.cat, test = "F")
## After inclusion, smoke is the most significant
add1(update(lm.null, ~ . +ui +race.cat), scope = ~ age + lwt +
race.cat + smoke + preterm + ht + ui + ftv.cat, test = "F")
## After inclusion, ht is the most significant
add1(update(lm.null, ~ . +ui +race.cat +smoke), scope = ~ age + l
+ race.cat + smoke + preterm + ht + ui + ftv.cat, test = "F")
...

21. test null model
ui most significant
F-test comparing ui-out
model to ui-in model
add ui, and test
race.cat most significant
add ui and race.cat

22. All-subset regression
using leaps package

23. library(leaps)
regsubsets.out <-
regsubsets(bwt ~ age + lwt + smoke + ht + ui + ftv.cat +
race.cat + preterm,
data = lbw,
nbest = 1,
nvmax = NULL,
force.in = NULL, force.out = NULL,
method = "exhaustive")
summary(regsubsets.out)

24. library(leaps) Result object
regsubsets.out <-
regsubsets(bwt ~ age + lwt + smoke + ht + ui + ftv.cat +
race.cat + preterm, Full model
data = lbw, How many best models?
Max
model size nbest = 1, Forced variables
nvmax = NULL,
force.in = NULL, force.out = NULL,
method = "exhaustive")
summary(regsubsets.out)

25. Forced variables
Variable combination Best 1
predictor
model
Best 7
predictor
model
Best 10
predictor
model

26. plot(regsubsets.out, scale = "adjr2", main = "Adjusted R^2")
the higher the better
~ lwt + smoke + ht + ui
+ race.cat + preterm
~ smoke + ht + ui + race
~ ui

27. library(car)
subsets(regsubsets.out, statistic="adjr2", legend = FALSE,
min.size = 5, main = "Adjusted R^2")
~ lwt + smoke + ht + ui
+ race.cat + preterm

28. subsets(regsubsets.out, statistic="cp", legend = FALSE,
min.size = 5, main = "Mallow Cp")
First model for which Mallow
Cp is less than number of
regressors + 1
~ lwt + smoke + ht + ui
+ race.cat + preterm