Lecture 16 from my mixed-effects modeling course: Empirical logit and Poisson models

1. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab

2. Clarification on Logit Models
• Dependent/outcome variable would be a
variable made up of 0s and 1s
• We don’t need to apply any kind of transformation
• glmer(Recalled ~ 1 + StudyTime * Strategy +
(1|Subject) + (1|WordPair),
data=cuedrecall, family=binomial)
• family=binomial tells R to analyze this using the logit
link function—everything handled automatically

3. Clarification on Logit Models
• When we get our results, they will be in terms
of log odds/logits (because we used
family=binomial)
• We may want to use exp() to transform our
estimates into effects on the odds to make them
more interpretable
• Elaborative rehearsal = +2.29 log odds of recall
• exp(2.29) = Odds of recall 9.87 times greater

4. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab

5. Poisson Regression
• glmer() supports other non-normal
distributions, such as family=poisson
• Good when the DV is a frequency count
• Number of gestures made in communicative task
• Number of traffic accidents
• Number of ideas brainstormed
• Number of doctor’s visits
• Different from both normal
& binomial distributions
• Lower-bound at 0
• But, no upper bound
• This is a Poisson distribution! -2 0 2 4 6 8 10
0.0
0.1
0.2
0.3
0.4
Count
Probability

6. • Link function for a Poisson distribution is the log()
• Thus, we would also want to use exp()
• Effect of ASD on number of disfluent pauses: 0.27
• exp(0.27) = 1.31
• ASD increases frequency of pauses by 1.31 times
= !0 + !1X1i + ei
Intercept
(Baseline)
ASD
Can be any number
Can be any number
log(yi)
Poisson Regression

7. Poisson Models: Other Variants
• Offset term: Controls for differences in the
opportunities to observe the DV (e.g., time)
• e.g., one child observed for 15 minutes, another
for 14 minutes
• glmer(Disfluencies ~ 1 + ASD + (1|Family),
offset=Time, family=poisson)

8. Poisson Models: Other Variants
• Zero-inflated Poisson: Sometimes, more 0s
than expected under a true Poisson distribution
• Often, when a separate process creates 0s
• # of traffic violations " 0 if you don’t have a car
• # of alcoholic drinks per week " 0 if you don’t drink
• Zero-inflated model:
Simultaneously models
the 0-creating process
as a logit and the rest
as a Poisson
• Use package zeroinfl

9. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab

10. • Remember our cued recall data from last week?
• A second categorical DV here is source
confusions:
• Lots of theoretical interest in source memory—
memory for the context or source where you learned
something
Odds are not the
same thing as
probabilities!
Source Confusion
Odds are not the
same thing as
probabilities!
WHO
SAID
IT?
VIKING—COLLEGE
SCOTCH—VODKA
Study: Test:
VIKING—vodka

11. Source Confusion Model
• Let’s model what causes people to make source
confusions:
• model.Source <- lmer(SourceConfusion ~
Strategy*StudyTime.cen + (1|Subject) +
(1|WordPair), data=sourceconfusion)
• This code contains two mistakes—can we fix them?

12. Source Confusion Model
• Let’s model what causes people to make source
confusions:
• model.Source <- glmer(SourceConfusion ~
Strategy*StudyTime.cen + (1|Subject) +
(1|WordPair), data=sourceconfusion,
family=binomial)
• Failed to converge, with very questionable results
exp(20.08) =
Source errors
525,941,212 times
more likely in
Maintenance condition
… but not significant

13. Low & High Probabilities
• Problem: These are
low frequency events
• In fact, lots of theoretically interesting things have
low frequency
• Clinical diagnoses that are not common
• Various kinds of cognitive errors
• Language production, memory, language
comprehension…
• Learners’ errors in math or other educational
domains

14. Low & High Probabilities
• A problem for our model:
• Model was trying to find the odds of making a
source confusion within each study condition
• But: Source confusions were never observed with
elaborative rehearsal, ever!
• How small are the odds?
They are infinitely small in
this dataset!
• Note that not all failures to converge reflect low
frequency events. But when very low frequencies exist,
they are likely to cause convergence problems.

15. Low & High Probabilities
• Logit is undefined if probability = 1
• Logit is also undefined if probability = 0
• Log 0 is undefined
• e??? = 0
• But there is nothing to which you can raise e to get 0
p(confusion)
1-p(confusion)
[ ]
logit = log
1
0
[ ]
= log
p(confusion)
1-p(confusion)
[ ]
logit = log
0
1
[ ]
= log = log (0)
Division by
zero!!

16. Low & High Probabilities
• When close to 0 or 1, logit is defined but unstable
0.0 0.2 0.4 0.6 0.8 1.0
-4
-2
0
2
4
PROBABILITY of recall
LOG
ODDS
of
recall
p(0.6) -> 0.41
p(0.8) ->1.39
Relatively
gradual change
at moderate
probabilities
p(0.95) -> 2.94
p(0.98) -> 3.89
Fast change at
extreme
probabilities

17. Low & High Probabilities
• A problem for our model:
• Question was how much less common source
confusions become with elaborative rehearsal
• But: Source confusions were never
observed with elaborative rehearsal
• Why we think this happened:
• In theory, elaborative subjects would
probably make at least one of these
errors eventually (given infinite trials)
• Not impossible
• But, empirically, probability was low
enough that we didn’t see the error
in our sample (limited sample size)
Probability = 1%
N = ∞
N = 8

18. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab

19. Empirical Logit
• Empirical logit: An adjustment to the regular
logit to deal with probabilities near (or at) 0 or 1
p(confusion)
1-p(confusion)
[ ]
logit = log

20. Empirical Logit
• Empirical logit: An adjustment to the regular
logit to deal with probabilities near (or at) 0 or 1
• Makes extreme values
(close to 0 or 1) less extreme
Num of “A”s
Num of “B”s
[ ]
logit = log A = Source confusion
occurred
B = Source confusion did
not occur
Num of “A”s + 0.5
Num of “B”s + 0.5
[ ]
emp. logit = log
e

21. Empirical Logit
• Empirical logit: An adjustment to the regular
logit to deal with probabilities near (or at) 0 or 1
• Makes extreme values
(close to 0 or 1) less extreme
Num of “A”s
Num of “B”s
[ ]
logit = log
Num of “A”s + 0.5
Num of “B”s + 0.5
[ ]
emp. logit = log
e
Num of As: 10
Num of Bs: 0
Num of As: 9
Num of Bs: 1
3.04
2.20
1.85
0.41
0.37
Num of As: 6
Num of Bs: 4

22. Empirical logit doesn’t go as
high or as low as the “true” logit
At moderate values, they’re
essentially the same

24. With larger
samples,
difference
gets much
smaller (as
long as
probability
isn’t 0 or 1)

25. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab

26. Empirical Logit: Implementation
• Empirical logit requires summing up events and
then adding 0.5 to numerator & denominator:
• Thus, we have to (1) sum across individual trials,
and then (2) calculate empirical logit
Num A + 0.5
Num B + 0.5
[ ]
empirical logit = log
Single empirical
logit value for
each subject in
each condition
Not a sequence
of one YES or
NO for every
item
Num of As for subject S10 in
Low associative strength,
Maintenance rehearsal
condition

27. Empirical Logit: Implementation
• Empirical logit requires summing up events and
then adding 0.5 to numerator & denominator:
• Thus, we have to (1) sum across individual trials,
and then (2) calculate empirical logit
• Can’t have multiple random effects with empirical
logit
• Would have to do separate by-subjects and by-
items analyses
• Collecting more data can be another solution
Num A + 0.5
Num B + 0.5
[ ]
empirical logit = log
Num of As for subject S10 in
Maintenance rehearsal
condition

28. Empirical Logit: Implementation
• Scott’s psycholing package can help calculate the
empirical logit & run the model
• Example script will be posted on Canvas
• Two notes:
• No longer using glmer() with family=binomial.
We’re now running the model on the empirical logit
value, which isn’t just a 0 or 1.
Here, the
value of
the DV is
-1.61

29. Empirical Logit: Implementation
• Scott’s psycholing package can help calculate the
empirical logit & run the model
• Example script will be posted on Canvas
• Two notes:
• No longer using glmer() with family=binomial.
We’re now running the model on the empirical logit
value, which isn’t just a 0 or 1.
• Because we calculate the empirical logit beforehand,
model doesn’t know how many observations went into
that value
• Want to appropriately
weight the model
-2.46 could be the
average across 10
trials or across 100
trials

30. Week 10.1: Empirical Logit
! Finish Generalized LMERs
! Clarification on Logit Models
! Poisson Regression
! Empirical Logit
! Low & High Probabilities
! Empirical Logit
! Implementation
! Lab