1. Stat310 Maximum likelihood
Hadley Wickham
Sunday, 11 April 2010
2. 1. Assessment
2. Feedback
3. Joint pdf
4. Maximum likelihood
Sunday, 11 April 2010
3. Assessment
• All grading now 100% up to date
(as far as I know)
• Overall grade to date in owlspace
(but doesn’t account for dropping
lowest homework)
• Quizzes were going to be worth 10%,
change to 5%?
Sunday, 11 April 2010
4. So far
• 2 / 2 tests * 10% = 20%
• 7 / 10 homeworks * 40% = 28%
• 3 / 5 quizzes * 5% = 3%
• Total: 51% of grade
Sunday, 11 April 2010
5. To come
• 1 final * 30% = 30%
• 3 / 10 homeworks * 40% = 12%
• 2 / 5 quizzes * 5% = 2%
• 5% TBA
• Total: 49% of grade
Sunday, 11 April 2010
6. Test
• Bad news: It was harder
• Good news: I’ve figured out why, so it
won’t happen on the final
Sunday, 11 April 2010
10. These are
minimums
15
described in
the syllabus
10
count
5
F C B A
0
10 20 30 40 50
Overall
Sunday, 11 April 2010
11. Options
• Do nothing.
• Add 3 points on to test. Distribute 5%
evenly across all assessment.
• 1 hour take home exam worth 5%.
2-3 problems from the book.
• 1 extra homework worth 5%.
4-5 problems from the book.
Sunday, 11 April 2010
12. Homeworks
• Due Thursday in class
• Out of the goodness of my heart I have
been accepting late homeworks
• But it is getting excessive - I shouldn’t
have to deal with 15 late homeworks a
week
• Please turn in on time or I will start
enforcing the late homework penalty.
Sunday, 11 April 2010
14. Feedback about me
Doing well: Lectures/teaching (13), engaging/
interesting lectures (11), website (10),
examples (10), homeworks (8), help sessions
(6), pace (4), funny (3), being awesome (2)
Needs improvement: test too hard (too many
to count), hard to study from ppt (7), more
activities (5), less mistakes (5), too fast (4),
homework session should be a tutorial (3)
Sunday, 11 April 2010
15. Changes
My notes are scattered between slides, the
board and my voice. Your notes should
not be!
Will continue to try and find interesting
examples and activities.
For final review session, will have voting
system and I’ll re-cover popular topics on
the board.
Sunday, 11 April 2010
16. You
Doing well
Needs
improvement
Sunday, 11 April 2010
17. You
Marijuana?
Doing well
Needs
improvement
Sunday, 11 April 2010
18. You
Doing well
Probably read
Needs ahead, but
improvement who does that
anyways
Sunday, 11 April 2010
19. You
I’m enjoying
Doing well the weather
Needs
improvement
Sunday, 11 April 2010
20. You
Doing well
Needs
my grade
improvement
Sunday, 11 April 2010
21. Why do we care
about random
variables?
Sunday, 11 April 2010
22. Experiments
If we capture all the relevant information
about an experiment, we can repeat
virtually (either mathematically or
computationally). This is usually easier
and cheaper than doing the real
experiment!
The mathematical abstraction we use to
do this is the random variable.
Sunday, 11 April 2010
23. So
The purpose of a random variable is to
describe (or at least approximate) the
behaviour of an experiment. So:
X ~ SomeDist(some params)
means we have a single experiment
whose behaviour is defined.
Sunday, 11 April 2010
24. Replications
X1 ~ SomeDist(some params)
X2 ~ SomeDist(some params)
Means we repeat the experiment twice - it’s the
same distribution, which implies that the
experiment is repeated under identical conditions.
f(x1, x2) is the bivariate pdf which allows us to
figure out the probability of any event involving
the two replicates
Sunday, 11 April 2010
25. Replicates
Xi ~ SomeDist(some params)
i = 1, 2, ..., n
Means we repeat the experiment n times.
f(x1, x2, ..., xn) is the joint pdf which allows
us to figure out the probability of any
event involving the n replicates
Sunday, 11 April 2010
27. Your turn
On Tuesday I was dismayed to find that if
Xi ~ Binomial(n, p) then an estimator for p
n
is i Xi /n 2
In fact, this estimator is basically correct,
but there is a problem with my notation.
Can you spot where I went wrong?
(everything you need is on this slide)
Sunday, 11 April 2010
28. Formal definition
The maximum likelihood estimator is a
value of the parameter that maximises the
likelihood function with respect to the
parameter.
ˆM L = max l(θ; x1 , x2 , . . . , xn )
θ
θ∈Θ
Sunday, 11 April 2010
29. Steps
Write out likelihood (=joint pdf)
Write out log-likelihood
(Discard constants)
Find maximum:
Differentiate and set to 0
(Check second derivative is negatice)
(Check end points)
Sunday, 11 April 2010
30. Maximum
• Derivative zero
• Derivative undefined
• At boundary points
Sunday, 11 April 2010
31. Your turn
Xi ~ Poisson(λ) i = 1,..., n
Use maximum likelihood to find an
estimator for λ
Sunday, 11 April 2010
32. Invariance principle
One neat property of maximum likelihood
estimators is invariance
Sunday, 11 April 2010
33. What else?
MLEs are:
Unbiased
Minimum variance
Have asymptotically normal distribution!
ˆM L ) = −1
V ar(θ δ2
E δθ2 l(X|θ)
Sunday, 11 April 2010
34. But
That math is too hard for this course :(
So we need some other ways to work out
how much error our estimators have.
Sunday, 11 April 2010
35. Your turn
What is the variance of ˆM L ?
λ
Sunday, 11 April 2010
36. Your turn
I repeated an experiment defined by
Poisson(λ) 10 times, and recorded the
following results:
6 11 10 6 12 7 8 5 7 10
What is the MLE of λ?
What is the standard deviation of our
estimate?
Sunday, 11 April 2010
37. Answer
Mean = 8.2
SD = 0.90
Can you create an interval around the
estimate that ensures that the true value
will be inside it 95% of the time?
(Use clt)
Sunday, 11 April 2010
