Presentation for reading session of Computer Vision: Models, Learning, and Inference

1. CHAPTER 3:
COMMON PROBABILITY
DISTRIBUTIONS
COMPUTER VISION: MODELS, LEARNING AND
INFERENCE
Lukas Tencer

2. 2
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

3. Why model these complicated
quantities?
3
Because we need probability distributions over model parameters as
well as over data and world state. Hence, some of the distributions
describe the parameters of the others:
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

4. Why model these complicated
quantities?
4
Because we need probability distributions over model parameters as
well as over data and world state. Hence, some of the distributions
describe the parameters of the others:
Example:
Parameters modelled by:
Models variance
Models mean
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

5. Bernoulli Distribution
5
or
For short we write:
Bernoulli distribution describes situation where only two
possible outcomes y=0/y=1 or failure/success
Takes a single parameter
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

6. Beta Distribution
6
Defined over data (i.e. parameter of Bernoulli)
• Two parameters both > 0 For short we write:
• Mean depends on relative values E[ ] = .
• Concentration depends on magnitude
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

7. Categorical Distribution
7
or can think of data as vector with all
elements zero except kth e.g. e4 =
[0,0,0,1,0]
For short we write:
Categorical distribution describes situation where K possible
outcomes y=1… y=k.
Takes K parameters where
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

8. Dirichlet Distribution
8
Defined over K values where
Or for short: Has k
parameters k>0
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

9. Univariate Normal Distribution
9
For short we write:
Univariate normal distribution
describes single continuous
variable.
Takes 2 parameters and
Computer vision: models, learning2and inference. ©2011
>0Simon J.D. Prince

10. Normal Inverse Gamma
10
Distribution
Defined on 2 variables and 2>0
or for short
Four parameters and
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

11. Multivariate Normal Distribution
11
For short we write:
Multivariate normal distribution describes multiple continuous
variables. Takes 2 parameters
• a vector containing mean position,
• a symmetric “positive definite” covariance matrix
Positive definite: is positive for any real
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

12. Types of covariance
12
Covariance matrix has three forms, termed spherical, diagonal and full
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

13. Normal Inverse Wishart
13
Defined on two variables: a mean vector and a symmetric positive definite
matrix, .
or for short:
Has four parameters
• a positive scalar,
• a positive definite matrix
• a positive scalar,
• a vector
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

14. Samples from
Normal Inverse
14
Wishart
(dispersion) (ave. Covar) (disper of means) (ave. of means)
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

15. Conjugate Distributions
15
The pairs of distributions discussed have a special
relationship: they are conjugate distributions
 Beta is conjugate to Bernouilli
 Dirichlet is conjugate to categorical
 Normal inverse gamma is conjugate to univariate
normal
 Normal inverse Wishart is conjugate to
multivariate normal
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

16. Conjugate Distributions
16
When we take product of distribution and it’s conjugate,
the result has the same form as the conjugate.
For example, consider the case where
then
a constant A new Beta distribution
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

17. Example proof
17
When we take product of distribution and it’s conjugate,
the result has the same form as the conjugate.
Computer vision: models, learning and inference. ©2011 Simon J.D.
17
Prince

18. Bayes’ Rule Terminology
18
Likelihood – propensity Prior – what we know
for observing a certain about y before seeing
value of x given a certain x
value of y
Posterior – what we Evidence – a constant to
know about y after ensure that the left hand
seeing x side is a valid distribution
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

19. Importance of the Conjugate
19
Relation 1
1. Choose prior
 Learning parameters: that is conjugate
to likelihood
2. Implies that posterior 3. Posterior must be a distribution
must have same form as which implies that evidence must
conjugate prior equal constant from conjugate
distribution relation
Computer vision: models, learning and inference. ©2011
Simon J.D. Prince

20. Importance of the Conjugate
20
Relation 2
 Marginalizing over parameters
2. Integral becomes easy --the product 1. Chosen so
becomes a constant times a distribution conjugate to othe
term
Integral of constant times probability
distribution
= constant times integral of probability
distribution = constant vision: models, learning and inference.
Computer x 1 = constant ©2011
Simon J.D. Prince

21. Conclusions
21
• Presented four distributions which model useful
quantities
• Presented four other distributions which model
the parameters of the first four
• They are paired in a special way – the second set
is conjugate to the other
• In the following material we’ll see that this
relationship is verymodels, learning and inference. ©2011
Computer vision:
useful
Simon J.D. Prince

22. 22
Thank You
for you attention
Based on:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
http://www.computervisionmodels.com/