it helps to understand the topic

1. Covariance & Correlation
Jerrell T.Stracener – Ph.D.
1

2. Covariance – Cov(X,Y)
Covariance between X and Y is a measure
of the association between two random
variables, X & Y
If positive, then both move up or down
together
If negative, then if X is high, Y is low, vice
versa
[
σ XY = Cov( X , Y ) = E ( X − µ X )( Y − µY )
]

3. Correlation Between X and Y
Covariance is dependent upon the units of
X & Y [Cov(aX,bY)=abCov(X,Y)]
Correlation, Corr(X,Y), scales covariance
by the standard deviations of X & Y so that
it lies between 1 & –1
ρ XY
σ XY
Cov ( X , Y )
=
=
1
σ X σ Y [Var ( X )Var (Y )] 2

4. More Correlation & Covariance
If σX,Y =0 (or equivalently ρX,Y =0) then X
and Y are linearly unrelated
If ρX,Y = 1 then X and Y are said to be
perfectly positively correlated
If ρX,Y = – 1 then X and Y are said to be
perfectly negatively correlated
Corr(aX,bY) = Corr(X,Y) if ab>0
Corr(aX,bY) = –Corr(X,Y) if ab<0

5. Properties of Expectations
E(a)=a, Var(a)=0
E(µX)=µX, i.e. E(E(X))=E(X)
E(aX+b)=aE(X)+b
E(X+Y)=E(X)+E(Y)
E(X-Y)=E(X)-E(Y)
E(X- µX)=0 or E(X-E(X))=0
E((aX)2)=a2E(X2)

6. More Properties
Var(X) = E(X2) – µx2
Var(aX+b) = a2Var(X)
Var(X+Y) = Var(X) +Var(Y) +2Cov(X,Y)
Var(X-Y) = Var(X) +Var(Y) - 2Cov(X,Y)
Cov(X,Y) = E(XY)-µxµy
If (and only if) X,Y independent, then

Var(X+Y)=Var(X)+Var(Y), E(XY)=E(X)E(Y)

7. Covariance of X and Y
Let X and Y be random variables with joint mass function
p(x,y) if X & Y are discrete random variables or with joint
probability density function f(x, y) if X & Y are continuous
random variables. The covariance of X and Y is
σ XY = E [ ( X − µ X )( Y − µ Y ) ] = ∑ ∑ ( x − µ x ) ( y − µ y ) p( x, y )
x
y
if X and Y are discrete, and
σ XY = E [ ( X − µ X )( Y − µ Y ) ] =
∞ ∞
∫ ∫ ( x − µ ) ( y − µ ) f ( x, y ) dxdy
x
− ∞− ∞
if X and Y are continuous.
Jerrell T.Stracener – Ph.D.
7
y

8. Covariance of X and Y
The covariance of two random variables X and Y with means
µX and µY , respectively is given by
σ XY = E ( XY ) − µ X µ Y
Jerrell T.Stracener – Ph.D.
8

9. Correlation Coefficient
Let X and Y be random variables with covariance σXY and
standard deviation σX and σY , respectively. The correlation
coefficient of X and Y is
ρ XY
σ XY
=
σ Xσ Y
Jerrell T.Stracener – Ph.D.
9

10. Theorem
If X and Y are random variables with joint probability
distribution f(x, y), then
σ
2
aX + bY
= a σ + b σ + 2abσ XY
2
2
X
Jerrell T.Stracener – Ph.D.
10
2
2
Y

11. Theorem
If X and Y are independent random variables, then
σ
2
aX + bY
= a σ +b σ
2
2
X
Jerrell T.Stracener – Ph.D.
11
2
2
Y

12. Correlation Analysis
A statistical analysis used to obtain a quantitative measure of
the strength of the linear relationship between a dependent
variable and one or more independent variables
Jerrell T.Stracener – Ph.D.
12

13. Correlation – Scatter Diagram
Visual Relationship Between X and Y
Jerrell T.Stracener – Ph.D.
13