Covariance

Summary

Definition

• \(X\) and \(Y\) are two random variables with \(μ_X=E(X)\) and \(μ_Y=E(Y)\) .
• Define \(Z=(X-μ_X)(Y-μ_Y)\) . We define the covariance of \(X\) and \(Y\) as \(E(Z)\) . Combined,

\[Cov\left( X,Y \right)=E\left( Z \right)=E\left[ \left( X-μ_X \right)\left( Y-μ_Y \right) \right]=E\left( XY \right)-μ_Xμ_Y\]

Discrete random variables

• If \(X,Y\) are discrete random variables with range \({x_1,…,x_n}\) and \({y_1,…,y_m}\) respectively and joint pmf \(f\left( x,y \right)\) then

\[Cov(X,Y)=\sum_{i=1}^{n}{ \sum_{j=1}^{m}{ \left( x_i-μ_X \right)\left( y_j-μ_Y \right)f\left( x_i,y_j \right) } }\]

Example

• \(f(x,y)\) is defined by the following table:
• \(μ_X=0.5, μ_Y=0.5\) and

\[Cov\left( X,Y \right)=\left( -0.5 \right)⋅\left( -0.5 \right)⋅0.4+\left( -0.5 \right)⋅0.5⋅0.1+0.5⋅\left( -0.5 \right)⋅0.1+0.5⋅0.5⋅0.4=0.15\]

• Alternatively, \(E\left( XY \right)=0.4\) and \(Cov\left( X,Y \right)=0.4-{0.5}^2=0.15\) .

Continuous random variable

• If \(X,Y\) are two continuous random variables with joint pdf \(f\left( x,y \right)\) t hen

\[Cov(X,Y)=\int_{-∞}^{∞}{ \int_{-∞}^{∞}{ \left( x-μ_X \right)\left( y-μ_Y \right)f\left( x,y \right)dxdy } }\]

Example

• \(f\left( x,y \right)=x+y\) for \(0≤x≤ 1\) and \(0≤y≤ 1\) . \(μ_X=μ_Y=7/12\) and

\[Cov\left( X,Y \right)=\int_{0}^{1}{ \int_{0}^{1}{ \left( x- \frac{7}{12} \right)\left( y- \frac{7}{12} \right)\left( x+y \right) dxdy } }=- \frac{1}{144}\]

• Alternatively,

\[E\left( XY \right)=\int_{0}^{1}{ \int_{0}^{1}{ xy\left( x+y \right) dxdy } }= \frac{1}{3}\]

• and \(Cov\left( X,Y \right)=1/3-{(7/12)}^2=-1/144\)