Covariance, correlation and independence (intro)
Summary
Covariance
- If \(X\) and \(Y\) are two random variables then the covariance between them, denoted by
\[Cov(X,Y)\]
- measures the degree to which they are linearly associated.
- A formal definition of covariance
Correlation
- If \(X\) and \(Y\) are two random variables then correlation between \(X\) and \(Y\) is defined as
\[ρ_{X,Y}= \frac{Cov(X,Y)}{σ_Xσ_Y}\]
- The correlation is a standardized version of the covariance ( \(-1≤ρ_{X,Y}≤1\) ) but they always have the same sign.
- If \(Cov\left( X,Y \right)=0\) we say that \(X,Y\) are uncorrelated .
- Example: Toss two dice and let \(X_1\) be the result of the first die and \(X_2\) be the result of the second die. Define \(Y=X_1+X_2\) (the sum) and \(Z=X_1-X_2\) (the difference) . Then
- \(Cov\left( Y,X_2 \right)>0\) , the sum is positively correlated with the outcome of the second die
- \(Cov\left( Z,X_2 \right)<0\) , the difference is negatively correlated with the outcome of the second die
- \(Cov\left( X_1,X_2 \right)=0\) , the two dice are uncorrelated
About the covariance
- Do not confuse the covariance (which is a property of two random variables) with the sample covariance (which is computed from a sample).
Independent random variables
- \(X\) and \(Y\) are independent random variables if they are not at all associated (linearly or nonlinearly).
- Formal definition of independent random variables
- If \(X,Y\) are independent then they are uncorrelated. The opposite is not true. If \(Cov\left( X,Y \right)=0\) then \(X,Y\) may still have a nonlinear association.
- Independence can be extended to several random variables.
- A random variable \(X\) is always independent of and uncorrelated with a constant.