Moments of an AR(1) process

Problem

We have

\[y_t=β_1+ρy_{t-1}+ε_t\]

where \(ε_t\) is white noise, \(Var\left( ε_t \right)=σ^2\) . Suppose that \(\left| ρ \right|<1\) and that \(y\) is a stationary process. Show that

Hint: For two random variables \(X\) and \(Y\) and for constants \(c\) and \(d\) we have

\[cov\left( cX,dY \right)=cdCov\left( X,Y \right)\]

Also,

\[Cov\left( X,X \right)=Var\left( X \right)\]

Solution

Begin with \(y_t=β_1+ρy_{t-1}+ε_t\) and take the expected value of both sides:

\[E\left( y_t \right)=E\left( β_1+ρy_{t-1}+ε_t \right)=β_1+E\left( ρy_{t-1} \right)+E\left( ε_t \right)\]

We have

\[E\left( ρy_{t-1} \right)=ρE\left( y_{t-1} \right)=ρE\left( y_t \right)\]

The last equality follows from stationarity. All \(y_t\) have the same expected value.

We also have \(E\left( ε_t \right)=0\) as it is white noise. Thus,

\[E\left( y_t \right)=β_1+ρE\left( y_t \right)\]

\[E\left( y_t \right)-ρE\left( y_t \right)=β_1\]

\[E\left( y_t \right)\left( 1-ρ \right)=β_1\]

\[E\left( y_t \right)= \frac{β_1}{1-ρ}\]

Now take variance of both sides ( \(ε_t\) is independent of \(y_{t-1}\) ) :

\[Var\left( y_t \right)=Var\left( β_1+ρy_{t-1}+ε_t \right)=Var\left( ρy_{t-1} \right)+Var\left( ε_t \right)\]

We have

\[Var\left( ρy_{t-1} \right)=ρ^2Var\left( y_{t-1} \right)=ρ^2Var\left( y_t \right)\]

The last equality follows from stationarity. All \(y_t\) have the same variance.

We also have \(Var\left( ε_t \right)=σ^2\) . Thus,

\[Var\left( y_t \right)=ρ^2Var\left( y_t \right)+σ^2\]

\[Var\left( y_t \right)-ρ^2Var\left( y_t \right)=σ^2\]

\[Var\left( y_t \right)\left( 1-ρ^2 \right)=σ^2\]

\[Var\left( y_t \right)= \frac{σ^2}{1-ρ^2}\]

For the covariance,

\[Cov\left( y_t,y_{t-1} \right)=Cov\left( β_1+ρy_{t-1}+ε_t,y_{t-1} \right)=Cov\left( ρy_{t-1},y_{t-1} \right)\]

The last equality follows since \(y_{t-1}\) does not covary with a constant \(β_1\) or the error term.

We now have

\[Cov\left( ρy_{t-1},y_{t-1} \right)=ρCov\left( y_{t-1},y_{t-1} \right)= \frac{ρσ^2}{1-ρ^2}\]