Stationary solution: existence and uniqueness

• A stationary solution exists and is unique if and only if ϕ(z)≠0,for all complex z with |z|=1

• The unit circle: the region in z∈C defined by |z|=1

• Stationary solution: Xt=θ(B)/ϕ(B)Zt=ψ(B)Zt=∞∑j=−∞ψjZt−j

Causality: ϕ(z) has no zeros inside the unit circle

• An ARMA(p,q) process is causal: if there exist ψ0,ψ1,… ∞∑j=0|ψj|<∞,and Xt=∞∑j=0ψjZt−j,for all t

• Theorem (equivalent condition of causaility): ϕ(z)=1−ϕ1z−⋯−ϕpzp≠0,for all |z|≤1

• Example: ARMA(1,1) Xt−ϕXt−1=Zt+θZt−1 1−ϕz=0⟹ only zero z=1/ϕ So |z|=1/|ϕ|>1, i.e., |ϕ|<1 is equivalent of causality

How do we get ψj’s?

• Letting θ0=1 and matching coefficients of zj based on 1+θ1z+⋯θqzq=(1−ϕ1z−⋯ϕpzp)(ψ0+ψ1z+⋯), gives θj1[j≤q]=ψj−p∑j=1ϕkψj−k,j=0,1,…

• Example: causal ARMA(1,1)
  1=ψ0θ=ψ1−ϕψ0⟹ψ1=θ+ψ0=ψj−ϕψj−1 for j≥2⟹ψj=ϕψj−1

Therefore, ψ0=1,ψj=ϕj−1(θ+ψ) for j≥1

Invertibility: θ(z) has no zeros inside the unit circle

• An ARMA(p,q) process is invertible: if there exist π0,π1,… ∞∑j=0|πj|<∞,and Zt=∞∑j=0πjXt−j,for all t

• Theorem (equivalent condition of invertibility): θ(z)=1+θ1z+⋯+θqzq≠0,for all |z|≤1

• Example: ARMA(1,1) Xt−ϕXt−1=Zt+θZt−1 1+θz=0⟹ only zero z=−1/θ So |z|=1/|θ|>1, i.e., |θ|<1 is equivalent of invertibility

How do we get πj’s?

• Letting ϕ0=−1 and matching coefficients of zj based on 1−ϕ1z−⋯ϕpzp=(1+θ1z+⋯θqzq)(π0+π1z+⋯), gives −ϕj1[j≤p]=πj+q∑j=1θkπj−k,j=0,1,…

• Example: invertible ARMA(1,1)
  1=π0−ϕ=π1+θψ0⟹π1=−(ψ+θ)0=πj+θπj−1 for j≥2⟹πj=−θπj−1

Therefore, π0=1,πj=(−1)jθj−1(ψ+θ) for j≥1

Calculation of the ACVF

• Assume the ARMA(p,q) process {Xt} is causal and invertible

• Method 1: If Xt=∑∞j=0ψjZt−j, then γ(h)=E(Xt+hEt)=σ2∞∑j=0ψjψj+|h|

• Method 2 (difference equation method): multiple the ARMA formula with Xt,Xt−1,… and take expectation

Example: ARMA(1,1)

• Recall that for a causal ARMA(1,1), in Xt=∑∞j=0ψjZt−j, ψ0=1,ψj=ϕj−1(θ+ψ) for j≥1

• Lag-0 autocorrelation γ(0)=σ2∞∑j=0ψ2j=σ2[1+(θ+ϕ)2∞∑j=0ϕ2j]=σ2[1+(θ+ϕ)21−ϕ2]

• Lag-1 autocorrelation γ(1)=σ2∞∑j=0ψjψj+1=σ2[θ+ϕ+(θ+ϕ)2ϕ1−ϕ2]

• Lag-k autocorrelation (k≥2) γ(k)=ϕk−1γ(1),k≥2

Use the difference equation method on ARMA(1,1)

1. Multiple Xt−ϕXt−1=Zt+θZt−1 by Xt, then take expectation E(X2t)−ϕE(XtXt−1)=E(XtZt)+θE(XtZt−1) Since E(XtZk)=E[(∑∞j=0ψjZt−j)Zk]=ψt−kσ2, we have γ(0)−ϕγ(1)=σ2+θ(θ+ϕ)σ2

2. Multiply by Xt−1 E(Xt−1Xt)−ϕE(X2t−1)=E(Xt−1Zt)+θE(Xt−1Zt−1) γ(1)−ϕγ(0)=0+θσ2ψ0=θσ2

Using the two equations from 1 and 2, we can solve γ(0),γ(1)

3. Multiply by Xt−k, for k≥2 E(Xt−kXt)−ϕE(Xt−kXt−1)=E(Xt−kZt)+θE(Xt−kZt−1) γ(k)−ϕγ(k−1)=0⟹γ(k)=ϕγ(k−1)

ACF of an MA(q) process

• Suppose {Xt} is an MA(q), then ρ(h)=0 for all h>q
• By asymptotic normality ˆρ(q+1)⋅∼N(0,wq+1,q+1n) and Bartlett wq+1,q+1=∞∑k=1[ρ(k+q+1)+ρ(k−q−1)−2ρ(k+1)ρ(q)]2=∞∑k=1ρ(k−q−1)2=1+2q∑j=1ρ(j)2

Test for an MA(q): from the ACF

1. Hypotheses H0:{Xt}∼MA(q)⟷HA:Not H0

2. Test statistic Z=ˆρ(q+1)−0√1+2∑qj=1ˆρ(j)2n

3. Reject H0 if |Z|≥zα/2

• Note: under the null hypothesis, we use the sample ACF plot with bounds ±1.96×√1+2∑qj=1ˆρ(j)2n to check if ˆρ(h) for all h≥q+1 are inside the bounds. But this may have some multiple testing problems.

Partial autocorrelation function (PACF)

• We define the partial autocorrelation function (PACF) of an ARMA process as the function α(⋅) α(0)=1,α(h)=ϕhh, for h≥1 Here, ϕhh is the last entry of ϕh=Γ−1hγh,where Γh=[γ(i−j)]hi,j=1, γh=[γ(1),…,γ(h)]′

• Sample PACF ˆα(⋅): change all γ(⋅) above to ˆγ(⋅)

• Recall: in DL algorithm ˆXn+1=ϕn1Xn+⋯+ϕnnX1, ϕnn=α(n),PACF at lag n

PACF property

• ϕnn is the correlation between the prediction errors α(n)=Corr(Xn−P(Xn|X1,…,Xn−1),X0−P(X0|X1,…,Xn−1))

• Theorem: A stationary series is AR(p) if and only if α(h)=0 for all h>p

• If {Xt} is an AR(p), then we have asymptotic normality ˆα(h)⋅∼N(0,1n),for all h>p

Test for an AR(p): from the PACF

1. Hypotheses H0:{Xt}∼AR(p)⟷HA:Not H0

2. Test statistic Z=ˆα(p+1)−0√1n

3. Reject H0 if |Z|≥zα/2

• Note: under the null hypothesis, we use the sample PACF plot with bounds ±1.96/√n to check if ˆα(h) for all h≥p+1 are inside the bounds. But this may have some multiple testing problems.