𝑗=0
(10.2.18)
Therefore, the variance is finite and independent of time. Correspondingly, the time independent autocovariances with 𝑘 > 0 can be calculated by using the result that 𝐸(𝑎𝑡−𝑖𝑎𝑡−𝑗) = 0 𝑓𝑜𝑟 𝑖 ≠ 𝑗 as indicated below (Mills, p. 68):
𝛾𝑘 = 𝐸(𝑧𝑡− 𝜇)(𝑧𝑡−𝑘− 𝜇) (10.2.19)
= 𝐸(𝑎𝑡+ 𝜓1𝑎𝑡−1+ ⋯ + 𝜓𝑘𝑎𝑡−𝑘+ 𝜓𝑘+1𝑎𝑡−𝑘−1+ ⋯ ) ∙ (𝑎𝑡−𝑘+ 𝜓1𝑎𝑡−𝑘−1 + ⋯ )
(10.2.20)
= 𝜎𝑎2(1 ⋅ 𝜓𝑘+ 𝜓1𝜓𝑘+1+ 𝜓2𝜓𝑘+2+ ⋯ ) (10.2.21)
= 𝜎𝑎2∑ 𝜓𝑗𝜓𝑗+𝑘
∞
𝑗=0
(10.2.22)
It can be inferred that the autocovariances are only functions of the time difference (i.e. the distance between two random variables) through fulfilling all conditions of covariance stationarity. The autocorrelation function is accordingly expressed below (Mills, p. 68):
𝜌𝑘 =∑∞𝑗=0𝜓𝑗𝜓𝑗+𝑘
∑∞𝑗=0𝜓𝑗2 (10.2.23)
10.2.3 The General Autoregressive Models
A general autoregressive model can be utilized for describing a univariate time series by regressing the current deviation 𝑧̃𝑡 on past deviations (𝑧̃𝑡−1, 𝑧̃𝑡−2, …,) of the corresponding stochastic process. Correspondingly, the representation of 𝑧̃𝑡 as a weighted sum of past values plus an added shock is given below (Box & Jenkins, p. 47):
𝑧̃𝑡 = 𝜋1𝑧̃𝑡−1+ 𝜋2𝑧̃𝑡−2+ ⋯ + 𝑎𝑡 = ∑ 𝜋𝑗𝑧̃𝑡−𝑗
∞
𝑗=1
+ 𝑎𝑡 (10.2.24)
119 The coefficients 𝜋𝑗 in the linear filter are called 𝜋 (pi)-weights whose number is infinite and relates 𝑧̃𝑡 to past deviations together with 𝑎𝑡. The corresponding operator, 𝜋(𝐵), which is utilized in establishing the mentioned relations is indicated below:
𝜋(𝐵) = 1 − 𝜋1𝐵 − 𝜋2𝐵2− ⋯ (10.2.25)
Thus, the Eq. (10.2.24) can be written in compact form as follows (Box & Jenkins, p. 51):
𝜋(𝐵)𝑧̃𝑡= 𝑎𝑡 (10.2.26)
The 𝜋-weights can be derived from the linear filter model according to the relationship 𝜋(𝐵) = 𝜓−1(𝐵) and by using the known values of the 𝜓-weights (Box & Jenkins, p. 48). The relationship between both weights can be derived, after multiplying both sides of the above Eq. (10.2.26) by 𝜓(𝐵) and cancelling 𝑧̃𝑡's on both sides, which results in Eq. (10.2.27) as shown below (Box & Jenkins, p. 48):
𝜓(𝐵)𝑎𝑡 = 𝑧̃𝑡 (10.2.27)
A stochastic process can be considered as a general autoregressive model of order p (i.e.
abbreviated as 𝐴𝑅(𝑝)), if only a finite number of 𝜋 weights are non-zero, i.e. 𝜋1 = 𝜙1, 𝜋2 = 𝜙2, … , 𝜋𝑝 = 𝜙𝑝 and 𝜋𝑘 = 0 for 𝑘 > 𝑝 as expressed below (Box & Jenkins, p. 51):
𝑧̃𝑡 = 𝜙1𝑧̃𝑡−1+ 𝜙2𝑧̃𝑡−2+ ⋯ + 𝜙𝑝𝑧̃𝑡−𝑝+ 𝑎𝑡 (10.2.28) The autoregressive operator of order 𝑝 can be subsequently expressed as follows (Box &
Jenkins, p. 51):
𝜙(𝐵) = 1 − 𝜙1𝐵 − 𝜙2𝐵2… − 𝜙𝑝𝐵𝑝 (10.2.29) A 𝑝𝑡ℎ order autoregressive model can be indicated in parsimonious form as follows:
𝜙(𝐵)𝑧̃𝑡 = 𝑎𝑡 (10.2.30)
The corresponding 𝐴𝑅(𝑝) characteristic equation can be expressed as indicated below (Cryer
& Chan, 2008, p. 76):
1 − 𝜙1𝐵 − 𝜙2𝐵2− 𝜙𝑝𝐵𝑝= 0 (10.2.31)
120 The stationarity of an 𝐴𝑅(𝑝) model, i.e. the convergence of the 𝜓-weights, is ensured by the requirement of the roots of the characteristic equation to lie outside of the unit circle94 (Box &
Jenkins, p. 54). As indicated below in Eq. (10.2.32), polynomial factorization of the characteristic equation is needed to be carried out for finding the roots of the characteristic polynomial. It is such that |𝐺𝑖| < 1 for 𝑖 = 1,2, … 𝑝, or the roots 𝐺𝑖−1 all lie outside the unit circle (Box & Jenkins, p. 54):
𝜙(𝐵) = (1 − 𝐺1𝐵)(1 − 𝐺2𝐵) … (1 − 𝐺𝑝𝐵) = 0 (10.2.32) An autoregressive model is considered to be always invertible (i.e. 𝜋(𝐵) = 𝜓−1(𝐵) is always valid), since ∑∞j=1|πj|= ∑pj=1|ϕj|< ∞ (Wei, 2006, p. 33). In addition, the model contains p+2 unknown parameters 𝜇, 𝜙1, 𝜙2, … , 𝜙𝑝, 𝜎𝑎2, which can be estimated from the observations (Box
& Jenkins, p. 9).
10.2.3.1 Autocorrelation function of 𝑨𝑹(𝒑) processes
The autocovariance function of a general AR process, indicated in Eq. (10.2.33), can be obtained by multiplying Eq. (10.2.28) by 𝑧̃𝑡, and then taking the expectations (Wei, p. 45).
Note that 𝐸(𝑎𝑡𝑧̃𝑡) = 𝐸(𝑎𝑡2) = 𝜎𝑎2.
𝛾𝑘 = 𝜙1𝛾𝑘−1+ 𝜙2𝛾𝑘−2+ ⋯ + 𝜙𝑝𝛾𝑘−𝑝, 𝑘 > 0 (10.2.33) Hence the autocorrelation function (ACF), represented in Eq. (10.2.34) as 𝜌𝑘, can be found through dividing the Eq. (10.2.33) by 𝛾0 (Wei, p. 46).
𝜌𝑘 = 𝜙1𝜌𝑘−1+ 𝜙2𝜌𝑘−2+ ⋯ + 𝜙𝑝𝜌𝑘−𝑝, 𝑘 > 0 (10.2.34)
The Eq. (10.2.34) can be written in parsimonious form as follows (Box & Jenkins, p. 55):
𝜙(𝐵)𝜌𝑘 = 0 (10.2.35)
In Eq. (10.2.35), B now backshifts 𝑘 and not 𝑡. Further, the polynomial factorization of the equation 𝜙(𝐵) can be written as given below (Box & Jenkins, p. 55):
𝜙(𝐵) = ∏(1 − 𝐺𝑖𝐵)
𝑝
𝑖=1
(10.2.36)
94 Unit circle is a circle with radius equaling to 1.
121 The general solution of the Eq. (10.2.34) is then
𝜌𝑘 = 𝐴1𝐺1𝑘+ 𝐴2𝐺2𝑘+ ⋯ + 𝐴𝑝𝐺𝑝𝑘 (10.2.37) where 𝐺1−1, 𝐺2−1, … , 𝐺𝑝−1 are the roots of the characteristic equation (Box & Jenkins, p. 55).
For a stationary 𝐴𝑅(𝑝) process (i.e. |𝐺𝑖| < 1), there can arise two situations, if it is assumed that the roots 𝐺𝑖 are distinct (Box & Jenkins, p. 55):
1) If a root 𝐺𝑖 is real, the ACF of the general 𝐴𝑅(𝑝) geometrically decays (i.e. called damped exponential) to zero as 𝑘 increases.
2) If a pair of roots 𝐺𝑖, 𝐺𝑗, is complex, the ACF follows a damped sine wave.
The general Yule-Walker equations, indicates the recursive relationship for 𝜌𝑘, can be obtained by setting 𝑘 = 1,2, … , 𝑝 in Eq. (10.2.34) to form a set of equations as follows (Box
& Jenkins, p. 55):
𝜌1 = 𝜙1 +𝜙2𝜌1 + ⋯ + 𝜙𝑝𝜌𝑝−1 𝜌2 = 𝜙1𝜌1 +𝜙2 + ⋯ + 𝜙𝑝𝜌𝑝−2 ⋮ ⋮ ⋮ … ⋮
𝜌𝑝 = 𝜙1𝜌𝑝−1 +𝜙2𝜌𝑝−2 + ⋯ + 𝜙𝑝
(10.2.38)
It should be noted that 𝜌0 = 1 and 𝜌𝑘 = 𝜌−𝑘. Once the values for 𝜙1, 𝜙2, … , 𝜙𝑝 are found, the set of linear equations can be solved recursively to obtain numerical values for 𝜌𝑘 at any number of higher lags (Box & Jenkins, p. 55). The so called “Yule-Walker estimates” of the parameters can be calculated by replacing 𝜌𝑘 with 𝜌̂𝑘.
The sample autocorrelations should be tested for statistical significance. Therefore, it is investigated whether 𝜌̂𝑘’s lie outside the confidence interval ±1.96/√𝑛 at a 5% level of significance (i.e. 𝐻0: 𝜌 = 0). Note that to test for uncorrelated observations (𝜌𝑘 = 0 ∀𝑘: 𝑘 ≠ 0), the corresponding standard error of 𝜌̂𝑘 can be approximated with 1/√𝑛. Further, the large lag standard error “𝑆𝑟𝑘” is used for approximating standard error of 𝜌̂𝑘 for large lags 𝑘 > 𝑞, i.e. for 𝜌𝑘’s which are tested to be zero beyond 𝑘 = 𝑞. 𝑆𝜌̂𝑘 can be calculated as follows (Box
& Jenkins, p. 35):
𝑆𝜌̂𝑘 ≃ √1
𝑛(1 + 2𝜌12+ ⋯ + 2𝜌𝑘−12 ) 𝑘 > 𝑞 (10.2.39)
122 10.2.3.2 The partial autocorrelation function of general 𝑨𝑹(𝒑) Processes
The partial autocorrelation function (PACF) provides the means for distinguishing between different orders of 𝐴𝑅(𝑝) models, since ACFs of all 𝐴𝑅(𝑝) models damp out. The PACF measures the correlation between 𝑧̃𝑡 and 𝑧̃𝑡−𝑘 after removing the effect of intermediate variables 𝑧̃𝑡−1, 𝑧̃𝑡−2, … , 𝑧̃𝑡−𝑘−1. In general, the correlation between two random variables is often due to both variables being correlated with a third variable. Namely, 𝑧̃𝑡−2 involves {𝑎𝑡−2, 𝑎𝑡−3, … } which are all uncorrelated with 𝑎𝑡, 𝑎𝑡−1. Since 𝑧̃𝑡 is dependent on 𝑧̃𝑡−2 through 𝑧̃𝑡−1, a correlation between 𝑧̃𝑡 and 𝑧̃𝑡−2 exists. The PACF indicates the correlation between 𝑧̃𝑡 and 𝑧̃𝑡−2 with the removed linear dependence of 𝑧̃𝑡−1. The lag 𝑘 partial autocorrelation is the last coefficient 𝜙𝑘,𝑘 in the 𝑘𝑡ℎ order autoregression as shown below (Mills, p. 78):
𝑧̃𝑡 = 𝜙𝑘,1𝑧̃𝑡−1+ 𝜙𝑘,2𝑧̃𝑡−2+ ⋯ + 𝜙𝑘,𝑘𝑧̃𝑡−𝑘+ 𝑎𝑡 (10.2.40) The term, 𝜙𝑘,𝑗, denotes the 𝑗𝑡ℎ coefficient in an autoregressive process of order 𝑘 (Mills, p.
78).
The partial autocorrelation can be estimated using least squares method for fitting successive autoregressive processes of orders 1,2,3, … and choosing the estimates 𝜙̂1,1, 𝜙̂2,2, 𝜙̂3,3, … which are the last coefficient fitted at each stage (Box & Jenkins, p. 65). Alternatively, a recursive method, by Durbin (1960), can also be utilized to find the estimates of the partial autocorrelations and the estimates of coefficients 𝜙̂𝑘,𝑗 (Mills, p. 80):
𝜙̂𝑘,𝑘= 𝜌̂𝑘− ∑𝑘−1𝑗=1𝜙̂𝑘−1,𝑗 𝜌̂𝑘−𝑗
1 − ∑𝑘−1𝑗=1𝜙̂𝑘−1,𝑗 𝜌̂𝑗 (10.2.41)
𝜙̂𝑘,𝑗= 𝜙̂𝑘−1,𝑗 − 𝜙̂𝑘,𝑘𝜙̂𝑘−1,𝑘−𝑗 𝑗 = 1,2, … , 𝑘 − 1 (10.2.42)
The partial autocorrelation function of an autoregressive process of order 𝑝, 𝜙𝑘,𝑘, will be non-zero for 𝑘 less than or equal to p and zero for 𝑘 greater than 𝑝 (𝑖. 𝑒. 𝜙𝑘,𝑘= 0 𝑓𝑜𝑟 𝑘 > 𝑝).
Hence, its correlogram of partial autocorrelations cuts off after lag 𝑝. The estimates of the partial autocorrelations should also be tested for statistical significance at 5% level (𝐻0; 𝜙𝑘,𝑘 = 0). The standard error of the sample PACF “ 𝑆𝜙̂𝑘,𝑘” can be calculated as follows (Box & Jenkins, p. 65):
123 𝑆𝜙̂𝑘,𝑘≃ √1
𝑛 𝑘 ≥ 𝑝 + 1 (10.2.43)