While CCL 2008 takes an initial step towards exploring the effect of the monetary policy in a system-dynamics context, there are several issues to be addressed. First, CCL 2008 study the asset returns “one-at-a-time” and hence this precludes the possibility of interactions among different assets. Second, CCL 2008 did not touch on the fore-casting issue, which is one of the major concerns of both the public and the policy makers. Third, CCL 2008 consider only the condi-tional impulse response and hence ignore the uncertainty generated by the possible stochastic regime switching along the path of adjust-ment. In light of these shortcomings, CCL 2009 takes a preliminary step in attempting to address some of these issues.
Econometrically speaking, CCL 2009 also employs a regime-switching structural VAR (RSSVAR) model, as in the case in CCL 2008. However, they make several important changes. First, they focus on the housing return and stock return and include both vari-ables in all the models they estimate. Second, they include a much longer list of variables and estimate many different versions of those RSSVAR models. Third, after the estimations of all those models, they use the calculated probabilities of regime switching for evalu-ating the forecasting performances of house and stock prices across various models, and then examine both in-sample (1975 Q1–2005 Q4) and out-of-sample (2006 Q1–2008 Q3) forecasting performances.
2005 Q4 is naturally chosen as the cut-off point because the rise in the house price growth rate starting in the 1990s peaked around the end
of 2005. As will become clear, CCL 2009 actually allow the models to “learn”, ie, they update the data starting at 2006 Q1 and see how these models perform when the growth of US house prices began to decline and, consequently, the subprime crisis unfolded.
While detailed discussion of the literature is beyond the scope of this chapter, a few points are worth attention.5
First, CCL use multi-variate regime-switching SVAR models, while many existing studies on forecasting either use a single-variate (ie, the variable to be forecasted) model or employ linear VAR (ie, single regime) models. The former approach suffers from an endo-geneity problem (see Sims (1980b) for a detailed discussion), while the latter implicitly rules out the possibility of regime switching (see, for example, Hamilton 1994).
Second, studies on the possible regime-switching nature of the US monetary policy typically adopt the Bayesian econometric method (Sargent et al 2006; Sims 1980a,b; Sims and Zha 2006). In order for this book to complement the literature, it may be a merit for this chapter to adopt the “classical econometric method”.
Third, CCL 2009 conduct out-of-sample forecasting using two dif-ferent approaches: conditional expectations and simulation-based methods. While the former approach is easier to implement, it does not really track the regime that occurs for each forecasting step, and the confidence intervals are not available. Following Sargent et al (2006), CCL 2009 adopt the simulation-based approach to calculate the median path and the confidence interval. CCL provide more discussion on this issue.
Fourth, CCL 2009 perform the forecasting in a “system-dynamics manner” and hence can deliver its prediction on both housing and stock returns simultaneously. In fact, from the investor’s point of view, since the returns of the two assets are imperfectly correlated, it is natural for agents to include both assets under some dynamic port-folio consideration (see, for example, Yao and Zhang 2005; Leung 2007). Moreover, some recent works identify channels in which the housing markets and stock returns are closely related (Lustig and Van Nieuwerburgh 2005; Piazzesi et al 2007). Sutton (2002) presents evidence that a significant part of house price fluctuations can be explained by stock prices in six countries (US, UK, Canada, Ireland, the Netherlands and Australia). A study by the Bank for International Settlements (2003) also shows that, for a large group
Table 5.5 Statistical summary of federal funds rate, term spread, gross domestic production growth rate, external finance premium, market liquidity, stock index return and housing market return (1975 Q2–2008 Q3)
FFR SPR GDP EFP TED SRET HRET
Mean 6.397 1.502 0.759 1.087 0.883 1.968 1.344 Median 5.563 1.604 0.731 0.957 0.637 2.263 1.313 Max. 17.780 3.611 3.865 2.513 3.307 18.952 4.511 Min. 0.997 −2.182 −2.038 0.560 0.097 −26.431 −2.713 Std. Dev. 3.508 1.335 0.750 0.422 0.742 7.659 1.040 Skewness 1.037 −0.627 −0.127 1.220 1.552 −0.664 −0.040 Kurtosis 4.283 2.941 6.150 4.229 4.917 4.070 4.691 Oberv. 134.000 134.000 134.000 134.000 134.000 134.000 134.000
FFR, federal funds rate; SPR, interest rate spread; GDP, gross domes-tic production growth rate; EFP, external finance premium; TED, market liquidity; SRET, stock index return; HRET, housing market return.
of countries, house prices tend to follow the stock market with a two- to three-year lag. Kakes and End (2004) find that stock prices in the Netherlands significantly affect house prices. On the other hand, Lustig and Van Nieuwerburgh (2005) find that the US housing col-lateral ratio predicts aggregate stock returns and investors seem to demand a larger risk compensation in times when the housing col-lateral ratio is low. Yoshida (2008) finds that the housing component serves as a risk factor in the pricing kernel of equities and this miti-gates the equity premium puzzle and the risk-free rate puzzle. Thus, it would be important to take into account the interactions of stock returns and housing returns by studying them at the same time.
Data
It may be instructive to quickly review the data here. Table 5.5 sum-marises the data series that are used in CCL 2009. They include the (three-month) FFR, which are a measure of the US monetary pol-icy, the SPR, which are the discrepancy between the long-term (10-year) interest rate and the short-term (three-month) counterpart, the external finance premium (EFP), which is equal to corporate bond spread (Baa-Aaa), the TED spread which is the difference between the three-month Eurodollar deposit rate and the three-month Trea-sury bill rate, growth rates of GDP, the SRET and the HRET, covering the period 1975 Q1–2008 Q3.6All these variables are widely used in
Table 5.6 Correlation coefficients (1975 Q2–2008 Q3)
FFR SPR GDP EFP TED SRET HRET
FFR 1.000 −0.557 −0.104 0.544 0.833 0.009 0.015 SPR 1.000 0.145 0.037 −0.437 0.021 −0.115
GDP 1.000 −0.179 −0.165 0.030 0.111
EFP 1.000 0.650 0.057 −0.151
TED 1.000 −0.049 −0.076
SRET 1.000 0.055
HRET 1.000
Numbers in bold denote best fit models.
Table 5.7 List of models
Model Model structure Variables
A Linear FFR, SPR, TED, EFP, GDP, SRET, HRET B Two-regime FFR, GDP, SRET, HRET
C Two-regime FFR, SPR, SRET, HRET D Two-regime FFR, EFP, SRET, HRET E Two-regime FFR, TED, SRET, HRET F Two-regime EFP, SPR, SRET, HRET G Two-regime EFP, TED, SRET, HRET H Two-regime SPR, TED, SRET, HRET
Key: (unless specified, all variables refer to quarterly data) FFR, Fed-eral Fund Rate; SPR, term spread, which is equal to the 10-year bond rate minus FFR; TED spread, which is equal to the interbank rate minus the T-bill rate, a measure of market liquidity; EFP, external finance pre-mium, which is equal to the lending rate minus the T-bill rate, a measure of EFP; “GDP” denotes GDP growth rate; SRET, stock market return; HRET, housing market return.
the literature.7A comparison of their forecasting ability in a unify-ing framework, with emphasis on the interactive (through the use of VAR) and the regime-switching nature, seems to be a novelty of CCL 2009. Figure 5.6 shows that several of these variables seem to be significantly correlated. For instance, the correlation between FFR and TED is above 0.8 and that between TED and EFP is above 0.6.
Thus, it seems appropriate to study their movement in a VAR-type model rather than a single-equation context. In addition, while some of those variables are correlated, it is still possible to compare (and rank) their forecasting performance formally.
Table 5.7 shows clearly how models are constructed in a way that would facilitate the comparison. For instance, models A to D would have FFR involved, which can highlight the potential role of monetary policy in the asset return dynamics. Models F to H dif-fer from the previous ones as the monetary policy variable FFR is removed. Instead, an additional financial market variable is intro-duced to the system. Thus, model F can be interpreted as model C with FFR replaced by EFP, model G as model E with FFR replaced by EFP, and model H as model E with FFR replaced by SPR. As it will become clear, in despite of all these similarities, models with only
“one variable difference” may have a very different performance in forecasting.
Econometric procedures
CCL 2009 conduct out-of-sample forecasting starting 2006 Q1, and thus we divide the sample into an in-sample period (1975 Q2–
2005 Q4) and an out-of-sample period (2006 Q1–2008 Q3). We then proceed with out-of-sample forecasting in two different approaches.
The first approach is the conventional conditional moment meth-od. Given the estimation window 1975 Q2–2005 Q4 and a forecasting horizon h= 1, . . . , 4, the estimated parameters are used to forecast house and stock prices h-steps ahead outside the estimation win-dow, using the smoothed transition probabilities. The h-step-ahead forecasted value of zt+hbased on information at time t, Ωt, is given by
E(zt+h| Ωt)=
2 i=1
E[zt+h| st+h= i, Ωt]× p(st+h = i | Ωt)
where zt ∈ yt. The estimation window is then updated one obser-vation at a time and the parameters are re-estimated. Again the h-step-ahead forecasts of house and stock prices are computed out-side the new estimation window. The procedure is iterated up to the final observation, 2008 Q3. The forecasts based on this method basically compute the h-step-ahead conditional expectations of the variable being predicted. Most existing (non-Bayesian) works follow this method.
The second approach is the simulation method. The idea is simply that the path of the forecasted values is obtained by simulating the model repeatedly. The procedure is as follows.
• Step 1. The RSSVAR model is estimated from the sampling period 1975 Q2–2005 Q4, and the parameters, transition prob-abilities and variance–covariance matrix, etc, are obtained.
Given the estimation results, the smoothed probabilities for identifying the regime at the period 2005 Q4 are computed.
• Step 2. Given the regime at 2005 Q4, the path of h-step-ahead regimes by random drawing is simulated (h = 1, . . . , 4).8 Given this particular path of h-step-ahead regimes, the path of predicted values of zt ∈ ytis obtained from Equation 5.2.
• Step 3. Steps 1 and 2 are repeated 50,001 times in order to obtain the median of the h-step-ahead forecasted values during 2006 Q1–2006 Q4 and their corresponding confidence intervals.
• Step 4 The sampling period is then updated with four more observations (ie, data for another year) and steps 1–3 are repeated to simulate the path of predicted values for the next four quarters.
• This procedure is repeated up to the end of our sample.
An advantage of this method over the computation of the mean of possible future values in the first approach is that this method takes full account of the regime-switching model by determining the path of future regimes using random drawing, rather than simply taking expectations over transition probabilities. Another advantage is that a confidence interval with which to evaluate its forecasting perfor-mances is generated naturally. It should be noted that the regime-switching nature of the model implies that the future forecast is path dependent and hence the conventional way to construct confidence interval becomes invalid.
To evaluate the performances of in-sample and out-of-sample forecasts, we compute two widely used measures for forecasting a variable zt ∈ yt: root-mean-square errors (RMSEs) and mean absolute errors (MAEs), which are defined respectively by
RMSE(h)=
1 T− h
T−h
t=1
(zt+h− ˆzt+h|t)2
1/2
MAE(h)= 1 T− h
T−h
t=1
|zt+h− ˆzt+h|t|
where ˆzt+h|t≡ E(zt+h| Ωt).
Table 5.8 A summary of goodness of fit for all eight models
Model Models AIC
A Single-regime model (FFR, SPR, TED, 11.230 EFP, GDP, SRET,HRET)
B Two-regime (FFR, GDP, SRET, HRET) 13.472 C Two-regime (FFR, SPR, SRET, HRET) 12.450 D Two-regime (FFR, EFP, SRET, HRET) 10.159 E Two-regime (FFR, TED, SRET, HRET) 11.134 F Two-regime (EFP, SPR, SRET, HRET) 9.747 G Two-regime (EFP, TED, SRET, HRET) 8.404 H Two-regime (SPR, TED, SRET, HRET) 11.274 Numbers in bold denote best fit models.
Estimation results
As in the case of CCL 2008, we can only highlight some of the findings of CCL 2009. Table 5.8 provides a summary of the esti-mation results. In general, a model allowing for regime switching attains a lower value of Akaike’s information criterion (AIC) and a higher log-likelihood value. Among all these models, the regime-switching model (EFP, TED, SRET, HRET) has the best fit, ie, a sig-nificantly lower value of AIC than other models, suggesting that the credit market frictions and asset returns are indeed significantly inter-related.
CCL 2009 identify regime 1 as the “high-volatility regime” and regime 2 as the “low volatility regime”. And, as in the case of CCL 2008, both regimes tend to be highly persistent. For instance, the transition probability matrix of the model (EFP, TED, SRET, HRET) is given by
P=
p11 p12
p21 p22
=
0. 854 0. 146 0. 068 0. 932
which suggests that the expected duration of regime 1 is 1/(1−p11)= 6. 8 quarters and the expected duration of regime 2 is 1/(1− p22)= 14. 7 quarters.
In-sample forecasting
Table 5.9 summarises the in-sample forecasts of asset returns. For the stock returns, model C (FFR, SPR, SRET, HRET) has the best perfor-mance. For housing return, however, it is model D (FFR, EFP, SRET,
Table 5.9 A summary of in-sample forecasting performance (four-quarter-ahead forecasts)
Stock returns Housing returns
Model RMSE MAE RMSE MAE
A Single-regime model 7.5842 5.6699 0.8226 0.6499 (FFR, SPR, TED, EFP,
GDP, SRET,HRET)
B Two-regime 7.6411 5.6640 0.8286 0.6508
(FFR, GDP, SRET, HRET)
C Two-regime 7.5103 5.5922 0.7974 0.6361
(FFR, SPR, SRET, HRET)
D Two-regime 7.6460 5.6561 0.7801 0.6129
(FFR, EFP, SRET, HRET)
E Two-regime 7.6232 5.6959 0.7984 0.6207
(FFR, TED, SRET, HRET)
F Two-regime 7.7767 5.7204 0.7940 0.6331
(EFP, SPR, SRET, HRET)
G Two-regime 7.7917 5.8092 0.8397 0.6468
(EFP, TED, SRET, HRET)
H Two-regime 7.6169 5.7064 0.8161 0.6313
(SPR, TED, SRET, HRET)
Numbers in bold denote best fit models.
HRET) that out-performs all others. The results hold regardless of whether the RMSE or MAE is used as the criterion. Notice that both models contain the monetary policy variable FFR. It is interesting to see that neither the linear model with seven variables nor model B, which contains GDP growth, give the best performance. In other words, the commonly used approach of using a linear VAR model with as many variables as possible may not be advised. Instead, care-ful selection of variables with the regime-switching consideration may be very important in understanding the “reality”.
Out-of-sample forecasting
We now turn to the out-of-sample forecasts results based on the con-ditional mean on housing and stock returns in CCL 2009. Table 5.10 summarises the results. In general, the regime-switching model H (SPR, TED, SRET, HRET) performs very well. In terms of forecasting stock returns, it out-performs all other models under the criteria of RMSEs. Under the criteria of MAEs, it is extremely close to model A,
Table 5.10 A summary of out-of-sample forecasting performance (four-quarter-ahead forecasts)
Stock returns Housing returns
Model RMSE MAE RMSE MAE
A Single-regime model 7.9841 5.6808 2.1292 1.8424 (FFR, SPR, TED,
EFP, GDP, SRET,HRET)
B Two-regime 7.2027 5.8760 2.1303 1.8739
(FFR, GDP, SRET, HRET)
C Two-regime 7.3392 6.0156 1.9161 1.7198
(FFR, SPR, SRET, HRET)
D Two-regime 7.3122 5.9867 1.9977 1.7797
(FFR, EFP, SRET, HRET)
E Two-regime 7.0037 5.7126 2.0761 1.7754
(FFR, TED, SRET, HRET)
F Two-regime 8.2423 6.7808 1.8184 1.6078
(EFP, SPR, SRET, HRET)
G Two-regime 7.2071 5.7972 2.0430 1.7617
(EFP, TED, SRET, HRET)
H Two-regime 6.9225 5.6933 1.8284 1.5201
(SPR, TED, SRET, HRET)
Numbers in bold denote best fit models.
which is the top performer. In terms of forecasting housing returns, it out-performs all other models under the criteria of MAEs. Under the criteria of RMSEs, it is extremely close to model F, which is the top performer. Notice that, unlike the case for in-sample forecasting, model H does not contain either the monetary policy variable FFR or the fundamental variable GDP. It seems to suggest that the in-sample and out-of-sample forecasting powers can be quite different.
We next turn to the simulation-based forecasting results in CCL 2009. They consider a forecasting window of four quarters start-ing in 2006 Q1, with h-quarter-ahead forecasts, h = 1, . . . , 4. After simulating the out-of-sample path 2006 Q1–2006 Q4 based on obser-vations up to 2005 Q4, the data is updated with four obserobser-vations and the parameters are re-estimated. The procedure is iterated up to the final observation, 2008 Q3. The purpose of this exercise is to see how the performances of the models change when information is updated. The simulated paths together with their 80% confidence
Figure 5.8 Simulation-based out-of-sample forecasts of stock returns with 80% CI from 2006 Q1 to 2006 Q4 based on information available at 2005 Q4
−15 05Q2 05Q4 06Q2 06Q4 15
−15 05Q2 05Q4 06Q2 06Q4
15
−15 05Q2 05Q4 06Q2 06Q4 15
−15 05Q2 05Q4 06Q2 06Q4
15
−15 05Q2 05Q4 06Q2 06Q4
Data
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).
intervals can be visualised in Figures 5.8–5.10 for stock returns and in Figures 5.11–5.13 for housing returns. Tables 5.11 and 5.12 provide a summary of the performances of different models.
For the predictions of stock returns, the predicted paths of the first two forecasting windows (Figures 5.8 and 5.9) and actual data are well within the boundaries of the 80% confidence intervals for all five models. In a sense, although the models did not predict what actually
Figure 5.9 Simulation-based out-of-sample forecasts of stock returns with 80% CI from 2007 Q1 to 2007 Q4 based on information available at 2006 Q4
−15 06Q2 06Q4 07Q2 07Q4 15
−15 06Q2 06Q4 07Q2 07Q4
15
−15 06Q2 06Q4 07Q2 07Q4 15
−15 06Q2 06Q4 07Q2 07Q4
15
−15 06Q2 06Q4 07Q2 07Q4
Data
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).
happened in 2006 and 2007, the models’ predictions are not that far
“off the mark”. But the last window (2008 Q1–2008 Q3 in Figure 5.8 performs much worse: except for the regime-switching model H (SPR, TED, SRET, HRET), all models have at least one period (ie, a quarter) which lies outside the confidence region. Notice that, for overall out-of-sample forecasting performance (four-quarter-ahead forecasts), it is also model H that performs best overall. While more
Figure 5.10 Simulation-based out-of-sample forecasts of stock returns with 80% CI from 2008 Q1 to 2008 Q3 based on information available at 2007 Q4
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).
research is clearly needed, the results here seem to suggest that the interest rate spread (SPR) and the TED spread are indeed very important in the forecasting of stock returns, even in a financial crisis.
For the predictions of housing returns, the forecasting perfor-mances of all the models in a sense “deteriorate” much faster than the predictions for stock returns. Figure 5.11 shows that most models basically capture the downward trend of the housing return in 2006
Figure 5.11 Simulation-based out-of-sample forecasts of housing returns with 80% CI from 2006 Q1 to 2006 Q4 based on information available at 2005 Q4
0 05Q2 05Q4 06Q2 06Q4 3.5
0 05Q2 05Q4 06Q2 06Q4 3.5
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).
within their 80% confidence intervals, although model A (the lin-ear model with all seven variables) and model D (FFR, SPR, SRET, HRET) are not totally successful even for the forecasting of 2006.
Unfortunately, Figure 5.12 seems to suggest that the models are “mis-led” by the “bound back” of housing return in 2006 Q4, which results in basically “flat predictions” for the 2007 returns. The reality is much worse, and hence the data for 2007 is basically outside the confi-dence intervals of all models, except for 2007 Q1. Figure 5.13 shows
Figure 5.12 Simulation-based out-of-sample forecasts of housing returns with 80% CI from 2007 Q1 to 2007 Q4 based on information available at 2006 Q4
−0.4−0.8 06Q2 06Q4 06Q2 06Q4 2.8
−0.4−0.8 06Q2 06Q4 06Q2 06Q4
2.8
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).
that there is another “bound back” of housing return in 2007 Q4.
This time, all the models even predict that the housing returns will increase and the confidence intervals are increasing in value over time. The reality again disappoints. As a result, for the forecasting window 2008 Q1–2008 Q3, the data lie completely outside the con-fidence interval. In other words, all models fail, as summarised by Table 5.12.
Figure 5.13 Simulation-based out-of-sample forecasts of housing returns with 80% CI from 2008 Q1 to 2008 Q3 based on information available at 2007 Q4
−307Q1 07Q3 08Q1 08Q3 4
−307Q1 07Q3 08Q1 08Q3
4
−307Q1 07Q3 08Q1 08Q3 4
−307Q1 07Q3 08Q1 08Q3
4
−307Q1 07Q3 08Q1 08Q3
Data
Model A: single-regime (FFR, SPR, TED, EFP, GDP, SRET, HRET); Model B: two-regime (FFR, GDP, SRET, HRET); Model C: two-two-regime (FFR, SPR, SRET, HRET);
Model D: two-regime (FFR, EFP, SRET, HRET); Model E: two-regime (FFR, TED, SRET, HRET); Model F: regime (EFP, SPR, SRET, HRET); Model G: two-regime (EFP, TED, SRET, HRET); Model H: two-two-regime (SPR, TED, SRET, HRET).