Optimal Window Selection for Forecasting in The Presence of Recent Structural Breaks

Transcription

1 Optimal Window Selection for Forecasting in The Presence of Recent Structural Breaks Yongli Wang University of Leicester June 23, 2017 Abstract: This paper proposes two feasible algorithms to select the optimal window size for forecasting in rolling regression. It develops the framework of Inoue, Jin, and Rossi (2017), keeping the desired properties, like the weak dependence, multi-step ahead forecasting and asymptotic validity. The Monte-Carlo experiments show that the proposed bootstrap method in this paper outperforms their original algorithm in almost all cases. It is also shown that the forecasts from the proposed methods are superior to those from other existing methods in some cases, and close to the best forecasts in other cases. However, when the break occurs far before the time of making forecasts and the break size is significant, using only post-break data is almost always the best strategy. Keywords: Parameter instability; Forecasting with breaks; Optimal window selection. Acknowledgement: This research used the ALICE High Performance Computing Facility at the University of Leicester. I am grateful to Wojciech Charemza, Carlos Díaz Vela and Deborah Gefang from the University of Leicester for their constructive comments and stimulative discussions. I also thank Lu Jin from the StataCorp for helpful correspondence at an early stage of this research. I am solely responsible for all remaining deficiencies. 1

2 1 Introduction It has been widely accepted that parameter instability is a crucial issue in forecasting (see Clements and Hendry, 1996, 1998; Hendry, 2000; Pesaran and Timmermann, 2002; Goyal and Welch, 2003; Pesaran, Pettenuzzo, and Timmermann, 2006; Koop and Potter, 2007; Castle, Clements, and Hendry, 2013; Rossi, 2013). In the presence of a structural break, 1 Clark and McCracken (2009) demonstrate using pre-break data in estimation may lead to biased parameter estimates and hence, give biased forecasts. 2 As empirical evidence of parameter instability has been found broadly in financial prediction (see Goyal and Welch, 2003), exchange rate forecasting (see Schinasi and Swamy, 1989; Sarno and Valente, 2009) and macroeconomic forecasting (see Stock and Watson, 1996, 1999, 2007), the usage of rolling windows has been increasing in practice as it discards remote observations (Clark and McCracken, 2009). However, the choice of the estimation window size in the rolling regression, which is usually conventionally fixed by an arbitrary number or by the past experience, 3 has always been a concern for practitioners. Inoue and Rossi (2012) suggest that different window sizes may lead to different empirical results in practice and good results might be obtained simply by chance. 4 window could be essential to the forecasting accuracy. Thus, the size of the estimation There has been a long history of detecting structural breaks and identifying break dates in stationary models (see Chow, 1960; Andrews, 1993; Bai and Perron, 1998, 2003; Altissimo and Corradi, 2003). Consequently a standard solution to treat instability is to estimate parameters by only using observations after the last structural break. But reducing sample size also increases the variance of the parameter estimates, which maps into the forecast errors. Hence, it may not always be optimal to only use the post-break data. Instead, Pesaran and Timmermann (2007), henceforth PT, claim that forecasters can select the optimal window size for each forecast by considering a trade-off between the potential forecast bias and the variance. Upon this, they propose a cross-validation 1 Clements and Hendry (2006) define a structural break as a permanent change in the parameter vector of the forecasting model. 2 Also see the forecast error decomposition in Clements and Hendry (1996, 1998). 3 For example, Meese and Rogoff (1983) use 93 observations of monthly data in each rolling window to forecast exchange rates, while Molodtsova and Papell (2009) considers a window size of 120 monthly observations to forecast exchange rates. 4 They also concern that the process of searching the best forecasting model and window sizes is not taken into account as researchers usually report the empirical results for a selected window size. 2

3 (CV) approach, which partitions the full sample into estimation and validation parts as a pseudo out-of-sample forecasting practice and ranks the recursive mean squared forecast errors (MSFEs) to select the starting point of the estimation window. However, PT s CV method requires strictly exogenous regressors and uncorrelated errors. Based on the same trade-off idea, the latest research by Inoue, Jin, and Rossi (2017), referred as IJR in the following context, proposes a feasible solution with the asymptotic validity of approximating the forecaster s quadratic loss function 5 to select the optimal estimation window size, allowing weak dependence and multiple-step ahead forecasts. However, their algorithm combining PT s CV approach may suffer from selection bias, once a break occuring shortly before the date of making forecasts 6 distorts the ranking in the validation. To treat the potential selection bias, this paper proposes two alternative feasible algorithms to select the optimal window size for forecasting by solving the forecaster s optimization problem. The first algorithm, referred as the bootstrap algorithm, incorporates a residual based bootstrap, which potentially captures the break date and break size, to estimate the key parameter vector in the forecaster s loss function. In seek to reduce the computation time and simplify the decision rule, a simplification referred as the simple selection method is proposed as the second alternative. Without using CV or bootstrap, it simply estimates the parameter vector in the loss function using only post-break data on an ad-hoc basis. Both proposed algorithms keep the desired properties of IJR s original method, such as the weak dependence, multi-step ahead forecasting and asymptotic validity. One-step ahead forecasts from these two proposed methods are compared with several different window selection approaches, in the presence of a single structural break. The Monte-Carlo experiments suggest that the forecasting performance of the proposed algorithms is as good as that of existing methods in most cases and is superior to others when the break size is medium or the break date is close to the forecast date. But when the break occurs sufficiently far before the time of making forecasts or the parameter shift is significant, forecasting using only the post-break data is almost always the best strategy. The following part of the paper is organised as follows. Section 2 describes the model setup and notations, following IJR. The proposed bootstrap and simple selection algo- 5 The forecaster s quadratic loss function in this paper is a function of the squared forecast error, as IJR. 6 In the following context, such break is defined as a recent break. 3

4 rithms are discussed in Section 3. Section 4 introduces the Monte-Carlo experiments and analyses the forecasting performance of the proposed methods against that of existing literature, followed by an extra group of simulations examining the effect of trimming values in Section 5. A brief conclusion is given in Section 6, along with a short discussion of the limitation and future development. 2 Model Framework Following the setup in IJR, consider the following data generating process (DGP), y t+h = x tβ h ( t T ) + ɛ t+h, t = 1, 2,, T, (1) where y t+h is the outcome observed at time t + h, x t = (x t1, x t2,..., x tk ) is a (k 1) vector of regressors at time t, β h ( t T ) = (β t1, β t2,..., β tk ) is a (k 1) vector of parameters at time t, T denotes the full sample size, h 1 denotes the forecast horizon and ɛ t+h is an unobservable stochastic disturbance with zero mean and finite variance, i.e., E[ɛ t+h Ω t ] = 0 and V ar[ɛ t+h Ω t ] = σt 2 < t, where Ω t = {x t, x t 1,, y t, y t 1, } is the information set at time t. This setup avoids parametric restrictions on β h ( ) and allows for multi-step ahead forecasts as h > 1, giving a lot of flexibility to IJR model. With unspecified form of parameter function, both types of parameter instability are covered under this framework. The setting of β h ( t ) is essential to derive consistent nonparametric estimates later T on, as demonstrated in Robinson (1989) that making β h ( ) depend on the sample size T is necessary to provide the asymptotic justification for any non-parametric smoothing estimators. 7 Noticeably, assumptions of strictly exogenous variables or no autocorrelation are not required. Hence, lagged dependent variables are allowed in the vector x t. The forecaster s objective is to predict the future value y T +h at time T. Since the conditional MSFE is commonly used as the measure of forecasting accuracy, such as in Meese and Rogoff (1983, 1988), Stock and Watson (1999, 2003, 2007, 2016), Pesaran and Timmermann (2007), Molodtsova and Papell (2009), Giraitis, Kapetanios, and Price (2013), the 7 The same setting is also used in Cai (2007). 4

5 forecaster s loss function is defined as E[(y T +h ŷ T +h ) 2 Ω T ], (2) where ŷ T +h is the forecast of y T +h at time T, which can be given by ŷ T +h x T ˆβ h,r ( T T ) = x T ˆβ h,r (1), (3) where ˆβ h,r (1) = ( ˆβ T 1, ˆβ T 2,..., ˆβ T k ) is a (k 1) vector of estimated parameters for h-step ahead forecasts at time T, using the most recent R observations, which can be estimated by the simple ordinary least squares (OLS) as ˆβ h,r (1) ( T h t=t R+1 x t x t) 1 ( T h t=t R+1 x t y t+h ). (4) Here R is regarded as the size of the estimation window. When R = T, the full sample is used to estimate the parameters. To simplify the notation, let ˆβ R (1) = ˆβ h,r (1), dropping the subscript h. By substituting equations (1) and (3) into equation (2), it can be shown that minimizing equation (2) is equivalent to minimizing [ ˆβ R (1) β(1)] x T x T [ ˆβ R (1) β(1)]. (5) Since the true value of β(1) from the true DGP in equation (1) is infeasible to be known in practice, searching R to minimize equation (5) with β(1) known is regarded as IJR s infeasible method. Instead, IJR proposes to approximate the true parameter vector β(1) in the forecaster s loss function using a local linear regression, which is a superior technique in theory and applications among non-parametric regressions, also used in Cai (2007). 8 They approximate β(1) with the OLS estimates β(1), as [ ] [ β(1) xt x t xt x = t( t T ) ] 1 [ ] xt y T t+h β (1) (1) xt x t( t T ) xt x T t( t T T )2 xt y t+h ( t T ), (6) T 8 See the discussion about the local linear regression against the Nadaraya-Watson estimator regression and the detailed derivation of the approximation in IJR. 5

6 where represents t=t h t=t S+1. Both β(1) and β (1) (1) are estimated using the most recent S observations, where S = 2k,, T 9 is an arbitrary number, which is the estimation window size of β(1). Replacing β(1) with β(1) in equation (5), the optimal window size ˆR satisfies ˆR arg min R Θ R [ ˆβ R (1) β(1)] x T x T [ ˆβ R (1) β(1)], (7) where Θ R = [R, R] is the searching zone for the optimal window size ˆR, β(1) is from equation (6), using the most recent S observations, and ˆβ R (1) is from equation (4), using the most recent R observations. The asymptotic validity of this approximation has been proved by IJR. Finally, substitute ˆR in equation (4) to get ˆβ ˆR(1). Then substitute it into equation (3) and the optimal forecast of y T +h at time T, regarding to minimizing the forecaster s loss function, is given by ŷ T +h = x ˆβ T ˆR(1). (8) It must be reminded that the optimal window sizes across different forecast horizons may differ, since the forecast horizon h appears in equations (4) and (6). Noticeably, the issue of selecting optimal window size has been transferred to an estimation question of computing β(1) in equation (6). However, it is not hard to show that the estimates of β(1) are related to the choice of its estimation window size S. IJR chooses S using PT s CV approach with unknown break dates (see Section 3.3 in Pesaran and Timmermann, 2007). However, since PT s method is not designed for local estimation, the estimated optimal forecasting window under CV approach may not be the optimal estimation window for β(1). More importantly, PT assumes that the last structural change occurs sufficiently long before the time when the forecasts are required. In the presence of a recent break, the ranking mechanism, which is the core in the CV method, may suffer bias and distort the window selection approach. Thus, alternative methods to select S need to be developed, especially in the presence of a single recent structural break. 9 S 2k is required so that the matrix in equation (6) is invertible. 6

7 3 Methodology As this paper focuses on the forecasting performance under a recent break, following equation (1), the parameter function is specified as β h ( t T ) = β 1 for t T b and β h ( t T ) = β 2 for t T b + 1, where T b < T is an integer that is usually known as the break date. There is no restriction on the parameters in two regimes. The forecaster s objective is the same as that in Section 2. So as discussed before, the best forecast is given by equation (8), but considering the following processes to select S in equation (6). Considering that the real DGP in equation (1) is known, M time series can be generated and different values of S Ψ are assigned to each series to make forecasts, where Ψ = {s} T s=2k. For example, [1] for the given data, give an initial value like S = 2k; [2] Estimate the values of β in equation (6) with the given value of S; [3] Substitute β into equation (7) and get the value of ˆR; [4] Given ˆR, estimate ˆβ(1) in equation (4) and then make a forecast using equation (3). [5] With the same series but a different value of S, repeat steps [2]- [4] until all the possible S Ψ has been picked. [6] Repeating the above steps for all the M time-series, the optimal S should be the value that minimizes the forecaster s loss function in equation (2). Hence, the optimal window size of estimating β(1) is defined as S arg min S Ψ M (y (m) T +h ŷ(m) m=1 T +h T,S )2, (9) where y (m) T +h is the outcome at time T + h for the m-th replication and ŷ(m) T +h T,S is the h-step ahead forecast at time T under S for the m-th replication, computed from equation (8) with the estimated parameters from equations (4),(6),(7). Apparently, it is unrealistic to know the realization of y T +h at time T, not to mention the real DGP. Thus equation (9) is not feasible to be applied in practice. However, it is possible to reproduce the time series by resampling. As Pesaran and Timmermann (2005) argue the choice of the window size depends on the nature of the possible model instability and the timing of the breaks, forecast accuracy can be realistically improved by exploiting the information of the data structure, i.e., the time of the parameter change T b and perhaps the size of the change. Instead of assuming the break occurs before the calibration period under CV (see the details of CV in Section 4.2), this study proposes a simple residual based bootstrap technique, within which such change can be recreated at time T b by resampling. 7

8 This is referred as the proposed bootstrap algorithm on the optimal window selection. The proposed resampling technique is similar with the bootstrap procedure described in Nankervis and Savin (1996) and the semiparametric bootstrap in Davidson and MacKinnon (2006), but partitioning the data into two period according to the break date T b. By resampling the centred residuals from the two estimated empirical distribution functions (EDFs) respectively, numerous bootstrap samples can be generated, containing the information of the break in the original series. The details of the proposed bootstrap algorithm are the following: First, divide the series {y t, x t } T t=1 into two parts according to the break date T b as {y t, x t } T b t=1 and {y t, x t } T t=t b +1. Thus, the OLS estimates of the corresponding partition are given by T b h T b h ˆβ 1 = ( x t x t) 1 ( x t y t+h ). (10) ˆβ 2 = ( t=1 T h t=t b +1 x t x t) 1 ( t=1 T h t=t b +1 Second, estimate residuals in each partition respectively as x t y t+h ). (11) ˆɛ 1,t+h = y t+h ˆβ 1x t, t = 1, 2,, T b h. (12) ˆɛ 2,t+h = y t+h ˆβ 2x t, t = T b + 1, T b + 2,, T h. (13) Third, the estimated residuals in equations (12) and (13) are centred respectively, as ɛ 1,t = ˆɛ 1,t ɛ 1, t = 1 + h, 2 + h,, T b, (14) ɛ 2,t = ˆɛ 2,t ɛ 2, t = T b h, T b h,, T, (15) where ɛ 1 and ɛ 2 are the respective mean of {ˆɛ 1,t } T b t=1+h and {ˆɛ 2,t} T t=t b +1+h. Hence, the corresponding EDFs of the two partitions are respectively E 1 = { ɛ 1,1+h, ɛ 1,2+h,, ɛ 1,Tb } and 8

9 E 2 = { ɛ 2,Tb +1+h, ɛ 2,Tb +2+h,, ɛ 2,T }. 10 Fourth, resample T b residuals with replacement from E 1 with equal probability 1 E1 = {ɛ 1,1+h, ɛ 1,2+h,, ɛ 1,T b +h }. Then a bootstrap sample can be generated as T b h as y t+h = ˆβ 1x t + ɛ 1,t+h, t = 1, 2,, T b, (16) where y t = y t for t = 1,, h (initial values), and x t = x t if there are no lagged dependent variables as regressors; Otherwise, update the value of the lagged dependent variables in x t with the corresponding values of the bootstrap series yt at each time. Similarly, 1 resample (T T b ) residuals with replacement from E 2 with equal probability E2 = {ɛ 2,T b +h+1, ɛ 2,T b +h+2,, ɛ 2,T +h } and resample dependent variable as T T b h as y t+h = ˆβ 2x t + ɛ 2,t+h, t = T b + 1, T b + 2,, T. (17) Hence, a bootstrap sample with a (T + h) 1 vector of dependent variable {y t } T +h t=1 and a T k matrix of regressors {x t } T t=1 is generated. Fifth, repeat step four B times, and then from equation (9), the optimal S for the series is given by minimizing forecaster s loss function, as Ŝ arg min S Ψ B i=1 (y (i) T +h ŷ (i) T +h T,S )2, (18) where y (i) T +h is the outcome at time T + h for the i-th bootstrap sample, and ŷ (i) T +h T,S is the h-step ahead forecast at time T under S for the i-th bootstrap sample, which is computed from equation (8) as ŷ (i) T +h T,S = x (i) T ˆβ (i) (1), (19) ˆR S where x (i) (i) T is the observations at time T in the i-th bootstrap sample, and ˆβ (1) is the ˆR S OLS estimates at time T for the i-th bootstrap sample, according to equations (4),(6),(7) with the given value of S. One may concern the computation burden of introducing the bootstrap. For example, 10 Davidson and MacKinnon (2006) argue that it gives a better estimate of the true DGP to rescale the centred residual to adjust the variance of the resampled bootstrap error terms. But this study chooses not to adapt as it might distort the scale of the error terms. 9

10 under rolling windows, the computation time increases dramatically as the size of the bootstrap samples, T, increases. Thus, this study also tests a simplified decision rule, which sets S = T T b, i.e., using only the post-break data to estimate β(1) in equation (6). It excludes the potential distortion from using pre-break data and can be seen as applying PT s post-break method on estimating β(1). This is referred as the proposed simple selection method. Without applying PT s CV method, it avoids suffering the ranking bias in the presence of a recent break and significantly reduces the computation burden in both IJR s algorithm and the proposed bootstrap algorithm. However, it must be pointed out that it may suffer from small sample bias if the break date is close to the forecasting date. In practice, the break date T b can be estimated by the break tests mentioned in Section 1. Bai and Perron (1998, 2003) provide consistent estimates of the break dates in linear models under a set of general conditions, and their tests are widely used in the literature, such as PT, Pesaran, Pick, and Pranovich (2013) and IJR. Alternatively, multiple breaks can also be detected by the tests in Altissimo and Corradi (2003). As this study assumes there is only one break, the sup-f test in Bai and Perron (1998, Section 4.1), referred as the BP test, provides a consistent estimate of the break date under the null of zero versus one break and will be applied in the following Monte-Carlo study. 4 Monte-Carlo Experiments A Monte-Carlo analysis is designed to test the forecasting performance of the two proposed algorithms against other existing methods under a structural break, where the position and the size of the break vary across a large number of different cases. It is expected to observe that there exist some critical points where the optimal window selection method for forecasting switches across different position and sizes of the break. 10

11 4.1 DGPs The DGPs are based on a bivariate VAR(1) model, which is also used in PT and IJR as follows, [ yt+1 w t+1 ] [ ] [ ] at b t yt = w t [ µt+1 υ t+1 ], (20) where [ µt+1 υ t+1 ] ([ ] [ ]) i.i.n,. (21) The detailed setup of DGPs is listed in Table 1 in the Appendix. This study proposes three groups of DGP settings. The first group (DGP 1) contains constant parameters. In theory, the benchmark forecast based on the full sample should perform the best in this case. The second group (DGPs 2-5) has two sizes of breaks with break dates moving towards the date of making forecasts. DGPs 2 and 3 respectively have a one-time small and large structural break at time T b on the AR parameter, while DGPs 4 and 5 respectively have a small and large structural break at time T b on the marginal coefficient. The position of the break T b varies from remotely (0.25T ) to very recently (0.95T ) with 0.05T segments, as T b {0.25T, 0.3T,, 0.95T }. The asymmetric positioning of breaks is set because it is more important to test the forecasting performance of window selection methods when there exists a recent break, other than when breaks occur sufficiently far before the time of making forecasts. It is expected that all feasible methods tend to perform worse as the time of breaks towards the date of making forecasts (T b T ). The third group (DGPs 6-7) gives different sizes of the breaks, which is fixed in a specific date (0.25T,0.50T,0.75T,0.85T,0.90T or 0.95T ). DGP 6 has a structural break with size 0.01θ 1 at time T b on the AR parameter, where θ 1 = [5, 40] with increment 5. When θ 1 = 40, it is indeed the DGP 3. DGP 7 has a structural break with size 0.01θ 2 at time T b on the marginal coefficient, where θ 2 = [10, 100] with increment 10. When θ 2 = 100, it is indeed the DGP 5. It is expected that when the size of the break is too small to be detected, forecasts using the optimal window selection method is no better than those using the full sample, and maybe even worse. 11

12 4.2 Window Selection Methods The forecasting model is set to be y t = c + ay t 1 + bw t 1 + ɛ t, (22) where a < 1 and ɛ t is assumed to be i.i.d (0, σ 2 ). Thus, model uncertainty is excluded. All the parameters are estimated using simple OLS with the following window selection methods, including five feasible methods and three infeasible methods. The former includes PT s post-break method (labelled as Postbk ), PT s CV method with unknown break dates (labelled as PTCV ), IJR s feasible approach (labelled as OptR ), the proposed simple selection algorithm with estimated break date (labelled as EstSS ) and the proposed bootstrap algorithm with estimated break date (labelled as EstBS ). The latter contains IJR s infeasible method (labelled as InfIJR ), the proposed simple selection algorithm with known break date (labelled as TrueSS ), and the proposed bootstrap algorithm with known break date (labelled as TrueBS ). [1] In the Postbk method, parameters in equation (22) are estimated by equation (4) with R = T ˆT b, where ˆT b is the estimated break date by the BP test with the trimming value The trimming value determines the minimum number of observations in a single regime, 0.15T. In other words, the range for possible break date under the BP test is [0.15T, 0.84T ]. 11 Critical values are set at 5% significant level and are available in Bai and Perron (1998). If the BP test shows there is no break, then set R = T. [2] In PTCV, following the setup in PT, the CV approach partitions the sample into estimation part [1, γ] and validation part [γ + 1, T ] to compute the pseudo out-of-sample MSFEs, where γ = 0.75T. This framework reserves the last 25%T observations for calibration. Consider a subsample [τ, γ] as an initial estimation window, the pseudo out-ofsample recursive MSFEs are defined as T 1 MSF E(τ T, γ) = (T γ) 1 (y i+1 x ˆβ i τ:i ) 2, (23) 11 One observation is lost due to the existence of the first-order lagged dependent variable. The effective sample size is 99 and a minimum of 15 observations is required in each regime, given T = 100. i=γ 12

13 where ˆβ τ:i = ( i 1 t=τ x tx t) 1 ( i 1 t=τ x ty t+1 ) is the estimated parameter vector in the estimation window [τ, i]. Then the optimal estimation window, denoted as [τ, T ], is selected by ranking the pseudo out-of-sample MSFE in equation (23), as 12 τ (T, γ, τ) arg min τ=1,, τ T 1 {(T γ) 1 (y i+1 x ˆβ i τ:i ) 2 }. (24) where τ is the upper bound of the searching zone for τ, so that the smallest estimation window size in equation (24) is restricted at τ τ i=γ In practice, τ is set to be the last break date τ = T b if the break date is known or τ = 65%T regardless of the breaks. This study incorporates the PTCV method with unknown break dates, other than the estimated break dates, because the latter requires the assumption that breaks occur before the last 25%T observations, which is not the case in this Monte-Carlo study. In other words, the window size selected by PTCV is no less than 35%T, which is R = T τ +1. [3] OptR follows IJR s setting of their OptR1 in Inoue, Jin, and Rossi (2017, Section 4.2), which is a two-step procedure. First, test the parameter instability with the BP test as Postbk. Second, if the null hypothesis of constant parameter could not be rejected in the first step, it is concluded that there is no parameter instability in the sample and hence, set R = T. Otherwise, set S = T τ + 1, R = max(1.5t 2/3, 20), and R = min(4t 2/3, T h), where τ is from equation (24) as PTCV. Then the optimal window size R = ˆR is selected from [R, R] by equation (7), minimizing the approximated conditional MSFE. [4] In EstSS, the following two steps are applied. First, test the parameter instability using the BP test with the same setting described in Postbk. Second, similarly with OptR, if the null hypothesis of constant parameter could not be rejected in the first step, set R = T. Otherwise, set S = T ˆT b, where ˆT b is estimated by the BP test in the first step. Then estimate β(1) by equation (6) and select the optimal window size R = ˆR by equation (7), subject to R = max(1.5t 2/3, 20), and R = T. 14 [5] The bootstrap based method EstBS with the estimated break dates, also has two steps. The first step is the same as OptR and EstSS, while the second step is the following. If the null hypothesis of constant parameter could not be rejected in the first 12 The optimal estimation window size is R = T τ The smaller the estimation window, the bigger the forecast variance in the validation stage. 14 Since ˆT b is restricted at least before 0.84T, it always satisfies the condition S 2k in equation (6). 13

14 step, set R = T. Otherwise, bootstrap B = 1000 samples with estimated break date ˆT b. Then select Ŝ with bootstrap samples by equation (18) with Ψ = [20, T ] and apply it in equation (6) to estimate β(1). The optimal window size R = ˆR is estimated by equation (7) with the same R and R in EstSS. [6] IJR s infeasible approach InfIJR is considered as a benchmark that gives the best forecasts all the time in theory. Following the setting in IJR, the optimal window size R = ˆR is chosen from [0.1T, 0.9T ] by the infeasible criterion in equation (5), minimizing the forecaster s loss function. [7] TrueSS sets S = T T b, where T b is assumed to be known. Estimate β(1) by equation (6) with given S and the optimal window size R = ˆR is chosen by equation (7), subject to the same R and R in EstSS. When there are no more than 10 observations after the break date, to satisfy the condition S 2k in equation (6), set S = 10. [8] TrueBS bootstraps B = 1000 samples with known break date T b and select Ŝ with bootstrap samples by equation (18). Substitute Ŝ in equation (6) and the optimal window R = ˆR is chosen by equation (7) with the same R and R in EstSS. If the DGP has constant parameters, then bootstrap samples without breaks. 4.3 Results The forecasting performance is evaluated over 5000 simulations for T = 100 and h = 1, measured by the square root of MSFE ratios (RMSFER), which is produced by the square root of MSFE under optimal window selection method against that using the full sample, as 5000 m=1 (y(m) T +1 ŷ(m) T +1 ) m=1 (y(m) T +1 ỹ(m) T +1 )2, (25) where ŷ (m) T +1 is the one-step ahead forecast using the optimal window size in the mth replication and ỹ (m) T +1 is the one-step ahead forecast using the full sample in the mth replication. If the value of RMSFER in equation (25) is less than one, that means the optimal window method performs better than the traditional method using the full sample. Table 2 in the Appendix reports the RMSFERs in the first two groups (DGPs 1-5), while Tables 3-8 in the Appendix report the RMSFERs in the third group (DGPs 6-7). The small- 14

15 est number (before rounding off to three decimal digits) is highlighted in bold, excluding the three infeasible methods, InfIJR, TrueSS and TrueBS. The last column in Tables 2-8, labelled as DR, reports the detection rate of the BP test in percentage. In addition, Figure 1 in the Appendix clearly indicates the forecasting performance of the optimal window selection methods in the presence of a single break with different positions, while Figures 2 and 3 in the Appendix show the forecasting performance of the optimal window selection methods in the presence of a single break with different sizes. The results are summarised here as follows. 1). As expected in DGP 1, no feasible methods perform better than the forecasts using the full sample, since the RMSFERs from all the feasible methods are larger than 1. It is also curious to find that the BP test wrongly reports a break in 14% cases, which is a little bit higher than the significant level 5%. 2). A detection rate as high as 80% does not guarantee that using the Postbk method yields the best forecast, although it is almost always true when it is above 90%. On the other hand, it is never the best forecast device when the detection rate is below 50%. Apart from the position of the break, the size of the break is also an important factor in selecting optimal window selection method. 3). The infeasible method InfIJR always produces the smallest RMSFER across all DGPs, as it assumes the true parameter vector in the DGP at time T is known. The fact that the RMSFER from InfIJR always declines against the break dates and sizes suggests that the gain of forecasting with optimal window sizes increases as the break date moves towards the date of making forecasts and the break size increases. 4). The proposed method EstBS almost always produces a smaller RMSFER than IJR s original method OptR, although it does not yield the smallest RMSFER in most cases. This extension can effectively improve the forecast performance against IJR s original method in the presence of a structural break, especially when the break date is at 0.75T or 0.85T and the break size is not small, where the detection rate is high. 5). In DGPs 2 and 3, Postbk dominates others when the break date is before or at time 0.85T. But it is indifferent from the forecasts using the full sample, if the break date is after that time. A possible explanation is the failure of the detection, as the detection rate drops significantly after time 0.85T. 15

16 6). In DGP 4, PTCV yields the smallest RMSFER when the break date is before or at time 0.65T, although Postbk produces a slightly larger RMSFER. However, as the ranking algorithm of PTCV suffers bias when there is a break after time 0.65T, the proposed EstBS performs the best when the break is between 0.70T and 0.85T. But because of the failure of the detection when the break is positioned at time 0.90T or 0.95T, EstBS is no better than PTCV, which does not require the information of the break dates. Figure 1, DGP 4 in the Appendix, shows this crossing of the lines and it is curious to know whether there exists such a critical break size that the optimal window selection method switches at. DGPs 6 and 7 are designed to answer the exact question. 7). In DGP 5, Postbk dominates others in all cases, except the last one when the break is at time 0.95T. Combining the results from DGP 3, the advantage of only using post-break data is increasing until the break is at time 0.80T, where the forecasts start to suffer more loss from the small sample bias than the gain from discarding pre-break data. But it is a different story when the break size is small (DGPs 2,4), where the gain from discarding pre-break data is much smaller than that in the DGPs which contain a large break (DGPs 3,5). 8). In DGPs 6 and 7, when the break size is tiny, no matter where the break is, there is almost no method which can beat the forecasts using the full sample significantly. If the break is positioned at time 0.95T, PTCV almost produces the smallest RMSFER no matter how small the break is, although such gain against other methods is very small. However, the results in other cases apart from the above are interestingly mixed. 9). When the break is at time either 0.25T or 0.50T in DGPs 6 and 7, if the break size is tiny, no method is significantly better than the forecasts using the full sample. Whilst, Postbk is the best when the break becomes larger. Moreover, the larger the break size is, the advantage of using Postbk is more significant. A possible explanation is the combined effects of the increased detectability of the break test and the huge gain from discarding pre-break data when the parameter shift is significant. It is also noticed that the Postbk method is more sensible to a small break on the AR parameter than that on the marginal coefficient. 10). When the break is at time either 0.75T or 0.85T in DGPs 6 and 7, there is a similar transition pattern where the best forecast method is moving across PTCV, EstBS and Postbk, as the break size increases. The proposed method EstBS yields the smallest 16

17 RMSFER when the break size is 0.10 in DGP 6 and 0.50 in DGP 7. If the break occurs at time 0.85T, it also gives the smallest RMSFER if the break size is 0.15 or 0.20 in DGP 6 and 0.60 in DGP 7. 11). When the break is at time 0.90T in DGPs 6 and 7, on the one hand, Postbk hardly performs well and even worse than the forecasts using the full sample in many cases. It also performs poorly when the break is at time 0.95T, if not more serious. On the other hand, the proposed method EstSS yields the smallest RMSFER in many cases, while the EstBS performs no significantly worse than EstSS in most cases. To summarize, when the break occurs before 0.85T and the break size is significant, Postbk is always the best window selection methods. But it is no better than the forecasts using the full sample if the break occurs close to the forecast date, i.e., at 0.90T or 0.95T. PTCV can be a good alternative if the break happens at 0.95T or if the break size is quite small but not tiny. The proposed method EstBS can be the best window selection method if the break size is medium and the break occurs at or after 0.75T. EstSS can only be a good window selection method when the break occurs at 0.90T or 0.95T, while IJR s original method OptR never yields the smallest RMSFER in the Monte-Carlo study. 5 Robust Check on Trimming Values with Recent Breaks Most results in Section 4 are based on a trimming value of 0.15 in the BP test, which restricts the estimated position of the break to be not after 0.84T. IJR shows that their algorithm is robust to the choices of the trimming value. This can be expected since their method does not require the information of the estimated break date, which is not the case for the proposed methods in this study. The effect of the choice of the trimming value to the proposed methods is examined in this section. DGPs 6-7 are replicated under four feasible methods, Postbk, OptR, EstSS and EstBS, with trimming values of 0.05 and 0.10, respectively. The significant level is still 5% and the other settings are the same. Tables 9-11 in the Appendix report the results with T b = 0.85T, 0.90T, 0.95T respectively. The smallest number (before rounding off to three decimal digits) is highlighted in bold, considering the results with a trimming value 17

18 of 0.15 in the previous section. The tables show that the proposed EstSS method and IJR s OptR method are both insensitive to the choice of the trimming value, while it is a totally different story to the proposed EstBS method and PT s Postbk method. Due to the significant improvement of the detection rate, EstBS can yield a smaller RMSFER given the alternative choices, especially for a trimming value of Whilst given alternative trimming values, the forecasting performance of Postbk may be improved or significantly jeopardized. In conclusion, when the break date is close to the date of making forecasts and the break size is not small, given a small trimming value (e.g. 0.05) can improve the forecasting performance of the proposed bootstrap algorithm, while it does not work for the other methods in the previous section. 6 Conclusion This study develops the framework in Inoue, Jin, and Rossi (2017, Section 3.3) and proposes two feasible algorithms to select the optimal window size for forecasting. Both proposed algorithms, assuming the regressors can be weakly dependent, keep the asymptotic validity property in IJR s original method. The multi-step ahead forecasting is also allowed in both algorithms in this paper. The main difference of the proposed algorithms in this paper against IJR s algorithm is how to estimate the key parameter vector in the forecaster s loss function in the sense of selecting the estimation window size. The proposed bootstrap algorithm improves the forecast accuracy against IJR s original algorithm in almost all cases at the expense of increasing computation time significantly. Meanwhile, the proposed simple selection method can reduce the computation burden against both algorithms significantly, but may not be ideal for forecasting in most cases. The Monte-Carlo study shows that the proposed methods can improve the forecasting performance, comparing to the forecasts using the full sample, unless there is no or almost no break (the break size is less than 0.05 on the AR parameter or less than 0.20 on the marginal coefficient). In the presence of a structural break, the proposed bootstrap method performs best among the existing window selection methods in the literature if the break size is medium ( on the AR parameter or on the marginal 18

19 coefficient) and the break occurs at 0.75T or 0.85T, while the proposed simple selection method can be a reliable window selection method when the break occurs at 0.90T or 0.95T. Both methods do not beat the forecasts using only the post-break data if the break size is large (more than 0.20 on the AR parameter or 0.60 on the marginal coefficient) and the break date is far before the date when the forecast is made (before 0.85T ). In addition, when the break date is close to the forecast date (after 0.85T ), it is found that a smaller trimming value (e.g. 0.05) can improve the forecasting performance of the proposed bootstrap method, since it allows the estimated break date to be closer to the end of the sample. There are also some caveats in this paper. First, in practice it is almost impossible for forecasters to distinguish structural breaks and continuous breaks 15 in the underlying model. Hence, forecasters may have to take additional care on choosing the appropriate method to apply by considering data idiosyncrasy and additional tests. Second, the proposed methods in this study mainly aim to treat the structural breaks. Thus it is likely that the proposed methods are not suitable to deal with continuous breaks. Third, although the proposed methods are superior to the exist window selection methods in the cases when there only exists a mild recent break in the Monte-Carlo study, it does not guarantee that they will outperform the others when there exists more than one break. The extension of multiple breaks and continuous breaks shall be left for future research. Fourth, this study engages with a symmetric loss function which is most commonly used in macroeconomic forecasting, while it may be more appropriate to assume an asymmetric loss function when the forecaster only suffers from the semi-variance. Fifth, the resampling technique used in the proposed bootstrap method is only valid if the error terms are independently and identically distributed. When a static model is used for forecasting and there exists autocorrelation, moving block bootstrap (see Kunsch, 1989; Liu and Singh, 1992) shall be considered instead. 15 Also known as the time-varying parameter. 19

20 References ALTISSIMO, F., AND V. CORRADI (2003): Strong rules for detecting the number of breaks in a time series, Journal of Econometrics, 117(2), ANDREWS, D. W. (1993): Tests for parameter instability and structural change with unknown change point, Econometrica: Journal of the Econometric Society, pp BAI, J., AND P. PERRON (1998): Estimating and testing linear models with multiple structural changes, Econometrica, pp (2003): Computation and analysis of multiple structural change models, Journal of applied econometrics, 18(1), CAI, Z. (2007): Trending time-varying coefficient time series models with serially correlated errors, Journal of Econometrics, 136(1), CASTLE, J. L., M. P. CLEMENTS, AND D. F. HENDRY (2013): Forecasting by factors, by variables, by both or neither?, Journal of Econometrics, 177(2), CHOW, G. C. (1960): Tests of equality between sets of coefficients in two linear regressions, Econometrica: Journal of the Econometric Society, pp CLARK, T. E., AND M. W. MCCRACKEN (2009): Improving forecast accuracy by combining recursive and rolling forecasts, International Economic Review, 50(2), CLEMENTS, M. P., AND D. F. HENDRY (1996): Intercept corrections and structural change, Journal of Applied Econometrics, 11(5), (1998): Forecasting economic time series. Cambridge University Press. (2006): Forecasting with breaks, in Handbook of economic forecasting, ed. by G. Elliott, and A. Timmermann, vol. 1, pp Elsevier. DAVIDSON, R., AND J. G. MACKINNON (2006): Bootstrap methods in econometrics, in Palgrave handbook of econometrics, ed. by C. M. Terence, and K. Patterson, vol. 1, pp Palgrave Macmillan. 20

21 GIRAITIS, L., G. KAPETANIOS, AND S. PRICE (2013): Adaptive forecasting in the presence of recent and ongoing structural change, Journal of Econometrics, 177(2), GOYAL, A., AND I. WELCH (2003): Predicting the equity premium with dividend ratios, Management Science, 49(5), HENDRY, D. F. (2000): On detectable and non-detectable structural change, Structural change and economic dynamics, 11(1), INOUE, A., L. JIN, AND B. ROSSI (2017): Rolling window selection for out-of-sample forecasting with time-varying parameters, Journal of Econometrics, 196(1), INOUE, A., AND B. ROSSI (2012): Out-of-sample forecast tests robust to the choice of window size, Journal of Business & Economic Statistics, 30(3), KOOP, G., AND S. M. POTTER (2007): Estimation and forecasting in models with multiple breaks, The Review of Economic Studies, 74(3), KUNSCH, H. R. (1989): The jackknife and the bootstrap for general stationary observations, The Annals of Statistics, pp LIU, R. Y., AND K. SINGH (1992): Moving blocks jackknife and bootstrap capture weak dependence, Exploring the limits of bootstrap, 225, 248. MEESE, R., AND K. ROGOFF (1983): Empirical exchange rate models of the seventies: Do they fit out of sample?, Journal of international economics, 14(1), (1988): Was it real? The exchange rate-interest differential relation over the modern floating-rate period, the Journal of Finance, 43(4), MOLODTSOVA, T., AND D. H. PAPELL (2009): Out-of-sample exchange rate predictability with Taylor rule fundamentals, Journal of International Economics, 77(2), NANKERVIS, J. C., AND N. SAVIN (1996): The level and power of the bootstrap t test in the AR (1) model with trend, Journal of Business & Economic Statistics, 14(2), PESARAN, M. H., D. PETTENUZZO, AND A. TIMMERMANN (2006): Forecasting time series subject to multiple structural breaks, The Review of Economic Studies, 73(4),

22 PESARAN, M. H., A. PICK, AND M. PRANOVICH (2013): Optimal forecasts in the presence of structural breaks, Journal of Econometrics, 177(2), PESARAN, M. H., AND A. TIMMERMANN (2002): Market timing and return prediction under model instability, Journal of Empirical Finance, 9(5), (2005): Small sample properties of forecasts from autoregressive models under structural breaks, Journal of Econometrics, 129(1), (2007): Selection of estimation window in the presence of breaks, Journal of Econometrics, 137(1), ROBINSON, P. M. (1989): Nonparametric estimation of time-varying parameters, in Statistical Analysis and Forecasting of Economic Structural Change, pp Springer. ROSSI, B. (2013): Exchange rate predictability, Journal of Economic Literature, 51(4), SARNO, L., AND G. VALENTE (2009): Exchange rates and fundamentals: Footloose or evolving relationship?, Journal of the European Economic Association, 7(4), SCHINASI, G. J., AND P. A. V. B. SWAMY (1989): The out-of-sample forecasting performance of exchange rate models when coefficients are allowed to change, Journal of International Money and Finance, 8(3), STOCK, J. H., AND M. W. WATSON (1996): Evidence on structural instability in macroeconomic time series relations, Journal of Business & Economic Statistics, 14(1), (1999): Forecasting inflation, Journal of Monetary Economics, 44(2), (2003): Forecasting output and inflation: The role of asset prices, Journal of Economic Literature, 41(3), (2007): Why has US inflation become harder to forecast?, Journal of Money, Credit and banking, 39(1), (2016): Core inflation and trend inflation, Review of Economics and Statistics, 98(4),

23 Appendix Abbreviations AR Model: the autoregressive model. BP Test: referred as the Sup-F test in Bai and Perron (1998). CV: the cross-validation method proposed in PT. DGP: the data generating process. DR: the detection rate of the BP test. It is the percentage of detecting a break among 5000 replications. EDF: the empirical distribution function, from which the error terms are resampled to bootstrap the dependent variable. EstBS : the proposed feasible window selection algorithm based on bootstrap with estimated break date. EstSS : the proposed feasible simple selection algorithm with S = T ˆT b. IJR: referred as Inoue, Jin, and Rossi (2017). InfIJR : the infeasible window selection algorithm proposed in (Inoue, Jin, and Rossi, 2017, Section 4.2). MSFE: the mean squared forecast errors. OLS: the ordinary least squares estimation method. OptR : IJR s feasible window selection algorithm with the same setting as that labelled OptR1 in Inoue, Jin, and Rossi (2017, Section 4.2). Postbk : the post-break method in Pesaran and Timmermann (2007, Section 3.1). PT: referred as Pesaran and Timmermann (2007). PTCV : the cross-validation based feasible window selection method with unknown break date in Pesaran and Timmermann (2007, Section 3.3). RMSFER: the ratio of square root MSFEs, i.e, the square root MSFE under optimal window selection method against that using the full sample. TrueBS : the proposed bootstrap algorithm with known break date. TrueSS : the proposed simple selection algorithm with known break date and S = T T b. VAR: the vector autoregressive model. 23

24 Table 1: DGP Setups DGP a t b t Note Constant Parameters if t T b A Small Break on if t T b + 1 AR Parameter if t T b A Large Break on if t T b + 1 AR Parameter if t T b 1.5 if t T b + 1 A Small Break on Marginal Coefficient if t T b A Large Break on 2 if t T b + 1 Marginal Coefficient if t T b 1 A Break on AR θ 1 if t T b + 1 Parameter 1 if t T b A Break on Marginal θ 2 if t T b + 1 Coefficient Table 2: RMSFER, T = 100,h = 1 DGP T b Postbk PTCV OptR EstSS EstBS InfIJR TrueSS TrueBS DR