Data-Based Ranking of Realised Volatility Estimators

Transcription

1 Data-Based Ranking of Realised Volatility Estimators Andrew J. Patton University of Oxford 9 June 2007 Preliminary. Comments welcome. Abstract I propose a formal, data-based method for ranking realised variance (RV) estimators. In contrast, most rankings of RV estimators currently in the literature are either graphical in nature, most notably the volatility signature plot, or rely on asymptotic approximations of the mean-squared errors of the estimators, or on simulations. The proposed method relies on the existence of a volatility proxy that is unbiased for the variable of interest, and satis es a certain zero correlation condition. The zero correlation condition has some similarities with instrumental variables estimation. The volatility proxy must be unbiased but it does not need to be very precise; a simple and widely-available proxy for volatility is the daily squared return. From a small empirical application to IBM volatility estimation, I nd that the daily squared return is signi cantly out-performed by an RV estimator based on intra-daily data, while simple RV estimators based on 5-minute returns (either in calendar time or in tick time) were not signi cantly out-performed by any of 32 competing RV estimators. Keywords: realised variance, volatility forecasting, instrumental variables. J.E.L. codes: C52, C22, C53. I thank Roel Oomen, Neil Shephard and Kevin Sheppard for helpful comments and suggestions, without implicating them for any errors, and I thank Runquan Chen for excellent research assistance. Contact address: Department of Economics, University of Oxford, Manor Road, Oxford OX 3UQ, United Kingdom. andrew.patton@economics.ox.ac.uk. A complete draft of this paper will be available from

2 Introduction Most rankings of realised volatility (RV) estimators in the existing econometrics literature are either graphical in nature (as opposed to formal statistical tests), notably the volatility signature plot of Zhou (996) and Andersen, et al. (2000) for example, or rely on asymptotic approximations of the mean-squared errors of the estimators, or on simulations. In this paper I propose a formal data-based ranking method, which does not rely on continuous record asymptotic approximations or on simulating a realistic description of real data. The proposed ranking method relies on the existence of a volatility proxy that is unbiased for the latent target variable, and satis es an uncorrelatedness condition, described in detail below. This proxy must be unbiased but it may not need to be very precise. A simple and widely-available proxy for volatility is the daily squared return. The use of a consistent, data-based ranking of RV estimators has numerous advantages over rankings obtained via asymptotic theory or simulations. Compared with the continuous record asymptotic approximations, it allows one to examine the nite-sample performance of these estimators, which can di er widely from their asymptotic performance, as noted by Bandi and Russell (2005). Furthermore, much of the asymptotic theory for RV estimators in the presence of market microstructure noise relies on very speci c assumptions about the noise process. For example, Hansen and Lunde (2006a) consider noise that is iid and additive to the e cient log-price process, or that is mean zero and covariance stationary. Zhang, et al. (2005) also consider the iid noise case, as do Barndor -Nielsen, et al. (2006) and Bandi and Russell (2005). Finally, a data-based ranking method allows one to make comparisons that are di cult using existing theory: estimators based on trade prices versus quote prices; estimators de ned using calendar-time sampling versus tick-time sampling (see Oomen, 2006, for a comparison of these estimators); or estimators based on very di erent assumptions, such as the multi-scale sub-sampled realised variance estimator of Zhang (2006) versus the alternation estimator of Large (2005). A data-based ranking has the obvious advantage over a simulation-based approach in that the latter requires a complete speci cation of the data generating process, and results obtained under one speci cation/parameterisation need not necessarily hold more generally. The data-based approach presented here allows one to answer the question of immediate interest to users of RV estimators: which estimator works best on my asset return series, for my sample period? I provide

3 conditions su cient for a consistent estimate of the average di erence in distance between an RV estimator and the latent target variable to be obtained, which then allows one to use standard tests for forecast comparisons, such as those of Diebold and Mariano (995), White (2000) and Hansen (2005). These tests rely on the usual large sample (T! ) asymptotics, but do not rely on continuous record (m!, where m is the number of intra-period observations) asymptotics. This allows for formal comparisons of estimators that di er only in their sampling frequency.. Notation t is the latent target variable. I assume that t is F t -measurable, though it is not observable to the econometrician. For the remainder of the paper I assume that t is a scalar; I discuss the extension to vector target variables in the conclusion. X it, i = ; 2; :::; n are the realised volatility estimators to be ranked. Often these will be the same estimator applied to data sampled at di erent frequencies, for example -minute returns vs. 30-minute returns. They could also be estimators based on di erent measures of the price: trades vs. mid-quotes, for example, or di erent sampling schemes, such as calendar-time sampling vs. tick-time sampling. In order to rank the competing estimators we need some measure of distance from the estimator, X it ; to the target variable, t : In rankings of estimators based on asymptotic approximations this distance is usually the mean-squared error (MSE). When the two estimators are both consistent this reduces to comparing the asymptotic variances of the two estimators. Barndor -Nielsen, et al. (2006) provide a detailed study of the asymptotic accuracy of a wide variety of kernel-based realised volatility estimators, Hansen and Lunde (2006a) study the asymptotic MSE of a variety of estimators under di erent assumptions on the microstructure noise, while Bandi and Russell (2005) study the nite-sample MSE of some kernel-based realised volatility estimators under the assumption of iid microstructure noise. The extensive simulation study of Gatheral and Oomen (2007) also uses MSE to measure the distance between the estimator and the target variable. I will consider ranking RV estimators using the average distance between the estimator and the target variable, using the general class of pseudo-distance measures proposed in Patton (2006): 2

4 E [L ( t ; X it )] vs. E [L ( t ; X jt )] () where L (; X) = C ~ (X) C ~ () + C (X) ( X) (2) Z and C ~ (z) C (z) dz with C being some decreasing, twice-di erentiable function. When both t and X it are strictly positive a:s:, the parametric family loss functions obtained from: C (z; b) = and so ~ C (z; b) = 8 >< >: 8 >< >: (b + ) z b+ ; for b =2 f ; 2g log z; for b = z ; for b = 2 (b + ) (b + 2) z b+2 ; for b =2 f ; 2g z z log z; for b = log z; for b = 2 has some attractive properties, see Patton (2006). This class nests MSE as a special case (b = 0; and so C (z) = z) and the popular QLIKE loss function (b = 2), up to location and scale constants in both cases. More generally, the shape parameter b a ects the penalty applied to over-estimation compared with under-estimation. This class is well-de ned when both and X are almost surely strictly positive, which is a reasonable assumption in applications involving realised variance. When either t or X it can be negative a di erent parametric family of pseudo-distance measures will be required. In all cases, the MSE distance measure can of course be employed. Our interest is in measuring the average distance between the estimator and the latent target variable. I will obtain a consistent estimator of this quantity by employing a proxy or instrument for t ; denoted Y t : The proxy must be observable by the econometrician, for the ranking to be data-based, and must satisfy certain unbiasedness and zero correlation conditions. Deriving these conditions and nding a proxy that satis es them is the main technical challenge in this paper. The method I propose below relies on the presence of a potentially noisy but conditionally unbiased proxy for the latent target variable. For many assets the squared daily return can reasonably 3

5 be assumed to be conditionally unbiased: the expected return is generally negligible at the daily frequency, and the impact of market microstructure e ects is often also negligible in daily returns. It should be noted, however, that the presence of jumps in the data generating process will a ect the inference obtained using the daily squared return as a proxy: in this case we can compare the RV estimators in terms of their ability to estimate quadratic variation, which is the integrated variance plus the sum of squared jumps in many cases, see Barndor -Nielsen and Shephard (2006) for example, but not in terms of their ability to estimate the integrated variance alone. If an estimator of the integrated variance that is conditionally unbiased even in the presence of jumps is available, however, the methods presented below apply directly. 2 Relation to the ranking of volatility forecasts Ranking volatility forecasts, as opposed to estimators, has received a lot of attention in the econometrics literature, see Poon and Granger (2003) and Hansen and Lunde (2005) for two recent and comprehensive studies, and this is the natural starting point for considering the ranking realised volatility estimators. Hansen and Lunde (2006b) and Patton (2006) show that if: E [Y t jf t ] = t (i.e., the proxy is conditionally unbiased for t ) then for any pseudo-distance measure in the class in equation (2) rankings based on the proxy are (T asymptotically) equivalent to rankings based on the true unobservable target variable, assuming that the expectations exist. That is, E [L ( t ; X t )] Q E [L ( t ; X 2t )], E [L (Y t ; X t )] Q E [L (Y t ; X 2t )] (3) However, this result does not go through when (X t ; X 2t ) are RV estimators rather than a volatility forecasts. To see this, consider a mean-value expansion of the pseudo-distance measure L (Y t ; X t ) given in equation (2) around ( t ; X t ) L (Y t ; X t ) = L ( t ; X t ) ( t; X t ) (Y t t ) 2 L t ; X 2 (Y t t ) 2 = L ( t ; X t ) + (C (X t ) C ( t )) (Y t t ) 2 C0 t (Y t t ) 2 where t = t t + ( t ) Y t ; t 2 [0; ] then E t [L (Y t ; X t )] = E t [L ( t ; X t )] + E t [(C (X t ) C ( t )) (Y t t )] (4) 2 E t hc 0 t (Y t t ) 2i 4

6 The third term in equation (4) does not depend on X t ; and so will not a ect the ranking of (X t ; X 2t ) : For the ranking obtained using Y t to be the same as that obtained using t we need to show that the second term equals zero: E t [(C (X t ) C ( t )) (Y t t )] = 0 In the standard case, X t is a volatility forecast and t is the conditional variance, and so is F t - measurable, which allows: E t [(C (X t ) C ( t )) (Y t t )] = (C (X t ) C ( t )) E t [Y t t ] = 0 by the conditional unbiasedness of Y t. However, when X t is a realised volatility estimator and t is the integrated variance or quadratic variation we have (X t ; t ) 2 F t but (X t ; t ) =2 F t ; and so we cannot take the rst term above out of the expectation. In short, the fact that the realised variance estimator of the target variable for day t is only available at the end of day t rules out the direct application of established results for volatility forecast comparison. If we could assume that Corr t [C (X it ) C ( t ) ; Y t t ] = 0 8 i in addition to E [Y t jf t ; t ] = t ; then we would have E t [(C (X it ) C ( t )) (Y t t )] = E t [(C (X it ) C ( t ))] E t [E [Y t jf t ; t ] t ] = 0 But it is not likely that Corr t [C (X it ) C ( t ) ; Y t t ] = 0 for all combinations of RV estimators and volatility proxies. For example, if X it = Y t and L = MSE, then C (z) = z so Corr t [C (X it ) C ( t ) ; Y t t ] = Corr t [ t Y t ; Y t t ] = Thus this correlation will in fact equal -! More generally, we would expect this correlation to be non-zero. It is the correlation between the error in Y t and something similar to the generalised forecast error, see Granger (999) or Patton and Timmermann (2003), of X t : If the proxy, Y t ; and the RV estimator, X t ; are both using the same or similar information sets then their errors will generally be correlated and this zero correlation restriction will not hold. This reveals the similarity of this problem to instrumental variables estimation: ignoring the correlation between the error in the RV estimator and the error in the proxy leads to invalid inference. 5

7 3 Data-based ranking of RV estimators I present results under two broad sets of assumptions: the rst allows for general behaviour in the target variable, t ; but restricts the behaviour of the RV estimators, X it : The second set of assumptions allows for general behavior of the RV estimators, at the cost of imposing some restrictions on the behaviour of the target variable. We present both sets of results as in di erent applications one set of assumptions may be more palatable than the other. 3. Rankings based on assumptions about the RV estimator bias This section presents results for data-based ranking of RV estimators that hold when we can assume that the time series behaviour of the bias in the RV estimators satis es restriction given below. Assumption T: t is a mean stationary process. Assumption P: Y t is a mean stationary process with E [Y t ] = E [ t ] : Assumption P2: Y t 2 F t : Assumption R: E [X it j t ; F t ] = t + c i k t 8 i, where k is known, and max i jc i j < : The rst two of these assumptions are standard, with only unconditional unbiasedness of Y t required (rather than conditional unbiasedness). Assumption P2 requires that the proxy is measurable at time t, which is non-standard. We would usually consider a proxy for t as being something measured on day t, such as the squared returns from day t. Assumption P2 suggests instead to use the rst lag of the daily squared return, or longer lags, or perhaps combinations of lags. (We will consider optimal choices of proxies below.) The result below shows that using lagged squared returns can be useful in obtaining a data-based ranking of RV estimators. Assumption R is the key assumption for this result. It requires that the bias in the RV estimators be proportional to some power of the target variable, with a common power but potentially di erent proportionality constants. This nests the interesting special cases where all RV estimators are unbiased (c i = 0 8 i). Also nested is the case where all RV estimators have some biases that are constant through time (k = 0) but which can di er across estimators (c i 6= c j ). This is relevant, for example, if the observed price is equal to the true e cient price plus some covariance stationary or iid noise, see Hansen and Lunde (2006a, Theorem and Lemma 2). Finally, assumption R also allows the biases in the RV estimators to be proportional to some power of t ; for example 2 t : This might be of interest as in many cases asymptotic variance of many RV estimators is re- 6

8 lated to integrated quarticity, which is in turn related to the square of integrated variance, see Barndor -Nielsen and Shephard (2004) for example. Alternatively, if the noise in the observed price is proportional to the observed price, and the RV estimator is based on tick-time sampling, then the bias is proportional to t, see Hansen and Lunde (2006a, Example ). Proposition Let assumptions T, P, P2 and R hold. positive a.s., Then if ( t ; Y t ; X t ; X 2t ) 0 is strictly E [L ( t ; X t ; b)] E [L ( t ; X 2t ; b)] = E [L (Y t ; X t ; b)] E [L (Y t ; X 2t ; b)], if b = k = 0 E [L (Y t ; X t ; b)] E [L (Y t ; X 2t ; b)], if b = k 6= 0 for any two RV estimators, X t and X 2t ; where k is from assumption R: If ( t ; Y t ; X t ; X 2t ) 0 can be negative this result holds for k = b = 0. All proofs are presented in the Appendix. This proposition shows that if we can make some assumption about the time series behaviour of the bias in the competing RV estimators, then there exists a unique pseudo-distance measure from equation (2) that yields a feasible data-based ranking of RV estimators. For example, if the biases in the RV estimators are constant through time, then we can rank the RV estimators using MSE (b = 0). If the biases in the RV estimators are proportional to 2 t, then the QLIKE pseudodistance measure ( b = 2) should be used to rank the RV estimators. Note, importantly, that we do not need to assume anything about the behaviour of the biases as m (the number of intra-daily observations) varies. Di erent choices of m correspond to di erent RV estimators in this framework and no relation between the competing estimators is imposed. 3.2 Rankings based on a random walk assumption The result from the previous section relied on a rather speci c assumption about the time series properties of the biases in the competing RV estimators. In this section I do away with such assumptions by imposing some structure on the time series dynamics of the target variable, t : Numerous papers on the conditional variance (see Bollerslev, et al., 994, Engle and Patton, 200, and Andersen, et al., 2005 for example), or integrated variance (see Andersen, et al., 2004 and 2005) have reported that these quantities are very persistent, close to being random walks. The popular RiskMetrics model, for example, is based on a unit root assumption for the conditional 7

9 variance. Wright (999) provides thorough evidence against the presence of a unit root in daily conditional variance for many stocks, however, despite this, it has proven to be a good approximation in many cases. Given this, consider the following assumption: Assumption T2: t = t + t, with E [ t jf t ] = 0: In the proof of the following proposition I need to strengthen the unconditional unbiasedness assumption in P to the standard conditional unbiasedness assumption. Let us denote the conditionally unbiased proxy as ~ t ; rather than Y t ; as below I will consider taking linear combinations of unbiased proxies to improve the power of tests in nite samples. Assumption P : ~ t = t + t, with E [ t jf t ; t ] = 0. For the proposition below I again consider using a proxy for t that is not measured on day t; but instead of considering lags of ~ t it turns out to be best to consider leads of ~ t : The reason for this becomes clear in the proof. Assumption P2 : Y t = P J i=! i ~ t+i, where J < ;! i 0 8 i and P J i=! i = : Proposition 2 Let assumptions T2, P and P2 hold. If ( t ; Y t ; X t ; X 2t ) 0 is strictly positive a.s., then E [L ( t ; X t ; b)] E [L ( t ; X 2t ; b)] = E [L (Y t ; X t ; b)] E [L (Y t ; X 2t ; b)] for any two RV estimators, X t and X 2t ; and any b: If ( t ; Y t ; X t ; X 2t ) 0 can be negative this result holds for MSE loss (b = 0): In some ways the above result is substantially more general than that in Proposition. The assumption that t follows a random walk allows us to leave the bias, if any, in the RV estimators completely unspeci ed: it can be constant, time-varying, a function of di erent powers of t ; or a function of other variables altogether. Furthermore, Proposition 2 shows that any pseudo-distance measure from the class in equation (2) may be used, according to the preferences of the user of the RV estimator. Note that for a formal RV comparison test to be implemented we will need certain moment conditions to be satis ed and this may restrict the choice of pseudo-distance measure. An alternative motivation for the empirical approach suggested by Proposition 2 is based on the asymptotics of rolling window volatility estimators given in Foster and Nelson (996), and used in Fleming, et al. (200), amongst many other applications. Foster and Nelson show that, under some conditions, estimators such as those covered in assumption P2 converge to true (spot) variance as 8

10 the length of the period H = =m (one day, in our application in Section 5) goes to zero and as the number of intra-period observations, m; goes to in nity. Before moving on, it is worth considering how the above proposition changes when the target variable is only close to a random walk. To that end, consider the following modi cation of the random walk assumption: Assumption T2 : t = + ( t ) + t, with E [ t jf t ] = 0; and ;, where is a small positive constant. Under this weaker assumption on the time series dynamics of the target variable I obtain the following result. For simplicity I restrict the proxy to be a simple lead of ~ t : Proposition 3 Let assumptions T2 and P hold, and set Y t = ~ t+ : (i) Then E [L ( t ; X t )] E [L ( t ; X 2t )] = E [L (Y t ; X t )] E [L (Y t ; X 2t )] E [ t (C (X t ) C (X 2t ))] + 2 E [C (X t ) C (X 2t )] for any two RV estimators, X t and X 2t : (ii) If we further assume R and let c 2 we have h E [ t (C (X t ; b) C (X 2t ; b))] = E h c ; then if ( t ; Y t ; X t ; X 2t ) 0 is strictly positive a.s. E k+ t k++b t i, for b = 0 i, for b 6= 0 and 2 E [C (X t ; b) C ( t ; b)] = h 2 E h 2 E k t i, for b = 0 i, for b 6= 0 k+b t When ( t ; Y t ; X t ; X 2t ) 0 can be negative the result for b = 0 holds. The rst part of Proposition 3 shows explicitly the extra terms that appear when the target variable follows an AR() rather than a random walk: The second part of the proposition provides some idea of the magnitudes of these terms as a function of ; which measures how close to a random walk the target variable is, and ; which measures the di erence in the proportionality constants in the biases of the two RV estimators. Empirically it is widely found that is positive 9

11 but small. Thus the third term in part (i) is O 2 and may be negligible. The second term is O () ; and becomes O () when we impose some structure on the biases in the RV estimators. If is small, and we think is small, then this term will also be negligible. Proposition 3 provides some reassurance of the empirical usefulness of the ranking method suggested by Proposition 2: if the target variable is close to a random walk, and/or the RV estimators being compared have similar biases, then ranking RV estimators by using a lead of a conditionally unbiased proxy for t in conjunction with a pseudo-distance measure from equation (2) will yield the same ranking as if t was directly observable. Proposition 2 above suggests the use of a convex combination of leads of ~ t ; but gives no guidance on how many leads, J; to consider or on the appropriate weights to apply to each lead individually. While the weighting function could theoretically have J free parameters (it must sum to one, pinning down the J th weight) let us simplify the problem and consider only equallyweighted proxies. In this case, the problem reduces to choosing J, the number of leads to combine: Proposition 4 Let P, P2 and T2 hold. Then, imposing! i = J 8 i = ; 2; :::; J; the variance of the error in the proxy for a given value of J is V [Y t t ] = J 2 + (J + ) (2J + ) 2 + 6J + J The number of leads that minimises the variance of the measurement error in Y t is given by s J + 6k + 6 p k = 2 where 2 = 2 and Corr [ t ; t ] When we constrain J to be an integer between and 0000, the optimal values are: J ,

12 These results reveal that the optimal integer values of J do not vary greatly with ; though they do change with : When <, it is intuitively clear that we should use only one lead of ~ t ; as in that case ~ t is a relatively accurate estimator of t and the gains from smoothing are low. When ; there is potentially some bene t to smoothing the proxy across a range of leads of ~ t : Only for very large values of do we average across more than a few leads of ~ t : It should be noted that the above result for the optimal value for J is very sensitive to the random walk assumption for t : if t is actually slowly mean-reverting then using leads of 00 or more periods will yield misleading results. In practice, it may be best to limit the value of J to be no more than 5 or 0 for daily data, depending on the estimated persistence in the latent target variable. 3.3 Rankings based on an AR(p) assumption I now present the most generally applicable result of this paper, which allows the latent target variable to follow any stationary AR(p) process, subject to the rst-order AR coe cient being di erent from zero. The work of Meddahi (2003) and Barndor -Nielsen and Shephard (2002) shows that integrated variance follows an ARMA(p,q) model for a wide variety of stochastic volatility models for the instantaneous volatility, motivating such this generalisation of the result for random walks in Proposition 2. Whilst allowing for a general ARMA model is possible, I focus on the AR case both for the ease with which this case can be handled, and for the fact that allowing for an ARMA model would theoretically involve estimating an in nite number, asymptotically, of autocovariance-type quantities, which is likely to have unsatisfactory nite sample properties. In the next section I show via simulations that an AR(p) approximation to the process for daily integrated variance is barely distinguishable from the ARMA approximation, for one common stochastic volatility model. This result requires a consistent estimator of the parameters of the AR(p) model for the latent target variable: we present such an estimator in the following lemma. This result follows from Baillie and Chung (200), although I focus on an AR(p) model and autocovariances, rather than ARMA models and autocorrelations, which simpli es the estimator and allows for a closed-form expression. Assumption T2 : t = 0 + P p i= i t i + t, with E [ t jf t ] = 0; 6= 0 and ; :::; p 0 such that t is covariance stationary.

13 Lemma Let assumptions P and T2 hold and de ne 0 ; ; 2 ; :::; p ^ p ^ p ^ 0 ^ p+ ^ p ^ ^A (k) T and let and B (k) T where ^ j h ^ T T 0 ^ 2p+k ^ 2p+k 2 ^ p+k ^ T ^ p+ ^ p+2 ^ 2p+k i 0, k 0 T j TX t= TX t=j+ ~ t ^ T = ^A(k)0 T W T ~t ^A (k) T ^ ~t j ^ ^A(k)0 T W T ^B (k) T (5) where W T! p W; a positive de nite matrix of constants. Then p T ^T! D N (0; V ) as T! : When the order of the autoregression is greater than one, I also require the following assumption: Assumption R2: X it is independent of t j for all j > 0: This assumption is mild given that almost all RV estimators (Barndor -Nielsen, et al., 2005, and Owen and Steigerwald, 2007, being two exceptions) are based solely on a single day of intra-day returns. Proposition 5 Let assumptions T2 and P hold, and let R2 hold if p > : (i) Then h i E [L ( t ; X t ; b) L ( t ; X 2t )] = E L ~t+ ; X t L ~t+ ; X 2t + E h(c (X t ) C (X 2t )) ~ i t+ 0 E [C (X t ) C (X 2t )] px j E h(c (X t ) C (X 2t )) ~ i t+ j j=2 2

14 for any two RV estimators, X t and X 2t : If ( t ; Y t ; X t ; X 2t ) 0 is strictly positive a.s. this result holds for any b in the pseudo-distance measures presented in equation (2). If ( t ; Y t ; X t ; X 2t ) 0 may be negative this result holds for the squared distance measure, MSE (b = 0) : (ii) Under standard regularity conditions, we have T TX t= + ^;T ^ ;T ^ 0;T ^ ;T T px j=2 n L ~t+ ; X t ; b T o L ~t+ ; X 2t TX n(c (X t ) C (X 2t )) ~ o t+ t= TX fc (X t ) C (X 2t )g t= ^ j;t ^ ;T T + j TX n(c (X t ) C (X 2t )) ~ o t+ j t=j! p E [L ( t ; X t ; b) L ( t ; X 2t )], as T! where ^ i;t ; i = 0; ; 2; :::; p are estimated using Lemma. Proposition 5 relaxes the assumption of a random walk, at the cost of introducing a bias term to the expected loss computed using the proxy, even when the error in the proxy is uncorrelated with the error in the RV estimators (e.g., when a lead of ~ t is used). This bias term, however, can be consistently estimated under the assumption that the target variable follows a stationary, non-trivial AR(p) process. The cost of the added exibility in allowing for a general AR(p) process for the target variable is the added estimation error induced by having to estimate the AR(p) parameters, and having to estimate the terms E h(c (X t ) C (X 2t )) ~ i t+ j, j = 0; 2; 3; ::; p: This estimation error will lead to reduced power to distinguish between competing RV estimators than would otherwise be the case. 4 Simulation study To examine the nite-sample performance of the proposed tests I present the results of a small simulation study. I use a standard log-normal stochastic volatility model with a leverage e ect, with the same parameters as in Gonçalves and Meddahi (2005): d log P (t) = 0:034d (t) + (t) 0:576dW (t) + p 0:576 2 dw 2 (t) (6) d log 2 (t) = 0:036 0: log 2 (t) d (t) + 0:48dW (t) 3

15 In simulating from these processes I use a simple Euler discretization scheme, with the step size calibrated to one second (i.e., with 23,400 steps per simulated trade day, which assumed to be 6.5 hours in length). To gain some insight into the impact of microstructure e ects, I also consider a simple iid error term for the observed log-price: where log P (t j ) = log P (t j ) + (t j ) (t j ) s iid N 0; V [r t ] f0; 0:2g where r t is the open-to-close return on day t: That is, I either set the noise to zero, or I set the variance of the noise to be such that the proportion of the variance of the 5-minute return (5/390 of a trade day) that is attributable to microstructure noise is 20%. The expression above is from Aït-Sahalia, et al. (2005), while the proportion of 20% is around the middle value considered in the simulation study of Huang and Tauchen (2005). I generated 250 independent sequences of 500 trade days. Existing results on the ARMA processes for integrated variance (IV) implied by various continuoustime stochastic volatility models, see Barndor -Nielsen and Shephard (2002) and Meddahi (2003), are derived under a no leverage assumption, whereas our simulated process exhibits a leverage e ect. This means that the goodness-of- t of an ARMA model to simulated IV needs to be veri ed. The results for a selection of ARMA models are presented in Table. The rst panel of Table shows the average R 2, across simulations, of a random walk model and an AR(), AR(2), AR(5), ARMA(,) and ARMA(2,2). The rst point to note is that all of these models t daily IV very well: The average R 2 s ranged from for the random walk model, to for the ARMA(2,2) model. The improvement in average R 2 of the AR() model over the random walk model is just , while the improvement of the most general model estimated, the ARMA(2,2), over the random walk model is This suggests that although the random walk model is mis-speci ed for daily IV, it is a reasonable approximation. The second panel of Table examines in more detail the t of the AR() model for daily IV across simulations. This table shows that the constant in this model is small, at 0.0, but is signi cantly di erent from zero: in all 250 simulations the estimate of this parameter is positive. The AR() coe cient is near one, averaging 0.98, but the 90 th quantile of the cross-simulation 4

16 distribution of these parameters is 0.99, and this parameter is strictly less than unity in all but three simulations. [ INSERT TABLE ABOUT HERE ] To investigate the nite-sample size and power properties of the tests proposed, I designed the following experiment. For simplicity I focused solely on pair-wise comparisons via a Diebold- Mariano (995) test. I set the each RV estimator equal to the true IV plus some iid noise: X it = IV t + it, i = ; 2 it s iid N 0; 2 i E [ t 2s ] = 0 8 t; s To study the size of the tests, I rst set 2 = 2 2 = 0:0 V [IV t] : In this case both estimators are noisy estimators of IV, and are equally accurate. Furthermore, in this example both estimators are conditionally unbiased, which means that the satisfy the conditions of Proposition and thus ranking by MSE, using a lead or a lag of the volatility proxy, can be done without any need to bias-correct. For comparison, I also show the results when a bias correction based on an AR() assumption is made, following Proposition 5. If the AR() is a good approximation to the IV process, then this test should also have reasonable nite-sample properties, though we would expect it to have lower power due to the additional error in the test statistic coming from the estimation of the AR parameters. In estimating the AR() parameters I follow Lemma, using either zero or three over-identifying moments. To study the power of the tests, I x 2 and let 2 2 =V [IV t] = 0:5; 0:20; 0:40; 0:75: The rejection frequencies under each scenario are presented in Table 2, using both MSE and QLIKE loss. I consider eleven tests in total. The rst two tests are the infeasible Diebold-Mariano (995) and White (2000) reality check tests that one would conduct if the IV were truly observable. The power of these tests represents an upper bound on what we can expect from the feasible tests. I consider the tests under both the random walk assumption (using Proposition 2) and the AR() assumption (using Proposition 5 and Lemma ), for three di erent proxies: daily squared returns, 30-minute RV and the true IV. The daily squared returns and the 30-minute RV are those obtained from the noisy returns data ( 2 > 0) to examine the impact of microstructure noise on nite-sample 5

17 size, while the true IV is used to see the limiting case of a proxy with no error being put through these tests. The rst row of each panel of Table 2 corresponds to the case when the null hypothesis is satis ed, and thus we expect these gures to be close to 0.05, the nominal size of the tests. For both MSE and QLIKE we see that the nite-sample size is reasonable, with rejection frequencies reasonably close to Most tests appear to be under-sized, meaning that they are conservative tests of the null. The results for the power of the tests are as expected: the power of the new tests are worse than would be obtained if IV were observable; the power is worse when a noisier instrument is used (daily squared returns versus 30-minute RV versus true IV); and the power of the test based on the AR() assumption is worse than that based on the random walk assumption. In this particular design, the power of the tests based on the MSE loss function is greater than those based on QLIKE loss, though this is likely due to the additive nature of the noise in the design of the RV estimators being compared. The test based on an AR() model estimated using three over-identifying moment conditions generally has slightly better power than the corresponding test based on an estimator with no over-identifying conditions, but the di erence in this particular simulation are not large. In addition to testing the size and power of tests based on the results of the previous section, it is also of interest to study the simple rankings of estimators obtained using the methods proposed in this paper. Table 3 reports the proportion of simulations for which the ranking of the two competing estimators obtained using one of the methods proposed above is the same as the ranking that would be obtained using the true IV. In the rst row of each panel the two RV estimators are equally accurate, and so we expect the proportion of correct rankings to lie anywhere between zero and one. As we move down the rows in each column, we expect the proportion of correct rankings to increase towards unity, with both methods correctly identifying the better estimator. This is indeed what is found. The proportion of correct rankings increases as the di erence in accuracy of the two estimators increases; decreases as the noise in the volatility proxy increases; and weakly decreases as we move from the random walk to the AR() assumption. No di erences are found between rankings based on an AR() model estimated using three over-identifying moment conditions versus no over-identifying conditions. Overall, I conclude that the proposed tests have reasonable nite-sample size properties. There is of course a loss in power when using a noisy proxy rather than the true, unobservable, target 6

18 variable, however this is the nature of the problem under analysis. Table 3 shows that although there is a loss in power, the rankings obtained using the proposed methods generally consistent with the rankings based on the unobservable target variable. [ INSERT TABLES 2 AND 3 ABOUT HERE ] 5 Empirical application In this section I consider the problem of estimating the quadratic variation of the daily return on IBM. I use data on trade prices from the TAQ database over the period from January 993 to May 998, yielding 364 daily returns. This sample period was used in the Andersen, et al. (200) study of realised variance of equity returns. I consider two types of simple realised volatility estimators: the rst using calendar time sampling, and the second using tick time sampling. The existing literature provides little guidance on calendar-time versus tick-time sampling. Oomen (2006) is a notable exception to this, and via a parametric pure jump process for transaction prices he nds that tick-time sampling has lower MSE than calendar-time sampling. For the rst type of RV estimator, I use last price interpolation of the trade price series to create a series of 30-second prices for the open hours of the New York Stock exchange (9:30am to 4:00pm), denoted fr t;j ; j = ; 2; :::; 780g T t=. From this series I compute: RV (m) t = where r t;j;m mx rt;j;m 2 j= 2hX r t;2h(j i= h = 390 m )+i r t;j;m is the h-minute return, computed from the original 30-second return series. I consider all values for h that are multiples of one-half (so that I can evenly aggregate these from the original 30-second return series) and that divide evenly into 390, the number of minutes in an NYSE trade day. This yields 7 sampling frequencies: h = 0.5,, 2, 3, 5, 6, 0, 3, 5, 26, 30, 39, 65, 78, 30, 95, and 390 minutes, the nal value for h corresponding to simply using the open-to-close return. The corresponding values for m are 780, 390, 95, 30, 78, 65, 39, 30, 26, 3, 0, 6, 5, 3, 2,. The tick time RV estimators, denoted RV tick (m) t ; are constructed by sampling the trade 7

19 prices so that I have m return observations per day: for example, for m = 2 I sample the rst trade, last trade and the trade closest to the middle trade (in terms of number of trades, not in terms of clock time) and use these to compute the 2 return observations. I use the same set of values for m that are used for the standard RV estimators. The total number of RV estimators considered is 33: 7 standard RV estimators and 6 RVtick estimators (for m = ; the calendar-time and tick-time estimators are identical and so I drop the last RVtick estimator). Figure presents the volatility signature plot and a plot of the standard deviation of these estimators. [ INSERT FIGURE ABOUT HERE ] Tables 4 to 6 present the rst empirical contribution of this paper. These tables present the average distance, under MSE and QLIKE, between the 33 RV estimators and the latent target variable relative to the average distance between the squared open-to-close ( daily ) return and the latent target variable. A negative value indicates that the daily squared return was out-performed, while a positive value indicates the opposite. In all cases the proxy is the squared daily return. The three tables show the estimated average distances under three assumptions on the DGP: Table 4 is based on an AR() assumption for the latent target variable (Assumption T2 ) and computes the average distance di erentials using the consistent estimator presented in Proposition 5, using three extra moments to estimate the AR() parameters. Table 5 is based on a random walk assumption for the latent target variable (Assumption T2) and computes the average distance di erentials using the estimator presented in Proposition 2. Table 6 is based on the incorrect assumption that the measurement error in the proxy is uncorrelated with the errors in the RV estimators, and uses the contemporaneous value of the proxy rather than a one-period lead as in the former two cases. The results from these three tables are depicted in Figure 2. [ INSERT FIGURE 2 ABOUT HERE ] Under the AR() assumption for the target variable, the best two estimators according to MSE and QLIKE are the RV estimators based on 30-minute and 5-minute returns. The worst two estimators under MSE are the RV-tick estimators based on m = 780 and m = 390 trades (corresponding to h = 0:5 minute and h = minute sampling on average). The worst estimators under QLIKE are daily squared returns and RV based on 95-minute returns. 8

20 Under the random walk assumption for the target variable, the best RV estimators according to MSE and QLIKE are the RV estimators based on 30-minute and m = 78 trade (2-minute) returns respectively. The second-best estimators are those based on h =5-minute returns and m =30 trades. The worst estimators are the same as those under the AR() assumption. To illustrate the distortions caused by ignoring the correlation between the error in the proxy and the RV estimators, I also present the ranking obtained under the naïve assumption that this correlation is zero. The resulting ranking suggests that daily squared returns are the best estimator of daily quadratic variation amongst all RV and RV-tick estimators, which is driven purely by the fact that the correlation between the measurement errors goes to unity for the standard RV estimator when m = ; far from the assumption that it is zero. [ INSERT TABLES 4, 5 AND 6 ABOUT HERE ] In Table 7 I present the results of formal comparisons of the 33 RV estimators considered in this empirical application. To do this I implement the reality check of White (2000), and a re nement of this test proposed by Hansen (2005). The reality check is a means of testing the null: H 0 : E [L ( t ; X 0t )] E [L ( t ; X it )], for all i = ; 2; :::; K vs. H a : E [L ( t ; X 0t )] > E [L ( t ; X it )] for some i where X 0t is some benchmark RV estimator. That is, we test whether the benchmark RV estimator generates losses that are weakly smaller in expectation than any competing RV estimator. The null hypothesis contains K weak inequalities, and the critical values for this test can be easily obtained using a bootstrap procedure. I use the stationary bootstrap of Politis and Romano (994) with an average block length of 20 days. Using the bootstrap also simpli es accounting for the impact of the estimator of the AR() parameters on the asymptotic distribution of the test statistic, see Corollary 2.7 of White (2000). Hansen s (2005) re nement of the White s reality check involves a form of trimming to limit the impact of very poor estimators and studentising the test statistic; both of these re nements should lead to improved power to reject the null. I consider four benchmark estimators of daily quadratic variation: the daily squared return, a standard RV estimator based on 5-minute returns, an RV-tick estimator based on 78 trades per day (5-minute returns on average) and the estimated volatility obtained from a Normal GARCH(,) model applied to the open-to-close return series. I present results under both the AR() assumption 9

21 and the random walk assumption, which allows for some insight into the impact of estimation error in the AR() parameter estimate on the power of the test. Finally, I consider two proxies: the squared daily return, and a standard RV estimator based on 2intra-daily returns (h = 95) : This latter estimator is approximately unbiased and is about 30% less volatile than daily squared returns, according to the plots in Figure, and so may lead to more powerful inference. [ INSERT TABLE 7 ABOUT HERE ] Table 7 reveals that the daily squared return can be rejected as being signi cantly beaten by some alternative RV estimator in many cases: for all but one case under the random walk assumption it is rejected, as well as under the AR() assumption and the QLIKE pseudo-distance measure. When using the AR() assumption the daily squared return is mostly not rejected either under MSE or QLIKE, perhaps indicating a loss of power for this application. The standard RV estimator based on 5-minute returns is rejected only twice, suggesting that for this sample period the competing RV or RV-tick estimators were not generally signi cantly better than this simple estimator. Similarly, the RV-tick estimate based on 78 returns per day is not rejected by any test. This nding provides some support for the rule-of-thumb that a simple 5-minute RV estimator (either in calendar time or in tick time) works well in practice. For comparison, I also considered the estimated volatility from a simple GARCH(,) model (see Engle, 982, and Bollerslev, 986) as a measure of daily quadratic variation. This estimator is almost certainly biased relative to RV estimators based on the current day s information, as the GARCH estimate for day t uses only data up until day t ; however the GARCH estimates will be smoother than the RV estimates, perhaps allowing for some bias-variance trade-o. This is indeed what is found: in no case is the GARCH estimator rejected in favour of one of the competing RV or RV-tick estimators. Overall, this small empirical application suggests that it is di cult to beat simple estimates of daily quadratic variation. Daily squared returns are signi cantly beaten by estimators that use intra-daily data, but a standard RV estimator based on 5-minute returns (computed either in calendar time or in tick time), and even an estimate obtained from a GARCH(,) model are not signi cantly out-performed by estimators based on higher frequency data. It remains to be seen whether this conclusion holds for other assets in other sample periods. 20

22 6 Conclusion This paper considers the problem of comparing realised volatility (RV) estimators. I propose a data-based method for formally ranking RV estimators that does not rely on simulations, detailed assumptions about the market microstructure noise process, or on large m (or continuous record ) asymptotics, though my method does rely on large T asymptotics. By either imposing some assumptions on the time series dynamics of the biases in the RV estimators, or by imposing a rather weak assumption on the time series dynamics of the latent target variable, I present results that allow for a consistent estimate of the ranking of competing RV estimators. These results can be used in formal Diebold-Mariano (995) pair-wise comparisons of RV estimators, or comparisons involving multiple estimators, such as the reality check of White (2000) or its re nement by Hansen (2005). In a small empirical application to IBM equity return volatility, I nd evidence that the daily squared return is out-performed as a measure of quadratic variation by RV estimators based on higher frequency data. However, I nd little evidence that a simple RV estimator constructed using 5-minute returns (either in calendar time or in tick time ) is out-performed by estimators using higher frequency data. This paper immediately suggests two extensions, which are being pursued in separate work. The rst is the comparison of di erent realised covariance estimators. The methods presented in this paper apply directly to this case, subject to a suitable pseudo-distance measure being selected in place of the parametric family presented in equation (2). The standard squared di erence distance measure (MSE) is applicable for comparing realised covariance estimators, and other measures for this case are discussed in Patton (2006). The second important extension of the results in this paper is to comparisons of estimators of the entire covariance matrix. Such comparisons are perhaps more relevant than comparisons of individual variances and covariances, given that these components are usually used together as a covariance matrix (and thus must satisfy conditions to ensure that the matrix is positive semi-de nite). For this application, the covariance matrix pseudodistance measures proposed in Patton and Sheppard (2006) may prove useful, when combined with a random walk or a vector AR assumption for the latent integrated covariance matrix. 2

23 7 Appendix: Proofs Proof of Proposition. Consider a rst-order Taylor series expansion of C (X t ; b) C (X t ; b) C ( t ; b) + C 0 ( t ; b) (X t t ) so E [(C (X t ; b) C ( t ; b)) (Y t t )] E C 0 ( t ; b) (X t t ) (Y t t ) h i = E t k (X t t ) (Y t t ) Under assumption P2 we have: h i E t k (X t t ) (Y t t ) = E h E i t (E [X t j t ; F t ] i t ) (Y t t ) c i k t (Y t t ) k h k t = c i E [Y t t ] = 0 Thus we have E [L (Y t ; X t ; b)] = E [L ( t ; X t ; b)] + E [(C (X t ; b) C ( t ; b)) (Y t t )] h 2 E C 0 t ; b (Y t t ) 2i h E [L ( t ; X t ; b)] 2 E C 0 t ; b (Y t t ) 2i and so E [L (Y t ; X t ; b)] E [L (Y t ; X 2t ; b)] E [L ( t ; X t ; b)] E [L ( t ; X 2t ; b)] up to the error term from the rst-order Taylor series expansion of C (X t ; b) around C ( t ; b) : When b = 0, C (z; b) = z and so the rst-order Taylor series expansion holds exactly. Furthermore, when b = 0 this distance measure can be applied to both positive and negative variables. Proof of Proposition 2. Consider the expectation second term in the second-order mean- 22

24 value expansion of L (Y t ; X t ; b) around L ( t ; X t ; b): E [(C (X t ; b) C ( t ; b)) (Y t t )] "!# JX = E (C (X t ; b) C ( t ; b))! i ~ t+i t i= 2 0 = E 4(C (X t ; b) JX ix C ( t ; i t+j + i= j= 3 JX! i t+i A JX ix = E 4(C (X t ; b) C ( t ; i E JX t+j jf t +! i E [ t+i jf t ] A5 = 0 This then yields i= E [L (Y t ; X t ; b)] E [L (Y t ; X 2t ; b)] = E [L ( t ; X t ; b)] E [L ( t ; X 2t ; b)] j= using the same calculations as in the proof of Proposition. Proof of Proposition 3. i= (i) Consider again the expectation second term in the mean-value expansion of L (Y t ; X t ; b) around L ( t ; X t ; b): h E [(C (X t ; b) C ( t ; b)) (Y t t )] = E (C (X t ; b) C ( t ; b)) (ii) First consider the case that b = 0 : i= ~t+ t i = E (C (X t ; b) C ( t ; b)) 2 t + t+ + t+ = 2 E [C (X t ; b) C ( t ; b)] E [ t (C (X t ; b) C ( t ; b))] E [ t (C (X t ; b) C (X 2t ; b))] = E [ t (X t X 2t )] h = E k+ t (c h i c 2 ) + t ( t i 2t ) = E k+ t since E [ it j t ; F t ] = 0, i = ; 2 For cases with b 6= 0 I employ a rst-order Taylor series approximation of C (X it ; b) around C ( t ; b) C (X it ; b) C ( t ; b) + C 0 ( t ; b) (X it t ) i so E [ t (C (X t ; b) C (X 2t ; b))] E h t b t (X t t ) + b t (X 2t t ) h i = E b+ t (X t X 2t ) h i = E 23 k++b t