An alternative route to performance hypothesis testing Received (in revised form): 7th November, 2003

Transcription

1 An alternative route to perforance hypothesis testing Received (in revised for): 7th Noveber, 3 Bernd Scherer heads Research for Deutsche Asset Manageent in Europe. Before joining Deutsche, he worked at Morgan Stanley, J.P. Morgan and Schroders, where he headed global fixed incoe research. He has published several books on asset anageent and is a lecturer in Finance at the University of Augsburg. Head of Investent Solutions, Deutsche Asset Manageent, Mainzer Landstr , 637 Frankfurt, Gerany Tel: ; Fax: ; e-ail: bernd.scherer@db.co Abstract A wide variety of risk return ratios are routinely reported in sales pitches as well as acadeic publications. Little attept has been ade, however, to look at the sall saple distributions of these estiators in order to derive confidence bands. The reason for this has been the extree difficulty of working out the required statistics for ost risk return ratios. Rather than following classical statistics, this paper relies on a general and robust ethod which not only provides confidence intervals for arbitrary risk return ratios, saple sizes and distribution, but is also fairly easy to ipleent. Keywords: Sharpe ratio, Sortino ratio, hedge funds, bootstrapping, confidence interval, sall saple Introduction Reported risk return ratios relate average returns to alternative easures of risk, and hence involve the ratio of a rando (owing to sapling error) noinator and denoinator. As such, point estiates of these ratios are easy to calculate, but confidence intervals are uch ore difficult to arrive at. Confidence intervals are needed, however, for any kind of statistical inference and decision aking. While Lo () has shown that an asyptotic distribution exists for the Sharpe ratio, this result provides a special case rather than a general concept. Traditional ethods only work well if the sapling distribution of the statistic in question is asyptotically norally distributed. There is little guidance on the sall saple behaviour of risk adjusted perforance easures or how any data points one needs to rely on for asyptotic results. Useful exceptions are Pedersen and Satchell (), Miller and Gehr (1978) and Jobson and Korkie (1981). Moreover, these analytical solutions are either extreely difficult to work out or siply do not exist for odifications of the popular Sharpe ratio which focus ore on downside risk. An exaple is the well-known Sortino ratio (it relates average return to standard deviation of downside returns). A general ethod is needed which provides confidence intervals for arbitrary risk return ratios, saple sizes and distributions. Henry Stewart Publications X () Vol. 5, 1, 5 1 Journal of Asset Manageent 5

2 Scherer Bootstrapping theory as an alternative Suppose a series of returns (total return inus risk-free rate) r 1, r,..., r is observed. Ex post risk return ratios ( ˆ) are calculated as the ratio of average return per unit of risk. This paper will focus on Sharpe and Sortino ratios (Sharpe, 199; Sortino and Price, 199). Both ratios differ with respect to the eployed risk easure. The Sharpe ratio applies a syetric risk concept, equally penalising (squaring) downside and upside deviations fro the saple ean return, while the Sortino ratio only includes negative perforance in their calculation of squared returns. The Sortino ratio is included for three reasons. First, it provides a better capture of risk if returns are non-norally distributed, as is the case for hedge fund returns. Secondly, it is known to suffer ore fro estiation error, as it uses fewer data (extree returns are by definition rare). Thirdly, no large saple approxiations exist. The saple calculations for both ratios are given below. Sharpe ratio Sortino ratio 1 1 r i i=1 i=1 (r i r) (1) r i i=1 i=1 I(r i <)(r i ) () where I(r i < ) denotes the indicator function. High ratios are preferable (everythingelseequal)astheyindicatea better return per unit of risk taken. If the sall saple distribution of ˆ is far fro noral, classical ethods are biased and unreliable. In any case, the analytical forula for the large sapling distribution of (1) is extreely hard to coe by, while it is unknown for (). In order to overcoe the above proble, bootstrapping techniques are relied on. Resapling treats the current saple as a good approxiation of the true distribution (in the absence of further inforation, it is the best available). It then repeatedly draws fro the epirical distribution to recalculate the statistic of interest any ties to arrive at the bootstrap sapling distribution that can now be used for hypothesis testing. Suppose one is given onthly returns on the HFR fund of funds index ranging fro January 199 to April 3 (16 observations). The JPM one-onth cash rate fro DataStrea is used to calculate a risk-free return. The bootstrapping procedure is as follows. Randoly draw 16 (original saple size) returns with replaceent fro the original saple. Calculate a new risk return ratio ˆ*b based on the resapled returns. Repeat this procedure for b 1,..., B ties arriving at ˆ*1, ˆ*1, L, ˆ*b,L ˆ*B resapled ratios. The bootstrap sapling distribution of ˆ*b can now be taken to judge whether the sapling distribution ˆ of in sall saples is noral (and hence traditional approaches ight not be so bad after all) or not. With B 1,, the following results are obtained (see Figure 1). 1 While the Sharpe ratio is well behaved (all resapled realisations plot on a straight line, equalising hypothetical and epirical percentiles), the sae cannot be said about the Sortino ratio. Deviations fro norality are large at both ends. Noral (sall saple) approxiations look only reasonable for the traditional Sharpe ratio. They see to be largely isleading, however, for the Sortino ratio. Taking the.5 per cent and 97.5 per cent percentile, one can now arrive at a syetric 95 per cent confidence band CI( ˆ*.5%, ˆ*97.5%). Note the apparent non-norality of the saple distribution for the Sortino ratio in 6 Journal of Asset Manageent Vol. 5, 1, 5 1 Henry Stewart Publications X ()

3 Perforance hypothesis testing SORTINO RATIO SHARPE RATIO Quantiles of Standard Noral Quantiles of Standard Noral Figure 1 QQ-plot for bootstrapped sapling distribution Figure Bootstrapped sapling distributions Henry Stewart Publications X () Vol. 5, 1, 5 1 Journal of Asset Manageent 7

4 Scherer Probability that double bootstrapped Sortino-ratio falls below initial Sortino-ratio Figure 3 Saple histogra of u b 1 Z Z z=1i( ˆ *b * z< ˆ) Figure. As suspected, the Sortino ratio shows a uch larger dispersion in resapled outcoes and hence estiation error. The corresponding values can be cut off fro the histogras in Figure. For the Sortino (Sharpe) ratio, the 95 per cent interval ranges fro.11 to.9 (.8.1). Both ratios are significantly different fro zero. Increasing the coverage ratio with the double bootstrap So far, reliance has been on the 95 per cent interval fro a siple bootstrap procedure. It is not guaranteed, however, that the 95 per cent confidence band calculated above indeed covers the true ratio with 95 per cent probability. The true ratio ight only fall 85 per cent of all ties into the estiated confidence band. One ethod of increasing the coverage probability is the double bootstrap, which can be thought of as bootstrapping the bootstrap. The double bootstrap as described by Nankervis () involves the following calculations. Perfor the siple bootstrap as described above. Save all b 1,..., B resapled data sets as well as resapled ratios ˆ*b. Thisiscalledthefirst stage resapling. For each of the B resapled datasets, start a second round of z 1,..., Z resaples, leading to a total of BZ resaples denoted as ˆ*b * z.foreach ˆ*b, there exists a new set of Z resapled ratios ˆ*b * 1,L, ˆ*b * Z. These are the second stage resaples. For each ˆ*b, calculate the percentage of second stage resaples ˆ*b * 1,L, ˆ*b * Z that fall below the original saple estiate of the risk return ratio ˆ, ie calculate u b 1 Z Z z=1 I( ˆ*b * z < ˆ) (3) Choose B 1, and Z. Under ideal conditions u b follows a unifor distribution. Figure 3 shows that this assuption is clearly violated for the double bootstrapped Sortino ratios ˆ*b * 1,L, ˆ*b * Z. Foral tests such as the Kologorov 8 Journal of Asset Manageent Vol. 5, 1, 5 1 Henry Stewart Publications X ()

5 Perforance hypothesis testing (a) RAW DATA (b) FILTERED DATA ACF. ACF Lag Lag Figure Autocorrelation for hedge fund series and Sirnov test as well as the adjustent test provide p values close to per cent. Hence, the null hypothesis that Figure 3 coes fro a unifor distribution can be safely rejected. Finally, calculate the.5 per cent and 97.5 per cent percentiles of u b.usethese value to adjust the first stage resaple confidence band to CI( ˆ*u.5%, ˆ*u97.5%). With the double bootstrap, the confidence interval changes to a uch higher bound of.18 (instead of.11) representing the 8 per cent quantile (rather than the.5 per cent quantile). In effect, one gets CI( ˆ*u7.9%.18, ˆ*u96%.96). What if one needs to deal with autocorrelated data? So far, it has been iplicitly assued that return data r 1, r,..., r have been drawn independently, ie neither returns nor volatilities are autocorrelated. Bootstrapping iplicitly reoves any tie dependence by its very definition. All draws are unconditional on the result of the previous draw. After a strong positive draw, there is no echanis that would favour another positive draw (in case of positive autocorrelation) or another large draw (in case of ARCH effects). Many financial tie series (real estate, corporate bonds, high yield, hedge funds), however, show strong autocorrelation due to the illiquidity (infrequent trading) of the underlying instruents. Bootstrapping fails if one does not account for return dependence. For the data analysed so far, the autocorrelation function can be plotted as in Figure (a). The first-order autocorrelation coefficient aounts to.31 and is well outside the confidence liits (given by dotted line), ie it is statistically significant. Henry Stewart Publications X () Vol. 5, 1, 5 1 Journal of Asset Manageent 9

6 Scherer FREQUENCY 8 6 FREQUENCY LOWER BOUND (.5% LEVEL) FOR RAW DATA LOWER BOUND (.5% LEVEL) FOR FILTERED DATA Figure 5 Lower confidence bound (.5 per cent) on Sharpe ratio for raw and adjusted data One way to deal with this issue is to reove the autocorrelation using a siple filter that adjusts for the inherent AR(1) process (lag one is statistically significant, while lag two is not) and reapply the ethods established in the previous sections to the transfored series. It is known that the first-order autocorrelation can be reoved. Estiate the AR(1) coefficient fro a linear regression r t 1 r t 1 t () Save the regression coefficient 1 and create a new filtered series r t * according to r t * r t 1 1 r t 1 (5) Obviously this represents a break with the non-paraetric approach used so far. As a coproise, one ight want to estiate Equation () using robust ethods. The above procedure will leave the average return of a series constant, but effectively increase (decrease) its variance for positive autocorrelation (negative correlation) as described in Scherer (). The effect of this procedure can be seen in the right-hand autocorrelation plot in Figure (b). Virtually all significant autocorrelation (confidence bounds are given by dotted lines) has been reoved. After the reoval of autocorrelation, one can proceed as in the previous section. Rather than focusing on a single series, all HFR series are used for the described frequency and data period (see the HFR website for a description of the underlying data series). Figure 5 plots the distribution of lower confidence bounds (.5 per cent percentile) of Sharpe ratios for raw data as well as adjusted (filtered) data. The Sharpe ratios of 5 out of 37 series 1 Journal of Asset Manageent Vol. 5, 1, 5 1 Henry Stewart Publications X ()

7 Perforance hypothesis testing FREQUENCY 6 FREQUENCY LOWER BOUND (.5% LEVEL) FOR RAW DATA LOWER BOUND (.5% LEVEL) FOR FILTERED DATA Figure 6 Lower confidence bound (.5 per cent) on Sortino ratio for raw and adjusted data (expected:.5% 37 1) are not statistically different fro zero. This nuber increases to 11, if filtered data (with higher volatility) are used. If this is cobined with a suspicion of upward bias in hedge fund returns, any hedge fund styles fail to show statistically significant Sharpe ratios of even onthly returns. An alost identical picture can be found for the Sortino ratio in Figure 6. Again, the historical track record of the hedge fund industry as a whole leaves soe doubt about the industry s ability to create added value. Suary Renewed interest in the significance of risk return ratios has been focusing on closed for solutions for the well-known Sharpe ratio. This paper provided a robust ethodology for evaluating the properties of the Sharpe ratio s sapling distribution, as well as how to derive confidence intervals without having to rely on asyptotic approxiations (as in reality saples are sall). It has also been shown that the sapling distribution of other risk return easures for which asyptotic results do not exist ight be highly non-noral. While the double bootstrap ethodology leads to significantly refined confidence bands, a ore realistic odelling of hedge fund returns leaves the case for hedge funds less optiistic than providers of these services ight wish. Notes 1 All calculations have been perfored in S-Plus. For relevant code and further exaples on the use of bootstrapping in portfolio anageent see Scherer and Martin (3). References Jobson, J. and Korkie, B. (1981) Perforance Hypothesis Testing with the Sharpe and Treynor Measures, Journal of Finance, 36, Lo, A. W. () The Statistics of Sharpe Ratios, Financial Analysts Journal, July/August, 58 (). Henry Stewart Publications X () Vol. 5, 1, 5 1 Journal of Asset Manageent 11

8 Scherer Miller, R. and Gehr, A. (1978) Saple Bias and Perforance Measures: A Note, Journal of Financial and Quantitative Analysis, 13, Nankervis, J. () Stopping Rules for Double Bootstrap Confidence Intervals, University of Surrey, /Nankervis_John.pdf Pedersen, C. and Satchell, S. () Sall Saple Measures of Perforance Measures in the Asyetric Response Model, Journal of Financial and Quantitative Analysis, 35(3), 5 5. Scherer, B. () Portfolio Construction and Risk Budgeting, Riskwater, London. Scherer, B. and Martin, D. (3) Portfolio Optization using Nuopt for S-Plus, Springer, New York. Sharpe, W. F. (199) TheSharpeRatio, Journal of Portfolio Manageent, Fall, Sortino, F. A. and Price, L. N. (199) Perforance Measureent in a Downside Risk Fraework, Journal of Investing, Fall. 1 Journal of Asset Manageent Vol. 5, 1, 5 1 Henry Stewart Publications X ()