The Predictability of Alternative UCITS Fund Returns

Transcription

1 The Predictability of Alternative UCITS Fund Returns Michael Busack, a Wolfgang Drobetz, b and Jan Tille c, * This version: May 2017 Abstract We study the out-of-sample predictability of the returns of pan-european harmonized mutual funds that apply hedge fund-like investment strategies ( Alternative UCITS ). Given these funds higher liquidity, investors could exploit relevant information easier than for hedge funds, and use it for asset allocation and risk management. We use a set of 13 variables, fundamental and technical, and apply single predictor models, combination forecasts, and multivariate regression models. In terms of out-of-sample R 2 predictive models do not lead to more accurate forecasts than historic average returns. Nevertheless, forming portfolios based on predicted returns can yield economic gains to investors, especially during crisis times. Combination approaches and multivariate models reduce estimation uncertainty that results from time-varying predictive performance of single predictor models, leading to economic gains across different market environments. Keywords: Alternative mutual funds, UCITS funds, hedge funds, return prediction JEL classification codes: G11, G14, G23 a Absolut Research GmbH, Grosse Elbstrasse 277a, Hamburg, Germany. b Faculty of Business, Hamburg University, Moorweidenstrassße 18, Hamburg, Germany. c Absolut Research GmbH, Grosse Elbstrasse 277a, Hamburg, Germany. * Corresponding contact information: Tel.: +49 (0) ; jan.tille@absolut-research.de We thank Viktoria-Sophie Bartsch, Hubert Dichtl, Simon Dörig, and participants of the Hamburg Finance Forum 2015, for helpful comments.

2 The Predictability of Alternative UCITS Fund Returns This version: May 2017 Abstract We study the out-of-sample predictability of the returns of pan-european harmonized mutual funds that apply hedge fund-like investment strategies ( Alternative UCITS ). Given these funds higher liquidity, investors could exploit relevant information easier than for hedge funds, and use it for asset allocation and risk management. We use a set of 13 variables, fundamental and technical, and apply single predictor models, combination forecasts, and multivariate regression models. In terms of out-of-sample R 2 predictive models do not lead to more accurate forecasts than historic average returns. Nevertheless, forming portfolios based on predicted returns can yield economic gains to investors, especially during crisis times. Combination approaches and multivariate models reduce estimation uncertainty that results from time-varying predictive performance of single predictor models, leading to economic gains across different market environments. Keywords: Alternative mutual funds, UCITS funds, hedge funds, return prediction JEL classification codes: G11, G14, G23

3 I. Introduction Mutual funds that use hedge fund-like strategies have surged in popularity in Europe since the credit crisis. These funds take advantage of the freedom offered by the pan-european regulatory UCITS regime, which coordinates the framework for retail mutual funds across Europe. 1 Since 2004, it has allowed the implementation of hedge fund-like strategies in a mutual fund vehicle (alternative UCITS funds) by using cash-settled derivatives for investment purposes. 2 According to a recent Bloomberg article, alternative UCITS funds offer higher levels of investor protection than hedge funds through increased transparency, higher liquidity, and lower fees. 3 The article also notes that, from 2010 through 2016, these funds received seventeen times more inflows than traditional hedge funds. The rise of alternative UCITS funds has allowed a higher proportion of investors to gain access to non-traditional investment strategies and also triggered academic research. 4 For example, Tuchschmid and Wallerstein (2013) and Busack et al. (2014) examine the performance characteristics of European alternative UCITS funds, and document that these funds have exposure toward similar risk factors as hedge funds, but to a different degree. Alternative UCITS funds, like hedge funds, follow heterogeneous strategies, and their exposure to risk factors and performance can vary over the market cycle. Therefore, from a research standpoint, it is important to determine whether it is possible to forecast these funds 1 UCITS (Undertakings for Collective Investments in Transferable Securities) is a coordinated pan-european regulatory framework for mutual funds with a strong focus on investor protection that facilitates cross-border marketing and distribution of funds inside the European Union. 2 More detailed descriptions of the possibilities and limitations to set up UCITS compliant alternative strategy funds can be found in Tuchschmid and Wallerstein (2013) or Busack et al. (2014). 3 Available under 4 Similar investment structures are available to U.S. investors under the name alternative mutual funds. For academic research on their performance, see Agarwal et al. (2009). 2

4 returns in an out-of-sample setting in order to enhance investors asset allocation, fund selection, and risk management. As Spiegel (2008) points out, there is an ongoing debate over whether stock market returns are predictable out-of-sample. Welch and Goyal (2008) find that none of their wellknown predictor variables are able to beat historical average return models. More recent research, such as Campbell and Thompson (2008), Rapach et al. (2010), Cenesizoglu and Timmermann (2012), and Meligkotsidou et al. (2014), show that investors can profit from return prediction models. Empirical evidence suggests that hedge fund returns are predictable as a result of their exposure to macroeconomic and market risk factors (Amenc et al., 2003; Wegener et al., 2010; Bali et al., 2011; Drobetz et al., 2012; Avramov et al., 2013; and Panopoulou and Vrontos, 2015; among others). As shown in Busack et al. (2014), alternative UCITS funds are less exposed to overall equity market risk factors than hedge funds. Busack et al. (2017) analyze performance persistence among alternative UCITS and find that it is less pronounced than performance persistence among hedge funds. Therefore, it is also not clear ex ante whether the returns of alternative UCITS offer a similar degree of predictability. In contrast to hedge funds, which often feature lockup and redemption periods, alternative UCITS funds are more liquid. Investors are more likely to actually profit from strategies (if any) that are based on exploiting return forecasts because they can redeem and/or reallocate funds in a much more flexible way. We use the sample period January 2004-June 2015 to analyze fund return predictability for a sample of 811 live and defunct alternative UCITS funds. Our analysis features a set of 13 predictive variables from the forecasting literature on stock market and hedge fund returns. Forecasts are calculated employing univariate prediction models, multivariate models (unconstrained, ridge, and lasso regression), and forecast combination approaches. According to Meligkotsidou et al. (2009), certain non-linear relationships between hedge fund returns and 3

5 risk factors across hedge fund return quantiles cannot be captured by conditional mean regression models such as ordinary least squares (OLS). Therefore, in addition to OLS, we estimate univariate models and the unconstrained multivariate model by quantile regression. We assess the out-of-sample return predictability in two ways: 1) statistically, by using out-of-sample R²s according to Campbell and Thompson (2008) and a test proposed in Clark and West (2007), and 2) economically, by calculating investor utility gains and the performance of simple portfolio strategies. We make several contributions to the extant literature. To the best of our knowledge, this is the first paper to address out-of-sample return predictability for a sample of liquid alternative mutual funds that is available to a broad investor clientele. Using a group of funds that has not been extensively studied in the past contributes to the robustness of the results documented in the broader literature. Moreover, prior literature on the predictability of hedge fund returns only used models that forecasted the conditional mean. We contribute to the literature by applying quantile regression to a large set of single funds. Our approach provides insights into whether robust estimation models that incorporate information about predictors and return distributions can provide better return forecasts than OLS despite noisy out-of-sample forecasts. Finally, Banegas et al. (2013) state that the European mutual fund market is under-researched, because most studies have been conducted on a country level. Using a pan-european sample, we are able to shed additional light on this important industry. Our main findings are as follows. First, on average, no predictive model is able to deliver out-of-sample forecasts for individual fund returns that are statistically more accurate than those based on historical average fund returns, as measured by out-of-sample R² (R 2 OS ). Nevertheless, selecting the funds with the highest out-of-sample return forecasts, rather than the highest historic average returns, can improve investor returns. Second, despite prior findings about the 4

6 advantages of using quantile regression-based forecasts, we are unable to find notable differences between conditional mean return forecast models and robust quantile regression forecasts. Third, splitting our sample period into three separate market phases, we find time variability in predictor performance. Prediction strategies are most valuable during times of market crisis. Finally, we find that combination and multivariate models can provide economic gains over historic average returns in every subperiod. They stabilize the performance of out-of-sample forecasts compared to using single predictor models, where no model can consistently beat the historic average return model. Our findings are robust for different fund categories. The remainder is structured as follows: Section 2 describes our data. Section 3 presents prediction models, and section 4 discusses forecast evaluation methodologies. Our results are presented in section 5, and we show robustness tests in section 6. Section 7 concludes. II. Data, Descriptive Statistics, and Database Biases A. Sample Selection We analyze a sample of 811 live and defunct alternative UCITS funds with at least 36 monthly returns during the January 2004-June 2015 sample period. Monthly (net of fee) total return data were taken from Morningstar Direct database. Because Morningstar introduced alternative fund categories for funds domiciled in Europe in May 2011, we obtain further information on alternative UCITS funds that were in existence prior to this date from Busack et al. (2014). They manually reclassified funds from Morningstar s prior absolute return category and tried to mitigate survivor ship bias by conducting a keyword-based search similar to Agarwal and Naik (2009) for defunct alternative UCITS funds. Therefore, our sample also accounts for survivorship. There are further potential biases, e.g., the backfill or look ahead bias that can plague studies on mutual fund performance. Backfill bias can arise if funds are included in a 5

7 database with their entire return history, despite having begun reporting only after an initial period of good returns. Comparing alternative UCITS funds performance with their performance after omitting the first twelve monthly returns, Busack et al. (2017) have shown that this bias does not affect alternative UCITS funds. This result is important because individual fund histories in our sample are short and truncating return histories would be costly in terms of performance track records. 5 We note that neglecting funds with fewer than thirty-six months of returns could lead to a multiperiod sampling bias. However, Fung and Hsieh (2000) argue that neglecting funds because of a required minimum return history will not induce erroneous inferences if investors are not willing to consider funds with shorter records. According to Del Guercio and Tkac (2002), institutional investors such as pension funds and consultants regularly require a thirtysix-month track record prior to investing. Morningstar also requires thirty-six months of returns to calculate its popular star rating, which suggests that that minimum record is relevant for investors. Finally, Fung and Hsieh (2000) and Bali et al. (2011) argue that requiring a certain minimum performance track record is necessary to ensure sufficient degrees of freedom. To obtain as long a sample as possible, we use the oldest fund share class. 6 Although funds are domiciled in Europe, their investment universe comprises a diverse set of global assets. Unlike funds domiciled in the U.S., UCITS funds do not always share the same base currency. To calculate consistent return forecasts, we use returns in excess of the local currency one-month interbank offered rates (Tuchschmid et al., 2010; Tuchschmid and 5 The median surviving fund s track record in our sample is only forty-six months (see Table 1). Cutting off the first twelve months of returns would halve our sample of available funds. 6 For multiple share classes with the same inception date, we use either 1) the class whose net asset value was calculated using the fund s base currency, or 2) the class with the lowest management fee, if there were still multiple share classes left. If a share class had a 0% management fee, we use the one with the next lowest published management fee. If it has closed, but other classes remained active, we selected a different share class according to the procedure described, and then merged the return histories to extend the time series. We found forty-eight cases where the oldest share class had closed, while others remained active. 6

8 Wallerstein, 2013; Busack et al., 2017). 7 This approach, which relies on the assumption that covered interest rate parity holds, converts fund returns into the returns of an investor who is perfectly hedged against exchange rate risks. It also enables us to estimate identical models for all funds in our sample. One-month interest rates come from Bloomberg. 8 Summary statistics for our sample are reported in Table 1. Panel A contains statistics for the entire fund sample, and panels B and C show statistics for live and defunct funds, respectively. Because our sample period includes the 2008 global financial crisis, monthly performance of alternative UCITS (0.12%) and hedge funds (-0.05%) were muted, as the low and negative excess returns show. [Insert Table 1 here] The major difference between live and defunct funds in terms of performance is their difference in returns. Whereas the average monthly return across live funds is 0.26%, it is only -0.11% for defunct funds. The size of this spread is in line with Busack et al. (2014, 2017) for alternative UCITS funds as well as for hedge funds (Brown et al., 1999; Fung and Hsieh, 2000; Malkiel and Saha, 2005; Bali et al., 2011). In terms of risk, the differences between live and defunct funds are negligible. Fund returns are usually not normally distributed, but are negatively skewed with fat tails. Due to their regulatory framework, illiquidity as a source of autocorrelation in hedge fund returns is not an issue in our sample (Getmansky et al., 2004). 7 By local, we mean the interest rate denominated in the same currency as the fund. We use one-month LIBOR rates for Euro-, USD-, GBP-, JPY-, and CHF-denominated funds, and one-month STIBOR, CIBOR, and NIBOR rates for SEK-, DKK-, and NOK-denominated funds, respectively. A caveat is that this approach does not lead to full hedging unless the fund return is known in advance. 8 We chose not to convert returns to a single numeraire currency. This is because alternative UCITS funds, whose base currency is denominated in a non-euro currency, usually offer share classes whose net asset values are hedged against currency movements to ensure a similar performance between different share classes. 7

9 B. Predictive Variables To predict future fund excess returns, we use a set of thirteen predictive variables that belong to the classes of macroeconomic and financial variables (bond yield measures, stock market valuation ratios, and market risk measures) or technical stock market indicators, which have already been applied as predictive variables in prior forecasting studies. 9 Specifically, we use the following variables: Yield spread between Moody s Baa and Aaa corporate bonds (DEF). 2. Yield on ten-year U.S. constant maturity government bonds (10YUS). 3. Yield on ninety-day U.S. treasury bills (3MUS; secondary market). 4. Yield spread between ten-year U.S. constant maturity government bonds and ninety-day T-bill rate (10Y3M). 5. Yield spread between three-month U.S. dollar LIBOR and ninety-day T-bill rate (TED spread). 6. Inflation rate, measured by the five-year U.S. break-even inflation rate (5YBEI) Chicago Board Options Exchange Volatility Index (VIX). 8. Log of twelve-month positive S&P 500 earnings log of S&P 500 price index (EP, earnings-to-price ratio). 9. Log of twelve-month S&P 500 dividends log of S&P 500 price index (DP, dividend-to-price ratio). 10. Log of twelve-month S&P 500 dividends log of twelve-month lagged S&P 500 price index (DY, dividend-to-yield). 11. Stock market illiquidity measure defined as the average daily ratio of absolute S&P 500 returns to the dollar trading volume on S&P 500 stocks (Amihud, 2002) (AML). 12. Difference between S&P 500 index and its 200-day moving average (P200). 13. Difference between S&P 500 index and its twenty-day moving average (P20). 9 Stock and Watson (2004) use some of the variables we apply to forecast international GDP growth. Campbell and Thompson (2008), Cenesizoglu and Timmermann (2008, 2012), Welch and Goyal (2008), Rapach et al. (2010), Meligkotsidou et al. (2014), and Neely et al. (2014), among others, use similar variables to predict equity market premiums. 10 Data for variables 1-7 come from the Federal Reserve Bank of St. Louis ( data for variables 8-13 were calculated from Bloomberg data. 11 We use this market-based measure of inflation, rather than inflation measured by changes in consumer price indexes, because it is available without a time lag or market participants estimates of future inflation. 8

10 We note that our predictive variables are U.S. related. However, the U.S. is the leading global economy, and shocks to the U.S. economy substantially influence the global economic environment. One reason is that most markets, especially developed economies, are integrated at least to some extent, see Bekaert and Harvey (1995). Therefore, U.S. factors are expected to be relevant predictors for the global economy as well. Despite the fact that funds in our sample invest in a broad spectrum of assets (e.g., equities, fixed income securities or commodities), and that these variables are typically used the in the literature on equity premium prediction, they are also viable predictors for fund returns in our sample. In particular, they can be interpreted as proxies for the state of the macroeconomic environment. Fama and French (1989) document that the spread between long and short dated bonds (term premium), the spread between high and low grade corporate bonds (default premium) and the dividend yield can predict both, stock and bond returns, which is attributed to their relation with the business cycle. More recently, Ludvigson and Ng (2009) report that models incorporating macroeconomic and financial variables improve the predictability of bond risk premia compared to traditional models based on forward rates. The same applies to the predictive ability of stock market volatility (Mueller et al., 2011). Thornton and Valente (2011) and Sarno et al. (2016), find that bond specific prediction models, which use forward rates or term structure models, do not deliver economic gains to investors when compared asset allocation decisions based on naïve benchmark models. We also note that variables used in our study are able to predict the returns of hedge funds (Amenc et al., 2003; Wegener et al., 2010; Bali et al., 2011; Drobetz et al., 2012; Avramov et al., 2013; Panopoulou and Vrontos, 2015) In fact, it is standard in the literature on hedge fund return predictability to use U.S. related predictors, although these funds invest globally as well. 9

11 III. Empirical Methodology In this section, we provide an overview of the models and approaches we use to forecast our out-of-sample fund returns. We forecast one-month-ahead returns using expanding windows. The first forecast for each fund is made using thirty-six monthly returns, and the estimation window is updated subsequently by one return observation each month. We use univariate models, models that combine forecasts from univariate models, and multivariate models that include all or a subset of predictor variables. Univariate models and the kitchen sink model that includes all the above predictors (Welch and Goyal, 2008) are estimated using both OLS and quantile regression. Contrary to forecast combinations, which shrink univariate forecasts toward the historical average (Rapach et al., 2010) and implicitly account for parameter uncertainty, the kitchen sink model does not provide shrinkage and may suffer from overfitting, which could lead to poor out-of-sample return forecasts. To mitigate this concern, we estimate ridge and lasso regression models (Tibshirani, 1996). We use both techniques, because ridge only performs coefficient shrinkage, while lasso can perform both shrinkage and variable selection by setting coefficients to zero. 13 A. Single Predictor Models The standard univariate prediction model estimated by OLS is of the form: r i,t+1 = α i + β i x j,t + ε i,t+1, i = 1, N and j = 1,.., K, (1) 13 Other methods to prevent overfitting are forward stepwise regression or principal components analysis. However, using stepwise regression, it is not guaranteed that this results in the best model containing a subset of the predictors, and it does not account for parameter uncertainty as it provides no coefficient shrinkage. A drawback of principal components analysis is that, in constructing components, no reference is made to the dependent variable. Thus there is no guarantee that linear combinations that best explain the predictors will also be the best linear combinations for predicting returns. 10

12 where r i,t+1 is the fund s observed excess return over the local one-month interest rate at time t+1, x j,t is the value of one of our K-predictors at time t, and ε i,t+1 is an error term, assumed to be i.i.d. with zero mean and unit variance. OLS gives estimates and forecasts of the conditional mean return. The quantile regression approach provides estimates of conditional return quantiles and allows modeling the entire conditional return distribution. Koenker and Basset (1978) propose this methodology as a means to obtain estimators that are more robust to outliers. It also leads to substantially higher efficiency if distributions deviate from normality, but with near-ols efficiency in case of normally distributed errors. Moreover, Cenesizoglu and Timmermann (2008) and Meligkotsidou et al. (2014) document that quantile regression can enhance equity premium forecasts over OLS because of its ability to account for non-linearity and non-normality patterns. They further note that most single predictor variables contain information only for parts of the return distribution, and that OLS can hide important return characteristics, especially in the tails of a distribution. As shown in Table 1, funds in our sample suffer from negative skewness and excess kurtosis. OLS assumes that predictors effects are constant across different return quantiles, while quantile regression allows for varying effects of predictors across return quantiles. 14 Univariate prediction models using quantile regression are of the following form: r i,t+1 = α i p + β i p x j,t + ε i,t+1,, i = 1, N and j = 1,.., K, (2) where r i,t+1 is the fund s observed excess return over the local one-month interest rate at time t+1, x j,t is the value of one of our thirteen predictors at time t, and ε i,t+1 is an error term. Errors 14 Basset and Chen (2001) use quantile regression to complement classical return-based style analysis because it enables investors to gather information on how style returns affect the entire return distribution, instead of just the expected returns. Cenesizoglu and Timmermann (2008) show that several predictive variables can have asymmetric effects on forecasting stock market return distributions. Specifically, they document that predicting the center of a return distribution is almost impossible, but variables such as the T-bill rate, inflation, the default spread, and stock market variance can predict the tails. Meligkotsidou et al. (2009) model return distributions of hedge fund indices using quantile regression, and find that hedge fund risk factors affect hedge fund returns differently across return quantiles. 11

13 are assumed to be independent of an unspecified error distribution g p (ε i ). p varies between 0 and 1, and the p-th quantile is equal to 0, such that 0 g p (ε i )dε i = p. Accordingly, the model assumes that the p-th (conditional) quantile of a fund s return distribution in period t+1 is Q ri,t+1 (p x j,t ) = α p i,j + β i p x j,t. Both coefficients α and β depend on the p-th quantile, which can vary according to the value of p. These varying coefficients extract more information from a fund s excess return than classical mean predictive regression models. Estimating equation (1) using OLS minimizes the sample estimate of a quadratic linear loss function with respect to the coefficients α i and β i. Similarly, the p-th quantile of r i,t+1 in equation (2) is obtained using a point estimate by minimizing an asymmetric linear loss function (see Koenker and Basset, 1978). Because we are using quantile regression to obtain more robust estimates of r i,t+1, we construct point forecasts based on the conditional quantile forecasts. We follow Meligkotsidou et al. (2014) and construct what they refer to as robust point forecasts, i.e., we use a fixed weighting scheme to combine the set of quantile forecasts for each predictor x j,t : 15 r i,t+1 = w p r i,t+1 (p), w p = 1 (3) There is no specific theory that defines an appropriate weighting scheme. We choose to estimate the 0.1, 0.25, 0.5, 0.75, and 0.9 quantiles, and use Judge et al. s (1988) weighting scheme to aggregate conditional quantile return forecasts as follows: 16 r i,t+1 = 0.05r i,t+1 (0.1) r i,t+1 (0.25) r i,t+1 (0.5)+ 15 Note that Meligkotsidou et al. (2014) use fixed and time-varying weighting schemes. Time-varying schemes require separate holdout periods to estimate the optimal combination of quantile forecasts. Because our funds return histories are short, we do not use time-varying weighting schemes. 16 For robustness, we use two other weighting schemes, w p = w = 0.2 and [w 0.1 = 0.1, w 0.25 = 0.25, w 0.50 = 0.30, w 0.75 = 0.25, w 0.90 = 0.10], to put more weight on tails. As per Cenesizoglu and Timmermann (2008), it is easier to forecast the tails than the center of a return distribution. However, these different weighting schemes, which we did not tabulate but are available upon request, do not alter our main results. 12

14 +0.25r i,t+1 (0.75) r i,t+1 (0.9) (4) B. Forecast Combinations The practice of combining forecasts dates back to Bates and Granger (1969), who showed that composite forecasts can result in lower mean squared errors than the use of several single forecasts. Stock and Watson (2004), for example, use combination approaches to predict international GDP growth. In a financial market context, Rapach et al. (2010) suggest using forecast combinations to predict U.S. equity premiums. Single predictors are related to different aspects of current and future business conditions, and are thus subject to structural market changes. Structural instability can induce time-varying predictive performance of univariate models, delivering reliable forecasts only during certain times or market phases. Combining several single predictive variables results in more accurate and less variable equity premium forecasts. Therefore, it adds value beyond the simple historic average return. Similarly, Avramov et al. (2013) and Panopoulou and Vrontos (2015) use forecast combination to predict out-of-sample hedge fund returns. They confirm that combining forecasts helps reduce model uncertainty and model instability. Despite the fact that single predictors might result in more accurate forecasts or deliver higher returns, it is not known ex ante which predictor will yield the best results. Therefore, combining several univariate forecasts is the most intuitively sensible approach. First, we assign equal weights to all univariate forecasts. Second, we use equal weights for the two categories, macroeconomic and technical. This second weighting scheme assigns higher weights to individual technical indicators. Neely et al. (2014) show that technical indicators complement and enhance equity premium forecasts from macroeconomic indicators We do not use more sophisticated combination schemes because they require out-of-sample holdout periods to calculate optimal weights. Another reason is the ensuing parameter instability inherent in estimating optimal 13

15 C. Multivariate Forecasts The third approach to forecasting alternative UCITS fund returns involves multivariate models. The simplest model contains all thirteen predictive variables ( kitchen sink model). When the kitchen sink model is estimated using quantile regression, we estimate the same quantiles and use the same weighting scheme as for the univariate models. In addition, we estimate multivariate models using ridge and lasso regressions. Ridge regression was introduced in Hoerl and Kennard (1970), and lasso was proposed in Tibshirani (1996). Both methods are intended to improve estimation and forecast accuracy for linear OLS regressions, when, for example, multicollinearity exists, or there are a number of true coefficients that are very small. Although it introduces a bias in the regression, shrinking coefficients can help reduce the prediction error by lowering the variance. Shrinking is achieved by adding different penalty terms to the minimization of the sum of squared OLS residuals. The penalty added in ridge regression is the sum of squared regression coefficients; in lasso regression, it is the sum of absolute regression coefficients. In other words, the multivariate model is estimated by minimizing either: T 1 t=1 (r i,t+1 x t β i )² + λ 1 K+1 2 j=0 β j in the case of ridge regression, or (5) T 1 t=1 (r i,t+1 x t β i )² + λ 2 K+1 j=0 β j for lasso regressions. 18 (6) Therefore, the primary difference between the two is the form of the penalty term. The parameters λ 1 and λ 2 control the strength of the penalty. λ = 0 yields the OLS estimate, and increasing values of λ shrink parameters toward 0. The penalty term used in ridge regressions, however, only shrinks the estimated parameters toward 0, but never exactly equal to 0 unless weights (DeMiguel et al., 2009). Moreover, as shown in Rapach et al. (2010) and Meligkotsidou et al. (2014), equally weighted combination forecasts for the equity premium yield good out-of-sample results. Panopoulou and Vrontos (2015) also find that simple equally weighted combination schemes deliver similar or even better results than more sophisticated schemes when using hedge fund return forecasts for portfolio construction. 18 We estimate both models using the R package glmnet. See Friedman et al. (2010) for details. 14

16 λ 1 =, in which case all the coefficients are zero. Lasso, however, allows for a more flexible selection by setting coefficients equal to 0. It thus fits a linear model of r t+1 on the x t s, where some coefficients are set to zero by shrinkage. The higher λ 2, the more coefficients are set to zero. We can thus think of both methods as a constrained form of linear regression. Because estimated coefficients vary with different values for λ, it is important to select the most appropriate λ values for the predictive models. No theory exists which values to use, thus we rely on a cross-validation method based on mean squared errors. Cross-validation estimates prediction errors for each value of λ. We use two different values for λ, one that yields minimum mean squared errors (Lambda_min), resulting in the model with the highest prediction accuracy, and one that yields the largest lambda, at which the mean squared error is within one standard deviation of Lambda_min (Lambda_1se). According to Friedman et al. (2010), this second value of lambda yields the most regularized model and mitigates problems that can arise from in-sample overfitting. Predictions from any multivariate model described above have the following form: r i,t+1 = x t β i, (7) where x t and β i are (K+1) 1 vectors containing the lagged predictor variables and regression coefficients, respectively. IV. Forecast Evaluation We evaluate forecasts from the models specified above both statistically and economically, because there can be substantial differences between the two approaches. For example, Leitch and Tanner (1991) find no systematic relationship between statistical accuracy and the profitability of interest rate forecasts. They show that forecast models perform poorly when 15

17 evaluated by statistical measures such as root mean squared errors or Theil s U, but result in more profitable investments than simple time series models. The natural benchmark for evaluation is the historical average return (Welch and Goyal, 2008; Rapach et al., 2010; Meligkotsidou et al., 2014; among others). Statistical evaluation is carried out using Campbell and Thompson s (2008) out-of-sample R² (R 2 OS ) together with a test proposed in Clark and West (2007). Statistical evaluation regularly reveals that out-of-sample return forecasts perform either worse or only marginally better than historic mean forecasts (Welch and Goyal, 2008). Camp- 2 bell and Thompson (2008) argue that economic gains can be large even if R OS is only marginally positive, as long as the squared Sharpe ratio of an asset is large relative to R 2 OS. 19 Rapach et al. (2010) find that some of the univariate predictors in their sample add economic gains to investors, despite having negative R 2 OS. Furthermore, Cenesizoglu and Timmermann (2012) report that a variety of forecast models, which perform poorly from a statistical standpoint, generate significant economic gains to investors. Therefore, according to Rapach and Zhou (2013), profit- and utility-based forecast evaluation is a more direct measure of the value of forecasts to investment professionals. A. Statistical Forecast Evaluation The out-of-sample R 2 is computed as R 2 OS = 1 T 1 t=s (r i,t+1 r i,t+1)², where r T 1 r i,t)² i,t+1 is a t=s (r i,t+1 fund s realized excess return, r i,t+1 is the forecasted return using data up to time t, r i,t is a fund s historical average excess return, and s is the length of the initial estimation window, i.e., s = 19 For example, using the smoothed EP ratio, which is associated with an R OS 2 of 0.43%, Campbell and Thompson (2008) calculate that a mean-variance investor could have increased average monthly returns by 36% by investing in the stock market, which had a squared Sharpe ratio of 1.2%. 16

18 2 36. R OS is positive if forecasted returns are closer to the realized than the historical average returns. 2 Because R OS remains quiet on the statistical significance of the reduction of the forecast errors, we use Clark and West s (2007) method to test whether R 2 OS 0, against the alternative hypothesis that R 2 OS > 0. This test, which the authors call MSFE-adjusted, is a more powerful version of those proposed in Diebold and Mariano (1995) and West (1996), especially when testing the differences in mean squared forecast errors between a richer and a more parsimonious model. 20 It yields a test statistic that is asymptotically normally distributed when comparing forecasts from both models. To calculate the test statistic, Clark and West (2007) suggest, in a first step, constructing time series consisting of the differences between models squared forecast errors, as follows: d i,t+1 = (r i,t+1 r i,t)² [(r i,t+1 r i,t+s+1) 2 (r i,t r i,t+s+1) 2 ] (8) To calculate (one-sided) p-values, the second step is to regress these time series on a constant, and then calculate the corresponding t-statistics. 21 B. Economic Forecast Evaluation To assess the economic significance of forecast models, i.e., the gains to investors, we calculate the certainty equivalent returns of a mean-variance investor who uses fund return forecasts instead of historic averages to split investments between the risky asset and cash. 22 Certainty equivalent returns can be interpreted as the maximum management fees investors are 20 In our context, the more parsimonious model is the historic mean forecast model. 21 To account for autocorrelated forecast errors, we use Newey and West (1987) standard errors. 22 Banegas et al. (2013) argue that returns of mutual fund selection strategies directly accrue to investors if they are not charged subscription and redemption fees. This is because fund returns are net of management fees and trading costs. However, we note that any returns from these fund trading strategies are gross, as we do not incorporate transaction costs of buying and selling mutual funds. Retail investors typically pay sales commissions (e.g., front load fees) to brokers, but institutional investor share classes are tradeable without fees, due to higher minimum initial investments. Therefore, our results are more illustrative for retail investors, but still somewhat representative for institutional investors. 17

19 willing to pay to be indifferent between using return forecasts and historic average returns. The utility function has the following form: u p,d = r p,d 1 2 γσ p,d 2 (9) where r p,d is the average realized return of decile portfolio d using prediction model p, 2 and σ p,d is its variance over the out-of-sample period. We calculate utility gains (certainty equivalent returns) as the difference between u p,d.and u avg,d, where u avg,d, is the investor s utility when decisions are based on historical returns. By incorporating strategies standard deviations, certainty equivalents are also a means to evaluate the risk-adjusted performance difference between prediction models. Similarly to Campbell and Thompson (2008), Rapach et al. (2010), Neely et al. (2014), Meligkotsidou et al. (2014), and Panopoulou and Vrontos (2015), we calculate average utilities for mean-variance investors with different levels of risk aversion γ (γ 0.5 5). Risky asset portfolios are formed monthly as equally-weighted fund portfolios holding the 10% of funds with the highest forecasted returns according to a forecasting model. 23 Portfolio returns are calculated on a euro-hedged basis, assuming validity of covered interest rate parity. period is: The amount of money put into the risky assets at the beginning of the out-of-sample w d,t = r d,t+1 2, (10) γσ d,t+1 2 where r d,t+1 is a decile portfolio s forecasted return for period t+1, and σ d,t+1 is its estimated variance. Similar to Campbell and Thompson (2008), Rapach et al. (2010), and Panopoulou 23 Note that we use equally weighted portfolios because DeMiguel et al. (2009) show that, due to estimation errors, the out-of-sample performance of in-sample optimally diversified portfolios is usually inferior to that of simple equally weighted portfolios. Kirner et al. (2011) show that this continues to be true when constructing balanced portfolios of international stock and bond index futures. Following Kosowski et al. (2007) we only take funds into account if they managed at least 20 million Euros at the time of the forecast. 18

20 and Vrontos (2015), we estimate portfolio variance by the historical variance of the decile portfolio s realized variance during the estimation window. Because alternative UCITS funds cannot be sold short, we restrict portfolio weights to lie between 0% and 100%. As a second measure to gauge economic significance, we evaluate decile portfolio performance directly and calculate several performance metrics. This approach captures the benefits from using return forecasts in fund portfolio construction and risk management (Panopoulou and Vrontos, 2015). V. Empirical Results Next, we present our out-of-sample results for the forecasting and evaluation procedures described above. Because our sample begins in January 2004, and we require thirty-six months of returns for initial forecasts, the evaluation period is January 2007 through June Table 2 gives the statistical evaluation results for the median out-of-sample R 2 (R 2 OS ), along with the p-values from Clark and West s (2007) MSFE-adjusted test statistic (in parentheses) for forecasting out-of-sample fund returns. The QR and OLS columns show the results for forecasts based on quantile regression and ordinary least squares, respectively. Rows 1-13 show the single predictor forecasts, followed by the combination forecast results. Finally, the last five rows give the results for the multivariate regression models. [Insert Table 2 here] 2 First, out-of-sample return forecasts are not very accurate. All R OS values for the univariate forecasts are negative, indicating that the return forecasts underperform the historic mean model for 50% of the funds. Second, combining univariate forecasts improves R 2 OS. The median 2 fund s R OS values are positive for both combination schemes, indicating that using combination 19

21 forecasts increases forecast accuracy for at least 50% of the funds. However, with a median p- value of around 0.2, this is not statistically significant. Third, the multivariate models (kitchen sink, Lasso_min, and Ridge_min) deliver the 2 poorest out-of-sample return forecasts, with the kitchen sink model showing the lowest R OS at -62.1%. The more regularized models, Lasso_1se and Ridge_1se, perform better than richer models in terms of forecast accuracy. Fourth, forecasts using quantile regressions do not deliver more precise out-of-sample forecasts than OLS-based return forecasts. This result is in line with Cenesizoglu and Timmermann (2008) and Meligkotsidou et al. (2014). Cenesizoglu and Timmermann (2008) document that quantile regression improves return forecasts only in the tails of the stock market s return distribution, but it is no better at predicting the median than OLS regression at predicting the mean. Because we assign a 40% weight to the median when aggregating quantile forecasts to the point forecast, it is possible that the ability to forecast the tails of a distribution is averaged out. Meligkotsidou et al. (2014) show that quantile regression improves forecasting results over simple OLS mean return forecasts only when they apply their more sophisticated time-varying weighting schemes. However, these schemes require substantial holdout periods in order to calculate optimal weights. 24 The subsequent tables present the results for economic significance. Table 3 reports annualized certainty equivalent returns for a mean-variance investor applying an asset allocation strategy of splitting investments between portfolios of the decile of funds with the highest return 24 Meligkotsidou et al. (2014) use ten years of data, or forty quarterly returns, as a holdout period to compute optimal weights. Compared to Cenesizoglu and Timmermann (2008) and Meligkotsidou et al. (2014), who use data dating back to 1871, or post-world War II data beginning in 1947, respectively, for estimating their predictive quantile regression models, our sample period is quite short. 20

22 forecasts and cash for varying levels of risk aversion γ. 25 Panel A reports results for quantile regression forecasts from Equations (2) and (3). Columns (1)-(13) show results for univariate predictors. The results for combination forecasts and the kitchen sink model are in columns (14)-(16). Lasso and ridge regression forecasts are in columns (17)-(20). Panel B gives the results for OLS-based (mean) return forecasts. [Insert Table 3 here] Despite their poor statistical performance, all predictors, except the three-month U.S. Treasury bill rate, used together with quantile regression deliver economic value to investors, i.e., result in positive certainty equivalents. This finding is in line with Campbell and Thompson (2008), who demonstrate the potential for large economic gains even with only marginally positive R 2 OS. Independent of which return prediction model is used, the certainty equivalents are very similar for different levels of risk aversion, varying by 10 to 20 basis points for low and high levels. This may be an indication that utility gains are most likely driven by higher portfolio returns, and not by risk reduction, when using forecast models instead of historic returns. 2 Although the R OS in Table 2 are negative for each of the univariate predictive variables, we observe some predictors that result in higher utility gains than a combination strategy. As panel A of Table 3 shows, constructing portfolios based on the TED spread variable (column (5)) results in certainty equivalents of approximately 3.8% per annum, while equally weighting all univariate forecasts (column (14)) yields certainty equivalents of only 3%. 26 However, given that most predictors underperform simple combination schemes, and since we cannot know ex 25 To facilitate the reading of Table 3, we only show results for risk aversion coefficients of 1, 3, and 5. Results for further values are available from the authors upon request. 26 This, however, is not unusual. Rapach et al. (2010), for example, report an increase in investor utility of 5.1% per annum when a term spread variable is used to predict the equity premium. This is more than twice the utility 2 gain for a mean combination strategy, despite the fact that R OS for their term spread variable is -3%, while it was positive and statistically significant for combination forecasts. 21

23 ante which predictor will work best, we believe combining return forecasts remains the most sensible approach. Note that utility gains from forecasts are slightly higher using technical than macroeconomic indicators. For example, looking at the 200-day moving average in column (12) reveals that only three macroeconomic variables deliver higher certainty equivalents. This is in line with Neely et al. (2014), who document that technical indicators fare as well as or even better than macroeconomic variables when forecasting the equity premium, and also show that macroeconomic and technical indicators are complementary over the business cycle. Although it is the statistically worst model, the kitchen sink model fares quite well economically, delivering certainty equivalent returns that are comparable to those from forecast combination. Forecasting several hedge fund index returns, Panopoulou and Vrontos (2015) also find that kitchen sink models can yield utility gains that are comparable to those obtained from combination approaches. 27 Among all the approaches that incorporate information from more than one predictor, Lasso_min, which performs coefficient shrinkage and variable selection, delivers the highest utility gains. For risk aversion coefficients above 3, utility gains are higher than for any of the univariate predictors in panel A. Lasso_1se and Ridge_1se, which return sparse models to prevent overfitting, do not add substantial utility gains when compared to historic mean return forecasts. This may be because the regularization algorithm of these models moves them close to the historic mean model. It shrinks most of the coefficients toward or to zero. 27 In fact, using a risk aversion coefficient of 3, Panopoulou and Vrontos (2015) document that the kitchen sink model adds economic value for seven of nine hedge fund strategies when compared to a naïve model. It also delivers higher certainty equivalent returns than any of the combination approaches. One explanation for this may be that the models produce similar relative return rankings for out-of-sample funds, and therefore tend to direct similar funds into the decile portfolios. 22

24 Table 3, panel B, shows that using OLS yields similar results as using quantile regression forecasts. Again, we note that using certain predictive variables would have resulted in higher utility gains than using information from all the predictive variables. But this method nevertheless still outperforms the bulk of the univariate predictors. However, when we compare quantile regression forecasts with OLS-based forecasts, there are some variations in the performance of the univariate predictors. Technical indicators tend to perform even better when OLS is used. In fact, the 200-day moving average delivers the highest utility gains for risk aversion coefficients of 3 and above. The results for investing in decile portfolios directly are in Table 4. Panels A and B report results for univariate prediction models based on quantile regression (panel A) and OLS (panel B), respectively. Panel C shows portfolio performance for combination strategies and the multivariate models. For comparison, the last column in panel C reports results for portfolios formed using only historic average returns (HAVG). [Insert Table 4 here] The main pattern is by and large similar to the results in Table 3, i.e., using return forecasts rather than historic mean returns can yield substantial economic gains. Using forecast models to select funds generally results in higher portfolio returns, as seen in rows (1). For example, the top decile portfolio formed using historic returns yielded annualized returns of 2.08%, while the portfolio formed using the TED spread in panel A delivered returns of 5.95% per annum. Again, some univariate models result in higher returns than using the entire set of predictive variables. However, using forecast combinations or multivariate prediction models 23