Out-of-sample performance of mutual fund predictors

Transcription

1 Out-of-sample performance of mutual fund predictors Christopher S. Jones Marshall School of Business University of Southern California Haitao Mo Ourso College of Business Louisiana State University First draft: August 2016 This version: September 2017 We are grateful to seminar participants at the University of Southern California, the University of Cincinnati, Louisiana State University, the 2017 Young Scholars Finance Consortium, the 2016 FMA annual meetings, and the 2017 SFS Cavalcade for helpful feedback and to Yong Chen, Timothy Chue, Richard Evans, Wayne Ferson, Juhani Linnainmaa, Oguz Ozbas, Jeff Pontiff, and Ravi Sastry for many useful comments. Data on hedge fund assets under management were provided by Matti Suominen, and corporate bond liquidity variables were provided by Eric Vogt. We are indebted to them both. All errors remain our own.

2 Out-of-sample performance of mutual fund predictors Abstract This study analyzes the out-of-sample performance of a variety of variables shown in prior work to forecast future mutual fund alphas. Overall, we find that the degree of predictability, as measured by alpha spreads from quintile sorts or by cross-sectional regression slopes, falls by at least half out of sample. We conclude that these shrinking spreads are in part consistent with data snooping biases but that a more important driver may be changes in the degree of market efficiency. We find little evidence that the declines are the result of learning by investors or fund managers. Keywords: mutual funds, out-of-sample performance, market efficiency

3 1 Introduction A central question in the mutual fund literature is whether funds with positive alpha, net of fees and costs, can be distinguished, ex ante, from those with negative alpha. An affirmative answer to the question requires that some variable in the investor s information set be associated with future alphas. To this end, a number of studies have investigated the ability of various theoretically or intuitively motivated variables to predict future fund alphas, and a modest number of variables that appear to do so have now been found. Whether these alpha predictors continue to perform outside of their original samples is a question we answer in this paper. As discussed in the seminal work of McLean and Pontiff (2016), there are a number of reasons why the ability of a variable to predict out of sample may be different from its ability in sample. Data snooping, perhaps resulting from journals favoring statistically significant results, will lead to an upward bias in in-sample predictive performance, which will naturally decline out of sample. Investors or fund managers who learn about return predictors from the publication of academic studies may act in a way that decreases the predictive ability of those variables as their relevance becomes more widely known. Finally, market conditions that differ between the in-sample and out-of-sample periods, such as a change in the amout of capital devoted towards arbitrage activity, may result in a difference in the ability of all investors to find nonzero alphas. Out of these three channels, data snooping is perhaps the one most frequently discussed. While it has long been recognized as a problem in non-experimental fields (Leamer, 1978), the more recent proliferation of empirical finance research has led Harvey, Liu, and Zhu (2016) towards the provocative conclusion that most claimed research findings in financial economics are likely false. 1 In particular, as Lo and MacKinlay (1990) show, the analysis of characteristic-sorted portfolios of securities creates a particularly high potential for data snooping when the characteristic considered is selected based on previous analysis of the same data. We emphasize, as have these authors, the existence of data snooping does not imply intellectual dishonesty, as it is an almost unavoidable result of one study s results 1 A similar claim has in fact been made recently that most empirical research findings in science are most likely incorrect (Ioannidis, 2005). 1

4 having an influence on another study s analysis. The second channel is learning, which McLean and Pontiff (2016) find to be an important force behind the out-of-sample decline in equity return anomalies. Although our mutual fund setting differs in terms of how the learning channel would operate, we also consider the possibility that investors or mutual fund managers learn from the academic literature and modify their investment decisions or management styles based on published research. There are several mechanisms by which learning may reduce the ability of some characteristic to predict future alphas. One is that investors (or their advisors) benefit from published research by becoming better able to identify funds with positive and negative alphas, leading to an increase in flows towards positive alpha funds and away from negative alpha funds. argued by Berk and Green (2004), declining returns to scale imply that these flows push the alphas of both types of funds towards zero. Alternatively, but with similar effect, fund companies respond to changes in investor interest by raising the management fees of good funds and lowering the fees of bad funds. In another version of the learning channel, variables lose their ability to predict future alphas because of manipulation by fund managers. As As a hypothetical example, consider the active share measure of Cremers and Petajisto (2009), which is positively related to fund alphas in their sample, a result that has garnered significant attention among mutual fund managers and investors. 2 If investors become aware of this relation and direct their investments towards high active share managers, who are on average skilled, then unskilled managers with low active share may attempt to mimic skilled managers by switching into a high active share strategy, even if they view that new strategy as having inferior performance. Over time, the measure becomes more contaminated by this pooling of high and low quality managers, causing active share to lose its effectiveness as an alpha predictor. The final channel we consider is that trends or fluctuations in market efficiency cause alpha predictors to vary over time in their effectiveness. As the aggregate amount of arbitrage activity increases, for instance, the alphas on individual assets might be expected to shrink towards zero, with mutual fund alphas shrinking similarly as a result. Chordia, 2 Advisers Using Active Share Hear From Critics. Concept first popularized in 2009 to identify active managers who were really closet indexers Maxey, Daisy. Wall Street Journal (Online), New York, NY, May 6,

5 Subrahmanyam, and Tong (2014) investigate how the returns to asset pricing anomalies (momentum, value, accruals, etc.) are related to various measures of arbitrage activity. For a number of anomalies, they find that the average return of a characteristic-sorted spread portfolio is reduced substantially by an increase in short interest, the assets under management of hedge funds, and aggregate share turnover. Jones and Pomorski (2016) analyze the evolution of the positive short-run autocorrelation in market returns identified by Lo and MacKinlay (1988). They showed that its disappearance began in the 1970s, far before the publication of the Lo and MacKinlay paper but coincident with the advent of electronic brokerages and accompanying the rise in trading volume. We follow this work by analyzing how arbitrage activity proxies are related to abnormal fund returns. Each of these three channels has a distinct prediction for how the performance of an alpha predictor should evolve over time. Data snooping biases should be evident only during the in-sample period and should therefore result in an immediate drop in alpha spreads as soon as the original study s sample ends. Learning presumably requires more time to have an impact, perhaps only starting to occur after the original study is published in an academic journal. We would therefore expect alpha spreads to decline gradually following the original study s sample as more investors and fund managers become aware of the value of a predictor. What is similar about both of these explanations is that they ascribe importance to the end of the original study s sample or its publication date. In contrast, changes in mutual fund alphas due to time variation in market efficiency or arbitrage activity should be independent of sample periods or publication dates. Alpha spreads could still be lower out of sample under this explanation, but the reason would simply be that equity mispricing has diminished over time. Our first main result is that alpha predictors largely fail, out of sample, to replicate their in-sample success. At least half of the alpha spread generated by predictors proposed in the literature disappears out of sample. In some specification, the decline is significantly higher. By the end of our sample period in January 2015, the only equity fund predictor that appears to retain a significant and robust ability to forecast future equity fund alphas is the expense ratio. Thus, to a potential mutual fund investor, advice from the academic literature on what mutual funds to hold is at best only moderately useful. 3

6 We find little support for the learning hypothesis, observing no significant evidence that alpha spreads diminish for a predictor as it becomes further away from the end of the sample period used in the original study. Following McLean and Pontiff (2016), we first compare alpha spreads in the period between the sample end and publication dates to those in the post-publication period. In no case do we observe a significant difference between these two periods, and point estimates are positive as well as negative. Similarly, we find no evidence that alpha spreads decline gradually following the end of the original sample. Additionally, implications of the learning hypothesis for fund flows and management fees are also unsupported. If academic studies foster learning by mutual fund investors and management, then the resulting changes in beliefs about fund manager skill should trigger, following the logic of Berk and Green (2004), greater inflows into higher quality funds and/or increased fees charged by those funds. We investigate the relation between alpha predictors and flows and find a significant positive relation (i.e., outperforming funds receiving more inflows), but it does not change out-of-sample or post-publication. The relation between alpha predictors and fees is generally negative (i.e., outperforming funds charging lower fees), becoming significantly more negative out-of-sample. This is the opposite of what learning under the Berk and Green model would imply, further undermining learning as the reason for declining alpha spreads. Finally, we examine the relation between alphas and arbitrage activity. Here we find strong evidence that higher levels of arbitrage are associated with smaller alpha spreads. Part of this effect is consistent with the view that market efficiency has in general increased over the sample we examine as the result of a trend towards greater arbitrage activity. However, the relation between arbitrage and alphas generally holds for detrended arbitrage activity measures as well, suggesting that cyclical variation in arbitrage forces is important as well. When we include both out-of-sample indicators and arbitrage proxies within the same specification, arbitrage activity usually retains economic and statistical significance. With controls for arbitrage activity in place, out-of-sample effects are generally reduced, retaining statistical significance in only a few cases. Overall, it appears that poor out-of-sample performance is largely driven by changes in market efficiency, though data snooping may be 4

7 an important secondary determinant. We confirm that the association between declining performance of fund alpha predictors and greater market efficiency holds among corporate bond mutual funds as well. Since the alpha predictors analyzed in the literature were almost exclusively popularized on the basis of their performance in equity funds, corporate bond funds provide another sample that is presumably unaffected by data snooping biases. Though this asset class differs, it is plausible that some of the same characteristics associated with strong equity fund performance translate to bond funds. Confirming this would provide additional evidence against data snooping as the primary explanation of the strong in-sample performance of mutual fund alpha predictors. Though the set of predictors we are able to analyze is limited by the unavailability of bond fund holdings data, we find strong evidence that most alpha predictors are also effective for corporate bond funds over our entire sample period. Again, however, predictive performance appears to depend strongly on measures of arbitrage activity, in this case measured from the bond market, with higher activity associated with lower alpha spreads. As with equity funds, upward trends in arbitrage activity mean that these spreads are likely much smaller at the end of our sample than they are at the start. Unlike equity funds, those smaller spreads remain statistically significant in most cases. Our paper relates to several strands of the finance literature. Most clearly, we follow a significant literature on mutual fund performance prediction, which we review in Section II. Our paper is also closely related to work on the persistence of asset pricing anomalies. In addition to the work of Chordia, Subrahmanyam, and Tong (2014), a number of studies have explored how the expected returns of anomaly strategies have held up since the original papers documenting their existence. Schwert (2003) shows, for example, that the size anomaly largely faded following its discovery in the early 1980s. Jones and Pomorski (2016) analyze several anomalies from the perspective of a Bayesian investor, finding that out-ofsample investment performance is improved by allowing, ex ante, for the possibility that the anomaly return may diminish over time. Finally, in the study most closely related to the current one, and whose framework we largely adopt, McLean and Pontiff (2016) examine the out-of-sample performance of 97 variables shown to predict equity returns and find a 5

8 substantial decline relative to in sample averages. Our work is also related to a large literature on data snooping in economics and finance. It has long been understood that specification searches, undertaken either by the individual researcher or collectively by many, may result in violations of the assumptions necessary for the validity of the statistical methods used. While a number of solutions have been proposed (Lo and MacKinlay, 1990; White, 2000; Harvey, Liu, and Zhu, 2016) for adjusting inference for such searching, these methods generally require assumptions about the nature of the search undertaken. Our own approach to dealing with the problem, which is simply to wait until additional data are available, is decidedly low tech, but it has the advantage of being relatively free of assumptions. We believe that our results show how poor out-of-sample performance need not be the result of data snooping if market conditions are changing over time, a conclusion that may have validity in other contexts as well. Finally, this paper is related to work seeking to characterize the distribution of mutual fund managers. This work has generally found that the average fund alpha, net of fees and expenses, is negative, but there is substantial disagreement over the fraction of managers that do provide positive alphas. In the Bayesian framework of Jones and Shanken (2005), the bootstrap analysis of Kosowski, Timmermann, Wermers, and White (2006), and the MLE approach taken by Harvey and Liu (2017), a substantial minority of funds do appear to outperform. In contrast, Fama and French (2010) conclude that funds with positive alphas are highly unusual, while Chen and Ferson (2015) find that they are largely nonexistent. Interestingly, in a similar study, Barras, Scaillet, and Wermers (2010) find a substantial fraction of funds with positive alphas prior to 1996, but find that almost no positive alphas existed by Our finding that arbitrage activity is associated with declining alpha spreads is consistent with this observation. In the following section we discuss the measurement of mutual fund alpha and the replication of mutual fund alpha predictors. In Section 3 we assess the out-of-sample performance of these predictors. We test the hypotheses on flows and expenses in Section 4 and examine the relation between alphas and arbitrage activity in Section 5. Section 6 assesses the performance of alpha predictors in corporate bond funds. Section 7 summarizes and concludes. 6

9 2 Mutual fund alpha predictors A variety of variables have been found to predict future fund performance. Most of the papers that document this predictability were published following the introduction of the CRSP Survivor-Bias-Free US Mutual Fund database, which first appeared in the work of Carhart (1997). The predictors that predate this database are the lagged one-year return which Hendricks, Patel, and Zeckhauser (1993) show to forecast future returns, and expense ratio and turnover which Elton, Gruber, Das, and Hlavka (1993) find negatively related to future returns. 3 Carhart (1997) also confirms the results of Hendricks, Patel, and Zeckhauser (1993). In addition, Carhart find that past alphas from his own four-factor model are more useful in forecasting future risk-adjusted performance. There is also some evidence that fund size is related to future performance. Chen, Hong, Huang, and Kubik (2004) find that larger funds perform worse, particularly for small-cap funds, while fund family size is positively related to performance. While the interpretation of this evidence is somewhat controversial (see Pástor, Stambaugh, and Taylor (2015)) in that it is not clear whether performance is an accurate reflection of managerial skill, the empirical relation between alphas and funds size appears relatively strong. The tendency of mutual funds to deviate from benchmark indexes also appears to be related to future performance. These deviations can be measured from fund holdings, as in the active share measure of Cremers and Petajisto (2009). They can also be measured from the R 2 of the fund s returns on one or more benchmarks, as in Amihud and Goyenko (2013). In both cases, funds that deviate more from their benchmarks perform better, suggesting that the greater conviction of managers who make larger stock-specific bets is associated with ability. 3 Literature reports mixed evidence on the relation between turnover and performance. While Elton, Gruber, Das, and Hlavka (1993) and Carhart (1997) find negative relation, studies such as Ippolito (1989), Wermers (2000), Edelen, Evans, and Kadlec (2007) and Grinblatt and Titman (1994) find no or positive relation. Mixed evidence may be attributable to different samples or different performance measures used. Since we use as our performance measure the Carhart four-factor adjusted alpha of reported net fund return, and we confirm its negative relation with turnover in the sample periods of Elton, Gruber, Das, and Hlavka (1993) and Carhart (1997), using the fund data that is either more comprehensive or comparable to that used in previous studies, we include turnover as a predictor in our analysis. 7

10 Several other measures are designed to identify managers that are more likely to display ability in stock selection. Kacperczyk, Sialm, and Zheng (2005) argue that fund managers may have informational advantages only in some industries, so that industry concentration indicates investment skill. Kacperczyk and Seru (2007) find that managers who appear to rely more on public information when making portfolio decisions tend to perform worse. Christoffersen and Sarkissian (2009) find evidence that funds in larger cities, where private information may be more accessible and knowledge spillovers are more likely, perform better than funds in smaller cities. Finally, funds whose holdings resemble other funds with strong performance records have been shown by Cohen, Coval, and Pástor (2005) to offer superior performance. Several performance measures have been derived by comparing the actual performance of the mutual fund to the performance of the portfolio formed on the basis of the most recent quarter-end fund holdings. The so called return gap, proposed by Kacperczyk, Sialm, and Zheng (2008), is defined as the difference between the actual fund returns and the holdingsbased returns. A measure of risk shifting can be obtained, as in Huang, Sialm, and Zhang (2011), by computing the difference between the volatility of the fund s actual returns to that of the holdings-based portfolio. These studies show that a higher return gap is positively related to future fund returns, while funds that exhibit greater risk shifting perform poorly. Both results are of a magnitude that is economically important and highly significant. Other performance predictors are based on asset market liquidity. Da, Gao, and Jagannathan (2010) find that funds that trade stocks with a high likelihood of informed trading (the PIN measure of Easley et al., 1996) perform better than those that do not. Liquidity at the aggregate level also appears to be important. Cao, Simin, and Wang (2013) find that funds that appear to time market liquidity by increasing market beta prior to periods of high liquidity on average perform better. Finally, several performance measures do not use additional fund-level characteristics but seek to predict future fund performance as a result of improvements in methodology. Mamaysky, Spiegel, and Zhang (2007) show how back testing can be used to better identify funds with nonzero alphas. Kacperczyk, Nieuwerburgh, and Veldkamp (2014) propose a skill index that combines both market timing and stock picking abilities and show that it strongly 8

11 forecasts future fund returns. In total, we find 20 different predictors from 17 papers. These 20 predictors represent the starting point of our analysis. 2.1 Mutual fund sample We obtain mutual fund returns (monthly) and fund characteristics such as expenses, total net assets (TNA), fund portfolio turnovers, and investment styles, from Center for Research in Security Prices Mutual Fund (CRSP MF) database, from November 1961 to January Fund returns are net of expenses but not of loads. Quarterly fund equity holdings data from 1980 to 2014 are from Thomson Reuters and, when we merge it with CRSP MF we use MFLINK. We exclude fixed income, international, money market, sector, index, and balanced funds, focusing on active US equity funds. 4 We subject the fund data to a number of screens to mitigate omission bias (Elton, Gruber, and Blake, 2001) and incubation and back-fill bias (Evans, 2010). We exclude observations prior to the first offer dates of funds, those for which the names of the funds are missing in the CRSP MF database, and those before the fund s TNA reaches $15 million. To prevent the impact of outliers when holdings data are used, we require a fund to hold at least 10 stocks to be eligible in our sample. We combine multiple share classes for each fund, focusing on the TNA-weighted aggregate share class. To construct various fund predictors, we obtain additional data: CRSP stock price data, constituents of nineteen indices from three families (S&P/Barra, Russell, and Wilshire) as used in Cremers and Petajisto (2009), daily fund returns data from CRSP MF, analysts recommendation data from IBES, and monthly/daily data of Fama-French-Carhart four factors from Kenneth French s website. For example, we need the data of index constituents to construct active shareness (Cremers and Petajisto, 2009), the analysts recommendation 4 We identify and remove index funds both by CRSP index fund flag and by searching the funds names with key words exchange-traded exchange traded etf dfa index inde indx inx idx dow jones ishare s&p s &p s& p s & p 500 WILSHIRE RUSSELL RUSS MSCI. US equity funds are defined as those with policy code CS; Weisenberger objective codes G, G-I, GCI, LTG, MCG, SCG, IEQ, I, I-G, SCG, AGG, G, G-S, S-G, GRO, LTG, I, I-S, IEQ, ING, GCI, G-I, G-I-S, G-S-I, I-G, I-G-S, I-S-G, S-G-I, S-I-G, GRI, MCG; SI objective codes AGG, GMC, GRI, GRO, ING, SCG; Lipper class codes EIEI, G, I, GI, LCCE, LCGE, LCVE, MCCE, MCGE, MCVE, MLCE, MLGE, MLVE, SCCE, SCGE, SCVE; or an average equity holding between 80% and 105% in the fund asset. 9

12 data to construct reliance on public information (Kacperczyk and Seru, 2007), and the daily fund returns to construct R-square (Amihud and Goyenko, 2013) and risk shifting (Huang, Sialm, and Zhang, 2011). There are 3652 unique funds in our final sample from November 1961 to January 2015 and the average number of months for a fund in our sample is Computing alpha spreads The ultimate object of interest in our study is mutual fund alpha. In order to control for standard risk exposures and to allow alphas to vary over time, we follow Carhart (1997) by computing alphas based on rolling window estimates of factor betas. Specifically, for each fund at each date, we use the previous 36 months to estimate the betas on the Fama and French (1993) and Carhart factors. We then use those betas to risk-adjust the current month s excess return. Given the lagged nature of the risk adjustment, we refer to the result as the ex post alpha. We label this quantity for fund i and time t as α it. A large part of our analysis is focused on the behavior of spreads in mutual fund alphas. These are formed by measuring the relation between fund alphas and some fund-level predictor x ijt, where the i subscript denotes the fund and j the predictor. For compactness of notation we specify the date as t but note that the predictor is always known as of the end of month t 1. In all cases, the predictor variable is defined such that high values are associated with good performance and low values with bad performance. This determination is made on the basis of the original paper in which the predictor was proposed, and we find no cases in which the sign of the prediction changes when we replicate the original paper s results. We measure alpha spreads in two ways, by sorting and by cross-sectional regression. Sort-based alphas are computed by sorting on x ijt and computing the difference between the equal weighted average alphas in the top and bottom quintiles. For shorthand we denote this spread as Q5-Q1. We also use cross-sectional regression to produce alpha spreads. In this approach, we simply run univariate monthly regressions of fund alphas on the predictor. The slope coefficients of these regressions constitute the CSR alpha spread. We feel that including both types of alpha spread is potentially important. Cross-sectional 10

13 regression maximizes dispersion in the predictor, which potentially allows funds with very large or very small values of the predictor to have more influence on the spread. This is appropriate if we believe that those funds are more heavily affected by whatever force underlies the effectiveness of that predictor. This is undesirable, however, if the relation between predictor and alpha is nonlinear. In this case, computing spreads based on extreme quintiles makes more sense. Given no guidance to choose one approach or the other, we include them both. 2.3 Criteria for including predictors As discussed in Section 2, we have identified 20 predictors that have been found in the mutual fund literature to forecast future fund performance. Unfortunately, four are impossible to replicate given that they rely on proprietary or hand collected data. 5 This leaves us with 16 predictors that we were able to analyze. Since our goal is to understand the out-of-sample performance of these predictors, we must first document in-sample performance. We follow McLean and Pontiff (2016) in requiring that predictors exhibit a degree of success in-sample that falls somewhat short of statistical significance. Specifically, we require that the average in-sample alpha spread have a t-statistic above For the extreme quintile (Q5-Q1) results, this t-statistic is computed on the basis of in-sample Q5-Q1 spreads. For the cross-sectional regression (CSR) results, the t-statistic is based on CSR spreads. Out of the 16 predictors we consider, 10 exceed the t-statistic threshold for both Q5-Q1 and CSR. One predictor meets the criteria for Q5-Q1 but not CSR, and two meet the criteria for CSR but not Q5-Q1. Three predictors have t-statistics below 1.4 for both Q5-Q1 and CSR. It is important to keep in mind that a failure to obtain a t-statistic of 1.4 or more does not indicate that the original study is incorrect, as many studies employ methods that are different than the ones we use here. In some studies, for instance, predictors are shown to be 5 In these cases obtaining data from the authors of the original studies does not help us since the datasets need to be extended beyond the original sample periods. 6 McLean and Pontiff (2016) require a t-statistic of 1.5 or more. We use a slightly lower cutoff so that two more predictors can be included. 11

14 significant only in the context of regressions in which other control variables are included. In addition, the data on which we base our analysis can be different from that used in prior work. The CRSP Mutual Fund Database, for example, has undergone several significant revisions. 3 Post-sample performance The central question we answer in this paper is whether mutual fund alpha predictors continue to work out of sample. We address this question in several ways. We begin by following McLean and Pontiff (2016) by analyzing how alpha spreads change following the end of the in-sample period. We then consider a related framework for analyzing predictability in a panel of mutual funds. 3.1 Predictor panel Our analysis begins by analyzing the panel of predictor-level alpha spreads, with one observation for each predictor in each month, where the alpha spread is defined either as the Q5-Q1 spread or the CSR coefficient. We examine these spreads in raw form or after scaling by the average in-sample value. Specifically, define A jt as the time-t alpha spread corresponding to predictor j. The scaled alpha spread is defined as A jt /Āj, where Āj is the average value of A jt over the in-sample period of predictor j. When averaged over the in-sample period, the average scaled alpha spread is equal to 1 by construction. When averaged over the out-ofsample period, the average scaled alpha spread is equal to the out-of-sample average divided by the in-sample average. Table 1 describes the data used in our predictor-level panel regressions. The table shows that of the 11 predictors included in our Q5-Q1 results, predictors had an average of 261 months of data in their sample period and 208 months of data past the end of their sample period. For the 12 predictors that were significant in the CSR-based analysis, the corresponding values were 259 and 195 months. Thus, the out-of-sample periods are comparable in length, on average, to the in-sample periods, though for some predictors the out-of-sample period is much shorter. 12

15 Both Q5-Q1 and CSR spreads show significant in-sample averages. For Q5-Q1 spreads, the average spread across all 11 predictors is 21.6 basis points per month, and the average t-statistic is The interpretation of CSR-based alpha spreads is not as straightforward. Recall that the CSR-based alpha spread is the slope of a cross-sectional regression. Because slopes computed in this way would not be comparable, for this table we normalize predictors to have zero mean and unit standard deviation. The average CSR alpha spread is the average coefficient on these normalized predictors. The 11.6 in-sample value implies, therefore, that a one standard deviation increase in a predictor increases average fund alpha by 11.6 bps. per month. The average t-statistic is again well above 2. Out-of-sample spreads are much lower. For Q5-Q1 spreads, the average spread drops to 7.1 bps. per month, with an average t-statistic of For CSR, the average spread drops from 11.6 to just 1.7. Average scaled spreads exhibit similar declines out of sample, with Q5-Q1 scaled spreads dropping by two thirds and CSR scaled spreads dropping by almost 90%. In fact, both for Q5-Q1 and CSR results, the average out-of-sample spread is lower than the average in-sample spread for every predictor analyzed. Unfortunately, hypothesis testing for average scaled spreads or differences in raw spreads is problematic due to the fact that we are averaging just 11 or 12 values, so small sample concerns are sizable. In addition, it would be difficult to adjust standard errors properly for cross-sectional dependence or differences in the samples available for each predictor. We therefore follow McLean and Pontiff (2016) by reassessing the out-of-sample performance of mutual fund predictors in a panel regression setting where all alpha predictors are analyzed jointly. In this panel we can control for both serial and cross-sectional dependence by clustering by predictor and date. Further, panel regression naturally takes into account that our predictors are observed over different time periods, with in-sample and out-of-sample periods of different lengths. Panel A of Table 2 begins by analyzing raw alpha spreads (in terms of basis points per month) in the regression A jt = a j + b D OOS jt + ɛ jt, where D OOS jt is an indicator variable that takes the value 1 if t is past the end of the original 13

16 sample used in the paper where predictor j was first proposed. We include predictor fixed effects (a j ) to control for persistent differences between predictors. 7 The top regressions for Q5-Q1 and CSR show the declines from the in-sample to out-ofsample periods, which are and -8.05, respectively. The value indicates that the average high minus low spread from quintile sorts declines by basis points per month following the end of the original study s sample. The value means that the impact of a one standard deviation increase in a predictor will reduce alphas by 8.05 basis points out of sample. These values are statistically significant but moderately smaller than those implied by Table 1. Scaled alphas, in Panel B, also exhibit significant deterioration out of sample, where the regression equation A jt /Āj = a + b D OOS jt + ɛ jt does not include fixed effects given the scaling of the dependent variable. Here, in addition to testing the null hypothesis of no deterioration in out-of-sample performance (b = 0), we can also easily test the hypothesis of complete disappearance of out-of-sample performance (b = 1). While no deterioration can always be rejected, complete disappearance can only be rejected for the Q5-Q1 results. As in McLean and Pontiff (2016), if investors or fund managers learn from academic studies, then the publication of those studies may lead to a further deterioration in alpha spreads. In Table 2, we investigate publication effects first by replacing the out-of-sample indicator D OOS jt with a post-publication indicator Djt P P. We then include both indicators, so that the post-publication indicator captures the incremental decline in alpha following publication, over and above the decline that occurs following the end of the in-sample period. Comparing the first and second regressions in each panel, it appears that post-publication declines are very similar to out-of-sample declines, which is inconsistent with learning. When both indicators are included, point estimates of the incremental impact of publication are negative but insignificant, though the economic magnitude is sizable in some cases. particular, for the CSR results for the scaled alphas, alpha spreads are estimated to decline 7 Results are similar without predictor fixed effects, with out-of-sample and post-publication indicators retaining statistical significance in all but one case. In 14

17 by approximately 50% in the out-of-sample but pre-publication period, declining another 25% in the post-publication sample. While this is a substantial effect, it is not statistically significant, perhaps due to collinearity of the two indicators. An alternative view of learning is that alpha spreads are likely to decline gradually, perhaps starting to decline before publication and continuing to decay as awareness of the study grows over time. We attempt to capture this possible dynamic by adding a regressor that captures gradual decay, max{t τ j, 0}, where τ j is the last in-sample period of predictor j. Regressions including this OOS decay variable are the last ones in each panel. In short, they show no evidence of patterns that might be implied by gradual learning. For the scaled alpha regressions, the positive point estimates imply alpha spreads that increase rather than decrease as the predictor moves further out of sample. In all cases the coefficient on the OOS decay variable is insignificant, and including it has little effect on the size or significance of the OOS indicator. In untabulated results, we examine whether out-of-sample and post-publication declines in alpha spreads are mostly from the long or short side of the quintile portfolio-based strategy. Similar to the limits to arbitrage literature (Shleifer and Vishny, 1997; Pontiff, 1996/2006), it is easier for an investor learning about alpha predictability to act on that information by investing into funds in the long portfolio, as short selling open end mutual funds is not feasible. When we analyze the Q5 and Q1 portfolios separately, there appears to be some tendency of raw alphas to diminish more on the long side, though the differences between the long and short portfolios are insignificant. When we analyze scaled alphas, however, the declines in long-side and short-side alphas are almost identical. We also segment our mutual fund sample by expense ratios and by loads, hypothesizing that an investor who learns about mutual fund predictability would be more likely to exploit that information in low cost and/or no-load funds. We find that all classes of funds appear to exhibit out-of-sample and post-publication declines in alpha. There appears to be little relation between loads, fees, and the decline in alpha that we observed. OOS and post- 15

18 publication effects are about the same between load and no-load funds. Low-expense funds exhibit greater declines in alpha spreads using the CSR approach, but smaller declines with quintile sorts. Overall, these results reinforce the informal ones in Table 1. Out-of-sample spreads in alphas are much smaller than their in-sample counterparts, suggesting that the economic value of the average predictor is modest at best. Furthermore, the lack of significant further deterioration of alpha spreads during the out-of-sample period suggests that learning by investors or fund managers is not the main cause of the decline in spreads and points instead to data snooping as an explanation more consistent with poor out-of-sample performance. 3.2 Fund panel We also analyze the out-of-sample and post-publication predictability of mutual fund alphas in a mutual fund-level panel regression. We do so for several reasons. One is that individual funds are likely to display substantial variation in alpha predictors relative to fund averages, such as those created as the result of a quintile sort. With larger variation in expected alpha, we may better be able to detect when realized alphas fall short of prior averages, for instance due to a publication effect. In examining individual funds, we will also find cases where a fund displays multiple characteristics associated with good performance. This will also increase predicted alpha and again give us the best chance to detect any changes in the relationship between predicted and realized alphas. A final reason to pursue a fund-level analysis is that it will allow us to examine fund expense ratios and asset growth in a natural way. We leave that analysis for Section 4. To understand our fund-level regressions, first imagine the regression that we would run if we were using just a single alpha predictor: α it = a i + a t + b S it + c S it D OOS t + d S it D P P t + ɛ it In this regression, S it represents a score computed based on fund i s date-t value of the predictor (which is assumed known prior to date t). As an example, the score might take the value +1 if the fund was in the bottom quintile based on the predictor and -1 if it was in the top quintile. As before, D OOS t and D P P t are out-of-sample and post-publication indicator 16

19 variables. Thus, c = 0 corresponds to the case in which the out-of-sample impact of the predictor is unchanged from its in-sample impact, while d = 0 indicates the absence of a publication effect. Note that our regression omits the direct effects of D OOS t and Dt P P, which are included only when interacted with the score variable. This is because our regressions include time fixed effects (a t ), which will absorb the two indicators and any other variable that exhibits variation only over time. Our results are nearly unchanged when we remove time fixed effects, whether or not the direct effects of the two indicators are included. With multiple predictors, we must make an assumption about how predictability aggregates. We do so by simply assuming that alphas are related to the average score across all predictors. If there are N predictors, we assume that α it = a i + a t + b 1 N S ijt + c 1 N S ijt Djt OOS + d 1 N N N j=1 j=1 N j=1 S ijt D P P jt + ɛ it, (1) where the j coefficient denotes the predictor. Under this specification, it remains true that c = 0 implies no deterioration in predictive ability out of sample and that d = 0 implies no decline following publication. We consider two different specifications of the score variable S ijt. One is the extreme quintile score described above, where funds in the bottom predictor quintile receive a score of +1 and funds in the top quintile receive a score of -1. This corresponds roughly to the Q5-Q1 approach in our earlier results. The other score is equal to the percentile of the predictor of a given fund, rescaled to lie between -1 and +1, within the contemporaneous cross section. This is more related to, but not analogous to, the earlier CSR results. We have experimented with other definitions of the score and find little effect on our results. The results, shown in Table 3, appear mixed. With fund fixed effects, out-of-sample and post-publication effects are both significant, consistent with the conclusions from the predictor panel analysis. Furthermore, the effects are large in terms of economic magnitude. For example, in the second column of Panel A, the coefficient on Average Score OOS Indicator (the c coefficient from (1)) implies that a fund whose predictors average in the 75th percentile (with a score of 0.5) will see its alpha fall by = 21 basis points per month, or 2.5% annualized, during the out-of-sample period. 17

20 As with the predictor panel, when both post-sample and post-publication effects are included, no significant post-publication effect is observed. This may be related to collinearity in the fund panel, however, as the post-sample effect (though much larger in magnitude) is also insignificant when both variables are included. Finally, the regressions with fund fixed effects do not show a very significant baseline relation between average score and performance, as the Average Score (b) coefficient is at best significant at the 10% level. Presumably this is due to the lack of time variation in average score, which causes its effect to be absorbed by the fund fixed effects. When we do not include fund fixed effects, these baseline effects are highly significant and economically large. For example, in the first regression without fixed effects reported in Panel A, the 31.0 coefficient on Average Score implies that a fund whose predictors are all in the 75th percentile (with a score of 0.5) will on average have an alpha that is 31 bps. per month higher than a fund whose predictors are all in the 25th percentile (with a score of -0.5). This is close to a 4% spread on an annualized basis. However, dropping fund fixed effects eliminates the significance of the out-of-sample and publication effects. This suggests that post-sample and post-publication changes in fund alphas are primarily a within-fund phenomenon, and that there exist persistent fund-level differences not captured by our average score variable that must be removed before out-ofsample and post-publication declines can be observed. What could these persistent fund-level differences be? In the model of Berk and Green (2004), equities are potentially mispriced, while the market for fund managers is efficient. The resulting equilibrium forces continuously push alphas (which are net of fees and costs) back towards zero via changes in fund size and fees. At the other extreme, in perhaps a more naive model, equities could be fairly priced while actively managed funds exist despite their uselessness. In this case, alpha will simply be determined by the fees and costs incurred by the fund. If those fees and costs are unaffected by sample end and publication dates, then the fund s alpha will be unaffected as well. To understand the role of fund fees in these results, we include a third set of regressions in which we add the fund s expense ratio back to the net alpha measure used previously 18

21 to construct a measure of gross alpha. 8 When we run these regressions, still without fund fixed effects, we restore the significant out-of-sample decline in alpha generation. The publication effect is stronger than it was using net alphas, but it is still only marginally significant. We conclude that fund alphas do not appear to diminish out of sample in regressions without fund fixed effects because there is a component of alpha that is driven by fees, and that this component does not show any tendency to diminish in the post-sample period. This fee component changes little over time and is absorbed by fund fixed effects. The regression with fixed effects therefore captures the non-fee component of alpha, which may be more related to skill, and which does seem to decrease in magnitude over time. 4 Flows and expenses The previous section showed evidence that variables that forecast future mutual fund alphas lose some of their predictive ability out of sample. While the evidence was insignificant, some point estimates implied substantial additional differences between the out-of-sample but pre-publication period and the post-publication period. As suggested by McLean and Pontiff (2016), these differences raise the possibility that market participants are learning about the drivers of alpha from the results of published studies. The idea that learning may affect expected return has been discussed in several papers investigating asset pricing anomalies, most recently Jones and Pomorski (2016) and McLean and Pontiff (2016). In these papers, the mechanism of how learning affects asset prices is well understood learning from an academic study that a set of securities offers positive alpha induces greater investment to those securities, which raises their prices and eliminates the alpha. In the mutual fund setting, the effects of learning are likely not as straightforward. One set of predictions is implied by the model of Berk and Green (2004). In that model, an increase in the predicted fund alpha would either drive the fund to increase its management fee or investors to increase their holdings of the fund. The former directly reduces alphas, which are measured net of expenses. The latter does so indirectly as the result of assumed 8 Ideally, our measure of gross alpha would also add back the transactions costs incurred by the fund, but these cannot be measured directly and would be difficult to estimate with any accuracy. 19

22 diseconomies of scale. Either way, if learning causes all parties to conclude that a fund has positive alpha, then the future performance of that fund (net of fees) should be expected to decrease. While the effects of raising fees are straightforward, there is some disagreement in the literature about whether fund inflows are likely to have a significant effect on the typical fund s alpha generation ability. Pástor, Stambaugh, and Taylor (2015) find inconclusive evidence and summarize conflicting results found elsewhere in the literature. Thus, a finding that the out-of-sample decay in alpha is associated with changing flow behavior would provide new support for a somewhat unsubstantiated hypothesis. It is also possible that learning may affect predictability through a channel unrelated to Berk and Green (2004). If the publication of academic research does affect mutual fund flows or allow some managers to raise fees, then fund managers may use that research to better mimic fund types that have been shown to offer better performance on average. For example, a fund manager with a low active share, which Cremers and Petajisto (2009) show signals to have poor performance, may decide to switch to a high active share strategy, even though his ability to generate good performance is unchanged. By doing so, this and other managers corrupt the active share measure, reducing its predictive power. Since they only have an incentive to do so if it generates inflows or allows them to raise fees, this hypothesis may be observationally equivalent to the one based on the Berk and Green model. We first investigate the relation between flows and alpha predictors in a framework that is similar to the fund panel analyzed in Section 3.2. We define new money growth (NMG it ) as the net inflows to the fund within a month divided by the total net assets of the fund at the end of the prior month. We regress this on the average score variable, the average score interacted with post-publication indicators, and the average score interacted with a time trend: NMG it = a i + a t + b 1 N N S ijt + c 1 N j=1 N j=1 S ijt D P P jt + ɛ it (2) Both time and fund fixed effects are included, though neither has a substantial effect on these results. The findings, in Table 4, are easily summarized. There is a strong and significant tendency 20