Hedge Funds: The Good, the Bad, and the Lucky

Transcription

1 Hedge Funds: The Good, the Bad, and the Lucky Yong Chen Texas A&M University Michael Cliff Analysis Group Haibei Zhao Georgia State University August 5, 2015 * We are grateful to Vikas Agarwal, Charles Cao, Heber Farnsworth, Wayne Ferson, Will Goetzmann, Feng Guo, Michael Halling, Petri Jylha, Greg Kadlec, Andrew Karolyi, Robert Kieschnick, Bing Liang, Andrew Lo, Hugues Pirotte, Jeffrey Pontiff, Zheng Sun, Josef Zechner, Harold Zhang, and seminar/conference participants at Cornerstone Research, the Institute for Quantitative Asset Management (IQAM), Pennsylvania State University, Shanghai University of Finance and Economics, Texas A&M University, University of North Carolina at Chapel Hill, University of Texas at Dallas, University of Virginia, Vienna University of Economics and Business, Virginia Tech, VU University of Amsterdam, the FBE 654 Asset Pricing class at University of Southern California, the 4th NYSE Euronext Hedge Fund Conference in Paris, and the 2015 Financial Intermediation Research Society (FIRS) Conference for helpful comments. The paper was previously circulated under the title Hedge Funds: The Good, the (Not-so) Bad, and the Ugly. All remaining errors are ours alone. The views expressed in this article do not necessarily represent those of Analysis Group, Inc. Mays Business School, Texas A&M University, College Station, TX 77843; ychen@mays.tamu.edu. Analysis Group, Washington, DC 20006; mike.cliff@analysisgroup.com. Robinson College of Business, Georgia State University, Atlanta, GA 30303; hzhao5@gsu.edu. Electronic copy available at:

2 Hedge Funds: The Good, the Bad, and the Lucky August 5, 2015 Abstract We develop a new method to evaluate hedge fund skill in the presence of luck. In the cross section, by assuming each fund comes from one of several skill groups, we estimate the number of groups, the fraction of each group, and the mean and variability of skill within each group. Our method allows luck to affect both unskilled and skilled funds. At the individual fund level, we propose a performance measure that combines the fund s estimated alpha with the crosssectional distribution of fund skill. In out-of-sample tests, a strategy using our measure outperforms those using estimated alpha and t-statistic. JEL Classification: C13, G11, G23 Keywords: Hedge funds, performance evaluation, EM algorithm, performance persistence Electronic copy available at:

3 1. Introduction The past two decades witnessed hedge funds, with less regulatory rigidity and more trading flexibility, grow into an important investment vehicle. Tremendous interest has emerged from both academics and practitioners in assessing whether hedge funds add value for investors. Indeed, a growing literature examines hedge fund performance from different angles. So far, there is no consensus about whether an average hedge fund can add value. 1 However, at the individual hedge fund level, prior studies have shown strong evidence of the existence and heterogeneity of fund skill. 2 Two important questions naturally arise from these findings. First, how many hedge funds have enough skill to add value? Second, how can we identify skilled hedge funds? These questions motivate our study. One major challenge in addressing the above questions is that fund managers true skill is not observable. 3 In practice, researchers typically measure skill with estimated performance measures such as alpha. Consequently, due to inaccuracies associated with the estimates, a zeroskill manager may be lucky and exhibit superior performance, while a good manager may be unlucky and show inferior performance. Prior studies have proposed several methods to control for the effect of luck on inference about fund skill in multiple hypothesis testing. Kosowski, Timmermann, White, and Wermers (2006) and Fama and French (2010) use bootstrap 1 For example, Ackermann, McEnally, and Ravenscraft (1999), Brown, Goetzmann, and Ibbotson (1999), and Liang (1999) show that in aggregate, hedge funds realize positive risk-adjusted performance. However, Griffin and Xu (2009) find little evidence that hedge funds, on average, deliver abnormal performance. 2 Kosowski, Naik, and Teo (2007) show that the superior performance of top hedge funds cannot be attributed to pure randomness. Several papers also investigate the cross-sectional relationship between hedge fund performance and fund characteristics. Aragon (2007) finds that hedge funds with stricter redemption restrictions offer higher returns. Agarwal, Daniel, and Naik (2009) find that hedge fund performance is positively related to fund managers incentives and discretion. Li, Zhang, and Zhao (2010) link hedge fund performance to fund managers educational background and work experience. Titman and Tiu (2011) show that hedge funds with lower R-squares against systematic factors realize better future performance. Sun, Wang, and Zheng (2012) find that hedge funds with different return patterns from peer funds are associated with better subsequent performance. 3 We use fund and fund manager interchangeably in this paper. 1

4 simulations to infer skill among mutual funds. Barras, Scaillet, and Wermers (2010) apply a false discovery approach to mutual funds and detect skill in only a small fraction of funds. 4 In this paper, we develop a new method to estimate the prevalence of fund skill and apply it to a sample of hedge funds. Our approach is based on the assumption that the skill of each fund, characterized by its alpha α i, comes from one of several skill groups with mean alpha μ j and variability of alpha σ j. 5 As a stylized example, we can view funds as being Good (say μ G = 3% per year), Neutral (μ N = 0%), or Bad (e.g., μ B = 2% per year), though our approach can accommodate more than three skill groups. Accordingly, the observed cross-sectional distribution of fund alphas is a mixture of the three distributions. Figure 1 illustrates the mixture of these distributions. Given an observed distribution of alphas (dashed line), our estimation algorithm identifies the three sub-distributions (solid lines) that match the cross-sectional distribution when combined together. 6 The shape of the crosssectional distribution dictates the number of skill groups and their distributional parameters. As in Fama and French (2010), skill gives rise to fat tails in the distribution of alpha. As shown in the figure, funds in the Good skill group can have bad realized performance. We use a modified Expectation-Maximization (EM) algorithm to estimate the average skill (μ j ), the variability of skill (σ j ), and the size of the group (π j ) for each skill group j. These parameter estimates not only describe the cross-sectional distribution of alphas across different 4 See Ferson and Chen (2015) for a refinement and generalization of the Barras et al. method, by using more of the structure of the model suggested by Barras et al. 5 We focus on net-of-fee returns. Hence, we adopt an investor s perspective in asking whether the manager can earn a gross return that is sufficient to cover costs. 6 In this example, we set π G = 0.2, π N = 0.7, π B = 0.1; μ G = 2%, μ N = 0, μ B = 2%, and σ j = 0.7% for all groups. This simple example does not incorporate estimation errors in alpha that are introduced later in the paper. All the parameter estimates in our empirical analysis incorporate the effects of estimation errors. 2

5 skill groups, but also provides useful information to make inference about skill of individual funds. In practice, fund alphas are estimated with noises. However, we show that the information from the cross section can be combined with estimated alphas to make more accurate inference for individual funds. At the individual fund level, we construct a new performance measure the conditional probability a fund comes from the highest-skilled group. This performance measure incorporates both a fund s estimated alpha and the information about the cross-sectional fund skill. When estimated alpha is very noisy with large estimation error, the measure relies more on the crosssectional information as opposed to estimated alpha. On the other hand, if estimated alpha has a high precision, it receives a great weight in the performance measure. This performance measure has advantages over the conventional way of using the t- statistic to adjust for the precision of estimated alpha. Though the t-statistic tells how strongly we can reject the null hypothesis of zero skill, it does not identify which funds are more skilled. For example, a fund with a t-statistic of 3.0 does not necessarily have more skill than another fund with a t-statistic of 2.0. This is because the t-statistic, as the product of estimated alpha and its precision, does not differentiate between these two components. In contrast, by weighting the fund s estimated alpha and the prior information about the cross section, our approach incorporates the magnitude of estimated alpha based on its precision and provides a ranking of fund skill. Having such a ranking is important for investors (like funds of hedge funds) facing capital constraints that limit the number of funds in which they can invest. In our empirical analysis, we employ a sample of 8,695 hedge funds by merging two major hedge fund databases Lipper TASS and Hedge Fund Research over the period of We use the Fung and Hsieh (2004) seven-factor model to estimate alpha from 3

6 historical fund returns, and we consider alternative models for robustness. We mitigate hedge fund data biases and propose a new way to correct backfill bias. Empirically, we find that a mixture of four skill groups best fits the empirical distribution of actual fund performance (compared with other numbers of skill groups), which we refer to as Excellent, Good, Neutral, and Bad. The first two groups have positive mean alpha, including 9% excellent funds with μ = 0.72%/month and 38% good funds with μ = 0.35%/month. Meanwhile, 43% of the fund are neutral funds with zero-alpha after fees (i.e., having skill just enough to cover their fees), and 9% are deemed as bad funds with μ = 0.80%/month. This finding is consistent with the notion that hedge fund skill tends to be heterogeneous. This result also depicts a remarkably different picture about hedge fund skill than the limited evidence of skill that prior studies find for mutual funds (e.g., Barras, Scaillet, and Wermers, 2010; Fama and French, 2010). To identify superior individual funds, we use the performance measure that computes the conditional probability a fund comes from each skill group, by combining the fund s estimated alpha with parameter estimates for the cross section. Specifically, in each month we form four portfolios based on funds conditional probabilities of being excellent, good, neutral, and bad estimated from the previous 24 months. Then, we examine out-of-sample performance of these monthly-rebalanced portfolios. We find that the portfolio of the predicted excellent funds (i.e., those with the greatest likelihood of being excellent) subsequently realize high alpha over a long horizon. In fact, the alpha spread between the predicted excellent and the predicted bad portfolios remains significantly positive even three years post-formation. This suggests that our performance measure is able to detect skill. Further, when comparing the investment value of our approach with alternative strategies based on past estimated alpha and its t-statistic, we find that our approach outperforms those competing strategies in out-of-sample tests. 4

7 Our paper makes several contributions to the literature. First, as an alternative to the false discovery method applied in Barras, Scaillet, and Wermers (2010), we use the EM algorithm to make inferences about mixture distributions of fund skill. Barras, Scaillet, and Wermers (2010) allow luck to affect zero-skill funds (i.e., false discoveries ), but by using a large test size (e.g., a size of 30%) they rule out the possibility that skilled fund can have zero-alpha due to bad luck. Our method allows luck to affect both zero-skill funds and skilled funds. The fact that we identify a larger fraction of skilled funds than simply counting statistically significant alphas suggests that it is important to consider imperfect test power. More importantly, for each individual fund, we construct a performance measure that combines the fund s own estimated alpha and the cross-sectional distribution of fund skill. Thus, the performance measure involves learning about skill from other funds. Jones and Shanken (2005) demonstrate how learning across funds affects the inference about the cross sectional distribution of fund skill. We extend their intuition to a setting of asset allocation across many funds with different skill. While Jones and Shanken (2005) consider one homogenous skill distribution, our method accommodates multiple skill groups, which is a necessary condition for comparing skill across funds. The rest of the paper proceeds as follows. In Section 2, we outline our approach to inferring fund skill. Section 3 describes the data. Section 4 presents the empirical results about the fractions of funds from different skill groups and fund performance persistence. Section 5 discusses additional analyses and robustness checks. Finally, Section 6 concludes. 5

8 2. Methodology In this section, we first lay out the general setup for inferring the characteristics of the skill groups and estimating the conditional probability that a fund belongs to the top skill group. Next, we relate our method to existing studies and discuss some properties of our performance measure. Finally, we describe our estimation procedure. Technical details about the estimation approach and simulations are provided in the Appendix The model We start by assuming that there is an unknown number J groups of funds with different skill levels. For each group j (j = 1, 2,, J), a representative fund is characterized by its alpha, which is assumed to follow a Normal distribution N(μ j, σ 2 j ), where μ j is the mean alpha in the group and σ j captures the dispersion in true skill across funds within the group. The clustering of performance within a group around the mean (μ j ) can be attributed to common investment styles (e.g., Brown and Goetzmann, 1997), while the variability σ j is driven by fund-specific traits ω i (e.g., infrastructure or trading intensity). Hence, the true alpha for manager i who belongs to group j is α i = μ j + ω i. True fund skill can vary through time due to changes in fund management or because the informational advantage that can generate alpha in one period erodes over time. We use π j to denote the fraction of the funds that come from skill group j, which is also the unconditional probability a fund belongs to the group. Thus, the sum of the group fractions equals one, i.e., J j=1 π j = 1. Consequently, the J sets of triples {μ j, σ j, π j } jointly define a composite distribution for fund i with the following density function: 6

9 J f(α i ) = j=1 π j φ (α i ; μ j, σ j ), (1) where α i denotes skill of the fund, φ(α i ; μ j, σ j ) is the Normal probability density with mean μ j and standard deviation σ j evaluated at α i. 7 The probability of observing α i in a population equals the weighted probability of observing α i in each group, weighted by that group s fraction in the population. As illustrated in Figure 1, the density function f(α i ) also describes the crosssectional distribution of fund skill. So far, we treat fund alpha α i as observable. However, empirical analysis in the literature routinely uses ordinary least squares (OLS) estimated alpha with a sample-specific estimation error e i for fund i. As a result, estimated alpha equals true alpha plus estimation error: α i = α i + e i = μ j + ω i + e i. Thus, estimated alpha satisfies the following density function: f(α i α i ) = φ(α i; α i, s i ), (2) where s i is the standard deviation of estimation error e i (i.e., the standard error of α i). The estimation error, e i, is assumed to follow a Normal distribution, which is a common assumption of OLS. By combining Equations (1) and (2) and marginalizing the joint distribution, we obtain the distribution of estimated alpha α i as follows: f(α i) = + f(α i α i )f(α i )dα i J + = j=1 π j φ(α i; α i, s i ) φ(α i ; μ j, σ j )dα i. (3) 7 We assume a Normal distribution for several reasons. First, the parameters of Normal distribution have clear economic meaning about the mean and variability of skill in our setting, as opposed to other distributions like t- distribution or inverse gamma. Second, according to the central limit theorem, Normal distribution seems natural to characterize true alpha as true alpha can be viewed as a sum of several random variables (e.g., fund-specific traits). Third, Normal distribution provides technical tractability to derive the iteration scheme used in our approach (see details in the Appendix A.1). Finally, even though each skill group is assumed to follow a Normal distribution, the composite distribution is non-normal with fat-tails, consistent with the empirical distribution for our data. 7

10 Next, evaluating the integral for each type j, we have: 8 J f(α i) = j=1 π j φ(α i; μ j, σ i,j ), where (σ i,j ) 2 = (s i ) 2 + (σ j ) 2. (4) This equation characterizes the density function for estimated alpha of fund i. Compared with Equation (1), the combined variance σ i,j incorporates two sources of variation in estimated alpha fund-specific estimation error s i and within-group variation σ j. Empirically, we find the average s i across funds (as shown in Table 2) to be of the same order of magnitude as σ j (as shown in Table 3). This suggests that the two sources of variations are roughly equally important. Given the importance of estimation error, making inferences based on estimated alpha alone without considering estimation error would lose a significant amount of information about fund skill. Figure 2 illustrates the effects of the two sources of variation from s i and σ j. Suppose we obtain a positive estimated alpha α for a fund. Here, both sources of variation in the estimated alpha affect our inference. The fund may come from the zero-skill group but exhibit a positive α due to ω and estimation error, i.e., (ω 0 + e 0 ). Alternatively, this fund may come from the positive-skill group, but a net negative (ω 1 + e 1 ) leads to an estimated alpha that is smaller than the group mean. Thus, we are uncertain exactly which group the fund comes from. As such, our method allows a zero-skill fund to appear to have positive estimated alpha, as well as allowing a skilled fund to have small (close to zero) estimated alpha. In other words, luck (e i ) can affect both unskilled and skilled funds in our model. 9 8 The detailed derivation of Equation (4) is provided in Equation (A6) of the Appendix A1. 9 This is different from Barras, Scaillet, and Wermers (2010), who assume perfect test power by using a large test size (e.g., 30%) and hence rule out the possibility that luck can affect skilled fund. 8

11 Once we have the estimates of {μ j, σ j, π j } (using the estimation procedure described below in Section 2.3), we can characterize f(α i) using by Equation (4) and make inference about each fund s skill. Specifically, we can make a probabilistic statement about how likely a fund is from each skill group, and use the probability as the basis for our performance measure. The conditional probability that the fund belongs to group j equals: 10 P j = Prob(fund i is from group j α i, σ i) = π j φ(α i; μ j, σ i,j )/f(α i). (5) For ease of illustration, we order the groups such that μ 1 < μ 2 < < μ J, and thus group J has the highest mean skill. From the perspective of investment practice, our focus is on the conditional probability that a fund comes from the group with highest mean skill, namely P J. The higher P J is, the more likely the fund has superior skill. The idea of this performance measure is as follows. When we make inference about fund i s skill, the estimation error of its alpha affects the relative importance between our prior based on the cross-sectional distribution and the fund s estimated alpha. In Equation (4), the total variation of estimated alpha α i is decomposed into fund-specific estimation error s i and within-group variation σ j. Hence, if fund alpha is estimated with high precision (i.e., when s i is small), our method assigns a relatively great weight to estimated alpha. On the other hand, if estimated alpha is of low precision, we rely less on estimated alpha and more on the prior. In the extreme case, when s i goes to infinity (i.e., no precision), the P j measure converges to π j (i.e., our prior knowledge). 10 To avoid notational clutter, we do not index P J for fund i, though it is a function of fund i s estimated alpha and standard error. 9

12 2.2. Relation of our method to existing studies Our estimation method builds on several existing studies. First, our paper is related to Kosowski, Timmermann, White, and Wermers (2006) and Fama and French (2010), who control for the effect of estimation error when inferring fund skill. However, unlike their studies that focus on whether the performance of top-performed managers comes from skill or luck, we estimate the probability that a fund comes from the highest-skilled group and our P J measure allows us to rank and compare skill across many funds. Second, our approach is closely related to Barras, Scaillet, and Wermers (2010), who apply the false discovery method to infer fund skill. Similar to their work, we consider the existence of multiple skill groups. However, unlike their study assuming perfect test power by using a large test size (e.g., a size of 30%), we allow for imperfect power that an estimated alpha close to zero could be a good (bad) manager with bad (good) luck. 11 More importantly, while their inference relies on the alpha t-statistic, we propose a fund-specific performance measure P J that accounts for estimated alpha and estimation error separately, and we show empirically in Section 4 that this separation leads to a significant improvement in out-of-sample investment value. Third, our measure P J depends on both a fund s own estimated alpha and the performance of other funds. As a result, learning about skill of other funds provides useful information to infer skill for a given fund. This is related to Jones and Shanken (2005), who study how learning across funds affects the inference about the cross-sectional distribution of fund skill in a Bayesian framework. Our study extends their idea of cross learning to a setting of asset 11 See Ferson and Chen (2015) for a study that considers imperfect power when inferring fund skill in the false discovery framework. 10

13 allocation across many funds. In out-of-sample tests, we show that the information about the cross sectional distribution of skill indeed has important implications for asset allocation across funds. Furthermore, our method accommodates a mixture of multiple skill groups, while theirs considers only one group. As shown below, our empirical analysis strongly rejects one skill group and favors a mixture of multiple skill groups in hedge funds. In fact, the existence of multiple skill groups is a necessary condition for obtaining a ranking of skill across funds The estimation procedure We now introduce the procedure to estimate the parameters with more technical details provided in the Appendix A.1. As explained above, our goal is to estimate a set of parameters {μ j, σ j, π j } that define the cross sectional distribution of true skill. To do so, we need to aggregate information about all individual funds estimated alphas. However, these estimated alphas are not true skill but estimates with noises. As a result, we refine our inference about each fund s skill by calculating the probability it belongs to each sub-distribution. As shown in equation (5), such probability estimates in turn depend on the cross sectional distribution of true skill. Thus, we have a simultaneous estimation problem. 12 Jones and Shanken (2005, p.545) state that although two examples demonstrate the substantial effects of learning on the allocation to a particular fund, the implications for asset allocation across funds remain unexplored. Since they assume one skill group, all funds come from the same distribution in their framework. As a result, learning about the cross-sectional distribution of skill provides the same information for all funds, and ranking based on posterior alpha would be the same as ranking based on the t-statistic. In our study, however, we consider multiple skill groups and allow the probability of belonging to each group to be different across funds. Therefore, learning from the cross sectional distribution is different for different funds. 11

14 We use a modified Expectation-Maximization (EM) algorithm to simultaneously estimate the cross sectional parameters {μ j, σ j, π j } and individual funds conditional probabilities. 13 The EM algorithm uses iterations of two steps. First, the expectation step calculates a conditional probability that α i is from group j, given previous estimates of the group parameters (or preset initial values in the case of the first iteration) and the funds estimated alpha and standard error. The expectation step refines our estimates of the fund s skill based on the cross-sectional information. Then, the maximization step aggregates the skill distribution of individual funds to obtain updated estimates of the cross sectional parameters. These two steps iterate until parameter estimates converge, and through the iteration process we solve the simultaneous estimation problem. In our setting, however, estimation errors in alphas complicate the EM algorithm, as the estimator σ j in the maximization step is highly non-linear without a closed-form solution. To overcome this difficulty, we modify the EM algorithm by deriving a separate iteration scheme to estimate σ j until convergence. To the best of our knowledge, our method is the first to incorporate estimation errors in the EM algorithm. The Appendix A.1 describes the details of the algorithm. Our method is flexible about the number of skill types. This is useful for inferring skill among entities (such as hedge funds) with highly heterogeneous skill. Empirically, we use the 13 The original EM algorithm developed by Dempster, Laird, and Rubin (1977) has been widely used in estimation and inference related to mixture distribution models and incomplete data. See, e.g., Wu (1983), Rudd (1991), McLachlan and Peel (2000), and McLachlan and Krishnan (2008). The EM algorithm, though powerful, was rarely used in finance research. Some exceptions are Kon (1984) who examines the distribution of daily stock returns and Asquith, Jones, and Kieschnick (1998) who study the heterogeneity of IPO returns. However, these previous studies examining stock returns do not deal with the complicating effects of estimation errors as we do, since in our setting fund alphas are not directly observed but estimated with estimation errors. 12

15 Bayesian Information Criterion (BIC) to identify the number of skill groups that best fit the actual data and confirm the existence of multiple skill groups in hedge funds. We perform sensitivity tests to validate our method. First, to address the concern that our parameter estimation of {μ j, σ j, π j } might be sensitive to the choice of initial values, we experiment with a grid of initial values and search for the global maxima of the likelihood function. Second, we run simulations in which we generate artificial datasets with known (population) group parameters, and compare the parameter estimates from the artificial datasets to their population counterparts. We find that our parameter estimates are reasonably close to the true parameters. Third, we run simulations in which we set values of alpha, and then rank the funds using our performance measure P J in comparison with alternative measures of estimated alpha and its t-statistic. The results show that the measure P J is better able to identify skilled funds than the alternative measures. Furthermore, an investment strategy based on our performance measure outperforms those based on the alternative measures. (In Section 4, we perform this comparison with actual hedge fund data in out-of-sample tests.) These results validate our estimation procedure. The Appendix A.2 provides the details of the simulations. 3. The Data 3.1. Hedge funds For our empirical analysis, we employ a large sample of hedge funds by merging data from the Lipper TASS and Hedge Fund Research (HFR) databases. Although these databases contain fund returns going back to as early as 1977, they do not retain information of defunct funds before 1994 and thus data in early years have survivorship bias (Fung and Hsieh, 2000; 13

16 Liang, 2000). To mitigate survivorship bias (see Brown, Goetzmann, Ibbotson, and Ross, 1992), we focus on the period from January 1994 onwards. Following the hedge fund literature, we only include funds that report net-of-fee returns on a monthly basis and have at least 24 months of returns. We exclude funds of funds from our analysis. Over the period January 1994 December 2011, our sample contains 8,695 funds, of which 3,076 are alive as of the end of the sample period and 5,619 are defunct funds. Table 1 reports summary statistics of fund returns. The average fund age in our sample is 73 months, slightly longer than six years. The mean (median) return is 0.70% (0.65%) per month or about 8.40% (7.80%) per year. At the top 25 th percentile, the mean return is 1.01% per month or about 12.12% per year. The average return volatility is 3.44% per month. Higher moments of fund returns suggest negative skewness and fat tails relative to a Normal distribution. Consistent with prior research, fund returns exhibit autocorrelation; the average first-order autocorrelation is Such autocorrelation is interpreted in prior studies as an indication of illiquidity holdings or return smoothing (e.g., Getmansky, Lo, and Makarov, 2004) Correcting backfill bias Hedge fund returns, as voluntarily reported, may have potential backfill bias (e.g., Fung and Hsieh, 2000; Liang, 2000). This bias arises as historical returns are often backfilled when new funds are added into a database. Since funds with good track records tend to join a database, neglecting backfilling generates an upward bias in average fund return. This is similar to the incubation bias (Evans, 2010). In the hedge fund literature, a typical treatment is to drop the first one or two years data. When estimating alpha for funds, it is common to require a certain number of observations (e.g., 24 months) to ensure test power. Thus, when early years are 14

17 truncated, only funds with relatively long track records remain in the analysis, which may introduce a survivorship bias. We propose a new way to correct backfill bias by simply adding an incubation dummy variable to the factor regression when estimating alpha. Specifically, the dummy takes a value of one for the backfill period, i.e. from the month when a fund s return becomes available in the data to the month when the fund joined the database. This dummy variable captures the incremental return during the backfill period, which we allow to vary for each fund. The advantages of our approach are that, first, we more accurately capture the actual backfill period than applying a same backfill period to all funds, and second, it retains a more complete fund history, which provides more information for the alpha estimates Factor model As hedge funds trade across different asset classes, a multi-factor model is often used to capture their risk exposures. We use the Fung and Hsieh (2004) seven-factor model as the benchmark model to estimate fund alpha. The seven factors include an equity market factor, a size factor, the change in the constant maturity yield of the ten-year Treasury, the change in the spread between Moody s Baa yield and the ten-year treasury, and three trend-following factors for bonds, currency, and commodities. 14 Our regression model has the following specification: r i,t = α + α 1 I(t t i,b ) + β f t + ε i,t, (6) 14 The bond, currency and commodity trend-following factors are constructed as portfolios of lookback straddle options on these assets (see Fung and Hsieh, 2001). The data on these factors are obtained from David Hsieh s website at 15

18 where r i,t is fund i s return in excess of the risk-free rate (proxied by the one-month T-bill rate) in month t, I( ) is an indicator function, t i,b is the month when fund i starts to report a database, and the vector f denotes the seven factors. The intercept α is the fund s alpha, and the coefficient on the dummy variable controls for potential backfill bias Estimated alpha Table 2 describes the result on estimated alpha, with and without controlling for backfill bias. As can be seen, controlling for backfill bias is important. Without the control, the average alpha for all funds is 0.39% per month, while with the control the average alpha shrinks to 0.11% per month. Hence, neglecting backfill bias would inflate alpha estimate by about threefold. Backfill bias appears particularly strong for defunct funds. The average alpha drops from 0.32% without the control to virtually zero after the adjustment. It turns out that the observed positive alpha for defunct funds is mostly concentrated in the backfill period. This result suggests that those funds that had no skill but joined the database after good incubation returns are more likely to fail. Thus, the superior performance observed for early months is likely to reflect a backfill bias. 15 In addition, live funds substantially outperform defunct funds, confirming the importance of controlling for the survivorship bias. The alpha estimates and their standard errors, after adjusting for backfill bias, are used in our later analysis as the main inputs to make inferences about hedge fund skill. 15 Aggarwal and Jorion (2010) find that, after adjusting for data biases, emerging hedge funds exhibit strong performance, which may reflect new funds incentive to perform well. 16

19 4. Empirical Results This section reports the main results from our empirical analysis. We first present the estimates of the parameters governing the different skill distributions. We then explore the performance persistence based on our performance measure. Finally, we compare the out-ofsample performance of an investment strategy based on our performance measure with that of alternative strategies based on estimated alpha and its t-statistic The distributions of fund skill As we do not observe the number of skill groups in data, we estimate our model using different values of J = 2, 3, 4, 5, 6, 7 and then compare model fit using the BIC. The BIC result indicates four skill groups in our sample. The BIC of 3, 4 and 5 groups are , and , respectively. The BIC of other numbers of groups are even greater than that of five groups. For robustness, we conduct a Likelihood Ratio test for the null H0: J = 3 (and J=5) against H1: J = 4, which rejects the null at the 1% significance level, suggesting that the case of four groups is significantly better than other numbers of groups in terms of fitting the data. Table 3 reports the parameter estimates. Since two skill groups have positive mean alpha, we refer to the groups as Excellent (μ =0.72% per month), Good (μ = 0.35%), Neutral (μ = 0% by assumption), and Bad (μ = 0.60%). The estimates of π j suggest the composition of funds is 9.3% excellent, 38.4% good, 43.0% neutral, and 9.3% bad. For investing in hedge funds in practice, the focus should be on the excellent group that has highest mean skill. The estimated variability of skill, i.e., σ j, of the excellent and bad groups is higher than that in the two middle groups, suggesting that funds with extreme skill have less in common and more specific to 17

20 themselves. This is consistent with Sun, Wang, and Zheng (2012) who find that hedge funds with distinctive return patterns from peer funds tend to have better performance. Equation (4) decomposes the total variation in fund skill into the within-group variation and fund-specific estimation error. We find that the estimated within-group variations σ j are in the same order of magnitude as the fund-specific estimation error as reported in Table 2, which suggests that both of them are important determinants of the total variation in estimated alphas. The fraction of funds with positive skill from our estimation is significantly higher than that judged by the t-statistic. Based on our method, 48% of the sample funds belong to either the Excellent or the Good group, whereas only 20.3% of the funds have t-statistic greater than 1.65 (see Table 2). The false discovery rate based on the Barras, Scaillet, and Wermers (2010) method is 3.7%, so after adjusting for the false discovery problem, the fraction of skilled funds inferred by the t-statistic would be even smaller (i.e., 20.3% 3.7%=16.6%) at the size of 10%. Thus, the result suggests that accounting for imperfect test power (i.e., allowing skilled funds to have bad luck) is important. We employ simulations to assess the statistical significance of our parameter estimates. We construct 1,000 artificial samples by drawing from the original sample with replacement. We estimate the parameters in each artificial sample, and then calculate standard errors as the standard deviation of the parameter estimates across the simulations. The bootstrapped standard errors, reported in parentheses Table 3, suggest that our parameters are estimated with reasonable precision. For instance, in the excellent group μ E, σ E, and π E are all more than three standard errors above zero. In sum, using a modified EM algorithm, we estimate skill distributions for hedge funds. Our results suggest that a significant portion of hedge funds have skill more than just covering 18

21 their fees. The finding is in sharp contrast with previous results (e.g., Barras, Scaillet, and Wermers, 2010; Fama and French, 2010) for mutual funds where few funds are found to deliver alpha after fees Performance persistence Now, we address another important question: Does superior performance persist in hedge funds? 16 In our setting, testing performance persistence is important because it can validate our grouping technique. If our grouping contains no information about fund skill, then there would be no persistence in performance identified by our method. Otherwise, if our method identifies skilled funds with superior performance, we expect a certain level of performance persistence. We examine performance persistence using portfolios formed in rolling windows. In each month starting from January 1996, we estimate the group parameters {μ j, σ j, π j } and each fund s performance measure P j from the previous 24 months. Then, we assign funds into one of four skill-based portfolios Excellent, Good, Neutral, or Bad. Specifically, each fund receives four conditional probabilities (i.e., P j s) corresponding to the different skill groups. In each of the rolling 24-month subperiods, for the fraction of each skill group estimated by our algorithm in that subperiod, we assign that fraction of funds that show the highest conditional probability of belonging to that group. 17 For example, if 10% of the funds are excellent in a subperiod according to our estimation procedure, then we assign the 10% of funds with the highest 16 Prior research shows mixed evidence about performance persistence in hedge funds. Brown, Goetzmann, and Ibbotson (1999) and Agarwal and Naik (2000) find little support for performance persistence, while Kosowski, Naik, and Teo (2007) and Jagannathan, Malakhov, and Novikov (2010) document significant evidence of performance persistence. 17 In reality, some years may have the number of skill groups different from four. However, assuming four groups for the whole sample facilitates the presentation of the results. For robustness, we allow the group number to change over time, and our inference about performance persistence is unchanged. 19

22 conditional probability of being excellent P J to the excellent group. This way, we assign funds into each group sequentially so that no fund will be assigned into multiple groups. As a result, four equal-weighted portfolios are formed with funds out of these groups. The portfolios are rebalanced monthly and held for different periods from three months to three years. The group parameters are re-estimated each month so that we only use information up to the month of portfolio formation. Funds that disappear during a holding period are included in the equalweighted portfolio until they disappear, and then their weights are reallocated to the remaining funds. In practice, it may not be realistic to immediately invest into these portfolios after formation, so we insert a one-month waiting window between the formation period and the holding period. Table 4 presents strong evidence of performance persistence. The out-of-sample alpha of the excellent portfolio is both economically and statistically significant. For example, for a 12- month holding period, the excellent portfolio has an alpha of 0.46% per month (t-statistic = 6.47), or about 5.52% per year. Further, the excellent portfolio outperforms the other portfolios significantly for as long as three years. The alpha spread between the excellent and the bad portfolios is about 0.48% per month (t-statistic = 5.69) for a 12-month holding period. The result of performance persistence suggests that our method groups funds with different skill well Comparing the P J measure with estimated alpha and t-statistic Next, is our measure P J better at identifying skilled funds than the conventional measures of estimated alpha and its t-statistic? A priori, we have good reasons to believe so. First, using estimated alpha alone omits important information about estimation precision. Second, though 20

23 the t-statistic equals estimated alpha multiplied by its precision (i.e., the inverse of the standard error of estimated alpha), it does not differentiate the contribution from the two components. As a result, the t-statistic only tells whether we can reject the null hypothesis of zero skill, but it cannot be used to rank individual fund skill. In contrast, our performance measure adjusts for precision of estimated alpha more efficiently by weighing the prior information and estimated alpha. Here, we make a comparison based on actual data. In particular, since most funds of hedge funds (FOFs) invest in about hedge funds (Brown, Gregoriou, and Pascalau, 2012), we form a strategy by selecting top 20 funds based on the P J measure, and compare its out-ofsample performance with that of alternative strategies picking top 20 funds based on estimated alpha and t-statistic. Similar to the procedure in Table 4, in each month starting from January 1996, we compute the performance measure P J for each fund from the previous 24 months. Then, we form an equal-weighted portfolio of investing in top 20 funds ranked by P J. 18 In a similar way, we form two other equal-weighted portfolios by selecting top 20 funds based on estimated alpha and t-statistic from the previous 24 months. The three portfolios are all rebalanced monthly and held for different periods. Table 5 reports the out-of-sample performance for the three strategies. The strategy based on the P J measure significantly outperforms the other two for up to 24 months. For example, for a six-month holding period, the portfolio of top funds ranked by our P J measure generates a riskadjusted return of 0.72% (t-statistic = 7.36), whereas the other two strategies yield 0.45% (t- 18 Note that if we hold the portfolio for multiple months, the actual number of funds held will exceed 20 since some of the top 20 funds in one month will not stay in top 20 in later months. For example, for a three-month holding period, the number of funds held in the portfolio will fall in the range of 20-60, depending on the transition of top funds over time. We report the transition probabilities for the hedge funds in our sample in the next section. 21

24 statistic = 2.52) and 0.35% (t-statistic = 6.03), respectively. In untabulated test, we find that the strategy based on the P J measure selects quite different funds from the other two strategies. 19 This confirms that our P J measure is substantially different from both estimated alpha and its t- statistic, and it provides more accurate information about fund skill. Hedge funds with longer lockup periods generally have better performance (Aragon 2007; Agarwal, Daniel and Naik, 2009). Hence, we are concerned that the P J measure may simply select funds with long lockup periods that restrict money redemption. As a robustness check, in Panel B of Table 5, we repeat the analysis by removing the funds with lockup periods longer than three months, and our inference is unchanged. Thus, the superior performance of P J is not driven by funds with long lockup periods. 5. Additional Tests and Robustness Checks In this section, we conduct additional tests to gain further insights about hedge fund skill as well as check the robustness of our results. We start with examining the transition probabilities across the skill groups. Next, we test the relation between fund skill and investor flows. Then, we link fund skill to fund characteristics. Finally, we examine the sensitivity of our results to alternative factor models. 19 In fact, the fraction of common funds selected by the P J measure and t-statistic is 48.5% for the sample, and the fraction of common funds selected by P J and estimated alpha is only 25.7%. 22

25 5.1. Transition probabilities across skill groups In Table 6, we present the transition probabilities across the skill groups. For each month we assign funds into one of four skill groups based on their P J measures from the previous 24 months. We then check how likely funds in each group remain in the group in 3, 6 and 12 months conditional on fund survival. As shown in Panel A, in three months, about 59% of excellent funds will remain to be excellent and 71% of good funds remain to be good. Furthermore, it is highly unlikely for an excellent fund in the current month to become a neutral or bad fund in the future. As discussed above, it is natural to expect some decay in skill as informational advantages erode over time. On the other hand, most bad funds either remain bad or improve to become neutral if they continue to survive Fund skill and investor flows How do investors affect and respond to fund skill? Given the persistence of performance, can investors infer fund skill from past performance? We answer the questions by examining the relation between fund skill and both prior and subsequent investor flows. As before, we use a 24- month period to evaluate fund skill. Then, we examine fund flows subsequent to the evaluation period as well as prior to the period. Following prior research (e.g., Sirri and Tufano, 1998), we measure fund flows as the percentage change of fund total assets adjusting for fund returns. Table 7 reports the results. First, investor flows chase past fund performance, as indicated by a significantly higher level of money flowing into recent excellent and good funds than into the other two groups. This is consistent with prior findings of Goetzmann, Ingersoll and Ross (2003) and Getmansky, Liang, Schwarz, and Wermers (2010), suggesting that hedge fund 23

26 investors infer managerial skill from past fund performance. Meanwhile, the level of money flows prior to the evaluation period is similar across the skill groups Skill type and fund characteristics Table 8 relates the skill types to fund characteristics by regressing the performance measures (i.e., conditional probabilities) on various fund characteristics. Funds with high probability of being excellent or good tend to be large funds. These funds charge high management and incentive fees and have long lockup and redemption notice periods. On the other hand, neutral and bad funds seem unable to retain capital and have small fund size; they also charge less incentive fee, perhaps indicating a lack of confidence in adding value. The results are consistent with prior studies (e.g., Aragon 2007; Agarwal, Daniel and Naik, 2009). Since our approach separates funds into different skill groups, our analysis has the richness to check the relation between each skill type and fund characteristics, rather than examining the association between one performance measure (e.g., estimated alpha) and fund characteristics. In untabulated tests, we assign funds into four skill groups and perform probit regressions and the main results are unchanged. Given the relationship between fund skill and fund characteristics, future work can incorporate fund characteristics into the estimation procedure. We leave this extension for future research Fund skill by investment styles As differential investment styles may have different skill levels, we now examine the skill distribution within each investment style. Since the two databases TASS and HFR use 24

27 somewhat overlapping but not identical classifications for investment strategies. We follow Agarwal, Daniel and Naik (2009) to reclassify the funds into four broad styles: directional trade, relative value, security selection, and multiple strategies. We exclude 479 funds in the analysis, as their strategy information is either undefined or missing. Table 9 reports the results. Overall, we observe a certain extent of variation in skill distribution across the styles. 20 The directional trade strategy group has the highest fraction (14.3%) of excellent funds and the skill distribution among the remaining strategies tends to be similar in general. Meanwhile, we find that the variability of skill in the directional trade style is also higher than that for the other styles. Finally, the result also suggests that our skill types do not simply reflect the difference in investment styles, since no single style seems to dominate others in terms of skill distribution Alternative factor models So far, we have estimated fund alpha using the Fung and Hsieh seven-factor model. In untabulated tests, we confirm the robustness of our results to alternative factor models. First, Agarwal and Naik (2004) show that returns of several hedge fund strategies bear significant exposure to factors built on returns on S&P 500 index options. We augment the Fung-Hsieh factor model with two out-of-money call and put option factors proposed by Agarwal and Naik (2004). Second, given their dynamic strategies, hedge funds risk exposures can vary over time. To control for the potential impact of time-varying risk exposures on alpha estimate, we use the Ferson and Schadt (1996) conditional model in which funds market beta varies with lagged macro variables such as the three-month T-bill rate, a term spread, a default spread, and the 20 The variability of alpha and bootstrap standard errors for the parameters are not reported in the table to conserve space, but they are available upon request. 25