Out of the dark: Hedge fund reporting biases and commercial databases

Transcription

1 Out of the dark: Hedge fund reporting biases and commercial databases Adam L. Aiken Department of Finance School of Business Quinnipiac University Christopher P. Clifford Department of Finance Gatton College of Business and Economics University of Kentucky Jesse Ellis Department of Finance Katz Graduate School of Business University of Pittsburgh This draft: September 3rd, 2010 Abstract We examine the self-reporting bias in hedge fund data. Using holdings data from a set of limited partners, we construct a novel set of returns for hedge funds that otherwise have never reported to a commercial database. These returns allow, for the first time, a direct comparison of performance between funds that choose to report and funds that do not. We find evidence that estimates of managerial skill using self-reported data have an economically significant positive bias (77bps/quarter). The nature of our data allows us to measure the performance of funds even after they exit the databases - the so-called dead funds. Quarterly returns for funds that have stopped reporting are dramatically lower than returns in a database. We show, however, that even when controlling for dead funds, self-reported returns still have a large, positive bias. We examine the risk implications caused by the self-reported data. Commonly used measures of tail-risk are larger for non-reported returns than for the reported data, indicating that the self-reporting may bias estimates of hedge fund risk downward. JEL Classification: G11, G23 Contact author: chris.clifford@uky.edu. We thank George Aragon, Jeffrey Coles, Brad Jordan, Michael Hertzel, Laura Lindsey and seminar participants at Arizona State University, the University of Kentucky, the Securities and Exchange Commission, and the FIRS Finance Conference in Florence, Italy for their useful comments. 1

2 Academic research on hedge fund performance readily acknowledges biases in commercially available data. 1 As private entities, hedge funds are not required to report to any regulatory agency with regards to most aspects of their business, including their performance. As a marketing tool, however, hedge funds often voluntarily submit their monthly returns to commercial database providers (e.g., Lipper TASS) in an effort to document their performance for potential investors. These commercial databases have been the primary data source used by academics and regulators to study hedge funds. Yet, the voluntarily nature of the disclosure decision creates a host of biases that affect inferences on hedge fund performance and risk. The self-reporting bias is the least studied, but perhaps most important of the biases in hedge fund data. Funds endogenously choose whether to list their returns in a database, making the trade-off between the costs of disclosure and the benefits of raising additional capital through marketing the fund s returns. Funds with poor past performance are unlikely to see much benefit in advertising their returns and subsequently choose not to report to a database. Conversely, funds that raise enough capital or simply view the costs of disclosure as too high also choose not to report. Databases of self-reported hedge fund returns are likely biased and missing both the best and worst performing funds (Fung and Hseih (2009)). In this paper, we employ an entirely new method for studying hedge fund returns that is free from many of the biases studied in the literature. Using a sample of publicly listed funds of funds, we hand-collect their quarterly hedge fund holdings. This holdings data allow us to construct a set of returns for funds that have never reported to a commercial database. These returns are free from self-reporting, backfilling, delisting, and survivorship bias. The nature of our sample selection prcoedure gives rise to concerns of endogeneity and self-selection bias given that our returns come from a sample of registered fund of funds. In Section II.C, we address these biases and demonstrate how the inferences in this paper are unlikely to be affected by such bias. As a break from the previous literature, we obtain our returns not from the hedge funds themselves, but rather their limited partners funds of funds. A primary contribution of our paper is to address both the size and direction of the self-reporting bias. In an effort to measure the effect of the bias on average managerial skill (Jensen s α), we examine a series of factor pricing models, controlling for whether the fund s quarterly return is listed in a commercial database. 2 1 Including, but not limited to, Agarwal, Daniel, and Naik (2008), Aragon (2007), Aragon and Martin (2008), Bollen and Pool (2009), Boyson, Stahel, and Stulz (2009), Griffin and Xu (2009), Kosowski, Naik, and Teo (2007), and Titman and Liu (2009). 2 Our unit of observation is a fund-quarter. As such, it is possible for our database return variable to vary over time for a fund if that fund originally reported to a database, but subsequently chooses to stop reporting its returns. 2

3 We find convincing economic evidence that self-reported hedge fund returns are upwardly biased on average. For the pricing models most likely to capture hedge fund returns, the Fung-Hsieh 7-factor model (2004) and a modified version of the Jagannathan, Malakhov, and Novikov (2010) benchmark model, we find the bias to account for 80% and 78%, respectively, of the average fund s quarterly alpha. The unique structure of our data allows us to examine the delisting bias in hedge fund returns. While hedge funds voluntarily agree to provide their returns to commercial databases, they can choose to stop reporting as well. Hodder, Jackwerth, and Kolokolova (2008) document that 8.1% of hedge funds delist from a commercial database annually. We identify 157 hedge funds that have delisted from two commercial databases, yet are still present in our data (i.e., they did not liquidate or have not done so yet). We find that returns for these dead funds are dramatically lower than the other hedge fund returns in our sample. 3 Funds delisting from a database lose an average 135bps/quarter when compared to those that continue listing in a database. Dead funds continue to operate for some time after the last database return is recorded, as 78% of our dead funds have at least two quarters of returns subsequent to the delisting date (52% of the funds have at least four quarters of returns following the delisting date). Therefore, the delisting result is not simply the case of a large, one-time write-down as the hedge fund begins to liquidate. Exacerbating the self-reporting problem is the fact that it may not only be the poor performing funds missing from self-reported data. Fung and Hseih (2009) report that 40% of the fund families in the 2007 Institutional Investor Top 100 do not report any of their funds to a commercial database. To test how the reporting decision affects the bias for not only the worst, but the best performing funds as well, we use a simultaneous quantile regression framework, estimating the self-reporting bias at various points throughout the return distribution and document an asymmetric effect. At the 10th percentile of returns, for example, returns found in a database outperform those that are not reported by 136bps/quarter. This is consistent with the fact that poor performing hedge funds have less incentive to list their returns with a database for marketing purposes. When returns are highest, we find modest evidence that funds in a database actually underperform those funds not in a database. Taken as a whole, our results suggest that the performance for an average fund listed in a database has a strong, upward bias. For researchers using self-reported data to study hedge fund performance, extrapolating their findings to the population of hedge funds is likely to be difficult. 3 Shumway (1997) investigates delisting returns for equities in CRSP. 3

4 As the best and worst performing funds are likely missing from commercially available databases, the observed distribution of hedge fund returns is effectively truncated. Of equal concern to return performance is how this truncated data affects estimates of hedge fund risk. We find that the standard deviation of non-reported quarterly returns is 26% greater than the standard deviation of returns self-reported by funds (8.45% vs. 6.66%). Furthermore, commonly used measures of tail risk, such as Value-at-Risk (VaR) and Expected Shortfall (ES), are higher when using returns that are not present in a database. Our paper sheds light on the public debate of hedge fund risk and provides a note of caution for regulators when assessing hedge fund risk from commercially available data. While limitations in our data (quarterly returns and short time-series) prohibit a thorough study of hedge fund risk, these findings suggest that the worst returns investigated by authors such as Boyson, Stahel, and Stulz (2010) are even lower in actuality. The remainder of the paper is organized as follows. Section I reviews the relevant literature. Section II describes our data and return methodology. Section III reports our results, and Section IV presents our conclusions. I Literature Review The literature has spent considerable time addressing many of the biases present in hedge fund data and has developed a variety of tools to mitigate them when possible. To address survivorship bias, most commercial data providers now include an archive of dead funds to capture those entities that no longer choose to report to the database. The inclusion of these funds in research may help to alleviate the survivorship bias, yet little is known about the funds that quit reporting. A fund, which has performed poorly, may choose to delist in an effort to hide poor performance or simply for the fact that it is near liquidation. However, many funds choose to leave a database not out of weakness, but strength (Fung and Hseih, 2009). Funds with strong performance that have increased their asset base may choose to stop listing once the gains from disclosure outweigh the costs. Compounding the issue further, funds that raise a considerable amount of assets from inception may never see the need to disclose their returns at all. Currently available commercial data offer no easy solutions to these issues. 4

5 Hedge funds often incubate a fund for some time before offering it to the market. These early stage funds are typically funded with capital from the general partner(s) and possibly a few limited partners. If the strategy works well, the fund is immediately listed on a database and the entire fund s previous history is subsequently backfilled. If the strategy does not work as planned, the fund is never listed. This backfilling of returns can generate a positive bias in observed hedge fund performance. Aggarwal and Jorion (2008) find that backfilling of returns imparts an upward bias of 5% over the first three years of a fund s life. Standard practices to reduce the backfill bias include eliminating all return observations before the date the fund was added to the database (e.g. Aragon, 2007), choosing funds that report an inception date close to the date the fund begin reporting to the database (e.g. Aggarwal and Jorion, 2008), or simply throwing out the first year or two of return data for a fund (e.g. Bollen and Whaley, 2009). However, none of these methods is fully satisfactory. For example, some funds may report to one database, stop, and then start reporting to another. Using the date a fund is added to a database in order to correct for the backfill bias will introduce a spurious bias into data if this is the case (Fung and Hseih, 2009). Further, the date a fund chooses to list as inception is in some sense arbitrary and provides the manager an ability to game the system. 4 This is not to say that previous research using commercially available hedge fund data is without value. Even in the presence of this biased data, there has been excellent research over the last decade that has informed the debate over performance, risk, and contracting environment. In the absence of better data, researchers are forced to document possible biases, mitigate them to the extent possible, and report their findings. Our paper is similar in spirit to a contemporaneous paper by Agarwal, Fos, and Jiang (2010), in that their paper proposes an alternative method to test the self-reporting bias. Their study uses SEC filings to compute quarterly hedge fund returns from holdings data. The authors then calculate the return differences between those fund advisors that report to a database and those that do not, finding little difference between the two on average. This result differs markedly from ours. This discrepancy is likely explained by differences in experimental design. The Agarwal, et. al. (2010) paper uses 13F filings from EDGAR to calculate implied returns for hedge funds (Griffin and Xu, 2009). This approach offers some advantages, namely that the returns are for a full 4 The hedge fund literature is not alone in having an incubation bias, as Evans (2010) finds that incubated mutual funds outperform non-incubated funds during the start-up period. 5

6 set of hedge funds (managing at least $100 million) as opposed to a sample of funds as our returns are. They are also able to look at a longer time horizon ( ), as well as investigate some of the underlying portfolio differences between reporting and non-reporting funds. 5 However, there are important drawbacks as well. The concern stems from the use of the 13F filing to generate hedge fund returns. The 13F is made at the advisor level, not the fund level. This is important, as one hedge fund advisor (e.g. D.E. Shaw) makes one quarterly 13F filing for its entire family of hedge funds. It is possible that these filings aggregate the holdings of seemingly different strategies (e.g., long-short equity and quantitative). Further complicating the matter is the fact that it is not uncommon for a fund family to self-report returns for only some of the funds in the family. Therefore, allocating the return of the advisor to individual funds becomes imprecise. Second, performance data using 13F holdings is likely to provide only a portion of the hedge fund s return. The 13F filing does not require the disclosure of short-selling (both equities and options), illiquid securities, the use of leverage, and intra-quartile trading, all strategies likely to be employed by hedge funds. For a hedge fund that engaged in any of these strategies, the return from the holdings is likely to be very different from the fund s actual return. Additionally, our sample creation methodology allows us to include funds with less than $100 million in assets (the threshold at which filing Form 13F is required), which reduces any possible bias caused by only using larger managers. We also report returns net of fees charged by the hedge fund. Therefore, our return measures are likely to be far more precise, albeit fewer in number, than the Agarwal, et.al. (2010) study and better reflect what the investor at the fund level actually experiences. Table I HERE We confirm the accuracy of our return measure for those funds in our sample which happen to report to database by comparing our calculated returns to those of the reported returns found in the database. From Table I, we can see that our returns are 94% correlated with the TASS database and 92% correlated with the BarclayHedge database. For comparison, Agarwal, et.al. (2010) report a correlation of 57%. We feel that the greater precision of our return measures allow us to more carefully test the null hypothesis that self-reported returns are unbiased. 5 Agarwal et al. (2010) also uses the union of five different hedge fund databases, while we only have access to two. 6

7 Fung and Hseih (2000) also study the self-reporting bias, by proposing the use of fund of funds return data, rather than hedge fund returns data. The track records of fund of funds retain the returns of hedge funds that stop reporting to a database for both good and bad performance, are typically unaffected by instant history bias, and may contain returns of both listed and non-listed funds. While their work helps to mitigate many of the biases in the data and provides evidence on the average skill in the hedge fund industry, issues still persist. Namely, fund of fund returns are indirect in that they do not fully control for the double layer of fees, leverage, or cash balances of the fund of funds. Our paper abstracts from these issues and provides for the first time a direct test of the size and direction of the self-reporting bias using actual hedge fund returns that are not present in a database. II Data A Registered Fund of Funds Here we describe the process for hand-collecting our set of hedge fund returns and provide a detailed discussion of potential endogeneity concerns regarding our data (Section II.C). Our data comes from a set of registered fund of funds (FoFs) that have received little, if any, attention from the literature. These funds register with the Securities and Exchange Commission (SEC) pursuant to the Investment Company Act of 1940 (40 Act). With similar filing requirements to that of a mutual fund, these registered funds allow the researcher the unique opportunity to use audited, regulatory filings to study FoFs. A registered FoF is organized as a closed-end investment company. Unlike the traditional closed-end fund, however, it is not listed on an exchange. Rather, the fund typically offers interests to investors on a periodic basis (usually monthly or quarterly). These funds often have minimum investments, which can be as small as $25,000 or $50,000. Similar to that of a typical FoF, the registered funds are marketed to qualified investors and charge both fixed and performance based fees. Additionally, investors often have their assets locked-up for periods of up to a year and receive liquidity only when fund management agrees to redeem interests through a tender offer. 7

8 The primary benefit to the fund in registering with the SEC is to allow the manager access to greater distribution channels. Typical FoFs offer interests under Regulation D of the Securities Act of While this allows the fund to avoid registration and sell interests in its fund through private placements, it explicitly limits the fund s ability to market or advertise. A registered fund faces no such restrictions. A registered fund files an offering prospectus with the SEC that allows them to actively advertise or market the fund to potential investors, including dedicated distribution platforms through investment advisers. Additionally, registered funds face less regulatory scrutiny under the Employee Retirement Income Security Act of 1974 (ERISA) and Section 4975 of the Internal Revenue Code. The ERISA act limits the amount of money a hedge fund (HF) can manage without becoming an ERISA fiduciary. 6 This designation can constrain the fund manager s use of leverage, diversification, and liquidity in its investments. By registering under the 40 Act, registered funds do not face this limitation and can accept ERISA money without concern of increasing their fiduciary responsibilities. Further, the registered FoFs are able to pass along this benefit to the HF they select, as registered money managed by a hedge fund is also exempt from the ERISA limits. We use forms NSAR-A/NSAR-B from the SEC to identify all funds that file as a closed-end fund, but do not list a closing price for the fund. 7 This results in a sample of 132 possible FoFs from We believe this to be the universe of registered FoFs. We eliminate 15 funds that either registered and never raised any funds or held primarily venture capital or private equity investments. We then eliminate funds that simply duplicate the holdings of another fund from the same management company (such as feeder funds or funds with special tax treatment that cross register). This screen eliminated 37 funds, which yields our final sample of 80 registered FoFs. B Generating Quarterly Hedge Fund Returns While the regulated nature of registered FoFs allows the researcher a unique look into performance of the FoFs and their organizational structure, that is not the focus of this paper. Rather, we use the registered 6 This is often referred to as the 25% rule. HFs that manage more than 25% of ERISA-regulated assets may be deemed an ERISA fiduciary. 7 Q76 on the NSAR lists the closing price for closed-end funds. Registered funds do not have a closing price, as they are not listed on any exchange. As such, they report 0.00 to this question. We further confirm that our sample is a registered fund of funds using additional regulatory filings (e.g., N-2). 8

9 funds as a vehicle to study their underlying holdings: hedge funds. As such, the description and analysis that follows will focus on the returns and performance of HFs, not the registered FoFs. 8 Registered funds have similar filing requirements to those of a mutual fund. 9 The funds semi-annual/annual reports (N-CSRS/N-CSR) and quarterly holdings statements (N-Q) disclose their quarterly holdings. These are the same forms utilized to generate the CDA/Spectrum mutual fund holdings database (s12). However, whereas the mutual fund holdings disclose the funds positions in various publicly traded equities, the FoF holdings disclose the funds positions in their underlying HF investments. The filings disclose the name of the underlying HF, the FoFs cost basis in the position, and the current value of the position. 10 The time-series combination of these forms allows us to utilize the underlying holdings to create a quarterly panel of HF returns. One of the primary innovations in our paper is that our HF returns are not supplied by a commercial data provider (e.g., Lipper TASS). That is, we avoid the self-reporting problem of reported HF returns (see Fung and Hsieh (2009) for a recent discussion). However, while this data allows us to systematically study returns for a set of HFs that have never been examined, the downside is that our returns have never been used in the literature either. As such, some time must be spent explaining our methodology for generating the quarterly series of HF returns and ensuring that we have accurately captured the funds returns. FIGURE 1 HERE Figure 1 provides a partial snapshot of two separate filings for one of the registered FoFs: Hatteras Multi-Strategy Institutional Fund, L.P. (Hatteras). As an example, one can see from Figure 1, Panel A that Hatteras held a position in the DE Shaw Composite Fund in September The cost basis of this holding was $16.00MM while the value of the holding was currently over $17.41MM. From Figure 1, Panel B, one observes that Hatteras maintained its position in the DE Shaw Composite Fund through December. Hatteras now reports the value of its holdings in the DE Shaw Composite Fund at over $18.47MM, while its cost basis remains $16.00MM. We use the following formula to generate our quarterly return: 8 See Aiken, Clifford, and Ellis (2010b) for an analysis of registered FoFs. 9 One notable exception here is form 13F. The SEC requires investment managers with more than $100MM in assets to disclose their quarterly holdings of exchange-traded or NASDAQ-quoted stocks, equity options and warrants, shares of closedend investment companies, and certain convertible debt securities. Because registered FoFs typically hold only underlying HFs, they are not required to report this form. 10 In less than 5% of our filings the FoF did not disclose the cost basis for the position. Because a return series cannot be generated without the cost basis, we remove these observations from our data. 9

10 Fund Return i,t = Value i,t Change in Cost i,(t 1,t) Value i,t 1 1 (1) In the case of the DE Shaw Composite Fund, its fourth quarter return for 2007 is 6.09%. We repeat this process for each FoFs quarterly filings and generate a sample of 20,376 quarterly HF returns from Generating a return series from cost and value inputs requires the information to be accurate and consistent across filings. While equation (1) generates the actual return for the holding assuming each FoF reports their costs and values the same, we find discrepancies in how different funds report their changes in cost basis. Many funds report the actual dollar change in cost, while some report the change as a percentage of the funds value. 11 The focus of this paper is to observe the return characteristics of HF returns that are not contained in commercial databases - we do not want artificial returns from different reporting regimes to affect the inferences of our tests. Given our inability to be certain of the regime, the remainder of the analysis in the paper focuses exclusively on those returns where cost basis does not change. We note however, that the inferences from our tests are unchanged when we use the entire sample of returns. In the appendix, we repeat the primary analysis in the paper on the full sample of returns. C Selection Bias and Endogeneity The motivation for this paper stems from the potential self-reporting bias present in hedge fund databases. To date, the hedge fund literature has only been able to study the portion of the universe that chooses to report to a database, leading to potentially biased estimates of performance and risk. As we discuss above, our data comes from a unique cross-section of returns; those funds that have been selected by FoFs that have chosen to register with the SEC. As such, it is possible that our data suffers from selection biases of its own. We address three potential biases below. Hedge funds that are selected by a FoF may differ from the general population of HFs (e.g., better performance). If the focus of this study were to compare the average performance of funds that are selected by FoFs to the entire sample of funds in commercial databases, these potential differences would be of 11 Conversations with several managers at registered funds revealed this discrepancy. The managers were unaware of any SEC guidance as to which methodology should be used. Mutual fund holdings largely avoid this issue as they list the number of shares they hold in a registered security, not their cost basis. 10

11 concern. Instead, our study examines only the funds chosen by FoFs in the first place. Within that subset, we examine how funds that report to a database differ from funds that choose not to report, conditional on the fact that both were selected by a FoF. That is, the control group for our tests is sampled from the same group of hedge funds. As such, we are less concerned with overall levels of performance, focusing instead on whether there is evidence of differences in performance between HFs that list returns in a database and those HFs that do not. A second form of bias could result from the fact that HFs selected by our FoFs differ given that our FoFs are registered with the SEC. A natural concern would be that the type of HFs that a registered fund could attract would differ from that of a more traditional FoF. We address this concern in a number of ways. First, we again note the fact that we only compare the returns of HFs selected by a registered FoF; therefore, this bias is likely mitigated when we then contrast the HFs that are selected by a registered FoF and report to a database and those HFs that are selected by a registered FoF and do not report to a database. Nonethless, the question of generalizing our results to the larger sample of traditional, non-registered FoFs remains. In Aiken, Clifford, and Ellis (2010b), we directly compare the performance, contracting environment (e.g., fee structure and lock-ups), and HF holdings of registered FoFs and those FoFs in commercial databases. We find little evidence that registered FoFs differ from traditional FoFs in any meaningful way. Alternatively, we compare (unreported) the returns of the HFs in a database and held by a registered FoF to those HFs that are in a database and not held by a registered FoF (i.e., the remaining portion of the commercial database) to examine whether registration by the FoF causes the fund to select from a different distribution of HFs. Differences in the mean and median returns between the two groups are economically and statistically indistinguishable. These results are likely driven by two opposing forces. On the one side, it is plausible that certain hedge funds will not allow registered FoFs to invest with them given the increased disclosure of their returns. However, registered FoFs offer a benefit to the HFs they invest in. Namely, their portion of the HFs AUM do not count against their ERISA cap (see Section II.A), providing a valuable funding source that does not constrain the HF manager. As a final examination of the HF coverage generated by our registered FoF database, we examine our databases coverage of the Institutional Investor s Top 100 hedge funds for 2008 and identify 64 fund families that report data to our database. This compares favorably to the Fung and Hsieh (2009) study which finds that only 60% of Institutional Investor s Top 100 HF families 11

12 of 2007 appear in any commercial database, indicating that our returns are likely to be drawn from a similar distribution to the commercial databases. 12 Finally, a third, albeit unlikely, source of bias could occur if FoFs use different fund selection screening when choosing funds that report to a database. Because selection skill and reporting status interact, this scenario would lead to our result of higher average returns for those funds found in a database if FoFs were better at selecting database HFs. While funds in a database have more publicly available information available initially, we note that the return and contract information found in the databases are all made available to FoFs during their due diligence process, regardless of the HF s listing decision. FoFs also typically conduct background checks and receive position-level data, as well as other private information not present in the commercial databases. We have been careful to structure our tests to mitigate any selection bias in the data. We focus less on the performance estimates of database and non-database funds separately, but rather on the differences in performance and risk between the two groups. For bias to exist in this environment, the factors leading to selection by the FoF manager would have to differ ex-ante between those funds in a database and those not in a database. We find it implausible that fund managers systematically have two separate decision making processes when allocating their investors capital; especially given that they likely would have access to the same information set regardless of the HFs decision to disclose to a database. D Descriptive Statistics and the Style Distribution It is possible for multiple FoFs to hold the same underlying HF during the quarter. The DE Shaw Oculus fund, for example, was held by 3 distinct FoFs during the first quarter of Based on the timing of when the different FoFs acquired their respective interests, differing high water marks, or small differences in fees, it is possible to generate different quarterly returns for the same HF. For completeness, we take the minimum, maximum, mean, and median return when multiple returns for the same HF appear, creating one unique fund-quarter return. In an effort to reduce the impact of outliers, the results in this paper will focus on median returns. 13 However, we note that our results our qualitatively unchanged when we instead use the 12 For example, HFs such as Citadel and SAC appear in our sample. These two large and successful hedge funds are both well-known for their secrecy and do not appear in our set of commercial database funds. 13 Additionally, we trim our returns at the 0.5% and 99.5% levels to reduce the outliers that accompany hand entered data. Our results are robust to the inclusion of the trimmed data. 12

13 min, max, or mean (99.9% correlation between the mean and median returns). After forming median returns across FoFs for every quarter, our final sample consists of 1,540 unique HFs and 10,169 unique fund-quarter returns. Of these 10,169 fund-quarters, 3,468 (34%) fund-quarters have a corresponding fund-quarter in either the Lipper TASS or BarclayHedge databases. The remaining 6,017 (66%) fund-quarter observations comprise the first set of HF returns that are not subject to self-reporting bias that we are aware of in the literature. 14 The focus of this paper will be to compare these two separate distributions of HF returns to understand the nature and magnitude of the biases inherent in self-reported hedge fund returns. We perform several robustness checks to ensure that our returns are calculated accurately. For the fundquarters that also match to either Lipper TASS or BarclayHedge, we can compare the funds self-reported returns (i.e. the actual returns given in the database) to the ones we calculate. The median discrepancy between the two samples is essentially zero (less than 0.1bp/quarter). Further, in Table I we report the correlations of our calculated returns for the 34% of returns in our sample that match to a database and those of the reported returns in the Lipper TASS and BarclayHedge databases. The correlations between our calculated returns and those of the reported returns vary from 92% to 94%, depending on database. As noted earlier, Agarwal et. al., report correlation of 57% in their sample. Given the presence of multiple returns for the same fund-quarter highlighted above, we believe this discrepancy to be random and unlikely to affect the inferences in our results. Finally, we note that for the 34% of our sample for which returns are available in a database, we instead use our calculated returns from eq. (1) for our tests. This is done to ensure that our return methodology and/or data errors in the inputs to calculate returns do not drive our results. Therefore, in the analysis that follows, our database variable is simply an indicator variable noting whether a HF was present in a database. Other than to determine presence in a database, no data in our paper is collected from commercially available sources. 14 Our final sample consists of 12,872 unique fund-quarter returns when we relax the cost change constraint. As before, (34%) of these fund-quarters have a corresponding fund-quarter in either the Lipper TASS or BarclayHedge databases, while (66%) do not; indicating that the propensity for cost basis to change is of equal probability for both database and non-database funds. 13

14 D.1 Database and Non-Database Returns Table II HERE Table II reveals the distribution of the quarterly returns by reporting the first two moments for both returns found in a database (database returns) and those returns that were not reported to a database (nondatabase returns). The table reveals that the median quarterly return for a fund in our sample is 1.91%. However, returns reported to a database are larger than returns that were not. For example, fund returns present in a database are 2.11% a quarter, while returns not reported to a database are 1.76%. 15 The results are similar when we evaluate the means. Tests for differences in medians and means between the two groups are rejected at the 1% level for both. When we focus of the standard deviation of returns, however, we note that returns not reported to a database are more volatile than returns included in a database (8.46% vs. 6.66%). A Kolmogorov-Smirnov test to assess whether the distributions of returns between the matched group and unmatched group are equivalent is rejected (p-value <.01). We will return to this fact and other implications for risk management in Section III.D. In addition to comparing the returns that are reported to a database to those that are not, we also explore the returns of funds which reported to a commercial database at some point in their life cycle, but subsequently chose to quit reporting (hereto after referred to as dead funds). As mentioned previously, the nature of our data allow us to track the performance of these funds even after they quit reporting and, as noted, the average performance of dead funds is dramatically lower than their live counterparts. The median quarterly return for a dead return is 1.43%, while the median quarterly return of these funds prior to delisting was 2.51%. The difference is even greater when we examine the means. Differences between means and medians are statistically significant at the 1% level for both. This is consistent the findings of Agarwal et al. (2010), who also find that fund performance after termination is significantly lower than prior performance. 15 Afund sreturnscanbeabsentfromadatabasebecausethefundneverreported,thereturnoccurredpriortothefund joining a database, or because the return occurs after a fund stopped reporting. 14

15 Figure 2 HERE Figure 2 shows kernel density estimations for the distributions of different return types. Panel A compares returns that are present in a database to those returns that are absent. Panel B compares dead returns (i.e. returns that are subsequent to a fund s last reporting date) to live returns (i.e. returns in our sample that are also in a database). Note that the distribution of database returns has more mass on the right side of zero than non-database returns. The same holds true for live returns vs. dead returns, as dead returns have a thicker left-hand tail than live returns. This figure visually confirms the descriptive statistics given in Table II. Table III HERE Table III examines how the different HF styles are distributed among the quarterly return observations in our matched and unmatched return samples. If the styles of funds that report to databases differ markedly from those that do not, making statistical inferences regarding performance differences between the two groups becomes more difficult, due to the pooled nature of our analysis. However, Table III shows that both samples contain very similar style distributions. Long/short equity is the most common style in both samples, while the short-bias style is one of the least common. Overall, there does not appear to be a style-bias among those funds that choose to report and those that do not. III Results Our results focus on four different ways the database self-reporting bias may affect inferences in HF research. First, we estimate the association between choosing to be in a database and abnormal return calculations using a pooled OLS regression. This allows us to test how selection bias affects inferences of the average HF manager s skill level. Second, we investigate the role dead funds play in the bias of database returns. Third, we look at the tails of the return distribution in order to examine whether the selection bias is symmetric among the best and the worst performing funds. Finally, we quantify how the distribution of biased HF returns affects risk measures such as Value-at-Risk. 15

16 A The Database Bias and Estimating Manager Skill We estimate abnormal returns with a series of common factor pricing models and a benchmark based method, but include an indicator variable for whether or not the return is found in a either the Lipper TASS or BarclayHedge databases (database ): J r i,t = α + β 1 database i,t + β j Factor j,t + i,t (2) j=2 J J r i,t = α + β 1 database i,t + β j Factor j,t + β j Factor j,t database i,t + i,t (3) j=2 j=2 where r i,t is a fund s quarterly return in excess of the 3-month risk-free rate and the controls are either the following: the Carhart (1997) four-factor model, the Fung and Hsieh (2004) seven-factor model 16, or a modified version of the Jagannathan, Malakhov, Novikov (2010) hedge fund benchmark model. 17 Our database indicator variable will shift the intercept from the regression, allowing us to estimate how much the self-reporting bias affects an alpha estimate from a pooled regression. In equation (3), we interact the database variable with the risk factors to allow for differences in risk between the two groups of funds. Standard errors are robust to heteroskedasticity and are clustered at the fund-level. We report only our estimates of the intercept and the indicator variable for expositional ease. Table IV HERE Table IV reports the results from these regressions. In Panel A, the coefficient on the database indicator is positive and statistically different from zero for each of the models. In the case of the 7-factor model the 16 The Fung and Hsieh (2004) model includes the following returns: the S&P 500 total return, a size spread return (Wilshire Small Cap Wilshire Large Cap 750), a bond market factor (quarterly change in the 10-year constant maturity treasury yield), a credit spread factor (quarterly change in the Moody s Baa yield less the 10-year treasury constant maturity yield), and three trend-following factors for the bond market, the currency market, and the commodities market. See David Hsieh s web page at for a complete description. 17 The Jagannathan, et al. (2010) model includes the fund s style index return and a market factor. Our version differs from theirs due to the pooled nature of our analysis - we are not able to directly correct for smoothed returns. However, the quarterly nature of our hand-collected returns mitigates the smoothing problem, as it is unlikely that quarterly returns exhibit as much autocorrelation due to holding illiquid securities and/or return smoothing behavior by the fund s manager. We use the return on the S&P 500 and the return on the Lipper TASS style benchmark associated with the fund (each less the risk-free rate) as our factors for this modified model. 16

17 database indicator in the 7-factor model indicates that returns found in a database are 77 bps/quarter larger than the set of returns that are not reported to a database. The estimates of the database variable and intercept are additive, which allows us to derive the economic impact on skill estimates for those funds that self report their returns to a database. Again focusing on the seven-factor model, this economic impact is % ( ) of observed alpha. This implies that most of what previous researchers and practitioners attribute to managerial skill may simply stem from the self-reporting bias in HF returns. In Panel B, we allow for the possibility of differences in risk exposures between the database and non-database groups by interacting the database variable with each of the factors in the model. In all three of the models, the database indicator variable remains positive and statistically different than zero. In the case of the sevenfactor model, estimates of alpha for the database funds are 107 bps/quarter higher than HFs that do not list in a database. In unreported results, we estimate models (2) and (3) for each of the years in the sample. Estimates for the self-reporting bias range from a low of 7 bps/quarter in 2006 to a high of 203 bps/quarter in Additionally, we estimate models (2) and (3) for each of the ten TASS hedge fund styles highlighted in Table III. For our three largest strategies, long-short equity, event driven, and market neutral, the database coefficient is 45, 181, and 44 bps/quarter, respectively. The database coefficient is positive for each of the hedge fund styles with the exception of short bias and managed futures. But, as shown in Table III, the latter two styles are the two of the strategies with the fewest observations, making precise estimations difficult. Finally, as noted above, it is possible for multiple FoFs to hold the same hedge fund in a given quarter. We perform a final robustness test using models (2) and (3) to test how the self-reporting bias varies when a HF is widely held by multiple FoFs. This test offers the additional advantage that the HF returns for that quarter are calculated with greater precision as they are provided by multiple FoFs. In unreported results, we find the self-reporting bias to persist regardless of whether a HF is widely held. When a hedge fund is held by only one FoF, the database coefficient is 86 bps/quarter. When a fund is held by 3 or more (5 or more) FoFs, the database coefficient is 84 bps/quarter (80 bps/quarter). On whole, the results from Table IV indicate that the average performance for self-reported hedge fund returns is biased upwards. The bias is robust to differences in risk exposures, time period, style, and multiple FoFs holding a given fund. No easy solutions exist to mitigate this bias. Until regulatory reporting for hedge fund returns becomes mandatory, it will likely be difficult to draw strong conclusions from hedge fund performance studies which use self-reported data. 17

18 B How Do Dead Funds Affect Hedge Fund Returns? As HFs can choose whether or not to report their returns to a database, they can just as easily choose to stop supplying returns to a commercial database. When a HF enters the dead portion of a database, this indicates that the fund has stopped reporting returns. Understanding the returns of dead funds is important because hedge fund investments are illiquid and the traditional investor cannot simply liquidate his position as soon as a fund goes dead. Thus, the true hedge fund investor experience necessarily comprises the returns of dead funds. Although no study examines the returns of dead funds directly, there have been two primary explanations put forth explaining why a fund would stop reporting its returns to a database. Most researchers suggest that funds chose to stop reporting because they have been unsuccessful (Malkiel and Saha, 2005). Funds that underperform their peers do not have an incentive to continue advertising the fact in a commercial database. These funds may benefit from ceasing reporting and waiting until their track record improves before they advertise to new investors. Additionally, very poor performers may stop reporting because they have ceased operations and begun the liquidation process. In this case dead funds should exhibit very poor returns on average. A competing hypothesis suggests that funds may choose to stop reporting because they have been very successful (Ackermann, McEnally, and Ravenscraft, 1999). A successful fund may not need to advertise its performance in a database because it could either have strong word of mouth reputation or the fund could have simply reached critical mass and closed to new investors. Additionally, a successful fund that has lost any benefits from advertising may choose to stop reporting returns in fear that their investment strategy could be replicated by competitors. If this were the case, then dead funds would likely exhibit higher than average returns. Although some studies have attempted to estimate the reason a fund has stopped delisting by looking at returns prior to the delist date (Grecu, Malkiel, and Saha, 2007), no study has examined actual hedge fund returns after the delist date. 18 Because of our unique sample, we have returns for HFs that have stopped reporting to a database, but continue to operate and are held by a FoF in our data. This attribute allows us to study the performance 18 Hodder et al. (2008) estimates the delisting returns of HFs by using reported FoF returns. They find that delisting returns are small, but statistically different from the average HF return. 18

19 of the so-called dead funds. To do this, we identify the funds that matched to a database and compare their dead date to the return observation date. 19 If the observation occurs after the date the fund stopped reporting, then the return is marked dead in our data. Table V HERE J r i,t = α + β 1 dead i,t + β j Factor j,t + i,t (4) j=2 J J r i,t = α + β 1 dead i,t + β j Factor j,t + β j Factor j,t dead i,t + i,t (5) j=2 j=2 In Table V, we report a series of tests using models (4) & (5) above with the Carhart (1997) four-factor model, the Fung and Hsieh (2004) seven-factor model, and the modified version of the Jagannathan et al. (2010) benchmark model. The dead indicator variable in model (4) allows a test of how funds that have delisted from the database perform following their delisting decision. The first three columns of Panel A, contain both funds that matched to a database and those that did not, and provides a test of whether dead funds underperform the general population of hedge funds in our sample. In each of the models, our indicator variable for dead returns is negative and statistically significant. The coefficient in the seven-factor model is -89bps/quarter, indicating that the average fund that delists from a database has an economically large reduction in average performance. In the second three columns of panel A, we limit the control group of funds to be only those funds that list in a commercial database at some point in their life. Again, in each of the models the indicator variable on dead returns is negative and statistically significant. Using the 7-factor model, funds that delist from a database underperform the returns in a database by 135 bps/quarter. In panel B, we repeat the analysis allowing for differences in risk exposure between dead and non-dead funds by interacting the dead fund indicator variable with each of the risk factors using model (5). Our results are stronger when we allow for differences in risk between the two groups, as the estimates for the dead fund indicator are larger for both the 4-Factor and the 7-Factor models. 19 Both Lipper TASS and BarclayHedge include a field that contains the date when the fund stopped reporting. 19

20 The results suggest that funds that choose to stop reporting are on average some of the worst performing funds. This result is important because although an investor could shield herself from the self-reporting bias by maintaining a trading strategy that only invests in funds that report to databases, the illiquid nature of hedge fund investment leaves investors vulnerable to the poor returns of dead funds. Table VI HERE One could argue that the upward bias in average HF returns, documented in Table IV, is simply driven by the strong negative returns of dead funds. That is, we previously show that funds that delist underperform significantly. By construction, these returns are missing from the database indicator in Table IV. We test this possibility in Table VI by simultaneously controlling for dead returns. That is, we add both the database and dead fund indicator variables from model (2) and (4) above into a series of factor pricing models. As before, the database indicator is positive and statistically different from zero for each of the models. Examining the database indicator in the 7-factor model indicates that returns found in a database are 65bps/quarter larger than the set of returns that are not reported to a database; even when controlling for the dead returns. While returns of those funds that delist from a database contribute to the upward bias in self-reported database returns, they do not fully capture the bias. Rather, the sample of funds that choose to report to a database are fundamentally different from those that do not. C Does the Self-Reporting Bias Affect the Best and Worst Funds Differently? Though the choice of disclosure in a HF database may alter estimates of average manager skill, there is a common assumption in the HF literature that not only the worst, but the best funds are missing from the databases. If this is the case, then it is important to study how returns in the tails of the distribution are viewed. For example, if the best funds choose not to report, then we expect that the portion of our hand-collected sample not in a database should outperform those funds in a database when returns are high. The opposite should hold true when returns are low. 20