The Benchmark s Benchmark: Measuring the Performance of a Manager s Long-Term Strategy

Transcription

1 The Benchmark s Benchmark: Measuring the Performance of a Manager s Long-Term Strategy October 2003 David E. Kuenzi Research Associate, Edhec Business School

2 Abstract When investment managers construct strategy benchmarks and manage their portfolios against them, they are making an implicit bet that some subset of the broader investment universe will produce better risk and return characteristics than a similar published index over the long term. Despite the long-term focus of this decision, it is nonetheless active in nature. Strategy benchmark performance should thus be evaluated as a source of manager value-added. This article highlights the concerns and issues involved with this evaluation process and puts forth suggestions as to how to mediate these issues. In doing so, an example drawn from the U.S. large cap equity universe is developed. EDHEC is one of the top five business schools in France. Its reputation is built on the high quality of its faculty and the privileged relationship with professionals that the school has cultivated since its establishment in EDHEC Business School has decided to draw on its extensive knowledge of the professional environment and has therefore focused its research on themes that satisfy the needs of professionals. 2 EDHEC pursues an active research policy in the field of finance. EDHEC-Risk Institute carries out numerous research programmes in the areas of asset allocation and risk management in both the traditional and alternative investment universes. Copyright 2015 EDHEC

3 As more and more managers adopt strategy benchmarks in order to measure both relative risk and relative performance, 1 a nagging question arises: how does one evaluate the performance of the strategy benchmark itself? Typically, an investment management group adopts a strategy benchmark because the universe of securities from which the group draws differs in a definable fashion from that of the closest fit published benchmark. 2 This manager-defined universe typically has characteristics that the manger believes will outperform over time. A momentum manager, for instance, might create a strategy benchmark by choosing all of the stocks in the Russell 1000 Growth Index that have higher expected earnings growth than the average of the Russell 1000 Growth Index. Implicit in this index construction rule is a view that, as a base case, growth stocks with higher than average earnings growth will outperform growth stocks with lower than average earnings growth. In other words, the manager is taking a longterm, strategic and semi-active view on a subset of the large- and mid-cap growth universe. 3 Given that there is a soft active element here, it is clear that the quality of this decision should be somehow evaluated. This article posits that there are three crucial and interdependent issues that must be considered as managers and clients attempt to evaluate the strategy benchmark decision: 1. The choice of the strategy benchmark s benchmark (what one should compare the performance of the strategy benchmark to); 2. The performance metrics that should be used in making this comparison and how best to present performance to clients on an ongoing basis; 3. The horizon across which this comparison should be made and necessary qualitative aspects of this analysis. The following is organised around each of these three issues with an example of a manager investing in high dividend yielding stocks developed throughout. The Benchmark s Benchmark What is the benchmark that one should compare the strategy benchmark to? Typically, the institutional client is trying to determine the relative merits between the manager and other and a passive allocation to a similar style. The institutional client wants to know the extent to which the strategy benchmark has outperformed a passive strategy with similar risk / return characteristics. Exhibit 1 shows how a strategy benchmark might overlap with various styles. It is clear that the strategy benchmark, represented here by the shared area, is a subset of the broader large-cap universe. As such, the investor essentially has a choice between the manager s universe and the closest fit published benchmark, which in this case would be the entire large-cap style. 4 As such, for the manager represented by the shaded area here, a comparison to the S&P 500 Index would make sense. Now assume that a particular manager believes that stocks with high dividend yields will provide superior risk-adjusted returns over the long term. This is based, shall we say, on the belief that companies paying out a significant portion of their earnings as dividends are more likely to use investor capital efficiently; they are likely to invest only in those projects which will produce outsized returns with a high probability. 5 As such, this hypothetical investment management group defines its benchmark as the 150 highest yielding stocks in the S&P 500, rebalanced on a quarterly basis. So what published benchmark should the manager use in order to measure the performance of this strategic tilt toward dividend yield? 1 - Anecdotal evidence would suggest this. See, for instance, Dynkin et al. [2003] in the commentary from the Lehman Brothers index team early in 2003: Given the great improvement in index technology that can make such non-standard benchmarks possible, we anticipate many more [inquiries concerning strategy benchmarks] during the course of For a full treatment of the appropriate use of strategy benchmarks and their importance to a cohesive investment process see Kuenzi [2003]. For articles dealing with strategy benchmark construction, see Campisi [2002] and Kritzman [1987]. 3 - For another approach to the measurement of a manager s performance along strategic versus tactical dimensions, see Engstrom [2001]. 4 - Bailey, Richards and Tierney [1990] present a framework that is reflective of this approach. 5 - See La Porta, Lopez-de-Silanes, Shleifer, and Vishny [2000] and Stein [2002] for a work on agency problems and dividends. 3

4 There is one significant complication in choosing such a benchmark for comparison against the strategy benchmark: the style attributes of the strategy benchmark can shift over time, thereby driving a shift in the best fit published benchmark. Consider our high dividend yielding strategy benchmark. Let us suppose that at 30 June, 1996, the manager set the high dividend yield strategy and established the related strategy benchmark. At this point, looking for the best fit published benchmark, the manager created the strategy benchmark as it would have existed during the previous seven years, and calculated tracking error against the S&P 500 (3.0%), the S&P/Barra 500 Value (5.6%), and the S&P/Barra Growth Index (3.2%). Based on this information, the manager would likely go with the S&P 500 as the benchmark s benchmark. During the ensuing seven years, however, this picture changes dramatically. Exhibit 2 shows the tracking error for the full 14 years (the backtesting period of July 1989 to June 1996 and the period during which the manager employed the strategy benchmark July 1996 to June 2003). While the S&P/Barra 500 Value was the worst fit benchmark during the first seven-year period, it was clearly the best fit benchmark during the second seven-year period as well as for the full 14 years. This migration toward value can also be seen in Exhibit 3, which shows the rolling 24-month tracking error of the strategy benchmark against the three indices. The arrows in Exhibit 1 indicate that, in general, a given strategy benchmark can shift in any style direction depending on its construction methodology. This analysis highlights two often competing issues involved with the choice of the appropriate benchmark s benchmark: 1. We would like the strategy benchmark to be a subset of the benchmark s benchmark. This assures that, despite style drift of the strategy benchmark, it will likely have a reasonable fit with its benchmark. 2. We would like the benchmark s benchmark to be that index which minimises the strategy benchmark s tracking error. This assures that we are comparing the strategy benchmark to the most similarly performing index. In our example, 1 above would suggest that the S&P 500 would be the better choice for the benchmark s benchmark. Despite the style drift that occurred over the full 14-year period, the strategy benchmark tracked reasonably well with the S&P 500 during both periods. As shown in Exhibit 3, there was no 24-month period during which use of the S&P 500 would have produced the highest tracking error. 6 Item 2 above would suggest that the S&P/Barra Value Index is the right choice, as it minimises tracking error over the full period and especially during the last four years of observations. It also produces the lowest tracking error of the three indices during every month starting from in April These conflicting considerations must be mediated qualitatively on a case by case basis. For a strategy benchmark with a consistent tilt and for which the style drift is not likely to be large, a least-tracking error benchmark that does not include all of the strategy benchmark s holdings might make sense. For a portfolio that seems to have a tilt but that also flips back and forth among style and / or size categories, as in our example, the all-inclusive benchmark makes more sense. Overall, the use of a best fit published benchmark is the appropriate index for comparison against a strategy benchmark. In determining the best fit, however, we must consider the potential style drift of the strategy benchmark. 7 Performance Measures for the Strategy Benchmark and Presentation of Results In general, the same types of analyses that one performs in the evaluation of individual funds or accounts applies in the case of strategy benchmarks. The main difference, however, is that The extreme increase in tracking errors in 1999 and 2000 were caused by extraordinarily high levels of return dispersion among stocks and stock portfolios during this period, where return dispersion is the cross-sectional standard deviation across stocks / portfolios. This high level of return dispersion is well documented in de Silva, Sapra, and Thorley [2001]. 7 - Henceforth, benchmark s benchmark and best fit published benchmark / index will refer to the index that the manager has chosen for comparison with the strategy benchmark in our example, the S&P 500.

5 investors will benefit from a two-way comparison the portfolio versus the strategy benchmark, and the strategy benchmark versus the closest fit benchmark. In order to better illustrate these ideas, we simulated a manager portfolio by randomly choosing stocks from our strategy benchmark. We started at 30 June,1996 with an equally-weighted portfolio of 40 stocks and proceeded to replace four companies per quarter by choosing randomly from the index, with the last portfolio formed using data as of 31 March, We will compare the resulting hypothetical manager returns (84 returns running from July 1996 to June 2003) to the strategy benchmark, which will in turn be compared to the benchmark s benchmark. To begin with, consider raw returns, as shown in Exhibit 4. Over the seven year period, the strategy benchmark provided a return of 14.48%, outperforming the S&P 500 (7.10%) as well as the hypothetical manager s portfolio (13.65%), albeit with slightly higher risk than the manager experienced. Additionally, the strategy benchmark and the portfolio have strong and nearly identical Sharpe ratios, which largely result from their sharing of the same peculiar style. Exhibit 5 shows the result of $100 invested in each of these three investments over the period. From the perspective of strategy benchmark evaluation, there are two things to note here. The return paths suggest that the portfolio tracks better with the strategy benchmark than with the benchmark s benchmark. This is verified in Exhibit 6, which shows a rolling 24-month tracking error of the portfolio to the strategy benchmark and to the S&P 500. The portfolio tracked better with the strategy benchmark in all periods except one of extreme dislocation. 8 Better tracking of the portfolio to the strategy benchmark is important to verify, as this is one of the essential reasons for using a strategy benchmark. After considering absolute risk and return and the efficacy of the strategy benchmark in terms of tracking error, one should further consider active risk and return analytics, as shown in Exhibit 7. This table allows investors to evaluate an investment management group based on its shortto intermediate-term tactical decisions and its long-term strategic tilt simultaneously. The first column of the table provides first a comparison of the portfolio to the strategy benchmark, while the second provides a comparison of the strategy benchmark to the established best fit published index the S&P This analysis is meant to isolate which decisions have added value over the period. First, we notice that the portfolio had quite a bit of total tracking error versus the strategy, but that the strategy had even more tracking error versus the best fit index. We see, however, that both the portfolio and the strategy benchmark were taking on much less systematic risk than their respective benchmarks, with betas of 0.75 and 0.64 respectively, which produces much smaller residual (beta-adjusted) tracking error. After taking account of a lower beta, the portfolio provided an alpha of 1.51%, producing a respectable information ratio of Now, however, it becomes clear that the majority of the value added came not from the manager s short term tactical decisions, but from his or her long term strategic positioning, as the alpha associated with the strategy benchmark is a considerable 8.08%, producing an information ratio of 2.25 versus the S&P Calculating the information ratio based on total tracking error rather than residual tracking error, a calculation used by many industry participants, produces the same result, although less dramatic. The last two rows of the table verify the generally conservative nature of both the portfolio and the strategy benchmark. The up capture ratio shows that the strategy benchmark generally 8 - This difference is almost entirely attributable to the strategy benchmarks returns from December 1999 to March We use the notation and definitions of Grinold and Kahn. Beta is defined as the slope of the regression r p (t) = a p + β p r B (t) + ε p. Alpha is defined from this equation as well; assuming that ε p is, on average, equal to zero, we get a p = r p (t) - β p r B (t). Total tracking error, Ψ p, is defined as std{r p (t) - r B (t)}, and residual tracking error, ω p, as std{r p (t) - β p r B (t)}. The information ratio, IR, is then defined as a p / ω p. We calculate a second information ratio based on standard practice by many in the industry, which is a p /Ψ p. Up capture is calculated as the return of the manager during periods when the index is up divided by the return of the index during such periods, or:, where n u is the number of months during which the index experienced positive returns. Down capture is calculated similarly The results are not significantly different if we use the S&P/Barra 500 Value Index as the benchmark s benchmark. 5

6 participated in only 72% of the upward move of the S&P 500 during months when the S&P 500 was up. During down months, on the other hand, the strategy benchmark only fell on average 47% of the decrease in the S&P 500. The same pattern holds for the portfolio. In evaluating the performance of a portfolio and strategy benchmark, it is necessary to revisit the issue of style drift. Arrington [2000] notes that the extent of style drift [is] a concern because investors have a strong preference for style consistency. (p.13) Investors want manager s to avoid the risks inherent in drifting away from the stated or historically established style in pursuit of boosting returns. To analyse style drift, we perform a rolling 24-month nonlinear least squares optimization in order to fit the returns of our manager to the returns of the S&P/Barra 500 Value and Growth indices. 11 The results, shown in Exhibit 8, reveal that style drift is present with this hypothetical manager. The manager s exposures begin nearly evenly weighted to value and growth, but after about a year the manager s exposure to growth disappears completely. Based solely on this data one might suppose that the manager was making a bet on a technology sell-off early on, and thus put all of his or her assets into value-oriented companies. This, however, does not represent the full story. If we perform the same analysis on the strategy benchmark, as shown in Exhibit 9, we produce the same pattern. It therefore becomes evident that the manager was, in fact, faithful to his or her discipline. The manager s entire universe, which is transparent and quantitatively defined, experienced style drift, which does not raise the red flags typically associated with manager style drift. Given this example, it is clear that style analysis of both the given portfolio and the strategy benchmark, and a comparison between the two sets of results, is crucial. The Horizon for Measuring Benchmark Performance Despite the strong performance of the strategy benchmark in our example, managers typically do not expect strategy benchmarks to outperform closest fit published benchmarks on a monthly, quarterly, or even annual basis. The manager is generally betting that the strategy will outperform over the course of a complete business cycle. This is due to the nature of most strategy benchmarks. Typically such benchmarks overweight factor exposures and / or sectors that may experience lengthy periods of underperformance but that the manager believes will lead to outperformance as markets mean-revert and the excesses of one period are undone over time. The complication arises from the fact that, while the strategy benchmark is expected to outperform over a complete business cycle, there is a non-zero probability that a superior strategy will underperform the closest fit benchmark simply due to the randomness of the markets. As such, one might require many business cycles in order to unequivocally determine the value of a particular strategy benchmark. Assuming a seven-year business cycle and a business cycle information ratio for the strategy benchmark of 0.5 (0.19 annually), we would need 112 years of data in order to establish a statistically significant information ratio for the strategy benchmark. 12 This requirement is obviously not tenable. Investors and investment managers must, therefore, employ both quantitative and qualitative techniques in determining the validity of a strategy benchmark. The most important qualitative questions are: What was the original rationale behind the design of the strategy benchmark and is such rationale still sensible given the intervening secular shifts in the markets, laws, regulations, and in local and global economies? Given the inability to quantitatively determine the quality of a strategy benchmark, a subjective examination of all relevant factors in the market environment is crucial. Exhibit 10 provides a schematic of this process first (Box 1) the manager develops a We use the methodology proposed in Sharpe [1992]. We perform the following minimisation: min ƒ (r P r B w)'(r P r B w), subject to [1 1]w= 1, and 0 w i 1, where r P is a 24 1 vector of trailing 24-month portfolio returns, r B is a 24 2 matrix consisting of trailing 24-month returns for the S&P/ Barra 500 Value and Growth indices, and w is a 2 1 vector of weights to the two indices. This optimisation is performed 61 times in order to generate the results in the graph, each time using data from the previous 24 months. Merrill Lynch uses a similar methodology in their evaluation of external managers. See Hanachi and Sun [2000] Grinold and Kahn [2000] p. 480 give the standard error of an annualised information ratio as SE[IR] 1/ Y 1/2, where Y is the number of years. Assuming an annual information ratio of 0.19 and a critical value of approximately 2 for testing significance at the 1% level, we have / (1 / 112 1/2 ). The business cycle information ratio is calculated as /2 0.5.

7 strategy benchmark based on clearly stated assumptions, second (Box 2) the environment undergoes change, and lastly (Box 3) the manager evaluates the strategy benchmark in the context of the original rationale, performance, and pertinent intervening changes in the environment. In our example, for instance, the use of dividend yield as a mechanism for determining a universe of stocks might have come into serious question in the late 1990s. This is evident in the extreme underperformance generated in the late 1990s (see Exhibit 5). Companies were cutting dividends (or at least failing to increase them), buying back stock, and emphasising the tax benefits to investors as they did so. 13 The tax arguments were compelling; long-term capital gains were taxed at 20% while the dividends were taxed at a marginal tax rate of as high as 39.6%. As these dynamics became evident, the manager would have done well to approach this issue systematically by first considering the rationale behind the strategy benchmark namely, that agency risk would be considerably reduced and corporate managers would be more likely to invest only in highly profitable projects if they were paying out substantial dividends. With this backdrop, one would consider the extent to which investor preferences and market behaviours were shifting in response to existing tax law. Investors seemed to be approving of corporate managers use of buybacks as a means of returning capital to shareholders. While this dynamic might lead to lower overall dividend payment levels and to more uneven payments among companies, it does not directly subvert the original rationale behind the strategy benchmark s construction. For the investment manager, there would not likely have been any simple answers, but the framework proposed in Exhibit 10 would have enabled the group to approach the issue with rigour. Conclusion As more and more managers adopt strategy benchmarks, it becomes increasingly important for both clients and managers to systematically evaluate this implicit long-term strategic decision. This paper suggests that there are three essential components of this evaluation: the choice of a best fit published benchmark for comparison with the strategy benchmark, ongoing active risk analysis of the strategy benchmark versus the best fit benchmark, and qualitative analysis meant to evaluate the ongoing viability of the assumptions behind the strategy benchmark s construction methodology. The main drivers of index choice should be the level of tracking error between the strategy benchmark and the benchmarks benchmark and the extent to which the benchmark s benchmark is a superset of the strategy benchmark. Ongoing examination of the strategy benchmark versus its best fit index should be performed using standard active risk and return analytics, but should be contextualised by performing the same analysis of the portfolio against the strategy benchmark. An important observation is that style analysis of the portfolio against published indices is only pertinent if one compares this to an analysis of the strategy benchmark against the same published indices. A manager should not be penalised because of the style drift inherent in a well-defined strategy benchmark. Finally, it would likely take several business cycles for a manager to demonstrate that a given strategy benchmark choice is a value-added decision. As such, the assumptions behind the construction methodology of a strategy benchmark should be revisited at any such time that fundamental shifts in laws, regulations, or the economic, business, or financial environment might challenge the viability of the strategy benchmark. This qualitative analysis is an essential element in any efforts to assure the relevance of a strategy benchmark. Looking ahead, it s a good bet that the use of strategy benchmarks will increase and that the 13 - See Grullon and Michaely [2002] and Fama and French [2001]. 7

8 construction rules for such benchmarks will become more complex. This points to an even greater active contribution to the manager s returns from this decision over the long term. As such, continued innovation in measuring manager and strategy benchmark performance in this environment will be important. Exhibits EXHIBIT 1: STRATEGY BENCHMARK VERSUS STYLE BOXES EXHIBIT 2: ANNUALIsED TRACKING ERROR OF THE STRATEGY BENCHMARK AGAINST VARIOUS INDICES EXHIBIT 3 ROLLING 24-MONTH TRACKING ERROR OF THE STRATEGY BENCHMARK AGAINST VARIOUS INDICES EXHIBIT 4: RAW RETURNS OF THE PORTFOLIO, THE STRATEGY BENCHMARK, AND THE BENCHMARK S BENCHMARK 8

9 EXHIBIT 5: VALUE OF $100 INVESTED IN THE PORTFOLIO, THE STRATEGY BENCHMARK, AND THE S&P 500 EXHIBIT 6: SIMULATED PORTFOLIO TRAILING 24-MONTH TRACKING ERROR VERSUS THE STRATEGY BENCHMARK AND THE S&P 500 EXHIBIT 7: ACTIVE RISK AND RETURN STATISTICS PORTFOLIO AGAINST STRATEGY BENCHMARK AND STRATEGY BENCHMARK AGAINST S&P 500 9

10 EXHIBIT 8: SHARPE STYLE WEIGHTS OF THE PORTFOLIO TO THE S&P / BARRA 500 VALUE AND GROWTH INDICES EXHIBIT 9: SHARPE STYLE WEIGHTS OF THE STRATEGY BENHCMAKT TO THE S&P / BARRA 500 VALUE AND GROWTH INDICES EXHIBIT 10: PROCESS FOR REEVALUATING THE STRATEGY BENCHMARK 10

11 References Arrington, George R. Chasing Performance Through Style Drift. Journal of Investing, Summer 2000, pp Bailey, Jeffery V., Thomas M. Richards, and David E. Tierney. Benchmark Portfolios and the Manager /Plan Sponsor Relationship. In Frank J. Fabozzi and T. Dessa Fabozzi, eds., Current Topics in Investment Management. Harper Collins, 1990, pp Campisi, Stephen. Creating and Managing Custom Benchmarks a Practitioner s Guide. Journal of Performance Measurement, Summer 2002, pp De Silva, Harindra, Steven Sapra, and Steven Thorley. Return Dispersion and Active Management. Financial Analysts Journal, September/October 2001, pp Dynkin, Lev, Jay Hyman, Vadim Konstantinovsky, Albert Desclee, Vasant Naik, Jeremy Rosten, and Hua He. Quantitative Portfolio Strategies: Lessons and Prospects. Lehman Brothers, January 6, Fama, Eugene F. and Kenneth R. French. How Disappearing Dividends: Changing Firm Characteristics of Lower Propensity to Pay. Journal of Applied Corporate Finance, Spring 2001, pp Grinold, Richard C., and Ronald N. Kahn. Active Portfolio Management, Second Edition. Chicago, IL: McGraw Hill, Grullon, Gustavo and Roni Michaely. Dividends, Share Repurchases, and the Substitution Hypothesis. Working Paper, Rice University, Hanachi, Shervin and Aline Sun. Comprehensive Guide to PASS. Merrill Lynch, December 12, Kritzman, Mark. How to Build a Normal Portfolio in Three Easy Steps. Journal of Portfolio Management, Summer 1987, pp Kuenzi, David E. Strategy Benchmarks: From the Investment Manager s Perspective. Journal of Portfolio Management, Winter 2003, pp La Porta, Rafael, Florencio Lopez-de-Silanes, Andrei Schleifer, and Robert W. Vishney. Agency Problems and Dividend Policies around the World. Journal Finance, February 2000, pp Sharpe, William F. Asset Allocation: Management Style and Performance Measurement. Journal of Portfolio Management, Winter 1992, pp Stein, Steve. Taxes, Dividends, and Distortions. Policy Review, June & July 2002, pp

12 Founded in 1906, EDHEC Business School offers management education at undergraduate, graduate, post-graduate and executive levels. Holding the AACSB, AMBA and EQUIS accreditations and regularly ranked among Europe s leading institutions, EDHEC Business School delivers degree courses to over 6,000 students from the world over and trains 5,500 professionals yearly through executive courses and research events. The School s Research for Business policy focuses on issues that correspond to genuine industry and community expectations. Established in 2001, EDHEC-Risk Institute has become the premier academic centre for industry-relevant financial research. In partnership with large financial institutions, its team of ninety permanent professors, engineers, and support staff, and forty-eight research associates and affiliate professors, implements six research programmes and sixteen research chairs and strategic research projects focusing on asset allocation and risk management. EDHEC-Risk Institute also has highly significant executive education activities for professionals. It has an original PhD in Finance programme which has an executive track for high level professionals. Complementing the core faculty, this unique PhD in Finance programme has highly prestigious affiliate faculty from universities such as Princeton, Wharton, Oxford, Chicago and CalTech. In 2012, EDHEC-Risk Institute signed two strategic partnership agreements with the Operations Research and Financial Engineering department of Princeton University to set up a joint research programme in the area of risk and investment management, and with Yale School of Management to set up joint certified executive training courses in North America and Europe in the area of investment management. Copyright 2015 EDHEC-Risk Institute For more information, please contact: Carolyn Essid on or by to: carolyn.essid@edhec-risk.com EDHEC-Risk Institute 393 promenade des Anglais BP Nice Cedex 3 France Tel: +33 (0) EDHEC Risk Institute Europe 10 Fleet Place, Ludgate London EC4M 7RB United Kingdom Tel: EDHEC Risk Institute North America One Boston Place, 201 Washington Street Suite 2608/2640 Boston, MA United States of America Tel: EDHEC Risk Institute France rue du 4 septembre Paris France Tel: +33 (0) EDHEC Risk Institute Asia 1 George Street #07-02 Singapore Tel: