Rating the Ratings. April Edhec Risk and Asset Management Research Centre

Transcription

1 Rating the Ratings April 25 Edhec Risk and Asset Management Research Centre

2 Contents Executive Summary. Introduction 3 2. The various quantitative fund ratings 5 2. Standard & Poor s Star Ranking Morningstar Aptimum 2.4 Lipper EuroPerformance-Edhec Style Rating 4 3. A comparative analysis of the various ratings 9 Bibliography 33 Web sites 34 Edhec Risk and Asset Management Research Centre 35 Copyright Edhec 25

3 About the Authors Noël Amenc Noël Amenc, PhD, is professor of finance at the Edhec Graduate School of Business, where he heads the Risk and Asset Management Research Centre. He has conducted active research in the fields of quantitative equity management, portfolio performance analysis and active asset allocation, resulting in numerous academic and practitioner articles and books. He is an Associate Editor of the Journal of Alternative Investments and Senior Academic Fellow of the Europlace Institute of Finance. Véronique Le Sourd Véronique Le Sourd has a Master s Degree in Applied Mathematics from the Pierre and Marie Curie University in Paris. From 992 to 996, she worked as Research Assistant within the Finance and Economics Department of the French Business School, HEC. She is currently a Senior Research Engineer at the Edhec Risk and Asset Management Research Centre. He is also a member of the scientific council of the French financial authority (AMF).

4 Executive Summary Foreword In 22, EuroPerformance, which is the leading French firm for the dissemination of mutual fund data, approached the Edhec research centre to consider the implementation of a value-added offering in the area of external analysis of the performance and risks of European investment funds. In view of the increasing importance of ratings in orienting the subscription flows in open distribution architecture, not only in the USA but also in Europe, it quickly became clear that one of the key success factors for this new offering would be the implementation of fund ratings. We noticed that the low level of predictability of the existing ratings, their conceptual and technical inadequacies with regard to the progress made by modern portfolio theory and, above all, the dissatisfaction of professionals and investors, could allow a new actor to play a leading role in a market that seemed at first sight to be saturated with offerings proposed by prestigious service providers, some of whom enjoy a certain international aura in the area of ratings. The method followed to implement the design of these new ratings was to carry out a thorough study of the insufficiencies of the existing rating methods in order to correct them by relying on the state-of-the-art in portfolio risk and performance measurement in a business context. The document which is presented to you here constitutes a detailed summary of the critical study and puts into perspective the responses given to the inadequacies of the existing ratings by the EuroPerformance-Edhec Style Ratings. The ratings compare funds that are not comparable The analysis of the main existing ratings carried out by Edhec shows that for numerous categories, notably the categories corresponding to equity funds invested in largecap stocks and internationally diversified funds, the rankings contain a mix of investment vehicles that present totally different risks and as such are not comparable. The style exposure averages in the "equities" category are not representative, since the mean dispersion (standard deviation) of the weights of each style is often greater than their average. The performance-related ranking methods based on the Information Ratio implemented by Lipper and S&P exacerbate the fragility of the ratings linked to the poor definition of categories, because the ranking is extremely sensitive to the choice of index, which is included in the calculation of both outperformance and relative volatility (tracking error). More globally, all of the ratings are based on rankings that are related to categories, which makes them particularly fragile and not very robust, since they depend on the prior definition of the categories. The stars attributed in each category are not comparable and do not have the same value. Depending on the breadth of the category to which they belong, the same funds can find themselves being attributed different levels of stars. The style of the funds is not genuinely taken into account Even though academic research work has shown that the choice of style (Growth, Value, Small Cap or Large Cap) was a determining factor in a portfolio's risk and performance over a long period, this notion is either poorly taken into account or not taken into account in the ratings. As a result, none of the major international rating agencies are capable of distinguishing clearly between value funds and growth funds in the European market. Moreover, when an agency attempts to do this, the integration of the style is not based on any serious analysis. Almost 7% of the funds ranked as Large Cap in the Morningstar Global Equity or Equity Euroland Large Cap categories actually have a majority exposure to the Small Cap style. Alphas missing from performance measurement and scores that depend on the chance of financial markets Furthermore, these approaches do not lead to each fund being attributed a benchmark that is representative of the risks taken. The fund's ranking is not therefore produced on the basis of its real outperformance (alpha), but depends on the overall performance which itself is conditioned strongly by whether the prevailing situation is favourable for the styles to which the mutual funds are exposed. With the exception of the scores attributed by Aptimum, the authors of the study estimate that, for the vast majority of rating categories, the scores attributed do not depend on either the manager's stock picking or tactical allocation capability, but on their exposure over a long period to one or more styles, without it being possible to say whether this selection of risks over the long period is due to skill or luck. This inability to distinguish between performance and outperformance makes any use of ratings as a tool for selecting managers or funds illusory. For the most part, the best scores obtained come from the evolution of the risk premiums of the styles to which the funds are exposed. Financial theory postulates that these follow a random walk or, at least, cannot be determined by only observing past values.

5 Potential extreme loss not taken into account; the lessons from the stock market crashes have still not been learnt The study also shows that the calculation of the extreme loss potential (VaR) to which the funds are exposed is either not performed (Morningstar, S&P and Aptimum) or not taken into account (Lipper) in the fund's overall score, even though funds are increasingly turning to derivative instruments, and are invested in assets that present nonnormal distributions, which makes the use of volatility as the only risk measure illusory. Performance persistence measurement is inadequate Finally, performance persistence measurement is not given enough attention. Most rating agencies do not provide a satisfactory solution, either because of a lack or inadequacy of the measure (S&P, Morningstar and Aptimum), or because it is not integrated into the overall score (Lipper). New ratings to respond to the inadequacies of the current ratings On the basis of the critiques and the solutions proposed by modern portfolio theory, EuroPerformance and Edhec have proposed a method that breaks free from the criticism described above: It is an absolute rating that only allows a fund-specific benchmark to be calculated for each of the funds. The attribution of stars no longer depends on a category. The outperformance (alpha) is the principal criterion for attributing stars and alpha is a magnitude that does not depend on the prevailing situation and thus on the chance of the financial markets, but only on the manager's skill, the persistence of which can be measured. The fund's extreme loss potential (VaR) is measured and integrated into the overall rating process. The measurement of the persistence of outperformance is subjected to a thorough quantitative analysis and is also integrated into the overall rating process. Evaluation of the different ratings Provider S&P Morningstar Lipper Aptimum EuroPerformance- Edhec Measure of risk actually taken Measure of extreme risks Measure of performance persistence Robustness and confidence Transparency and comprehensibility Poor Average Poor Very Good Very Good No No Average No Good No No Very Good Good Very Good Poor Poor Poor Good Very Good Very Good Very Good Good Poor Very Good 2

6 . Introduction With the increasing number of investment products in Europe, the funds being made available to investors are more numerous and increasingly diversified in terms of assets and style management, which renders the choice between them more complex. In order to promote their products, the various financial establishments are seeking tools to evaluate the performance of their funds in an easily understandable way. Fund ratings satisfy their needs by offering a tool that evaluates the respective qualities of the funds and allows them to be compared. The interest in ratings is shared by all the players in the investment management field. For fund subscribers, ratings allow funds to be evaluated and compared quickly on the basis of return and risk, making it possible to check whether the performance of the fund is up to the level of risk taken. For management firms, ratings serve as a sales argument, allowing them to underline the quality of the funds they offer. The economic press is interested in publishing fund star ratings as these ratings can be easily understood by the general public. Finally, the firms that supply the ratings base part of their reputaion on those ratings. However, the various existing ratings do not all exhibit the requisite properties for offering investors and fund managers reliable information. Fund rating methods are divided into two kinds: qualitative methods and quantitative ones. The present document will focus on quantitative ratings. Qualitative ratings were introduced in the mid-97s with Standard and Poor's rating. This type of rating is not free of charge and only concerns a restrictive number of funds. The method used is based on financial analysis and fund manager interviews. The rating uses letters, from AAA for the best rating to D for the lowest rating. The rating is updated each year. Apart from Standard and Poor's, this type of rating is also proposed by Fitch IBCA. Unlike qualitative ratings, quantitative ratings are free of charge and concern all funds. The evaluation is based on an estimation of the fund's risk-adjusted performance. These ratings use stars (or equivalent symbols), from five for the highest rating to one for the lowest. Quantitative rating is an area that is tending to grow and is becoming a reference for investors. In the United States, quantitative ratings have been used since the middle of the 98s with the creation of the Morningstar ratings. Standard and Poor's and Lipper's - a branch of Reuters - also offer quantitative ratings. These ratings are published in an extensive way on the promoters' web sites. Beside these three ratings, which are aimed at the general public, Aptimum developed a rating system based on Arbitrage Pricing Theory in 999. Launched at the same time as the Euro, these ratings are of particular interest to us as they are disseminated on a European level by several European newspapers working in tandem. In order to make it accessible to a broader audience, Aptimum has recently made the presentation of its ratings simpler. The latest quantitative rating created is the EuroPerformance-Edhec Style Rating, launched at the end of 24 by EuroPerformance in association with Edhec. it f i Numerous studies performed on US mutual funds have concluded that fund subscribers are widely influenced by fund ratings in making their choice. Moreover, if the rating of a fund is modified and becomes lower, one observes a transfer of money towards other funds. For example, Del Guercio and Tkac (22) found that an initial Morningstar 5-star rating results, on average, in six months of abnormal flows (53% above the normal expected flow). They also found significant abnormal flow in the case of rating changes, with positive flow for rating upgrades and negative flow for rating downgrades. Adkisson and Fraser (23a) also present significant evidence that investors withheld funds from mutual funds that lost stars, but did not proportionately reward funds that gained stars. With the development of open architecture distribution for investment products in Europe, it is very likely that European investors will be influenced by ratings in the same way when subscribing to funds. This increasing influence of ratings in the area of investment management makes both investment management firms and investors more careful about the quality of these ratings and, specifically, their robustness. Meanwhile, the predictive capability of the existing fund ratings has not been proved. Using Morningstar ratings, Khoran and Nelling (998) found that fund ratings exhibit persistence over a thirty-month period: highly rated funds tend to continue to be successful in the future, and poorly rated funds continue to underperform relative to their peers. Blake and Morey (2) found somewhat different results. They found that low ratings indicate relatively poor future performance, while there is little evidence that the highest-rated funds outperform the medium-rated funds. Moreover, the ratings are not a better predictor of performance than the fund's past average monthly returns. Finally, Morey (23) showed that three years after a fund received its initial Morningstar 5-star rating, fund performance falls off severely. As far as the study of fund return predictability is concerned, research work proceeds in two different ways. Some consider the fund performance as a whole, without making a distinction between risk premiums, the reward for the various risk factors to which the fund is exposed, and the specific performance due to manager skill. In that case, it is difficult to try to identify any kind of persistence in the performance. Assuming that the performance as a whole can persist would be equivalent to supposing that style and asset class returns are auto-correlated and persistent, which is not in accordance with the weak form of market efficiency. Others evaluate the performance by considering the fund's alpha. In separating alphas from betas, it is possible to isolate the success of management decisions from the returns due to the economic situation. We recall that the return of a portfolio or a fund may be written in the following way: R - r = a + b ( F - r ) + e i( F t - rf ) + bi2( F2 t - rf ) bik a i r f where is the abnormal return of the fund due to stockpicking and/or market timing; is the risk free rate; Kt f it 3

7 b ik is the sensitivity of the fund to factor k; return of factor k during period t and the products evaluate the return due to the risks taken. is the Alpha is in fact the only reliable performance measure for active management. When the persistence of style and asset class returns over a long period has not been proved, there is a higher probability that the manager's ability may persist and it is possible to seek to evaluate this persistence. This duality in fund return analysis is to be found in the rating methodologies. An initial category of rating (Standard & Poor's, Morningstar) compares the performance of a fund to an index or to the performance of funds belonging to fixed peer groups, without making a distinction between risk premiums and abnormal returns due to the portfolio manager's decisions. Rankings are compiled according to the main asset categories (equities, bonds, etc.), which are split into sub-categories according to different kinds of style, but without taking the real risk exposure or portfolio style into account. The creation of fixed style classes is in fact an artificial construct since there is no obvious way to determine where the breakpoints should be between the classes. It is apparent that funds can have very different risk levels within a specific category. By placing a portfolio in a fixed style class, there will always be portfolios near the boundary of the class. As a result, the performance of such portfolios will be compared with other portfolios that are in the same class, but at the other end of the class, and are quite different in terms of style character. Conversely, the performance of these portfolios will not be compared with that of portfolios in an adjacent style class with which they have much in common. Moreover, some portfolios change their investment style considerably over time due to the nature of their investment strategy, so assigning such portfolios to a single fixed style class is not suitable. Therefore, when using fixed asset classes, it seems difficult to qualify the performance observed. In particular, it is impossible to distinguish between normal returns, provided through exposure to different risks, from abnormal performance (or outperformance) due to the manager's skill, whether through market timing or stock picking. In that case, it seems hopeless to try to identify any persistence in performance. Indeed, the fact that an investment style performs well or badly should not be confused with the manager's skill in picking the right stocks within the style he has chosen. A manager's skill in practising a well-defined style should thus be evaluated in comparison with a benchmark that is adapted to that style, so that his ability to achieve returns for the different risk factors may be identified. Therefore, a second category of rating makes a distinction between the amount of performance due to the manager's ability and that due to the performance of the asset classes or style in which he is invested. In that case, a portfolio manager's performance is evaluated using the fund's alpha and it becomes possible to seek to identify persistence in alpha. The new ratings launched by EuroPerformance and Edhec belong to this second category. F kt b ik F k Having observed that the rankings of the main existing rating services no longer satisfy the principles of reliability required by both investors and management firms, EuroPerformance and Edhec wanted to create new ratings that would overcome the failings of the ratings developed previously. These requirements, which will be developed in the present study, concern the following points: genuinely taking risk into account in performance measurement; being able to evaluate performance persistence; obtaining robust and reliable results; and presenting transparent and comprehensive ratings. Existing ratings encounter two main difficulties in taking risk into account. The first concerns the measurement of the risks that were really taken by the manager, and the second concerns the necessity of taking extreme risks into account. The risk measure, which is the core of fund ratings, often refers to the average performance of a category, without the rating agency being completely sure that the category or the index really describes the fund's strategic allocation, and thus the risk to which the fund has been exposed. Moreover, most of the rating systems do not explicitly take into account or do not try to evaluate the funds' extreme loss risks. Whether it is the indicator chosen (information ratio, volatility) or how risk aversion is modelled, risk-adjusted measures consider that the investor is averse to average investment risk, but do not have any specific point of view with regard to extreme risks. The second point concerns the fact that ratings are used by investors to make investment decisions and not only to reward past performance. Unfortunately, despite its usefulness, performance persistence measurement does not seem to be of major concern for rating agencies. The third point relates to confidence in the results. Most of the rating systems use a relative ranking of fund performance. This approach makes the ratings attributed totally dependent on the definition of the categories. However, the increase in the number of investment products and managers does not allow the rating agency to be sure that all the funds in the same category are like each other. Ratings are thus compiled for funds that do not have the same risk profile, and consequently the information given to investors is not reliable. Finally, as major information for investors and management firms, fund ratings must be easily understandable and must rely on calculations that are significant to professionals and individuals. The present study will describe the various quantitative methods available for fund ratings and introduce the EuroPerformance-Edhec Style Ratings. It will then give a detailed comparison on the various methodologies underlining their respective weak and strong points. The aim of this study is to supply precise information on each rating system, so that they may be used with all due caution. 4

8 2. The various quantitative fund ratings 2. Standard & Poor's Star Ranking Principles This rating is based on Markowitz's model, according to which investors seek to maximize the expected return of their portfolio, while minimising the risk. The Standard & Poor's Star Ranking proposes to assist investors in evaluating the performance of a fund and the consistency of that performance relative to other funds in the same sector. Fund Selection and Categories Standard & Poor's track 9, funds in 52 countries, with 54, funds in Europe and, international indices. In France, Standard & Poor's cover 9, funds, of which 4,5 are governed by foreign law. These 9, funds are divided into 6 asset categories. The categories relate to the country, sector, asset type, etc. Standard & Poor's stars are computed for funds that have a minimum period of historical data of 3 years and belong to a GIFS (Global Investment Fund Sector) category containing at least 6 funds during the evaluation period. Of the 9, funds, only 4,5 receive a star ranking. Methodology Rankings are based on a fund's monthly performance relative to the sector average over each of the preceding 36 months. The return of each fund in the category is computed for each of the 36 months in the period. The return of fund i during month t is denoted by R it. The mean average return of all the funds in the category is also calculated for each month. Assuming there are N funds in category S, the monthly mean average return R St is given by: R = The relative monthly return of each fund is computed as the difference between the fund's monthly return and the monthly mean average return of the funds in the category, i.e. Rit - R St. The calculation of the average of the fund's latest 36 relative monthly returns enables its average outperformance or underperformance compared to the other funds in the same sector to be measured. The higher the average, the more the fund has outperformed its peers over the 36 months. The volatility of the fund's relative return is then computed as: N Â St R it N i= 36 s ( Ri - RS ) = Â(( R where R i = Â R it and R S = Â R St. 36 t= 36 N We note that RS = Â R i. N i= The volatility measures how much a series of values deviates around its average. The calculation of the volatility of the fund's 36 relative monthly performances allows for the measurement of how consistently the fund has outperformed or underperformed its peers. The higher the volatility, the less consistent the fund's performance relative to its peers. It can be seen as a tracking error indicator between the fund and the average of the category. Finally, Standard & Poor's Relative Risk Adjusted Ratio is calculated by dividing the fund's average relative performance by the volatility of its relative performance. This is in fact an information ratio. The information ratio is equivalent to a Sharpe ratio where the riskless asset has been replaced by the return of a benchmark. Here, the benchmark is the average return of the sector. In a formal way, the information ratio (IR) can be written as follows: IR = This ratio measures the ability of the fund to outperform its peers on a consistent basis. The funds are ranked on the basis of this ratio. Funds that tend to do well under this method are those that behave similarly to their sector (they rise and fall together) but steadily pull ahead of their peers over time. For this calculation, each month is given an equal weight. Management fees, custodian charges, and all other running costs are taken into account in the computation of fund returns. Attribution of Stars t= 36 Â t= ( R s ( R The higher the information ratio, the greater the fund's past ability to consistently outperform its peers and the more stars it will be given. Five stars are given to funds with ratios in the top %, four stars go to those in the next 2%, three stars go to those in the following 2%, while funds in the next 25% receive two stars and those in the bottom 25% are awarded just one star. The stars are recalculated every month. Standard & Poor's stress that their star rating is not intended to be predictive and that past performance does not give information on future performance. Observations The methodology is simple to understand. However, the existence of numerous categories may make it difficult it i it - R - R - R S St St ) - ( R - R ) ) i S t= )) 2 5

9 to compare the performance of funds between categories. The aim of Standard & Poor's is to reward funds that behave in the same way as the funds belonging to the same category, while exhibiting outperformance, so only the relative risk of the fund with regard to the average of the category is taken into account and no indication is given about the fund's absolute risk. The information ratio used to perform the rating is very dependent on the benchmark chosen to calculate this ratio: different benchmarks yield different information ratios. The use of an average of the funds belonging to a specific category as a benchmark does not ensure that the risk of all the funds in the category will be perfectly represented. 2.2 Morningstar Principles The Morningstar Rating was introduced in 985 in the United States. It has most often been used to help investors and advisors choose one or a few funds from among the many available within broadly defined asset classes. Over time, though, increasing emphasis has been placed on the importance of funds as portfolio components rather than "stand-alone" investments. In this context, it is important that funds within a particular rating group be valid substitutes for one another in the construction of a diversified portfolio. Moreover, according to Morningstar the relative star ratings of two funds should be affected more by manager skill than by market circumstances or events that lie beyond the fund managers' control. For these reasons, Morningstar has revised its methodology. The revised approach was made available on June 3, 22. Morningstar now assigns ratings based on comparisons of all funds within a specific Morningstar Category, consistent with the Morningstar practice in Europe. The star rating is based on risk-adjusted performance. The Morningstar Risk-Adjusted Return considers that, assuming investor preferences, higher return is good and higher risk is bad under all circumstances, without regard to how these two outcomes are combined. Fund Selection and Categories Fund categories are defined according to the following principles: Funds are grouped by the type of investments that are predominant in their portfolios. In general, a single return benchmark should form a valid basis for evaluation of the returns for all funds in a single category In general, funds in the same category can be considered reasonable substitutes for the purposes of portfolio construction Category membership is based on a fund's long term or "normal' style profile. At a given point in time, the fund's current Morningstar Style Box assignment may differ from its Morningstar Category. The list of fund categories depends on the area of the world in which the star rating system is used. In Europe, Morningstar has created a single set of fund categories for the entire European onshore and offshore fund universe. Each fund in Europe is allocated to only one category and so will receive a unique Morningstar Rating, applicable anywhere in Europe. Fund categories define groups of funds whose members are similar enough in their risk factor exposures for the return comparisons between them to be useful. The main elements on which the categories are based are: Equity funds (invest in stocks): country or region, economic sector and market capitalisation. Style will be selectively introduced in 25, bringing the European approach closer to the US practice. Bond funds: credit quality, currency and maturity. Balanced funds (invest in a mixture of bonds and stocks): the denomination currency and the allocations to domestic bonds (in the relevant currency) Morningstar categories will keep evolving to reflect a changing marketplace and innovation by managers. Methodology Performance measurement Morningstar calculates a fund's total return for a given month as follows: where: P e is the NAV per share at the end of the month; is the NAV per share at the beginning of the month; P b D i P i Ï n P = Ì Á Ê e D + i R Ó Pb i= Ë Pi is the distribution per share at time i; is the NAV reinvested per share at time i; n is the number of distributions during the month. Distributions include dividends and distributed capital gains, where applicable. All calculations are based on pretax return data. If there were no loads or redemption fees, the cumulative value of one Euro over a period of T months would be: If there are loads or redemption fees, the cumulative value adjusted for loads and redemption fees is given by: V = ( - F)( - R) V T V u = ( + R t ) t= - D( u - ˆ - min( P F) P, PT ) 6

10 where: F is the front load; D is the deferred load; R is the redemption fee; P is the NAV per share at the beginning of the period; is the NAV per share at the end of the period. P T Morningstar Risk-Adjusted Return The Morningstar Risk-Adjusted Return (MRAR) measure has the following characteristics. It assumes no particular distribution of excess returns and risk is penalised in all cases. MRAR is motivated by expected utility theory, according to which an investor ranks alternative portfolios using the mathematical expectation of a function (called the utility function) of the ending value of each portfolio. According to financial theory, each individual has his own utility function. Nevertheless, in order to be able to establish a fund ranking, Morningstar assumes that all investors use the same utility function. The function chosen is the isoelastic (or power) function defined by: -g Ï W Ô- u( W ) = Ì g Ô Ó ln( W ) g > -, g g = where W is the ending value of the portfolio being considered and g is a parameter that describes the degree of risk aversion. The choice of this particular function is based on the desirable properties required to describe investors' attitude towards risk. To be meaningful, the utility function must satisfy two conditions. First, it must always be positively sloped; i.e., u ( W ) >. That is, more expected ' wealth is always preferred to less expected wealth. Second, it must imply that investors are risk averse. This means that the investor prefers a riskless portfolio with a known end-of-period value to a risky portfolio that is expected, but not certain, to have the same end-of-period value. This is described by the fact that the utility of the expected wealth must be greater than the expected utility of the wealth: u ( E[ W ]) > E[ u( W )] From probability theory, it follows that this can be true " only if the utility function is concave, i.e. u ( W ) <. The degree of risk aversion of investors can be measured by the coefficient of relative risk aversion (RRA), defined by: " Wu ( W ) RRA( W ) = - ' u ( W ) In the specific case of the utility function defined above, the relative risk aversion is equal to the constant g +. Constant relative risk aversion utility functions are particularly suitable, as they imply that the risk aversion of the individual does not depend on the amount of his wealth. Assuming is the value of the portfolio at the beginning of the investing period and R the return of the portfolio during the period, the terminal wealth of the investor will be W = W ( + R). Replacing W with this expression in the utility function will lead to: Instead of holding a risky portfolio, the investor could buy a risk-free asset. Let R F be the return on the risk-free asset. To compare risky portfolios to the risk-free asset, we assume that the investor initially has all his wealth invested in the risk-free asset and that the beginning-of-period wealth is such that the end-of-period wealth will be. Hence has the following value: and Ï Ê + R ˆ Ô- Á g Ì Ë+ RF u( W ( + R)) = Ô Ê + R ˆ ln Ô Á Ó Ë+ RF Defining the geometric excess return as: we have: The certainty equivalent geometric excess return of a risky investment is the guaranteed geometric excess return that the investor would accept as a substitute for the uncertain geometric excess return of that investment. Letting CE r G denote the certainty equivalent geometric excess return for a given value of hence: and: u( W ) = (g ) r u( + r CE G W W -g Ï W Ô- Ì g Ô Ó ln( W G r W G - Ï ( + rg ) Ô- ) = Ì g Ô Ó ln( + rg ) Ï Ô = Ì ( E[( + r ) E ÔÓ e + R = + R -g G [ln(+ r G )] ( + R) ( + R)) = +, this means that: CE u ( + rg ( g )) = E[ u( + rg )] CE -g ( + rg ( g )) = E[( + rg ) g - -g R F F g - g ]) - - g > -, g -g g = g > -, g g > -, g g = -g g = ] g > -, g g = 7

11 MRAR( ) is defined as the annualised value of -g using the time series average of ( + r G ) as an -g estimate of E[( r ) ]. With g, we have: where: is the geometric excess return in month t; is the return on the risk-free asset in month t. R Ft When : r G g g =, MRAR is the annualised geometric mean of A rating system based solely on performance would rank funds on their geometric mean return, or equivalently, MRAR(). A rating system that provides a heavier penalty for risk requires that g >. Morningstar considers that g = 2 results in fund rankings that are consistent with the risk tolerances of typical retail investors. Hence, Morningstar uses a g equal to 2 in the calculation of its star ratings. It also calculates a risk indicator as MRAR() - MRAR(2), i.e. the difference between the mean annual geometric return and the certainty equivalent annual geometric excess return. To calculate MRAR when there are loads and redemption fees, total monthly returns must be adjusted. Let a be the adjustment factor: where: V is the cumulative value adjusted for loads and redemption fees; V u is the cumulative value not adjusted for loads and redemption fees The adjusted total return for month t R t + G È MRAR( g ) = Í ÎT r È MRAR() = Í Î AR Ê a = Á Ë = t a( + R ) - where is the total return for month t. Gt t T Â t= + Rt = + R T t= ( + V V u ( + AR t is given by: Loads and fees can then be incorporated into the calculation of MRAR by replacing R t with AR t. Ft ˆ T r Gt - r Gt ) -g ) 2 - T 2 - g - - CE r G Attribution of Stars Each fund is placed in the category indicated in the most recent monthly record. For each category, Morningstar calculates a three-year star rating for all member funds that have at least 36 continuous months of total return data, up to and including the evaluation month. In extreme cases where the funds in a category vary widely in their risk factor exposures (i.e. it is a "convenience category") a star rating would have little value and is not assigned. To assign three-year ratings to funds in a given category, Morningstar calculate the load-adjusted MRAR(2) of total returns for the 36 months ending in the evaluation month. The ranking is then produced as follows. All the funds in a peer group are ranked in descending order of their MRAR(2). A number of stars is then attributed to each fund according to its position in the distribution of MRAR(2) values. The funds in the top % of the distribution obtain five stars; those in the following 22.5% obtain four stars; those in the following 35% obtain three stars; those in the next 22.5% obtain two stars; and, finally, those in the bottom % obtain one star. The rankings are updated each month. In the United States, ratings are also performed on longer periods if the data is available: five-year ratings are assigned using 6 months of data and -year ratings are assigned using 2 months of data. An overall star rating for each fund is then based on the weighted average of the number of stars assigned to it in the three-year, five-year, and -year rating periods. If the fund in question has been in its current category over its entire evaluation period, the weights are:.6 for the 5-year rating and.4 for the 3-year rating if there are less than years of historical data; and.5 for the -year rating,.3 for the 5-year rating and.2 for the 3-year rating if there are years or more of historical data. Morningstar note that though the long-term overall star rating formula seems to give the most weight to the -year period, the most recent three-year period actually has the greatest impact because it is included in all three rating periods. If the fund has changed categories over time, a matrix is used to measure the similarity between the current category and the fund's historical categories. (For months that do not contain a category record, the category is assumed to equal that of the closest month that contains a category record.) The weights given above are then modified based on the fund's average degree of similarity to the current category for the months contained in the rating period. The length of period used to perform the rating is the main difference in methodology between Europe and the US, although this discrepancy will disappear over time. Observations Based on Morningstar's old methodology, Chiang, Kozhevnikov and Wisen (23) compared Morningstar's ranking with other risk-adjusted perform- 8

12 ance evaluations. They found that the RAR and the excess return from the CAPM regression yield similar star ratings, but they find systematic differences between the star ratings produced by the RAR and the excess return estimated from the Fama-French (993) three-factor model. A problem of age bias in risk estimation was described in Blume (998), Morey (22) and Vinod and Morey (22) before the new methodology was introduced. Morningstar used to rate funds regardless of age differences: the estimates upon which younger fund ratings were based have significantly higher estimation risk than the estimates upon which the ratings of older funds were based. According to Adkisson and Fraser (23b, 24), despite the improvements to the methodology, a potential age bias still exists in the ratings. This problem is linked to the weighting of the different periods. Morningstar first calculates a risk-adjusted return for the 3-year, 5-year and - year horizons; it then multiplies these adjusted returns by weights of.2,.3 and.5 respectively. These weights are quite arbitrary. A mutual fund with relatively poor recent performance may still be highly rated if its longer-term returns are strong, due to the fact that Morningstar puts relatively little weight on the most recent performance. The improvements between the old star rating system and the new one have related to the following points. First, Morningstar have replaced the broad categories with narrow groups. The old star rating system was too heavily influenced by cyclical market trends. The stars were rewarding the asset class favoured by the market more than good managers. For instance, during periods in which large-cap stocks dominated, large-cap funds received a disproportionate percentage of high star rankings in the broad "domestic equity" asset class. Likewise, when small-cap stocks dominated, it was small-cap funds that earned a higher percentage of five-star rankings. This was a source of major problems when the speculative bubble burst. Funds with five-star ratings encountered considerable losses. Concerning bond funds, most high-yield bond funds were awarded four or five stars with the previous methodology, because high-yield bonds produced higher returns than the rest of the bond universe. Morningstar's new rating system has thus been designed to correct this type of problem. The new system compares a large-cap growth fund's performance with the performance of other large-cap growth funds. In addition, a fund's rating no longer depends on the relative strength or weakness of the fund's style. Investors who only used Morningstar ratings to select funds in the 99s tended to select growth funds. This was beneficial during the latter part of the nineties, but became unfavourable during the 2-22 period. Before the tech plunge in 2, for example, most fivestar-rated funds were either growth or technology-oriented. Those funds dropped 4% that year, while one-star funds, which were then dominated by value and small caps, gained nearly 6%. In theory, the new star allocation makes it less likely that the one-star funds will outperform the five stars. Narrow groups take style management into account better. However, some categories had to be defined broadly to incorporate enough funds for a meaningful comparison group. This implies that there are larger style differences among funds in some categories. Moreover it is difficult to define funds as "similar". Apart from the problem of broad categories, the old system did not evaluate fund risk accurately. In the late nineties, a lot of Internet funds obtained very high ratings. They were not penalised for being risky, when in fact their extraordinary upside returns were indicating that a potential risk of loss existed. The problem of risk evaluation was also brought to light for bond funds when two Heartland Group Inc. high-yield municipal bond funds, which both had five-star ratings, lost 7% and 44% of their value in one day. These two funds were among the best performing municipal funds, but it appears that they were full of illiquid and distressed securities. Their risk was not evaluated accurately. Risk calculation has been modified in the new version of the star rating. Risk is measured by monthly variations in fund returns and now takes not only downside risk but also upside volatility into account, but with more emphasis on downward volatility. Funds with highly volatile returns are penalised, whether the volatility is upside or downside. With the new process, risks that have not yet surfaced will be apparent in funds that have extraordinary upside volatility. The advantages of this improvement can be understood by looking at Internet funds. These funds were not considered risky in 999, as they only exhibited upside volatility, but their extreme gains indicated serious potential for extreme losses, as has been demonstrated since. The new risk measure would have qualified those funds with a higher risk level than the previous measure did. As a result, the possibility of strong short-term performance masking the inherent risk of a fund has now been reduced and it is more difficult for high-risk funds to earn high star ratings. Despite all these improvements, we do not think that all the previously encountered problems have been solved. Fund returns are still considered as a whole, without separating alpha from risk premiums. Even if funds are now grouped into categories according to their style, the use of style categories defined a priori does not guarantee that all the funds in the same category are genuinely homogeneous in their risk profile, as was explained in the introduction. It is not therefore possible to isolate the amount of manager skill - the only component of performance that may persist - from the performance of the style, and to determine whether the performance observed over a period may persist in the future. As a result, the rating is not a reliable indicator for choosing future investments. This is corroborated by Morningstar, who stress that their ratings tell more about the past than they do about the future. They also indicate that the stars show how well a fund has balanced risk and return in the past and that they have little predictive value. Finally, the implementation of the methodology in itself constitutes a difficulty for Morningstar. The rating agency is not able to verify and monitor the composition of portfolios, even though knowledge of the composition is essential for classifying funds. Consequently, Morningstar has to rely on the categories that are auto-proclaimed by managers. Moreover, the managers are obliged to declare a specific category for their portfolio, while it may be diversified among several styles. This discrepancy 9 A detailed study of the fund styles for several Morningstar categories will be presented in the third section

13 between the reality of the investment management and the requirement for a "black and white" view of the style leads to very significant errors of appreciation in the relative performance of the funds. As such, their ranking is more often due to an error or insufficiency in the categorisation than to manager skill. 2.3 Aptimum Principles APT model The rating system developed by Aptimum relies on APT (Arbitrage Pricing Theory). This model, which was developed by Stephen Ross in 976, is based on the principle of valuing assets through arbitrage theory in an efficient market. According to this model, the return on an asset depends linearly on several factors, shared by all the assets, and a specific factor for each asset, which is expressed by the following formula: where: denotes the expected mean return of asset i; a it b ik R K it = it + Â k = denotes the sensitivity (or exposure) of asset i to factor k; F kt denotes the return of factor k with E( F k ) = ; e it denotes the residual (or specific) return of asset i, i.e. the share of the return that is not explained by the factors, with E( e i ) =. The APT model offers more detailed risk decomposition than the CAPM, which is a one-factor model. However, despite its usefulness, the model has for many years been neither widely used nor well known within the financial community, due to the difficulties in implementing it. These difficulties related to the choice of the factors and the extensive calculations involved. As computers became more powerful, the use of this theory was rendered easier. The form of the model used in the ranking proposed by Aptimum has been developed since 985 by the American firm Advanced Portfolio Technology, of which Aptimum is the French branch. This company has made the APT theory operational by deriving a methodology to construct accurate risk factors for the model. Instead of using macroeconomic factors defined a priori, as in the original model, they evaluate these factors in an implicit way. The idea behind this approach is to use asset returns to characterise the unobservable factors. Factors are extracted directly from the historical returns, with the help of methods drawn from factor analysis. Risk factor construction a b F + e To identify the basic factors that drive security prices, ik kt it Aptimum analyses the movements of approximately 3, share, bond, commodity and currency prices in various markets and countries over a 8-week period. A standard approach would consist of factorising the variance/covariance matrix of asset returns through principal component analysis. However, applied to large sample data this will cause biased results. So Aptimum do not carry out traditional factor analysis, but use a specific statistical estimation technique, which enables the factors to be estimated from a large sample of variables, and produces unbiased estimates of the covariance matrix. The idea is to group assets into homogeneous classes according to the similarities they exhibit in their behaviour. The combination of the individual asset returns within each group serves to derive an index for each group. These indices constitute the factors. They are constructed in such a way as not to be correlated, which is an advantage compared to the economic variables, which are often correlated. These factors have strong explanatory power. The optimal number of factors is obtained by the best compromise between the search for precision in replicating the initial matrix, and the need to retain a reasonably-sized model. About 2 factors which are intended to explain the behaviour of the market are determined in that way. The analysis is repeated every quarter, adding the most recent quarterly return data, and dropping the oldest. The factors identified are then used to analyse the funds' performance. Fund Selection and Categories The funds analysed are those available in Europe that are able to supply their net asset value daily, or at least on a Wednesday, for a minimum of two years. The analysis is performed using coherent fund categorisation dedicated to the European marketplace. This categorisation is objectively arrived at by statistically discriminating clusters of similarly behaving funds rather than taking managers' self-determination of how they wish to be labelled. Using the risk factors identified in deriving the APT model, the exposure of every fund to each of the factors is estimated, together with the specific fund return component not taken into account by factors. These calculations are carried out by regressing the factors onto the fund returns. For each fund, the set of exposures to the risk factors determines its risk profile. In particular, it becomes possible to determine a "distance" between each fund in terms of the difference between their estimated exposures. The distance is a measurement of how much the values of two funds have been likely to diverge from one another. Funds that have similar risk profiles will have the same type of behaviour and will be gathered in the same group. Identification of peer groups is performed by looking at two variables: the APT correlation between any two funds; and the overall volatility of the fund compared to the other funds. This volatility is expressed in relative terms by calculating the beta relative to a reference. As a result, funds are then grouped according to the types of risks to which they are exposed. Funds that are more heavily exposed to a particular country or region are grouped together, as are funds that are more heavily exposed to a particular industry or investment strategy.