Sngapore Management Unversty Insttutonal Knowledge at Sngapore Management Unversty Research Collecton School Of Accountancy School of Accountancy 5-2006 Loss Functon Asymmetry and Forecast Optmalty: Evdence from Indvdual Analysts' Forecasts Stanmr Markov Emory Unversty Mn Yen TAN Sngapore Management Unversty, mnyen@aslouken.com Follow ths and addtonal works at: http://nk.lbrary.smu.edu.sg/soa_research Part of the Fnance and Fnancal Management Commons, and the Portfolo and Securty Analyss Commons Ctaton Markov, Stanmr and TAN, Mn Yen. Loss Functon Asymmetry and Forecast Optmalty: Evdence from Indvdual Analysts' Forecasts. (2006). Research Collecton School Of Accountancy. Avalable at: http://nk.lbrary.smu.edu.sg/soa_research/165 Ths Workng Paper s brought to you for free and open access by the School of Accountancy at Insttutonal Knowledge at Sngapore Management Unversty. It has been accepted for ncluson n Research Collecton School Of Accountancy by an authorzed admnstrator of Insttutonal Knowledge at Sngapore Management Unversty. For more nformaton, please emal lbir@smu.edu.sg.
Loss functon asymmetry and forecast optmalty: Evdence from ndvdual analysts forecasts Stanmr Markov (Stanmr_Markov@bus.emory.edu) and Mn-Yen Tan (Mnyen_Tan @bus.emory.edu) Gozueta Busness School Emory Unversty 1300 Clfton Rd. Atlanta, GA 30322 Ths verson: May 9, 2006 Abstract: We examne the optmalty of quarterly earnng forecasts ssued by ndvdual analysts. When we conduct Ordnary Least Squares (OLS) and Least Absolute Devatons (LAD) analyses, whch assume loss functon symmetry, we reect the null of forecast optmalty at 5% sgnfcance level more than 5% of the tme. Relaxng the symmetry assumpton reduces the frequency of reectons below 5%. We demonstrate that the cross-sectonal varaton n the asymmetry parameter of the loss functon s related to analyst employment. Overall, our evdence s consstent wth the ont hypothess of asymmetrc loss and forecast optmalty rather than the alternatve of symmetrc loss and lack of optmalty. We thank Orre Barron, Sudpta Basu, Murge Krshnan, Larry Brown, Marty Butler, Andy Leone, Jng Lu, Bruce Mller, Mke Parzen, Grace Pownall, Ane Tamayo, and semnar partcpants at Emory Unversty, Georga State Unversty, and Penn State Unversty. We also thank semnar partcpants at Baruch College and UCLA for ther comments on a related paper. Fnally, we thank Ron Harrs and Chrstopher Baum for programmng assstance and IBES for analysts data.
Introducton Many studes have documented that consensus forecast errors are predctable based on easly accessble publc nformaton, and have then concluded that fnancal analysts forecasts are not optmal (Mendenhall, 1991; Abarbanell and Bernard, 1992). 1 These conclusons have been questoned recently from two perspectves. The frst one questons the evdence of forecast error predctablty based on the fndngs lack of robustness (Abarbanell and Lehavy, 2003; Cohen and Lys, 2003). The second one questons the nference about lack of optmalty. For example, Basu and Markov (2004) argue that ths nference s drven by an napproprate assumpton of quadratc loss, and they document almost no predctablty n consensus forecast errors when conductng the tests under the assumpton of lnear loss. They conclude that analysts expectatons are consstent wth ratonalty and that suggestons that nvestors make the same cogntve mstakes are premature. 2 We extend pror research on forecast ratonalty n two ways. Our frst nnovaton s that we explore loss functon-based explanatons for the evdence of forecast predctablty at the ndvdual analyst level (Lys and Sohn, 1990; Jacobs and Lys, 1999; Mkhal et al., 2003, among others). Whle most of the debate about analyst ratonalty takes place at the consensus level, there are good reasons to study forecast optmalty and nformaton use at the ndvdual analyst level. Frst, heterogenety n loss functons makes ambguous the nterpretaton of the tests of 1 Whle dfferent studes use dfferent terms n nterpretng the evdence of forecast error predctablty, we thnk that the common theme n these nterpretatons s that the forecasts are not optmal. For example, seral correlaton n forecast errors s explaned wth analysts underreacton to nformaton, or neffcent nformaton use, due to the exstence of cogntve bases (Mkhal et al., 2003) and analysts underestmaton of the persstence of ther forecast errors (Mendenhall, 1991). See Kothar (2001) and Ramnath et al. (2006) for addtonal references to studes that document forecast error predctablty, and nfer that forecasts lack optmalty. 2 A thrd perspectve s taken by Markov and Tamayo (2006), who explore Bayesan learnng as an explanaton for the evdence of forecast error predctablty. 1
ratonalty at the consensus level. 3 When we average the forecasts to construct the consensus, we average not only over ndvdual analysts mstakes, but also over heterogeneous loss functons. Snce no optmzng ndvdual constructed the mean forecast to mnmze her expected loss, t s not clear what reecton of the null, or falure to reect the null at the consensus level, means for how analysts use nformaton. Second, forecast users make decsons based on access to specfc analysts. Thus, knowledge of ndvdual analysts forecastng obectves and forecast neffcences s of enormous practcal mportance to forecast users. Our second nnovaton s that we conduct bootstrap nferences to address Abarbanell and Lehavy s (2003) crtcsm about pror studes lack of attenton to the severe non-normalty of the forecast errors. Bootstrap does not requre assumptons about error dstrbuton or nfntely large samples, and, n most samples, t typcally provdes a more accurate approxmaton to the dstrbuton of an estmator than asymptotc theory (Horowtz, 1997; Efron and Tbshran, 1998). We analyze a comprehensve sample of forecasts ssued late n the current quarter over the perod of 1985 to 2004 by 2,489 analysts employed by 303 nvestment frms. Assumng that fnancal analysts loss functon s symmetrc, we fnd that analysts do not effcently use nformaton n past forecast errors and earnngs. In partcular, our OLS (LAD)-based tests reect the null of forecast optmalty at a 5% sgnfcance level between 11% and 16% (6 % and 8%) of the tme. These reecton rates are based on the bootstrap approach, whch does not assume normalty or large sample szes. We note that the asymptotc theory reecton rates can be twce 3 Heterogenety n loss functons lkely arses from dfferences n the mx and relatve mportance of varous actvtes fundng nvestment research (Cohen et al., 2006). It seems unlkely that the set of mplct and explct contracts faced by an analyst employed by a frm that sells research for a fee are the same as the ones faced by an analyst workng for a full-servce nvestment frm where research s bundled wth the sale of other products and servces. See Krgman et al. (2001) and Irvne (2000) for evdence of nvestment research s contrbuton to underwrtng and brokerage revenues. 2
as hgh as the bootstrap reecton rates. Thus, we are confdent that the severe non-normalty of the forecast errors dscussed n Abarbanell and Lehavy (2003), whle an mportant cause for concern n pror research on earnngs forecast errors, cannot be an explanaton for these fndngs. We next explore whether the reectons of the hypothess of forecast optmalty are due to the fact that ndvdual analysts forecastng obectves are () heterogeneous and () poorly approxmated by a symmetrc loss functon mpled n the OLS and LAD analyses. In partcular, we conduct tests of forecast optmalty under more general loss functons that nest the tradtonal quadratc and lnear loss functons. The ln-ln and quad-quad loss functons generalze the tradtonal lnear and quadratc loss functons by ntroducng a sngle asymmetry parameter, α, to allow for the possblty that the cost of a forecast error depends on ts sgn. A parameter value of 0.75 (0.25) means that the penalty for a postve forecast error s three tmes as hgh (low) as the penalty for a negatve forecast error. The quadratc and lnear loss functon obtan when α s equal to 0.50. We have several reasons for usng the ln-ln and quad-quad loss functons. Frst, snce the arguments for why fnancal analysts obectves are best captured by a symmetrc loss functon are not very strong (Lambert, 2004), t seems ustfed to use an approach that allows rather than assumes loss functon symmetry. Second, many practtoners and researchers beleve that, late n the quarter, fnancal analysts forecastng obectves are, n fact, to ssue forecasts that are easy to beat (Matsumoto, 2002; Bartov et al., 2002; Rchardson et al., 2004). Such ncentves are nconsstent wth a symmetrc loss functon, α=0.50, but consstent wth α beng less than 0.50. If at the end of the quarter the cost of underpredctng earnngs s lower than the cost of overpredctng earnngs, then analysts wll systematcally underpredct earnngs by ssung forecasts that are easy to beat. Fnally, these loss functons are conventonal n the 3
economcs lterature, wth recent advances n econometrc theory enablng the estmaton of the asymmetry parameter α (Granger and Newbold, 1986; Ellott et al., 2004; Ellott et al., 2005; Rodrguez, 2005; Markov and Tan, 2005). When we use the ln-ln functon, whch nests the lnear loss functon, we reect the null of optmalty less than 5% of the tme, whch s consstent wth pure chance. When we use the quad-quad functon, whch nests the quadratc functon, we reect between 6% and 12% of the tme. The asymmetry parameter of the loss functon, α, s generally lower than 0.5, whch means that the cost of underpredctng earnngs s generally lower than the cost of overpredctng earnngs. We conclude that the tradtonal OLS and LAD tests reect the hypothess that all analysts have the same symmetrc loss functon rather than the hypothess that ther forecasts devate from optmalty. An alternatve explanaton for our fndngs s that we are overfttng the data by ntroducng a parameter devod of any economc substance. To preclude ths explanaton, we examne whether ln-ln and quad-quad loss functons are reasonable representatons of analysts forecastng obectves. We fnd that analysts wth smlar α s tend to work for the same employer, and that an analyst whose α devates from that of her current employer s more lkely to experence turnover. These results are especally strong for the asymmetry parameter of the ln-ln loss functon, whch valdates the ln-ln functon as a more reasonable representaton of analysts ncentves than the quadratc loss functon. We conclude that the varaton n our estmates of the shape of the loss functon s related to varaton n ncentves rather than nose, and that overfttng does not explan our fndngs of forecast optmalty. Next, we brefly dscuss pror lterature wth an emphass on studes that depart from the tradton of assumng symmetrc loss. Secton 3 descrbes our sample and reports the results from 4
the tradtonal OLS and LAD analyses. Secton 4 presents our man analyses based on Ellott et al. s (2005) framework that allows for loss functon asymmetry. Secton 5 concludes. 2. Pror lterature on analysts loss functons and forecast optmalty The tradtonal approach to examnng whether forecasts are optmal nvolves estmatng wth OLS the model FE = β + β X + δ, (1) t+ 1 0 1 t t+ 1 where FE t+1 s analyst forecast error at tme t+1 and X t s a vector of varables known at the tme of the forecast, t. Reectng the hypothess that β 0 =0 and β 1 =0 s vewed as evdence of lack of optmalty as one can use publcly avalable nformaton, X t, to further reduce the mean squared forecast error. Postve values of β 1 are typcally nterpreted as evdence of underreacton to nformaton or neffcent nformaton use. Basu and Markov (2004) pont out that mplct n ths approach s the assumpton of a quadratc loss. In other words, the analyst s vewed as tryng to mnmze her mean squared error. Followng Gu and Wu (2003), Basu and Markov (2004) argue that analysts ncentves are better represented by a lnear loss functon; analysts attempt to mnmze ther mean absolute error. They estmate a verson of equaton (1) usng the LAD method rather than OLS and fnd that the coeffcents are ndstngushable from the predcted values under the null of optmalty. They stll, however, often reect the null hypothess that consensus forecasts are consstent wth forecast optmalty. A number of recent studes on analyst forecasts queston the approprateness of the quadratc and lnear loss functons (Lambert, 2004; Rodrguez, 2005; Clatworthy et al., 2005; Markov and Tan, 2005), and n partcular, the assumpton that the consequences of 5
overpredctng earnngs are the same as the consequences of underpredctng earnngs. The symmetry of these loss functons s nconsstent wth the vew that analysts have ncentves to ssue optmstc reports early n the quarter and pessmstc forecasts late n the quarter, and the evdence of generally negatve mean and medan forecast errors for long-term forecasts and postve mean and medan forecast errors for short-term forecasts (Rchardson et al., 2004). 4 Clatworty et al. (2005) examne the propertes of fnancal analysts forecast errors under an asymmetrc loss functon ntroduced frst by Varan (1974), the lnex loss functon vs-à-vs the propertes of the forecast errors under the lnear loss functon. A relaton between forecast error bas and varance of the forecast error dstrbuton s predcted under the asymmetrc loss functon, but not under the lnear loss functon (Chrstoffersen and Debold, 1996, 1997). The evdence n Clatworty et al. (2005) reects the lnear loss functon n favor of the asymmetrc loss functon. Ths evdence, however, does not shed lght on the ssue of forecast optmalty, whch s at the center of the other two emprcal studes, Rodrguez (2005) and Markov and Tan (2005). Both Markov and Tan (2005) and Rodrguez (2005) rely on the econometrc framework of Ellott et al. (2005), also used n ths study, to descrbe the asymmetry of the fnancal analysts loss functon and examne forecast optmalty. Rodrguez (2005) analyzes a sample of 107 analysts and fnds evdence consstent wth forecast optmalty. He also shows that rsk averson can be mportant for understandng the propertes of analysts forecast errors. An mportant dfference between hs study and our study s that our sample s much more comprehensve (2,489 analysts). Ths allows us to examne whether low power explans hs falure to reect the null of optmalty. In addton, we show that the ndvdual analysts loss functon asymmetry vares systematcally across frms, whch allevates the concern that the tests ntroduce an extra parameter devod of economc content. 4 The quadratc (lnear) loss functon predcts a zero mean (medan) forecast error. 6
Markov and Tan (2005) examne the optmalty of the consensus forecasts and fnd that consensus forecasts are more consstent wth ratonalty n the perod after Regulaton FD became effectve. We vew emprcal analyses of ndvdual and consensus forecast errors as complementary. Researchers examne the propertes of the consensus forecast, defned as the mean or medan forecast, when ther prmary focus s on studyng the belefs of the margnal nvestor. They use the consensus forecast as a proxy for the unobservable margnal nvestor s belefs. However, there are good reasons to study ndvdual analysts errors. Frst, wthout knowledge of the forecastng obectves and ratonalty of ndvdual analysts, we cannot really understand how the market for nvestment research functons. Second, snce forecast users rely on nvestment research by ndvdual analysts, t s mportant for them to know the forecastng obectves and bases of the analysts whose research they have purchased or consder for purchase. Thrd, f dfferent analysts have dfferent loss functons, then evdence about mean or medan forecast becomes hard to nterpret. For example, ndvdual analysts may ssue forecasts that are optmal under ther own loss functons, but the average of ther forecasts may be suboptmal as each analyst pays attenton only to her own loss functon. The opposte s also possble. Indvdual analysts ssue sub-optmal forecasts, but the average of ther forecasts s optmal. Evdence of forecast optmalty at the consensus level does not dstngush between the hypotheses that () all analysts have the same loss functon and they make errors that cancel out, or () analysts have dfferent loss functons, make systematc mstakes, but there exsts a loss functon parameter value that ratonalzes the consensus. 5 5 If analysts have dfferent nformaton sets, then tests of ratonalty at the consensus level can reect the null of ratonalty even f ndvdual analysts are ratonal (Fglewsk and Wachtel, 1983). 7
3. Tests of forecast optmalty under loss functon symmetry In ths secton we descrbe our sample and report evdence about forecast optmalty based on OLS and LAD regressons, whch assume loss functon symmetry. 3.1. Sample constructon Our prmary data come from the Insttutonal Brokers Estmate System (I/B/E/S) database. We use the I/B/E/S Detal Earnngs Estmate Hstory Fle, whch contans 1,564,054 ndvdual analysts forecasts of current quarter earnngs of U.S. companes for the perod from 1985 to 2004. We elmnate forecasts of quarter t earnngs that are dated after the announcement of quarter t earnngs (160,656 observatons). We focus on forecasts that are ssued n the second half of the quarter because they are the ones most lkely to determne the earnngs surprse on announcement dates (530,541 observatons). These forecasts are more lkely to ncorporate nformaton avalable at the begnnng of the quarter (Soffer and Lys, 1999), and more lkely to be systematcally based downward relatve to forecasts ssued at the begnnng of the quarter (Rchardson et al., 2004). We elmnate forecasts ssued n the frst half of the quarter (52,793), observatons wthout two pror earnngs announcements (41,928), observatons where share prce from two quarters ago s less than $1 (2,108), and observatons wthout pror forecast errors (120,049). These flters ensure that we use the most up-to-date analyst forecasts, that we can examne analysts use of publcly avalable nformaton such as earnngs and past forecast errors, and that we do not nflate forecast errors when we dvde them by low stock prce to allevate heteroscedastcty concerns. The number of forecasts satsfyng these crtera s 313,663. These forecasts were ssued by 7,439 analysts. To ensure precson n our analyst-specfc estmates, we requre at least 30 observatons for an analyst, whch elmnates 42,942 forecasts (4,949 8
analysts). Fnally, we drop forecasts ssued by an analyst wth I/B/E/S code of 0000000, as ths code s assgned to all analysts who wsh to stay anonymous (2,070). Our sample has 268,651 forecasts ssued by 2,489 analysts. 3.2. Descrptve statstcs Panel A of Table 1 provdes descrptve statstcs about stock coverage for our sample of 2,489 analysts. These analysts were employed by 303 nvestment frms. They ssued a total of 268,651 forecasts on 7,379 companes. Over her tenure, an analyst n our sample ssued on average about 108 quarterly earnngs forecasts for 22 frms over 27 quarters. The medan number of forecasts ssued, frms covered, and quarters on I/B/E/S tends to be lower, whch suggests that there s a relatvely hgh proporton of analysts who were very experenced, followed many stocks, and ssued multple forecasts. Lkewse, some nvestment frms were much larger than others as the mean (medan) number of analysts employed and stocks covered are 16 (4) and 178 (39), respectvely. The reason for the relatvely low number of analysts employed and stocks covered by an nvestment frm s that we nclude only analysts wth at least 30 forecasts ssued n the second half of the current quarter. As we noted above, there are about 5,000 analysts who ssued fewer than 30 forecasts durng our sample perod. We denote frm s quarterly I/B/E/S earnngs per share (EPS) for quarter t+1 as A t + 1, and analyst s forecast of frm s EPS for quarter t+1 as F t + 1. The forecast error, denoted as FE t + 1, s defned as At+ 1 Ft+ 1. All varables are scaled by share prce recorded for the earnngs-announcement month of the quarter t-1 obtaned from I/B/E/S to allevate heteroscedastcty concerns and are wnsorzed at the 1% level on both tals to elmnate outlers. The mean and medan forecast errors of analyst are calculated based on all forecasts, F t + 1 9
ssued by analyst. Thus, they are analyst-specfc rather than analyst-frm-specfc. There are both advantages and dsadvantages to analyzng the propertes of FE rather than FE. An obvous advantage s that we are less prone to survval bas; we do not requre that analyst ssued at least 30 forecasts on frm, whch s the approach taken by Mkhal et al. (2003), but only that analyst ssued 30 forecasts. In addton, our choce to combne the dstrbutons FE and + 1FE s ustfed by Jacobs and Lys s (1999) fndngs about the exstence of common components n the propertes of the dstrbuton of FE and + 1FE. The dsadvantage s that analyst s forecast errors n quarter t are cross-correlated, a problem that we address later on by clusterng observatons or conductng a block bootstrap. We report mean, medan, standard devaton, and the 25 th and 75 th percentle of the cross-sectonal dstrbuton of 2,489 mean and medan forecast errors n Panel B of Table 1. We fnd strong evdence that the dstrbuton of medan forecast errors s centered at a postve number. Both the mean and the medan of the dstrbuton of medan forecast errors are postve, and we reect the null of zero medan forecast errors at 5% level n favor of greater than zero 44% of the tme. The frequency of reectng the null n favor of smaller than zero s only 2% and consstent wth chance. The evdence on mean forecast errors s mxed. The mean of the dstrbuton of mean forecast errors s negatve, -0.0002, but the medan s zero. In addton, we obtan hgh reectons of the null n favor of greater than zero 22% of the tme, and n favor of less than zero, 14% of the tme. The hgher frequency of reectons of the null n favor of greater than zero s not necessarly nconsstent wth the negatve mean of the dstrbuton of mean forecast errors. If statstcal precson s hgher n the sub-sample of analysts wth postve forecast errors, then we may reect more often. Overall, the general tendency s toward ssung low, beatable forecast errors that result n postve earnngs surprses. 10
3.3 OLS and LAD evdence about forecast optmalty The OLS (LAD) method s approprate f analysts have quadratc (lnear) loss functon. Basu and Markov (2004) document that the estmated coeffcents devate less from the predcted values when they use the LAD method than when they use the OLS method and argue that consensus annual forecasts are generally consstent wth optmalty under the lnear loss functon. However, even wth the LAD method, Basu and Markov (2004) often reect the null hypothess of forecast optmalty. To provde a baselne for our emprcal analyses, we frst conduct tests of forecast optmalty under the tradtonal assumpton of symmetrc lnear or quadratc loss functon. In partcular, for every analyst, we regress the forecast errors ( FE t + 1 ) on ntercept and nformaton varables known to the analyst at the tme of the forecast; past forecast errors ( FE ) and ntercept (Model 1), earnngs at lags 1 and 2 ( A t and At 1 ) and ntercept, (Model 2), and past forecast errors and past earnngs at lags 1 and 2 ( FE, A t, and At 1 ) and ntercept (Model 3). 6 We estmate these models usng OLS and LAD methods. In Table 2 we report the mean parameter estmates from our 2,489 analyst-specfc regressons. The last column summarzes the results from our 2,489 tests of optmalty; we report the percentage of tmes that we reect the null of optmalty at 5%. In vew of the severe non-normalty of forecast error dstrbuton (Abarbanell and Lehavy, 2003), we conduct both asymptotc-theory and bootstrap nferences. t t 6 Consstent wth pror lterature, we assume that the cost of accessng and processng publcly avalable nformaton for a sell-sde analyst s zero. We thnk that ths assumpton s more approprate for analyst s own forecast errors and past earnngs than for other varables such as accruals and stock returns, hence our focus on forecast errors and past earnngs. Our analyss, however, can be extended to other nformaton varables. 11
The mean coeffcents on past forecast errors and past earnngs are postve, whch s consstent wth pror evdence. The common nterpretaton s that analysts do not understand the propertes of the quarterly earnngs process, or underreact to earnngs nformaton. We note that the coeffcent on past forecast errors from our analyst-specfc regressons s 0.0823, whch s lower than the correspondng coeffcent n Mkhal et al. (2003), 0.1400. Gven that our sample covers a more recent perod, 1985 to 2004, vs-à-vs 1980 to 1995 n Mkhal et al. (2003), and that analysts ssue more accurate forecasts n recent years (Brown and Caylor, 2005), we vew our fndngs as beng n the same range. When we base our nferences on asymptotc theory, we get very hgh reectons of the null hypothess that all coeffcents are ontly equal to zero. The OLS regressons reect the null hypothess at 5% level between 34% and 56% of the tme. When we use the LAD method, we reect the null between 49% and 74% of the tme. These results are not drectly comparable, however. The LAD regressons assume ndependent and dentcally dstrbuted errors, whle the OLS regressons assume only tme ndependence. Wth heteroscedastcty and cross-correlaton lkely present n the data, the LAD frequency of reectons lkely overstates the true frequency of reectons. Due to the lack of asymptotc results about the dstrbuton of LAD parameters n the presence of heteroscedastcty or cross-correlaton that s analogous to the results about OLS parameters, usng the bootstrap n LAD analyss should be the preferred choce. 7 7 Rogers (1992) provdes Monte Carlo evdence that LAD asymptotc standard errors n the presence of heteroscedastcty sgnfcantly understate the true standard errors; he recommends that nferences be based on bootstrap standard errors nstead. 12
The Jarque-Bera test (results not tabulated) reects the null hypothess that the resduals come from a normal dstrbuton more than 90% of the tme. 8 The dstrbuton of resduals has both hgh skewness and kurtoss. The lack of normalty of the resduals combned wth the lmted sample szes used n the analyst-specfc estmatons makes asymptotc nferences suspect and provdes addtonal motvaton for conductng bootstrap nferences. We vew observatons from dfferent quarters as ndependent and observatons from the same quarter as dependent. Therefore, we sample blocks of observatons rather than ndvdual observatons; each block conssts of observatons for the same quarter. Drawng observatons by quarters ensures that each draw s ndependent of the other draws. We draw 1,000 samples wth replacement from our orgnal sample. We estmate our model 1,000 tmes to obtan the parameters bootstrap dstrbuton. Ths dstrbuton s the bass for calculatng standard errors and for conductng statstcal tests. We document a substantal drop n reecton rates when we conduct bootstrap nferences. In the case of the OLS regresson, the reecton rates are now between 12% and 20%, whch are stll too hgh to be explaned by chance. In the case of LAD regressons, we obtan reecton rates between 6% and 8%. Gven the substantal drop n reecton rates, we recommend that researchers studyng the propertes of analyst forecast errors conduct bootstrap nferences n addton to asymptotc theory nferences. 9 8 The Jarque-Bera test determnes whether the sample skewness and kurtoss are unusually dfferent from ther n ( ) 2 2 K 3 2 expected values under the normalty assumpton of 0 and 3. The test statstc s S + χ 2, where 6 4 n s number of observatons, S and K are sample skewness and kurtoss. It s possble that non-normalty of the forecast errors s at least partly drven by poolng quarter t s forecast errors of analyst. 9 Our recommendaton s most pertnent to studes on ndvdual analysts where we have a combnaton of small sample sze and hghly non-normal resduals. 13
3.4. Optmalty tests when the true loss functon dffers from the one assumed by the researcher The evdence so far suggests lack of forecast optmalty at the ndvdual analyst level assumng, of course, that analysts have symmetrc loss functon. The obectve of the analyss n ths sub-secton s to demonstrate that an earnngs forecast constructed under a slghtly asymmetrc loss functon would be found sub-optmal n OLS and LAD tests, whch assume symmetrc loss. 3.4.1. Generatng optmal forecasts We smulate forecasts of A t + 1 that ncorporate all nformaton n A t and At 1 to mnmze the sum of absolute forecast errors, where postve and negatve forecast errors are weghted by α and (1-α) respectvely; α {0.40,045,0.50,0.55,0.60}. In partcular, we estmate the regresson A, (3) t+ 1 = χ 0, α + χ1, α At + χ 2, α At 1 + ε t+ 1, α at varous quantles of the earnngs dstrbuton at tme t+1 for the same values of α; α {0.40, 045, 0.50, 0.55, 0.60}. The estmated coeffcents, by constructon, descrbe the condtonal quantle of A t+1 as a lnear functon of A t+1 and A t+1. They mnmze the sum of absolute errors where postve and negatve errors are weghted α and (1-α) respectvely. 10 Coeffcents and standard errors are reported n Panel A of Table 3. If a forecaster has a loss functon n whch the cost of underpredctng earnngs s 2/3 of the cost of over-predctng earnngs (α=0.40), then her optmal forecast of A t+1 would be equal to 0.006+ 0.7034*A t +0.2086* A t-1. A forecaster wth an α of 0.60 would construct her forecast 10 In contrast, OLS coeffcents descrbe the condtonal mean of the dependent varable as a functon of the dependent varable, and mnmze the sum of squared resduals. 14
dfferently; her forecast would be equal to 0.0038+ 0.6702*A t +0.1724* A t-1. In other words, each regresson decomposes A t+1 nto forecast Aˆt + 1, α and forecast error, ε t, α. The forecast and the forecast error are ndexed by α because they depend on how errors are penalzed. ˆ +1 3.4.2. Are generated forecasts optmal n OLS and LAD analyses? In the second step of our analyss, we examne whether the forecast errors ˆ ε t +1, α, can be predcted by A t and t 1 A usng OLS and LAD regresson methods. We estmate the model ˆ t+ 1, = β 0, α + β1, α At + β 2, α At 1 + δ t+ 1 ε α (4) usng OLS and LAD regresson methods. In Panel B of Table 3, we report the OLS and LAD coeffcents from the estmaton of equaton (4) as well as F-statstc from the ont test that all coeffcents equal zero. We reect the null of forecast optmalty n all specfcatons wth the excepton of α=0.5 n the LAD regresson. 11 These reecton rates are based on the more conservatve bootstrap approach. It s mportant to note that not only the ntercept but also the slope coeffcents sgnfcantly devate from the predcted values of zero. We conclude that forecast that are constructed to be optmally under asymmetrc loss, appear sub-optmal n OLS and LAD tests that assume symmetrc loss. 12 In practce, however, we do not know the value of the asymmetry parameter of the analyst s loss functon. Ellott et al. (2005) develop a method to estmate ths parameter and 11 Ths s not surprsng snce we generate the forecasts and tests for optmalty under lnearty and for the same α. 12 The bootstrap was mplemented by drawng pars of observatons. Thus, the bootstrap reecton rates are probably too hgh gven the lack of ndependence n the cross-secton. In our OLS analyss we also used heteroscedastcty consstent and robust to ntra-quarter cross-correlaton standard errors, whch rely on asymptotc theory, and smlarly reected the null of forecast optmalty for all values of α. 15
examne the extent to whch the forecasts are consstent wth forecast optmalty. 13 The next secton provdes a bref overvew of ther framework, used n our study, and reports our man fndngs. 4. Tests of forecast optmalty that do not assume symmetry 4.1 Econometrc method The consequences of makng an naccurate forecast are represented by the loss functon ( ) ( ) ( ) ( ) t+ 1 t+ 1 t+ 1 t+ 1( ) p L p, αθ, α+ 1 2α 1 A f θ < 0 A f θ. (5) The second term, A f ( θ ) t+ 1 t+ 1 p s the analyst s forecast error defned as the dfference between earnngs, A t+1 and the earnngs forecast, f ( θ ) observed by the analyst at tme t, f ( θ) t 1 +. The latter s a lnear functon of varables W t t 1 θ Wt. + = Dfferent values of θ represent dfferent forecastng rules, whch n turn result n dfferent forecast errors. The frst term n equaton (5), ( t+ 1 t+ 1 ) ( ) A f ( ) α + 1 2α 1 θ < 0 makes the cost of a forecast error condtonal on ts sgn. If α s equal to 0.5, then postve and negatve forecast errors are equally costly. In fact, when α=0.50 and p=1 or p=2, the loss functon reduces to that of the famlar cases of a lnear or a quadratc loss functon, wdely used n pror research on fnancal analysts. If α>0.5, however, then overpredctons are less costly to the analyst. In other words, the analyst has ncentves to overpredct earnngs. In sum, our gnorance about the analysts obectves conssts only of not knowng the value of the sngle parameter α, α ( 0,1). 13 The general dea of recoverng a parameter from the data that s most consstent wth optmzng behavor and assessng the extent to whch optmalty restrctons are satsfed n the data appears frst n Hansen and Sngleton s semnal (1982) study. 16
As an optmzng agent, the analyst chooses a forecastng rule f ( θ) mnmze her expected loss ( α θ ) t+ 1 = θ Wt to mn E L p,,.14 (6) θ If θ s chosen optmally, then the forecast errors must satsfy the frst-order condtons ( ( ) ) p 1 E Wt 1 εt+ 1 < 0 α ε t+ 1 = 0, (7) where εt+ 1 = At+ 1 θ Wt. 15 Havng access only to a subset of the nformaton avalable to the analyst at tme t, whch we denote as V t, does not prevent us from estmatng α. Snce an optmzng analyst explots any nformaton avalable to her at tme t, we can substtute V t for W t n the moment condtons and use the correspondng sample moments to back out the asymmetry parameter α. As long as we have more moment condtons than parameters to estmate, we are able to recover the asymmetry parameter wthout ad hoc ratonalzng the forecasts. The reason for ths s that the same α has to set two or more sample moments smultaneously to zero. Our estmator of α mnmzes a quadratc form where g ( ) T q = g Sg (8) T ( α ) ( α ) T α s the sample equvalent of equaton (7), and S s a weghtng matrx. Our weghtng matrx s the nverse of the covarance matrx of the moment condtons, whch 14 In other words, we vew the forecast as a choce that analysts make n tryng to enhance ther welfare a departure from the lterature s tradton of vewng forecasts as exogenously gven (Demsk, 2004). In a survey of the use of expectatons n accountng research, Demsk forcefully argues that relance on exogenous expectatons structures lmts the depth and boundares of teachng and research (p. 519). 15 Ths s proposton 1 n Ellot et al. (2005). 17
mnmzes the asymptotc varance of the GMM estmator. 16,17 In sum, GMM pcks the value of α that mnmzes the squared dstance between zero and the moment condtons dvded by the covarance of the moment condtons. Hansen s J-statstc, whch s equal to T tmes the mnmzed value of the quadratc form, measures the dstance between zero and the moment condtons, or how well the frst order condtons from the analyst s optmzaton problem, are satsfed n the data. It follows a chsquare dstrbuton wth degrees of freedom equal to number of moments mnus 1, the number of parameters estmated. Large values of the J-statstc mean that the dstance between zero and the moment condtons s too large to be explaned by chance, and that we should reect the ont hypothess that () analyst s loss functon s well approxmated by equaton (5) and () the forecasts are optmal. 18 4.2. Man fndngs We use as nstruments the regressors n the OLS and LAD regressons of Table 2; past forecast errors and an ntercept; past earnngs and an ntercept; forecast errors, past earnngs, and an ntercept. Panel A of Table 4 provdes evdence about the cross-sectonal dstrbuton of the asymmetry parameter and the frequency wth whch we reect the symmetry null of α=0.50 n favor of α>0.50 or α<0.50. In the case of the ln-ln functon, we reect the null of symmetry at the 5% level n favor of α<0.50 about 80% of the tme. We conclude that analysts have ncentves to systematcally underpredct earnngs (Brown, 2001, Bartov et al., 2002, among 16 The weghtng matrx determnes the relatve mportance of settng a partcular moment condton to zero when estmatng α. 17 We used Stata s vreg2 command and ts optons cluster and robust to produce heteroscedastcty-consstent and robust to ntra-quarter cross-correlaton standard errors. Sample code s avalable upon request. 18 It s standard to refer to the test as a test of over-dentfyng restrctons. In our settng, the restrctons hold f the forecast solves the mnmzaton problem (equaton (6)), and, thus, we refer to the test as an optmalty test. Cochrane (2001) provdes an nsghtful dscusson of the J-test and ts applcatons to tests of asset prcng models. 18
others). We fnd no evdence that any analysts have ncentves to overpredct earnngs snce the frequency of reectons, whch occur 2% of the tme, s low enough to be due to chance. The cross-sectonal mean and medan are about 0.31, whch means that the cost of underpredctng earnngs by 1 cent s about three tmes lower than the cost of overpredctng earnngs by 1 cent. In other words, analysts have ncentves to consstently underpredct earnngs. The asymmetry parameter of the quad-quad loss functon s 0.48. Ths should not be surprsng gven our sample evdence that the mean forecast error s very close to zero. As we have argued above, the assumpton that analysts true loss functon s quadratc (lnear) leads to the predcton that the mean (medan) forecast error s equal to zero. However, we observe a sgnfcant amount of varaton n the cross-sectonal dstrbuton of the asymmetry parameter, as the 25 th and 75 th percentle are about 0.32 and 0.63 respectvely. Thus, we get hgh frequency of reectons of the null not only n favor of α<0.50, between 27% and 32%, but also n favor of α>0.50, between 24% and 30% of the tme. The dfference between α quad-quad and α ln-ln s due at least partally to the fact that the quad-quad loss functon estmatons use squared forecast errors, whch ncreases the senstvty of our estmates to extreme observatons n ether tal of the dstrbuton. In Panel B of Table 4 we present the frequency wth whch we reect the null of forecast optmalty for the ln-ln and quad-quad functons. To stress the sgnfcance of relaxng the symmetry assumpton, we also present the reecton frequences when we assume symmetry (OLS and LAD analyss). In the case of the ln-ln loss functon, the reecton frequency s about 5%, whch s consstent wth pure chance. We conclude that relaxng the symmetry assumpton mplct n the LAD estmatons sgnfcantly changes our nferences about forecasts apparent lack of optmalty. 19
In the case of the quad-quad loss functon, we reect the null of forecast optmalty less often than n the case of OLS estmatons, but more than 5% of the tme: between 8% and 12% of the tme under the quad-quad loss functon, and between 11% and 16% of the tme under the quadratc loss functon. The evdence about the forecasts lack of optmalty s weakened, but stll exstent. We conclude that relaxng the symmetry assumpton s generally mportant for makng correct nferences about forecast optmalty. Under the ln-ln loss functon, whch nests the tradtonal lnear loss functon, the evdence s consstent wth forecast optmalty. Under the quad-quad loss functon, whch nests the tradtonal quadratc loss functon, the evdence s nconsstent wth forecast optmalty. The frequency of reectons of the null hypothess of forecast optmalty s slghtly hgher than what we would expect by pure chance. A mantaned assumpton of ths study s that the asymmetrc loss functon s a meanngful representaton of the forecaster s ncentves, and that varaton n α s not ust statstcal nose, but captures varaton n ncentves. We document slghtly hgher reecton rates of the null under the quad-quad loss functon, whch rases the queston of whch specfcaton s more approprate. To valdate the ln-ln and quad-quad loss functons as reasonable representatons of analysts forecastng obectves and to help determne whch specfcaton s more reasonable, we next explore the lnk between the estmated asymmetry parameters of the ln-ln and quad-quad loss functons and analyst employment. 4.3. Valdatng the asymmetry parameter We vew analysts employed by the same frm as conductng nvestment research n the same nsttutonal settng, whch means that they should face smlar sets of explct and mplct 20
compensaton contracts. If α s nformatve about analyst ncentves, not ust nose, we should observe that analysts employed by the same frm have smlar α s. We test ths predcton by regressng ndvdual analysts α s on 210 nvestment-frm ndcator varables whch are equal to 1 when an ndvdual analyst s employed by a partcular frm and are 0 otherwse. The dfference between 210 and 303 (number of nvestment frms n Table 1) s due to the fact an analyst who works for more than one employer s pared up wth her frst employer. If employment does not nfluence α, then these ndcator varables wll not help explan the crosssectonal varaton n α. We estmate α ln-ln and α lquad-quadn usng all nstruments (reported n the frst column of Panel A of Table 5). 19 The second and thrd columns report adusted R-squared and F-stats from the test that the coeffcents on the employment ndcator varables are ontly equal to 0. The employment ndcator varables explan over 12% (about 8%) of the varaton n α lnln (α quad-quad ). In all specfcatons we strongly reect the null that α s unrelated to employment. We conclude that varaton n α ln-ln and α quad-quad s related to varaton n ncentves rather than drven by statstcal nose. The hgher R-squared and stronger reectons n the case of α ln-ln suggest that varaton n α ln-ln captures better varaton n the crcumstances n whch forecastng performances are evaluated. Thus, we have more confdence n our results from the tests under the ln-ln loss functon. We also examne whether the dscrepancy between an analyst s α and the current employer s α, estmated by poolng observatons of all analysts employed by the same frm, has an effect on analyst turnover. The hgher the dscrepancy between an analyst s and an employer s α, the more dvergent the analyst s behavor from the behavor of the other analysts 19 The results do not change when we use subsets of the avalable nstruments. 21
employed at the same frm and the more lkely that she wll be separated from her current employer. There are 22 employers wth not enough observatons to estmate employer s α. Ths reduces our sample from 2,489 analysts to 2,457 analysts. The number of turnover observatons s reduced from 1,388 to 1,356. 20 We document a statstcally sgnfcant postve effect of the dscrepancy n α ln-ln on the probablty of a turnover (Panel B of Table 5). To help nterpret the logt coeffcent of 1.44, we calculate the probablty of a turnover at the mean value of the ndependent varable and at the mean-plus-one standard devaton of the ndependent varable. We fnd that ncreasng the ndependent varable by one standard devaton ncreases the probablty of a turnover from 54% to 57%. The farly small effect s not surprsng, however, gven the parsmony of the loss functon specfcaton and the estmaton error n α ln-ln. The coeffcent on α quad-quad s postve, but not statstcally sgnfcant. Fnally, we examne whether the turnover event matches analysts and employers wth smlar α s. We predct that, when an analyst wth hgh α (greater than 0.50) changes obs, he s more lkely to par up wth a hgh α employer. We examne ths predcton condtonal on the analyst beng currently employed by a hgh α or low α frm. The need to estmate new employer s α reduced our sample from 1,356 turnover observatons to 1,304 observatons. The results from our estmatons are reported n Panel B of Table 5 (Model 2). In three out of four specfcatons, we document that an analyst wth hgh α s more lkely to par up wth an employer wth hgh α. 21 In sum, we fnd that our parameter estmate, α, can explan turnover 20 There are 681 analysts who have more than one turnover observaton. The analyzed sample ncludes only the frst turnover observatons. 21 In the fourth specfcaton, we do not have enough varaton n the ndependent varable to estmate the model. 22
outcomes. Ths evdence makes t less lkely that our fndngs are due to overfttng the data by ntroducng a parameter devod of any economc content. 5. Conclusons Our study makes several contrbutons. Frst, we document that reectons of the null hypothess of forecast optmalty at the ndvdual analyst level are drven by the nvald assumpton of loss functon symmetry. Ths assumpton s () nconsstent wth the argument that analysts have ncentves to ssue beatable forecasts and () reected n the data. After allowng for loss functon asymmetry, we fnd evdence consstent wth forecast optmalty. Second, we address Abarbanell and Lehavy s (2003) concern about the senstvty of pror fndngs to dstrbutonal assumptons by conductng bootstrap nferences. Whle the use of bootstrap does not reverse the OLS fndngs of lack of forecast optmalty, we fnd a sgnfcant drop n reecton rates. We beleve that n some crcumstances the use of bootstrap can change our nferences, and recommend ts use. Thrd, we further establsh the role of ncentves n nvestgatons of the tme-seres propertes of analyst forecast errors. Whle we are able to reect the quadratc and lnear loss functon n favor of the quad-quad and ln-ln loss functons, we acknowledge that there could be alternatve loss functons that better approxmate the analysts forecastng problem. We thnk that proposng and estmatng loss functons that better approxmate analysts forecastng problem should be an exctng area of future research. Our evdence should not be consdered to be a general ndctment of the quadratc and lnear loss functons. There could be crcumstances n whch symmetry s a vald assumpton. Assumng symmetry because t s convenent, however, not only leads to nvald nferences 23
about forecast optmalty, but also prevent us from learnng about the nature of the forecasters ncentves from the data. There are other potental avenues for future research. In ths study, we consder only a few nformaton varables suggested by pror research as beng neffcently used by fnancal analysts. An mmedate extenson of our study s to examne addtonal nformaton varables such as extreme past earnngs (Easterwood and Nutt, 1999), accruals (Bradshaw et al., 2001), and past returns (Lys and Sohn, 1990). Another potental avenue for future research would be to examne the shape of the manager s loss functon mplct n her earnngs forecasts and the optmalty of her earnngs forecasts. 24
References Abarbanell, J. S., and V. L. Bernard, 1992, Tests of Analysts Overreacton/ Underreacton to Earnngs Informaton as an Explanaton for Anomalous Stock Prce Behavor, The Journal of Fnance 47 (3), 1181-1207. Abarbanell, J. S., and R. Lehavy, 2003, Based forecasts or based earnngs? The role of reported earnngs n explanng apparent bas and over/underreacton n analysts earnngs forecasts, Journal of Accountng and Economcs 36 (1-3), 105-146. Bartov, E., D. Gvoly, and C. Hayn, 2002, The rewards to meetng or beatng earnngs expectatons, Journal of Accountng and Economcs 33, 173-204. Basu, S., and S. Markov, 2004, Loss functon assumptons n ratonal expectatons tests on fnancal analysts earnngs forecasts, Journal of Accountng and Economcs 38, 171-203. Bradshaw, M., S. Rchardson, and R. Sloan, 2001, Do analysts and audtors use nformaton n accruals? Journal of Accountng Research 39 (1), 45-74. Brown, L., 2001, A temporal analyss of earnngs surprses: Profts vs. losses, Journal of Accountng 39, 221-241. Brown, L., and M. Caylor, 2005, A temporal analyss of quarterly earnngs thresholds: Propenstes and valuaton consequences, The Accountng Revew 80 (2), 423-440. Chrstofferson, P., and F. Debold, 1996, Further results on forecastng and model selecton under asymmetrc loss, Journal of Appled Econometrcs 11, 561-572. Chrstofferson, P., and F. Debold, 1997, Optmal predcton under asymmetrc loss, Econometrc Theory 13, 808-817. Clatworthy M., Peel, P., and P. Pope, 2005, Are analysts loss functons asymmetrc?, Workng Paper, Lancaster Unversty Management School. Cochrane, J., 2001, Asset Prcng, Prnceton Unversty Press. Cohen, A., Groysberg, B., and P. Healy, 2006, Whch types of analyst frms are more optmstc? Journal of Accountng and Economcs 41, 119-146. Cohen, D. A., and T. Z. Lys, 2003, A note on analysts earnngs forecast errors dstrbuton, Journal of Accountng and Economcs 36 (1-3), 147-164. Demsk, J., 2004. Endogenous expectatons, The Accountng Revew 70 (2), 519-539. Easterwood, J., and S. Nutt, 1999, Ineffcency n analysts earnngs forecasts: systematc msreacton or systematc optmsm? The Journal of Fnance 54 (5), 1777-1797. 25
Efron, B., and R.J. Tbshran, 1998, An Introducton to the bootstrap, Chapman and Hall, New York. Ellot, G., I. Komuner, and A. Tmmerman, 2005, Estmaton and testng of forecast ratonalty under flexble loss, Revew of Economc Studes 72, 1107-1125. Ellot, G., I. Komuner, and A. Tmmerman, 2004, Bases n macroeconomc forecasts: rratonalty, or asymmetrc loss? Workng Paper, Unversty of Calforna at San Dego. Fglewsk, S., and P. Wachtel, (1983), Ratonal expectatons, nformaton effcency, and tests usng survey data, Revew of Economcs and Statstcs 65, 529-531. Gu, Z., and J. Wu, 2003, Earnngs skewness and analyst forecast bas, Journal of Accountng and Economcs 35 (1), 5-29. Granger, C., and P. Newbold, 1986, Forecastng Economc Seres, 2 nd Edton, Academc Press. Hansen, L., and K. Sngleton, 1982, Generalzed nstrumental varables estmaton of nonlnear ratonal expectatons models, Econometrca 50, 714-735. Horowtz, J., 1997, Advances n econometrcs: theory and applcatons, Vol. 3, Ed. by D. Kreps, and K. Walls, Cambrdge Unversty Press. Irvne, P., 2000. Do analysts generate trades for ther frms? Evdence from the Toronto stock exchange, Journal of Accountng and Economcs 30 (2), 209-226. Jacobs, J., and T. Lys, (1999), Determnants and mplcatons of the seral correlaton n analysts earnngs forecast errors, Workng Paper, Unversty of Colorado and Northwestern Unversty. Kothar, S.P., 2001, Captal markets research n accountng, Journal of Accountng and Economcs 31, 105-231. Krgman, L., Shaw, W., and K. Womack, 2001, Why do frms swtch underwrters? Journal of Fnancal Economcs 60, 245-284. Lambert, R., 2004, Dscusson of analysts treatment of non-recurrng tems n street earnngs and loss functon assumptons n ratonal expectatons tests on fnancal analysts earnngs forecasts, Journal of Accountng and Economcs 38, 205-222. Lys, T., and S. Sohn, 1990, The assocaton between revsons of fnancal analysts earnngs forecasts and securty-prce changes, Journal of Accountng and Economcs 13 (4), 341-363. Markov, S., and A. Tamayo, 2006, Predctablty n fnancal analyst forecast errors: Learnng or rratonalty, Forthcomng n Journal of Accountng Research. 26