Transparency, Expectations, and Forecasts 1

Transcription

1 Transparency, Expecaions, and Forecass 1 Andy Bauer Senior Economic Analys Rober Eisenbeis Execuive Vice Presiden and Direcor of Research Daniel Waggoner Assisan Policy Advisor Tao Zha Policy Advisor Federal Reserve Bank of Alana Sepember This aricle is prepared for publicaion in he Federal Reserve Bank of Alana Economic Review. We hank John Roberson and Ellis Tallman for criical commens. Cindy Soo and Eric Wang provided excellen research assisance.

2 Transparency, Expecaions, and Forecass I. Inroducion Many macroeconomiss have argued ha a cenral bank should be ransparen abou is objecives, is views abou he economic oulook, and he reasoning behind is policy changes. 2 In 1994, he Federal Open Marke Commiee (FOMC) began o release saemens accompanying changes in he federal funds rae arge. Since hen, he degree of specificiy of he saemen and guidance provided on he likely course of fuure policy has evolved significanly. 3 In a recen paper, Woodford (2005) discusses wo kinds of cenral-bank communicaions: curren policy decisions and he cenral bank s view of likely fuure policy. He ariculaes four differen kinds of informaion ha a cenral bank migh seek o communicae o he public. These include informaion abou he cenral bank s view of curren economic condiions, curren operaing arges, sraegies guiding policy decision making, and he oulook for fuure policy. Woodford argues ha hese open communicaions are beneficial, no only from he poin of view of reducing he uncerainy wih which raders and oher economic decision makers mus conend, bu also from ha of enhancing he accuracy by keeping he expecaions of marke paricipans more closely synchronized wih is own. 2 See Faus and Leeper (2005). 3 Kohn and Sack (2003) characerize, in deail, he pre and pos saemens periods as he period wih he FOMC saemens released on changes in he discoun rae ( ), he period wih he FOMC saemens released on changes in he federal funds rae ( ), he period wih he FOMC saemens including policy il ( ), and he period wih he FOMC saemens including assessmen of he balance of risks ( ). In May 2003, a furher refinemen was added o separaely sae is views in he risks o inflaion and growh. And finally, in Augus of 2003, he Commiee provided explici guidance on he likelihood ha policy would remain accommodaive. 1

3 This paper invesigaes wheher he public s views abou he curren pah of he economy and of fuure policy have been affeced by changes in he Federal Reserve s communicaions policy as refleced in privae secor s forecass of fuure economic condiions and policy moves. In paricular, has he abiliy of privae agens o predic where he economy is going improved since 1994 when he FOMC began o release saemens conaining he Commiee s views of he economic oulook improved? If so, on which dimensions has he abiliy o forecas improved? The focus is on boh he shor-erm and he longer-erm economic forecass of key macroeconomic variables such as inflaion, GDP growh and unemploymen and policy variables such as shor erm ineres raes. Curren-year and nex-year forecass of privae agens are used o proxy for he public s shor-erm and longer-erm expecaions and empirical evidence is presened regarding wheher are performed o deermine hese forecass have performed beer in predicing fuure economic and policy condiions since The privae agens forecass used in his aricle are hose of individual paricipans, as well as he consensus (average) forecass, conained in he monhly Blue-Chip Economic Indicaors Surveys from 1986 o 2004, which include boh pre-fomcsaemen sub-period 1986: :12 and pos-fomc-saemen sub-period 1994: :12. We employ he economeric mehodology of Eisenbeis, Waggoner, and Zha (2002). I permis one o evaluae how good forecass are boh in cross-secion and across ime and o examine he errors in forecasing key economic variables on boh a univariae basis and a mulivariae basis. The laer is imporan, because agens are no simply forecasing one economic variable, bu raher a se of variables ha presumably are iner-relaed and joinly capure imporan dimensions of economic performance. 2

4 Good forecas abiliy on one dimension, bu poor overall performance may provide some indicaion abou how inernally consisen he forecaser s approach is. This cross-secional daa se enables us o decompose forecas accuracy ino wo componens: he common error ha affecs all individual paricipans and he idiosyncraic error ha reflecs discrepan views across individuals abou fuure economic and policy condiions. To preview he main resul, we find ha since 1994 he idiosyncraic errors for key macroeconomic variables have seadily declined and he expecaions of marke paricipans are more closely synchronized o one anoher. We find no evidence, however, ha he common error has become smaller since 1994, especially for he longer-erm forecass. II. Mehodology Le µ be a n 1 vecor of economic variables a ime, y be he realized value of hese economic variables, and i y is he ih individual s forecas value of he variables. Assume ha y is normally disribued wih mean covariance marix i y is normally disribued wih mean µ and forecaswise covariance marix R Ω and ha forecas. The covariance marix realized value of µ ; he covariance marix µ and economy-wise (common) F Ω. The super-scrips R sands for realized and F for R Ω reflecs he aggregae shocks ha affec he F Ω capures he discrepancy in forecass across individual paricipans. The assumpion ha he mean forecas among individual paricipans is µ is reasonable because previous work has suggesed ha he Blue-Chip consensus forecas as a proxy o he mean forecas is close o being an unbiased esimae 3

5 of µ (Bauer, Eisenbeis, Waggoner, and Zha, 2003). Denoe he forecas error for he ih forecaser by x = y y. I follows ha he individual forecas error i i i x has mean zero and variance marix Ω =Ω +Ω, R F which indicaes ha i x is subjec o boh idiosyncraic and common shocks. 4 The sandard saisical heory implies ha 2 where chi ( ) χ x Ω x chi n, i i 1 i 2 i n denoes he chi-square disribuion wih n degrees of freedom and χ is a square error weighed by ( ) Ω. The above expression says ha he weighed square error i χ follows he chi-square disribuion wih n degrees of freedom. To measure he forecas accuracy for each individual paricipan, we compue a score value ( p-value) associaed wih his chi-square disribuion and call i an accuracy score. The score for i individual forecaser i a forecas ime is a funcion of χ and n: where chi2cdf ( i, n) ( i i χ, ) 1 chi2cdf ( χ, ) p n = n, χ is he probabiliy ha a random observaion from he chi-square i disribuion wih n degrees of freedom falls in he inerval [0 χ ]. 5 p As Eisenbeis, Waggoner, and Zha (2002) poined ou, he summary measure ( i, n) χ is a probabiliy ha is invarian o he underlying scales of error variances. I 4 In fuure research, we inend o relax he assumpions ha he Consensus forecas is equal o µ and idiosyncraic shocks are independen of common shocks. 4

6 can be inerpreed ha he i h paricipan s forecas is closer o he realized value han do 100 p ( χ i, n) percen of all possible forecasers. Moreover, he score p ( i, n) compared across forecasers, wihin a forecas period, and across periods. χ can be Bauer, Eisenbeis, Waggoner, and Zha (2003) show how o esimae he covariance marices R Ω and F Ω. The marix R Ω can be esimaed as he sample covariance marix of he Blue-Chip Consensus forecas errors across ime under he assumpion ha R Ω is he same across years for each monh bu varies across monhs wihin a year. Thus, he variances on he diagonal of R Ω become smaller as ges closer o he end of he year, for more informaion becomes available o forecas economic condiions for he curren year. The covariance marix F Ω can be esimaed as he sample covariance marix of forecas errors across individual forecasers; his covariance varies boh across monhs and across years. 6 The esimae of Ω, denoed by Ω ˆ, is he sum of he esimaes of and F Ω. Given his esimae, he weighed square error can be calculaed as R Ω χ = Ω ˆ. 1 ˆ i i i x x A each ime, he average accuracy score is where N 1 i pˆ ( ) ( ˆ n = p χ, n), N i= 1 N is he number of individual forecasers a ime. One can also calculae he cross-secional disribuion of accuracy scores, which is described in deail in Box I. 5 If he assumpions used are valid, he disribuion of accuracy scores from 1986 o 2004 should be uniform. We have verified ha such a disribuion is more or less uniform, aking ino accoun smallsample uncerainy. 6 Oher esimaes can also be consruced using model-based mehods. 5

7 III. Vinage Daa and Forecas Errors The monhly Blue-Chip Economic Indicaors repor he forecass of key macroeconomic variables for he curren and nex years. We sudy he annual average forecass of five key variables: he 3-monh reasury bill rae, he consumer price index (CPI) inflaion rae, real gross naional produc (GNP) for 1986 o 1995 or real gross domesic produc (GDP) from 1996 o 2004, he unemploymen rae, and he long-erm bond yield (he corporae bond yield from 1986 o 1995 or he en-year reasury noe yield from 1996 o The hree-monh T-bill rae, he CPI inflaion rae, he unemploymen rae, and he long-erm bond yield are monhly variables while real GNP/GDP is a quarerly variable. This frequency difference is imporan o noe when evaluaing forecass. As he year ges close o he end, more informaion is available abou he acual curren-year daa and herefore he forecas errors for boh he curren and nex years ge smaller. For example, he forecasers paricipaing in he December Blue Chip survey will have daa monhly daa on he hree-monh T-bill rae and he long-erm bond yield hrough November. They will have daa on he unemploymen rae hrough Ocober or November. They will have daa on he CPI inflaion rae hrough Ocober. However, since GNP/GDP is released quarerly, forecasers will only have informaion regarding GNP/GDP hrough he hird quarer of he year. The weighed square error ˆ χ i is designed o avoid he influence of differen amouns of available daa so ha he errors are comparable across ime. To gauge forecas errors, he realized values of each variable a a given ime mus be used. The values of some of variables are revised over ime by he agencies 6

8 responsible for reporing hose variables. In paricular, real GNP/GDP is repored quarerly and revised wice. Every year addiional benchmark revisions may be made in July o he pas daa of GDP. Hence, wha is repored are he coninuously changing esimaes of he final values of many key economic variables. Finally, someimes he definiion of GDP is changed and he series is compleely revised. Wih such revisions aking place, he quesion arises as o wha vinage daa should one use o evaluae forecas errors? From a macro policy perspecive, we would argue ha he focus should be on he bes esimae of he final value of he variable of ineres. However, ofen ha value is no known for several years, and someimes he difference beween even a preliminary esimae and is neares neighbor esimaes can be very large. For example, he advanced esimae for real GDP for Q was 3.1% which was revised up by he Bureau of Economic Analysis from 3.4% and finally o 3.8% as more daa on he performance of he economy became available. The difference beween he firs and mos recen esimae could cause policy makers o infer ha he economy was growing below rend according o he firs number, bu above rend based on he final esimae. Such differences could have significanly differen implicaions for policy. For his reason, we would argue ha he focus should be on forecas mehods ha bes approximae he final number raher han he iniial esimae. Moreover, wih such a focus, a priori knowledge of he expeced performance of a model or forecasing mehod, can help inform he policy maker as o which evidence o give greaer weigh o, when here are significan differences beween he iniial releases of daa and forecass of hose daa. 7

9 For he purposes of his sudy of he curren-year forecass, we use he vinage daa available a he end of January following he curren year; and for he nex-year forecass, we use he daa available a he end January following he nex year. We use vinage daa so ha he resuls here will be comparable wih previous sudies. We also provide a comparison beween he average Blue Chip Consensus score using vinage and final daa, using January 2005 daa as our final daa. IV. Accuracy Scores In his secion we look a he disribuion of scores a each monh and examine wheher he disribuion has changed over ime, especially from he pre-saemen subperiod o he pos-saemen period. The echnical deails of how o characerize he cross-secional disribuion of scores are provided in Box I. Char 1A shows he ime-series pahs of average scores and sandard deviaions of scores for he curren year and Char 1B shows he ime-series pahs of skewness and kurosis for he curren year. Chars 2A and 2B show he similar pahs for he nex year. The measure of sandard deviaion is ofen used o approximae he volailiy of he public s expecaions or forecass a each poin in ime. As i can be seen from Char 1A, he average score flucuaes over ime and so does he sandard deviaion of scores. There are no noiceable differences in he degree of flucuaion before and afer 1994, and nor are here differences for any sub-periods afer There is no rend in which he average score has increased or he sandard deviaion of scores had decreased since There are clearly periods when forecasers made big errors, such as missing he onse of he recessions in 1990 and In addiion, while he average scores have increased in 8

10 he 2004, so have he sandard deviaions of hose scores. Similarly, he average scores dropped significanly in 1995, which is mainly caused by he definiion change of he GDP series. In January of 1996 he Bureau of Economic Analysis changed he measuremen of GDP o a chain-weighed sysem, bu he forecass made before January 1996 migh be based on he non-chain-weighed series. Ineresingly, his change seems having relaively less effec on he longer-erm forecas errors (Char 2B). The average score for he nex year (Char 2A) shows no improvemen since 1994 and in fac appears o have drifed lower since There has been a seady upward drif in he sandard deviaion of he scores since The paern of he drif in he sandard deviaion is similar o ha which occurred jus prior o and coming ou of he recession. As will be discussed furher in he nex secion, hese lower scores afer 1996 are mos likely associaed wih he naure of he business cycle and unexpeced growh in produciviy ha surged in he lae 1990s. We now look a he skewness and kurosis of accuracy scores (Chars 1B and 2B). Skewness measures how asymmeric he score disribuion is. The more negaive his measure is, he more scores spread ou oward 0%. Conversely, he more posiive his measure is, he more scores spread ou oward 100%. Kurosis measures how likely he score disribuion has exreme ouliers ha may affec he average score. The bigger he value of his measure is, he more likely we have ouliers in he score disribuion. For he curren-year forecass, he skewness and kurosis have remained sable excep for a few periods (Char 1B). The spike ha occurred in 1995 is due o he redefiniion of GDP and he small spikes around 2001 are associaed wih he recen recession. For he nex-year forecass, again, here is no clear paern or rend in which skewness and 9

11 kurosis have changed since 1994 (Char 2B). There were a couple of spikes in skewness and kurosis, whose periods correspond o he Asian financial crisis and he recen recession. To provide furher informaion abou disribuional changes of accuracy scores, we display in Char 3 he ime-series pahs of accuracy scores of Blue-Chip consensus forecas and he average of he op and boom 5 forecass for each monh. The currenyear resuls are repored in Char 3A and he nex-year resuls are in Char 3B. The consensus forecas is of paricular ineres because is score is on average he highes (see Appendix II for deails) and because i performs beer han any single individual forecaser over he sample. Again, i can be seen from Char 3 ha here is no endency ha hese scores have improved over ime since In fac, he scores of consensus forecass appear o be slighly lower afer 1996 han before, especially for he nex-year forecas. Moreover, he drop in he consensus scores around he recen recession and again following 9/11 in 2001, suggess ha evens and exogenous shocks affeced forecas performance much more han FOMC saemens. The drop in he scores owards he end of 1995 is due o he redefiniion of GDP. We also show he average scores for he op five forecasers in each period as well as he average score for he 5 poores performers. The evidence suggess ha daa have fa ails, wih mos of he forecass being clusered a he high end wih a few really poor performers on he boom. All hese findings sugges ha he individual paricipan s forecas performance relaive o oher paricipans has no improved beween he pre-saemen and possaemen periods. Alhough he accuracy score is a powerful summary measure of forecasing performance, i is a nonlinear funcion of he square forecas errors weighed 10

12 by he overall covariance marix Ω. I would be informaive o separae Ω and forecas errors for furher analysis. In he nex secion, we examine wheher he covariance marix F Ω has changed over ime and sudy he sources of forecas errors ha do no depend on F Ω and R Ω. 7 V. Transparency and Sources of Forecas Errors Kohn and Sack (2003) and Woodford (2005) argue ha he conens in FOMC saemens have become more ransparen since I is herefore imporan o see wheher he expecaions of marke paricipans via he forecass of key economic variables have become more synchronized in he pos-saemen period han in he presaemen sub-period. If here is useful informaion conen in he saemen, hen one migh expec ha here may be an overall improvemen in forecas accuracy, ceeris paribus, or a leas more agreemen among forecasers (ie. a igher disribuion of idiosyncraic errors.) A posiive answer may provide evidence abou he effecs of he FOMC saemens on he privae secor s agreemen on he direcion of he fuure economy. We also examine he sources of forecas errors by direcly decomposing he mean square error (MSE) ino he idiosyncraic componen reflecing he discrepancy in individual paricipans from he Surveys and he common componen ha is associaed wih unanicipaed aggregae shocks and affecs all paricipans. The echnical deails of his decomposiion are provided in Box II. 7 The reader may recall ha by assumpion relax his assumpion in fuure research. R Ω does no change from one year o anoher. We inend o 11

13 The MSE is he average of square errors across individual forecasers. Arguably, boh he idiosyncraic and common errors may show a decreasing rend if here is useful informaion in he saemen and forecasers gain beer undersanding of he economy over ime, especially since To he exen ha he common error is affeced by exogenous aggregae shocks, and he disribuion of he shocks is no consan, he here may be no clear inference abou he size of he common error. However, we hypohesize ha he more imporan impac is likely o be seen for he idiosyncraic componen, in ha he idiosyncraic errors should be igher ha is here should be greaer agreemen among he forecasers. The empirical resuls presened below confirm his hypohesis. We firs sudy how synchronized he expecaions of marke paricipans are. The degree of synchronizaion is measured by he cross-secional sandard deviaions of all he variables, which are equal o square roos of he diagonal elemens of F Ω. Chars 4-8 repor he cross-secional sandard deviaion of each of he five macroeconomic variables considered in his paper. Chars 4A-8A display he sandard deviaions for boh he curren-year and nex-year forecass; Chars 4B-8B display he 12-monh moving averages of he sandard deviaions o show he rend more clearly. I is clear from hese chars ha for no only he ineres raes bu also he oher variables, he rend has been downward and he sandard deviaions afer 1994 end o be smaller han before These findings sugges ha individual paricipans forecass have indeed been more synchronized since 1994, boh in erms of heir overall view of he economy and of he ineres rae variable mos closely ied o policy. We now sudy he decomposiions of forecas errors for each of he five key macroeconomic variables. Chars 9-14 show he ime-series pahs of decomposiions for 12

14 individual variables as well as all he variables joinly. As eviden in Panel A of each char of Chars 9-14, one uniform resul is ha he ime pah of idiosyncraic errors shows a paern of seady decline as well as very seasonal paern for he curren-year forecass. Wihin he curren year, he individual paricipan s forecas error becomes much smaller as he ime ges close o December. The seasonal paern is much less obvious for he nex-year forecass (Panel B of each char of Chars 9-14), parly because he uncerainy abou he economy nex year is sill large even if one ries o forecas as of December las year. For boh he curren-year and nex-year forecass we see a clear paern of smaller idiosyncraic errors afer Again, hese resuls are consisen wih he hypohesis ha individual forecass have been more synchronized since Paerns of common errors are disincively differen from hose of idiosyncraic ones and he difference seems o be associaed wih business cycles unrelaed o he FOMC saemens. One can see from Chars 9-14 ha he common errors in he curren year forecas are large relaive o he idiosyncraic errors, whereas he common errors are dominan in he nex year forecass. Bu here is no apparen paern ha he common errors are smaller afer 1994 han before. According o Char 9A, he unusually large common errors for he curren-year forecass of he shor-erm ineres rae occur in These common errors are associaed wih he unexpeced sharp decline of he federal fund rae. The large common errors of longer-erm (nex-year) forecass seem o be associaed wih missing he urning poin of he federal funds rae in he early 2000s and failing o predic he unchanged rae in 2002 and 2003 (Char 9B). 13

15 For CPI inflaion, excep for a couple of unusually large common errors before 1994, he common errors of he curren-year forecass have he similar paerns before and afer 1994 (Char 10A). The common errors for he nex-year forecass end o be larger in he period afer 1996 han before (Char 10B), and here shows no endency ha hese errors have become smaller han before Typically as he ime ges closer o he end of he year, boh idiosyncraic and common errors become smaller for he curren year forecass. Bu for 1995, here are unusually large common errors of he curren-year forecass of real GNP/GDP owards he end of 1995, caused mainly by he definiion change of he GDP series. These errors are amplified when divided by he diminishing variances of forecas errors, which explains he seep drop of accuracy scores oward he end of 1995 in Char 3A. In Char 11A, he errors are no divided by he variances of forecas errors and hus are no as visually dramaic as in Char 3A. The subsanial, persisen common errors of he nexyear forecass in he lae 1990s are consisen wih he susained increase in produciviy growh largely unexpeced by he public, while he federal funds rae did no change much in he lae 1990s. The common errors in forecasing he unemploymen rae for he curren year appear o be somewha smaller afer 1994 han before, bu hose errors for he nex year have similar paerns before and afer 1994 (Chars 12A and 12B). The large common errors for he nex-year forecass have much o do wih business cycles and wih he errors in predicing oupu growh (Char 12B). There are no clear paerns in which he common forecas errors of he long-erm bond yield have become smaller since 1994 (Chars 13A and 13B). In paricular, he 14

16 errors around he recen recession are relaively large in magniude. I is ineresing, however, ha here was a noiceable drop in he idiosyncraic errors in boh he curren year and nex year forecas afer 1987 when Chairman Greenspan became chairman and he effecs of he sock marke problems dissipaed. Chars 14A and 14B summarize he decomposiion of he MSE for all he five variables combined. For he curren-year forecass, he seasonal paern is eviden, as explained early in his aricle. For he nex-year forecass, he large common errors occurred in he periods around he las wo recessions. The persisen and volaile common errors since 1994 are mainly due o he correlaion effec among forecas errors across variables, because he forecas errors for individual variables oher han GNP/GDP do no share hese feaures. Overall here is no evidence ha he public s forecass of key macroeconomic variables have improved since 1994, following he FOMC s effors o increase ransparency. Table 1 repors he average of percenages of he MSE ha are aribued o he idiosyncraic componen and he common componen. Two mehods are used o compue he average percen conribuions. The firs mehod is o calculae he percen conribuions of idiosyncraic and common errors for each period and hen average over all he periods. This mehod helps eliminae ouliers of exremely large errors, so he resuls may no conform o he impression by looking a he chars. The op panel of Table 1 repors hese resuls. The second mehod is o accumulae he forecas errors of boh ypes hroughou he enire sample and hen calculae he percen conribuions of idiosyncraic and common errors. The resuls generaed by his mehod are repored in he boom panel of 15

17 Table 1. This mehod is likely o be influenced by ouliers bu will be consisen wih he impression given by he chars. Le us look a he op panel of Table 1 firs. For he curren-year forecass, excep for GNP/GDP he idiosyncraic errors conribue much more o he oal errors han he common errors, despie he fac ha he common errors are much larger a imes. Bu when one examines all he variables joinly, he common errors become more imporan. This resul implies ha while i may be relaively easy o predic a single variable, he problem of predicing a se of economic variables may be more difficul. 8 As for he longer-erm (nex-year) forecass, he picure is compleely differen: he common errors are clearly a driving force for almos all variables (excep for CPI), individually and joinly. Compared o he op panel of Table 1, he resuls from he boom panel give more dominae role o he common errors, parly because he common errors are much larger han he idiosyncraic errors in some periods. All in all, i is clear ha he common errors play a dominan role in overall forecas errors. This finding suggess ha unanicipaed shocks, which of course are also no anicipaed in he FOMC saemens, are dominan facors in affecing forecas performance and improvemens in policy ransparency would be unlikely o make he forecas errors smaller excep on he margins. 9 Bu i is also possible ha more clear paerns may show up as more observaions become available in he fuure, as he FOMC only began o sae is views abou he economic oulook in May Given he daa we have oday, however, we find no empirical evidence in significan improvemen in he 8 One migh also infer ha here are differen models being used and hey perform beer on some variables han ohers, bu in aggregae here are significan differences among he forecass. 16

18 common forecas errors over he period in which he FOMC aemped o clarify is views of he economy or he likely course for fuure policy. This does no sugges ha he movemen owards ransparency has been a failure. I may simply sae ha here isn sufficien informaion conen in he saemens as presenly srucured, relaive o he kinds of disclosures made by oher cenral banks in heir inflaion repors. VI. Vinage Daa Versus Final Daa One could argue ha whenever forecas errors are evaluaed, final daa available a ha ime should be used. The reason is obvious. From a policy perspecive, being able o predic well iniially released daa ha subsequenly ge revised, may lead o policy errors, especially when close o urning poins or where he revisions may subsanially change one s view of he economy. However, when policy formulaion relies heavily upon model forecass, i is imporan ha hose forecass capure, as bes as possible he rue underlying pahs for key economic variables. If hey do no, hen he risk of series policy errors may be increased. Furhermore, how o choose he vinage daa a various poins in ime is compleely arbirary, and here is no saisical or economical foundaion for such an arbirary decision. The public knows ha daa such as GDP are ofen revised and someimes horoughly revised. They ake such unpredicable oucomes ino accoun and make heir forecass as accuraely as possible on average. In his secion, we use he revised and mos currenly available daa a he beginning of 2005 o re-compue he forecas errors. Char 15 displays he Blue Chip Consensus accuracy scores wih he vinage daa and he final daa for boh he currenyear and nex-year forecass. Ineresingly, he scores using final daa do no fall 9 This inerpreaion is consisen wih he resuls of Sock and Wason (2003) and Sims and Zha (2005). 17

19 considerably. The average curren-year score using vinage daa is 70.9 while he average curren-year score using final daa is 67.0, jus 3.9 poins lower. For he nexyear forecas scores here is very lile difference beween he scores using vinage and final daa. The average score using vinage daa is 57.4 while he average score using final daa is There are several periods, in 1992, , and in 1998 where he nex-year forecas score was lower using final daa, bu a he same ime here are several periods (1994, 1999 and 2002) where i is higher. These resuls indicae ha fuure daa revisions are random enough such ha hey do no inroduce a bias ha does no significanly negaively affec forecas scores on average. More imporan, i also suggess ha he daa revisions do no pose significan risks for policy makers. One would expec, perhaps, a greaer dispariy beween he wo scores given ha addiional revision errors are unpredicable. However, an imporan advanage of using he final daa is ha one can avoid he disored forecas errors of GDP caused by he daa revision in Comparing Char 6A wih Char 11A, one can see ha he disorion is compleely eliminaed when he final daa are used o measure he forecas accuracy. Sill, we find ha when he 1995 period is excluded he difference beween he currenyear scores using vinage and final daa increases from 3.9 o 7.7. Looking more closely a he source of his difference, we find ha i can be mosly aribued o he forecas error for GNP/GDP. Char 16 displays he decomposiions of forecas errors for GNP/GDP using he final daa as realized values. Comparing his char wih Char 11, one can see ha here are some noable differences in he breakdown in he composiion for boh he currenyear and nex-year forecass. In Char 11A, we see larger overall errors in 1992 and in 18

20 he 1996 o 2004 period. These larger errors are due o increases in error aribuable o he common componen of he forecas error. Consequenly, a greaer proporion of he error each period is due o he common componen. The average conribuion of he common componen o he overall error rises o 73.9 percen from 56.7 percen. In addiion, he overall error in 1995 using vinage daa (which resuled from he changing o chain-weighed GDP) is no longer presen. For he nex-year forecass in Char 11B, we again see ha he overall error has increased bu o a considerably more modes degree. The overall forecas error prior o he recession is less using final daa bu is greaer (on aggregae) for he 1996 o 2000 period. Bu once again, his increase in overall error is aribuable o he common componen of he forecas error. The average conribuion of he common componen of he overall error rises o 61.8 percen from 59.0 percen. Our findings sugges here may make lile difference o use final daa or vinage daa when evaluaing forecass. We have shown ha he average Blue Chip Consensus score was modesly affeced for curren-year forecass and was almos unchanged for nex-year forecass. In addiion, we have shown ha he decrease in score for currenyear and nex-year forecass was as a resul of an increase in he common componen of he forecas error and did no affec he idiosyncraic componen. Therefore, he effec of a swich o final daa for evaluaing individual forecass scores should be roughly equal across forecass. And lasly, he use of final daa eliminaes he need for arbirarily choosing among differen vinages. 19

21 VII. Conclusion In 1994 he FOMC began o release saemens afer each meeing. The amoun of informaion released in he saemens has increased and changed over ime. The findings from Kohn and Sack (2003) and Ehrmann and Frazscher (2004) sugges ha financial markes are sensiive o he informaion revealed in he saemens. While i is imporan o know wheher he saemens have affeced markes, bu i is also imporan o undersand wheher he saemens are providing useful signals concerning he Commiee s views abou he fuure pah for he economy and/or economic policy, or wheher he saemens are simply adding noise. Tha is, has he public s abiliy o forecas fuure economic and financial condiions been improved since 1994? This quesion is imporan because one would hope ha ransparency, if appropriaely communicaed, would enhance he marke paricipans abiliy o forecas (Woodford 2005). We analyze he forecas errors across a large secion of forecasers and for a se of five key macroeconomic variables. We find evidence ha he individuals forecass have been more synchronized since 1994, implying he possible effecs of he FOMC s ransparency. On he oher hand, we find lile evidence ha he common forecas errors, which are he driving force of overall forecas errors, have become smaller since In fac, hey have worsen and become more volaile on several dimensions. These common errors seem o be associaed wih business cycles and oher economic shocks. I is possible ha ransparen moneary policy may no necessarily enhance he public s predicabiliy of business cycles. I is also possible ha because Augus 2003 he Commiee has jus begun o provide more explici guidance on he likely pah of fuure 20

22 policy and is coningen naure on fuure economic condiions. We hope ha our findings will generae more fuure research on his imporan opic. Box I Characerizing he Disribuion of Accuracy Scores The disribuion of accuracy scores can be summarized by he firs four momens. We show how o calculae he mean or average score pˆ ( n ) in he ex. In his appendix we show how o calculae he oher hree momens: sandard deviaion, skewnesss, and kurosis. These hree informaive measures can be calculaed as follows: ˆ σ 1 N i ( ) ( ( ˆ n = p χ, n) pˆ( n) ) N, i= 1 sˆ ( n) = 1 N N i ( p( ˆ χ, ) ˆ ( ) ) 3 n p n i= 1 ˆ σ ( n) 3, uˆ ( n) = 1 N N i ( p ( ˆ χ, ) ˆ ( ) ) 4 n p n i= 1 ˆ σ ( n) 4, where σ sands for he sandard deviaion, s he skewness, and u he kurosis. Le he esimae of µ be Box II Decomposiion of Mean Square Error N 1 i ˆ µ = y. N i= 1 Noe ha ˆ µ is also he Blue-Chip consensus forecas. The weighed mean square error a ime can be decomposed as 21

23 N N 1 i i 1 i i x x = ( y ˆ µ ) ( y ˆ µ ) ( y ˆ µ ) ( y ˆ µ ) N i 1 N = i= 1, N N 1 i i 1 = ( y ˆ ) ( ˆ ) ( ˆ ) ( ˆ µ y µ y µ + y µ ) N N i= 1 i= 1 where he firs erm a he righ hand side is he MSE aribued o he idiosyncraic componen and he second erm is he MSE aribued o he common componen. The cross erm is zero because 1 N N i ( y µ ) ( y µ ) ( µ µ ) ( y µ ) ˆ ˆ = ˆ ˆ ˆ = 0. i= 1 Appendix I: Daa Descripion Three-monh Treasury bill rae: Secondary marke, monhly average. Source: Board of Governors of he Federal Reserve Sysem. Consumer price index: CPI-U (all urban consumers). Source: Bureau of Labor Saisics, U.S. Deparmen of Labor. Gross naional/domesic produc: , no chained; , chained. Source: Bureau of Economic Analysis, U.S. Deparmen of Commerce. Unemploymen rae: All 16 years or older workers. Source: Bureau of Labor Saisics, U.S. Deparmen of Labor. Corporae bond yield: Aaa, monhly average. Source: Moody s Invesors Service, Inc. Ten-year Treasury noe yield: Consan mauriy, monhly average. Source: Board of Governors of he Federal Reserve Sysem. Appendix II: Scores and Ranks for Individual Forecasers 22

24 In his appendix we repor in Table 2 he average scores for all he individual forecasers who have coninued o paricipae in he surveys in recen years. We include also he consensus forecas and he Bayesian vecor auoregressive (BVAR) model. The BVAR model is ofen used in he empirical lieraure as a benchmark for model comparison (Roberson and Tallman 1999, 2001), and reporing he real-ime forecasing performance of his model is of paricular ineres o academic researchers. We also repor oher forecasers scores oward he end of he able for compleeness. The years in which each forecaser paricipaed in he Survey are also repored in he able. 23

25 References Andy Bauer, Eisenbeis, Rober A. Daniel F. Waggoner, and Tao Zha, Forecas evaluaion wih cross-secional daa: The Blue Chip Surveys, Federal Reserve Bank of Alana Economic Review (Q2), Ehrmann, Michael, and Marcel Frazscher, Cenral Bank Communicaion: Differen Sraegies, Same Effeciveness? Unpublished Manuscrip (November), he European Cenral Bank. Eisenbeis, Rober A. Daniel F. Waggoner, and Tao Zha, Evaluaing Wall Sree Journal Survey Forecasers: A Mulivariae Approach, Business Economics 37(3), Faus, John and Eric M. Leeper, Forecass and Inflaion Repors: An Evaluaion, manuscrip prepared for he Sveriges Riskbank conference Inflaion Targeing: Implemenaion, Communicaion and Effeciveness, June 11-12, Kohn, Donald L., and Brian P. Sack, Cenral Bank Talk: Does I Maer and Why? Finance and Economics Discussion Series (November), Board of Governors of he Federal Reserve Sysem. Roberson, John C. and Ellis W. Tallman, Vecor Auoregressions: Forecasing and Realiy, Federal Reserve Bank of Alana Economic Review (Q1), Roberson, John C. and Ellis W. Tallman, Improving Federal-Funds Rae Forecass in VAR Models Used for Policy Analysis, Journal of Business and Economic Saisics 19(3), July, Sims, Chrisopher A. and Tao Zha, Were There Regime Swiches in US Moneary Policy? American Economic Review, forhcoming. Sock, James H. and Mark W. Wason, Has he Business Cycles Changed? Evidence and Explanaions, Moneary Policy and Uncerainy: Adaping o a Changing Economy, Federal Reserve Bank of Kansas Ciy Symposium, Jackson Hole, Wyoming, Augus Michael Woodford, Cenral-Bank Communicaion and Policy Effeciveness, manuscrip prepared for he Federal Reserve Bank of Kansas Ciy Conference on The Greenspan Era: Lessons for he Fuure, Jackson Hole, Wyoming, Augus 25-27,

26 Char 1A 100 Blue Chip Individual Scores: Average & Sandard Deviaion Curren Year Average Score S Deviaion Char 1B 14 Blue Chip Individual Scores: Skewness & Kurosis Curren Year Skewness Kurosis 25

27 Char 2A 100 Blue Chip Individual Scores: Average & Sandard Deviaion Nex Year Average Score S Deviaion Char 2B 14 Blue Chip Individual Scores: Skewness & Kurosis Nex Year Skewness Kurosis 26

28 Char 3A 100 Blue Chip Consensus Score & Average of Top/Boom 5 Forecaser Scores Curren Year Blue Chip Consensus Average of Top 5 Scores Average of Boom 5 Scores Char 3B 100 Blue Chip Consensus Score & Average of Top/Boom 5 Forecaser Scores Nex Year Blue Chip Consensus Average of Top 5 Scores Average of Boom 5 Scores 27

29 Char 4A 1.2 Cross-secional Sandard Deviaion of 3-Monh Treasury Bill Curren Year Nex Year Char 4B 1.0 Cross-secional Sandard Deviaion of 3-Monh Treasury Bill 12-monh moving average Curren Year Nex Year 28

30 Char 5A 1.0 Cross-secional Sandard Deviaion of CPI Curren Year Nex Year Char 5B 0.8 Cross-secional Sandard Deviaion of CPI 12-monh moving average Curren Year Nex Year 29

31 Char 6A 1.4 Cross-secional Sandard Deviaion of GDP Curren Year Nex Year Char 6B 1.2 Cross-secional Sandard Deviaion of GDP 12-monh moving average Curren Year Nex Year 30

32 Char 7A 0.7 Cross-secional Sandard Deviaion of Unemploymen Rae Curren Year Nex Year Char 7B 0.6 Cross-secional Sandard Deviaion of Unemploymen Rae 12-monh moving average Curren Year Nex Year 31

33 Char 8A 1.2 Cross-secional Sandard Deviaion of 10-Year Treasury Bill Curren Year Nex Year Char 8B 1.0 Cross-secional Sandard Deviaion of 10-Year Treasury Bill 12-monh moving average Curren Year Nex Year 32

34 Char 9A 4.0 3M-TBill -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Targe Char 9B 14 3M-TBill -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Targe 33

35 Char 10A 3.00 CPI -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Char 10B 2.50 CPI -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae 34

36 Char 11A 5.5 GDP -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Char 11B 11 GDP -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae 35

37 Char 12A 0.8 Unemploymen Rae -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Char 12B 2.00 Unemploymen Rae -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae 36

38 Char 13A Y-Tbond -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Char 13B Y-TBond -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae 37

39 Char 14A 8 All Variables -- Mean Square Error: Curren Year Idiosyncraic Common Error Overall 11-Sep Federal Funds Rae Char 14B 20 All Variables -- Mean Square Error: Nex Year Idiosyncraic Common Error Overall Federal Funds Rae 11-Sep 38

40 Table 1 Decomposiion of Mean Square Error By Average Percen Conribuion o Error in Each Period Curren Year Forecass ( ) All Variables 3MTB CPI GDP UR 10YTB Idiosyncraic Componen Common Componen Nex Year Forecass ( ) All Variables 3MTB CPI GDP UR 10YTB Idiosyncraic Componen Common Componen By Percen Conribuion of Toal Error Across Sample Curren Year Forecass ( ) All Variables 3MTB CPI GDP UR 10YTB Idiosyncraic Componen Common Componen Nex Year Forecass ( ) All Variables 3MTB CPI GDP UR 10YTB Idiosyncraic Componen Common Componen

41 Char 15A 100 Blue Chip Consensus Scores: Curren vs Real Time Acual Daa Curren Year Average Scores Real Time Acuals = 70.9 Jan2005 Daa for Acual = Real Time Acual Daa Jan2005 Daa for Acual Char 15B 100 Blue Chip Consensus Scores: Curren vs Real Time Acual Daa Nex Year Average Scores Real Time Acuals = Jan2005 Daa for Acual = Real Time Acual Daa Jan2005 Daa for Acual 40

42 Char 16A GDP -- Mean Square Error*: Curren Year * Using January 2005 Daa as Acual Daa Percen of Toal Error (Per Period Average) Idiosyncraic = 26.1 Common = Idiosyncraic Common Error Overall Char 16B GDP -- Mean Square Error*: Nex Year Percen of Toal Error (Per Period Average) Idiosyncraic = 38.2 Common = 61.8 * Using January 2005 Daa as Acual Daa Idiosyncraic Common Error Overall 41

43 Table 2 Overall Performance: Score Overall Curren Year Nex Year Paricipaion Average Sandard Average Sandard Average Sandard Forecaser Name Score Deviaion Score Deviaion Score Deviaion CY NY BC- Average of Top BC Consensus Macroeconomic Advisers, LLC Schwab Washingon Research Grou Alana BVAR U.S. Trus Co ClearView Economics Banc of America Corp Norhern Trus Company Wayne Hummer & Co Moody's Invesors Service Perna Associaes Merrill Lynch Wells Capial Managemen Naional Assn. of Home Builders Nomura Securiies Na. Ciy Bank of Cleveland DuPon Georgia Sae Fannie Mae DaimlerChrysler AG Sandard & Poors Egger Economic Enerprises Siff, Oakley, Marks, Inc Evans, Carrol and Associaes Bank One Bear Seams & Co., Inc BC- Ave of Individual Scores La Salle Naional Bank Prudenial Securiies Prudenial Financial Goldman Sachs & Co Naional Assn. of Realors Conference Board Chamber of Commerce, USA General Moors Corporaion Econoclas Eaon Corporaion Turning Poins (Micromerics) Comerica UCLA Business Forecas Moorola, Inc J P Morgan Chase Kellner Economic Advisers Geneski.com Wachovia Securiies Federal Express Corp DRl-WEFA Morgan Sanley & Co Inforum-U. of Md Deusche Banc Alex Brown Naroff Economic Advisors Ford Moor Company BC- Average of Boom

44 Table 3 Overall Performance: Rank Overall Curren Year Nex Year Paricipaion Average Sandard Average Sandard Average Sandard Forecaser Name Rank Deviaion Rank Deviaion Rank Deviaion CY NY BC- Average of Top BC Consensus Macroeconomic Advisers, LLC ClearView Economics Schwab Washingon Research Grou Banc of America Corp Moody's Invesors Service Norhern Trus Company Model U.S. Trus Co Perna Associaes Wayne Hummer & Co Fannie Mae Nomura Securiies Naional Assn. of Home Builders Merrill Lynch Sandard & Poors Wells Capial Managemen DaimlerChrysler AG DuPon BC- Ave of Individual Scores Na. Ciy Bank of Cleveland Siff, Oakley, Marks, Inc Georgia Sae Evans, Carrol and Associaes Egger Economic Enerprises Naional Assn. of Realors Bear Seams & Co., Inc Bank One Goldman Sachs & Co Prudenial Financial Prudenial Securiies La Salle Naional Bank Wachovia Securiies Eaon Corporaion Chamber of Commerce, USA Turning Poins (Micromerics) Comerica General Moors Corporaion Kellner Economic Advisers Conference Board Moorola, Inc Econoclas Federal Express Corp J P Morgan Chase Geneski.com UCLA Business Forecas Inforum-U. of Md DRl-WEFA Morgan Sanley & Co Deusche Banc Alex Brown Naroff Economic Advisors Ford Moor Company BC- Average of Boom