Transparency, Expecaions, and Forecass ANDREW BAUER, ROBERT A. EISENBEIS, DANIEL F. WAGGONER, AND TAO ZHA Bauer is a senior economic analys in he macropolicy secion, Eisenbeis is execuive vice presiden and direcor of research, Waggoner is a research economis and assisan policy adviser in he financial secion, and Zha is a research economis and policy adviser in he macropolicy secion, all in he Alana Fed s research deparmen. They hank Jinill Kim, Brian Madigan, John Roberson, and Ellis Tallman for criical commens and Cindy Soo and Eric Wang for research assisance. A similar version of his research is also published wih he same ile as Federal Reserve Bank of Alana Working Paper -3. Many macroeconomiss have argued ha a cenral bank should be ransparen abou is objecives, is views abou he economic oulook, and he reasoning behind is policy changes (see Faus and Leeper 5). In 99 he Federal Open Marke Commiee (FOMC) began o release saemens accompanying changes in he federal funds rae arge. Since hen, he degree of specificiy of he saemens and he guidance provided on he likely course of fuure policy have evolved significanly. In a recen paper, Woodford (5) discusses wo kinds of cenral-bank communicaions: curren policy decisions and he cenral bank s view of likely fuure policy. He ariculaes four caegories of informaion he cenral bank s view of curren economic condiions, curren operaing arges, sraegies guiding policy decision making, and he oulook for fuure policy ha a cenral bank migh seek o communicae o he public. Woodford argues ha hese open communicaions are beneficial, no only from he poin of view of reducing he uncerainy wih which raders and oher economic decision makers mus conend, bu also from ha of enhancing he accuracy wih which he FOMC is able o achieve he effecs on he economy ha i desires, by keeping he expecaions of marke paricipans more closely synchronized wih is own. This aricle invesigaes wheher he public s views abou he economy s curren pah and abou fuure policy have been affeced by changes in he Federal Reserve s communicaions policy as refleced in privae-secor forecass of fuure economic condiions and policy moves. In paricular, has privae agens abiliy o predic he direcion of he economy improved since 99, when he FOMC began o publicly sae is views of he economic oulook? If so, on which dimensions has he abiliy o forecas improved? The analysis focuses on boh he shor-erm and longer-erm economic forecass of key macroeconomic variables such as inflaion, gross domesic produc (GDP) growh, and unemploymen and of policy variables such as shorerm ineres raes. Privae agens curren-year and nex-year forecass are used as proxies for he public s shor-erm and longer-erm expecaions, and empirical E C O N O M I C R E V I E W Firs Quarer
evidence is presened regarding wheher such forecass have performed beer in predicing fuure economic and policy condiions since 99. The privae-agen forecass used in his aricle are hose of individual paricipans as well as he consensus (average) forecass conained in he monhly Blue Chip Economic Indicaors surveys from 9 o, which include boh he pre-fomcsaemen subperiod (9: 993:) and he pos-fomc-saemen subperiod (99: :). We employ he economeric mehodology of Eisenbeis, Waggoner, and Zha (), which permis us o evaluae he accuracy of forecass boh in cross secion and across ime and o examine he errors in forecasing key economic variables on boh a univariae and a mulivariae basis. The laer is imporan because agens are no simply forecasing one economic variable bu raher a se of variables ha presumably are inerrelaed and joinly capure imporan dimensions of economic performance. Good forecass on one dimension bu poor overall performance may provide some indicaion of he inernal consisency of he forecaser s approach. This cross-secional daa se enables us o decompose forecas accuracy ino wo componens: he common error ha affecs all individual paricipans and he idiosyncraic error ha reflecs discrepan views across individuals abou fuure economic and policy condiions. According o Woodford (5), one should expec he idiosyncraic error o become smaller as FOMC open communicaions become more ransparen. Bu he common error may no change much because i is likely o be affeced by facors oher han changes in policy ransparency, such as unforeseen business cycles. To preview he main resul, we find ha since 99 he idiosyncraic errors for key macroeconomic variables have seadily declined and he expecaions of marke paricipans are more closely synchronized o one anoher. We find no evidence, however, ha he common error has become smaller since 99, especially for he longer-erm forecass. The Mehodology Le µ be an n vecor of economic variables a ime, le y be he realized value of hese economic variables, and le y i be he ih individual s forecas value of he variables. Assume ha y is normally disribued wih mean µ and an economywise (common) covariance marix Ω R and ha yi is normally disribued wih mean µ and a forecaswise covariance marix Ω F. (The superscrips R and F sand for realized and forecas, respecively.) The covariance marix Ω R reflecs he aggregae shocks ha affec he realized value of µ ; he covariance marix Ω F capures he discrepancy in forecass across individual paricipans. The assumpion ha he mean forecas among individual paricipans is µ is reasonable because previous work has suggesed ha he Blue Chip Consensus forecas, serving as a proxy for he mean forecas, is close o being an unbiased esimae of µ (Bauer e al. 3). We denoe he forecas error for he ih forecaser by x i = yi y. Therefore, he individual forecas error x i has mean zero and a variance marix Ω = Ω R + Ω F, which indicaes ha x i is subjec o boh idiosyncraic and common shocks. The sandard saisical heory implies ha χ i i x Ω x i χ n ( ), where χ (n) denoes he χ disribuion wih n degrees of freedom and χ i is a square error weighed by Ω. The above expression shows ha he weighed square error E C O N O M I C R E V I E W Firs Quarer
χ i follows he χ disribuion wih n degrees of freedom. To measure he forecas accuracy for each individual paricipan, we compue a score value (p value) associaed wih his χ disribuion and call i an accuracy score. The score for individual forecaser i a forecas ime is a funcion of χ i and n: ( ) = cdf ( ) p χ i i n χ, χ, n, where χ cdf (χi,n) is he probabiliy ha a random observaion from he χ disribuion wih n degrees of freedom falls in he inerval [ χ i ].3 As Eisenbeis, Waggoner, and Zha () poin ou, he summary measure p(χ i,n) is a probabiliy ha is invarian o he underlying scales-of-error variances. One possible inerpreaion is ha he ih paricipan s forecas is closer o he realized value han ha of p(χ i,n) percen of all possible forecasers. Moreover, he score p(χi,n) can be compared across forecasers, wihin a forecas period, and across periods. Bauer e al. (3) show how o esimae he covariance marices Ω R and Ω F. The marix Ω R can be esimaed as he sample covariance marix of he Blue Chip Consensus forecas errors across ime under he assumpion ha Ω R is he same across years for each monh bu varies across monhs wihin a year. Thus, he variances on he diagonal of Ω R become smaller as approaches he end of he year because more informaion becomes available o forecas economic condiions for he curren year. The covariance marix Ω F can be esimaed as he sample covariance marix of forecas errors across individual forecasers; his covariance varies boh across monhs and across years. The esimae of Ω, denoed by ˆΩ, is he sum of he esimaes of Ω R and Ω F. Given his esimae, he weighed-square error can be calculaed as i i χˆ ˆ i = x Ω x. A each ime, he average accuracy score is N pˆ ( n) p ˆ i, n, χ N = ( ) i= where N is he number of individual forecasers a ime. One can also calculae he cross-secional disribuion of accuracy scores; he process is described in deail in he sidebar on page.. Kohn and Sack (3) characerize several disinc periods of increasing ransparency in FOMC saemens: saemens on changes in he discoun rae (99 93), saemens on changes in he federal funds rae (99 9), saemens including policy il (99 99), and saemens including assessmen of he balance of risks ( ). In May 3 a furher refinemen was added o separaely sae he commiee s views on he risks o inflaion and growh. And, finally, in Augus 3 he commiee provided explici guidance on he likelihood ha policy would remain accommodaive.. In fuure research, we inend o relax he assumpions ha he Blue Chip Consensus forecas is equal o µ and idiosyncraic shocks are independen of common shocks. 3. If he assumpions used are valid, he disribuion of accuracy scores from 9 o should be uniform. We have verified ha such a disribuion is more or less uniform, aking ino accoun small-sample uncerainy.. Oher esimaes can also be consruced using model-based mehods. E C O N O M I C R E V I E W Firs Quarer 3
Figure Blue Chip Average of Individual Scores for he Curren Year Average scores and sandard deviaions Skewness and kurosis Average score Kurosis Sandard deviaion Skewness 9 99 99 995 99 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Vinage Daa and Forecas Errors The monhly Blue Chip Economic Indicaors repor he forecass of key macroeconomic variables for he curren and nex years. We sudy he annual average forecass of five key variables: he hree-monh Treasury bill (T-bill) rae, he consumer price index (CPI) inflaion rae, real gross naional produc (GNP) for 9 o 995 or real gross domesic produc (GDP) from 99 o, he unemploymen rae, and he long-erm bond yield (he corporae bond yield from 9 o 995 or he enyear Treasury noe yield from 99 o ). The hree-monh T-bill rae, he CPI inflaion rae, he unemploymen rae, and he long-erm bond yield are monhly variables while real GNP/GDP is a quarerly variable. This frequency difference is imporan o noe when evaluaing forecass. (See Appendix for a descripion of and sources for hese daa.) More informaion becomes available abou he acual curren-year daa as he end of he year approaches, and herefore he forecas errors for boh he curren and nex years ge smaller. For example, he forecasers paricipaing in he December Blue Chip survey will have monhly daa on he hree-monh T-bill rae and he longerm bond yield hrough November, daa on he unemploymen rae hrough Ocober or November, and daa on he CPI inflaion rae hrough Ocober. However, since GNP/GDP daa are released quarerly, forecasers will have informaion regarding i GNP/GDP only hrough he hird quarer of he year. The weighed-square error ˆχ is designed o avoid he influence of differen amouns of available daa so ha he errors are comparable across ime. To gauge forecas errors, he realized values of each variable a a given ime mus be used. The values of some variables are revised over ime by he agencies responsible for reporing hose variables. In paricular, real GNP/GDP is repored quarerly and revised wice. Every year addiional benchmark revisions may be made in July o pas GDP daa. Hence, he informaion repored is acually he coninuously changing esimaes of many key economic variables final values. Finally, someimes he definiion of GDP is changed and he series is compleely revised. E C O N O M I C R E V I E W Firs Quarer
Figure Blue Chip Average of Individual Scores for he Nex Year Average scores and sandard deviaions Skewness and kurosis Average score Kurosis Sandard deviaion 9 99 99 995 99 Skewness 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Such revisions raise he quesion, Wha vinage daa should one use o evaluae forecas errors? From a macropolicy perspecive, one could argue ha he focus should be on he bes esimae of he final value of he variable of ineres. Ofen, however, ha value is no known for several years, and someimes he difference beween even a preliminary esimae and is neares neighbor esimaes can be very large. For example, he advanced esimae for real GDP for he firs quarer of 5 was 3. percen. This number was revised upward by he Bureau of Economic Analysis (BEA) o 3. percen and finally o 3. percen as more daa on he performance of he economy became available. Policymakers migh have inferred ha he economy was growing below rend according o he firs number bu above rend based on he final esimae. Such differences could have significanly differen implicaions for policy. For his reason, we would argue ha he focus should be on forecas mehods ha bes approximae he final number raher han he iniial esimae. Also, a priori knowledge of he expeced performance of a model or forecasing mehod can help policymakers decide how o weigh he evidence when significan differences exis beween he iniial releases of daa and forecass. For he purposes of his sudy, for he curren-year forecass, we use vinage daa available a he end of January following he curren year; for he nex-year forecass, we use daa available a he end of January following he nex year. This sudy uses vinage daa so ha is resuls will be comparable wih hose of previous sudies. I also provides a comparison beween he average Blue Chip Consensus score using vinage and final daa, using January 5 for he final daa. Accuracy Scores This secion looks a he disribuion of scores a each monh and examines wheher he disribuion has changed over ime, especially from he presaemen subperiod o he possaemen period. The echnical deails of how o characerize he crosssecional disribuion of scores are provided in he sidebar on page. The firs panel of Figure shows he ime-series pahs of average scores and sandard deviaions of scores for he curren year. The firs panel of Figure shows E C O N O M I C R E V I E W Firs Quarer 5
Characerizing he Disribuion of Accuracy Scores The disribuion of accuracy scores can be summarized by he firs four momens. The mehod for calculaing he mean or average score pˆ ( n) is shown in he ex. The oher hree momens sandard deviaion, skewnesss, and kurosis can be calculaed as follows: ( ) N i σˆ ( n) = χˆ, ˆ ( ) p( n p n ) N i=, N sˆ ( n) = N uˆ ( n) = i= i= ( p( χˆ i, n pˆ ( n) ) ), σˆ ( n) N ( p( χˆ i, n pˆ ( n) ) ) N 3 σˆ ( n), and where σ sands for he sandard deviaion, s he skewness, and u he kurosis. 3 similar pahs for he nex year. The measure of sandard deviaion is ofen used o approximae he volailiy of he public s expecaions or forecass a each poin in ime. As he firs panel of Figure shows, boh he average score and he sandard deviaion of scores flucuae over ime. No noiceable differences exis in he degree of flucuaion before and afer 99, nor are here differences for any subperiods afer 99. No rend appears in which he average score has increased or he sandard deviaion of scores has decreased since 99. The figures clearly display periods when forecasers made big errors, such as missing he onse of he recessions in 99 and. In addiion, while he average scores increased in, so did he sandard deviaions of he scores. Similarly, he average scores dropped significanly in 995 primarily because he definiion of he GDP series changed. In January 99 he BEA changed he measuremen of GDP o a chain-weighed sysem, bu he forecass made before January 99 migh be based on he non-chain-weighed series. Ineresingly, his change seems o have had relaively less effec on he longer-erm forecas errors (he second panel of Figure ). The average score for he nex year (Figure ) shows no improvemen since 99 and in fac appears o have drifed lower since 99. The sandard deviaion of scores since has drifed seadily upward. The paern of he drif in he sandard deviaion is similar o ha jus prior o and coming ou of he 99 9 recession. As discussed furher in he nex secion, hese lower scores afer 99 are mos likely associaed wih he naure of he business cycle and a surge of unexpeced produciviy growh in he lae 99s. The second panels of Figures and display he skewness and kurosis of accuracy scores. Skewness measures he asymmery of he score disribuion. The more negaive his measure is, he more scores spread ou oward percen. Conversely, he more posiive his measure is, he more scores spread ou oward percen. Kurosis measures he likelihood ha he score disribuion has exreme ouliers ha may affec he average score. The bigger he value of his measure is, he more likely he presence of ouliers in he score disribuion is. For he curren-year forecass, he skewness and kurosis have remained sable excep for a few periods. The 995 spike is he resul of he redefiniion of GDP, and he small spikes around are associaed wih he recen recession. For he nex-year forecass, again, no clear paern or rend is apparen in which skewness and kurosis have changed since 99. Two spikes in skewness and kurosis correspond o he Asian financial crisis and he recen recession. E C O N O M I C R E V I E W Firs Quarer
Figure 3 Blue Chip Consensus Scores and he Averages of he Five Top and Boom Forecaser Scores Curren year 9 9 99 99 99 99 99 Nex year 9 9 99 99 99 99 99 Blue Chip Consensus Average of five op scores Average of five boom scores Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Furher informaion abou he disribuional changes of accuracy scores is provided in Figure 3, which displays he ime-series pahs of accuracy scores of he Blue Chip Consensus forecas and he average of he op and boom five forecass for each monh. The consensus forecas is of paricular ineres because is score is on average he highes (see Appendix for deails) and because i performs beer han any single individual forecaser over he sample. Again, Figure 3 demonsraes ha hese scores have had no endency o improve over ime since 99. In fac, he E C O N O M I C R E V I E W Firs Quarer 7
Figure Cross-Secional Sandard Deviaions of Three-Monh Treasury Bill Forecass. Monhly daa. Twelve-monh moving average.. Nex year.. Nex year..... Curren year Curren year 9 99 99 995 99 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa scores of consensus forecass appear o be slighly lower afer 99 han before, especially for he nex-year forecas. Moreover, he drop in he consensus scores around he recen recession and again following Sepember,, suggess ha evens and exogenous shocks affeced forecas performance much more han FOMC saemens did. The drop in he scores oward he end of 995 is aribuable o he redefiniion of GDP. The average scores for he five op and he five poores forecasers sugges ha he daa have fa ails, wih mos of he forecass being clusered a he high end wih a few really poor performers on he boom. All hese findings sugges ha he individual paricipan s forecas performance relaive o oher paricipans has no improved beween he presaemen and possaemen periods. Alhough he accuracy score is a powerful summary measure of forecasing performance, i is a nonlinear funcion of he square forecas errors weighed by he overall covariance marix Ω. Separaing Ω and forecas errors for furher analysis would be informaive. In he nex secion, we examine wheher he covariance marix Ω F has changed over ime and sudy he sources of forecas errors ha do no depend on Ω F and Ω R.5 Transparency and Sources of Forecas Errors Kohn and Sack (3) and Woodford (5) argue ha he conens of FOMC saemens have become more ransparen since 99. To evaluae his argumen, i is imporan o deermine wheher he expecaions of marke paricipans as refleced in he forecass of key economic variables have become more synchronized in he possaemen period han in he presaemen subperiod. If he saemen conains useful informaion, hen one migh expec an overall improvemen in forecas accuracy, ceeris paribus, or a leas more agreemen among forecasers (ha is, a igher disribuion of idiosyncraic errors). A posiive answer may provide evidence abou he effecs of he FOMC saemens on he privae secor s agreemen on he direcion of he fuure economy. E C O N O M I C R E V I E W Firs Quarer
Figure 5 Cross-Secional Sandard Deviaions of CPI Forecass. Monhly daa. Twelve-monh moving average.. Nex year. Nex year.... Curren year Curren year 9 99 99 995 99 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa We also examine he sources of forecas errors by direcly decomposing he mean square error (MSE) ino he idiosyncraic componen ha reflecs he discrepancy in individual paricipans from he Blue Chip surveys and he common componen ha is associaed wih unanicipaed aggregae shocks and affecs all paricipans. The echnical deails of his decomposiion are provided in he sidebar on page 9. The MSE is he average of square errors across individual forecasers. Arguably, boh he idiosyncraic and common errors may show a decreasing rend if he saemen conains useful informaion and forecasers gain beer undersanding of he economy over ime, especially afer 99. To he exen ha he common error is affeced by exogenous aggregae shocks and he disribuion of he shocks is no consan, no clear inference may exis abou he size of he common error. However, we hypohesize ha he more imporan impac is likely o be seen for he idiosyncraic componen, in ha he idiosyncraic errors should be igher ha is, greaer agreemen should be eviden among he forecasers. The empirical resuls presened below confirm his hypohesis. The degree of synchronizaion among marke paricipans expecaions is measured by he cross-secional sandard deviaions of all he variables, which are equal o square roos of he diagonal elemens of Ω F. Figures repor he cross-secional sandard deviaion of each of he five macroeconomic variables considered in his sudy. These chars clearly show ha he rend for hese variables has been downward, and he sandard deviaions end o be smaller afer 99 han before 99. These findings sugges ha individual paricipans forecass have indeed been more synchronized since 99 in erms of boh heir overall view of he economy and he ineres rae variable mos closely ied o policy. 5. The reader may recall ha by assumpion Ω R does no change from one year o anoher. We inend o relax his assumpion in fuure research. E C O N O M I C R E V I E W Firs Quarer 9
Figure Cross-Secional Sandard Deviaions of GDP Forecass. Monhly daa. Twelve-monh moving average..... Nex year.. Nex year... Curren year 9 99 99 995 99. Curren year 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Figure 7 Cross-Secional Sandard Deviaions of Unemploymen Rae Forecass.7 Monhly daa. Twelve-monh moving average..5 Nex year.5...3.3 Nex year.. Curren year.. Curren year 9 99 99 995 99 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Figures 9 show he ime-series pahs of decomposiions for each of he five key variables as well as all he variables joinly. One uniform resul seen in he firs panel of each figure is ha he ime pah of idiosyncraic errors shows a paern of seady decline as well as a seasonal paern for he curren-year forecass. Wihin he curren year, he individual paricipan s forecas error becomes much smaller as December approaches. The seasonal paern is much less obvious for he nex-year forecass (he second panel of each figure) parly because he uncerainy abou he economy during he coming year is sill large even if one ries o forecas as of E C O N O M I C R E V I E W Firs Quarer
Figure Cross-Secional Sandard Deviaions of Ten-Year Treasury Noe Forecass. Monhly daa. Twelve-monh moving average. Nex year.... Nex year.... Curren year Curren year 9 99 99 995 99 9 99 99 995 99 Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa December in he curren year. For boh he curren-year and nex-year forecass, a clear paern of smaller idiosyncraic errors emerges afer 99. Again, hese resuls are consisen wih he hypohesis ha individual forecass have been more synchronized since 99. Paerns of common errors are disincively differen from hose of idiosyncraic ones, and he difference seems o be associaed wih business cycles unrelaed o he FOMC saemens. One can see from Figures 9 ha he common errors in he curren-year forecas are large relaive o he idiosyncraic errors whereas he common errors are dominan in he nex-year forecass. Bu here is no apparen paern ha he common errors are smaller afer 99 han before. According o he firs panel of Figure 9, unusually large common errors for he curren-year forecass of he shor-erm ineres rae occur in. These errors are associaed wih he unexpeced sharp decline of he federal funds rae. The large common errors of longer-erm (nex-year) forecass seem o be associaed wih missing he urning poin of he federal funds rae in he early s and failing o predic he unchanged rae in and 3 (he second panel of Figure 9). For CPI inflaion, excep for wo unusually large common errors before 99, he common errors of he curren-year forecass have similar paerns before and afer 99 (he firs panel of Figure ). The common errors for he nex-year forecass end o be larger in he period afer 99 han before (he second panel of Figure ), and no endency is apparen ha hese errors have become smaller han before 99. Typically, as he end of he year approaches, boh idiosyncraic and common errors become smaller for he curren-year forecass. Bu unusually large common errors of he curren-year forecass of real GNP/GDP develop oward he end of 995, caused mainly by he definiion change of he GDP series. When divided by he diminishing variances of forecas errors, hese errors are amplified, accouning for he seep drop of accuracy scores oward he end of 995 (see he firs panel of Figure 3). In he firs panel of Figure, he errors are no divided by he variances of forecas errors and hus are no as visually dramaic as in Figure 3. The subsanial, E C O N O M I C R E V I E W Firs Quarer
Figure 9 Mean Square Errors of Three-Monh Treasury Bill Forecass 5 Curren year 9// Errors (percenage poins) 3 Federal funds rae (percen) 9 9 99 99 99 99 99 5 Nex year 9// Errors (percenage poins) 9 Federal funds rae (percen) 3 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa persisen common errors of he nex-year forecass in he lae 99s are consisen wih he susained increase in produciviy growh being largely unexpeced by he public, while he federal funds rae did no change much. The common errors in forecasing he unemploymen rae for he curren year appear o be somewha smaller afer 99 han before, bu hose errors for he nex year have similar paerns before and afer 99 (Figure ). The large common errors for he nex-year forecass have much o do wih business cycles and wih he errors in predicing oupu growh. E C O N O M I C R E V I E W Firs Quarer
Figure Mean Square Errors of CPI Forecass Curren year 3. 9// Errors (percenage poins)... Federal funds rae (percen). 9 9 99 99 99 99 99.5 Nex year 9//. Errors (percenage poins).5. Federal funds rae (percen).5 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa No clear paerns exis in which he common forecas errors of he long-erm bond yield have become smaller since 99 (Figure 3). In paricular, he errors around he recen recession are relaively large in magniude. Ineresingly, a noiceable drop in he idiosyncraic errors in boh he curren-year and nex-year forecass occurs afer 97, when Alan Greenspan became chairman and he effecs of he sock-marke problems dissipaed. Figure summarizes he decomposiion of he MSE for he five variables combined. For he curren-year forecass, he seasonal paern is eviden, as explained E C O N O M I C R E V I E W Firs Quarer 3
Figure Mean Square Errors of GDP Forecass 5 Curren year 9// Errors (percenage poins) 3 Federal funds rae (percen) 9 9 99 99 99 99 99 Nex year 9// Errors (percenage poins) Federal funds rae (percen) 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa early in his aricle. For he nex-year forecass, he large common errors occurred in he periods around he las wo recessions. The persisen and volaile common errors since 99 are mainly caused by he correlaion effec among forecas errors across variables because he forecas errors for individual variables oher han GNP/GDP do no share hese feaures. Overall no evidence indicaes ha he public s forecass of key macroeconomic variables have improved since 99, following he FOMC s effors o increase ransparency. The able (on page ) repors he average of percenages of he MSE ha are aribued o he idiosyncraic componen and he common componen. Two meh- E C O N O M I C R E V I E W Firs Quarer
Figure Mean Square Errors of Unemploymen Rae Forecass. Curren year 9//. Errors (percenage poins).. Federal funds rae (percen). 9 9 99 99 99 99 99. Nex year 9//. Errors (percenage poins).. Federal funds rae (percen). 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa ods are used o compue he average percen conribuions. The firs is o calculae he percen conribuions of idiosyncraic and common errors for each period and hen average hem over all he periods. This mehod helps eliminae ouliers of exremely large errors, so he resuls may no conform o he paerns in he chars. The op panel of he able repors hese resuls. The second mehod is o accumulae he forecas errors of boh ypes hroughou he enire sample and hen calculae he percen conribuions of idiosyncraic and common errors (see he boom panel of he able). This mehod is likely o be influenced by ouliers bu will be consisen wih he paerns shown in he chars. E C O N O M I C R E V I E W Firs Quarer 5
Figure 3 Mean Square Errors of Ten-Year Treasury Noe Forecass. Curren year 9//. Errors (percenage poins).. Federal funds rae (percen). 9 9 99 99 99 99 99 5 Nex year 9// Errors (percenage poins) 3 Federal funds rae (percen) 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa In he op panel of he able, he idiosyncraic errors for he curren-year forecass, excep for GNP/GDP, conribue much more o he oal errors han he common errors do despie he fac ha he common errors are much larger a imes. Bu for all he variables joinly, he common errors become more imporan. This resul implies ha while predicing a single variable may be relaively easy, predicing a se of economic variables may be more difficul. For he longer-erm (nex-year) forecass, he picure is compleely differen: The common errors are clearly a driving force for almos all variables (excep for CPI), individually and joinly. E C O N O M I C R E V I E W Firs Quarer
Figure Mean Square Errors of All Variables Forecass Curren year 9// Errors (percenage poins) Federal funds rae (percen) 9 9 99 99 99 99 99 Nex year 9// Errors (percenage poins) Federal funds rae (percen) 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa Compared o he resuls in he op panel of he able, he resuls in he boom panel give a more dominan role o he common errors, parly because he common errors are much larger han he idiosyncraic errors in some periods. All in all, he common errors clearly play a dominan role in overall forecas errors.. One migh also infer ha differen models are being used and ha hese models perform beer on some variables han ohers, bu in aggregae significan differences exis among he forecass. E C O N O M I C R E V I E W Firs Quarer 7
Table Decomposiion of he Mean Square Error All 3-monh Unempl. -year variables T-bill CPI GDP rae T-noe By average percen conribuion o error in each period Curren-year forecass (9 ) Idiosyncraic componen.5 57. 9.7 3.3. 5.7 Common componen 55.5 3. 3.3 5.7 3..3 Nex-year forecass (9 3) Idiosyncraic componen 3.. 5.7. 3..5 Common componen 7.. 7.3 59. 3. 5.5 By percen conribuion of oal error across sample Curren-year forecass (9 ) Idiosyncraic componen 3.9 3.9.. 39. 3. Common componen. 9. 59. 7... Nex-year forecass (9 3) Idiosyncraic componen. 5. 3...7 3. Common componen 77.9.9. 79.9 75.3 7.9 This finding suggess ha unexpeced shocks, which of course are also no anicipaed in he FOMC saemens, are dominan facors in affecing forecas performance, and improvemens in policy ransparency would be unlikely o make he forecas errors smaller excep on he margins. 7 Anoher possibiliy is ha clearer paerns may show up as more observaions become available; he FOMC only began in Augus 3 o provide explici guidance on he likely pah of fuure policy and sae-coningen economic condiions in he fuure. Given he daa available oday, however, we find no empirical evidence of significan improvemen in he common forecas errors over he period in which he FOMC aemped o clarify is views of he economy or he likely course for fuure policy. This finding does no necessarily sugges ha he movemen oward ransparency has been a failure. I may simply indicae ha no new informaion was provided in he saemens ha had no already been inferred by marke paricipans. Given he unpredicable naure of business cycles, moreover, he common error may be mosly affeced by facors oher han moneary policy ransparency. Vinage Daa versus Final Daa One could argue ha whenever forecas errors for a paricular period are evaluaed, final daa available a ha ime should be used. The reason is obvious: From a policy perspecive, being able o accuraely predic iniially released daa ha are subsequenly revised may lead o policy errors, especially when urning poins are imminen or when he revisions may subsanially aler one s view of he economy. However, when policy formulaion relies heavily upon model forecass, i is imporan ha hose forecass capure, as well as possible, he rue underlying pahs for key economic variables. If hey do no, hen he risk of serious policy errors may be increased. Furhermore, deciding how o choose he vinage daa a various poins in ime is compleely arbirary, and no saisical or economical foundaion exiss o E C O N O M I C R E V I E W Firs Quarer
Decomposiion of he Mean Square Error Le he esimae of µ be N i µ ˆ = y. N i= Noe ha ˆµ is also he Blue Chip Consensus forecas. The weighed mean square error a ime can be decomposed as N i N y y i i x x = ( µ ˆ ) µ ˆ N N i= i= i ( y µ ˆ y ) ( µ ˆ ) ( ) ( ) ( ) N i i = y µ ˆ y µ ˆ N i= ( ) ( ) N + y µ ˆ y µ ˆ, N i= where he firs erm on he righ-hand side is he MSE aribued o he idiosyncraic componen and he second erm is he MSE aribued o he common componen. The cross erm is zero because N i y µ ˆ y µ ˆ µ ˆ µ ˆ y µ ˆ N i= ( ) ( ) = ( ) ( ) =. guide such decisions. The public know ha daa such as GDP are ofen revised and someimes horoughly revised. They ake such unpredicable oucomes ino accoun and make heir forecass as accuraely as possible on average. In his secion, we use he revised and mos curren daa available a he beginning of 5 o recompue he forecas errors. Figure 5 displays he Blue Chip Consensus accuracy scores wih he vinage daa and he final daa for boh he curren-year and nex-year forecass. The average curren year score using vinage daa is 7.9 while he average curren-year score using final daa is 7, jus 3.9 poins lower. For he nexyear forecas, he average scores using vinage daa and final daa are very similar: 57. using vinage daa and 5. using final daa. During several periods (99, 995 9, and 99) he nex-year forecas scores are lower using final daa, bu several periods (99, 999, and ) have higher scores. These resuls indicae ha fuure daa revisions are random enough ha hey do no inroduce a bias ha significanly affecs forecas scores on average. More imporan, he findings also sugges ha he daa revisions do no pose significan risks for policymakers. One would expec, perhaps, a greaer dispariy beween he wo scores given ha addiional revision errors are unpredicable. However, an imporan advanage of using he final daa is ha one can avoid he disored GDP forecas errors caused by he 995 daa revision. By comparing he firs panels of Figures and, one can see ha he disorion is compleely eliminaed when he final daa are used o measure he forecas accuracy. Sill, when he 995 period is excluded, he difference beween he curren-year scores using vinage and final daa increases from 3.9 o 7.7. Looking more closely a he source of his difference, we find ha i can be aribued mosly o he GNP/GDP forecas error. Figure displays he decomposiions of forecas errors for GNP/GDP using he final daa as realized values. A comparison of his figure wih Figure reveals some noable differences in he breakdown in he composiion for boh he curren-year and nex-year forecass. In he firs panel of Figure, we see larger overall errors in 99 and in he 99 period ha are due o increases in he common componen 7. This inerpreaion is consisen wih he resuls of Sock and Wason (3) and Sims and Zha (). E C O N O M I C R E V I E W Firs Quarer 9
Figure 5 Blue Chip Consensus Scores: Curren versus Real-Time Acual Daa Curren year Jan. 5 daa for acual Real-ime acual daa 9 9 99 99 99 99 99 Nex year Real-ime acual daa Jan. 5 daa for acual 9 9 99 99 99 99 99 Noe: The shaded verical bars indicae recessions. In he firs panel, he average score using real-ime acual daa is 7.9; he average score using January 5 daa for acual daa is 7.. In he second panel, he average score using real-ime acual daa is 57.; he average score using January 5 daa for acual daa is 5.3. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa of he forecas error. Consequenly, a greaer proporion of he error each period is due o he common componen. The average conribuion of he common componen o he overall error rises o 73.9 percen from 5.7 percen. In addiion, he overall error in 995 using vinage daa (which resuled from he changing o chain-weighed GDP) is no longer presen. For he nex-year forecass in he second panel of Figure, we again see ha he overall error has increased bu o a considerably more modes degree. The overall forecas error prior o he 99 9 recession is less using final daa bu is greaer (on aggregae) for he 99 period. Bu once again, his increase E C O N O M I C R E V I E W Firs Quarer
Figure Mean Square Error (Using January 5 Daa as Acual Daa) of GDP Forecass Curren year 5 Errors (percenage poins) 3 Federal funds rae (percen) 9 9 99 99 99 99 99 Nex year Errors (percenage poins) Federal funds rae (percen) 9 9 99 99 99 99 99 Idiosyncraic error Common error Overall MSE Federal funds rae Noe: The shaded verical bars indicae recessions. In he firs panel, he idiosyncraic percen of oal error (per period average) is.; he common percen of oal error (per period average) is 73.9. In he second panel, he idiosyncraic percen of oal error (per period average) is 3.; he common percen of oal error (per period average) is.. Source: Auhors calculaions from monhly Blue Chip Economic Indicaors daa in overall error is aribuable o he common componen. The average conribuion of he common componen rises o. percen from 59 percen. Our findings sugges ha using final daa or vinage daa may make lile difference when evaluaing forecass. The resuls show ha he average Blue Chip Consensus score is modesly affeced for curren-year forecass and almos unchanged for nexyear forecass. In addiion, he decrease in score for curren-year and nex-year forecass resuls from an increase in he common componen of he forecas error and does no affec he idiosyncraic componen. Therefore, he effec of a swich o final daa E C O N O M I C R E V I E W Firs Quarer
for evaluaing individual forecass scores should be roughly equal across forecass. The use of final daa eliminaes he need for arbirarily choosing among differen vinages. Conclusion In 99 he FOMC began o release saemens afer each meeing. The amoun of policy informaion released in he saemens has increased and changed over ime. The findings from Kohn and Sack (3) and Ehrmann and Frazscher () sugges ha financial markes are sensiive o he informaion revealed in hese saemens. While knowing wheher he saemens have affeced markes is imporan, undersanding wheher he saemens are providing srong signals concerning he FOMC s views abou he fuure pah of he economy or economic policy is also imporan. Tha is, has he public s abiliy o forecas fuure economic and financial condiions improved since 99? This quesion is imporan because one hopes ha ransparency, if appropriaely communicaed, enhances marke paricipans abiliy o forecas (Woodford 5). This aricle analyzes he forecas errors across a large secion of forecasers and for a se of five key macroeconomic variables. The analysis finds evidence ha he individuals forecass have been more synchronized since 99, implying he possible effecs of he FOMC s ransparency. On he oher hand, we find lile evidence ha he common forecas errors, which are he driving force of overall forecas errors, have become smaller since 99. In fac, common forecas errors have increased and have become more volaile on several dimensions. These common errors seem o be associaed wih business cycles and oher economic shocks. Transparen moneary policy may no necessarily enhance he public s abiliy o predic business cycles. On he oher hand, i is possible ha we do no have a long-enough sample o observe he effecs of ransparency because he FOMC jus began in Augus 3 o provide more explici guidance on he likely pah of fuure policy and is coningency on fuure economic condiions. We hope ha our findings will generae more research on his imporan opic. E C O N O M I C R E V I E W Firs Quarer
Appendix Daa Descripion Three-monh Treasury bill rae: 9. Secondary marke, monhly average. Source: Board of Governors of he Federal Reserve Sysem. Consumer price index: 9. CPI-U (all urban consumers). Source: U.S. Deparmen of Labor, Bureau of Labor Saisics. Gross naional/domesic produc: 9 95, no chained; 99, chained. Source: U.S. Deparmen of Commerce, Bureau of Economic Analysis. Unemploymen rae: 9. All workers sixeen years or older. Source: U.S. Deparmen of Labor, Bureau of Labor Saisics. Corporae bond yield: 9 95. Aaa, monhly average. Source: Moody s Invesors Service Inc. Ten-year Treasury noe yield: 99. Consan mauriy, monhly average. Source: Board of Governors of he Federal Reserve Sysem. Appendix Scores and Ranks for Individual Forecasers In his appendix, he following able shows he average scores for all he individual forecasers who have coninued o paricipae in he surveys in recen years. The able also includes he consensus forecas and he Bayesian vecor auoregressive (BVAR) model. The BVAR model is ofen used in he empirical lieraure as a benchmark for model comparison (Roberson and Tallman 999, ), and reporing he real-ime forecasing performance of his model is of paricular ineres o academic researchers. For compleeness, we also repor oher forecasers scores oward he end of he able. The years in which each forecaser paricipaed in he Blue Chip surveys are also repored in he able. Table Overall Performance: Score Overall Curren year Nex year Paricipaion Avg. Sd. Avg. Sd. Avg. Sd. Curren Nex Forecaser Name score dev. score dev. score dev. year year BC average of op...5 5.99 77.. BC consensus.3 3.9 7.9.7 57.3.77 Macroeconomic Advisers, LLC.5 7.7 7.57.5 53.. 7 5 Schwab Washingon Research Group.. 9.97 7. 53. 7.7 97 Alana BVAR 59.9 3.9 9. 9.5 9. 9.75 U.S. Trus Company 59.5 7.5..5 9.9.5 7 3 ClearView Economics 59.3.9.9 7.7 5. 7.99 5 Banc of America Corporaion 59. 7. 3. 7. 5.7 5. 9 Norhern Trus Company 5.75. 3.3 7.7 53.7 7.95 3 Wayne Hummer & Company 55.9 7.7 5.5 7. 53.5.7 Moody s Invesors Service 55..3 5.77.3.35.3 7 Perna Associaes 5..3.9.35 7.. 7 55 E C O N O M I C R E V I E W Firs Quarer 3
Appendix (coninued) Overall Curren year Nex year Paricipaion Avg. Sd. Avg. Sd. Avg. Sd. Curren Nex Forecaser Name score dev. score dev. score dev. year year Merrill Lynch 5.5 7. 5.3.97 5.3 5. 9 Wells Capial Managemen 53.5.7 59.3.9.3 7. 9 Naional Associaion of Home Builders 53.5. 5.77.9 7.93. 7 3 Nomura Securiies 5.55.7 55.77 9..57 7. 3 5 Naional Ciy Bank of Cleveland 5.. 5.75.5.93. 9 DuPon 5. 5. 57....3 Georgia Sae Universiy 5.7 7.39 5.7. 5..7 3 Fannie Mae 5.3. 59.7 9.3. 3.35 7 DaimlerChrysler AG 5.3 9.3 5.9 9. 3.35.5 5 Sandard & Poors 5.5 3.3 5. 3.9.7.3 Egger Economic Enerprises 5.79 5.9 5. 7.5 5..9 5 5 Siff, Oakley, Marks Inc. 5..9 5.5 7..77 7.7 97 97 Evans, Carrol and Associaes 5.3 9.77 5. 3.35. 7. Bank One 9. 3.39 5.7 3.75. 9. 5 9 Bear Searns & Company Inc. 9.7 9.9 53. 3.39 3.95.59 9 59 BC average of individual scores.3. 5... 5. La Salle Naional Bank 7.7 9.73 5.3 3...97 5 5 Prudenial Securiies 7.7 3. 7. 33..57. 75 7 Prudenial Financial 7.. 5.5.97 3.3 3.55 9 Goldman Sachs & Company. 7.9 59.7 5.9 3.9 9.5 79 Naional Associaion of Realors. 9. 5. 9...5 53 Conference Board 5. 9.3 5. 3.3 37. 5. Chamber of Commerce, USA.97 7..35.3..5 9 General Moors Corporaion.3.3.5 9... 5 Econoclas 3.9 7.3.3 3.9.3. 7 5 Eaon Corporaion 3..5.9 3.7 5.37. 7 5 Turning Poins (Micromerics) 3. 7..5 9. 5..9 5 7 Comerica. 5. 3. 9.3.. 7 UCLA Business Forecas. 3.9 5.3 3.3 3.75 7.55 7 5 Moorola Inc...7 5.3 3.73 3.9.9 9 JPMorgan Chase.9 7. 7.57 9. 33..55 9 Kellner Economic Advisers.79 3...5 39.55. 9 79 Geneski.com. 3.5 5. 3. 9.53.37 5 3 Wachovia Securiies.39 7.9. 3.5 35.9.33 9 Federal Express Corporaion 39.9.5..7 37.3. 5 53 DRl-WEFA 39. 7.9.3. 7.99.5 77 5 Morgan Sanley & Company 35.95 9.39 3.7 3.9 3.3.75 5 5 Inforum Universiy of Maryland 35.7. 33.5 7.5 3. 5.7 Deusche Banc Alex Brown 3.7. 3..7 9. 3.33 9 Naroff Economic Advisors 9.9.7 33.3 3.5 5..59 7 5 Ford Moor Company 5. 5. 7.3.9 3.9 3.73 3 7 BC average of boom 7.5.35..3.9.5 E C O N O M I C R E V I E W Firs Quarer
R E F E R E N C E S Bauer, Andy, Rober A. Eisenbeis, Daniel F. Waggoner, and Tao Zha. 3. Forecas evaluaion wih crosssecional daa: The Blue Chip Surveys. Federal Reserve Bank of Alana Economic Review, no. :7 3. Ehrmann, Michael, and Marcel Frazscher.. Cenral bank communicaion: Differen sraegies, same effeciveness? European Cenral Bank, unpublished paper. Eisenbeis, Rober A., Daniel F. Waggoner, and Tao Zha.. Evaluaing Wall Sree Journal survey forecasers: A mulivariae approach. Business Economics 37, no. 3:. Faus, Jon, and Eric M. Leeper. 5. Forecass and inflaion repors: An evaluaion. Paper presened a he Sveriges Riksbank conference Inflaion Targeing: Implemenaion, Communicaion and Effeciveness, Sockholm, June. Kohn, Donald L., and Brian P. Sack. 3. Cenral bank alk: Does i maer and why? Board of Governors of he Federal Reserve Sysem Finance and Economics Discussion Series No. 3-55, November. Roberson, John C., and Ellis W. Tallman. 999. Vecor auoregressions: Forecasing and realiy. Federal Reserve Bank of Alana Economic Review, no. :... Improving federal-funds rae forecass in VAR models used for policy analysis. Journal of Business and Economic Saisics 9, no. 3:3 3. Sims, Chrisopher A., and Tao Zha.. Were here regime swiches in U.S. moneary policy? American Economic Review 9, no. :5. Sock, James H., and Mark W. Wason. 3. Has he business cycle changed? Evidence and explanaions. In Moneary policy and uncerainy: Adaping o a changing economy. Federal Reserve Bank of Kansas Ciy. Woodford, Michael. 5. Cenral-bank communicaion and policy effeciveness. In The Greenspan era: Lessons for he fuure. Federal Reserve Bank of Kansas Ciy. E C O N O M I C R E V I E W Firs Quarer 5