The Reliability of Output Gap Estimates in Canada

Transcription

1 The Reliabiliy of Oupu Gap Esimaes in Canada Jean-Philippe Cayen 1 and Simon van Norden 2 1 Bank of Canada jcayen@bankofcanada.ca 2 HEC (Monréal) and CIRANO simon.van-norden@hec.ca In his paper, we measure, wih Canadian daa, he scope of he revisions o real-ime esimaes of he oupu gap generaed wih several univariae and mulivariae echnics. We also make an empirical evaluaion of he usefulness of he oupu gap esimaes for predicing inflaion. Our findings sugges ha, for all echnics, he sandard deviaion of he revisions is of he same order of magniude as he oupu gap iself. We also find ha, wih excepion of he Beveridge-Nelson echnic, all revisions are very persisen, which means ha here is a long lag before he scope of he oupu gap revisions is fully known. Finally, we found ha he oupu gap esimaes do no significanly improve he inflaion forecass. We infer from hese resuls ha esimaes of he oupu gap are no very reliable. * The views expressed in his paper are hose of he auhor. No responsibiliy for hem should be aribued o he Bank of Canada.

2 Conens 1. Inroducion Oupu Gap and Esimaion Errors Definiion Esimaion echniques Deerminisic Trends The Beveridge-Nelson decomposiion Hodrick-Presco filer Srucural VAR wih long-run resricions Unobserved componen models Facors explaining real-ime esimaions errors Sudies ha measured errors in oupu gap esimaes Evaluaion Mehods Daa Daa revisions Oher variables Firs evaluaion mehod: revisions of oupu gap esimaes Final esimaes Real-ime esimaes Quasi-real esimaes Quasi-final esimaes Reliabiliy indicaors used o evaluae he revisions Second evaluaion mehod: inflaion forecass Procedure used o forecas inflaion Crieria for selecing he number of lags Evaluaion of forecass Resuls Oupu Gap esimaes Firs evaluaion mehod: revisions of oupu gap esimaes Reliabiliy indicaors Decomposiion of revisions Second evaluaion mehod: inflaion forecass Conclusion Appendix References ii

3 1. Inroducion The oupu gap, defined as he discrepancy beween acual oupu and poenial oupu, is an imporan variable of models used by moneary auhoriies o projec and analyze he main macroeconomic variables. For counries like Canada, commied o mainaining a low and sable rae of inflaion, he oupu gap is considered o be a useful ool for gauging he exen of inflaionary pressures presen in goods and services markes in he economy. Unforunaely, he oupu gap is no a direcly observable measure. Many echniques have been developed o esimae i, bu hese esimaions are subjec o considerable uncerainy, paricularly a he end of he sample. Moreover, differen echniques ofen produce differen esimaes of he oupu gap. Many auhors evaluae how moneary policy decisions can be affeced by he uncerainy ha surrounds oupu gap esimaes. 1 Research on his opic usually involves simulaing of a macroeconomic model in which he auhors evaluae he effec ha he uncerainy around he oupu gap esimaes has on moneary policy rules such as he Taylor (1993) rule or inflaionforecas-based rules. 2 To conduc hese simulaions, he auhors mus beforehand make some imporan assumpions on he oupu gap. They mus firs decide how o model he oupu gap and specify how i is linked wih he oher variables of he model. They mus also formulae assumpions on he naure and he magniude of he errors ha affec he oupu gap esimaes. Bu he validiy of hese assumpions is no always esed. 3 Indeed, many auhors recognize ha heir resuls depend on he assumpions hey made on he specificaion of he model and on he ype and he level of uncerainy hey inroduce in heir model. For he moneary auhoriies, i is imporan ha he conclusions ha emanae from hese sudies are reliable. Therefore, he assumpions inroduced in hese simulaions mus be esed. In paricular, i is imporan o know wheher here are some echniques ha produce beer real-ime esimaes of he oupu gap han ohers; ha is, echniques ha produce reliable end-of-sample esimaes. I is also imporan o undersand he naure and he magniude of he errors ha affec 1 See, for example, Orphanides (1998, 2), Orphanides e al. (2), Smes (1998), Ehrmann and Smes (21), Rudebusch (1999), Swanson (2), Svensson and Woodford (2), Gaiduch and Hun (2), Yeman (2), and McCallum (21). 2 See Haldane and Baini (1998) and Amano e al. (1999) for an overview of inflaion-forecas-based rules. 3 Gaiduch and Hun (2) is one of he rare sudies ha ess hese assumpions. 1

4 he oupu gap esimaes. Finally, policymakers need o know wheher here is a sable predicive relaionship beween he real-ime esimaes of he oupu gap and inflaion. The goal of his sudy is o answer hese quesions. We measure, wih Canadian daa, he reliabiliy of differen univariae and mulivariae echniques of esimaing he oupu gap. In paricular, we invesigae he revisions o real-ime esimaes of he oupu gap over ime. We also es wheher he addiion of oupu gap esimaes in a simple equaion helps improve inflaion forecass. Orphanides and van Norden (1999, 21a, b) applied he same mehodology on U.S. daa and found ha, in general, he reliabiliy of oupu gap esimaes is quie low. For all he echniques hey esed, he magniude of he revisions is similar o he size of he gap esimaes hemselves. They also found ha oupu gap esimaes do no improve ou-of-sample inflaion forecass. Our resuls for Canada are similar o hose of Orphanides and van Norden. We also find ha he reliabiliy of he real-ime esimaes of he oupu gap ends o be quie low. In fac, for all he echniques ha we esed, he ex pos revisions are of he same order of magniude as he ex pos esimaes of he oupu gap, he real-ime esimaes frequenly misclassify he sign of he gap, and he esimaion errors appear o conain a highly persisen componen ha is subsanial in size. Also, he oupu gap esimaes generaed from he differen echniques, excep for he Beveridge- Nelson decomposiion, do no improve ou-of-sample inflaion forecass. This paper is organized as follow. Secion 2 briefly describes he differen echniques of esimaing he oupu gap, as well as he differen ypes of errors ha can affec hese esimaes. Secion 3 describes he mehodology used o evaluae he reliabiliy of he differen esimaion echniques. Secion 4 presens he oupu gap esimaes generaed by he differen esimaion echniques and he reliabiliy of hese esimaes based on wo evaluaion mehods. Secion 5 offers some conclusions. 2. Oupu Gap and Esimaion Errors This secion describes he differen approaches of esimaing he oupu gap, as well as he errors ha can affec he esimaes. In secion 2.1, we define an imporan concep for he analysis of oupu gap: poenial oupu. In secion 2.2, we describe he differen esimaion echniques sudied in his paper. In secion 2.3, we discuss he differen sources of errors ha may affec he oupu gap esimaes. Secion 2.4 presens sudies ha evaluae errors ha surround oupu gap esimaes. 2

5 2.1 Definiion The oupu gap represens he difference beween he observed level of oupu and poenial oupu. Thus, he key concep ha mus be defined is poenial oupu. axon and Telow (1992) noe ha he definiion of poenial oupu has changed over ime. In he 196s and he early 197s, poenial oupu was considered as he maximum level of oupu ha he economy can generae. Therefore, he analysis of business cycles consised of idenifying he cyclical peaks and explaining he facors ha caused he economy o approach or move away from he cyclical peaks. Under such a definiion, he oupu gap is always negaive. A he end of he 196s and early 197s, poenial oupu sared o be defined as he maximum level of oupu ha he economy can susain wihou creaing inflaionary pressures. This definiion brings ou he relaionship beween excess demand and inflaion. As he observed level of oupu increases relaive o is poenial level, excess demand increases, which encourages economic agens o increase he prices of he goods and services ha hey supply. This definiion of poenial oupu is sill he one generally used oday. I is also he one ha we use for his paper. 2.2 Esimaion echniques Claus, Conway, and Sco (2) idenify hree approaches for esimaing he oupu gap. The firs approach consiss of surveying businesses on heir producion capaciy. The survey makes i possible o consruc measures of capaciy uilizaion, giving an overview of poenial oupu when paired off wih producion daa. However, his approach is plagued by some imporan problems. Firs, i is no clear ha all he surveyed businesses have he same inerpreaion of he survey's quesions. In paricular, i is ypically hard o incorporae he noion of work inensiy in quesions ha refer o producion capaciy. Second, surveys ofen cover only a small porion of he economy, since hey usually arge he manufacuring secor. Therefore, we do no evaluae his approach in his paper. A second approach consiss of measuring a producion funcion, which reflecs he relaionships beween he observed level of oupu and he amoun of resources used in he producion process. This relaionship can be used o calculae he level of oupu ha would be produced if resources were fully employed and used a normal inensiy; ha is, a a level of inensiy ha would no creae inflaionary pressures. This approach also has imporan problems. Firs, i is very difficul o deermine a normal level of inensiy. Second, he exac form of he producion funcion is no clear. Finally, an imporan componen of he producion funcion is echnical progress. Jus like 3

6 poenial oupu, echnical progress is no observable and is herefore hard o measure. For hese reasons, we do no evaluae his approach in his paper. The hird approach, which is used in his paper, incorporaes all he imes-series echniques of decomposing oupu ino wo elemens: poenial oupu and he oupu gap. The number of exising echniques is really large, and so we limi ourselves o a number of hem. Because we wan o evaluae boh univariae and mulivariae echniques, we choose nine echniques ha can be regrouped ino he following five caegories: (i) deerminisic rends (ii) Beveridge-Nelson decomposiion (iii) Hodrick-Presco filer (iv) srucural VAR wih long-run resricions (v) unobserved componen models Wihin each caegory, we can reain many specificaions. We decide o keep specificaions ha have been used in oher sudies. Thus, for he deerminisic rend models, we examine he linear rend and he quadraic rend, while for unobserved componen models, we sudy he Wason (1986), Clark (1987), Harvey-Jaeger (1993) and Kichian (1999) models. The Beveridge-Nelson model could also be considered as an unobserved componen model, bu since i is esimaed differenly han he oher unobserved componen models, we separae i from hem. For he srucural VAR model, we use he specificaion proposed by alonde, Page, and S-Aman (1998). Oher esimaion echniques, such as he mulivariae filer used a he Bank of Canada (see Buler 1996), will probably be evaluaed in fuure work. The res of secion 2.2 describes he differen echniques ha we have chosen Deerminisic Trends The firs wo univariae derending mehods we consider are he linear and he quadraic rend. Boh mehods assume ha we can decompose oupu ino wo componens: a deerminisic rend and a cycle componen, which corresponds o he oupu gap. The general form of deerminisic rends is: I i y = + i =1 β i + α c, (1) 4 axon and Telow (1992), Buler (1996), S-Aman and van Norden (1998), and Cerra and Saxena (2) give a more complee picure of he exising echniques and of heir advanages and disadvanages. 4

7 where y is our chosen measure of oupu (in logarihms), α is a consan, is a ime rend and c is he oupu gap. When I is equal o 1, equaion (1) corresponds o he linear rend. When i is equal o 2, i represens he quadraic rend. To obain an esimae of he oupu gap, one only has o do an ordinary leas-squares esimaion of equaion (1). The esimaed residuals correspond o he oupu gap series The Beveridge-Nelson decomposiion Beveridge and Nelson (1981) consider he case of an ARIMA(p,1,q) series y, which is o be decomposed ino a rend and a cyclical componen. For simpliciy, we can assume ha all deerminisic componens belong o he rend componen and have already been removed from he series. Since he firs-difference of he series is saionary, i has an infinie-order moving average represenaion of he form y = ε + β + = e, (2) 1ε 1 + β 2ε 2 where ε is assumed o be an innovaions sequence. The change in he series over he nex s periods is simply: y s + s y = y+ j = j= 1 s j= 1 e + j. (3) The rend is defined o be: s ( ) lim E = + y+ s y lim E e+ j. (4) s s j= 1 From equaion (2), we can see ha E ( ) ( ) e+ j = E + j + β1ε + j 1 + β 2ε + j 2 + = i= ε β ε. (5) Since changes in he rend are herefore unforecasable, his has he effec of decomposing he series ino a random walk and a cyclical componen, so ha j+ i i y = τ + c, (6) where he rend is τ + e, (7) = τ 1 5

8 and e is whie noise. To use he Beveridge-Nelson decomposiion, we mus herefore: (i) idenify p and q in our ARIMA(p,1,q) model (ii) esimae he parameers in equaion (2) (iii) choose some large-enough bu finie value of s o approximae he limi in equaion (4) 5 (iv) for all and for j = 1, 2,, s, calculae E (e +j ), from equaion (5) (v) calculae he rend a ime as in he righ-end side of equaion (4), and he cycle as he difference beween observed oupu and he rend. For he specificaion of he ARIMA(p,1,q) model of oupu, we have no found any papers ha sudy, for Canada, he opimal values for p and q. We herefore follow Box and Jenkins' (1976) recommendaion of using a parsimonious model. We apply he Schwarz (1978) crierion 6 on he full sample and find ha Canadian oupu is following an ARIMA(1,1,) process Hodrick-Presco filer Anoher mechanical derending echnique is he HP filer, developed by Hodrick and Presco (1997). The HP filer decomposes a ime series (y ) ino an addiive cyclical componen (c ) and a growh componen (τ ), such as y = c - τ, and hen chooses he series τ o minimize he variance of he cyclical componen subjec o a penaly for he variaion in he second difference of he growh τ. Formally, he HP-filered rend is given by: 2 { } T + 1 T 2 { } = arg ( y τ ) + λ[ ( τ τ ) ( τ τ )] min = = τ. (8) and c is he resuling measure of he oupu gap. λ is called he smoohness parameer and penalizes he variabiliy in he growh componen. The larger he value of λ, he smooher he growh componen and he greaer he variabiliy of he oupu gap. As λ approaches infiniy, he growh componen corresponds o a linear ime rend. For quarerly daa, Hodrick and Presco propose seing λ equal o Beveridge and Nelson (1981) use s = 1. We also use ha value even if he forecass converge before s equal o 1. 6 This crierion is described in secion

9 2.2.4 Srucural VAR wih long-run resricions Anoher way of esimaing he oupu gap is o impose long-run resricions on he resuls obained for a mulivariae auoregressive model (VAR), o recover he srucural represenaion from which we can deduc a measure of he oupu gap. Blanchard and Quah (1989) were he firs o recover he srucural represenaion of a model from a reduced-form VAR, based on a priori assumpions on he long-run relaionship beween supply and demand. They made he simple assumpion ha demand shocks have only emporary effecs on real oupu, while supply shocks may have permanen effecs. Dupasquier, Guay, and S-Aman (1997), alonde (1998), and alonde, Page, and S-Aman (1998) use he Blanchard and Quah assumpions o recover esimaes of he oupu gap. 7 Since alonde, Page, and S-Aman (1998) (hereafer PS) apply his mehodology o Canadian daa, we use heir specificaion o esimae he oupu gap. 8 As wih PS, we esimae he reduced form of an eigh-lags VAR, which incorporaes he following variables: he firs differences of (he logarihms of) real oupu ( y ), of he inflaion rae ( π ), and of he real ineres rae( r ): ( ) φ12 ( ) φ13( ) y 1 ε y, ( ) ( ) ( ) ( ) ( ) ( ) φ22 φ23 π 1 + ε, φ φ r ε y µ φ11 + π = µ φ21 π. (9) r µ φ r, The hree series of residuals (ε y,, ε π,, and ε r, ) esimaed wih his reduced-form model include a he same ime, boh permanen and ransiory shocks ha affec real oupu. To isolae he srucural shocks, PS impose resricions similar o he one proposed by Blanchard and Quah (1989): ha a firs group of shocks has permanen effecs on all variables (η p ); a second group has ransiory effecs on oupu, bu permanen effecs on inflaion (η cp ); and a hird group has ransiory effecs on boh inflaion and oupu (η c ). 9 Since hese resricions are made only on he long-run dynamic of he model, he shor-run movemens are no consrained. Once he srucural shocks are idenified, i is possible o decompose oupu ino he hree shocks, as follows: 7 These auhors replace he erms supply and demand shocks by permanen and ransioy shocks. 8 Their sudy presens a more deail discussion of he choosen specificaion. Differen resuls could be obained wih a differen specificaion, such as he one proposed by alonde (1998) for he U.S. for example. 9 Oher echnical resricions mus be imposed o idenify he differen srucural shocks. These are described in he appendix. 7

10 y y p y p p* p cp cp c ( ) η + Γ ( ) η + Γ ( ) η + ( ) η c = µ + Γ 1 Γ, (1) y y y where µ y is he deerminisic componen on he rend, and Γ i y is he moving average represenaion of he differen shocks. In heir sudy, PS explain ha he erm Γ p y (1) is he long-run muliplicaor of permanen shocks, while Γ p* y () = Γ p y () - Γ p y (1) represens he ransiory componens of he permanen shocks. Thus, poenial oupu, which is represened by µ y +Γ p y (1) η p +Γ p* y () η p, can flucuae over ime. PS also explain ha wo measures of he oupu gap can be exraced from his specificaion. The firs measure, RTP, correspond o he erm Γ cp y ()η cp +Γ c y ()η c, he sum of he ransiory componens ha affec real oupu. The second measure, RTP1, corresponds o he erm Γ cp y ()η cp. Thus, RTP1 exclude from he oupu gap ransiory movemens ha are no due o changes in he rend of inflaion. PS find ha he oupu gap measured by he RTP1 mehod is a good (in-sample) predicor of inflaion Unobserved componen models The las ype of models considered are he unobserved componen models, which are also known as dynamic facor models and sae-space models. In his caegory, we evaluae hree univariae models, he Wason (1986), he Clark (1987), and he Harvey-Jaeger (1993) models, and one mulivariae model, he Kichian (1999) model. Unobserved componen models can all be represened by he same general framework, which includes he measuremen equaion (11) and he ransiion equaion (12): y = Zα + βx + ε (11) α = T α 1 + δw + U. (12) where α is an M-dimensional vecor of unobserved sae variables ; y is an N-dimensional vecor of observed variables; X and W are K- and S-dimensional vecors of observable exogenous variables; Z and T are marices of coefficiens; and ε and U are G- and M-dimensional vecors of i.i.d. gaussian errors, which are muually independen wih variance-covariance marices H and Q respecively. Wha differeniae he differen models described in his secion are he resricions made o he measuremen and ransiion equaions, as well as he dynamics chosen o describe he movemens of he unobserved variables. 8

11 The Wason (1986) model assumes ha oupu can be decomposed ino wo elemens: a rend componen (τ ) and he oupu gap (c ). In his model, he measuremen equaion (equaion 13) is herefore an ideniy equaion. For he ransiion equaions, he Wason model assumes ha he rend componen follows a random walk wih drif (equaion 14), while he oupu gap follows an AR(2) process (equaion 15) o allow some persisence in he business cycle. The model can be wrien as follows: y = τ + c (13) τ = µ + τ 1 + η (14) = 1c 1 + φ2c 2 c φ + e. (15) The Clark (1987) model is similar o he Wason model, excep ha he drif erm m is allowed o flucuae over ime (equaion 17). This variable also follows a random walk (equaion 18). This represens a major difference, since he Wason model assumes ha poenial oupu increases on average a a consan rae, which is no he case for he Clark model. The Clark model can be formalized as follows: y = τ + c (16) τ = µ + τ 1 + η (17) µ = µ 1 + υ (18) = 1c 1 + φ2c 2 c φ + e. (19) The Harvey and Jaeger (1993) model is differen han he Wason and Clark models in ha i includes a residual erm in he oupu equaion. The measuremen equaion (equaion 2) is herefore no an ideniy equaion anymore. Anoher major difference in he Harvey and Jaeger model is ha he AR(2) cycle is replaced by a sinusoidal sochasic process (equaions 23 and 24). The rend componen is, however, he same as in he Clark model. The model can be wrien as follows: y = τ + ψ + ε (2) τ = µ + τ 1 + η (21) µ = µ 1 + υ (22) ψ = ρ cos( λψ 1 ) + ρ sin( λψ * 1 ) + χ (23) * = ρ sin( λψ 1) + ρ sin( λψ * 1 ) χ *. (24) ψ + where ρ and λ are respecively he facors ha deermine he ampliude and he frequency of he cycle (ψ ). 9

12 The las unobserved componen model ha we evaluae is he Kichian (1999) model. This model is a modified version of he Gerlach and Smes (1997), adaped for he Canadian economy. This model has he same specificaion han he Clark model, excep ha i adds a Phillips curve ha links he oupu gap wih inflaion (equaion 29). This model can be wrien as follow: 1 y = τ + c (25) τ = µ + τ 1 + η (26) µ = µ 1 + υ (27) = 1c 1 + φ2c 2 e β + β1c 1 + γ ( ) ω + c φ + (28) π π = µ + c δ ( ) ε. (29) π where π is he firs difference of he inflaion rae (calculae wih CPI excluding food and energy), µ p is a consan, ε p are i.i.d. gaussian errors, which are uncorrelaed wih he oher errors erms, and ω is a se of exogenous variables. 11 Kichian also inroduces a moving average process MA(3) in he Phillips curve. Since hese models conain variables ha are no observed, he esimaion of he parameers is no rivial. In fac, he esimaion of such model is done in wo seps. The firs one consiss in consrucing he likelihood funcion of each parameer of he model, ha is, all he coefficiens and he variance of he residuals; and o esimae hese parameers wih he maximum likelihood mehod. In he second sep, we esimae he unobserved variables wih he Kalman filer Facors explaining real-ime esimaions errors One of he reasons ha explain he presence of many differen echniques of esimaing he oupu gap is ha he uncerainy around he esimaes is quie large. I is herefore hard o prove ha one echnique gives beer esimaes han he ohers. Gaiduch and Hun (2) disinguish hree sources of errors ha can affec oupu gap esimaes and which are responsible for he large uncerainy around he oupu gap esimaes. 1 In her sudy, Kichian (1999) ess differen varians of he Gerlach and Smes model. She noes ha he one ha has he bes fi is assuming a sochasic rend wih a consan drif, bu wih a break in he drif erm in However, since i is hard o deec such breaks in real ime, we decide o use anoher varian proposed by Kichian in which he rend follows a similar process as in he Clark model. 11 For he exogenous variables, Kichian (1999) includes he log changes in he real exchange rae and in he real oil prices (WTI), as well as changes in indirec axes. 12 The Kalman filer is a echnique ha recursively esimae unobserved variables (α i, ) by opimising heir mean a i, = E (α i, ) and heir variance-covariance marix P i, = E ((a i, - α i, ) (a i, - α i, )'). Harvey (1989) gives more deail on his esimaion mehod. I should be noed ha for he Kichian (1999) model, we use he GAUSS programs ha she developed a he Bank of Canada (see Kichian 2). 1

13 The firs source of errors concerns he saisical uncerainy around he esimaion of he parameers of he model used o esimae he oupu gap. This form of uncerainy is generally illusraed by he confidence inervals around he esimaes of he oupu gap. arge confidence inervals mean ha his ype of error is poenially big. However, some esimaion echniques, such as he Hodrick-Presco filer, do no necessiae he esimaion of any parameer. I is herefore no possible o measure his ype of errors for such echniques. For his reason, and because i is generally well documened in he lieraure, we do no consider his form of error in our evaluaion of he reliabiliy of oupu gap esimaes. The second source includes errors due o he uilizaion of an inadequae model. For example, he omission of imporan variables or he uilizaion of incorrec assumpions in a model can cause significan errors. The main difficuly wih his ype of errors is ha we do no perfecly know he srucure of he model ha explains poenial oupu. I is herefore impossible o direcly measure his ype of errors. The hird source of errors idenified by Gaiduch and Hun concerns he revisions made o he realime esimaes of he oupu gap over ime. As new daa become available, analyss change heir oupu gap esimaes for he recen periods (and someimes for longer periods). These ex pos revisions are considered as errors made in he pas. This form of error is closely linked wih he wo oher ypes of errors described before, since hese revisions can be caused by he uilizaion of an inadequae model or can simply reflec he normal saisical uncerainy around parameers' esimaes. Daa revisions can also be he cause of he revisions o he real-ime esimaes of he oupu gap. 13 Only he hird ype of errors, ha is, revisions o he real-ime esimaes of he oupu gap over ime, is direcly evaluaed in his paper. However, we will also show ha errors caused by he uilizaion of an inadequae model can poenially be large. 2.4 Sudies ha measured errors in oupu gap esimaes Even if we do no evaluae his ype of errors in his paper, i is ineresing o know ha he saisical uncerainy around oupu gap esimaes can be quie large. axon and Telow (1992), Kuner (1994), and S-Aman and van Norden (1997), among ohers, show ha he confidence inervals around he oupu gap esimaes are quie big for mulivariae echniques of esimaion. 13 Secion describes he differen revisions made o he daa in Canada. 11

14 For he errors caused by he choice of an inadequae model, we have no find any sudy able o precisely evaluae hem. Besides, i would almos be impossible o conduc such evaluaion, since nobody knows he exac srucure of he economy. Gaiduch and Hun (2), however, find ha his ype of errors can poenially be large. They observe ha he differences in oupu gap esimaes across echniques are relaively large. If we assume ha one of hese models is he righ one, i auomaically means ha he oher echniques are relaively bad a reproducing he real oupu gap. We find six sudies ha evaluae errors link o revisions o he oupu gap esimaes. Using Canadian daa, S-Aman and van Norden (1997) show ha he differences beween he end-ofsample esimaes and he middle-of-sample oupu gap esimaes generaed by he Hodrick- Presco filer are raher large. Using U.S. daa, Kuner (1994) and Orphanides and van Norden (1999, 21b) find similar resuls for a broader se of echniques. In fac, Orphanides and van Norden (1999, 21b) noe ha he revisions are persisen and of he same order of magniude han he esimaes hemselves. Gaiduch and Hun (2) find similar resuls for New Zealand. Finally, Orphanides (2) shows ha he official esimaes of he oupu gap used by he Federal Reserve Board from he mid 196s unil he mid 199s have also been subjec o subsanial revisions. 3. Evaluaion Mehods We use wo differen mehods o evaluae he reliabiliy of real-ime oupu gap esimaes. The firs one invesigaes he revisions of real-ime esimaes of he oupu gap over ime. The second one ess wheher he addiion of real-ime oupu gap esimaes in a simple equaion help improve inflaion forecass. To conduc hese ess, we need real-ime daa on Canadian oupu. Secion 3.1 describes he daa se ha we use. Secions 3.2 and 3.3 describe, respecively, he wo evaluaion mehods proposed in his paper. 3.1 Daa Real-ime oupu daa are necessary o generae real-ime esimaes of he oupu gap. Since Saisics Canada does no have such a daabase, we build our own one. To do his, we use he daa published each quarer in he Naional Income and Expendiure Accouns of Saisics Canada. We also consul special caalogues when major mehodological changes are made o he series. 12

15 Cerain poins should be noed. Firs, our daabase covers each vinage beween 1972Q2 and 21Q1. 14 Saisics Canada sared o publish quarerly oupu daa in Unforunaely, he series was compleely revised a he end of he 196s and we have no found any documen showing he new series. The firs vinage for which we have reliable daa on he enire hisorical period is 1972Q2. Second, before 1997, he firs observaion for each vinage was 1947Q1. However, Saisics Canada made a major revision of he naional accouns in 1997 and hereafer sopped publishing he daa prior o Therefore, for he 1997Q3 vinage and he following ones, he firs observaion of he series is 1961Q1. For his sudy, we use only daa afer 1961Q1 for hree reasons. Firs, quarerly oupu daa prior o 1961 are no reliable. Second, o evaluae he impac of adding new observaions in he esimaions, i is preferable ha all vinages sar a he same dae. Third, alonde, Page, and S-Aman (1998) and Kichian (1999) esimae heir models wih a daa se ha sars afer To obain resuls ha coincide wih heirs, we use he same sample period hey use. Third, we have o ignore a cerain number of available vinages. Mulivariae echniques necessiae a large number of observaions, because of he number of coefficiens ha have o be esimaed. For some of he oldes vinages, he number of observaions is no large enough o make he esimaion converge. 16 To be able o compare he differen echniques, we exclude he oldes vinage. Therefore, he firs vinage for which we show resuls is 1982Q Daa revisions The oupu daa ha we use are seasonally adjused and expressed in consan dollars. These wo characerisics necessarily imply ha he oupu series will be revised periodically. In fac, Saisics Canada revises seasonal facors each year. This form of revision usually affecs he pas four years of daa. Revisions ha resul from he change of he year of reference happen less ofen. In he pas 3 years, he reference year has changed only six imes. 17 In he laes change of he 14 The release of oupu daa is always done wih a one-quarer lag. For example, daa for he fourh quarer of 2 were released in he firs quarer of 21. To avoid confusion in he ex, we will always refer o he vinage of publicaion. Thus, he las observaion of he 1972Q2 vinage is he firs quarer of The observaions prior o 1961 were inerpolaed from annual daa. Given he poor reliabiliy of hese daa, Saisics Canada decided no o publish hem anymore. 16 This is he case for he esimaion of he alonde, Page, and S-Aman (1998) mehod using daa of he 1982Q2 vinage. 17 Before he 1975Q2 vinage, real oupu was expressed in 1961dollars. Beween 1975Q3 and 1986Q1, i was expressed in 1971dollars; beween 1986Q2 and 199Q1, in 1981 dollars; beween 199Q2 and 1997Q3, in 1986 dollars; beween 1997Q4 and 21Q1, in 1992 dollars; and since 21Q2, he reference year has been

16 reference year, Saisics Canada also modified is mehodology o calculae oupu a a consan rae. Since 21Q2, Saisics Canada has used he Fisher chained index, whereas before i was using a aspeyres index. 18 The concep of real oupu has also evolved over ime. In Canada, unil 1986Q1, he benchmark series was he gross naional produc (GNP), and since hen i has been he gross domesic produc (GDP). This modificaion is anoher form of revision of he daa. Oher major changes have been made o he oupu series over ime, he mos recen case being in 1997, when Saisics Canada modified he definiion of cerain componens of oupu o comply wih inernaional sandards. This modificaion caused a major revision of he enire oupu series. Each quarer, Saisics Canada revises he daa for he curren year as i receives new informaion from is various sources Oher variables In addiion o he oupu series, oher variables are required o esimae he mulivariae mehods. The srucural VAR model of alonde, Page, and S-Aman (1998) necessiaes he uilizaion of he oal consumer price index and he modified overnigh series proposed by Armour e al. (1996). Kichian's (1999) model also incorporaes he oal consumer price index; he consumer price index excluding food and energy; he price of Wes Texas Inermediae; he U.S. consumer price index; he bilaeral exchange rae beween Canada and he U.S.; and indirec axes. Some of hese series can be revised, as can he differen price indexes, which are seasonally adjused. However, we do no have any real-ime daa for hese series. We herefore use he mos recen vinage available for each of hese series. This means ha we will underesimae he impac of daa revisions for he mulivariae echniques. The magniude of revisions made o price indexes, however, is ypically small relaive o he revisions made o he oupu series. 3.2 Firs evaluaion mehod: revisions of oupu gap esimaes The firs evaluaion mehod consiss of measuring he degree o which esimaes of he oupu gap a any poin in ime vary as daa are revised and as daa abou subsequen evoluion of oupu becomes available. Presuming ha revisions improve our esimaes, he oal amoun of revision gives us a lower bound on he measuremen errors hough o be associaed wih he real-ime 18 Because our las vinage is 21Q1, our resuls do no capure his recen change. Normally, we would expec ha his modificaion would reduce he magniude of revisions o he daa, since changes in he reference year should no longer affec he growh rae of real variables. 14

17 oupu gap. 19 This is informaive when and if we find ha revision errors are relaively large, since we can conclude ha he oal error of hese esimaors mus be larger sill. We apply each derending mehod in a differen number of ways, o esimae and decompose he exen of he revisions in he esimaed gap series. To undersand how he exen of he revisions is measured, we define several concepually differen ways in which any exising derending mehod may be applied. In he remainder of his secion, we describe how hese mehods are applied and heir corresponding inerpreaions. We also presen reliabiliy indicaors ha are used o evaluae he revisions Final esimaes The firs of hese mehods gives rise o a final esimae of he oupu gap. This simply akes he las available vinage of daa we have available (in our case, he series as published in 21Q1) and derends i. The resuling series of deviaions from rend consiues he final esimae of he oupu gap. This is he ypical way in which such derending mehods are employed Real-ime esimaes The real-ime esimae of he oupu gap is consruced in wo sages. Firs, we derend each and every vinage of daa available o consruc an ensemble of oupu gap series. 2 Of course, earlier vinage oupu gap series are shorer han laer vinages, since he oupu series on which hey are based end earlier. Nex, we use hese differen vinages o consruc a new series ha consiss enirely of he firs available esimae of he oupu gap for each poin in ime. This new series is he real-ime esimae of he oupu gap. I represens he mos imely esimae of he oupu gap ha policy-makers could have consruced a any poin in ime. The difference beween he real-ime and he final esimae gives us he oal revision in he esimaed oupu gap a each poin in ime. We use he saisical properies of hese revisions as our guide o reliabiliy and accuracy of esimaed oupu gaps, recalling, of course, ha his is an overesimae of he rue reliabiliy of he real-ime esimaes, since i ignores he esimaion error in he final series. 19 This lower bound comes from he fac ha oher sources of errors can affec he esimaes of he oupu gap. In paricular, i is reasonable o assume ha some uncerainy remains wih long-pas hisorical esimaes of he oupu gap. 2 For he mulivariae echniques, only oupu daa are available in real-ime. For he oher variables, we use he mos recen vinage available. 15

18 3.2.3 Quasi-real esimaes The differences beween he real-ime and he final esimaes have several sources, one of which is he ongoing revision of published daa. To isolae he imporance of his facor, we define a hird oupu gap measure, he quasi-real esimae. ike he real-ime esimae, he quasi-real esimae is consruced in wo seps. The firs sep is o consruc an ensemble of rolling esimaes of he oupu gap. Tha is, we begin by aking he final vinage of he oupu series, bu use only he observaions up o and including 1982Q2 o compue he quasi-real esimae for 1982Q2. Nex, we exend he sample period by one observaion and repea he derending. We coninue in his way unil we use he full sample period for he final oupu series and have a full se of corresponding oupu gap series. The second sep is he same as ha used o consruc he real-ime series: We collec he firs available esimae of he oupu gap a each poin in ime from he various series we consruced in sep one. This sequence of oupu gaps is he quasi-real series. The difference beween he realime and he quasi-real series is enirely due o he effecs of daa revision, since esimaes in he wo series a any paricular poin in ime are based on daa samples ha cover exacly he same ime period Quasi-final esimaes For unobserved componen models, we are able o furher decompose he revision in he esimaed gap by defining anoher esimae of he oupu gap. This quasi-final esimae uses more informaion han he quasi-real esimae (which uses subsamples of final daa) bu less han he final esimae (which uses he full sample of final daa.) This is relevan because unobserved componen models use he daa in wo disinc phases. Firs, hey use he available daa sample o esimae he parameers of a ime-series model of oupu. Nex, hey use hese esimaed parameers in he Kalman filer o arrive a esimaes of he oupu gap. However, hey disinguish beween filered and smoohed esimaes of he oupu gap. The smoohed esimae uses he full sample parameer esimaes and daa from 1 o T o form an opimal esimae of he gap in quarer T. The filered esimae uses only daa from 1 o wih he full sample parameer esimaes o make an opimal esimae of he oupu gap a (1 T). For his class of models, smoohed esimaes of he oupu gap are used o consruc he final series, while filered esimaes are used for he quasi-final series. The difference beween he quasi-final and he quasi-real series hen reflec solely he effecs of using differen parameer esimaes for he model o filer he daa (i.e. he full-sample ones versus 16

19 he parial sample ones). The exen of he difference will reflec he imporance of parameer insabiliy in he underlying unobserved componen model. The difference beween he quasi-final and he final series reflecs he imporance of ex pos informaion in esimaing he oupu gap given he parameer values of he process generaing oupu Reliabiliy indicaors used o evaluae he revisions The mean of he absolue value of oal revisions and he roo-mean-square of he oal revision series are wo good measures of he magniude of oupu gap revisions over ime. Bu because he various echniques may have subsanial variaion in he size of he cyclical componen hey produce, i is easier o compare heir reliabiliy in real-ime by looking a comparably scaled measures of he revisions. A good scaled measure is he noise-o-signal raio. There are wo differen ways o measure his indicaor. The firs one (NS1) divides he sandard deviaion of he oal revision series over he sandard deviaion of he final oupu gap series. The second one (NS2) divides he roo-meansquare of he oal revision series over he sandard deviaion of he final oupu gap series. High values for hese raios mean ha he size of he revisions is large relaive o he magniude of he oupu gap esimaes. Thus, a reliable esimaion echnique should generae low values for boh measures of he noise-o-signal raio. Anoher indicaor of he reliabiliy of esimaion echniques is he correlaion beween he realime and he final esimaes of he oupu gap (COR). This indicaor shows wheher he real-ime business cycle has he same shor-erm flucuaions as he final one. The correlaion coefficien would be 1 in he ideal case where no revisions o he real-ime esimaes are ever required. A las reliabiliy indicaor is he frequency wih which he real-ime and final gaps are of opposie signs (OPSIGN). Reliable esimaion echniques would have an OPSIGN saisic close o. 3.3 Second evaluaion mehod: inflaion forecass The second evaluaion mehod is used o deermine wheher oupu gap esimaes generaed by he differen echniques help improve ou-of-sample forecass of inflaion. To do his, we incorporae he gap esimaes in a simple equaion and compare he ou-of-sample forecass produced by his equaion wih hose produced by a similar equaion ha excludes he oupu gap. Our approach also allows us o deermine wheher cerain echniques generae oupu gap esimaes ha do a beer job a forecasing inflaion han he oher esimaion echniques. 17

20 To perform such an evaluaion, we mus make cerain choices and assumpions. Firs, o forecas inflaion, we use a quasi-reduced form equaion ha includes lags of inflaion and of oupu gap as independen variables. Of course, he omission of cerain key variables, such as inflaion expecaions, will probably reduce he qualiy of he forecass. I is herefore possible ha oupu gaps migh improve on simple univariae forecass of inflaion, bu no on forecass using a broader range of inpus. For his reason, we feel ha he experimen ha we perform may acually oversae he uiliy of empirical oupu gap models. Second, we perform he es for a forecasing horizon of four quarers. This es is jusified by he fac ha moneary policy acions require ime o ake effec. We could, however, repea in fuure work he same procedure for differen forecasing horizons, wih a more complee se of variables in he forecasing equaion Procedure used o forecas inflaion Based on he assumpions explained in he previous secion, we consruc wo forecasing equaions. One includes he oupu gap and he oher does no: ~ ~ π ~ ~ c ε (3) i + 4 = α + β = 1 + = 1 + l lπ l λ l l l + 4 π ˆ + +, (31) 4 = αˆ + β ˆ l= 1 lπ l ε + 4 where π is he level of inflaion a quarer (year-over-year inflaion measured wih oal CPI) and c i is he oupu gap esimae generaed by echnique i a quarer. 21 Thus, he dependen variable ha we wan o forecas is he level of inflaion in four quarers. Because of reporing lags, informaion a quarer is no available before quarer +1. This is why conemporaneous values of inflaion and oupu gap are no used as righ-end side variables. In our ou-of-sample forecasing exercise, we use quarerly daa from 1961Q1 o 1982Q2 22 o esimae he parameers of each equaion, which are hen used o produce a forecas of inflaion for 1983Q3. The sample is hen exended o 1982Q3, he equaions are re-esimaed and we produce a forecas for 1983Q4. This procedure is repeaed unil we have a forecas for 2Q4. 21 For equaion (3), we esimae a differen equaion for each mehod of esimaing he oupu gap. Since each unobserved componen model provides wo esimaes, one filered and he oher smoohed, and he srucural VAR mehod also generaes wo esimaions (RTP and RTP1), his makes 14 differen forecasing equaions. 22 This represens he daa se ha was available in 1982Q3. 18

21 We perform he same exercise wih final oupu gap esimaes. Normally, we would expec forecass using final esimaes of he oupu gap o be more precise, given ha hese esimaes incorporae a broader se of informaion Crieria for selecing he number of lags For each equaion, we mus choose he number of lags for β and λ. We used hree differen crieria: he Akaike (1973) informaion crierion, he Schwarz (1978) informaion crierion, and a crierion based on he ou-of-sample forecas errors (OSFE). The Akaike (AIC) and Schwarz (SBC) informaion crieria boh use he enire sample available. These crieria are illusraed by equaions (32) and (33): AIC = T log Σ + 2N (32) SBC = T log Σ + N log( T ), (33) where T is he number of observaions, N is he number of coefficiens o esimae, and Σ is he deerminan of he covariance marix of he residuals. For boh echniques, he opimal specificaion is he one ha minimizes he value of he es. The OSFE crierion formalized by Murchison (1999) 23 is oally differen han he ones proposed by Akaike and Schwarz. Firs, we have o divide he sample ino wo pars. Wih observaions of he firs par of he sample, we esimae he coefficiens of he model and use hem o forecas inflaion in he second par of he sample. The opimal specificaion is he one ha minimizes forecass errors in he second par of he sample. This crierion can be formalized as follows: min φ [ 1, p] k 1 p k ( y ˆ β i y l ) s 2 V ' = 1 h= 1l = 1 Γ Γ = ; Γ = i= = s+ v ˆ β l, h k ( y ˆˆ βy ) 1 1, (34) where k is he number of lags, p is he maximum number of lags ha we wan o es, T is he oal number of observaions in he full sample, s is he number of observaions in he firs subsample, v is he forecasing horizon, Γ is he forecasing error, y is he variable ha we wan o forecas, and φ = T-(s+v). Again, we have o make some choices. Firs, we se he parameer p a 2, which is quie large for a forecasing model ha uses quarerly daa. Of course, we fix v a 5 [since (+4) - (-1) = 5]. For 23 Murchison menions ha his is a mehodology commonly applied by forecasers. The goal of his paper is o compare his procedure wih oher selecion crieria. 19

22 he parameer s, Murchison finds ha he formula s =.7T gives good resuls. However, we adap Murchison's approach o beer mee our needs. We arbirarily adap Murchison's rule o s =T-v- 15, since his rule does no allow enough observaions for he esimaion of he firs vinages Evaluaion of forecass The evaluaion procedure in his case consiss of comparing he ou-of-sample forecass wih acual inflaion. To deermine wheer he oupu gap esimaes significanly improve inflaion forecass, we use a es similar o he one proposed by Diebold and Mariano (1995). They build a es ha follows a normal disribuion, N(,1), and ha shows wheher one series of forecas errors is significanly differen from anoher one. 24 This es can be wrien as follows: d DM =, (35) f ( ) T 2πˆd 1 T where d = [ g( e ) g( e )] = 1 i j (36) T T () 1 ˆ τ 2π f = ˆ d 1 γ d ( τ ) (37) τ = ( T 1) S( T ) T 1 ˆ γ τ = ( d d )( d d ). (38) d ( ) T τ = T + 1 The wo series of forecas errors are designaed by he erms e i and e j. The expression g() designaes he ransformaion made o he series of errors. For his paper, we use squared errors [g(e i ) = e 2 i ]. The expression 2πf d () represens he weigh sum of sample auocovariances and is used o correc serial correlaion problems. Since he forecasing horizon is longer han one period (i is five quarers), we follow Diebold and Mariano's recommendaion and use a recangular kernel. Harvey, eybourne, and Newbold (1997) propose wo small modificaions o he Diebold Mariano es, o improve he properies wih a small sample. In his paper, we use his modified version of he Diebold and Mariano es. The firs modificaion ha hey propose is as follows: τ ( h 1) n + 1 2h + h / n MDM = DM, (39) n 24 Of course, boh series mus forecas he same variable. 2

23 where n is he number of observaions in each series of forecas errors and h is equal o he forecasing horizon minus 1. The second modificaion ha Harvey, eybourne, and Newbold propose is o use a n-1 disribuion insead of a normal disribuion o deermine he criical value of he es. By applying he modified version of he Diebold and Mariano es, we can verify wheher equaions ha incorporae an oupu gap measure significanly improve inflaion forecass relaive o an equaion ha excludes such a measure. This es also allows us o deermine wheher cerain mehods generae oupu gap esimaes ha do a beer job of forecasing inflaion han he oher esimaion echniques. 4. Resuls This secion presens he resuls obained for he wo evaluaion mehods. Firs, we show he oupu gap esimaes generaed by he differen mehods. 4.1 Oupu Gap esimaes Figures 1 and 2 compare he final and real-ime oupu gap esimaes for he nine esimaion mehods described in secion 2.2 (he srucural VAR mehods generae wo oupu gap esimaes: RTP and RTP1). Table 1 provides descripive saisics on he various real-ime, quasi-real, quasi-final, and final esimaes. Firs, we can clearly see ha he linear rend and he Wason model do no generae realisic business cycles. In paricular, heir real-ime esimaes are negaive for he enire period presened in Figures 1 and 2. Second, he AR1 saisics of Table 1 show ha all echniques generae persisen business cycles, wih he excepion of he Beveridge-Nelson decomposiion, which has a firs-order auocorrelaion coefficien in he range of.55 o.58. In fac, he Beveridge-Nelson business cycle shows many more urning poins han any oher esimaion echniques (see he TP saisics in Table 1). Figures 1 and 2 also show a lo of variabiliy in erms of he magniude of he gap esimaes. This is rue for boh final and real-ime esimaes. These figures also show ha differen oupu gap esimaes do no necessarily display he same shor-erm flucuaions. In fac, some echniques frequenly display posiive oupu gaps while ohers display negaive ones. 21

24 Figure 1 Per cen Final Esimaes 1982Q2-1999Q4 inear rend Quadraic rend Hodrick-Presco Beveridge-Nelson VAR - RTP VAR - RTP1 Wason Clark Harvey-Jaeger Kichian ' -1 22

25 Figure 2 Per cen Real-Time Esimaes 1982Q2-1999Q4 Quadraic rend Hodrick-Presco Beveridge-Nelson VAR - RTP VAR - RTP1 Wason Clark Harvey-Jaeger Kichian inear rend ' -2 *Noes: The linear rend refers o he righ scale, while all oher series refer o he lef scale. Therefore, he line represening an oupu gap of zero does no apply for he linear rend since his series is always negaive in real-ime. 23

26 Table 1 : Saisics on oupu gap esimaes 1982Q2 1999Q4 25 Mehods MEAN SD MIN MAX AR1 COR TP inear rend Finale Temps réel Quasi-réelle Quadraic rend Finale Temps réel Quasi-réelle Hodrick-Presco Finale Temps réel Quasi-réelle Beveridge-Nelson Finale Temps réel Quasi-réelle VAR RTP Finale Temps réel Quasi-réelle VAR RTP1 Finale Temps réel Quasi-réelle Wason Finale Temps réel Quasi-réelle Quasi-finale Clark Finale Temps réel Quasi-réelle Quasi-finale Harvey-Jaeger Finale Temps réel Quasi-réelle Quasi-finale Kichian Finale Temps réel Quasi-réelle Quasi-finale Noes: The saisics shown for each variable are: MEAN, he mean; SD, he sandard deviaion; MIN and MAX, he minimum and maximum values; AR1, he firs-order serial correlaion; COR, he correlaion wih he final esimae for ha mehod; and TP, he number of urning poins. 24