Implementing the Wild Bootstrap using a Two-Point Distribution

Transcription

1 Implemening he Wild Boosrap using a Two-Poin Disribuion James Davidson Universiy of Exeer Andrea Monicini Universiy of Exeer David Peel Universiy of Lancaser June 006 Keywords: Wild Boosrap, wo-poin disribuion, heeroscedsiciy, skewness JEL Classi caion: C5 Absrac We consider he problem of selecing he auxiliary disribuion o implemen he wild boosrap for regressions feauring heeroscedasiciy of unknown form. Asympoic re nemens are nominally obained by choosing a disribuion wih second and hird momens equal o. We show ha his sipulaion may fail in pracice, due o he disorion imposed on higher momens. We propose a new class of wo-poin disribuions and sugges using he Kolmogorov-Smirnov saisic as a selecion crierion. The resuls are illusraed by a Mone Carlo experimen. Corresponding auhor, james.davidson@exeer.ac.uk. We hank Bernard Pearson for helpful discussion.

2 Inroducion The wild boosrap is a varian of he boosrap mehod for applicaion o daa which are no i.i.d. and which, in paricular, are heeroscedasic; see Wu (986), Beran (986), Liu (988), Mammen (993), Davidson and Flachaire (00), and for a recen empirical applicaion, Paya and Peel (006). Consider a regression model y = 0 x + u where x are xed in repeaed samples and u s iid(0; ), wih 6= s for 6= s; in general. Leing ^ denoe he OLS esimaor, our objec is o consruc a boosrap analogue for he disribuion of p n ^ = p n x X x 0 n = x u = whose variance marix, in paricular, is X ne(^ )(^ ) 0 n = n x X x 0 n X = = x x 0 n x x 0 = : (.) Le ^u denoe a random resampling of he rescaled leas squares residuals, such ha r n P ^u = n k ^u = n and le p n ^ ^ = p n x X x 0 n = x ^u = denoe he boosrap analogue of p n(^ ). Leing E denoes he expeced value under he boosrap disribuion, i is easily esablished ha E (^ ^) = 0, bu if s denoes he usual unbiased residual variance esimaor hen ne (^ ^)(^ ^) X 0 = ns n x x 0 = = = ^u x x 0 = = = x x 0 = + Op (n = ) (.) Noe ha he implici variance esimaor is inconsisen for (.), so ha he boosrap fails in his case. However, leing ( ; : : : ; n ) be random drawings from a disribuion having E = 0 and E =, and independen of (^u ; : : : ; ^u n ), so-called wild boosrap replicaes have he form r n ^u = n k ^u. Leing E denoe expeced values under he wild boosrap disribuion, noe ha ne (^ ^)(^ ^) 0 = n n k x X x 0 n X = = ^u x x 0 n x x 0 = : = ne(^ )(^ ) 0 + O p (n = ): This is one of several possible ransformaions of he residuals - see Davidson and Flachaire (00).

3 Thus, under he usual regulariy condiions his disribuion is consisen for he sampling disribuion of p n(^ ). The regular boosrap sill provides asympoically valid ess, because alhough he variance in (.) does no mach he sampling disribuion he boosrap -raios are noneheless asympoically N(0; ). However, he coverage probabiliies of con dence inervals are asympoically biased, and asympoic re nemens of he error in rejecion probabiliy (ERP) are no aained. As well as asympoically valid con dence inervals, he wild boosrap has been shown (Liu 988, Davidson and Flachaire 00) o yield asympoic re nemens in he disribuions of pivoal saisics. The key fac is ha if E 3 =, hen E^u 3 = E^u 3 in view of he independence of he componens. Agreemen of he hird momens of he boosrap shocks wih ha of he paren disribuion of he rescaled residuals means ha he rs-order erms in he Edgeworh expansions of an asympoically pivoal saisic in he wo cases agree likewise, wih a corresponding reducion of ERP. By he same oken, if we could arrange for addiional higher momens of o equal, hen we should correspondingly mach he higher momens of ^u and ^u, leading o addiional re nemens. Unforunaely, no disribuions wih his desirable propery exis, in view of he inequaliy E 4 + (E 3 ) (.3) (Pearson, 96). However, noe ha if E^u 3 = 0, hen E(^u ) 3 = 0 regardless of he value of E 3. In his case, if we could arrange o have E 4 =, which is possible according o (.3), hen E^u 4 = E^u 4 in addiion o E^u 3 = E^u 3, implying agreemen of he second-order erms of he expansions: For example, i is shown by Davidson and Flachaire ha in he rs of hese cases, he leading erm in he asympoic expansion of he ERP of a -ailed es is of O(n ). In he second case, however, he leading erm of O(n ) in he expansion also vanishes. Choice of he Auxiliary Disribuion A number of disribuions can be considered o play he role of generaion process for he, ful lling one or more of he requiremens deailed above. Liu (988) and Mammen (993) sugges alernaive schemes o mee he requiremen E 3 =, of which he mos widely adoped appears o be he wo-poin disribuion, 8 + p p 5 5 >< wih probabiliy p = p 5 A = p (.) >: 5 wih probabiliy p. This has he properies E A = 0, E A = E3 A = and E4 A = : Two-poin disribuions are he only class for which (.3) holds as an equaliy, an imporan propery in heir favour for his role. An alernaive case is he so-called Rademacher disribuion, aking he form = ( wih probabiliy p = wih probabiliy p. (.) and has he properies E = 0, E =, E3 = 0 and E4 =. This laer disribuion o ers he possibiliy of he higher-order improvemens noed in he las secion when he paren disribuion is symmeric. 3

4 a 0.5 Figure : Skewness (solid line) and kurosis (dashed line) of -poin disribuions a Focussing on he choice beween hese wo-poin alernaives, here is evidenly a con ic beween achieving he bes improvemen of he ERP in he wors case in which he paren disribuion is skewed, hence favouring A, and aking advanage of possible symmery of he paren disribuion o achieve a beer re nemen by using. Insead, we propose exending he range of possibiliies o achieve a poenial balance of advanages. A wo-poin disribuion being compleely speci ed by wo poin values and he associaed probabiliy, he condiions of zero mean and uni variance resric he remaining free parameers o one. De ne a class of wo-poin disribuions a indexed on a parameer a > 0, such ha E a = 0 and E a =, by 8 >< a = >: a wih probabiliy + a a wih probabiliy a + a. Seing a = A = ( + p 5) :68 yields (.) while seing a = yields (.). Moreover, since hese are wo-poin disribuions, he relaion E 4 a = (E 3 a) + holds for each a implying E 4 a < E 3 a + in he range < a < A. Figure shows he relaionship beween he wo momens, ploed as funcions of a. The di erence beween he curves aains is minimum of 0:8 a a :47. Raher han choosing beween he wo exremes, hese facs sugges choosing a o opimize he agreemen beween boosrap and paren disribuions. We denoe he class of wild boosrap disribuions so de ned by ^u a = ^u a where ( a ; : : : ; an ) are independen drawings from (.3). An easily implemened mehod for checking he choice of a empirically is o compue he Kolmogorov-Smirnov (KS) saisic for he residuals ^u, relaive o he quaniles of he boosrap (.3) 4

5 disribuion of ^u a. Le ^u () denoe he h order saisic associaed wih he observed residuals, such ha ^u () ^u (n). Given B independen replicaions drawn from some chosen wild boosrap disribuion, say f^u (j); : : : ; ^u n (j)g for j = ; : : : ; B, de ne he funcion ^G() = Bn BX nx j= s= (^u s (j) ^u () ) where () is he indicaor funcion aking he value when is argumen is rue, and 0 oherwise. ^G() can be viewed as an esimae of he quaniy F (^u () ), where F is he CDF of he wild boosrap disribuion, and we expec o observe ^G() =n when f^u ; : : : ; ^u n g is a drawing from his disribuion. Accordingly, de ne KS = n = max n j ^G() =nj as our indicaor of he agreemen beween he wild boosrap and paren disribuions. Noe, in his applicaion he saisic is used solely as a basis for ranking alernaive choices of he parameer a. In he presence of skewness, he hypohesis of acual agreemen beween he disribuions canno ruly hold. 3 Experimenal Evidence We performed Mone Carlo experimens o compare di eren choices of a in a regression model exhibiing boh heeroscedasiciy and skewness of he disurbances. The model is (.4) y = + x + u ; = ; : : : ; n where = = 0, x = sin(=n) and u = p exp( + =n)" where " has a skew-suden disribuion. The laer is a mixure disribuion of he form " = jz j e x e ( x) where z is a Suden variae wih degrees of freedom ( > ) and x is independenly Bernoulli disribued wih probabiliy of success P (x = ) = + e : is a parameer o capure he skewness, wih = 0 represening symmery (see Fernandez and Seel, 998). By choice of and, we may arrange for " o have any desired con guraion of skewness and kurosis. Noe he array formulaion of he model, ensuring a comparable paern of heeroscedasiciy a each sample size. In our experimens we chose = 5:5 and =, and = 5. Table shows he rejecion relaive frequencies in 00,000 replicaions, in he wo-sided -es of he rue hypohesis = 0. The es was conduced using he robus saisic j j where r P n n = = (x q n Pn = (x x)^u x) ^u : (3.) The choice of rigonomeric rend ensures he esimaor of is consisen, in his seup. 5

6 Asympoic Regular Wild Boosrap n Crierion Boosrap a = a = :06 a = :4 a = : [0.398] [0.56] [0.667] [0.770] [0.548] [0.730] [0.849] [0.954] [0.765] [0.990] [.66] [.34] [.089] [.389] [.587] [.773] [.587] [.98] [.69] [.453] Table : Rejecion Frequencies of he Null Hypohesis in 00,000 Replicaions. (Mean KS saisics in square brackes.) In he able, asympoic crierion means ha he rejecion crierion ook he form j j > :96, and noe ha his es is asympoically correcly sized. The rejecion crierion for he boosrap ess is P ~ ( j j) < 0:05, where is de ned by (3.) wih ^u replaced by ^u s or ^u as, and P ~ denoes he probabiliy under he boosrap EDP, esimaed by Mone Carlo wih 99 replicaions. The gures in square brackes in he las four columns are he mean values of he KS saisic from (.4), which is compued in each Mone Carlo replicaion. Repeaing he experimens wih = 00, he case of virually normal kurosis, revealed lile or no di erence in he resuls. 4 Discussion The example chosen is deliberaely a wors case, wih heavy skewness as well as pronounced heeroscedasiciy. The skew-correced wild boosrap migh be expeced o perform bes relaive o is rivals, and i is herefore noable ha he wild boosrap ERP noneheless increases monoonically wih a, he case a = A barely improving on he asympoic crierion. The wild boosrap wih he Rademacher disribuion (a = ) is unambiguously he winner amongs he six alernaives, a all sample sizes. Moreover, his resul is re eced accuraely in he repored KS saisics. We cie his as srong evidence ha he KS saisic is a reliable guide o he relaive ERP of he corresponding wild boosrap es. We noe in conclusion ha Liu (988) and Mammen (993) sugges an alernaive auxiliary disribuion based on normal variaes, = W W EW EW where W i s N( i ; ) and p p 7=6 + =6 = p = 7=6 p =6 : I can be veri ed ha his disribuion has E = 0, E = E 3 =. However, i is also easily veri ed ha E 4 = 5:65. As we have noed previously, wo-poin disribuions o er he mos favourable rade-o beween hird and fourh momens. I appears unlikely ha alernaive disribuions of his ype could provide a beer remedy for our problem. 6

7 References Beran, R. (986) Discussion: Jackknife, boosrap and oher resampling mehods in regression analysis, Annals of Saisics 4, Davidson, J. (006) Time Series Modelling 4.8, a hp:// Davidson, R., Flachaire, E. (00) The wild boosrap, amed a las. Economic Research Working Paper No Queen s Insiue for Doornik, J. A. (004) Ox: an Objec-Oriened Marix Programming Language. Timberlake Consulans Ld. Fernandez, C., and M. Seel (998) On Bayesian modelling of fa ails and skewness, Journal of he American Saisical Associaion 93, Liu, R. Y. (988) Boosrap procedure under some non-i.i.d. models. Annals of Saisics 6, Mammen, E. (993) Boosrap and wild boosrap for high dimensional linear models. Annals of Saisics, Paya, I., Peel, D. A. (009) On he speed of adjusmen in ESTAR models when allowance is made for bias in esimaion. Economics Leers 90, Pearson, K. (96) Mahemaical conribuions o he heory of evoluion XIX; second supplemen o a memoir on skew variaion, Philos. Trans. Roy. Soc. London, Ser A 6, 43. Wu, C. F. J.,(986). Jackknife boosrap and oher resampling mehods in regression analysis. Annals of Saisics 4,