A Robust Modification of the Goldfeld-Quandt Test for the Detection of Heteroscedasticity in the Presence of Outliers

Transcription

1 Journal of Mahemaics and Saisics 4 (4): 77-83, 8 ISSN Science Publicaions A Robus Modificaion of he Goldfeld-Quand Tes for he Deecion of Heeroscedasiciy in he Presence of Ouliers 1 Md. Sohel Rana, 1 Habshah Midi and A.H.M. Rahmaullah Imon 1 Laboraory of Applied and Compuaional Saisics, Insiue for Mahemaical Research, Universiy Pura Malaysia, 434 Serdang, Selangor, Malaysia Deparmen of Mahemaical Sciences, Ball Sae Universiy, Muncie, IN 4736, USA Absrac: Problem saemen: The problem of heeroscedasiciy occurs in regression analysis for many pracical reasons. I is now eviden ha he heeroscedasic problem affecs boh he esimaion and es procedure of regression analysis, so i is really imporan o be able o deec his problem for possible remedy. The exisence of a few exreme or unusual observaions ha we ofen call ouliers is a very common feaure in daa analysis. In his sudy we have shown how he exisence of ouliers makes he deecion of heeroscedasiciy cumbersome. Ofen ouliers occurring in a homoscedasic model make he model heeroscedasic, on he oher hand, ouliers may disor he diagnosic ools in such a way ha we canno correcly diagnose he heeroscedasic problem in he presence of ouliers. Neiher of hese siuaions is desirable. Approach: This aricle inroduced a robus es procedure o deec he problem of heeroscedasiciy which will be unaffeced in he presence of ouliers. We have modified one of he mos popular and commonly used ess, he Goldfeld-Quand, by replacing is nonrobus componens by robus alernaives. Resuls: The performance of he newly proposed es is invesigaed exensively by real daa ses and Mone Carlo simulaions. The resuls sugges ha he robus version of his es offers subsanial improvemens over he exising ess. Conclusion/Recommendaions: The proposed robus Goldfeld-Quand es should be employed insead of he exising ess in order o avoid misleading conclusion. Key words: Heeroscedasiciy, ouliers, robus es, modified goldfeld-quand es, Mone Carlo simulaion INTRODUCTION I is a common pracice over he years o use he Ordinary Leas Squares (OLS) as he inferenial echnique in regression. Under he usual assumpions, he OLS possesses some nice and aracive properies. Among hem homogeneiy of error variances (homoscedasiciy) is an imporan assumpion for which he OLS esimaors enjoy he minimum variance propery. Bu here are many occasions [] when he assumpion of homoscedasic error variance is unreasonable. For example, if one is examining a cross secion of firms in one indusry, error erms associaed wih very large firms migh have larger variance han hose of error erms associaed wih smaller firms. If he error variance changes we call he error heeroscedasic. Heeroscedasiciy ofen occurs when here is a large difference among he sizes of he observaions. I is really imporan o deec his problem because if his problem is no eliminaed he leas squares esimaors will sill be unbiased, bu hey will no longer have he minimum variance propery. This means ha he regression coefficiens will have larger sandard errors han necessary. A large number of diagnosic plos are now available in he lieraure [3,4,9,11,14] for deecing heeroscedasiciy. Bu graphical mehods are very subjecive so we really need analyical mehods o deec he problem of heeroscedasiciy. Rigorous procedures for esing he homoscedasiciy of daa are available in he lieraure [1,5,15]. Mos of hese echniques are based on he leas squares residuals bu here is evidence ha hese residuals may no exhibi heeroscedasic paern if ouliers are presen in he daa. According o Hampel e al. [7] he exisence of 1-1% ouliers in a rouine daa is raher rule han excepions. We suspec ha hese analyical ess may suffer from possessing poor power in he presence of ouliers. In his sudy we firs invesigae how he commonly used heeroscedasic ess perform in he Corresponding Auhor: Md. Sohel Rana, Laboraory of Applied and Compuaional Saisics, Insiue for Mahemaical Research, Universiy Pura Malaysia, 434 Serdang, Selangor, Malaysia 77

2 presence of oulier. We observe ha all he ess considered in our sudy suffer in his siuaion and for his reason we robusify he Goldfeld-Quand es. Real daa ses and simulaion experimens show ha he proposed modified Goldfeld-Quand es ouperforms oher ess in deecing heeroscedasiciy in he presence of ouliers. An excellen review of differen analyical ess for he deecion of heeroscedasiciy is available in Kuner, Nachsheim and Neer [1] and in Chaerjee and Hadi [3]. In our sudy we consider he ess which are very popular and commonly used in economerics. Firs we consider he Goldfeld-Quand es. This es is applicable if one assumes ha he heeroscedasic variance σ s is posiively relaed o one of he explanaory variables in he model. For simpliciy, le us consider he usual wo-variable model: Y = α + β X + u (1) Suppose σ is posiively relaed o X as: σ = σ () X where, σ is a consan. Such an assumpion has been found quie useful in family budges. If () is appropriae, i would mean σ would be larger for he larger values of X. If ha urns ou o be he case, heeroscedasiciy is mos likely o be presen in he model. To es his explicily, Goldfeld and Quand [9] sugges ordering he observaions according o he values of X, beginning wih he lowes X value. To possess beer power hey sugges omiing c cenral observaions. OLS regressions are fied separaely o he firs and las (n-c)/ observaions and he respecive residual sum of squares RSS 1 and RSS are obained. Under normaliy of errors, each RSS follows a chisquare disribuion wih (n-c-k)/ degrees of freedom, where k is he number of parameers o be esimaed, including he inercep. Then he raio: RSS / df RSS / df RSS RSS λ = = (3) Under he assumpion of normaliy and homoscedasiciy λ follows an F disribuion wih numeraor and denominaor d.f. each of (n-c-k)/. The Goldfeld-Quand es is a naural es o apply when one can order he observaions in erms of he increasing variance of he error erm (or one independen variable). An alernaive es, which does J. Mah. & Sa., 4 (4): 77-83, 8 78 no require such an ordering and is easy o apply, is he Breusch-Pagan es. To illusrae his es, consider he k-variable linear regression model: Y = 1 + X + + βkxk + u (4) Assume ha he error variance σ is described as: σ = f (α 1 + α Z + + α m Z m ) (5) ha is, σ is some funcion of he nonsochasic variable Z s; some or all of he X s can serve as Z s. We also assume ha: σ = α 1 + α Z + + α m Z m (6) ha is, σ is a linear funcion of he Z s. If α = α = = α m =, we ge σ = α 1, which is a consan. Therefore, o es wheher σ s are homoscedasic, one can es he hypohesis ha α = α 3 = = α m. This is he basic idea behind he Breusch-Pagan es. In his es we Esimae he model (6) by he leas squares mehod and obain he residuals. Then he mean of squared leas square residuals ˆσ = û / n are compued and he û variable p = is consruced. Nex we regress p σ on ˆ he Z s o Obain he SSE (error sum of squares). Finally we compue he es saisic: T = SSE/ (7) Under he assumpion of normaliy and homoscedasiciy T ~ χ m 1 and if he value of T exceeds he criical value, we conclude ha heeroscedasiciy is presen in he daa. Unlike he Goldfeld-Quand es, which requires reordering he observaions wih respec o he X variable ha supposed o cause heeroscedasiciy, or he Breusch-pagan es, which is sensiive o he normaliy assumpion, he general es of heeroscedasiciy proposed by Whie [] does no rely on he normaliy assumpion and is very easy o implemen. As an illusraion of he basic idea, consider he following hree-variable regression model (he generalizaion o he k-variable model is sraighforward). Given he daa, we esimae regression parameers by he OLS mehod and obain he residuals. We hen run he following (auxiliary) regression: û = α + α X + α X + α X + α X + α X X + e (8)

3 J. Mah. & Sa., 4 (4): 77-83, 8 where e is he random error erm. In oher words, he squared residuals from he original regression are regressed on he original X variables or regressor, heir squared values and he cross produc(s) of he regressors. Higher power of he regressors can also be inroduced. I is imporan here o noe ha here is a consan erm in his equaion even hough he original regression may or may no conain i. Now we obain he R from his (auxiliary) regression. Under he null hypohesis ha here is no heeroscedadiciy, i can be shown ha sample size (n) imes he R obained from he auxiliary regression asympoically follows he chisquare disribuion wih d.f. equal o he number regressors (excluding he consan erm) in he auxiliary regression as given in (8). Tha is, for a regression model wih p regressors: n.r ~ Asympoic χ a (9) Trimmed of Squares (LTS) mehod suggesed by Rousseeuw and Leroy [13] o fi he regression line. We compue he deleion residuals [8] for he enire daa se based on a fi wihou he poins idenified as ouliers by he LTS fi. Sep 4: For boh he groups compue he Median of he Squared Deleion (MSDR) and compue he raio MSDR MGQ = MSDR 1 (1) where, MSDR 1 and MSDR are he median of he squared deleion residuals for he smaller and he larger group variances respecively. Under normaliy, he MGQ saisic follows an F disribuion wih numeraor and denominaor degrees of freedom each of (n-c-k)/. The degrees of freedom a = p +p-1. If he chisquare value hus obained exceeds he criical chisquare value a he chosen level of significance, he conclusion is ha here exiss heeroscedasiciy. MATERIALS AND METHODS Modified Goldfield-Quand es: we have briefly discussed some popular ess for heeroscedasiciy deecion. Bu here is evidence ha all hese ess suffer a huge seback when ouliers are presen in he daa. So we need o develop a es which is no much affeced by ouliers. Here we propose a new es which is a modificaion of he Goldfeld-Quand es. We firs idenify which componens of he Goldfeld-Quand es are affeced by ouliers and hen replace hese componens by robus alernaives. I is worh menioning ha his kind of replacemen does no help he oher wo ess ha we consider in he previous secion. We call his es he Modified Goldfeld-Quand (MGQ) es which, we believe, will be more powerful han he exising ess in he presence of ouliers. Here we ouline he proposed modified Goldfeld- Quand es. This es conains he following seps: Sep 1: Likewise he classical Goldfeld-Quand es, we order or rank he observaions according o he value of X, beginning wih he lowes X values Sep : We omi cenral c observaions, where c is specified a priori and hen we divide he remaining (n-c) observaions ino wo groups each of (n-c)/ observaions Sep 3: Check for he ouliers by any robus regression echnique. We prefer o use he robus Leas 79 RESULTS AND DISCUSSION Numerical examples: Here we presen few examples o show he advanage of using he modified Goldfeld- Quand es in he deecion of heerogeneiy of error variances problem. Housing expendiures daa: Our firs example is he housing expendiures daa given by Pindyck and Rubinfeld [1]. This single-predicor daa se (Table 1) conains observaions ha give housing expendiure for four differen income groups. As expeced, peoples wih higher income have relaively more variaion in heir expendiures on housing. The scaer plo as shown in Fig. 1 clearly exhibis he heeroscedasic paern of he daa Fi. Values Fig. 1: vs. fied plo for original housing expendiures daa

4 Table 1: Original and modified housing expendiures daa Index Income Housing Exp. Index Income Housing Exp (4.9) (.) Table : Heeroscedasiciy diagnosics for housing expendiures daa Wihou ouliers Wih ouliers Tes Saisic p-value Saisic p-value Goldfeld Quand Breusch Pagan Whie MGQ Fi. Values Fig. : vs. fied plo for modified housing expendiures daa We now deliberaely pu wo ouliers ino he daa se by replacing he housing expendiures of he cases indexed by 1 and (modified values are presened wihin he parenheses). Figure shows he residuals-fis plo for he housing expendiures daa. This plo is no as clear as Fig. 1 in exhibiing variance heeroscedasiciy and we definiely need analyical ess o draw a definie conclusion. J. Mah. & Sa., 4 (4): 77-83, We apply all he convenional ess like he Goldfeld-Quand, he Breusch-Pagan and he Whie ess on he original and modified housing expendiures daa and he resuls are presened in Table. We observe from his able ha all he convenional ess variance when he daa se is free from ouliers. heeroscedasiciy on boh he occasions Fi. Values Fig. 3: vs. fied values for original consumpion expendiure daa Bu all hese ess fail o deec he problem of heeroscedasiciy when ouliers occur in he daa se. We now apply our proposed modified Goldfeld-Quand es on he original and modified housing expendiures daa and hese resuls are presened in Table. We observe from his able ha he modified Goldfeld- Quand es performs in a similar way as he Goldfeld- Quand es when here is no oulier. Bu unlike he oher ess, i can successfully deec he heeroscedasiciy in he presence of ouliers yielding a highly significan p-value. Consumpion expendiure daa: Our nex example is he consumpion expendiure and income daa given by Gujarai [6]. This daa conains 3 observaions and i shows ha he expendiure of peoples vary wih heir income. So we can guess ha in his daa he variaion is no consan. We now deliberaely pu hree ouliers ino he daa se by replacing he housing expendiures of he cases indexed by 1, and 3 (modified values are presened wihin he parenheses). This daa se ogeher wih he ouliers is shown in Table 3. Figure 3 and 4 shows he residuals-fis plo for he original and modified consumpion daa. The variance heerogeneiy is clearly visible wih he original daa bu when ouliers are presen in he daa his phenomenon is no clearly visible. Table 4 offers a comparison beween he newly proposed modified Goldfeld-Quand es and oher exising ess in he deecion of heeroscedasiciy for he consumpion daa. Table 4 shows ha he Goldfeld- Quand, he Breusch-Pagan and he Whie ess can correcly ideniy he heeroscedasic paern of variance when he daa se is free from ouliers bu hey become unsuccessful in he presence of ouliers. The modified Goldfeld-Quand es can successfully deec he

5 J. Mah. & Sa., 4 (4): 77-83, 8 Table 3: Original and modified consumpion expendiure daa Index Expendiure Income Index Expendiure Income Index Expendiure Income 1 55 (1) (1) (1) 5 Table 4: Heeroscedasiciy diagnosics for consumpion expendiure daa Wihou ouliers Wih ouliers p-value p-value Tes Saisic Saisic Goldfeld Quand Breusch Pagan Whie MGQ Fi. Values Fi. Value Fig. 5: vs. fied plo for original resaurans food sales daa 1 Fig. 4: vs. fied values for modified consumpion expendiure daa Resauran food sales daa: Finally we consider resauran food sales daa given by Mongomery e al. [11] In his daa se here is a relaion of income wih adverising expense. Again we deliberaely pu hree ouliers ino he daa se by replacing he income of he cases indexed by 1, 7 and 3 (modified values are presened wihin he parenheses). The original and he modified daa are shown in Table 5. Figure 5 and 6 show he residuals-fis plo for he original and modified consumpion daa. The varianc heerogeneiy is clearly visible wih he original daa bu when ouliers are presen in he daa his phenomenon is no clearly visible Fi. Values Fig. 6: vs. fied plo original resaurans food sales daa Likewise he previous examples we employ he Goldfeld-Quand, he Breusch-Pagan, Whie and modified Goldfeld-Quand es o he resaurans food sales daa and obain similar resuls ha we go earlier. Tes resuls as shown in Table 6 shows ha he hree convenional ess perform well in deecion of heeroscedasiciy bu heir performances become poor when ouliers are presen in he daa. 81

6 J. Mah. & Sa., 4 (4): 77-83, 8 Table 5: Original and modified resauran food sales daa Index Income Ad. Exp. Index Income Ad. Exp. Index Income Ad. Exp (3) (3) (1431) 1935 Table 6: Heeroscedasiciy diagnosics for resaurans food sales daa Wihou ouliers Wih ouliers Tes saisic p-value saisic p-value Goldfeld Quand Breusch Pagan Whie MGQ The modified Goldfeld-Quand es performs bes. Irrespecive of he presence of ouliers i can successfully deec he heeroscedasic error variance in he daa. Simulaion Sudy: Now from our experience wih individual daa ses we wan o confirm our resuls by reporing a Mone Carlo simulaion experimen. In our simulaion experimen, we consider a design of 5 and 1% ouliers in heeroscedasic daa. Here we consider a simple bu ineresing heeroscedasic variance problem where he variance is he square of he mean of he response variable. Le us consider a simple wo variable linear model: Y = X + (11) In our simulaion sudy, all he values of X are being aken equally spaced such as 1,,, 1 and hese values are replicaed several imes o ge higher sample sizes. We generae he random errors from Normal disribuions wih mean and sandard deviaions X, X and 3X. We pu ouliers in he error erm in every h or 1h posiion o generae 5 and 1% ouliers respecively. The magniude of he oulier is 5 imes he sandard deviaion of he original errors. The Y values are obained from he Eq. 11. We run his simulaion experimen for five differen sample sizes n =, 3, 4, 6 and 1. To assess which of he ess does he bes in deecing heeroscedasiciy in he presence of ouliers we consider powers of four ess, he convenional Goldfeld-Quand, Breusch-Pagan and Whie ess and he newly proposed modified Goldfeld-Quand es. For each es we se he level of significance.5 and he resuls of his experimen are shown in Table 7-9 each of which is based on he average of 1, simulaions. Table 7-9 offer comparisons beween he newly proposed modified Goldfeld-Quand es and convenional Goldfeld-Quand, Breusch-Pagan and Whie ess in he deecion of heeroscedasiciy for he 5 and 1% oulier daa. All he hree convenional ess perform very poorly in simulaion. The Goldfeld- Quand es performs relaively well for 5% ouliers. The Bruesch-Pagan es performs relaively well for 1% oulier cases bu is performance ends o deeriorae wih he increase in sample size. The Whie es performs wors in every siuaion. Throughou he simulaion experimen each of he convenional ess shows inconsisence paern for he sample size n = 3. Bu he newly proposed modified Goldfeld-Quand es performs superbly hroughou. For small sample size (n = ) and lower conaminaion (5%) is performance is similar o he Goldfeld-Quand es, bu is power ends o increase wih he increase in sample size. We also observe ha he es is robus in he sense ha i performs exacly in he same way when ouliers occur in a daa wih differen levels of error variances. Thus he modified Goldfeld-Quand es ouperforms he convenional ess in every respec and is proved o be he bes overall. 8

7 J. Mah. & Sa., 4 (4): 77-83, 8 Table 7: Simulaions resuls of heeroscedasiciy ess for error variance = X 5% Ouliers 1% Ouliers Tes GQ BP Whie MGQ Table 8: Simulaions resuls of heeroscedasiciy ess for error variance = 4X 5% Ouliers 1% Ouliers Tes GQ BP Whie MGQ Table 9: Simulaions resuls of heeroscedasiciy ess for error variance = 9X 5% Ouliers 1% Ouliers Tes GQ BP Whie MGQ CONCLUSION In his research we show ha all commonly used ess for deecing heeroscedasiciy fail when ouliers are presen in he daa. We develop a new es in his regard which is a simple bu robus modificaion of he Goldfeld-Quand es. The real daa ses and Mone Carlo simulaions show ha modified Goldfeld-Quand es offers subsanial improvemens over he exising ess and performs superbly in he deecion of heeroscedasiciy in he presence of ouliers. REFERENCES 1. Breusch, T. and A. Pagan, A simple es for heeroscedasiciy and random coefficien variaion. Economerica, 47: hp://ideas.repec.org/a/ecm/emerp/v47y1979i5p hml. Carroll, R.J. and D. Rupper, Transformaions and Weighing in Regression. nd Edn., Wiley, New York, pp: 49. ISBN: Chaerjee, S. and A.S. Hadi, 6. Regression Analysis by Examples. 4h Edn., Wiley, New York, pp: 375. ISBN: Draper, N.R. and H. Smih, Applied Regression Analysis. 3rd Edn., Wiley, New York, pp: 736. ISBN: Goldfeld, S.M. and R.E. Quand, Some ess for homoskedasiciy. J. Am. Sa. Assoc., 6: hp:// /readings/goldfeld-quand-1965.pdf Gujarai, D.,. Basic Economerics. 4h Edn., McGraw-Hill, New York, pp: 1. ISBN: Hampel, F.R., E.M. Ronchei, P.J. Rousseeuw and W. Sahel, Robus Saisics: The Approach Based on Influence Funcion. 1s Edn., Wiley, New York, pp: 536. ISBN: Imon, A.H.M.R., 3. from deleion in added variable plos. J. Applied Sa., 3: DOI: 1.18/ Greene, W.H., 8. Economeric Analysis. 6h Edn., Pearson Educaion, Inc., pp: ISBN: Kuner, M.H., C.J. Nachsheim and J. Neer, 4. Applied Linear Regression Models. 4h Edn., McGraw-Hill/Irwin, New York, pp: 71. ISBN: Mongomery, D.C., E.A. Peck and G.G. Vining, 1. Inroducion o Linear Regression Analysis. 3rd Edn., Wiley, New York, pp: 641. ISBN: Pindyck, S.R and L.D. Rubinfeld, Economeric Models and Economeric Forecass. 4h Edn., Irwin/McGraw-Hill, New York, pp: 634. ISBN: Rousseeuw, P.J. and A. Leroy, Robus Regression and Oulier Deecion, 1s Edn., Wiley, New York, pp: 39. ISBN: Ryan, T.P., 8. Modern Regression Mehods. nd Edn., Wiley, New York. pp: 664. ISBN: Whie, H., 198. Heeroscedasiciy-consisen covariance marix esimaor and a direc es for heeroscedasiciy. Economerica, 48: