RJOAS, 5(65), May 2017

Transcription

1 DOI hps://doi.org/ /rjoas THE EFFECT OF OUTLIERS ON THE PERFORMANCE OF AKAIKE INFORMATION CRITERION (AIC) AND BAYESIAN INFORMATION CRITERION (BIC) IN SELECTION OF AN ASYMMETRIC PRICE RELATIONSHIP Acquah De-Graf H., Associae Professor Deparmen of Agriculural Economics and Exension, Universiy of Cape Coas, Cape Coas, Ghana ABSTRACT Asymmeric price ransmission modelling aims o selec one model ha bes capures he asymmeric daa generaing process from a se of compeing models using model selecion mehods. However, such an ineres in model selecion oupace an awareness ha ouliers in daa can have a disproporionae impac on model ranking. In order o explore he issue, he effec of ouliers on he performance of commonly used Akaike Informaion Crieria (AIC) and Bayesian Informaion Crieria (BIC) in selecion of asymmeric price relaionship are evaluaed under condiions of differen sample size. Mone Carlo experimenaion indicaed ha he abiliy of he model selecion mehods o idenify he rue asymmeric price relaionship decreases wih an increase in ouliers in moderae and large samples. Wih 5% oulier-conaminaion in large samples, boh AIC and BIC fail o idenify he rue asymmeric price relaionship. BIC ouperforms AIC in selecing he asymmeric daa generaing process in large samples wih ouliers. However, in small samples, he effec of ouliers on he performance of AIC and BIC in selecion of he correc asymmeric model remains unclear. KEY WORDS Model selecion, Akaike s Informaion Crieria (AIC), Bayesian Informaion Crieria (BIC), asymmery, Mone Carlo, ouliers. Asymmeric price ransmission modelling aims o selec one model ha bes capures he asymmeric daa generaing process (DGP) from a se of compeing models. This enails choosing he model ha provides he bes fi o he daa on he basis of informaion crieria. The jusificaion for his approach is ha he model providing he bes fi is he one ha closely approximaes he underlying asymmeric daa generaing process. However, such an ineres in model selecion oupace an awareness ha ouliers in daa can have a disproporionae impac on model ranking. In he presence of ouliers, a model may provide he bes fi among a se of compeing models wihou necessarily closely approximaing he daa generaing process. Ouliers are observaions ha are rare and for some reasons, differen from majoriy of he observaions (see Barne & Lewis, 1994 for a deail discussion). The effec of ouliers on he ranking of compeing models has been menioned several imes in he saisics lieraure (e.g., Hoeing, Rafery, & Madigan, 1996; McCann, 2006; Ronchei, 1997; Ronchei, Field, & Blanchard, 1997). Previous research on Akaike Informaion Crieria (AIC) and Bayesian Informaion Crieria (BIC) has found hem sensiive o ouliers in regression analysis (e.g., Akinson & Riani, 2008; Chik, 2002; Laud & Ibrahim, 1995; Le, Rafery, & Marin, 1996). Akinson and Riani (2008) noes ha he sensiiviy of model selecion indices, such as AIC, o ouliers is an ofen overlooked issue. Lile is known abou he relaive performance of differen informaion crieria in asymmeric price ransmission modelling when he daa conains ouliers. Empirically, less effor has been made in examining he influence of ouliers on model selecion wihin he asymmeric price ransmission modelling conex. Noably, he abiliy of he commonly used model selecion mehods (AIC and BIC) o selec he rue asymmeric price ransmission model in he presence of ouliers has no ye been exensively invesigaed and is no well undersood. An imporan quesion which remains unanswered is how well will AIC and BIC 32

2 perform when ouliers are presen in he daa used for price ransmission analysis. In he presence of ouliers, will AIC and BIC poin o he correc asymmeric price ransmission model? In order o address his issue, his paper empirically evaluae and compare he performance of he wo commonly used model selecion crieria, AIC and BIC in choosing beween alernaive mehods of esing for asymmery in he presence of ouliers. The paper conribues owards undersanding he effec of ouliers on he model selecion performance of AIC and BIC in asymmeric price ransmission modelling framework. The rue daa generaing process is known in all experimens and he Mone Carlo simulaions are essenial in deriving he model recovery raes of he rue model. METHODOLOGY OF RESEARCH The process of selecing a saisical model from a se of candidae models is called model selecion. Informaion crieria provides he basis of choosing beween compeing models. The basic concep of informaion crieria is o selec saisical models ha simplify descripion of he daa and model. In effec, informaion crieria emphasizes minimising he amoun of informaion required o express he daa and model. In addressing he problem of choosing among compeing models, informaion crieria allows one o selec he model ha gives he mos accurae descripion of he daa. I addresses he rade-off beween descripive accuracy and minimizing he number of parameers. Akaike Informaion Crieria (AIC). A widely used informaion crieria, Akaike Informaion Crieria (AIC) was inroduced by Akaike (1973; 1974) via Kullback Lieber divengence. AIC is an esimae of he relaive expeced Kullback Lieber disance of a given model from he rue model. AIC is derived as an asympoically unbiased esimaor of he expeced Kullback- Liebler discrepancy beween he rue and a fied model. I is defined as: A I C 2 lo g ( L ) 2 p (1), Where: he firs erm log( L) is he negaive maximum log-likelihood of he daa given he model parameer esimaes and he second erm p, is he number of parameers in he model. AIC aims o find he bes approximaing model o he daa generaing process. Models wih smalles AIC values are deemed as bes. Bayesian Informaion Crieria (BIC). Anoher widely used informaion crieria, he Bayesian Informaion Crieria (BIC) was proposed by Schwarz (1978) as an asympoic approximaion o a ransformaion of he Bayesian poserior probabiliy of a candidae model. The compuaion of he BIC is based on he empirical log-likelihood of he candidae model and does no require he specificaion of priors. BIC is defined as: B I C 2 lo g ( L ) p lo g ( n ) (2), Where: n is he sample size, p is number of parameers in he model and log( L) is he negaive maximum log-likelihood of he daa given he model. BIC is consisen and ends o selec he rue model wih a probabiliy of one as sample size increases. Under his selecion crieria, models wih minimum BIC are preferred. Asymmeric Price Transmission Models. Several economeric models have been developed o esimae asymmeric price ransmission. They include he Houck (1977) model (HKM), Sandard Error Correcion Model (SECM) and he Complex Error Correcion Model (CECM). For he purpose of breviy, he sandard asymmeric error correcion model, he complex asymmeric error correcion model and he Houck s model are denoed by SECM, CECM and HKM respecively. The Houck s model is specified as follows: 33

3 y 1 x 1 x (3) 2 ~ N(0, ) The Houck s Model (HKM) relaes changes in he response price ( y ) o he posiive and negaive changes in he oher price ( x, x ). Dynamic varians of his model can be esimaed o disinguish beween shor and long run asymmeries. The Sandard Error Correcion Model (SECM) is specified as follows: = + ( ) + ( ) + ~ (0, ) (4) The Sandard Error Correcion Model relaes changes in response price ( changes in he oher price ( ) as well as changes in he Error Correcion Term (ECT) x which is decomposed ino posiive and negaive componens (( y x) 1,( y x) 1). Eqn. (4) was proposed in Granger and Lee (1989). Engle and Granger (1987) noes ha if y and x are coinegraed, hen an error correcion represenaion exis. Coinegraion is firs esablished by esimaing he long run relaionship beween price y and x. The lagged residuals from he Eqn. (5) denoes he Error Correcion Term and is included in he sandard error correcion model. y x (5) o 1 y ) o The conemporaneous response erm ( x ) is segmened in Von Cramon-Taubadel and Loy (1996). This leads o he following specificaion in which conemporaneous and shor run response o deparures from he coinegraing relaion are asymmeric if 1 1 and 2 respecively: y = β x + β x + β (y x) + β (y x) + ε (6) ~ (0, ) In his case, a join F-es can be used o deermine symmery or asymmery of he price ransmission process. Noably, asymmeries specified affecs he direc impac of price increases and decreases as well as adjusmens o he equilibrium level. Where x and x are he posiive and negaive changes in x and he remaining variables are defined as in he sandard error correcion model. The asymmeric ECM wih complex dynamics ness he Houck s model in firs difference. RESULTS AND DISCUSSION In order o illusrae he abiliy of he model selecion mehods o recover he rue model when he daa conains ouliers, a series of Mone Carlo simulaion experimens are conduced and he resuls are repored below. The simulaion is based on he Sandard Error Correcion Model (SECM) daa generaing process specified as follows: y 0.7x 0.25( y x ) 0.75( y x ) (7), 1 1 y and x are generaed as I (1) non-saionary variables ha are coinegraed. The error (( y x ),( y x ) ) represen he posiive and negaive deviaions correcion erms 1 1 from he long run equilibrium relaionship beween y and x. For daa wihou ouliers, he 34

4 errors are generaed from a normal disribuion wih a mean 0 and a variance of 1 ( N(0,1) ). In order o creae ouliers in he daa, various percenages of ouliers (0, 2, 3, 4 and 5 percen) are inroduced ino he daa wihou ouliers. For example, wo percen of he number of observaions of he errors generaed for he normal daa wih values generaed from a normal disribuion wih a mean of 0 and a variance of 1, were replaced wih wo percen of he number of observaions from he normal disribuion wih a mean of 20 and variance of 1 ( N(20,1) ) for a chosen sample size. This is repeaed for 3, 4 and 5 percen of ouliers given he various sample sizes 50, 150 and 500 respecively. The daa generaing process is simulaed 1000 imes wih differen percenages of ouliers and across differen sample sizes. For each simulaion, he abiliy of he model selecion mehods o recover he rue daa generaing process is evaluaed. The percenage of samples in which each compeing model provides a beer model fi han he oher compeing models is referred o as he model recovery raes. The recovery raes are derived using 1000 Mone Carlo simulaions. In effec, he amoun of samples in which each model fis beer han he oher compeing models is measured ou of he 1000 samples and expressed as a percenage. Thus, he values obained from each model selecion crieria are calculaed as he arihmeic mean based on 1000 samples. Generally, he overall power of he model selecion mehods (AIC and BIC) o selec he rue daa generaing process decreased wih increase in he percenage of ouliers. Noiceably, ouliers have a subsanial impac on selecion power in moderae and large samples. In effec, he abiliy of AIC and BIC o selec he rue asymmeric daa generaing process decreased wih increase in he percenage of ouliers in moderae and large samples. Table 1 Relaive performance of he model selecion mehods (Small Sample) % of Ouliers Experimenal Crierion 0 n=50 σ = 1 2 n=50 σ = 1 3 n=50 σ = 1 4 n=50 σ = 1 5 N=50 σ = 1 Recovery raes based on 1000 replicaions. Model fied Mehods CECM (%) HKM (%) SECM (DGP) (%) AIC BIC AIC BIC AIC BIC AIC BIC AIC BIC Comparison of he differen crieria are illusraed in Tables 1, 2, and 3. Model recovery raes are presened for each crieria under various sample size condiions wih varying percenage of ouliers. In small samples, he performance rends of he model selecion crieria remains unclear. Noably in small samples, ouliers have an unclear effec on he abiliy of he model selecion mehods o recover he correc model. This is because in small samples, he expeced number of ouliers is n imes he percenage of ouliers. This leads o a few ouliers since n is small. The oulier effec becomes unclear in small samples and more pronounced in large samples. For ouliers percenages of 0, 2, 3, 4 and 5, he abiliy of he model selecion mehods o choose he rue asymmeric daa generaing process is seriously disored in small samples. For example, wihou ouliers in he daa (n=50), AIC and BIC recovered 78.7% and 81.8% respecively of he rue asymmeric daa generaing process. However, wih inroducion of 5% ouliers ino he daa (n=50), AIC and BIC recovered 78.2% and 83.7% respecively. In small sample size of 50, he model recovery raes derived when he daa conains ouliers does no show any subsanial difference from hose recovery raes derived when here was no oulier in he daa. 35

5 Table 2 Relaive performance of he model selecion mehods (Moderae Sample) % of Ouliers Experimenal Crierion 0 n=150 σ = 1 2 n=150 σ = 1 3 n=150 σ = 1 4 n=150 σ = 1 5 n=150 σ = 1 Recovery raes based on 1000 replicaions. Model fied Mehods CECM (%) HKM (%) SECM (DGP) (%) AIC BIC AIC BIC AIC BIC AIC BIC AIC BIC In moderae sample size of 150, a clear paern is seen in he performance of he model selecion mehods o recover he rue asymmeric daa generaing process as he percenage of ouliers increases. For example, as he percenage of ouliers increase from 0 o 5%, here is a subsanial decline in he abiliy of he model selecion mehods o selec he rue asymmeric daa generaing process, as he recovery raes for AIC and BIC decrease from 84.7% and 97.4% o 47.6% and 27.7% respecively. Noiceably, he performance of AIC and BIC in he selecion of asymmeric price relaionship is affeced by ouliers. Similarly, previous sudies (Laud & Ibrahim, 1995) found ha AIC and BIC are no robus o ouliers or influenial daa poins. Ronchei, Field and Blanchard (1997) also observed ha ouliers have an undue influence on model chosen. Table 3 Relaive performance of he model selecion mehods (Large Sample) % of Ouliers Experimenal Crierion 0 n=500 σ = 1 2 n=500 σ = 1 3 n=500 σ = 1 4 n=500 σ = 1 5 n=500 σ = 1 Recovery raes based on 1000 replicaions. Model fied Mehods CECM (%) HKM (%) SECM (DGP) (%) AIC BIC AIC BIC AIC BIC AIC BIC AIC BIC In large samples of 500, ouliers have a more pronounced effec on asymmeric price ransmission model selecion performance. This is because he effec of ouliers become more pronounced in large samples. Subsequenly, as sample size increases, he number of ouliers in he sample increases. This is because in large samples, he expeced number of ouliers is n imes he percenage of ouliers. This increases he number of ouliers since n is large. For example, he recovery raes of AIC and BIC decreases from 84.6% and 98.4% o 34.4% and 38.7% respecively when oulier percenages in he daa increases from 0 o 5%. Wih 5% oulier conaminaion, boh AIC and BIC performs poorly and fail o idenify he rue asymmeric model in large samples. Similarly, McCann (2006) noed ha sandard model selecion mehods such as AIC and BIC performs poorly when he daa conains ouliers. Under he influence of 5% oulier conaminaion in large samples, AIC seleced he complex asymmeric model agains he rue daa generaing process. Similarly, BIC seleced he simpler asymmeric model agains he rue daa generaing process. BIC ouperforms AIC in selecing he suiable asymmeric price relaionship in large samples wih ouliers. 36

6 Similarly, in a model selecion analysis using poserior probabiliies, Le, Rafery and Marin (1996) noe ha BIC ouperforms AIC for daa wih ouliers. CONCLUSION The sudy examined he abiliy of AIC and BIC o clearly idenify he rue asymmeric daa generaing process in he presence of ouliers in he daa. Generally, he Mone Carlo simulaion resuls indicae ha he abiliy of AIC and BIC o clearly idenify he correc model among compeing models decreases wih increases in ouliers in he daa. Under unsable condiions such as small sample size, he effec of ouliers on he performance of AIC and BIC in selecion of he correc asymmeric model remains unclear. However, in moderae and large samples, here is a persisen decline in he performance of AIC and BIC o recover he rue asymmeric daa generaing process. As he percenage of ouliers increase o 5% oulier conaminaion, boh AIC and BIC fail o idenify he rue asymmeric model in large samples. The comparison provided conribues o knowledge and undersanding of he effecs of ouliers on he relaive performance of AIC and BIC in an asymmeric price ransmission modelling framework. The sudy conribues o he lieraure on asymmeric price ransmission modelling by making researchers aware of he failure of AIC and BIC o selec he correc asymmeric price ransmission model when oulier percenages in daa (large samples) are high. Invesigaion ino he performance of robus modificaions of AIC and BIC in selecion of asymmeric price ransmission models in he presence of ouliers represens a fruiful avenue for fuure research. REFERENCES 1. Akaike, H. (1973). Informaion Theory and an Exension of he Maximum Likelihood Principle. 2 nd Inernaional Symposium on Informaion Theory, Budapes. pp Akaike, H. (1974). A new look a he Saisical Model Idenificaion. IEEE Transacions on Auomaic Conrol, 19: Akinson, A. C., & Riani, M. (2008). A robus and diagnosic informaion crierion for selecing regression models. Journal of he Japanese Saisical Sociey, 38: Barne, V. & Lewis T. (1994). Ouliers in saisical daa, 3 rd ed., John Willey, Chicheser. 5. Chik, Z. (2002). The effec of ouliers on he performance of order selecion crieria for shor ime series. Pakisan Journal of Applied Sciences, 2: Engle, R.F., & Granger, C.W.J. (1987). Co-inegraion and error correcion: Represenaion, esimaion and esing. Economerica, 55: Granger, C.W. J., & Lee, T.H. (1989). Invesigaion of Producion, Sales and Invenory Relaionships using Mulicoinegraion and non-symmeric Error Correcion Models. Journal of Applied Economerics 4: Hoeing, J., Rafery, A. E., & Madigan, D. (1996). A mehod for simulaneous variable selecion and oulier idenificaion in linear regression. Compuaional Saisics and Daa Analysis, 22: Houck, J. P. (1977). An Approach o specifying and esimaing nonreversible Funcions, American Journal of Agriculural Economics, 59: Laud, P. W., & Ibrahim, J. G. (1995). Predicive model selecion. Journal of he Royal Saisical Sociey, Series B, 57: Le, N. D., Rafery, A. E., & Marin, R. D. (1996). Robus Bayesian model selecion for auoregressive processes wih addiive ouliers. Journal of he American Saisical Associaion, 91: McCann, L. (2006). Robus model selecion and oulier deecion in linear regression. Docoral disseraion, Massachuses Insiue of Technology, Cambridge, MA. 13. Ronchei, E. (1997). Robusness aspecs of model choice. Saisica Sinica, 7: Ronchei, E., Field, C., & Blanchard, W. (1997). Robus linear model selecion by crossvalidaion. Journal of he American Saisical Associaion, 92: Schwarz, G. (1978). Esimaing he Dimension of a Model. Annals of Saisics, 6:

7 16. Von Cramon-Taubadel, S., & Loy, J.-P. (1996). Price Asymmery in he inernaional Whea Marke: Commen. Canadian Journal of Agriculural Economics, 44, pp