Department of Econometrics and Business Statistics

Transcription

1 ISSN X Australia Departent of Econoetrics and Business Statistics Applications of Inforation Measures to Assess Convergence in the Central Liit Theore Ranjani Atukorala, Maxwell L. King and Sivagowry Sriananthakuar Deceber 04 Working Paper 9/4

2 APPLICATIONS OF INFORMATION MEASURES TO ASSESS CONVERGENCE IN THE CENTRAL LIMIT THEOREM RANJANI ATUKORALA Statistics & Reporting Unit RMIT University, MAXWELL L. KING Departent of Econoetrics & Business Statistics Monash University SIVAGOWRY SRIANANTHAKUMAR * School of Econoics, Finance and Marketing RMIT University, GPO Box 476 Melbourne Australia 300 Phone: Fax: , Eail: sivagowry.sriananthakuar@rit.edu.au Abstract The Central Liit Theore (CLT) is an iportant result in statistics and econoetrics and econoetricians often rely on the CLT for inference in practice. Even though different conditions apply to different kinds of data, the CLT results are believed to be generally available for a range of situations. This paper illustrates the use of the Kullback-Leibler Inforation (KLI) easure to assess how close an approxiating distribution is to a true distribution in the context of investigating how different population distributions affect convergence in the CLT. For this purpose, three different non-paraetric ethods for estiating the KLI are proposed and investigated. The ain findings of this paper are ) the distribution of the saple eans better approxiates the noral distribution as the saple size increases, as expected, ) for any fixed saple size, the distribution of eans of saples fro skewed distributions converges faster to the noral distribution as the kurtosis increases, 3) at least in the range of values of kurtosis considered, the distribution of eans of sall saples generated fro syetric distributions is well approxiated by the noral distribution, and 4) aong the nonparaetric ethods used, Vasicek s (976) estiator sees to be the best for the purpose of assessing asyptotic approxiations. Based on the results of this paper, recoendations on iniu saple sizes required for an accurate noral approxiation of the true distribution of saple eans are ade. Keywords: Kullback-Leibler Inforation, Central Liit Theore, skewness and kurtosis JEL codes: C, C, C4, C5 * Corresponding author We would like to thank two anonyous referees and Professor Farshid Vahid for their helpful coents.

3 . INTRODUCTION A large part of asyptotic theory is based on the CLT. However, convergence in the CLT is not unifor in the underlying distribution. There are soe distributions for which the noral approxiation to the distribution can be very poor. We can iprove on the noral approxiation using higher order approxiations but that does not always provide good results. When higher order ters in the expansion involve unknown paraeters, the use of estiates for these paraeters can soeties worsen the approxiation error rather than iprove it (Rothenberg, 984). Fro tie to tie, researchers point out probles associated with the CLT. In contrast to textbook advice, the rate at which a sapling distribution of eans converges to a noral distribution depends not only on saple size but also on the shape of the underlying population distribution. The CLT tends to work well when sapling fro distributions with little skew, light tails and no outliers (Little, 03; Wilcox, 003; Wu, 00). Wu (00) in the psychological research context, discovered that saple sizes in excess of 60 can be necessary for a distribution of saple eans to reseble a noral distribution when the population distribution is non-noral and saples are likely to contain outliers. Sith and Wells (006) conducted a siulation study to generate sapling distributions of the ean fro realistic non-noral parent distributions for a range of saple sizes in order to deterine when the distribution of the saple ean is approxiately noral. Their findings suggest that as the skewness and kurtosis of a distribution increase, the CLT will need saple sizes of up to 300 to provide accurate inference. Other studies revealed that standard tests such as z, t and F, can suffer fro very inflated rates of Type error when sapling fro skewed distributions even when the saple sizes are as high as 00 (Bradley, 980; Ott and Longnecker, 00). Wilcox (005) observed that the noral approxiation s quality cannot be ensured for highly skewed distributions in the context of calculating confidence intervals using the noral quantiles even in very oderate sized saples (e.g. 30 or 50). Shilane et al. (00) established that the noral confidence interval significantly under-covers the ean at oderate saple sizes and suggested alternative estiators based upon gaa and chi square approxiations along with tail probability bounds such as Bernstein s inequality. Shilane and Bean (03) proposed another ethod, naely the growth estiator, which provides iproved confidence intervals for the ean of negative binoial rando variables with unknown dispersion. They observed that their growth estiator produces intervals that are longer and ore variable than the noral approxiation. In the censored data context, Hong et al. (008) pointed out that the noral approxiation to confidence interval calculations can be poor when the saple size is not large or there is heavy censoring. In the context of approxiation of the binoial distribution, Chang et al. (008) ade siilar observations.

4 Econoetric textbooks loosely define the CLT as the distribution of the su (or average) of a large nuber of independent, identically distributed variables will be approxiately noral, regardless of the underlying distribution. The question is how large the saple size should be for the noral distribution to provide a good approxiation. Also which distribution fro a class of distributions, causes the slowest convergence in the CLT. These are the iportant questions this paper seeks answers to using the KLI easure. In particular, the KLI is used to find which saple sizes are reasonably good for the noral distribution to be an accurate approxiation to the true distribution of the saple ean. To do so, we use the KLI of the density functions for true distributions of eans of a sequence of rando saples with respect to the asyptotic noral distribution. Using siulation ethods, we generate rando saples fro a range of different underlying population distributions and calculate their saple eans. In particular, Tukey s Labda distribution is used for generating rando nubers with known skewness and kurosis. We also find the axiu value of the KLI aong a range of distributions in order to investigate the slowest convergence in the CLT. For convenience, we use the Lindeberg-Levy CLT which is the siplest and applies to independent and identically distributed rando observations. Only one diensional variables are considered for convenience. The estiated KLI nubers are used to study how large the saple size should be to have an accurate noral approxiation. We also try to find which distributions give poor noral approxiations for a particular fixed saple size using this concept. In suary, this paper investigates four iportant issues; ) how large the saple size should be for the noral distribution to provide a good approxiation to the distribution of the saple ean, ) which distribution fro a class of distributions, causes the slowest convergence in the CLT, 3) which distributions give poor noral approxiations for particular fixed saple sizes and 4) of the nonparaetric ethods used, which sees to be the best for the purpose of assessing asyptotic approxiations. The rest of the paper is planned as follows. Section outlines the theory and the details of estiating the KLI. The design of the Monte Carlo experients including the data generation process is discussed in Section 3. Section 4 reports the Monte Carlo results. Soe concluding rearks are ade in Section 5. 3

5 . THE THEORY. Generating observations fro Tukey s Labda ( ) distribution Our siulation experients used rando drawings fro a generalisation of Tukey s distribution proposed by Raberg and Scheiser (97, 974). The distribution is defined by the percentile function (the inverse of the distribution function) Rp ( ) = p ( p), 0 p, () where p is the percentile value, is a location paraeter, is a scale paraeter and 3 and 4 are shape paraeters. It has the advantage that rando drawings fro this distribution can be ade using (), where p is now a rando drawing fro the unifor distribution on the unit interval. The density function corresponding to () is given by 3 4 f() z = f R( p ) = p ( p) 3 4 () and can be plotted by substituting values of p in () to get z = Rp ( ) and then substituting the sae values of p in () to get the corresponding f() z values. Raberg et al. (979) discuss this distribution and its potential use in soe detail. They also give tables that allow one to choose,, 3 and 4 values that correspond to particular skewness and kurtosis values when the ean is zero and the variance is one. Therefore by an appropriate choice of skewness and kurtosis values, a nuber of distributions can be approxiated by a distribution that has the sae first four oents. These include the unifor, noral, Weibull, beta, gaa, log-noral and Student s t distributions. For exaples of the use of this distribution in econoetric siulation studies see Evans (99), Brooks and King (994) and King and Harris (995).. Estiation of KLI In order to evaluate the quality of an approxiating distribution, we need a convenient way to easure divergence between distributions. One such tool is the KLI easure, introduced by Kullback and Leibler The siultaneous equations (for any ean, variance, skewness and kurtosis values) which can be solved to obtain the corresponding values are also given by Raberg et al. (979). 4

6 (95). Let gx ( ) be the true density function of a q rando vector x and f ( x ) be an approxiating density for x. The KLI easure is defined as: a I( g; f a) = Elog{ g( x) / fa( x )} = a q R log g( x) / f ( x) g( x) dx = g( x)log g( x) dx - ( )log ( ) a R g x f x dx. (3) Its usefulness as a easure of the quality of approxiation coes fro the following properties I( g; f a) 0 for all g and f a. I( g; f a) = 0 if and only if gx ( ) = fa( x ) alost everywhere. R As observed by Renyi (96, 970), the KLI easure can be interpreted as the surprise experienced on average when we believe fa( x ) is the true underlying distribution and we are told it is in fact gx. ( ) The saller the value of I( g; f a) the less the surprise, and the closer we consider the approxiating distribution fa( x ) to be to the true distribution gx. ( ) Also note that I( g; f a) is the expected value of the log of the likelihood ratio which, according to the Neyan-Pearson Lea, provides the best test of H 0 : x ~ gx ( ) against H : x ~ fa( x ). Let x, x,..., x be a siulated iid rando saple in which x i, i =,..,, is an n vector fro either H 0 or H, then the ost powerful test can be based on rejecting H0 for sall values of log g( xi ) / fa( xi ) (4) i which is the standard estiate of I g f = Elog(( ( ) / ( )) ( ; ) a g x f x (5) a fro a siple rando saple of size. In this sense we feel confident in using I( g; f a) as a easure of distance between gx ( ) and f ( x ). For further discussion of the KLI easure, see Kullback (959), a Renyi (96, 970), Vuong (989), Maasoui (993) and White (98, 994). Our ai is to estiate (3) where gx ( ) not known but a siple rando saple of observations fro g can be taken. The negative value of the first ter of (3), 5

7 H( g ) = g( x)log g( x) dx (6) is the continuous version of the entropy of the probability density function gx ( ) 3. When the distribution gx ( ) is known, it is obvious that the KLI easure can be easily estiated via the estiation of the entropy for the known distribution. But when the true distribution of gx ( ) is unknown, nonparaetric estiation ethods are needed to estiate the unknown true distribution or the entropy of the unknown true distribution. A nuber of nonparaetric techniques are available for estiating the entropy of the true distribution, however, we use the Vasicek s (976) estiator because of its reliability (Atukorala, 999; Guo et al., 00), the kernel estiator because of its popularity and siplicity and the Maxiu Entrophy (ME) principle because of its popularity... The use of kernel density estiation (hereafter referred to as M) The kernel estiator is the ost coonly used density estiator. Even though this ethod is not the best to use in all circustances, it is widely used particularly in the univariate case. We use this ethod in estiating the true density function g in equation (3). A nonparaetric estiator of the Shannon entropy defined as in (6) for an absolutely continuous distribution g, is given by ˆ ( ) k where x, x,,, (7) H g = log gx ˆ( ) i i x is a rando saple generated fro g and gx ˆ( ) is the kernel estiate of g (Rosenblatt, 956; Parzen, 96; Ahad and Lin, 976; Rao, 973). Accordingly, an estiator for the first ter in (3) is, Iˆ T ( g ) = log gx ˆ( i). (8) i in which gx ˆ( ) can be calculated as gx ˆ( ) = ( x xj ) k h j h. (9) 3 The entropy easure is nonparaetric since it needs not assue the probability distribution is in any paraetric for. 6

8 Thus the estiation aounts to drawing a siple rando saple, estiating gx ˆ( ) using this saple and then taking a second saple to calculate I ˆ T ( g ). The kernel density function k() and the soothing paraeter h have to be chosen appropriately. The choice of the kernel does not see very iportant to the statistical perforance of the estiation ethod. That is, the shape of the kernel does not significantly influence the final shape of the estiated density because it just deterines the local behaviour (Bolance et al., 0). Therefore, in our study, we use the standard Gaussian density for k(). For the noral kernel, our best choice of the soothing paraeter is 4 h = ˆ.06 /5, (0) where ˆ is the standard error of the observed data and is the nuber of observations in the data set. Then the KLI can be estiated as I ˆk = log gˆ ( x ) log f ( x ) () i N i i i In (), ĝ is the estiated density function of the true distribution of eans, x i = n z j j n, i =,,,, () where n is the size of the saples generated fro Tukey s distribution, for calculating eans as explained in Section. and f is the noral density function with zero ean and variance N n. We also calculated the standard errors of estiated KLI using the square root of the statistic, var( I ˆ ) = k gx ˆ( ) i log Iˆ k i fn( xi). (3).. The use of the Maxiu Entropy (ME) distribution (hereafter referred to as M) Suppose we have a siple rando saple of observations fro an unknown continuous distribution with range, ; say x, x,, x. In the ME approach, the objective is to exploit the knowledge that the parent distribution is continuous in constructing an estiated density function, written h (.). This 4 See Silveran (978, 986). 7

9 function is derived by axiising its entropy subject to certain constraints. Those constraints reflect the knowledge of the parent distribution provided by the saple. Calculating the univariate ME distribution aounts to ordering the saple observations x < x < < x. As given by Theil and Fiebig (984), the two constraints called (i) the ass-preserving constraint and (ii) the ean-preserving constraint have to be iposed in order to calculate the univariate ME distribution. Then, the interediate points between successive order statistics need to be defined as, i = ( i i x, x ), i =,,, (4) where () is a syetric differentiable function of its two arguents whose values are not outside the range defined by these arguents. The ME density function (Theil and Fiebig, 984) is as follows: fi( x ) = i i x x, for i x i, (5) f ( ) x = x 4 x x x exp, x x 4 x for x, (6) exp, 4 f ( x ) = 4 x x x x x x x for x. (7) The ME distribution is obtained by axiising the entropy and the value of that axiu is called the axiu entropy. The value of the axiu entropy is 8

10 H = ME log log x i. (8) + i i x The first ter, ( log ) 0.637, which is called an end-ter correction, results fro the exponential tails. In this paper, we use (8) to estiate the entropy of the true density function involved in (3) 5. This aounts to i i i log x x log f N ( x ) i - I ˆME = log, (9) where f is the noral density function with ean zero and variance N n...3 The use of the Vasicek s entropy (hereafter referred to as M3) When our saple observations are rearranged in the for of order statistics as given by x < x < < x, the entropy estiate introduced by Vasicek (976) can be written as log x x, i (0), i i Hv = where is a positive integer saller than /. If the variance of the underlying distribution is finite, H, converges to H( g ) in (6) as, and / 0. When we use Vasicek s ethod for estiating KLI, we replace the first integral of (3) with inus the estiate H v given by (0). Then the estiate for the KLI can be given as, x x f x i i i i i Î = log log N ( ) () 5 Note that the second ter of (8) is identical to Vasicek s (976) saple entropy which is used to test for norality. 9

11 In (), an appropriate value for has to be chosen. Our approach to choosing is explained in the next section. Because the theoretical variance for these estiators given in (9) and () are coplicated and difficult to derive, we use the nonparaetric bootstrap ethod for estiating the standard errors for these cases (Efron, 979) bootstrap saples were used in our experients. 3. MONTE CARLO EXPERIMENT As explained in Section., data is generated fro a generalisation of Tukey s Labda distribution with = 0 and =. The following grid points for the skewness, kurtosis and n values were used in the Monte Carlo experients. Skewness: 0, 0.5, 0.5, 0.75,.0,.,.5,.7 and.0. Kurtosis: 3, 4, 5, 6, 7, 8, 9, 0,,, 3, 4 and 5. Saple size ( n ): 3, 4, 5, 6, 8, 0,, 4, 6, 8, 0,, 4, 6, 8, 30, 3, and 34. In cases of skewness values of 0, 0.5, 0.75,.,.5,.7 and.0 with kurtosis values of 6, 7, 8, 9, 0 and, saple sizes of 3, 5, 0, 5, 0, 5, 30, 35 and 40 were used. The grid of skewness and kurtosis values given above, is sufficient for our purposes because those cobinations of skewness, kurtosis and saple sizes cover a wide range of values. By generating rando nubers for each cobination of these paraeters, we calculate eans of each saple. To deterine the value of, we estiated our final estiator given in () for a range of different values for until our estiated values are stable. The values of range fro,000 to,000 in steps of,000 in this case. In the case of M, the density functions for true distributions of eans were estiated using these values under the different cobinations of paraeters given above. Estiates of KLI decline as the value of increases for all ethods used in the experients. For different values between 8,000 and,000, there was not uch difference between the KLI 6 Researchers find that bootstrap standard errors perfor better than the conventional asyptotic standard errors in the linear regression context (Goncalves and White, 005). 0

12 estiates. For the M estiator, even fro the point of view of density estiation, this range sees to be reasonably good for to use because it gives sooth density functions for ost of the paraeter cobinations. We found = 0,000 is a reasonably good nuber to use in Monte Carlo experients by considering both the density functions and the KLI estiates. For M3, in addition to the selection of, an appropriate value for in equation () has to be chosen. We know that the true distributions of eans of independent rando saples taken fro the standard noral distribution is noral, so there is no approxiation error and the true value of KLI is zero. Therefore, estiates of KLI in this case with respect to the approxiate noral distribution should be near zero, typically insignificantly different fro zero. Thus, the case of generating saple eans using a distribution with the sae first four oents as the standard noral distribution as the underlying population distribution can be considered as a benchark for coparing results and choosing an optial value. Table lists soe of the results for estiated KLI for different values of when saple sizes are 3, 5 and 0. Most of the estiates for high values such as 70 to 00 are within two standard errors of zero. Based on these results, we selected = 85 as the best value to use in our experients for estiating KLI. 4. MONTE CARLO RESULTS Selected estiated results for the KLI obtained using the three ethods, naely, M, M and M3, for different saple sizes ( n ) and different skewness and kurtosis values are given in Tables to 5. First, we shall consider the results in the case of generating rando observations fro Tukey s Labda distribution with the sae first four oents as the noral distribution as they provide a bench ark for the coparison of Monte Carlo results for different ethods. The corresponding results are given in the first colun of Table. The KLI estiates obtained using M and M3, are very sall and for all the saple sizes are not significantly different fro zero. Copared to M, they also have low standard errors. This iplies that the noral distribution approxiates the true distribution of the saple eans extreely well, as expected. M gives estiates uch higher and uch ore variable than the other two ethods. Alost all these estiated values are significantly different fro zero even in the case of the underlying population distribution s skewness being zero and kurtosis being three which is the case for data generation fro the noral distribution (see Table ). These results iply that divergence between true distributions of saple eans and the asyptotic distribution is high, even when the underlying population distribution is syetric with the sae fourth oent as the standard noral distribution. Even at the highest saple size for these oent values, 30 in our experients, the results behave in a siilar anner. It sees that

13 M clearly produces biased and very variable estiates of KLI. Thus, it is clearly not appropriate to use it for assessing asyptotic approxiations in our settings. One reason for getting these biased and variable estiates of KLI is that the axiu entropy principle provides an extree entropy estiate. It sees that the forula for the value of the axiu entropy given in (8) should not be used as an estiate for the entropy of an unknown distribution in our case. The results obtained using the other two ethods, (M and M3), show that the distributions of the eans of rando saples taken fro syetric distributions with zero skewness and kurtosis of three give KLI values very close to zero (see Table ). This indicates that the noral distribution better approxiates the true distribution of saple eans taken fro such syetric distributions. This is not surprising because the ean of rando saples taken fro (syetric) noral distributions have a noral distribution. If we look at the sall saple results, when M3 is used and the kurtosis of the underlying population distribution increases, the KLI estiates becoe significantly different fro zero at the 5% level of significance. However M does not produce results with a siilar pattern. Only the estiates for kurtosis values of 8, 9 and 0 when skewness is equal to 0.5, are significantly different fro zero at the 5% level (see Table 3). In sall saple sizes such as 3 and 4, we observed that as kurtosis of less skewed underlying population distributions increases, the KLI estiates increase. When sapling is done fro distributions with lower skewness values such as 0, 0.5, and.0, soe of the estiated values were sall negative nubers near zero (see Tables & 3). One of the reasons for this could be sapling errors because all these negative values are insignificantly different fro zero. Thus, these values can be considered as negligible positive values because KLI cannot be negative by definition. Overall, the standard errors show that M3 gives estiates with uch less variation than those of M for all the different paraeter cobinations, with a few exceptions. Thus M3 sees better than M for estiating KLI for our purpose. Consequently, we shall now interpret the results based on M3. According to the Monte Carlo results, the estiated KLI values range between 0 and 0.4 when considering insignificant negative values (only 3 nubers) as zero. Aong the KLI estiates which are significantly different fro zero, the lowest value is whereas the highest value is 0.4. The lowest value occurs for a saple size of 5 when skewness is zero. If we can categorise this range to subranges such as higher or oderate values of to 0.4, and sall values such as values less than 0.004, we can discuss how well the true distribution converges to the noral distribution based on these low and high liits of estiates. The asyptotic noral approxiation sees to be very reasonable for the distributions which have very sall KLI values close to zero (values less than 0.004). The reverse occurs when the KLI values are very high. In order to illustrate how well the true distributions are

14 approxiated by the noral distribution using our estiates of KLI, we can choose a reasonably appropriate value within the range of significant KLI estiates, as a threshold. The next question is, what should the threshold value be? We observed that as the saple size increases, the values of the estiated KLI decline. Thus the lowest saple size, which is 3, gives the highest KLI for all kurtosis values. According to the results, as the kurtosis of the underlying population distribution increases, values of estiated KLI increase iplying that when the underlying population distribution is away fro the noral distribution, KLI increases. For the saple size of 3, the KLI estiates for kurtosis values of 3, 4, 5, 6, 7, 8 and 9 are , , , , 0.045, and 0.09, respectively. Aong these, only the KLI estiates for kurtosis values of 5, 6, 7, 8 and 9 are significantly different fro zero. Aong the significant values, is the one with the id value of kurtosis which give significant KLI estiates. The average of significant KLI estiates is also nearly Therefore sees to be a good choice for the threshold value for KLI estiates. Thus we can use the following rule concerning the distributions of saple eans: KLI estiates < well approxiated by the noral distribution < KLI estiates reasonably approxiated by the noral distribution. KLI > poorly approxiated by the noral distribution. We find there are sall KLI estiates which are less than 0.045, for saple sizes 30 and above for kurtosis value of 9 and 0 when data is generated fro highly skewed distributions (see Table 4). But when data is generated fro distributions with kurtosis of and, the iniu saple sizes required for having low KLI estiates are 6 and 4, respectively. When skewness is, and as the kurtosis of the underlying population distribution increases, the iniu saple size required for a reasonably noral approxiation sees to decrease. For exaple, for kurtosis in the range 9 0, iniu saple size required sees equal to 30 whereas for kurtosis in the range 5, this becoes 4 (see Tables 4 & 5). Based on our results, saple sizes greater than 30 can be recoended for use of the asyptotic noral approxiation in the CLT when sapling fro skewed and leptokurtic or ediu tailed distributions (see Table 4). However, saple sizes less than that also give a relatively good noral approxiation when the population distribution s skewness is less than or equal to one. But as the skewness increases, the possibility of getting a good noral approxiation for a sall saple diinishes 7. 7 For brevity these and the following results are not reported. They are available fro the corresponding author. 3

15 If we look at the behaviour of estiates with changes of kurtosis and skewness values, for leptokurtic distributions with sall skewness values, even saple sizes of 3 0 can be used for a reasonably good noral approxiation. However, the saple ean of rando saples taken fro highly positive skewed distributions (for exaple, skewness of ) does not have a good noral approxiation copared to the others. Thus, for saple sizes such as 3 0, the noral approxiation cannot be recoended when sapling fro such distributions because the divergence between the true distribution and the approxiating noral distribution is coparatively high. When saples are taken fro skewed distributions (for exaple, skewness of.5), saple sizes less than 0 ight give poor noral approxiations to the distributions of saple eans. When sapling is done fro asyetric distributions 8, we clearly see that the KLI values of the true distribution of the saple eans with respect to the noral distribution, decreases and converges to zero as saple sizes increase. The results are justifiable due to the CLT. When a threshold value such as is chosen, then saple sizes higher than or equal to 4, give KLI estiates less than Therefore at least 8 observations should be used for the true distribution to be better approxiated by the noral distribution, when sapling fro an underlying population distribution with skewness of.5. When skewness is, a siilar pattern in KLI estiates can be observed but the iniu saple sizes required for a better noral approxiation is higher. For saple sizes greater than 6-8, alost all the KLI estiates are less than in the case of generating data fro distributions with skewness of. Therefore, it sees that these distributions are reasonably approxiated by the noral distribution for saple sizes greater than 8 for all the kurtosis values used in the experients. Based on the estiated KLI values, Table 6 suarises the iniu saple size needed for the true distribution of the saple ean to be reasonably approxiated by the noral distribution, for particular choices of skewness and kurtosis values. It should be noted that these recoendations are ade on the basis of the distributions used in this study. One should not assue that they extend to all distributions with these particular values of skewness and kurtosis. Obviously the shape of the underlying population distribution influences the rate at which a sapling distribution of eans converges to a noral distribution. 5. CONCLUSION This paper considers three nonparaetric estiators (kernel, axiu entropy principle and Vasicek s entropy) of the KLI easure to investigate how well the true distribution of eans of independent rando saples are approxiated by the approxiating noral distribution in the context of the CLT. For this study, a range of saple sizes were used and the saples were generated fro Tukey s labda distribution with different skewness and kurtosis values. Overall, the Vasicek s entropy perfors better 8 The skewness values.5 used in this paper can be considered as such asyetric cases. 4

16 than the other ethods in ters of estiating KLI for assessing asyptotic approxiations. Based on this best ethod, we investigate how distributions affect convergence in the CLT and find which type of distributions give poor asyptotic approxiations. As expected, the results suggest that the distribution of the saple ean better approxiates the noral distribution as the saple size increases. We have also ade soe recoendations on iniu saple sizes required for an accurate noral approxiation of the true distribution of the saple ean. Our results indicate that the true distribution of the saple ean when the saple is taken fro a highly skewed distribution better approxiates the noral distribution as the thickness of the tail of the population distribution increases. In the range of kurtosis values considered, eans of sall saples generated fro syetric distributions are well approxiated by the noral distribution. REFERENCES Ahad, P. and I. Lin (976). A nonparaetric estiation of the entropy for absolute continuous distributions, IEEE Transactions on Inforation Theory, Atukorala, R. (999). The use of an inforation criterion for assessing asyptotic approxiations in econoetrics, PhD Thesis, Monash University, Melbourne. Bolance, C., Guillen, M., Gustafsson, J. and J.P. Nielsen (0). Quantitative Operational Risk Models, Taylor & Francis Group, LLC. Bradley, J.V. (980). Nonrobustness in z, t and F tests at large saple sizes, Bulletin of the Phychonoic Society 6, Brooks, R.D. and M.L. King (994). Testing Hildreth-Houck against return to noralcy rando regression coefficients, Journal of Quantitative Econoics 0, Chang, C.H., Lin, J.J., Pal, N. and M.C. Chiang (008). A Note on Iproved Approxiation of the Binoial Distribution by the Skew-Noral Distribution, The Aerican Statistician 6, Evans, M.A. (99). Robustness of size of tests of autocorrelation and heteroscedasticity to nonnorality, Journal of Econoetrics 5, 7-4. Efron, B. (979). Bootstrap ethods: Another look at the jackknife, Annals of Statistics 7, -6. Goncalves, S. and H. White (005). Bootstrap standard error estiates for linear regression, Journal of the Aerican Statistical Association 00, Guo, J., Aleayehu, D., and Y. Shao (00). Tests for norality based on entropy divergence, Biopharaceutical Research, Hong, Y., Meeker, W.Q. and L.A. Escobar (008). Avoiding probles with noral approxiation confidence intervals for probabilities, Technoetrics 50, King, M.L. and D.C. Harris (995). The application of the Durbin-Watson test to the dynaic regression odel under noral and non-noral errors, Econoetric Reviews 4,

17 Kullback, S. (959). Inforation Theory and Statistics. New York: John Wiley and Sons. Kullback, S. and R.A. Leibler (95). On inforation and sufficiency, Annals of Matheatical Statistics, Little, T.D. (03). The Oxford Handbook of Quantitative Methods in Psychology: Vol. : Statistical Analysis, Oxford University Press. Maasoui, E. (993). A copendiu to inforation theory in econoics and econoetrics, Econoetric Reviews, Ott, R. and M. Longnecker (00). An Introduction to Statistical Methods and Data Analysis, Cengage Learning, USA. Parzen, E. (96). On estiation of a probability density and ode, Annals of Matheatical Statistics 33, Raberg, J.S., Dudewicz, E.J., Tadikaalla, P.R. and E.F. Mykytka (979). A probability distribution and its uses in fitting data, Technoetrics, 0-4. Raberg, J.S. and B.W. Scheiser (97). An approxiate ethod for generating syetric rando variables, Counication of the Association for Coputing Machinery 5, Raberg, J.S. and B.W. Scheiser (974). An approxiate ethod for generating asyetric rando variables, Counication of the Association for Coputing Machinery 7, Rao, C. (973). Linear Statistical Inference and its Applications, Wiley, New York. Renyi, A. (96). On easures of entropy and inforation, Proceedings of the Fourth Berkeley Syposiu in Matheatical Statistics, University of California Press. Renyi, A. (970). Probability Theory, Asterda: North-Holland. Rosenblatt, M. (956). Rearks on soe nonparaetric estiates for a probability density function, Annals of Matheatical Statistics 7, Rothenberg, T.J. (984). Approxiating the distributions of econoetric estiators and test statistics, Handbook of Econoetrics, Z. Griliches and M.D. Intriligator (eds.), North-Holland, Asterda, Shilane, D. and D. Bean (03). Growth estiators and confidence intervals for the ean of negative binoial rando variables with unknown dispersion, Journal of Probability and Statistics, Volue 03, Article ID Shilane, D., Evans, S.N. and A. Hubbard (00). Confidence intervals for negative binoial rando variables of high dispersion, The International Journal of Biostatistics 6, -9. Silveran, B.W. (978). Choosing the window width when estiating a density, Bioetrika 65, -. Silveran, B.W. (986). Density Estiation for Statistics and Data Analysis, Chapan and Hall, London. 6

18 Sith, Z.R. and C.S. Wells (006, October 8 0). Central Liit Theore and Saple Size, Retrieved fro The Annual Meeting of the Northeastern Educational Research Association: Theil, H. and D.G. Fiebig (984). Exploiting Continuity: Maxiu Entropy Estiation of Continuous Distributions, Ballinger Publishing Copany, Cabridge. Vasicek, O. (976). A test for norality based on saple entropy, Journal of the Royal Statistical Society B 38, Vuong, Q. (989). Likelihood ratio tests for odel selection and non-nested hypotheses, Econoetrica 50, -6. White, H. (98). Maxiu likelihood estiation of isspecified odels, Econoetrica 57, White, H. (994). Estiation, Inference and Specification Analysis, Cabridge University Press, USA. Wilcox, R.R. (003). Applying Conteporary Statistical Techniques, San Diego, CA: Acadeic Press. Wilcox, R.R. (005). Robust estiation and Hypothesis Testing, Elsevier Acadeic Press, Burlington, MA. Wu, P.C. (00). The central liit theore and coparing eans, tried eans, one step -estiators and odified one step -estiators under non-norality, Unpublished onograph, University of Southern California. 7

19 Table : Selected KLI estiates for different values of and associated standard errors when the underlying population distribution has the sae first four oents as the standard noral Saple size (n) KLI s.e KLI s.e KLI s.e

20 Table : Selected estiates of KLI and associated standard errors for different ethods (M, M and M3) and saple sizes (n) when skewness = 0 and kurtosis = 3-6 kurtosis n KLI s.e KLI s.e KLI s.e KLI s.e 3 M M M M M M M M M M M M M M M M M M M M M M M M

21 Table 3: Selected estiates of KLI and associated standard errors for different ethods (M, M and M3) and saple sizes (n) when skewness = 0.5 and kurtosis = 8-0 kurtosis n KLI s.e KLI s.e KLI s.e 3 M M M M M M M M M M M M M M M M M M M M M M M M

22 Table 4: Selected estiates of KLI and associated standard errors for different ethods (M, M and M3) and saple sizes (n) when skewness = and kurtosis = 9- kurtosis n 9 0 KLI s.e KLI s.e KLI s.e KLI s.e 3 M M M M M M M M M M M M M M M M M M M M M M M M

23 Table 5: Selected estiates of KLI and associated standard errors for different ethods (M, M and M3) and saple sizes (n) when skewness = and kurtosis = 3-5 kurtosis n KLI s.e KLI s.e KLI s.e 3 M M M M M M M M M M M M M M M M M M M M M M M M

24 Table 6: Miniu saple size needed for the true distribution of the saple ean to be reasonably approxiated by the noral distribution Kurtosis skewness = 0 skewness = 0.5 skewness = skewness =.5 skewness = Note: As noted in Section 3, not all cobinations of skewness and kurtosis values are estiated, which explains the issing values. 3