THESIS. "Dennis J. AFIT/GSO/MA/ "Approved for public rel7cas. Diatribution Unlimited THE AIR FORCE. iited

Transcription

1 00 CD DTIC _ZLECTE! S A COMPARISON OF ESTIMATION TECHNIQUES FOR THICTHE THREE PARAMETER PARETO DISTRIBUTION THESIS "Dennis J. Major, Charek USAF AFIT/GSO/MA/ "Approved for publi rel7as. Diatribution Unlimited "6. DEPARTMENTOF THE AIR FORCE "' Sl"]! U AIR FORCE INSTITUTE OF iited TECHNOLOGY "2" AIR UNIVERSITY _l_ "Wright-Patterson Air Fore Base, Ohio n:i

2 AFIT/GSO/MA/8SD-3 S S DTIC ELECTEW FEB1j0196 D A COMPARISON OF ESTIMATION TECHNIQUES FOR THE THREE PARAMETER PARETO DISTRIBUTION THESIS Dennis J. Charek Major, USAF AFIT/GSO/MA/8SD-3 Approved for publi release; distribution unlimited

3 AFIT/GSO/MA/8SD-3 A COMPARISON OF ESTIMATION TECHNIQUES FOR THE THREE PARAMETER PARETO DISTRIBUTION THESIS Presented to the Faulty of the Shool of Engineering of the Air Fore Institute of Tehnology Air University In Partial Fulfillment of the Requirements for the Degree of Master of Siene in Spae Operations Aesion For NTIS CRA&I U I IC TAB 5 U annoj: ed 13 Dennis J. Charek, B.S., M.S. By Diý.t ibu~tio / Major, USAF Avawiab,;ity Codes Deember Dist Ava: a. d or Spial Approved for publi release; distribution unlimited I- U -- 2.

4 Ii Prefae The purpose of this 15se-n is to ompare the minimum distane estimation tehnique with the best linear unbiased estimation tehnique to determine whih estimator provides more aurate estimates of the underlying loation and sale parameter values for a given Pareto distribution. Two forms of the Kolmogorov, Anderson-Oarling, and Cramer-von Mises minimum distane estimators are tested. A Monte Carlo methodology is used to generate the Pareto random variates and the resulting estimates. A mean square error omparison is then performed to evaluate whih estimator provides the best results. Additionally, various sample sizes and shape parameters are also used to determine whether they have an influene on a given estimator's performane., I wish to express my sinere appreiation to my advisor, Dr. Albert H. Moore, for his guidane and diretion throughout this thesis projet. I also wish to thank my reader, Lt Col Joseph Coleman, for his advie and omments whih greatly supplemented my efforts during this study. In addition, I would like to thank mydlassmate, Capt James Porter, for his assistane and omntributions to this study. Finally, I am very grateful to my wife, Monia, tolerane, and support throughout this thesis effort. for her love, I also Wish to thank my daughter, Emily, and my son, Brian, for their understanding when playtime was interrupted by homework. Dennis J. Charek,.1

5 Table of Contents Page Prefae... ii List of Figures v List of Tables... vi Abstrat vii I. Introdution..... Speifi Problem... 4 "Researh Question... 5 General Approah... S Sequene of Presentation... 5 II. Estimation Tehniques... 7 Estimation Estimator Properties... 9 Unbiased Estimators... 9 Consistent Estimators Effiient Estimators... 1Z Invariant Estimators z Summary Empirial Distribution Funtion (EDF) Best Linear Unbiased Estimator (BLUE) Minimum Distane (MD) Estimator Kolmogorov Distane Cramer-von Mises Distane..... z Anderson-Darling Distane III. Pareto Distribution..... zz History zz Appliations Soio-eonomi Related Apliations... Z4 "Militarily Related Appliations... zs Pareto Funtion... Z6 Grouping Pareto Distributions By Kind... Z7 Grouping Pareto Distributions By Parameter Number Parameter Estimation Best Linear Unbiased Estimator BLUEs for Shape Greater Than Z "BLUEs for Shape Equal to or Less Than Minimum Distane Estimator *1

6 IV. Monte Carlo Analysis Monte Carlo Method Monte Carlo Steps and Proedures Step 1: Data Generation Step Z: Estimate Computation Step 3: Estimate Comparison V. Results, Analysis and Conlusions Results Analysis Z Conlusions VI. Summary and Reommendations Summary S6 Reommendations Appendix A: Tables of Mean Square Errors Appendix Bf Computer Program for Estimator Comparison B bliography Vita IV

7 List pf Figures "Figure Page 1. Pseudoode for Program BLUMO Z. Sample Table of Mean Square Errors... so -S 4.-.

8 List of Tables Table Page I. Mean Square Error for = 1 and n = II. Mean Square Error for = I and n = III. Mean Square Error for = I and n = IV. Mean Square Error for = I and n = V. Mean Square Error for = I and n = VI. Mean Square Error for = Z and n = VII. Mean Square Error for = Z and n = VIII. Mean Square Error for = Z and n = IX. Mean Square Error for = 2 and n = t X. Mean Square Error for = Z and n = XI. Mean Square Error for 3 and n = XII. Mean Square Error for 3 and n = XIII. Mean Square Error for = 3 and n = IZ XIV. Mean Square Error for = 3 and n = XV. Mean Square Error for = 3 and n XVI. Mean Square Error for = 4 and n = XVII. Mean Square Error for = 4 and n = XVIII. Mean Square Error for = 4 and n = IZ XIX. Mean Square Error for 4 and n = XX. Mean Square Error for = 4 and n = 1B VI

9 AFIT/GSO/MA/8SO-3 Abstrat This investigation ompared the minimum distane estimation tehnique with the best linear unbiased estimation tehnique to determine whih tehnique provided more aurate estimates of the loation and sale parameter values when applied to the three parameter Pdreto distribution. Six distint minimum distane estimators were developed. Of these six, two were based on the Kolmogorov distane, two were based on the Anderson-Darling distane, and two were based on the Cramer-von Mises distane. For a given sample size and Pareto shape parameter, the loation and sale parameters were estimated. Additionally, varying ombinations Of sample sizes (6, 9, 12, 1S, or 18) and shape parameters (1.0, 2.0, 3.0, or 4.0) were tested to investigate the affet of suh hanges. A Monte Carlo methodology was used to generate the 1000 sample sets of Pareto random variates for eah sample size - shape parameter ombination with loation and sale parameters both set to a value of 1. The best linear,nblased estimator and the six minimum distane estimators then provided parameter estimates based on the sample sets. Finally, these estimates were ompared using the mean square error as the evaluation tool. The results of this investigation indiate that the best linear unbiased estimation tehnique provided more aurate estimates of loation and sale for the three parameter Pareto "distribution than did the minimum distane estimation tehniques. Vii

10 A COMPARISON OF ESTIMATION TECHNIQUES FOR THE THREE PARAMETER PARETO DISTRIBUTION I. Introdution Parameter estimation is an important underlying tehnique in statistial analysis. Although the statistiian an perform some analysis intuitively, estimation requires a speifi method. For example, if a statistiian is asked to analyze some sample data, he ould order it in asending order and draw a histogram refleting the ourrene frequeny of values within ertain intervals. Further, from the histogram's shape, he ould guess the underlying population distribution. However, he ould not easily determine the parameters (e.g. mean, standard deviation) of the population. At this point, the statistiian needs a method to estimate the true population parameters from the sample data. The method is alled the estimator, and the approximations based on the sample are the statistis (i.e. the estimates). Mendenhall defines an estimator as "a rule whih speifially states how one may alulate the estimate based upon information ontained in a sample" (23:13). Using these rules, the statistiian an estimate the parameters of a population distribution based on sample data drawn from the populetion. These estimates then summarize the properties of the population for the investigator. One estimation tehnique, alled the best linear unbiased estimator (BLUE), relies on a linear ombination of order statistis (10:26S). Order statistis are a set of variables arranged aording

11 to their magnitudes. For instane, ordering a set of observed random variables (e.g. fastest times in an automobile rae) from smallest to largest results in a set of order statistis (24:229). The best linear unbiased estimator MT an be used to estimate an unknown population parameter (8) where T is only dependent on the values of n independent random variables. In addition, the estimator, T, must be in the set of n S~linear random variables. The estimator must also - display the minimum variane among linear estimators and must be -- unbiased (10:26S-266). In simple terms, unbiased means that on the Ii average, the value of the estimator equals the parameter being estimated (33:197). Therefore, by ombining a set of order statistis in a linear ' " fashion, one an produe estimators for the underlying population parameters. If these estimators also Possess the properties of minimum variane and unblasedne55, then they are alled best linear unbiased estimators. Another parameter estimation tehnique is minimum distane estimation, introdued by Wolfowitz in the as a method whih "in a wide variety of ases, Will furnish super onsistent estimators even when la551al methods... fail to give onsistent estimators" (38:9). A minimum d1stane estimator 15 onsistent if, as the sample size inreases, the probability that the estimate approahes the true value of the parameter also inreases (33:199). estimation tehnique 15 Closely related in The minimum distane theory to the statistial proedure alled goodness Of fit beause a distane measure is the i evaluation riteria for both proedures. In goodness Of fit, one tests to"the sample data to identify Its underlying unknown distrvbution. A

12 K goodness of fit test is "a test designed to ompare the sample obtained with the type of sample one would expet from the hypothesized distribution to see if the hypothesized distribution funtion 'fits' the data in the sample" (8:189). Certain goodness of fit tests are based on a distane measure between the sample and a hypothesized distribution with known population parameters. Minimum distane estimation, however, reverses the goodness of fit approah by assuming a probability distribution type and then finding the values that minimize the distane measure. These values beome the estimates of the population parameters (18:34). Even though the minimum distane estimation tehnique was developed in 1953, researhers have not extensively studied the tehnique until kip reently. Parr and Shuany reported in 1979 that the method yields "strongly onsistent estimators with exellent robustness properties" (27:5) when used to estimate the loation parameter of symmetri distributions (27:5). Robustness of an estimator is its ability to serve as a good estimator even when the distribution assumptions are not stritly followed (27:3). Additionally, several Air Fore Institute of Tehnology (AFIT) students, under the guidane of Dr. Albert H. Moore, have ompleted thesis researh projets by applying the minimum distane estimation tehnique to speifi distributions and omparing this tehnique with other estimation methods. These former students inlude Maj MNeese, working with the generalized exponential power distribution; Capt Daniels, working with the generalized t distribution; Capt Miller, working with the three parameter Weibull distribution; Capt James, working with the three parameter gamma distribution; 2Lt 3

13 Bertrand, working with the four parameter beta distribution; and 2Lt Keffer, working with the three parameter lognormal distribution. Results have generally shown that minimum distane estimators provide better estimates (i.e., estimates loser to the atual population parameters) than the other tehnioues used (4:9). The literature searh reveals that the apabilities of the minimum distane estimation tehnique have not been ompared with those of the best linear unbiased estimator with regard to the Pareto distribution, a distribution of onsiderable value. The Pareto distribution has a variety of uses in the ommerial setor. Johnson and Kotz identify several Pareto distribution analysis areas, inluding ity population distribution, stok prie flutuation, and oil field loation (16:242). In addition to ommerial users, the Air Fore also uses the Pareto distribution in a number of analysis areas: time to failfre of equipment "omponents (9), maintenane servie times (14), nulear fallout partiles' distribution (11), and error lusters in ommuniations iruits (3). In sum, the Pareto distribution proves to be a distribution worthy of further investigation. Use of the minimum distane estimation tehnique applied to the Pareto distribution offers the researher a hane to expand the frontier of knowledge in this "area. SPECIFIC PROBLEM Researhers have not explored the potenti3l of the minimum distane estimation tehnique to improve upon the best linear unbiased estimation tehnique as applied to the Pareto distribution. A omparison of the tehniques in a ontrolled environment is needed to evaluate whih 4

14 tehnique performs better under given irumstanes. The ontrolled environment should speify the sample size and the value of the parameters of the underlying Pareto distribution funtion for eah omparison attempt. RESEARCH QUESTION For speified parameter values and sample sizes, whih estimation tehnique, minimum distane or best linear unbiased, performs better when applied to the Pareto distribution? GENERAL APPROACH Monte Carlo analysis is the analytial method to be used to make the estimation tehnique omparison. Monte Carlo analysis of estimation methods onsilts of three steps. First, one generates random variates fronm a speified Pareto distribution (i.e., a Pareto distribution with known parameters). Seond, the two estimation tehniques are used to obtain parameter estimates based on the random sample data from the first step. Third, the resulting estimates are ompared to Jetermine whih estimation tehnique provided the better parameter estimates "(4:27). The mean square error tehnique an be used to perform this "evaluation (4:31). SEQUENCE OF PRESENTATION This report will proeed with five additional hapters. The seond hapter will disuss the estimation tehniques used in this study while the third hapter will present the Pareto Distribution. The fourth hapter will desribe the Monte Carlo analysis methodology used to make - S

15 the estimation tehnique omparisons. The fifth hapter will present the results and onlusions of the study while the sixth hapter will provide a short summary and some reommendations for future study in this area. 6

16 This hapter will first provide a disussion on estimation in general, some desirable properties of estimators, and the empirial distribution as an estimator of the true distribution. Following this disussion, the two estimation tehniques to be ompared in this thesis will be presented. First the best linear unbiased tehnique will be disussed along with its inherent properties. Then the minimum distane "tehnique will be presented in the three distane measure forms to be used throughout the rest of this study. ESTI OLM "Estimation is part of a larger area of study alled statistial inferene. The statistiian makes inferenes about the state of nature, or the "way things really are" (22:187), based on data gathered from experiments done to disover something about the state of nature (22;187). Lindgren then narrows his disussion of statistial problems to deision problems, eliminating the areas of experimental design and representative data gathering. Some statistial problems, notably in business and industry, are deision problems, in whih the partial information about the state of nature provided by data from experimentation is used as the basis of making an immediate deision (22:188]. Lindgren then desribes the general deision problem as onsisting of "a set or 'spae' A of possible ations that might be taken, the individual 'points' of this spae being the individual ations" (22:188). He finally A is defines estimation problems as "those in whih the ation spae idential with the spae of parameter values that index the family 7

17 of possible states of nature" (22:188). In this ase, states of nature o,.'d be desribed by the distribution funtion family members, eah member being defined through its own set of parameter values. Pritsker desribes the onept of parameter estimation by presenting two supporting definitions. He first defines the 'population' as the set of data points onsisting "of all possible observations of a random variable" (31:46). He then defines a 'sample' as being "only part of these observations" (31:46). A method to summarize a set of data is "to view the data as a sample whih is then used to estimate the parameters of the parent or underlying population" (31:46). Runyon and Haber simply define a parameter as "a summary "." numerial value alulated from a population" (33:4). Liebelt indiates that the estimation problem, defined earlier by Lindgren, is diffiult to solve. In fat, beause there an be many estimates regarding a problem, the solution is not unique. Therefore, the statistiian begins searhing for the 'best' estimate; but, sine the riteria for a 'best' estimate is arbitrary, there annot be an optimal estimate to solve all problems (21: ). "Eah problem may require a different set of optimal riteria; the hoie is always left to the user of estimation theory" (21:136). So, the searh always ontinues for a better estimator. This thesis is a ontinuation of that searh. Before we ontinue by listing and defining some of the agreed upon properties of a good estimator, we must larify the differene between an estimator and an estimate. Mendenhall explains that an estimator is "a rule whih speifially states how one may alulate the estimate based upon information ontained in a sample" (23:13). However, when 8

18 the estimator is used to produe a partiular value based on speified sample data, "the resulting predited value is alled an estimate" (23:13). Wine draws an analogy to desribe the differene. He indiates the distintion between the two is the same as the differene between a funtion, f(x), and the evaluated funtional value, f(). "f(x) is a variable defined in some domain of x, and f() is a onstant orresponding to a speified value of x equal to onstant " (37: ). Before a sample is drawn, we have an estimator. After the sample is drawn, the estimator produes a partiular value whih is an estimate (37:171). ESTIMATOR PROPERTIES The searh for better estimators ontinues; but, what is the riteria for determining a good estimator? Certain properties of estimators have been defined and seem to be reasonable guides for hoosing good estimators, although these riteria annot be fully "justified exept on the basis of intuition" (21:136). This setion will disuss four of these desirable properties. If an estimator is to be used in repeated samplings from the same population, then unbiasedness is a desirable property; otherwise, a biased estimator ould possibly be found whih provides better parameter estimates. Additionally, a good estimator should be onsistent, effiient and invariant. Eah of thele properties will now be desribed in more detail. Unbiased Estimators. The first property a good estimator to be used in repeated samplings from the same population is unbiasedness. "Freeman defines an unbiased estimator as follows: 9

19 We have a population desribed by the density funtion f(x;8), where f is knoun and the value of the parameter is unknown. A random sample x.,x,...,,x is drawn from this population. The statisti t x.,x_,...,x) is an unbiased estimator of the parameter 8 if E(t) = (Z.1) for all n and for any possible value of 8 [i1:zz9]. Wine points out that this definition "requires that the mean of the sampling distribution of any statisti equals the parameter whih the statisti is supposed to estimate" (37:172). In other words, the expeted value o? the statisti t equals the parameter being estimated, where "the expeted value of a random variable x with density funtion f(v) is defined as E(x) v f(v)dv (2.2) (21:85). Freeman defines the term density funtion as "a funtion f(x.) whih is onneted to probability statements on the random variable x by p(x = = f'x.) (Z.3) (10:18). Looking at unbiasedness from a slightly different perspetive, Liebelt says that unbiasedness "is desirable, for it states that in the absene of Measurement error, and unertainty in the estimation proedure, the estimate beomes the true value' (21:137). Freeman adds "a final note onerning unbiased estimators. He indiates that for an estimator to be truly unbiased. Eq (2.1) "is required to hold for all "sample sizes n" (10:229). There are ases when Eq (2.1) roughly holds St10

20 only for very large sample sizes. In these ases, the estimator is merely 'asymptotially unbiased' (10:229). Unbiasedness is an important property for an estimator to have in repeated samplings from the same population. The reason for this Etatement beomes apparent when ane looks at what an happen if an estimator is biased. "Any estimating proess used repeatedly and whih on the average (mean) is not equal to the parameter leads to a sure umulation of error in one diretion" (10:229). To avoid this aumulation of error in one diretion, the statistiian seeks to find and use unbiased estimators. However, in a single estimation situation, unbiasedness may not be desireable. Instead, one ould seek to minimize the mean square error of the estimate whih ould then result in a better estimate. Consistent Estimators. The seond property of a good estimator is that of onsisteny. As the sample size inreases, one would want the risk assoiated with the estimator to derease. "That is, the estimator ought to be better when it is based on twenty observations than when it is based on two observations" (25:172). This supposition portrays the idea of onsisteny. "An estimator is onsistent if for a large sample there is a high probability that the estimator will be near the parameter it is intended to estimate" (5:140). A similar definition expressed by Wine uses the idea of onvergene to define a onsistent estimator. An estimator, t, of the parameter 0 is onsistent if, for any small numbers d and 6, "there exists an integer n"' suh that the probability that [It - 01 < 6] is greater than E1-d] for all n > n' " (37:171). This "definition introdues the idea of onvergene by saying, "given any Z1

21 small 161, we an find a sample size large enough 50 that, for all larger sample sizes, the probability that [t] differs from tha true value 8 [by] more than 6 is as small as we please" (37:171). Therefore, the estimator, t, onverges in probability to 8 (37:171). Consisteny, then, implies that as 5ample sizes inrease, the probability also inreases that the estimator provides estimates whih more losely approximate the true value of the parameier being estimated. Effiient Estimators. The third desirable property of a good estimator is that of effiieny. Effiieny is generally used as a measure to ompare two estimators. The effiieny is the ratio of their mean square errors. Mendenhall and Sheaffer indiate that the mean square error an be written as the summation of the variane and the square of the bias of an estimator (24:267). Sine variane is a measure of the dispersion of the distribution of an estimator about the parameter value, the statistiian seeks an estimator with small variane. By seleting an estimator with the smaller variane, he ensures that his estimates will onverge more rapidly to the true parameter value (32:155). Therefore, "one estimator is said to be more effiient than another when the variability of its sampling distribution is less" (33:198). Invariant Estimators. The final property of a good estimator is that of invariane. Invariane is partiularly desirable when funtional transformations must be made regarding the parameter. As Freeman states: 12

22 "We all a method of estimation invariant under transformation of a parameter if, when the method leads to t as the estimator of 8, the method also leads to g(t) as the estimator of g(e). We an speak of t as an invariant estimator for a ertain lass of transformations g if, when the parameter 8 is transformed by g to g(o), the estimator t is transformed to g(t) [10:233]. If the statistiian is working with an invariant estimator where the estimate of 8 is t, then he an onlude that his estimate for 8 + k is t + k and his estimate for k8 is kt (10:233). Thus, the property of invariane permits the transformation of a parameter to be translated into the transformation of its estimator. Summary. Three desirable properties of an estimator are "onsisteny, effiieny, and invariane. Unbiasedness is desirable when the estimator is used in repeated sampling from the same population. Unblasedness means that, on the average, the estimator equals the parameter being estimated. Consisteny means that as the tample size inreases, the estimator will more losely approximate the true parameter value. Eff:-leny is a omparative measure between estimators where the estimatr with the smaller mean square error is more effiient. Finally, invariane means that if a transformation operation is performed on a parameter, the idential transformation an be performed on the estimator resulting in the transformed estimator beoming a valid estimator for the transformed parameter. Although these properties are desirable, estimators generally do not possess all of these properties. Therefore, the statistiians must find an estimator with the properties needed for their partiular applietions. 13

23 ", EMPIRICAL DISTRIBUTION FUNCTION (EDF) An empirial distribution is a distribution based solely on sample values of a random variable. The empirial distribution an be thought of as an estimation of the true underlying population distribution. The empirial distribution is developed "by observing several values of the random variable and onstruting a graph S(x) that may be used as an estimate of the entire unknown distribution funtion F(x) of the random variable (8:59). Conover defines the empirial distribution as follows: Let X 1, Xe,.., X be a random sample. The empirial distributlon funtion S(x) is a funtion of x, whih equals the fration of X.s that are less than or equal "to x for eah x, -00( x (u [8:69]. "Based on this definition, the graph of the empirial distribution "funtion, S(x), is a step funtion starting at zero. As eah bimple value (ordered from lowest to highest) is enountered, a step of height I/n is entered on the graph. This proedure ontinues until all the sample values have been entered and a height of one has been reahed. "S(x) resembles a distribution funtion in that it is a nondereasing funtion that goes from zero to one in height. However, S(x) is empirially (from a sample) determined and therefore its name" (8:70). The empirial distribution funtion is used a5 an estimator for the population distribution funtion of the random variable (8:70). From the empirial distirbution funtion, one an "ompute the expetation of the empiri random, variable, E(x). We have E(x) = n n i= X, ( = /n) = (1/n) i =l X (. (Z.4) whih is just the sample mean, x (S:137). Eq (2.4) uses the 14

24 disrete random variable form of the expeted value definition. Therefore, assuming the empirial distribution aeptably estimates the population distribution leads to the sample mean being an aeptable estimate for the population mean (5:138). BEST LINEAR UNBIASED ESTIMATOR (BLUIE) Knowing what properties are desirable in an estimator still leaves the statistiian with the problem of developing an estimator. One estimator is alled the best linear unbiased estimation tehnique. As was mentioned in Chapter I, the BLU estimator is based on order statistis, whih is simply an arrangement of random variables in order of magnitude (24:229). A population parameter (B) an be estimated by a statisti (T) whih depends only on the values of n independent random variables, x,,.. xn (10:Z65). The title of this estimator indiates some of the properties that it possesses. Namely, the estimator must be unbiased, 'best', and * linear. As was disussed ealier in this hapter, an unbiased estimator has a bias term equal to zero, and, on the average over many trials, the estimator provides estimates equal to the parameter value. Eq 2.1 states the property mathematially. unbiased, the BLU estimator must be 'best'. In addition to being To be best among unbiased estimators, the estimator must have the minimum mean square error (10:26S). The mean square error is the sum of the variane term and the square of the bias term (24:267). Sine we are dealing with an estimator whih 1s inherently unbiased (i.e., the bias term equals zero), the mean square error simply redues to the variane term. "Therefore, in this ase, best implies minimum variane. Finally, the

25 t ' BLU estlmator must be linear. Linearity demands that we onsider only "estimators whih are linear in the random variables xl,. xn, for it is only in omparison with other esimators within this restrited lass that we an always find estimators [whih are best unbiased]" (10:266). Stated mathematially, the estimator appears as * follows: T -x e.h. +x (2.5),. I where the oeffiients (.) must be determined (10:266). i "In addition to the properties desribed above, the best linear ' unbiased estimator possesses another desirable feature, that of "* invariane. Mood and Graybill indiate that BLU estimators are a subset of least-squares estimators (25:349). Further, they state that, in general, least square estimators do not possess the invariane property. "There is one important ase, however, when the invariant property holds for least-squares estimators, and this is the ase of linear funtions" * (25:350). Therefore, in addition to being unbiased, and possessing minimum variane, the BLU estimator is also invariant. MINIMUM DISTANCE (MD) ESTIMATOR Chapter I presented a partial history and desription of the "minimum distane estimation tehnique. The efforts of Wolfowitz ulminated in his 1957 paper whih refined his work toward "developing the minimum distane method for obtaining strongly onsistent estimators (i.e., estimators whih onverge with probability one)" (39:75). In the paper, he emphasized that his method ould be used with a variety of I 16 distane measuring tehniques (39:75). Additionally, Wolfowitz si.azd

26 "':" that "it is a problem of great interest to deide whih, if any, definition of distane yields estimators preferable in some sense" (39:76). This thesis will in part respond to this hallenge, sine three distane measures will be used in the minimum distane method for omparison against the best linear unbiased estimation method. The three distane measures to be used are the Kolmogorov, the Anderson-Darling, and the Cramer-von Mises disrepany measures. Wolfowitz finally summarizes the minimum distane method as follows: The estimator is hosen to be suh a funtion of the observed hane variables that the d.f. of the observed "hane variables (when the estimator is put in plae of the parameters and distributions being estimated) is losest' to the empiri d.f. of the observed hane variables [39:76]. Sine 19S7, the minimum distane estimation tehnique has been studied by many other statistiians and has been found to display other desirable estimator properties. The tehnique has "been onsidered as a method for deriving robust estimators by Knusel (1969) and Parr and Shuany (1980)" (28:178). Additi.nally, Parr and Shuany indiate that the method yields "strongly onsistent estimators with exellent robustness properties" (27:5) when used to estimate the loation parameter of symmetri distributions (27:S). They define robust estimation as "effiient or nearly effiient (at a model) estimation proedures whih also perform well under moderate deviations from that model" (27:2). They attempt to explain why the minimum distane estimator possesses robustness properties: It may well be inquired as to why an estimator obtained by minimization of a disrepany measure whih is useful for goodness-of-fit purposes (and, hene, in many ases extremely sensitive to outliers or general disrepanies "from the model) should be hoped to possess any desirable robustness' properties. 't turns out that, in most 17

27 r" L ases. while the disrepany measure itself may be fairly sensitive to the presene of outliers, the value whih minimizes the disrepany... is muh less so (27:5-6]. Finally, they state that the Method presents a trade-off between effiieny onsiderations and robustness onsiderations (28:179). In addition to onsisteny, robustness and effiieny, investigators have revealed other attrative features of the minimum distane estimation tehnique. Parr and Shuany indiate that "minimum distane estimators share an invariane property with maximum likelihood estimators... It operates in a manner analogous to maximum likelihood methods in simply seleting a 'best approximating distribution' from those in the model" (27:9). Additionally, Parr states that the method is very easy to implement. "Given a set of data, a parmetri model, and "a distane measure between distribution funtions, all that is needed is an omnibus minimization routine to ompute the estimator" (26: ). Finally, minimum distane estimators provide meaningful results even if the onjetured parametri model is inorret. MD-estimation still provides the best approximation in terms of probability urits with regard to the onjetured distribution (26:1208). "This is a feature not enjoyed by other estimation methods suh as the maximum likelihood" (26:1208). Therefore, MD-estimation an be a very useful tool for the statistiian. The minimum distane estimation tehnique uses a distane measure and, for this reason, is losely linked with ertain goodness-of-flt tests. As explained by Stephens, go,dness-of-fit statistis are "based on a omparison of F(x) with the empirial distribution funtion F (x)" (35:730). In a goodness-of-fit test, one is n interested in fitting an empirial distribution funtion, desribed 18

28 9. :- -:>. earlier, with a fully speified (i.e., with known paramters) distribution funtion. The test for whether the fit is 'good' is normally a measure ol distane oetween the two distribution urves. In ontrast, minimum distane estimation uses a parent distribution family with ertain unknown parameters. The estimates of the unknown parameters are those parameter values whih minimize the distane measure between the empirial distribution and the parent distribution being investigated. The three distanze measures to be used in this study are desribed next. Kolmooorov D.stane. The statisti suggested by Kolmogorov in 1933 is the largest absolute distane between the graphs of the empirial distribution funtion, S(x), and the hypothesized "distribution funtion, F(x.;G) measured in the vertial diretion (8:345). Symbolially, the Kolmogo-ov distane (D) is given by:. D = suplf(x.;1) - S(x)lI i (2.6) whih reads D equals "the supremum, over all x, of the absolute value of the differene F(x ;8) - S(x) " (8:347). Stephens i provides a omputational form for all of the distane measures to be used in this study where he lets z F(x ), i= 1,2,...,n For the Kolmogorov distane, the omputational form is as follows: D max (1(n - z I l-mx(i(n i ) max [z - (i-l)/n] l~i(n i. D = max (0+,D ) (2.7) (35:731). These omputational formulae provide the maximum distane 19

29 betueen the empirial distribution funtion, whih is a step funtion, and the onjetured distribution funtion, Fix.;8). 1 Cramer-von Mises Distane. The Cramer-von Mises statisti is * atually a member of the Cramer-von Mises family of distane measures whi-1 is "based on the squared integral of the differene between the EDF and the distribution tested: ~2 {F00 2(28 S 0(F n(x) -F(x;O)) B(x) dx The funtion... [(x)1... gives a weighting to the squared differene" (34:2). The Cramer-von Mises statisti is produied by setting the weighting funtion equal to one, O(x) = I (34:Z). The omputational form of the Cramer-von Mises statisti is given by Stephens as follows: n LI = lz - (Zi - i)/2n) + (i/12n) i= (Z.9) (35:73i). This formula uses the same symbology as the omputational form of the Kolmogorov distane measure. Anderson-Darling DistAne. The Anderson-Darling distane measure is atually another member of the Cramer-von Mises family. In this ase, however, the weighting fator is l/{u(l - u)} where 0 ( u ( I (27:4). "This weight funtion ounterats the fat that the disrepany [in Eq 2.91 between F (x) and Fix;8) is neessarily n beoming smaller in the tails, sine both approah 0 and 1 at the extremes (34:Z). Therefore, the Anderson-Darling weighting funtion gives "greater importane to observations in the tail than do most of 2S

30 I" "4 'the EDF statistis" (34:Z). Stephens gives the omputational form of the Anderson-Darling statisti as follous: 2n AZ n (2i - 1) In z. + in (I -)] )/n - n -1) in( Z n+l-i (Z. (35:731). Again, this omputational formula uses the same symbology used for the other two distane measures' omputational formulae. 21

31 III. Pareto Distribution P This hapter will first relate the history of the Pareto t distribution. A summary of various soio-eonomi and military appliations will follow this historial perspetive. desription of the Pareto funtion will be presented. Then a detailed Finally, this hapter will desribe the best linear unbiased and the minimum distane estimation tehniques as applied speifially to the Pareto funtion. HISTORY In 1897 Vilfredo Pareto ( ), an Italian-born Swiss professor of eonomis, formulated an empirial law whih bears his name (16:233). Pareto's Law was based on his study of the distribution of inomes in several European ountries during the nineteenth entury. The mathematial results of the study were summarized as follows: N = Ax- (3.1) where N 1s the number of people haveing inomes equal to or greater than inome level x. A and are parameters where is sometimes referred to as Pareto's onstant or the shape parameter (16:233). Pigou summarized Pareto's findings in the following statement: It is shown that, if x signify a given inome and N the number of persons with inomes exeeding x, and if a urve be drawn, of whih the ordinates are "logarithms of x and the absissae logarithms of N, this urve, for all the ountries examined, is approximately a straight line, and is, furthermore, "inlined to the vertial axis at an angle, whih, in -_ no ountry, differs by more than three or four degrees *- from 56%. This means (sine tan 56" = 1.5) that, if the number of inomes greater than x is equal o N, the "number greater than mx is equal to [ N(l/m) 1, z2

32 whatever the value of m may be. Thus the sheme of inome distribution is everywhere the same [29:647). The Pareto premise, then, as dedued from his mathematial findings and stated in eonomi rather than mathematial terms is as follows: Hene, what this thesis amounts to in effet is that, on the one hand, anything that inreases the national dividend must, in general, inrease also the absolute share of the poor, and, on the other hand--and this is the side of it that is relevant here--that it is impossible for the absolute share of the poor to be inreased by any ause whih does not at the same time inrease the national dividend as a whole.. we annot be onfronted with any proposal the adoption of whih would both make the dividend larger and the absolute share of the poor smdller, or 'vie versa' [Z9:6481. Pareto felt, therefore that his law was *universal and inevitable--regardless of taxation and soial and politial onditions" %16:233). Sine the statement of Pareto's Law, several renowned eonomists have refuted the law's sweeping appliability (16:233). In partiular, Pigou identified defets in its statistial basis, arguing that the differenes in inlination of the plotted lines were signifiant. Additionally, he argues that suh a generalization from an empirial study under ertain onditions (ertain avenues of inome suh as inheritane and personal effort) annot justifiably be extended to all soial onditions (29: ). The general defene of "Pareto's Law" as a law of even limited neessity rapidly rumbles. His statistis warrant no inferene as to the effet on distribution of the introdution of any ause that is not already present in approximately equivalent form in at least one of the ommunities--and they are very limited in range--from whih these statistis are drawn. This onsideration is really fatal; and Pareto is driven, in effet, to abandon the Su whole laim... [29: "Additv-nally, Champernowne identifies weaknesses in the Pareto Law. 23

33 ".~'. He indiates that the use of the Pareto onstant as a measure of inome distribution inequality between ommunities suffers from two problems. Firstly, the measure only addresses inome before taxation. Seondly, the measure only applied to inome distributions among the rih and breaks down when applied to those with medium inomes (7:609). Finally, Fisk disusses the value of the Pareto distribution regarding its ability to desribe distributions of inome. He states that the "Pareto urve fits inome distributions at the extremities of the inome range but provides a poor fit over the whole inome range" (12 :171). Therefore, Pareto's Law with regard to inome distributions is no longer highly touted. However, other disiplines have found appliation "-"- of the Pareto distribution to be very useful. APPLICATIONS Soio-eonomi Related Appliations. Although the Pareto distribution was formulated as a refletion of inome distribution, the Pareto distribution has proven to be useful in many other areas of investigation. Johnson and Kotz indiate the Pareto distribution an be useful in desribing many soio-eonomi or naturally ouring quantities. Examples inlude the distributions of ity population sizes, flutuations in the stok market, and the ourrene of natural resoures. The Pareto is useful in these areas beause they often "display statistial distributions with very long right tails (16:242). "Koutrouvelis listed some additional areas where the Pareto distribution had suessfully been used. These areas inlude: business mortality rates, worker migration, property values and inheritane, and 24

34 "servie times in queues (19:7). Johnson and Kotz additionally identified the area of personal inome investigation as an area where the Pareto distribution was appliable (16:242). In 1982, Wong used the Pareto in his analysis of inome. He indiates that many individuals underreport their true inomes to avoid a portion of their tax payments. Wong shows the appliability of the Pareto in refleti ig this underreporting phenomena (40:1). Militarily Related Appliations. In addition to soio-eonomi interests, the Pareto distribution has proven useful in many areas of interest to the military. These areas inlude fallout mass-size distributions, interarrival time distributions, and failure time distributions. This setion will address eah of these areas in turn. "E. C. Freiling onduted a study for the U.S. Naval R.diologial Defense Laboratory onerning a omparison of distribution types for desribing "the size distribution of partile mass in the fallout from land-surfae bursts" (11:1). distribution with the Pareto. In this study, he ompared the lognormal He determined that with the effets of the unertainties playing in the problem, the differenes in desriptive ability of the two distributions were trivial. He indiated that the lognormal "has the estheti advantage of an observationally onfirmed theoretial basis in the ase of airburst debris" (11:12). However, if trunation is required, the Pareto distribution has "the pratial advantage of simplifying further alulations of partile surfae distribution" (11:12). "A Pareto desription of interarrival times has played an important part in two other studies, one involving interarrival times in general 25

35 ", and the seond involving telephone iruit error lustering. Bell, Ahmad, Park and Lui performed the general interarrival time study supported by a grant from the Offie of Naval Researh. They indiate that interarrival time distributions are usually thik-tailed as ompared to Gaussian or Poisson proesses for like distributions. They state that the Pareto an provide a variety of tail thiknesses depending on the value of the shape parameter employed (2:1). In the telephone iruit paper, Berger and Mandelbrot propose a new model to desribe error ourrene on telephone lines. They onlude that the Pareto distribution an well be used to approximate the distribution of inter-error intervals. Finally, the Pareto distribution has proven useful in life testing and replaement poliy situations. Davis and Feldstein show the Pareto as a ompetitor to the Weibull distribution with regard to time to failure of a system sine, "unlike the Weibull, it does not give rise to infinite hazard at the origin nor hazard inreasing without bound" (9:306). Kaminsky and Nelson illustrate the use of the Pareto in developing replaement poliy. The Pareto an be used to predit omponent replaement times based on an aumulation of early failure data (17:145). PARETO FUNCTION The mathematial formulation of Pareto'G Law on inome distribution is shown in Eq (3.1). This law orresponds to the following Pareto probability density funtion as given by Johnson and Kotz: "P(x) Pr[X ) x] (a/x) a)@, )0, x)a (3.2) 26

36 In this equation P(x) gives the probability that inome is equal to or greater than x, while a orresponds to some minimum inome (16:234). The umulative distribution funtion (df) of X resulting from Eq (3.2) gives the following Pareto distribution: F (x) = I - (a/x) a)@, )0, x)a (3.3) x (16:234). During Mandelbrot's investigation onerning the Pareto distribution, he distinguishes between two forms of the Pareto Law: the Strong Law of Pareto and the Weak or Asymptoti form of the Law of Pareto. Mandelbrot's Strong Law of Pareto is of the form shown in Eq (3.3) and is written as follows: - F (x) (x/a) x)a - 1 x(a (3.4) Mandelbrot's Weak or Asymptoti form of the Pareto Law is follows: written as 1 - F (x) ~ (x/a)- as x -- C (3.5) x The Weak form implies that if the log of the left side of the relation is graphed against log x "the resulting urve should be asymptoti to a straight line with slope equal to f-] as x approahes infinity" (16:245). Grougino Pareto Distributions by Kind. There are several versions of the Pareto umulative distribution funtion. Often, these versions are grouped aording to 'kind'. There are three labels used in this type of grouping sheme: Pareto distributions of the first kind, of the 27

37 seond kind, and of the third kind. A distribution of the form shown in Eq (3.3) is referred to as a Pareto distribution of the first kind (16:234). A Pareto distribution of the seond kind is written as follows: F(x) = 1 - K/[(x + C) (3.6) (16:234). This form differs from the Pareto distribution of the first kind through the addition of another quaniity, C, in the denominator of the seond term on the right hand side of the equation. In addition to the two distribution kinds above, Pareto suggested a third law, the distribution of whih Mandelbrot alls a Pareto distribution of the third kind. The mathematial form is as follows: -Fx) = 1- Ek e-hx/x /( C)C 1 (3.7) z (16:234). The Pareto distribution of the third kind degenerates to that of the seond kind when h - 0. Groupina Pareto Distributions by Parameter Number. Perhaps a more understandable method of grouping the various forms of the Pareto distribution funtion is by grouping them aording to the number of parameters the form ontains. However, before desribing these funtions, three basi parameters will be defined. Hastings and Peaok desribe three types of parameters whih always have a physial or geometrial meaning. These three parameters are those of loation (a), sale (b) and shape (). This study will use this symbology when using these parameters. The loation parameter, a, is "the absissa of a loation point (usually the lower or mid point) of 28

38 the range of the variate" (15:20). The sale parameter, b, "determines the sale of measurement of the fratile, x" (15:20). A fratile is a general element within the range of the variate, X (15:5). Finally, the shape parameter,, "determines the shape (in a sense distint from loation and sale) of the distribution funtion (and other funtions) within a family of shapes assoiated with a speified type of variate" (15:20). Using the normal distribution as an example, the mean is the loation parameter beause It speifies a kind of mid point for the distribution. The standard deviation is the sale parameter beause it provides a fratile measurment devie for the distribution. "The normal distribution does not have a shape parameter" (15:20). With this bakground on loation, sale and shape parameters, we an now proeed with the disussion on grouping Pareto distributions aording to the number of parameters ontained in the distribution expression. The most ommonly used form of the Pareto distribution is the two parameter form; however, there is a more general form whih uses all three basi parameters of loation (a), sale (b), and shape (). This setion will present this more general form and show how the simpler forms are derived from it. The three parameter form of the Pareto distribution is written as follows: F(x) I - [I + (x-a)/b] x)a (3.8) where b)o and a)@ (20:Z18). As stated earlier, the notation of Hastings and Peaok is used in this equation and in those that follow. "The two parameter Pareto distribution is the most ommon form of the distribution and is derived from Eq (3.8) by eliminating either the Z9

39 loation or the sale parameter from the equation. One way to obtain a two parameter distribution funtion is to set the loation parameter equal to zero. For a=@ we obtain a Pareto distribution of the seond kind as shown in Eq (3.6) where K=b and C=b. This speial ase is sometimes referred to as the Lomax distribution (Z1:Z18). method of effetively eliminating one of the parameters is Another to set the loation parameter equal to the sale parameter. Setting a=b in Eq (3.8) results in the usual formulation of the Pareto distribution and is the Pareto distribution of the first kind as shown in Eq (3.3). The simplest form of the Pareto distribution is the one parameter version whih an be obtained by setting both the loation and the sale parameter equal to one. Setting a=b=l in Eq (3.8), the foliowing ditribution funtion results: -S. F(x) = 1 - x~l (3.9) This one parameter form 1s regarded as the 'standard form' of the Pareto distribution (16:240). Sine most of the many versions of the Pareto distribution an be derived from the more general three parameter model, this thesis investigates the three parameter distribution. This should ensure that results of this study an be used in a wider variety of appliations where estimation is required. PARAMETER ESTIMATION This setion desribes the estimation methods used in this study as applied speifially to the Pareto distribution. First the best linear unbiased estimators are presented along with the proedure used to 30