BIAS-CORRECTED MAXIMUM LIKELIHOOD ESTIMATION OF THE PARAMETERS OF THE WEIGHTED LINDLEY DISTRIBUTION

Transcription

1 Michiga Techological Uiversity Digital Michiga Tech Dissertatios, Master's Theses ad Master's Reports - Ope Dissertatios, Master's Theses ad Master's Reports 2015 BIAS-CORRECTED MAXIMUM LIKELIHOOD ESTIMATION OF THE PARAMETERS OF THE WEIGHTED LINDLEY DISTRIBUTION Wetao Wag Michiga Techological Uiversity Copyright 2015 Wetao Wag Recommeded Citatio Wag, Wetao, "BIAS-CORRECTED MAXIMUM LIKELIHOOD ESTIMATION OF THE PARAMETERS OF THE WEIGHTED LINDLEY DISTRIBUTION", Master's report, Michiga Techological Uiversity, Follow this ad additioal works at: Part of the Mathematics Commos

2 BIAS-CORRECTED MAXIMUM LIKELIHOOD ESTIMATION OF THE PARAMETERS OF THE WEIGHTED LINDLEY DISTRIBUTION By Wetao Wag A REPORT Submitted i partial fulfillmet of the requiremets for the degree of MASTER OF SCIENCE I Mathematical Scieces MICHIGAN TECHNOLOGICAL UNIVERSITY Wetao Wag

3

4 This report has bee approved i partial fulfillmet of the requiremets for the Degree of MASTER OF SCIENCE i Mathematical Scieces. Departmet of Mathematical Scieces Advisor: Dr. Mi Wag Committee Member: Dr. Jigfeg Jiag Committee Member: Dr. Qiuyig Sha Departmet Chair: Dr. Mark S. Gockebach

5

6 Cotets List of Figures vii List of Tables ix Ackowledgmets xi Abstract xiii 1 Itroductio The iverse Lidley distributio Itroductio The maximum likelihood estimatio Bias-corrected Simulatio studies Cocludig remarks Bias-corrected maximum likelihood estimatio of the parameters of the weighted Lidley distributio Itroductio v

7 3.2 Maximum likelihood estimatio Bias-corrected s A corrective approach A bootstrap approach Simulatio studies Real data examples Cocludig remarks Cocludig remarks ad future work Refereces vi

8 List of Figures 2.1 Pdf of IL distributio with differet values of θ Average bias of the cosidered estimate of θ versus for θ = {0.1, 0.5, 1, 1.5, 7.5, 15} RMSE of the cosidered estimate of θ = {0.1, 0.5, 1, 1.5, 7.5, 15} Average bias of the cosidered estimate of θ Average bias of the cosidered estimate of c RMSE of the cosidered estimate of θ RMSE of the cosidered estimate of c Estimated fitted desity fuctios of the lifetime failure of a electroic device for Example Estimated fitted desity fuctios of the failure stresses (i GPa) of 65 sigle carbo fibers of legth 50mm for Example vii

9

10 List of Tables 3.1 The time to failure of 18 electroic devices Poit estimates of θ ad c for Example The failure stresses (i GPa) of 65 sigle carbo fibers of legth 50mm Poit estimates of θ ad c for Example ix

11

12 Ackowledgmets First ad foremost I offer my sicerest gratitude to my supervisor, Dr. Mi Wag, who has supported me thoughout my report with his patiece ad kowledge whilst allowig me the room to work i my ow way. I attribute the level of my Master s degree to his ecouragemet ad effort ad without him this report, too, would ot have bee completed or writte. Oe simply could ot wish for a better or friedlier supervisor. Secod, I would like to thak to all those who helped me i learig ad livig durig my time here. Fially, I thak to my parets ad my wife. I ca ot do this without their support. Chapter 3 has bee accepted for publicatio by Commuicatios i Statistics - Computatio ad Simulatio. I am very grateful for the editors. xi

13

14 Abstract This report discusses the calculatio of aalytic secod-order bias techiques for the maximum likelihood estimates (for short, s) of the ukow parameters of the distributio i quality ad reliability aalysis. It is well-kow that the s are widely used to estimate the ukow parameters of the probability distributios due to their various desirable properties; for example, the s are asymptotically ubiased, cosistet, ad asymptotically ormal. However, may of these properties deped o a extremely large sample sizes. Those properties, such as ubiasedess, may ot be valid for small or eve moderate sample sizes, which are more practical i real data applicatios. Therefore, some bias-corrected techiques for the s are desired i practice, especially whe the sample size is small. Two commoly used popular techiques to reduce the bias of the s, are prevetive ad corrective approaches. They both ca reduce the bias of the s to order O( 2 ), whereas the prevetive approach does ot have a explicit closedform expressio. Cosequetly, we maily focus o the corrective approach i this report. To illustrate the importace of the bias-correctio i practice, we apply the bias-corrected method to two popular lifetime distributios: the iverse Lidley distributio ad the weighted Lidley distributio. Numerical studies based o the xiii

15 two distributios show that the cosidered bias-corrected techique is highly recommeded over other commoly used estimators without bias-correctio. Therefore, special attetio should be paid whe we estimate the ukow parameters of the probability distributios uder the sceario i which the sample size is small or moderate. xiv

16 Chapter 1 Itroductio I recet years, umerous distributios have bee developed i the literature. The mai motivatio of developig the ew distributio is that researchers wat to provide a better model to aalyze the real data from differet research areas. However, i may cases, the poor performace of the distributio is due to iaccurate estimates of the ukow parameters, ot its ier properties. It is well-kow that the maximum-likelihood estimator () is the most popular oe for estimatig the ukow parameter, due to its good properties. However, the is biased i fiite sample space. Such bias may sigificatly affect the fitess of the distributio. This observatio motivates us to adopt some bias-corrected techique to reduce the bias of the from order O( 1 )toordero( 2 ). 1

17 I Chapter 2, we cosider the oe-parameter iverse Lidley distributio (shortly, IL), which is applicable of modelig the upside-dow bathtub shape data. We firstly estimate the ukow parameter based o the. The we adopt a corrective approach to derive the modified that is bias-free to the secod order. As compariso, a alterative bias-correctio mechaism based o the parametric bootstrap is cosidered i this chapter. I Chapter 3, we focus o the two-parameter weighted Lidley distributio. This distributio is useful for modelig survival data with differet shapes, whereas its s are biased i fiite samples. This motivates us to costruct early ubiased estimators for the ukow parameters. We cosider a corrective approach to derive modified s that are bias-free to secod order. I additio, we adopt a alterative bias-correctio mechaism based o the parametric bootstrap. Mote Carlo simulatios are coducted to compare the performace betwee the proposed ad two previous methods i the literature. The umerical evidece shows that the biascorrected estimators are extremely accurate eve for very small sample sizes ad are superior tha the previous estimators i terms of biases ad root mea squared errors. Fially, applicatios to two real data sets are preseted for illustrative purposes. I Chapter 4, We preset our coclusios ad discuss some future research. Due to the importace of the bias-correctio for the s illustrated above, we should pay special attetio o estimatig the ukow parameters of the lifetime distributios. 2

18 It is oteworthy that the cosidered bias-corrected techique ca be easily applied to other commoly used lifetime distributios, such as the weighted expoetial distributio ad the three-parameter Lidley geometric distributio, which are curretly uder ivestigatio ad will be reported elsewhere. 3

19

20 Chapter 2 The iverse Lidley distributio 2.1 Itroductio The Lidley distributio was origially itroduced by Lidley [1] i the cotext of Batesia statistics as a couter example of fiducial statistics. Its probability desity fuctio (pdf) is give by f(t; θ) = θ2 θ +1 (1 + t)e θt, t > 0, where the parameter θ > 0. It has bee discussed by may authors i differet practical cases, such as Bayesia estimatio [2], loadig-sharig system mode [3] ad stress-stregth reliability model [4]. It deserves metioig that the Lidley 5

21 distributio provides a flexible shape to model the lifetime data. However, the Lidley distributio may perform poorly for fittig the o-mootoe shapes data. This motivates the researchers to develop a modified Lidley distributio discussed as follows. The iverse Lidley (for short, IL) distributio was origially proposed by [5]. The radom variable X is said to follow the IL distributio with the parameter θ, deoted by X IL(θ). Its pdf ca be writte as f(x; θ) = θ2 1+θ ( 1+x x 3 ) ( exp θ ), x > 0, (2.1) x where the parameter θ>0. The correspodig cumulative distributio fuctio (cdf) of the IL distributio is give by F (x; θ) = ( 1+ θ ) ( 1 exp θ ), x > 0. (2.2) 1+θx x Figure 2.1 shows differet shapes of the pdf of the IL distributio with differet values of θ. It ca be see from figure that the shape of the IL distributio ca be upsidedow bathtub, right skewed ad heavy-tailed. The flexibility of the shape is very useful to model the survival data i practice. 6

22 Desity θ=0.5 θ=1 θ=2 θ= x Figure 2.1: Pdf of IL distributio with differet values of θ. 7

23 2.2 The maximum likelihood estimatio Suppose that X 1,X 2,,X are observatios of idepedet uits take from the IL distributio. The log likelihood fuctio of θ is give by l(θ; x) =2 log(θ) log(θ +1)+ log(1 + x i ) 3 log(x i ) θ i=1 i=1 i=1 1 x i, (2.3) where x =(x 1,x 2,,x ). The score fuctio is give by l θ = 2 θ 1+θ i=1 1 x i. Defie x = i=1 x i/. Theofθ, deoted by ˆθ, ca be easily obtaied. Simple algebra shows that, ˆθ = 1 x x + x 2. (2.4) 2 x Sice the is biased to order O( 1 ) i fiite samples, we adopt a corrective approach to reduce the bias of to order O( 2 ). 8

24 2.3 Bias-corrected Let l(τ) be the log-likelihood fuctio with a p-dimesioal vector of ukow parameters τ =(τ 1,,τ p ) based o a sample of observatios. The joit cumulates of the derivatives of l(τ) aregiveby [ 2 l κ ij =IE τ i τ j [ 3 l κ ijl =IE τ i τ j τ l [( )( 2 l l κ ij,l =IE τ i τ j τ l ], for i, j =1, 2,,p, (2.5) ], for i, j, l =1, 2,,p, (2.6) )], for i, j, l =1, 2,,p, (2.7) κ l ij = κ ij τ l, for i, j, l =1, 2,,p, (2.8) respectively. It is assumed that the log-likelihood fuctio is well behaved ad regular with respect to all derivatives up to ad icludig the third order ad that all of the four equatios give by (2.5) (2.8) are of order O(). Let K =[ κ ij ] deote the Fisher s iformatio matrix of τ for i, j =1, 2,,p. [6] show that whe the sample data are idepedet but ot ecessarily idetically distributed, the bias of the sth elemet of ˆτ s ca be writte as Bias(ˆτ s )= p p p [ 1 ] κ si κ jl 2 κ ijl + κ ij,l + O( 2 ), s =1, 2,,p, (2.9) i=1 j=1 l=1 9

25 where κ ij is the (i, j)th elemet of the iverse of Fisher s iformatio matrix. Thereafter, [7] show that whe all equatios i (2.5) (2.8) are of order O(), equatio (2.9) still holds eve if observatios are ot idepedet. They thus advocate the followig coveiet form Bias(ˆτ s )= p i=1 κ si p j=1 p l=1 [ κ (l) ij 1 2 κ ijl + κ ij,l ] + O( 2 ), s =1, 2,,p, (2.10) istead of equatio (2.9). Defie a (l) ij = κ (l) ij 1 2 κ ijl for i, j, l =1, 2,,p. They also show that the O( 2 ) bias expressio of ˆτ ca be reexpressed as Bias(ˆτ) =K 1 A vec(k 1 )+O( 2 ), where vec is a operator that creates a colum vector from a matrix by stackig the colum vectors below oe aother, ad A = [ A (1) A (2) A (p)] with A (l) = [ a (l) ] ij. A bias-corrected for τ, deoted by ˆτ, ca thus be costructed as ˆτ =ˆτ ˆK 1 Â vec( ˆK 1 ), where ˆτ is the of the ukow parameter τ, ˆK = K τ=ˆτ,adâ = A τ=ˆτ. It ca be show that the bias of ˆτ will be of order O( 2 ). 10

26 For our problem, we have the case of p =1,thatis,τ = θ. The derivatives of the log-likelihood fuctio of θ ca be easily obtaied as follows. 2 l θ = 2 2 θ + 2 (1 + θ), 2 3 l θ = 4 3 θ 2 3 (1 + θ). (2.11) 3 I additio, we have [ ] 2 K = θ, 2 (1 + θ) 2 k (1) 11 = k 111 = 4 θ 2 3 (1 + θ), (2.12) 3 A = a (1) 11 = 2 θ 3 (1 + θ). 3 The bias-corrected estimator of the for the IL distributio ca be obtaied as ˆθ = ˆθ (ˆθ 3 +6ˆθ 2 +6ˆθ +2)(ˆθ +1)ˆθ (ˆθ 2 +4ˆθ +2) 2. (2.13) Note that, the bias-corrected estimator, ˆθ has a simple closed-form expressio. So it is easily computed. It should be oted that ˆθ is a bias-corrected of θ to order O( 1 ) ad that its bias is of order O( 2 ), because IE [ˆθ ] = θ + O( 2 ). 11

27 2.4 Simulatio studies I this sectio, we coduct Mote Carlo simulatios to compare the performace of the, bias-corrected, ad bootstrap estimator. Let ˆθ, ˆθ, ˆθ stad for the, bias-corrected, ad bootstrap estimator. Geerate the data from IL distributio by the followig algorithm: step 1. Geerate U Uiform (0, 1), step 2. Geerate V 1/Expoetial (θ), step 3. Geerate W 1/Gamma (shape =2,scale=1/θ), step 4. If U θ/(θ + 1), the X = V, otherwise, let X = W. We draw radom samples of size = {8, 11, 14,, 125}, with the parameter θ = {0.1, 0.5, 1, 5, 7.5, 15}. The replicatios of simulatio studies are based o M =20, 000, ad the replicatios of bootstrap are B =5, 000. We calculate the average bias of each estimator ad its root mea squared error(rmse), give by Bias(ˆθ est )= 1 M M i=1 (ˆθest i θ ) ad RMSE(ˆθ est )= 1 M M i=1 (ˆθest i θ ) 2, where ˆθ est is a estimator of the parameter θ. Figure 2.2 depicts the bias versus the sample size for a certai value of θ. Figure 2.3 represets the RMSE versus the sample size for a fixed value of θ. Some coclusios from the two figures ca be draw as follows. 12

28 (i) The of θ is positively biased, idicatig that the o average overshoots the target parameters θ, ad the average of the gets decreasig as the sample size is icreasig. (ii) The of θ clearly outperforms the uder the same sceario above ad these corrected estimators provide substatial bias-correctio, especially for the small or moderate sample sizes. (iii) The bias of the is icreasig, whe the parameter θ gets larger, as show i Figure 2.2. Whe the sample size gets larger, the bias ad RMSE of each estimator decrease ad the magitude of reductio becomes smaller. This is expected because most estimators i statistical theory perform better with icreasig. Therefore, oe ca oly expect that the performaces of all the estimators become closer with icreasig i terms of biases ad RMSEs. (iv) The reductios i biases ad RMSEs of each estimator are very substatial eve for small sample sizes. For istace, whe =9,θ =0.1, Bias(ˆθ )= , Bias(ˆθ )= , Bias(ˆθ )= , RMSE(ˆθ )= , RMSE(ˆθ )= , RMSE(ˆθ )=

29 θ=0.1 θ=0.5 Bias Bias θ=1 θ=5 Bias Bias θ=7.5 θ=15 Bias Bias Figure 2.2: Average bias of the cosidered estimate of θ versus for θ = {0.1, 0.5, 1, 1.5, 7.5, 15}. 14

30 θ=0.1 θ=0.5 RMSE RMSE θ=1 θ=5 RMSE RMSE θ=7.5 θ=15 RMSE RMSE Figure 2.3: RMSE of the cosidered estimate of θ = {0.1, 0.5, 1, 1.5, 7.5, 15}. 15

31 2.5 Cocludig remarks I this chapter, we have studied the for the ukow parameter of the iverse IL, which is positively biased i fiite samples. We have proposed the bias-corrected estimator, the, which reduces the bias of the from order O( 1 )toorder O( 2 ). Numerical evidece shows that the bias-corrected estimator is strogly recommeded over other commoly used estimators without bias-correctio, especially whe the sample size is small or moderate. 16

32 Chapter 3 Bias-corrected maximum likelihood estimatio of the parameters of the weighted Lidley distributio 3.1 Itroductio The Lidley distributio was origially itroduced by [1] i the cotext of Batesia statistics as a couterexample of fiducial statistics. Its probability desity fuctio (pdf) is give by f(t; θ) = θ2 θ +1 (1 + t)e θt, t > 0, 17

33 where the parameter θ>0. Sice the distributio was proposed, it has bee overlooked i the literature partly due to the popularity of the expoetial distributio i the cotext of reliability aalysis. Noetheless, it has recetly received cosiderable attetio as a lifetime model to aalyze survival data i the competig risks aalysis ad stress-stregth reliability studies; see, for example, [8], [9], [10], [11], [12], amog others. [8] provide a ice overview of various statistical properties of the Lidley distributio. Furthermore, they argue that the Lidley distributio could be a better lifetime model tha the expoetial distributio usig a real data set. I a recet paper, [13] itroduce the two-parameter Weighted Lidley (shortly LW) distributio as follows. The radom variable X is said to follow the WL distributio with parameters θ ad c, deoted by X WL(θ, c), if its pdf ca be writte as f(x; θ, c) = θ c+1 (θ + c)γ(c) xc 1 (1 + x)e θx, x > 0, (3.1) where the parameters θ>0, c>0, ad Γ(c) = 0 y c 1 e y dy, c > 0, is the complete gamma fuctio. The WL distributio ca be viewed as a mixture 18

34 of two gamma distributios: oe with shape parameter c ad scale parameter θ, deoted by Gamma(c, θ), the other with shape parameter c + 1 ad scale parameter θ, Gamma(c +1,θ). This property ca be used to geerate radom samples from the WL distributio. The correspodig cumulative distributio fuctio (cdf) of the WL distributio is give by F (x; θ, c) =1 (θ + c)γ(c, θx)+(θx)c e θx, x > 0, θ, c > 0, (3.2) (θ + c)γ(c) ad the hazard rate fuctio of the WL distributio is give by h(x; θ, c) = θ c+1 x c 1 (1 + x)e θx (θ + c)γ(c, θx)+(θx) c e θx, x > 0, θ, c > 0, where Γ(a, b) = b x a 1 e x dx, a > 0, b 0, is the upper icomplete gamma fuctio. It is widely kow that the maximum likelihood method is ofte adopted to estimate the ukow parameters of a statistical model because the maximum likelihood estimators (s) have may appealig properties; for example, they are asymptotically ubiased, cosistet, ad asymptotically ormally distributed, etc. It should be oted 19

35 that most of those properties heavily rely o the large sample size coditio, which idicates that they, such as ubiasedess, may ot be valid for a small or eve moderate sample size; see [14]. As show by [13], the s of the WL distributio are positively biased o average i fiite samples, i.e. the expected value of the estimators exceeds the true value of the parameters. Later o, besides the maximum likelihood method, [15] cosider other estimatio methods, such as the method of momets estimatio (MME), ordiary least-squares estimatio (OLSE), ad weighted leastsquares estimatio (WLSE) methods. The umerical evidece they preset shows that all of the estimators uder cosideratio are positively biased i fiite samples. For this reaso, it has become stadard practice to develop early ubiased estimators for the WL distributio. To the best of our kowledge, such bias-corrected estimators have ot yet bee fully explored for the WL distributio i the literature. I this paper, we adopt a corrective approach to derive modified s that are bias-free to secod order. Here, the corrective approach meas that the bias-correctio ca be achieved by subtractig the bias (estimated at the of the parameter) from the origial. As ca be see i the simulatio study, the proposed estimators are extremely accurate eve for very small sample sizes ad are far superior tha the previous estimators i terms of biases ad root mea squared errors. Additioally, they have simple closed-form expressios, which meas they are quite attractive because they are easy to compute for practitioers. Ideed, such a bias-correctio techique has bee applied successfully for parameter estimatio i other distributios; see, for 20

36 example, [16], [17], [18], [19], [20], [21], ad refereces cited therei. As a alterative to the aalytically bias-corrected s metioed above, we cosider the bias-corrected s through Efro s bootstrap resamplig because the bootstrap estimator is also secod-order correct. Note that the bootstrap estimator does ot require aalytical derivatio of the bias fuctio ad that the bias-correctio is performed umerically. We here refer the iterested readers to [22], [23], [24], [25], to ame just a few. It deserves metioig that aother aalytically bias-corrected s ca be developed based o a prevetive approach itroduced by [26]. This approach ca also reduce the bias of the s to order O( 2 ), whereas it ivolves modifyig the score vector of the log-likelihood fuctio prior to solvig for the s, ad thus, this approach is ot simply attempted i this paper. The remaider of this paper is orgaized as follows. I Sectio 3.2, we briefly discuss poit estimatio by the maximum likelihood method for the WL distributio. I Sectio 3.3, we adopt a corrective approach to derive modified s that are biasfree to secod order. I additio, a alterative bias-correctio mechaism based o Efro s bootstrap resamplig is also cosidered. I Sectio 3.4, Mote Carlo simulatios are coducted to compare the performace betwee the proposed ad two previous methods; ad MME. I Sectio 3.5, applicatios to two real data sets are preseted for illustrative purposes. Fially, Sectio 3.6 cocludes the paper. 21

37 3.2 Maximum likelihood estimatio Suppose that X 1,X 2,,X are observatios of idepedet uits take from the WL distributio. The log-likelihood fuctio of θ ad c is give by l(θ, c) = [ (c + 1) log(θ) log ( Γ(c) ) log(θ + c) ] +(c 1) log(x i ) + log(1 + x i ) θ x i. (3.3) i=1 i=1 i=1 The score fuctios are thus give by [ l c +1 (θ, c) = 1 ] x i, θ θ θ + c i=1 [ l (θ, c) = log(θ) 1 ] c θ + c ψ(c) + log(x i ), i=1 where ψ(x) =(d/dc)logγ(c) is the digamma fuctio. The s ˆθ ad ĉ of the ukow parameters θ ad c ca be easily obtaied by puttig the two equatios above equal to 0. [13] show that the s of θ ad c are, respectively, give by ˆθ = ĉ( x 1) + [ĉ( x 1) ] 2 +4ĉ(ĉ +1) x 2 x η(ĉ), say, (3.4) 22

38 where x is the sample mea ad ĉ is the solutio of the oliear equatio [ log ( η(c) ) ] 1 η(c)+c ψ(c) + log(x i )=0. (3.5) i=1 3.3 Bias-corrected s A corrective approach For ease of expositio ad without loss of geerality, let l(τ) be the log-likelihood fuctio with a p-dimesioal vector of ukow parameters τ =(τ 1,,τ p ) based o a sample of observatios. The joit cumulats of the derivatives of l(τ) aregive by [ 2 l κ ij =IE τ i τ j [ 3 l κ ijl =IE τ i τ j τ l [( )( 2 l l κ ij,l =IE τ i τ j τ l ], for i, j =1, 2,,p, (3.6) ], for i, j, l =1, 2,,p, (3.7) )], for i, j, l =1, 2,,p, (3.8) κ l ij = κ ij τ l, for i, j, l =1, 2,,p, (3.9) 23

39 respectively. It is assumed that the log-likelihood fuctio is well behaved ad regular with respect to all derivatives up to ad icludig the third order ad that all of the four equatios give by (3.6) (3.9) are of order O(). Let K =[ κ ij ] deote the Fisher s iformatio matrix of τ for i, j =1, 2,,p. [6] show that whe the sample data are idepedet but ot ecessarily idetically distributed, the bias of the sth elemet of ˆτ s ca be writte as Bias(ˆτ s )= p p p [ 1 ] κ si κ jl 2 κ ijl + κ ij,l + O( 2 ), s =1, 2,,p, (3.10) i=1 j=1 l=1 where κ ij is the (i, j)th elemet of the iverse of Fisher s iformatio matrix. Thereafter, [7] show that whe all equatios i (3.6) (3.9) are of order O(), equatio (3.10) still holds eve if observatios are ot idepedet. They thus advocate the followig coveiet form Bias(ˆτ s )= p i=1 κ si p j=1 p l=1 [ κ (l) ij 1 2 κ ijl + κ ij,l ] + O( 2 ), s =1, 2,,p, (3.11) istead of equatio (3.10). Defie a (l) ij = κ(l) ij 1 2 κ ijl for i, j, l =1, 2,,p. They also show that the O( 2 ) bias expressio of ˆτ ca be reexpressed as Bias(ˆτ) =K 1 A vec(k 1 )+O( 2 ), where vec is a operator that creates a colum vector from a matrix by stackig the 24

40 colum vectors below oe aother, ad A = [ A (1) A (2) A (p)] with A (l) = [ a (l) ] ij. A bias-corrected for τ, deoted by ˆτ, ca thus be costructed as ˆτ =ˆτ ˆK 1 Â vec( ˆK 1 ), where ˆτ is the of the ukow parameter τ, ˆK = K τ=ˆτ,adâ = A τ=ˆτ. It ca be show that the bias of ˆτ will be of order O( 2 ). For our problem, we have the case of p = 2, i.e., τ =(θ, c). Before adoptig the above corrective approach to bias-corrected s, we eed the followig higherorder derivatives of the log-likelihood fuctio of θ ad c i (3.3). Simple algebra 25

41 shows that 2 l +1) = (c + θ2 θ 2 (θ + c) = k 11, 2 2 l θ c = θ + (θ + c) = k 12, 2 2 l c 2 = (θ + c) 2 ψ (c) =k 22, 3 l 2(c +1) = 2 θ3 θ 3 (θ + c) = k 111, 3 3 l θ 2 c = θ 2 2 (θ + c) = k 112, 3 3 l θ c = 2 2 (θ + c) = k 122, 3 3 l c = 2 3 (θ + c) 3 ψ (c) =k 222, where ψ (c) adψ (c) are the first ad secod derivatives of ψ(c), respectively. Of particular ote is that the higher-order derivatives do ot ivolve the sample data ad thus are equal to their expectatios give above. I additio, we have k (1) 11 = k 11 θ = k 111, k (1) 12 = k 12 θ = k 112, k (1) 22 = k 22 θ = k 122, k (2) 11 = k 11 c = k 112, k (2) 12 = k 12 c = k 122, k (2) 22 = k 22 c = k

42 To implemet the corrective approach, we obtai the elemets of A (1) : a (1) 11 = k (1) k (c +1) 111 = θ 3 (θ + c), 3 a (1) 12 = a (1) 21 = k (1) k 112 = 2θ 2 (θ + c), 3 a (1) 22 = k (1) k 122 = (θ + c) 3. The elemets of A (2) are a (2) 11 = k (2) k 112 = 2θ 2 (θ + c), 3 a (2) 12 = a (2) 21 = k (2) k 122 = (θ + c) 3, a (2) 22 = k (2) k 222 = (θ + c) 3 2 ψ (c). The matrix of A ca thus be writte as A = [ A (1) A (2)] = c+1 θ 3 1 (θ+c) 3 1 2θ 2 1 (θ+c) 3 1 2θ 2 1 (θ+c) 3 1 (θ+c) 3 1 2θ 2 1 (θ+c) 3 1 (θ+c) 3 1 (θ+c) 3 1 (θ+c) ψ (c). (3.12) 27

43 The Fisher iformatio matrix for the WL distributio is give by K = c+1 θ 2 1 (θ+c) 2 1 θ 1 (θ+c) 2 1 θ 1 (θ+c) 2 1 (θ+c) 2 + ψ (c). (3.13) The bias of the of the WL parameters (θ, c) is give by Bias ˆθ ĉ = K 1 A vec ( K 1) + O( 2 ). The bias-corrected estimators of the s of the WL distributio ca be obtaied as ˆθ ĉ = ˆθ ĉ ˆK 1  vec ( ˆK 1 ), (3.14) where ˆK = K θ=ˆθ,c=ĉ ad  = A θ=ˆθ,c=ĉ. Note that the bias-corrected estimators i (3.14) have simple closed-form expressios, which meas they are quite attractive because they are ot computatioally burdesome. It should be oted that (ˆθ, ĉ ) is a bias-corrected of (θ, c) to order O( 1 ) ad that its bias is of order O( 2 ), i.e., IE [ˆθ ] = θ +O( 2 )adie [ ĉ ] = c +O( 2 ). As oe would expect, ˆθ ad ĉ have superior fiite-sample behavior relative to ˆθ ad ĉ, respectively, whose biases are of order O( 1 ). 28

44 3.3.2 A bootstrap approach As a alterative to the aalytically bias-corrected s metioed above, we here cosider the Efro s [27] bootstrap resamplig method for derivig the bias-corrected s. Let y =(y 1,,y ) be a radom sample of size from the radom variable Y with distributio fuctio F.Letη = t(f ) be a fuctio of F kow as a parameter ad ˆη = s(y) be a estimator of η. I Efro s bootstrap resamplig, we choose a large umber of pseudo-samples y =(y1,,y )fromthesampley ad calculate the correspodig bootstrap replicates of ˆη, sayˆη = s(y ). Thereafter, the empirical distributio of ˆη is used to estimate the distributio fuctio of ˆη. IfF belogs to a parametric family which is kow ad has fiite dimesio, F η, we ca the obtai a parametric estimate for F by usig a cosistet estimator for Fˆη. The bias of the estimator ˆη = s(y) ca be writte as B F (ˆη, η) =IE F [ s(y) ] ˆη(F ), (3.15) where the subscript F deotes that expectatio is take with respect to F. The bootstrap bias estimate is obtaied by replacig F, from which the origial sample was obtaied, by Fˆη. Hece, the bias ca be writte as B Fˆη (ˆη, η) =IE Fˆη [ˆη] ˆη. 29

45 For N bootstrap samples geerated idepedetly from the origial sample y, we calculate the correspodig bootstrap estimates (ˆη (1),, ˆη () ). Whe N is gettig larger, the expected value E Fˆη (ˆη) ca be approximated by ˆη ( ) = 1 N N ˆη (i). i=1 The bootstrap bias estimate, obtaied from the N replicates of ˆη, isthusb Fˆη (ˆη, η) = ˆη ( ) ˆη. The secod-order bias-corrected s of the WL distributio ca be obtaied as η B =ˆη B Fˆη (ˆη, η) =2ˆη ˆη ( ). (3.16) Note that the estimator η B shall be called the costat bias-corrected sice it approximates the fuctio by a costat; see [28]. 3.4 Simulatio studies I this sectio, we carry out Mote Carlo simulatios to compare the performace betwee the proposed ad two previous methods i the literature. The WL radom variables are geerated usig the acceptace-rejectio algorithm: Step 1. Geerate u 1,,u for Uiform(0, 1); 30

46 Step 2. If u i p = θ/(θ+c)(u i >p), geerate x i from Gamma(c, θ) (Gamma(c+1,θ)). For ease of otatio, let ˆβ, ˆβ, ˆβ, ad ˆβ MME stad for the corrective, bootstrap,, ad MME of the ukow parameter β for β = θ, c, respectively. [9] show that the MMEs of θ ad c are give by [ĉmme ˆθ MME = ĉmme ( x 1) + ( x 1) ] 2 +4 xĉ MME (ĉ MME +1), (3.17) 2 x ad ĉ MME = b( x, s2 )+ [b( x, s2 ) ] 2 [ +16s 2 s 2 +( x +1) 2] x 3 2s 2[ s 2 +( x +1) 2], (3.18) respectively, where b( x, s 2 )=s 4 x( x 3 +2 x 2 + x 4s 2 )with x ad s 2 beig the sample mea ad biased sample variace. Followig the similar sceario of [9], we draw radom samples of size =10, 20,, 100 with parameters θ =0.5, 2ad c =0.5, 1, 2. The umber of Mote Carlo replicatios is M =5, 000 ad the umber of bootstrap replicatios is B =1, 000 for each combiatio of (, θ, c). Hece, each combiatio etails a total of 50 millio replicatios. I each simulatio, to assess the performace of the methods uder cosideratio, we calculate the average bias ad root mea squared error (RMSE) of a estimator β est of the parameter β, which are defied as Bias( ˆβ est )= 1 M M i=1 ( ˆβest i β ) ad RMSE( ˆβ est )= 1 M M i=1 ( ˆβest i β )2, 31

47 respectively. Figures 3.1 ad 3.2 depict the biases of the simulated estimates of θ ad c agaist the sample sizes. The correspodig RMSEs of the simulated estimates of θ ad c are also displayed i Figures 3.3 ad 3.4, respectively. The four figures reveal importat iformatio. (i) The s ad MMEs of θ ad c appear positively biased, idicatig that the s ad MMEs o average overshoot the target parameters θ ad c, particularly whe the sample size is small. We also observe that i each simulatio, the outperforms the MME i terms of bias ad RMSE. (ii) Note that the s ad s of θ ad c clearly perform better tha the s ad MMEs uder the same sceario above ad that these corrected estimators provide substatial bias-correctio, especially for the small or moderate sample sizes. Cosequetly, we may treat them as better alteratives of the s ad MMEs for θ ad c for the case i which bias is a cocer. (iii) Whe gets larger, the bias ad RMSE of each estimator decrease ad the magitude of reductio becomes smaller. This is expected because most estimators i statistical theory perform better with icreasig. Therefore, oe ca oly expect that the performaces of all the estimators become closer with icreasig i terms of biases ad RMSEs. (iv) The reductios i biases ad RMSEs of each estimate are very substatial eve for small sample sizes. For istace, whe = 10, θ = 2, 32

48 ad c = 1, Bias(ˆθ )= , Bias(ˆθ )= , Bias(ˆθ )= , Bias(ˆθ MME )= , Bias(ĉ )= , Bias(ĉ )= , Bias(ĉ )= , Bias(ĉ MME )= ; RMSE(ˆθ )= , RMSE(ˆθ )= , RMSE(ˆθ )= , RMSE(ˆθ MME )= , RMSE(ĉ )= , RMSE(ĉ )= , RMSE(ĉ )= , RMSE(ĉ MME )= (v) The proposed estimator cosistetly outperforms the bootstrap estimator i terms of bias ad RMSE. I particular, the bootstrap biascorrectio procedure may lead to a icreased RMSE, as show i Figures 3.3 ad 3.4. For example, whe θ =2adc = 2, the RMSE of the bootstrap estimator is larger tha that of the for 20,. Hece, the corrected estimators proposed i this paper should be preferred for the WL distributio, istead of the oes via the bootstrap. 3.5 Real data examples I this sectio, we illustrate the practical applicatio of the proposed bias-corrected estimators for the WL distributio usig two real data sets with oe ivolvig a small sample ad the other with a moderate sample. 33

49 θ = 0.5 ad c = 0.5 θ = 0.5 ad c = 1 Bias MME Bias MME θ = 0.5 ad c = 2 θ = 2 ad c = 0.5 Bias MME Bias MME θ = 2 ad c = 1 θ = 2 ad c = 2 Bias MME Bias Figure 3.1: Average bias of the cosidered estimate of θ. 34

50 θ = 0.5 ad c = 0.5 θ = 0.5 ad c = 1 Bias MME Bias MME θ = 0.5 ad c = 2 θ = 2 ad c = 0.5 Bias MME Bias MME θ = 2 ad c = 1 θ = 2 ad c = 2 Bias MME Bias MME Figure 3.2: Average bias of the cosidered estimate of c. 35

51 θ = 0.5 ad c = 0.5 θ = 0.5 ad c = 1 RMSE MME RMSE MME θ = 0.5 ad c = 2 θ = 2 ad c = 0.5 RMSE MME RMSE MME θ = 2 ad c = 1 θ = 2 ad c = 2 RMSE MME RMSE MME Figure 3.3: RMSE of the cosidered estimate of θ. 36

52 θ = 0.5 ad c = 0.5 θ = 0.5 ad c = 1 RMSE MME RMSE MME θ = 0.5 ad c = 2 θ = 2 ad c = 0.5 RMSE MME RMSE MME θ = 2 ad c = 1 θ = 2 ad c = 2 RMSE RMSE MME Figure 3.4: RMSE of the cosidered estimate of c. 37

53 Example 3.1 We shall ow aalyze a data set o the lifetime failure of a electroic device. The data were used by [29] as a illustratio of the additive Burr XII distributio. Later o, the data were further aalyzed by [9] for comparig differet estimatio methods for the WL distributio. The data are give i Table Table 3.1 The time to failure of 18 electroic devices The poit estimates for the WL distributio are provided i Table 3.2. Note that the bias-corrected estimates of θ ad c are smaller tha the s ad MMEs, especially for estimatig c. This would justify that estimatio by the maximum likelihood ad method of momets are overestimatig both θ ad c. Figure 3.5 depicts the WL desity give by (3.1) evaluated at differet estimates of θ ad c i Table 3.2. It ca be see from Figure 3.5 that give the small sample size, the shape of desities based o the ad MME may be misleadig ad that correctio for bias i the estimatio for the WL distributio should be extremely importat i real data aalysis. Estimate θ c MME Table 3.2 Poit estimates of θ ad c for Example

54 Desity MME x Figure 3.5: Estimated fitted desity fuctios of the lifetime failure of a electroic device for Example 3.1. Example 3.2 The data set is give by [30] o the failure stresses (i GPa) of 65 sigle carbo fibers of legth 50mm. The data were recetly used as a illustrative example for the WL distributio by [9]. The data are preseted i Table Table 3.3 The failure stresses (i GPa) of 65 sigle carbo fibers of legth 50mm. The poit estimates of θ ad c obtaied by all the cosidered methods are summarized i Table 3.4. It is worth poitig out that all the estimatios are obviously differet, 39

55 which idicates that eve whe the sample size is moderate, the bias correctio is still ecessary because it cotais useful iformatio. Figure 3.6 cotais the WL desity give by (3.1) evaluated at the poit estimates of θ ad c i Table 3.4. Note that the estimated desity obtaied from the is too peaked ad the ad desity estimates are almost overlappig ad less peaked tha the two previous estimates i the literature. Estimate θ c MME Table 3.4 Poit estimates of θ ad c for Example 3.2. Desity MME x Figure 3.6: Estimated fitted desity fuctios of the failure stresses (i GPa) of 65 sigle carbo fibers of legth 50mm for Example

56 3.6 Cocludig remarks I this chapter, we have adopted a corrective approach to derive simple closedform expressios for the secod order biases of the s of the parameters that idex the weighted Lidley distributio. The biases of the proposed estimators are of order O( 2 ), whereas for the s they are of order O( 1 ), idicatig that the ewly proposed estimators coverge to their true value cosiderably faster tha those of the s. I additio, we have also cosidered a alterative bias-correctio mechaism through Efro s bootstrap resamplig. The umerical evidece shows that the proposed estimators are quite attractive because they outperform those of the ad MME i terms of bias ad RMSE. It deserves metioig that ulike the bias-corrected s via the bootstrap, the proposed estimators are available i closed form ad are thus easy to compute without requirig data resamplig. Cosequetly, the proposed bias-corrected estimators are strogly recommeded over other estimators without bias-correctio, especially whe the sample size is small or moderate, which is ofte ecoutered i the cotext of reliability aalysis. 41

57

58 Chapter 4 Cocludig remarks ad future work The mai goal of this report is to illustrate the importace of the bias-correctio of the s of the probability distributios, especially whe the sample size is small or moderate. It has bee show that the fitted distributios based o the s ad bias-corrected s ca be sigificatly differet for both the oe-parameter iverse Lidley distributio ad the two-parameter weighted Lidley distributio. We thus have a preferece of the cosidered bias-corrected techique, because it reduces the bias of the from order O( 1 )toordero( 2 ), idicatig that the bias-corrected estimator coverges to the true value faster tha the oe based o the. Moreover, the cosidered techique ca be easily implemeted i practical situatios as log as 43

59 the of the ukow parameter is available. Recetly, umerous distributios have bee developed i the literature. The mai motivatio of these ew distributios is that researchers wat to provide a better fit for the real-data applicatios. I this report, we have show that the poor performace of a distributio maybe due to iaccurate estimators of the ukow parameters, rather tha the ier properties of the distributio. Cosequetly, special attetio should be paid whe we apply a distributio to aalyze the real data i practice. I a o-goig work, we study the applicatio of the bias-corrected techique to some other commoly used lifetime distributios, such as weighted expoetial distributio [31] ad the three-parameter Lidley geometric distributio[32], which are curretly uder ivestigatio ad will be reported elsewhere i the ear future. 44

60 Refereces [1] Lidley, D. V. Joural of the Royal Statistical Society. Series B. (Methodological) 1958, 20, [2] Ali, S. ; Aslam, M. ; Kazmi, S. Applied Mathematical Modellig. Simulatio ad Computatio for Egieerig ad Evirometal Systems 2013, 37(8), [3] Sigh, B.; Gupta, P. K. Mathematics ad Computers i Simulatio 2012, 82(9), [4] Al-Mutairi, D. K.; Ghitay, M. E.; Kudu, D. Commuicatios i Statistics. Theory ad Methods 2013, 42(8), [5] VikasKumarSharma,SajayKumarSigh,U.S.V.A.Applicatios (stat.ap) 2014, (17), [6] Cox, D. R.; Sell, E. J. Joural of the Royal Statistical Society. Series B. (Methodological) 1968, 30,

61 [7] Cordeiro, G. M.; Klei, R. Statistics & Probability Letters 1994, 19(3), [8] Ghitay, M. E.; Atieh, B.; Nadarajah, S. Mathematics ad Computers i Simulatio 2008, 78(4), [9] Mazucheli, J.; Achcar, J. A. Computer Methods ad Programs i Biomedicie 2011, 104(2), [10] Gupta, P.; Sigh, B. Iteratioal Joural of System Assurace Egieerig ad Maagemet 2012, pages 1 8. [11] Al-Mutairi, D. K.; Ghitay, M. E.; Kudu, D. Commuicatios i Statistics - Theory ad Methods 2013, 42(8), [12] A ew three-parameter lifetime distributio ad associated iferece. Wag, M. arxiv: [stat.me], [13] Ghitay, M.; Alqallaf, F.; Al-Mutairi, D.; Husai, H. Mathematics ad Computers i Simulatio 2011, 81(6), [14] Kay, S. Sigal Processig, IEEE Trasactios o 1995, 43(4), [15] Mazucheli, J.; Louzada, F.; Ghitay, M. Applied Mathematics ad Computatio 2013, 220(0), [16] Lemote, A. J. Joural of Statistical Computatio ad Simulatio 2011, 81(12),

62 [17] A ote o improved estiamtio of the Topp-Leoe distributio. Giles, D. Ecomometrics Workig Paper EWP1203, Departemet of Ecoomics, Uiversity of Victoria, [18] Bias reductio for the maximum likelihood estimator of the geeralized Rayleigh family of distributios. Xiao, L.; Giles, D. To appear i Commuicatios i Statistics - Theory ad Methods, [19] O the bias of the maximum likelihood estimator for the two-parameter Lomax distributio. Giles, D.; Feg, H.; Godwi, R. To appear i Commuicatios i Statistics - Theory ad Methods, [20] Schwartz, J.; Godwi, R. T.; Giles, D. E. Joural of Statistical Computatio ad Simulatio 2013, 83(3), [21] Teimouri, M.; Nadarajah, S. Statistical Methodology 2013, 13(0), [22] Efro, B. The Jackkife, the Bootstrap ad Other Resamplig Plas; [23] Efro, B.; Tibshirai, R. Statistical Sciece 1986, 1(1), [24] DiCiccio, T.; Efro, B. Biometrika 1992, 79(2), [25] Efro, B.; Tibshirai, R. A Itroductio to the Bootstrap; [26] Firth, D. Biometrika 1993, 80(1), [27] Efro, B. The Aals of Statistics 1979, 7(1),

63 [28] MacKio, J.; Smith, A. Joural of Ecoometrics 1998, 85, [29] Wag, F. Reliability Egieerig & System Safety 2000, 70(3), [30] Statistical aspects of fiber ad budle stregth i hybrid composites. Bader, M. G.; Priest, A. M. I: Hayashi, T., Kawata, K., Umekawa, S. (Eds.), Progress i Sciece ad Egieerig Composites. ICCM-IV, Tokyo, [31] Gupta, R. D.; Kudu, D. Statistics 2009, 43(6), [32] Rezaei, S.; Nadarajah, S.; Tahghighia, N. Statistics 2013, 47(4), [33] Iferetial techiques for Weibull populatios. McCool, J. Jue, [34] Giles, D. E.; Feg, H.; Godwi, R. T. Commuicatios i Statistics-Theory ad Methods 2013, 42(11),