Gram Charlier and Edgeworth expansion for sample variance arxiv:809.06668v [math.st] 8 Sep 08 Eric Benhamou,* A.I. SQUARE CONNECT, 35 Boulevard d Inkermann 900 Neuilly sur Seine, France and LAMSADE, Universit Paris Dauphine, Place du Marchal de Lattre de Tassigny,7506 Paris, France.e-mail: * eric.benhamou@aisquare.com,eric.benhamou@dauphine.eu Abstract: In this paper, we derive a valid Edgeworth expansions for the Bessel corrected empirical variance when data are generated by a strongly mixing process whose distribution can be arbitrarily. The constraint of strongly mixing process makes the problem not easy. Indeed, even for a strongly mixing normal process, the distribution is unknown. Here, we do not assume any other assumption than a sufficiently fast decrease of the underlying distribution to make the Edgeworth expansion convergent. This results can obviously apply to strongly mixing normal process and provide an alternative to the work of Moschopoulos (985) and Mathai (98). Keywords and phrases: sample variance, Edgeworth expansion.. Introduction Let X,...,X n be a random sample and define the sample variance statistic as: X n = n n X i, s n = n n (X i X n ), X n = (X,...,X n ) T (.) where X n is the empirical mean, s n the Bessel corrected empirical variance also called sample variance, and X n the vector of the full history of this random sample. We are interested in the distribution of the sample variance under very weak conditions, namely that it admits a valid Edgeworth expansion. It is insightful to notice that even with the additional constraint of a multi dimensional Gaussian distribution N(0,Σ) for the underlying random vector X n, the distribution of the sample variance is not known. In this particular setting, the sample variance is the squared norm of a multi dimensional Gaussian and can be seen as the linear combination of independent but not Homoscedastic variables. Standard theory states that the sample variance of a collection of independent and identically distributed normal variables follows a chi-squared distribution. But in this particular case, the different variables X i X n are not independent and the result can not apply. Though they marginally have the same variance (if we conditioned by X n ), they are correlated with each other. with Sigma arbitrarily
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance Hence, in general, the distribution of the sample variance in the normal case is not a known distribution. There is however one special case, where it is a χ distribution. Remember that when X N(0,Σ), X T BX has a χ distribution of degree d the rank of B if and only if BΣB = B (see for instance page 43 remark following theorem of Hogg et al. (978)). Notice that in our case, the B matrix is B = I n n ½T n ½ n, where I n is the identity matrix of size n and ½ n is the vector of size n filled with as we can write s n = = n n (X i X n ) (.) we obtain that it is a χ distribution if and only if n XT (I n n ½T n ½ n)x (.3) (I n n ½T n ½ n)σ(i n n ½T n ½ n) = I n n ½T n ½ n In other cases, the sample variance is a linear combination of Gamma distribution, and one has to rely on approximations as explained in Moschopoulos (985) and Mathai (98). This simple example explains the interest in deriving an approximation of the distribution of the sample variance by means of Gram Charlier and Edgeworth expansion.. Gram Charlier and Edgeworth expansion.. Key concepts GramCharlier expansion, and Edgeworth expansion 3, are series that approximate a probability distribution in terms of its cumulants. The series are the same but, they differ in the ordering of their terms. Hence the truncated series are different, as well as the accuracy of truncating the series. The key idea in these two series is to expand the characteristic function in terms of the characteristic function of a known distribution with suitable properties, and to recover the concerned distribution through the inverse Fourier transform. In our case, a natural candidate to expand around is the normal distribution as the central limit theorem and its different extensions to non independent and non identically distributed variable state that the resulting distribution is a normal distribution (or in the most general case to truncated symmetrical and α stable distributions 4. named in honor of the Danish mathematician, Jrgen Pedersen Gram and the Swedish astronomer, Carl Charlier 3 named in honor of the Anglo-Irish philosopher, Francis Ysidro Edgeworth 4 please see the extension of CLT in Gnedenko and Kolmogorov (954)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 3 Let denote by ˆf (respectively ˆφ the characteristic function of our distribution whose density function is f (respectively φ), and κ j its cumulants (respectively γ j ). Cumulants definition state ˆf(t) = exp (it) j κ j and ˆφ(t) = exp j! j= j= (it) j γ j, (.) j! which gives the following formal identity: ˆf(t) = exp (κ j γ j ) (it)j ˆφ(t). (.) j! j= Using Fourier transform property that say (it) jˆφ(t) is the Fourier transform of ( ) j [D j φ]( x), where D is the differential operator with respect to x, we get the formal expansion: f(x) = exp (κ j γ j ) ( D)j φ(x) (.3) j! j= If φ is chosen as the normal density φ(x) = πσ exp [ (x µ) σ ] with mean and variance as given by f, that is, mean µ = κ and variance σ = κ, then the expansion becomes f(x) = exp ( D) j κ j φ(x), (.4) j! j=3 since γ j = 0 for all j >, as higher cumulants of the normal distribution are 0. By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the GramCharlier A series. Such an expansion can be written compactly in terms of Bell polynomials as exp j=3 κ j ( D) j j! = j=0 B j (0,0,κ 3,...,κ j ) ( D)j. (.5) j! Since the j-th derivative of the Gaussian function φ is given in terms of Hermite polynomial as ( ) φ (j) (x) = ( )j x µ σ j He j φ(x), (.6) σ this gives us the final expression of the Gram-Charlier A series as f(x) = φ(x) j=0 j!σ jb j(0,0,κ 3,...,κ j )He j ( x µ σ ). (.7)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 4 If we include only the first two correction terms to the normal distribution, we obtain [ f(x) = φ(x) + κ ( ) 3 x µ 3!σ 3He 3 + κ ( ] 4 x µ σ 4!σ 4He 4 )+R 5, (.8) σ with He 3 (x) = x 3 3x and He 4 (x) = x 4 6x +3 and R 5 (x) = j=5 j!σ jb j(0,0,κ 3,...,κ j )He j ( x µ If in the above expression, the cumulant are function of a parameter n, we can rearrange terms by power of n and find the Edgeworth expansion. σ )... Cumulant for weak conditions In order to derive our Gram Charlier or Edgeworth expansion, we need to compute in full generality our different cumulants. Using similar techniques as in Benhamou (08), we can get the various cumulants as follows: The first two cumulants are easy and given by: κ =( µ ) ( µ ) (.9) κ = A,0 n + A 0, n + A, +R (.0) (n )n with the different numerator terms given by: A,0 = 4( µ 4 ) +8( µ µ ) ( µ ) (.) A 0, =( µ 4) 4( µ µ 3) (.) A, =6( µ 4 ) ( µ µ ) +3( µ ) (.3) R =( µ ) ( µ µ ) +( µ 4 ) (κ ) (.4) with the natural symmetric moment estimators whose expressions are provided in appendix section A.. The term R is the second order rest and is equal to zero if the sample is i.i.d. For general case, this term does not cancel out and should be taken into account. It can be rewritten as R =( µ ) ( µ ) +(( µ ) ( µ ) ( µ µ ) )+( µ 4 ) ( µ ) (.5) The third cumulant is more involved and given by: κ 3 = A3,0 (n ) + A3, (n )n + A3 0, n + A3, (n ) n + A3, (n ) n + A 3, (n ) +R3 (.6) n
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 5 where the different numerator terms are given by: A 3,0 = 40( µ 6 ) 3 +0( µ 4 µ ) 3 56( µ 3 µ 3) 3 78( µ µ ) 3 +48( µ µ µ 3) 3 +( µ 3 ) 3 6( µ 3) 3 (.7) A 3, =8( µ µ 4) 3 3( µ µ 4) 3 (.8) A 3 0, =( µ 6) 3 6( µ µ 5) 3 (.9) A 3, =36( µ 6 ) 3 408( µ 4 µ ) 3 +60( µ 3 µ 3) 3 +88( µ µ ) 3 44( µ µ µ 3) 3 4( µ 3 ) 3 +( µ 3) 3 (.0) A 3, =5( µ µ 4) 3 30( µ µ 4) 3 (.) A 3, = 0( µ 6 ) 3 +360( µ 4 µ ) 3 0( µ 3 µ 3) 3 70( µ µ ) 3 +0( µ µ µ 3) 3 +30( µ 3 ) 3 0( µ 3) 3 (.) R 3 = 3µ µ +(κ ) 3 (.3) with the natural symmetric moment estimators given in appendix section A.. The fourth cumulant is even more involved and given by: κ 4 = A4 3,0 (n ) 3 + A4, (n ) n + A4, (n )n + A4 0,3 n 3 + A4 3, (n ) 3 n + A 4, (n ) n + A4,3 (n )n + A 4 3, 3 (n ) 3 n + A 4,3 (n ) n + A 4 3,3 3 (n ) 3 +R4 (.4) n3
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 6 with the different numerator terms are given by: A 4 3,0 = 67( µ 8 ) 4 +688( µ 6 µ ) 4 6( µ 5 µ 3) 4 30( µ 4 µ ) 4 +400( µ 4 µ 4) 4 +40( µ 3 µ µ 3) 4 +960( µ µ 3 ) 4 384( µ 3 µ ) 4 480( µ µ µ 4) 4 64( µ µ µ 3) 4 +44( µ µ 3 µ 4) 4 6( µ 4 ) 4 +96( µ µ 3) 4 3( µ 4) 4 +( µ µ 4) 4 (.5) A 4, = 8( µ 3 µ 5) 4 +96( µ µ µ 5) 4 4( µ 3 µ 5) 4 (.6) A 4, =3( µ µ 6) 4 4( µ µ 6) 4 (.7) A 4 0,3 =( µ 8) 4 8( µ µ 7) 4 (.8) A 4 3, =379( µ 8 ) 4 568( µ 6 µ ) 4 +644( µ 5 µ 3) 4 +844( µ 4 µ ) 4 90( µ 4 µ 4) 4 50( µ 3 µ µ 3) 4 6336( µ 3 µ ) 4 +680( µ µ 3) 4 +50( µ µ µ 4) 4 +3600( µ µ µ 3) 4 64( µ µ 3 µ 4) 4 +34( µ 4 ) 4 43( µ µ 3) 4 +33( µ 4) 4 5( µ µ 4) 4 (.9) A 4, =400( µ 3 µ 5) 4 336( µ µ µ 5) 4 +48( µ 3 µ 5) 4 (.30) A 4,3 =8( µ µ 6) 4 56( µ µ 6) 4 (.3) A 4 3, = 7440( µ 8 ) 4 +9760( µ 6 µ ) 4 0880( µ 5 µ 3) 4 36480( µ 4 µ ) 4 +304( µ 4 µ 4) 4 +099( µ 3 µ µ 3) 4 +384( µ µ 3 ) 4 75( µ µ 3) 4 4368( µ µ µ 4) 4 748( µ µ µ 3) 4 +976( µ µ 3 µ 4) 4 738( µ 4 ) 4 +800( µ µ 3) 4 57( µ 4) 4 +6( µ µ 4) 4 (.3) A 4,3 = 336( µ 3 µ 5) 4 +336( µ µ µ 5) 4 56( µ 3 µ 5) 4 (.33) A 4 3,3 =5040( µ 8 ) 4 060( µ 6 µ ) 4 +670( µ 5 µ 3) 4 +500( µ 4 µ ) 4 680( µ 4 µ 4) 4 3440( µ 3 µ µ 3) 4 0080( µ µ 3 ) 4 +680( µ µ 3) 4 +50( µ µ µ 4) 4 +5040( µ µ µ 3) 4 560( µ µ 3 µ 4) 4 +630( µ 4 ) 4 560( µ µ 3) 4 +35( µ 4) 4 40( µ µ 4) 4 (.34) with the natural symmetric moment estimators given in appendix section A.3. Proof. see appendix section B 3. Conclusion In this paper, we have derived the most general formula for the Gram Charlier and the resulting Edgeworth expansion for the sample variance under very weak conditions. Our formula does not assume that the underlying sample is independent neither identically distributed. This formula can therefore be applied to strong mixing processes like sample of an auto regressive process of order (AR()). It extends in particular the work of Mikusheva (05)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 7 Appendix A: Notations A.. Empirical Moments of order and Notation We adopt the following notations moment of order : n ( µ ) = Xi n ( µ i j ) = E[XiXj] n(n ) moments of order : n ( µ 4) = Xi 4 n i j Xi 3 X j ( µ µ 3) = n(n ) ( µ i j ) = Xi Xj n(n ) i j k Xi X jx k ( µ µ ) = n(n )(n ) ( µ 4 i j k l ) = E[XiXjX kx l ] n(n )(n )(n 3) (A.) (A.) (A.3) (A.4) (A.5) (A.6) (A.7)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 8 A.. Empirical Moments of order 3 Notation ( µ 6 i j k l m n ) 3 = E[XiXjX kx l X mx n] n(n )(n )(n 3)(n 4)(n 5) i j k l m Xi X jx k X l X m ( µ 4 µ ) 3 = n(n )(n )(n 3)(n 4) i j k l XiX 3 jx k X l ( µ 3 µ 3) 3 = n(n )(n )(n 3) i j k l XiX jx k X l ( µ µ ) 3 = n(n )(n )(n 3) i j k Xi 4 X jx k ( µ µ 4) 3 = n(n )(n ) i j k Xi 3 XjX k ( µ µ µ 3) 3 = n(n )(n ) i j Xi 5 X j ( µ µ 5) 3 = n(n ) ( µ 3 i j k ) 3 = Xi XjX k n(n )(n ) i j ( µ µ 4) 3 = Xi 4 Xj n(n ) ( µ i j 3) 3 = Xi 3 Xj 3 n(n ) i ( µ 6) 3 = Xi 6 n (A.8) (A.9) (A.0) (A.) (A.) (A.3) (A.4) (A.5) (A.6) (A.7) (A.8)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 9 A.3. Empirical Moments of order 4 Notation ( µ 8 i j k l m n o p ) 4 = E[XiXjX kx l X mx nx ox p] n(n )(n )(n 3)(n 4)(n 5)(n 6)(n 7) i j k l m n o Xi X jx k X l X mx nx o ( µ 6 µ ) 4 = n(n )(n )(n 3)(n 4)(n 5)(n 6) i j k l m n XiX 3 jx k X l X mx n ( µ 5 µ 3) 4 = n(n )(n )(n 3)(n 4)(n 5) i j k l m n XiX jx k X l X mx n ( µ 4 µ ) 4 = n(n )(n )(n 3)(n 4)(n 5) i j k l m Xi 4 X jx k X l X m ( µ 4 µ 4) 4 = n(n )(n )(n 3)(n 4) i j k l XiX 5 jx k X l ( µ 3 µ 5) 4 = n(n )(n )(n 3) i j k l m Xi 3 XjX k X l X m ( µ 3 µ µ 3) 4 = n(n )(n )(n 3)(n 4) i j k l XiX 4 jx k X l ( µ µ µ 4) 4 = n(n )(n )(n 3) i j k l m Xi XjX kx l X m ( µ µ 3 ) 4 = n(n )(n )(n 3)(n 4) i j k l XiX 3 jx 3 k X l ( µ µ 3) 4 = n(n )(n )(n 3) i j k Xi 6 X jx k ( µ µ 6) 4 = n(n )(n ) i j k l XiX 3 jx kx l ( µ µ µ 3) 4 = n(n )(n )(n 3) i j k Xi 5 XjX k ( µ µ µ 5) 4 = n(n )(n ) i j k Xi 4 XjX 3 k ( µ µ 3 µ 4) 4 = n(n )(n ) i j Xi 7 X j ( µ µ 7) 4 = n(n ) ( µ i j k µ 4) 4 = Xi 4 XjX k n(n )(n ) ( µ 4 i j k l ) 4 = XiX jx kx l n(n )(n )(n 3) i j ( µ µ 6) 4 = Xi 6 Xj n(n ) ( µ µ i j k 3) 4 = Xi 3 XjX 3 k n(n )(n ) (A.9) (A.0) (A.) (A.) (A.3) (A.4) (A.5) (A.6) (A.7) (A.8) (A.9) (A.30) (A.3) (A.3) (A.33) (A.34) (A.35) (A.36) (A.37)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 0 i j ( µ 3 µ 5) 4 = XiX 5 j 3 n(n ) ( µ i j 4) 4 = XiX 4 j 4 n(n ) i ( µ 8) 4 = Xi 8 n (A.38) (A.39) (A.40) Appendix B: Cumulant computation First of all, the first four cumulants, denoted by κ i for ii =,...,4 are obtained through standard relationships with respect to moments denoted by µ i as follows: κ = µ κ = µ µ κ 3 = µ 3 3µ µ +µ 3 κ 4 = µ 4 4µ µ 3 3µ +µ µ 6µ 4 (B.) (B.) (B.3) (B.4) Hence we are left with computing the first four moments of the sample variance. The first two moments of the sample variance are easy to compute and given for instance in Benhamou (08). For the cumulant of order 3 and 4, we first compute the different moments and then regroup the terms. B.. Third Moment for s n Let us do some routine algebraic computation. We have 3 s 6 n n = (n ) n 3 (n ) 3 Xi X i X j (B.5) i j n n = (n ) 3 n 3 (n ) 3 ( Xi )3 3(n ) ( Xi )( X k X l ) k l n +3(n )( i X )( X k X l ) +( X k X l ) 3 (B.6) k l k l Let us expand. The first expansion ( n X i )3 is easy and immediate: n n ( Xi) 3 = XiX 4 j + XiX jx k (B.7) X 6 i +3 i j i j k In the expansion of ( n X i ) ( j k X jx k ), the possibilities are:
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance i j X5 i X j i j k X4 i X jx k i j X3 i X3 j i j k X3 i X j X k i j k l X i X j X kx l In the expansion of ( n X i )( j k X jx k ), the possibilities are: i j X4 i X j i j k X4 i X jx k i j k X3 i X j X k i j k l X3 i X jx k X l i j k X i X j X k i j k l X i X j X kx l i j k l m X i X jx k X l X m In the expansion of ( i j X ix j ) 3, the possibilities are: i j X3 i X3 j i j k X3 i X j X k i j k X i X j X k i j k l X3 i X jx k X l i j k l X i X j X kx l i j k l m X i X jx k X l X m i j k l m n X ix j X k X l X m which leads to E[s 8 n] = (n 5)(n 4)(n 3)(n ) µ6 (n ) n + 3(n 5)(n 4)(n 3)(n ) µ µ 4 (n ) n + 4(n 3)(n )(3n 5) µ 3 µ 3 (n ) n 3(n 3)(n ) ( n 6n+5 ) µ µ (n ) n 3(n 5)(n ) µ 4 µ (n ) (n )n ( 3n 6n+5 ) µ 3 (n ) n + 3 ( n 4n+5 ) µ µ 3 µ (n ) n 6 µ 5 µ n ( n n+5 ) µ µ 4 (n )n + µ 6 n + (n )( n 3 3n +9n 5 ) µ 3 (n ) n (B.8) Regrouping all the terms leads to E[s 6 n ] =M3 0,0 + M3,0 n + M3 0, n + M3,0 (n ) + M3, (n )n + M3 0, n + M3, (n ) n + M3, (n )n + M 3, (n ) n (B.9)
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance with M,0 3 = 60 µ6 +80 µ4 µ 68 µ 3 µ 3 9 µ µ +60 µ µ 3 µ +3 µ 3 6 µ 3 M0,0 3 = µ 6 +3 µ 4 µ 3 µ µ + µ 3 M, 3 = 0 µ 6 +360 µ 4 µ 0 µ 3 µ 3 70 µ µ +0 µ µ µ 3 +30 µ 3 0 µ 3 M,0 3 = µ6 36 µ4 µ + µ 3 µ 3 +7 µ µ µ µ µ 3 3 µ 3 M, 3 =54 µ6 46 µ µ 4 +7 µ3 µ 3 +333 µ µ 56 µ µ µ 3 33 µ 3 + µ 3 M,0 3 = 3 µ µ 4 +3 µ µ 4 M, 3 =5 µ µ 4 30 µ µ 4 M, 3 = µ µ 4 6 µ µ 4 M0, 3 = µ 6 6 µ µ 5 (B.0) (B.) (B.) (B.3) (B.4) (B.5) (B.6) (B.7) (B.8) Using previous results in the relationship between cumulant and moment leads to the result. B.. Fourth Moment for s n Let us do some routine algebraic computation. We have s 8 n = = (n ) n 4 (n ) 4 n 4 (n ) 4 ( n X i) k l n 4 X i X j Xi i j n n (n ) 4 ( Xi) 4 4(n ) 3 ( X k X l ) +4(n )( n X i) 3 ( k l X i)( k l X k X l ) 3 +( k l (B.9) X k X l )+6(n ) X k X l ) 4 (B.0) Again, one needs to expand all the terms and look at all the various possibilities to demonstrate the following relationship where we have regrouped against each of the symmetric empirical moment estimator
E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 3 E[s 8 n ] =(n 7)(n 6)(n 5)(n 4)(n 3)(n ) µ8 (n ) 3 n 3 4(n 7)(n 6)(n 5)(n 4)(n 3)(n ) µ µ 6 (n ) 3 n 3 8(n 5)(n 4)(n 3)(n )(3n 7) µ 3 µ 5 ( 6(n 5)(n 4)(n 3)(n ) n 0n+35 ) µ (n ) 3 n 3 + µ4 (n ) 3 n 3 + (n 4)(n 3)(n )( 3n 30n+35 ) µ 4 µ 4 6(n 4)(n 3)(n ) (n ) 3 n 3 + (n ) 3 n 3 + 8(n 3)(n )(3n 7) µ 5 µ 3 (n ) n 3 + 8(n 3)(n )( 9n 30n+35 ) µ 3 µ (n ) 3 n 3 4(n 7)(n ) µ 6 µ 4(n 3)(n ) (n )n 3 ( 3n 0n+35 ) µ µ 3 µ 3 ( 4(n 4)(n 3)(n ) n 3 9n +45n 05 ) µ 3 µ (n ) 3 n 3 ( (n 3)(n ) n 3 9n +35n 35 ) µ µ 4 µ (n ) 3 n 3 ( n 3 7n +5n 35 ) µ µ 3 µ (n ) 3 n 3 8(n )( 3n 3 n +45n 35 ) ( µ 3 µ 4 µ 4(n ) n 4n+7 ) µ µ 5 µ (n ) 3 n 3 (n ) n 3 8 µ ( 7 µ (n 3)(n ) n 4 4n 3 +8n 60n+05 ) µ 4 n 3 + (n ) 3 n 3 8(n )( 3n 3 5n +35n 35 ) µ µ 3 (n ) 3 n 3 + + 6(n )( n 4 4n 3 +6n 40n+35 ) µ µ 4 (n ) 3 n 3 8 ( 3n 4 n 3 +4n 60n+35 ) µ 4 (n ) 3 n 3 ( 3n 6n+7 ) µ 3 µ 5 (n ) n 3 + 4( n n+7 ) µ µ 6 (n )n 3 + µ 8 n 3 (B.) Once this is done, one needs to collect all the terms and can get the final result.
References E. Benhamou/Gram Charlier and Edgeworth expansion for sample variance 4 E. Benhamou. A few properties of sample variance. arxiv, September-October 08. B. Gnedenko and A. Kolmogorov. Limit distributions for sums of independent random variables. Addison-Wesley, Cambridge, Mass., 954. R. V. Hogg, J. W. McKean, and A. T. Craig. Introduction to Mathematical Statistics. Pearson, 4th edition, 978. A. M. Mathai. Storage capacity of a dam with gamma type inputs. Annals of the Institute of Statistical Mathematics, pages 59 597, 98. A. Mikusheva. Second order expansion of the t-statistics in ar() models. Econometric Theory, 3(3):46 448, 05.. P. G. Moschopoulos. The distribution of the sum of independent gamma random variables. Annals of the Institute of Statistical Mathematics, pages 54 544, 985.