APPENDIX M: NOTES ON MOMENTS

Transcription

1 APPENDIX M: NOTES ON MOMENTS Every stats textbook covers the propertes of the mea ad varace great detal, but the hgher momets are ofte eglected. Ths s ufortuate, because they are ofte of mportat real-world sgfcace. For example, returs from facal stocks are kow to have larger-tha-normal skew ad kurtoss: Madelbrot [1963, footote 3] traces awareess of ths o-normalty as far back as 1915; see also, e.g., Blattberg ad Goedes [1974] or Ko [1984]. Models that assume Normal skew ad kurtoss wll uderpredct extreme outcomes. Ths techcal appedx goes over a few propertes ad otes to help you better uderstad the skew ad kurtoss. As the ma textbook, x s a data vector of sze, ad x s a represetatve elemet. The skew s defed as the thrd cetral momet, S(x) = (x µ) 3 /, ad the kurtoss s defed as the fourth cetral momet, 1 K(x) = (x µ) 4 /. The sample momet s much lke the populato momet, but the populato momet s based o (x µ) m, where µ s the true mea of the data ad m s two for varace, three for skew, or four 0 Although t s ot boud wth the book tself, ths s a techcal appedx to Modelg wth Data: Tools ad Techques for Scetfc Computg. It uses the same otato as the book, as o pp Should you eed to cte ths appedx, smply cte the book ad gve a page umber of the form ole appedx M, page. 1 Notce that the defto of skew ad kurtoss are ot dvded by σ 3 or σ 4, or otherwse modfed from the defto of a cetral momet. Such reormalzatos make comparso to the stadard Normal dstrbuto very easy, but make absolutely every other computato more dffcult. 1

2 for kurtoss; whle the sample momet s based o (x x) m, where x s the sample mea. We ca expect that x µ. Also, x has a varace (ad skew ad kurtoss), whle µ s a costat. O the other had, we ca calculate x from the data we have, whle µ may be etrely ukowable. As o page 222, the ubased sample estmator of the sample varace, ˆσ 2 (x) = (x x) 2 /( 1), volves a sum dvded by 1, ot. There are smlar adjustmets to be made to acheve ubased skew ad kurtoss. The dervato s volved, but t gves us a chace to look over some other propertes of the cetral momets. To fd the relatoshp betwee the sample momets based o x ad the populato momets based o µ, we wll frst fd a few termedate expected values: 1. σ 2 ( x), S( x), K( x) 2. E(x ), E(x 2 ), E(x 3 ), E(x 4 ) 3. E( x 2 ), E( x 3 ), E( x 4 ) 4. E(x x 2 ), E(x 2 x), E(x3 x), E((x x) 2 ), E((x x) 3 ), E((x x) 4 ) 6. ˆσ 2 (µ), Ŝ(µ), ˆK(µ) Seres (1) s relatvely easy, ad the same dervato works for all cetral momets. Seres (2) ad (3) are the ocetral momets, where the mea s ot subtracted; the slght dfferece betwee these two seres proves to be mportat later. Seres (4) cossts of hybrd terms that multply a power of a represetatve elemet by a power of x, ad requre ew techques for estmato. Seres (5) s a estmate of the sample momets usg the pror results. Seres (6) uses seres (5) to produce a ubased estmate of the true populato momets, usg sample data. Frst, we ca calculate the varace, skew, ad kurtoss of the mea. 2 The steps volved are to expad the expected value to the sum over that t s (the ellpses dcate addtoal terms whose expected value s zero), take the 1 out of the momet calculato, ad the ote that the sum of d elemets ca be broke dow to momets of ay represetatve elemet (here, x 1 ). The secod momet of a represetatve elemet s smply σ 2 (x), ad smlarly for the other momets. MOMENTS OF THE MEAN 2 Followg Casella ad Berger [1990, p 208]. 2

3 σ 2 ( x) = E ( 1 = 1 2 E ( = 1 2 E ( (x µ) (x µ) ) 2 ) 2 (x µ) 2 + = 1 2 E ( (x 1 µ) 2) ) (x µ)(x j µ) j = 1 σ2 (x 1 ) S( x) = E ( 1 = 1 3 E ( (x µ) = 1 3 E ( (x 1 µ) 3) ) 3 (x µ) ) (x µ) 2 (x j µ) +... j = 1 2 S (x 1) K( x) = E ( 1 = 1 4 E ( (x µ) ) 4 (x µ) ) (x µ) 2 (x j µ) j> = 1 4 E ( (x 1 µ) 4 + 6(( 1)/2)σ 2 (x )σ 2 (x j ) ) = 1 3 (K (x 1 ) + 3( 1)σ 4 (x 1 ) ) 3

4 To summarze the above: σ 2 ( x) S( x) K( x) = σ2 (x) = S(x) 2 = K(x)+3( 1)σ4 (x) 3 It s reasoably tutve that the mea of several draws wll have a smaller varace, skew, or kurtoss tha a sgle draw, ad these relatos embody that tuto. They wll drve much of the story to follow. NONCENTRAL MOMENTS The expresso E(x 2 ) s ot qute the varace, whch s the expected squared dstace to the mea: E((x µ) 2 ). The further µ s from zero, the further E(x 2 ) wll be from the varace. The trck to calculatg E(x m ) s to rewrte t as E ((x µ + µ) m ). For the varace: E [ ((x µ) + µ) 2] = E [ (x µ) 2 + 2(x µ)µ + µ 2 ]. The expectato of a sum s the sum of expectatos; that s, ths expresso ca be broke dow to ts compoets: E(x 2 ) = E [(x µ) 2 ] +E [2(x µ)µ] +E [µ 2 ] = σ 2 (x) +0 +µ 2 Ths ca be rewrtte to produce the famlar statemet that σ 2 (x) = E(x 2 ) E 2 (x). We ca repeat the same steps for the skew ad kurtoss: wrte E(x m ) as E ((x µ + µ) m ), expad, ad fd the expectato of each dvdual term. For the skew: E [ ((x µ) + µ) 3] = E [(x µ) 3 ] +E [3(x µ) 2 µ] +E [3(x µ)µ 2 ] +E[µ 3 ] = S(x) +3σ 2 (x)µ +0 +µ 3. For kurtoss: E [x 4 ] = E [(x µ) 4 ] +E [4(x µ) 3 µ] +E [6(x µ) 2 µ 2 ] +E [4(x µ)µ 3 ] +E[µ 4 ]. = K(x) +4S(x)µ +6σ 2 (x)µ µ 4. To summarze: 4

5 E (x 2 ) = σ 2 (x) + µ 2 E (x 3 ) = S(x) + 3σ 2 (x)µ + µ 3 E (x 4 ) = K(x) + 4S(x)µ + 6σ 2 (x)µ 2 + µ 4 So the ocetral varace depeds o the mea, the ocetral skew depeds o the varace ad mea, ad the ocetral kurtoss depeds o the skew, varace, ad mea. I all cases, the relatoshp s postve; for example, a larger varace meas a larger ocetral skew ad kurtoss. We ca reuse much of the above calculato regardg E[x ] for E[ x]. The results from the seres of cetral momets of the mea already come hady, as we ca wrte the varace of x as σ 2 (x)/, for example. Expadg E( x 2 ), we get: NONCENTRAL MOMENTS FOR x E [ (( x µ) + µ) 2] = E [( x µ) 2 ] +E [2( x µ)µ] +E [µ 2 ] = σ 2 (x)/ +0 +µ 2. For the skew: E [ (( x µ) + µ) 3] = E [( x µ) 3 ] +E [3( x µ) 2 µ] +E [3( x µ)µ 2 ] +E [µ 3 ] = S(x)/ 2 +3σ 2 (x)µ/ +0 +µ 3. For kurtoss: E [x 4 ] = E [( x µ) 4 ] +E [4( x µ) 3 µ] +E [6( x µ) 2 µ 2 ] +E [4( x µ)µ 3 ] +E[µ 4 ]. = K(x)+3( 1)σ4 (x) 3 +4S(x)µ/ 2 +6σ 2 (x)µ 2 / +0 +µ 4. E ( x 2 ) = σ 2 (x)/ + µ 2 E ( x 3 ) = S(x)/ 2 + 3σ 2 (x)µ/ + µ 3 E ( x 4 ) = K(x)/ 3 + 3( 1)σ 4 (x)/ 3 + 4S(x)µ/ 2 + 6σ 2 (x)µ 2 / + µ 4 You ca see that ths seres s aalogous to the last, save for the dvsos by powers of. As, the varatos amog the data pots become less mportat, ad the fal value of E( x m ) s smply µ m. That s, these are asymptotcally ubased estmates of the powers of µ. So far, we have cosdered oly x or a sgle elemet x, but what about combatos, lke E(x 1 x)? [As above, use x 1 or x 2 as represetatve elemets.] x s the sum of all of the elemets of the data set, but elemets of the form x 1 x 1 behave dfferetly from elemets of the form x 1 x 2, so we eed to break dow the sum. For E(x 1 x): HYBRIDS 5

6 E[x 1 x] = E[x 1 =1 x /] = E[x 2 1/ +x 1 =2 x /] = E[x 2 1]/ +( 1)E[x 1 x 2 ] = σ 2 (x)/ + µ 2 / +( 1)µ 2 / = σ 2 (x)/ + µ 2 = E( x 2 ) I ths case, thgs worked out well fact, E(x 1 x m ) = E( x m+1 ) for ay m. But other cases are ot so elegat. For example, E(x 2 1 x) s a ew form: E[x 2 1 x] = E[x2 1 =1 x /] = E[x 3 1/ +x 2 1 =2 x /] = E[x 3 1]/ +( 1)E[x 2 1x 2 ] = (S(x) + 3σ 2 (x)µ + µ 3 )/ +( 1)(σ 2 (x) + µ 2 )µ/ = S(x)/ + +2 σ2 (x)µ + µ 3 Here s a catalog of some of the forms that wll be used below. The especally tedous dervatos have ot bee show, but take the same form as the dervatos above, expadg the mea or square-of-mea ad coutg staces of the form x 2 1x 2, x 2 1x 2 2, et cetera. E[x 1 x] = σ 2 (x)/ + µ 2 = E( x 2 ) 2 E[x1 x] +2 = S(x)/ + σ2 (x)µ + µ 3 E[x 1 x 2 ] = S(x)/ σ2 (x)µ + µ 3 = E( x 3 ) 3 E[x1 x] = K(x)/ + S(x)µ + σ2 (x)µ 2 + µ 4 E[x 1 x 3 ] = K(x)/ 3 + 3( 1)σ 4 (x)/ 3 + 4S(x)µ/ 2 + 6σ 2 (x)µ 2 / + µ 4 = E( x 4 ) E[x 2 1 x2 ] = K(x)/ 2 + 2(+1) S(x)µ σ2 (x)µ σ 4 + µ 4 2 We ow have the compoets eeded to fd the sample cetral momets. A sample cetral momet s cetered ot aroud µ, where t would have a form lke (x µ) 2 /, but s cetered aroud x, lke (x x) 2 /. THE SAMPLE CENTRAL MOMENTS I all three cases, the sample cetral momet based o x ad the actual cetral momet based o µ are ot equal expectato. The sample mea x tself has some varace, so the fal 6

7 sample varace wll be a combato of the true populato varace ad the varace of the mea; smlarly for skew ad kurtoss. Let the sample cetral momets based o x be σ 2 x(x), S x (x), ad K x (x). The procedure for calculatg ths sample cetral momet s exactly as above: expad the square, cube, or quadratc, the take the expectato of each dvdual term. E [(x x) 2 ] = E[x 2 2x x + x 2 ] = E[x 2 x 2 ] = E [x 2 ] E [ x 2 ] = σ 2 (x) + µ 2 σ 2 (x)/ µ 2 = 1 σ2 (x) I the last step, the µ terms easly cacel out, but the remag expected value of σ 2 (x) dffers from the remag expected value of x 2 by that fracto of, ad that meas that there s the coveet ( 1)/ term whe they merge. We ca rewrte the fal form to reveal a smple relatoshp amog the varous types of varace. Because σ 2 ( x) = σ 2 (x)/, ths result becomes: σ 2 x(x) = σ 2 (x) σ 2 ( x). Now repeat the process for the skew. Here we beg to use the hybrd expressos above, such as that E[x x 2 ] = E[ x 3 ]. E [(x x) 3 ] = E[x 3 3x 2 x + 3x x 2 x 3 ] = E[x 3 ] 3E[x 2 x] + 2E[ x3 ] The expaso for each term the equato s gve at some pot above, ad each cossts of a sum of S, µσ 2, ad µ 3. To keep thgs orgazed, each row the table below s the expaso of oe of the three terms the above expresso, ad each colum gves the coeffcet o S, µσ 2, or µ 3 for the gve expresso. For example, the frst row ca be read as E(x 3 ) = 1S + 3µσ 2 + 1µ 3. term S µσ 2 µ 3 E(x 3 ) E(x 2 +2 x) 3/ E( x 3 ) 2/ 2 6/ 2 Wth terms eatly colums, you ca quckly verfy that the coeffcets o µσ 2 ad o µ 3 sum to zero. We are left wth: ( 2)( 1) S x (x) = S(x) Lke the varace, ths ca be expressed terms of S(x) ad S( x): 2 S x (x) = S(x) (3 2)S( x). 7

8 Tragcally, ay smple patter establshed by the varace ad skew does ot carry forward, ad the sample kurtoss s a mess. Expadg the quadratc gves us several terms to evaluate: E [(x x) 4 ] = E [x 4 ] E [4x 3 x] + E [6x2 x2 ] E [4x x 3 ] + E [ x 4 ]. Aga orgazg the expaso of the expectatos to a table of coeffcets: term K Sµ µ 2 σ 2 σ 4 µ 4 E(x 4 ) E(x 3 x) 4/ 4( + 3)/ 12( + 1)/ 4 +6E(x 2 x2 ) 6/ 2 12( + 1)/ 2 6( + 5)/ 6( 1)/ 2 6 4E(x x 3 ) + E( x 4 ) 3/ 3 12/ 2 18/ 9( 1)/ 3 3 You ca aga quckly verfy that the coeffcets the Sµ, µ 2 σ 2, ad µ 4 colums sum to zero. The remag colums gve us two rather upleasat terms the fal expected value of the sample kurtoss. E [(x x) 4 ] = ( 1)(2 3+3) 3 = ( 1) 3 K(x) + ( 1)(6 9) σ 4 (x) 3 (( )K(x) + (6 9)σ 4 (x)) Remarkably, ths mess ca also be expressed as a weghted sum of K(x) ad K( x): K x (x) = ( 2)2 K(x) + (2 3)K( x). 2 σ 2 x(x) = 1 σ2 (x) = σ 2 (x) σ 2 ( x) S x (x) = ( 2)( 1) S(x) 2 = S(x) (3 2)S( x) K x (x) = ( 1) 3 (( )K(x) + (6 9)σ 4 (x)) = ( 2)( 2) K(x) + (2 3)K( x) 2 As a bous calculato, let us calculate the expected value of the square of the varace, σ 4 x(x): 8

9 E [σ 4 x(x)] = E [(x x) 2 (x j x) 2 ] = E [ 1 (x x) 2 1 [ (xj x) 2] = 1 E 2 j (x x) 2 (x ] j x) 2 = 1 E [ [ ] 2 (x x) 4 ] + 1 E 2 j (x x) 2 (x j x) 2 ] = 1 2 [K x (x) + ( 1) E [σ2 x(x)σ 2 x(x)] 2 = K x(x) + ( 1)3 3 [σ 2 (x)] 2 We are terested the true varace, based o µ, but µ s typcally ot kowable from fte data, so we must make do wth estmates based o statstcs of the data, lke x. We just saw that σ 2 x(x), S x (x), ad K x (x) are based estmators of ther respectve true momets, so they wo t work wthout modfcato. Fortuately, we kow exactly how large the bas s, ad so ca produce a ubased estmate of the the true varace, skew, or kurtoss by smply movg the true momet to the left-had sde of above equatos. SAMPLE ESTIMATES OF THE TRUE MOMENTS To make the computatos clearer (ad the cocepts a lttle more opaque), I preset the estmates of the cetral momets fully wrtte out. Because we are movg from the expected value of a potetally calculable value to a operatoal estmate, we put a hat o the cetral momet to dcate that ths s a sample-based estmator. For the varace ad skew, we ca easly produce a estmate that does ot deped o kowledge of the true value of µ, but uses oly formato from the data set at had. The value of K x (x) s expressed terms of the true fourth ocetral momet ad a σ 4 term; above, the expectato of σ 4 s expressed terms of K x (x) ad the estmable-from-data value σ 2 x(x). We ca use these equatos to solve for a estmator of K(x) usg oly estmable values, but t s ot pretty (whch s why may authors, such as Cramér [1946, p 349], start to lea o Taylor expasos aroud here.) ˆσ 2 (x) = 1 1 (x x) 2 Ŝ(x) = ( 1)( 2) (x x) 3 ˆK(x) = 2 [[(( ( 1) 3 ( 2 3+3) 1)2 + (6 9))] K x (x) (6 9)σ 4 x(x)] The sample estmate of the populato varace should be famlar to you from textbooks, usg the defto of varace but wth x replacg µ ad dvdg the sum by 1 stead of. You ca see that t derves rather smply from the fact that the expected value of the x-based varace ad the µ-based varace dffer by σ 2 ( x) = σ 2 (x)/; from there t s smple algebra to get the commo form. 9

10 Especally amog ANOVA users, the varace formula s ofte explaed usg a degrees-offreedom story: 3 gve the average x, there are oly 1 free elemets the data, because the last ca be determstcally calculated from the frst 1 ad x. That story s a useful fcto, that t makes a great deal of sese the ANOVA cotext, ad saves the trouble of the algebra above. However, f t were really just a story of coutg degrees of freedom, the we are hardpressed to expla the forms for the ubased estmate of the sample skew or kurtoss. Fortuately, researchers make use oly of aalyss of varace ad ot aalyss of skew or kurtoss (ANOSK ad ANOKU?), ad so the useful fcto of the degrees-of-freedom story s cosstet ad a help. REFERENCES Robert C Blattberg ad Ncholas J Goedes. A comparso of the stable ad studet dstrbutos as statstcal models for stock prces. The Joural of Busess, 47(2): , George Casella ad Roger L Berger. Statstcal Iferece. Duxbury Press, Harald Cramér. Mathematcal Methods of Statstcs. Prceto Uversty Press, Staley J Ko. Models of stock returs a comparso. The Joural of Face, 39(1): , Beot Madelbrot. The varato of certa speculatve prces. The Joural of Busess, 36(4): , George W Sedecor ad Wlla G Cochra. Statstcal Methods. Iowa State Uversty Press, 6th edto, See, e.g., Sedecor ad Cochra [1976, p 45]. 10