:46 /6/ TOPIC Momen generaing funcions The n h momen of a random variable X is EX n if his quaniy exiss; he momen generaing funcion MGF of X is he funcion defined by M := Ee X for R; he expecaion in exiss since e X is a nonnegaive bu may be + This secion shows how he MGF generaes he momens, and how he momens can in cerain cases be used o recover he MGF Noe ha if Y = a + bx for numbers a and b, hen M Y := Ee Y = Ee a e bx = e a Ee bx = e a M X b for all R Consequenly, given a locaion/scale family of disribuions, i suffices o compue he MGF for one simple member in he family; he MGFs of he oher members follow from Example The mgfs in his example are illusraed in Figure a Suppose X N, Then M = Ee X = = e / e x e x / π dx e /+x x / π dx = e / In paricular, { R : M < } = R b Suppose X Gammar, Then M = Ee X = = e x xr e x Γr dx r x r e x r dx = Γr e x / π dx = e / 3 r 4 for < ; however M = for In paricular, { R : M < } =, c Suppose X UF,, wih densiy fx = / + x for x and oherwise Then M = Ee X = e x + x dx = e e y e E, if <, y dy =, if =, 5, if > ; here E n u := e uy /y n dx is he value of he so-called exponenial inegral of order n a u In paricular, { R : M < } =, ] In Maple, E n u is Ein,u d Suppose X Cauchy, Then M = Ee X = Here { R : M < } = {} e x π + x dx = These examples moivae he following resul {, if =,, oherwise Theorem Le M be he MGF of a random variable X Then B := { R : M < } 7 is a perhaps degenerae inerval conaining Moreover M is convex on B Proof The exponenial funcion x e x is convex on R Hence for and in R and α, we have e α + α Xω αe Xω + αe Xω for all ω = E e α + α X E αe X + αe X = M α + α αm + αm 8 Consequenly if M < and M <, hen M < for all [, ]; i follows ha B is an inerval B conains because M = Ee X = Ee = 8 says ha M is convex on B 6
Figure : Graphs of M = Ee X for he random variables X in Example M 7 5 3 M = e / X N, M = X UF, { e E,, > M 3 M = X Γr, for r = { r, <, 3 X Cauchy, asympoe as The derivaives of he MGF Le X be a random variable wih MGF M Le B = { : M < } be he inerval of finieness of M, and le B be he inerior of B Wih luck, we should have d d M = d d EeX? = E d d ex = E Xe X for B The following heorem assers ha his is correc Theorem Le X, M, B, and B be as above, and suppose ha B is nonempy Then for each B, M is differeniable a, he random variable Xe X is inegrable, and M = E Xe X 9 Proof Le B Le ɛ > be such ha he poins ɛ and +ɛ are boh in B Le,, be poins in B such ha n as n and < n ɛ for all n For each n where M n M n = EenX Ee X n = EY n Y n = e nx e X n As n, we have Y n d d ex = = Xe X := Y M M According o he DCT, provided here exiss an inegrable rv D such ha Y n D for all n, 3 3 he Y n s and Y will be inegrable and we will have EY n EY This is exacly he conclusion we wan: i implies ha Xe X is inegrable and ha M = EXe X 4
:46 /6/ : D is inegrable and Y n D for all n ± ɛ B We need o produce a D saisfying heorem, e nx e X = n Xe X By he mean value for some poin beween and n ; depends on n and he implici sample poin ω, bu we don indicae ha in he noaion Hence Y n = enx e X = X e X X e ɛx + e +ɛx n Now Thus We have X = ɛ ɛ X ɛ + ɛ X + ɛ X = ɛ eɛ X ɛ e ɛx + e ɛx Y n ɛ! + ɛ3 X 3 3! e ɛx + e ɛx e ɛx + e +ɛx D := ɛ e ɛx + e X + e +ɛx + ED = ɛ M ɛ + M + M + ɛ < by he choice of ɛ Hence D is he inegrable dominaor we need Ieraing his argumen do i! gives Theorem 3 Le X, M, B, B be as in Theorem, and suppose ha B is nonempy Then M is infiniely differeniable on B For each B and each k N, X k e X is inegrable and M k = E X k e X In paricular if B, hen for all k N, X k is inegrable and M k = EX k 5 B = M k exiss and equals EX k, for all k N Example Suppose X Gammar, By 4 M = r for < Consequenly B =, and EX k = M k = dk r = d k r+ = rr + r + r + k r+k = = rr + r + r + k = Γr + k 3 Γr A relaed bu differen argumen see Exercise 5 gives his companion o Theorem 3: Theorem 4 Le X, M, B, and B be as in Theorem 3 Suppose ha / B, bu ha here exiss an ɛ > such ha [, ɛ B Then for all k N, X k is quasi-inegrable and EX k = M k M k M k + + := lim 4 M k + is called he k h righ-hand derivaive of M a ; by definiion M + = M = A similar resul holds for he lefhand derivaives M k of M a when ɛ, ] B Example 3 Suppose X UF, as in Example c Since B =, ], we have M k = EX k = for all k N x k dx = + x 6
The series expansion of he MGF Consider he following heurisic M = Ee X = E k X k?= EXk k 5 This couldn be rue in general, because he power series on he RHS converges o a finie value in an inerval which is symmeric abou, whereas he domain of finieness of M can be an asymmeric inerval abou Theorem 5 Le X be a random variable wih MGF M Then he following wo saemens are equivalen: S X k is inegrable for all k N and he series k EX k / has radius of convergence R > S R := sup{ > : M < and M < } > If S and S hold, hen S3 R = R, and S4 M = k EX k / for all such ha M < and M <, and in paricular for all such < R = R Example 4 a Suppose X Gammar, Then B =, = R = = R = = 5 holds for < Noe ha he LHS of 5 is defined and finie for <, even hough he series on he RHS does no converge for such s b Suppose X N, Then B =,, so R = R = Consequenly X k is inegrable for all k N and = M = / e n= EXn n n! = / k = k! k k k! 7 S: X k is inegrable for all k N and he series k EX k / has radius of convergence R > S: R := sup{ > : M < and M < } > S3: R = R S4: M = EXk k / for < R = R By he uniqueness of he coefficiens of a power series, EX n = for n odd, whereas EX k = k! k = 3 k = 3 5 k 6 4 6 k is he produc of he odd posiive inegers less han k In paricular, EX =, EX 6 = 3 5 = 5, EX 4 = 3 = 3, EX 8 = 3 5 7 = 5 Remember hese formulas! c Suppose S holds Then M = EXk k / for all wih < R = R I follows from he heory of power series ha M is infiniely differeniable in he inerval R, R and ha M k = EX k for all k N This gives anoher proof of Proof of Theorem 5 Sep : If and M± <, hen X k is inegrable for all k and M = EXk k / Indeed, we have E k X k = Ee X Ee X + e X = M + M < In paricular, E k X k / < = E X k < = EX k exiss and is finie, for all k N Moreover by Fubini s Theorem M = E k X k = 8 k EX k
:46 /6/ S: X k is inegrable for all k N and he series k EX k / has radius of convergence R > S: R := sup{ > : M < and M < } > S3: R = R S4: M = EXk k / for < R = R Sep : S = S and R R Suppose < < R Then M < and M < By Sep, X k is inegrable for all k N and M = k EX k / Since his series converges for all wih < R, is radius of convergence R mus be a leas R > Sep 3: S = S and R R Suppose < R Then M + M = Ee X + Ee X = Ee X + e X X k = E = E = <, n= k X k k! k EX k k! n EX n n! + X k by Fubini he las sep holding since a power series converges absoluely in he inerior of is inerval of convergence I follows ha R R > Sep 4: S and/or S = S3 This follows from Seps and 3 Sep 5: S and/or S = S4 This follows from Sep 9 Exercise Suppose X,, X n are independen random variables Pu S n = X + + X n Show ha Ee S n = n for all R i= EeX i 7 Exercise Suppose U Uniform, Show ha Ee U = e 8 for all evaluae he RHS via l Hospial s rule for = Use 8 and or S4 o calculae he mean and variance of U Exercise 3 Suppose X Poissonλ, so P [X = k] = e λ λ k / for k =,,, Show ha Ee X = exp λe 9 for all R Use 9 and or S4 o calculae he mean and variance of X Exercise 4 Le X be a rv wih densiy fx = e e x e x for x R This disribuion arises in he heory of exreme values Show ha Ee X = Γ for <, and = oherwise Express he mean and variance of X in erms of he Gamma funcion and is derivaives Exercise 5 a Le h: R R be he funcion which maps u o e u /u Show ha h is nondecreasing in u b Use he MCT o prove 4 for he case k =
Exercise 6 As in Example c, suppose X UF, and pu M = Ee X Show ha M u u = log/u + logy e y dy + o as u [Hin: Wrie M u as M M u, inegrae by pars, make he change variables y = u + x, and inegrae by pars again] Exercise 7 Suppose X has finie momens of all orders Show ha he radius R of convergence of he series in 5 is given by he formula /R = lim sup n n EX n /n! = lim sup n e n EX n /n = lim sup n e n E X n /n Hin: According o Sirling s formula n! πn n n e n as n ] Exercise 8 Suppose X Gammar, Compue he momens of X by direc inegraion and use 5 o compue Ee X for s wihin he inerval of convergence of he series Le X be a random variable wih MGF M The funcion K defined by K = log M is called he cumulan generaing funcion of X; i s derivaives a, namely κ r := K r, r =,,,, are called he cumulans of X or of M; he cumulans exis if M is finie in an open inerval conaining = Exercise 9 Le X be a random variable having cumulans κ, κ, κ, Show ha κ =, κ = EX, and κ = VarX 3 The exercises below explore exponenial families, a opic of impor hroughou saisical heory The exercises are wordy, bu ex- cep for Exercises 5 and 6, and he las pars of Exercises, 7, and 8, he answers should be almos immediae The following seup is used hroughou exercises 8 Le f be a piecewise coninuous densiy on he real line whose MGF M = e x fx dx is finie for all s in some nonempy open inerval Denoe he larges such inerval by l, r, and noe ha l and r l and r may be infinie Le K = log M be he corresponding cumulan generaing funcion Consruc a one-parameer family {f θ } r <θ< r of densiies relaed o f by exponenial iling, as follows: for each θ l, r, se f θ x = e θx fx/mθ = e θx Kθ fx 4 Exercise Show ha: a f θ is a densiy; b f θ has MGF M θ = M + θ/mθ, finie for l θ, r θ Exercise a Le fx = exp x // π for x real Wha are M, l, and r? Name he densiy corresponding o f θ b Repea a wih fx = exp x for x nonnegaive c Repea a wih fx = / π coshx for x real [Hin for c: coshx := e x +e x / There is no common name for f θ Show ha f θ is he densiy of logf, where F has an unnormalized F disribuion; give he degrees of freedom] Exercise Show ha in general he cumulan generaing funcion K θ of f θ saisfies K θ = K + θ Kθ, 5 d d K θ = K + θ, 5 d r d r K θ = K r + θ, 5 3
:46 /6/ K and K r denoing derivaives of he funcion K In hese formulas, l θ, r θ Exercise 3 Denoe he r h cumulan of f θ by κ r θ so κ r θ = K r θ Show ha κ θ = K θ, κ θ = K θ, 6 κ r θ = K r θ = dj dθ j κ r jθ, 6 he las equaion holding for any r and j < r Exercise 4 Show ha κ θ > and deduce ha on he inerval l, r, K is sricly increasing and K is sricly convex [Hin: Use he resul of Exercise 9] Exercise 5 Le a = inf{ x : fx > } and b = sup{ x : fx > } be respecively he smalles and larges poins of suppor of f Show ha provided moreover provided κ l + := lim θ κ θ = a 7 l = or l > and M l + = ; κ r := lim θ r κ θ = b 7 r = or r < and M r = [Hins: You need only prove 7, because he argumen for 7 is similar In he case ha r <, firs argue ha b = In he case ha r =, firs argue ha if fx > for all x in some nonempy inerval α, β, hen here exiss a finie number c such ha K c + α for all ] 3 Exercise 6 Show by example ha 7 need no hold Exercise 7 Suppose 7 holds Show ha for each ξ a, b, he equaion κ θ = ξ 8 has a unique roo θ l, r, and hence ha corresponding o each such ξ here is a unique densiy in he family {f θ } ha has mean ξ This resul is of cenral imporance o he saddlepoin approximaions o which we shall reurn Solve 8 in closed form for he densiies in Exercise [Hin: For he hird densiy in Exercise, you may find he formula ΓzΓ z = π/ sinπz helpful The formula is valid for < z < ; you do no have o prove i] Exercise 8 Everyhing in he preceding discussion applies as well o discree disribuions ha are no concenraed on a single poin, wih he undersanding ha f is now o be inerpreaed as a frequency funcion aka probabiliy mass funcion Le f = bn, / be he Binomial frequency funcion for n rials and success probabiliy / Find M, l, and r, and idenify he family {f θ } as he binomial family bn, p, < p < How does he so-called canonical parameer θ correspond o he so-called naural parameer p? Find dp/dθ as a funcion of p To avoid noaional confusion, le κ r,p = κ r θ refer o he cumulans of he binomial, considered as a funcion of p and θ, respecively From he general formula 6 on cumulan recursion and he chain rule, derive he recurrence relaion κ r,p = pq d dp κ r,p 9 Use 9 and Maple o find he firs 8 binomial cumulans 4