Supplemental notes for topic 9: April 4, 6

Transcription

1 Sta-30: Probablty Sprg 017 Supplemetal otes for topc 9: Aprl 4, Polyomal equaltes Theorem (Jese. If φ s a covex fucto the φ(ex Eφ(x. Theorem (Beaymé-Chebyshev. For ay radom varable x, ɛ > 0 P( x ɛ Ex ɛ. Ex E(x I { x ɛ} ɛ P( x > ɛ. Theorem (Markov. For ay radom varable x, ɛ > 0 ad P( x ɛ Eeλx e λɛ P( x ɛ f λ<0 e λɛ Ee λx. P(x > ɛ = P(e λx > e λɛ Eeλx e λɛ. 9. Expoetal equaltes For the sums or averages of depedet radom varables the above bouds ca be mproved from polyomal 1/ɛ to expoetal ɛ. Theorem (Beet. Let x 1,..., x be depedet radom varables wth Ex = 0, Ex = σ, ad x M. For ɛ > 0 ( P x > ɛ e σ φ( M ɛm σ, where φ(z = (1 + z log(1 + z z. We wll prove a boud o oe-sde of the above theorem P x > ɛ. 9-1

2 9- Lecture 9: Aprl 4, 6 P x > ɛ e λɛ Ee λ x = e λɛ Π Ee λx = e λɛ (Ee λx. Ee λx (λx k = E = k! k=0 k=0 λ k = 1 + k! Ex x k 1 + = 1 + σ M The last le holds sce 1 + x e x. k= λ k M k k= k! k= λ k Exk k! λ k k! M k σ = 1 + σ M (eλm 1 λm e σ M (eλm λm 1. Therefore, P x > ɛ e λɛ e σ M (eλm λm 1. (9.1 We ow optmze wth respect to λ by takg the dervatve wth respect to λ 0 = ɛ + σ M (MeλM M, e λm = ɛm σ + 1, λ = 1 M log ( 1 + ɛm σ The theorem s prove by substtutg λ to equato (9.1. The problem wth Beet s equalty s that t s hard to get a smple expresso for ɛ as a fucto of the probablty of the sum exceedg ɛ. Theorem (Berste. Let x 1,..., x be depedet radom varables wth Ex = 0, Ex = σ, ad x M. For ɛ > 0 ( P x > ɛ e ɛ σ + 3 ɛm. Take the proof of Beet s equalty ad otce φ(z z + 3 z..

3 Lecture 9: Aprl 4, Remark. Wth Berste s equalty a smple expresso for ɛ as a fucto of the probablty of the sum exceedg ɛ ca be computed x 3 um + σ u. where we ow solve for ɛ P x > ɛ e ɛ σ + 3 ɛm = e u, u = ɛ σ + 3 ɛm. ɛ 3 ɛm σ ɛ = 0 ad Sce a + b a + b So wth large probablty If we wat to boud we cosder Therefore ad Smlarly, I the above boud ɛ = 1 3 um + u M + σ 9 u. ɛ = 3 um + σ u. x 3 um + σ u. 1 f(x Ef(x f(x Ef(x M. (f(x Ef(x 4 3 um + σ u 1 Ef(x 1 f(x Ef(x 4 um 3 + σ u. σ u f(x 4 um 3 + σ u. whch mples u σ 8M 4uM ad therefore 1 σ u f(x Ef(x for u σ,

4 9-4 Lecture 9: Aprl 4, 6 whch correspods to the tal probablty for a Gaussa radom varable ad s predcted by the Cetral Lmt Theorem (CLT Codto that lm σ. If lm σ = C, where C s a fxed costat, the 1 f(x Ef(x C whch correspods to the tal probablty for a Posso radom varable. We ow look at a eve smpler expoetal equalty where we do ot eed formato o the varace. Theorem (Hoeffdg. Let x 1,..., x be depedet radom varables wth Ex = 0 ad x M. For ɛ > 0 P ( x > ɛ e ɛ M. P x > ɛ e λɛ Ee λ x = e λɛ Π Ee λx. It ca be show E(e λx e λ M 8. The boud s prove by optmzg the followg wth respect to λ e λɛ Π e λ M 8. Applyg Hoeffdg s equalty to we ca state that wth probablty 1 e u 1 1 f(x Ef(x f(x Ef(x Mu, whch s a sub-gaussa as the CLT but wthout the varace formato we ca ever acheve the 1 rate we acheved whe the radom varable has a Posso tal dstrbuto. We wll use the followg verso of Hoeffdg s equalty later lectures o Kolmogorov chag ad the Dudley s etropy tegral. Theorem (Hoeffdg. Let x 1,..., x be depedet radom varables wth P(x = M = 1/ ad P(x = M = 1/. For ɛ > 0 ( P x > ɛ e ɛ M.

5 Lecture 9: Aprl 4, P x > ɛ e λɛ Ee λ x = e λɛ Π Ee λx. Optmze the followg wth respect to λ E(e λx = 1 eλm + 1 e λm, 1 eλm + 1 e λm = k=0 (M λ k (k! e λɛ Π e λ M. e λ M. 9.3 Strog law of large umbers Lemma 9.1 (Borel-Catell. Cosder a set of evets {A : 1} a probablty space. If the P(A occurs ftely ofte = 0. P(A < =1 The dea behd the Borel-Catell Lemma s that f a evet happes ftely ofte ad the state space s fte the the probablty of the evet heppeg s zero. So we ca say somethg happes wth probablty zero. We ow use the Borel-Catell Lemma ad the weak law of large umbers to prove the strog law of large umbers. d Cosder a fte sequece of Beroull radom varable X 1,..., X Be(p. S P(lm = p = 1. We wat to show that S If we ca show that P( S > ε.o. = 0 the P(lm = p = 1 because these are complemetary evets. To make otato easer we cosder the case where p = 1/ the geeralzato s straghtforward. Cosder the evet A = {ω Ω : S ε}, We have a expoetal equalty for the above evet sce we kow by Hoeffdg s equalty ( P x > ε e ε 1, P ( x > ε e ε.

6 9-6 Lecture 9: Aprl 4, 6 So we ow check f for all ε > 0 P(A e ε <. =1 =1 The sum of expoetal tals s bouded so Borel-Catell holds ad we see that for Beroull radom varables the strog law of large umbers holds.