508-B (Statistics Camp, Wash U, Summer 2016) Asymptotics. Author: Andrés Hincapié and Linyi Cao. This Version: August 9, 2016

Transcription

1 Asymtotics Author: Anrés Hincaié an Linyi Cao This Version: August 9, 2016

2 Asymtotics 3 In arametric moels, we usually assume that the oulation follows some istribution F (x θ) with unknown θ. Knowing θ yiels knowlege about the entire oulation. samle. Here comes estimators. Our task is then to estimate θ from our An estimator is a function that mas from the samle sace to a set of samle estimates. With ranom samling, an estimator is a r.v., while an estimate (from a given samle) is a realization of the estimator. Now comes the question of what a goo estimator is. To answer that, first we nee to introuce some concets in asymtotic analysis.

3 Asymtotics 4 1 Convergence Concets What haens if we have large samles? Useful aroximations for finite-samle case (exressions ten to be simler) Sequence of Ranom Variables: collection of ranom variables T n inexe by (samle size) n = 1, 2, with CDF G n (t) = P r(t n t). EXAMPLE: Samle mean X n

4 Asymtotics Convergence in Probability A sequence of ranom variables X 1, X 2... converges in robability to a ranom variable X if, for every ɛ > 0 or equivalently lim P r( X n X ɛ) = 0 n lim P r( X n X < ɛ) = 1 n X 1, X 2... are not necessarily i.i.. It is written as X n X or lim Xn = X Imortant concets: o(1) an O(1)

5 Asymtotics Weak Law of Large Numbers Let {X 1, X 2...} be a set of i.i.. ranom variables with EX i = µ V arx i = σ 2 <, then lim X n = µ Proof: Use Chebyshev s Inequality Remark: There are many versions of WLLN. They are ifferent in reliminary conitions (all sufficient), but all result in convergence in robability.

6 Asymtotics 7 If X n a an Yn b then a) X n ± Y n a ± b b) X n Y n a b c) X n /Y n a/b if b 0

7 Asymtotics 8 If X n c an g is a continuous function at c, g (Xn) g (c)

8 Asymtotics Convergence in Distribution A sequence of ranom variables {X n } converges in istribution to a ranom variable X if lim F X n n (x) = F X (x) At all oints x where F X (x) is continuous. We write X n X 1.4 Almost sure convergence A sequence of ranom variables {X n } converges in istribution to a ranom variable X if: a.s. P r[ω Ω : lim n X n (ω) = X(ω)] = 1. We write X n X a.s. Strong Law of Large Numbers: Given the same conitions as we have in WLLN, Xn µ. Har to rove. 1.5 Which convergence concet is stronger? X n a.s. X X n X Xn X X n C Xn C Proof. htts://en.wikieia.org/wiki/proofs_of_convergence_of_ranom_variables

9 Asymtotics 10 2 Other Asymtotics 2.1 Central Limit Theorem (CLT) Let {X 1, X 2...} be a set of i.i.. ranom variables. Let EX i = µ an V arx i = σ 2 <. Let F n (x) be the CDF of the r.v. Z n = n ( X n µ) σ. Then, for any x lim F n(x) = n x 1 2π e t2 /2 t We can also write Z n N(0, 1). Aroximation roceure. Use the limiting istribution of Z n to aroximate its exact CDF for a given samle size n. Problematic if n is too small. Remark: There are many versions of CLT.

10 Asymtotics 11 Aroximating the samling istribution of Z n with its limiting istribution amounts to aroximating the cf of X by the cf of a N(µ, σ 2 /n). F Xn (c) = Pr( X n c) ( ) n (c µ) = Pr Z n σ ( n ) (c µ) = F Zn σ F n (c) Φ ( n(c µ) ) σ is equivalent to treating Xn as a N(µ, σ 2 /n). We say N(µ, σ 2 /n) is X n s asymtotic istribution an write X n a N(µ, σ 2 /n) µ is the exectation of Xn as well as its asymtotic exectation. σ 2 /n is X n s asymtotic variance.

11 Asymtotics Slutsky s Theorem Assume X n X an Yn c, where c is a constant. Then a) X n + Y n X + c b) X n Y n c X c) X n /Y n X/c, c 0

12 Asymtotics 13 EXAMPLE n ( X n µ) S n N (0, 1) First, consier the biase samle variance S 2 n = i (X i X n) 2 n σ 2 Write ( (Xi µ) ( Xn µ )) 2 S 2 n = n i = (X i µ) 2 + ( Xn µ ) 2 n i = M 2 + ( Xn µ ) 2 = h ( M 2, X ) n By LLN i (X i µ) 2 n σ 2

13 Asymtotics 14 By continuity of h, S 2 n σ 2 + (µ µ) = σ 2 By continuity of, S n σ By Slutsky c) n ( X n µ) n ( Xn µ ) S n = σ }{{} S n }{{ σ } 1 N(0,1) N (0, 1)

14 Asymtotics Delta Metho Let Y n be such that a n (Y n θ) Y, θ R where {a n } is a sequnce of ositive numbers an a n. Let h be a continuously ifferentiable at θ, an h (θ) is not zero. Then a n (h (Y n ) h (θ)) h (θ) Y Proof: First we know that it must be Y n θ. Then using Taylor s theorem an Slutsky s Theorem...

15 Asymtotics 16 Secial case: If n(y n θ) N(0, σ 2 ) an h is continuously ifferentiable at θ, an h (θ) is not zero, then n(h(yn ) h(θ)) N(0, [h (θ)] 2 σ 2 ) Proof: Uses Taylor exansion aroun X n = θ, or Mean Value Theorem

16 Asymtotics 17 EXAMPLE: Suose n(y n θ) N(0, σ 2 ) an we are estimating Construct an asymtotic istribution for θ 2 What if θ = 0?