STATISTICAL LABORATORY, May 18th, 2010 CENTRAL LIMIT THEOREM ILLUSTRATION

Transcription

1 STATISTICAL LABORATORY, May 18th, 2010 CENTRAL LIMIT THEOREM ILLUSTRATION Mario Romanazzi 1 BINOMIAL DISTRIBUTION The binomial distribution Bi(n, p), being the sum of n independent Bernoulli distributions, falls within the framework of the CLT. When the parameter n (sample size) is large, it can be approimated by a normal distribution with parameters µ = np and standard deviation σ = (np(1 p)) 1/2. if p = 1/2, the binomial distribution is symmetric and the rate of convergence is high: the approimation is already good for n equal to The result is illustrated below with n = 50 and p = 1/2. > n <- 50 > p <- 1/2 > dma <- ma(dbinom(0:n, n, p)) > plot(0:n, dbinom(0:n, n, p), type = "h", lwd = 5, ylim = c(0, + dma), lab = "", ylab = "PDF", main = "Binomial Distribution Bi(n=50, p=1/2)") > plot(function() dnorm(, mean = n * p, sd = sqrt(n * p * (1 - + p))), 0, n, lwd = 2, col = "red", add = TRUE) > plot(0:n, pbinom(0:n, n, p), type = "s", lwd = 2, lab = "", + ylab = "CDF", main = "Binomial Distribution Bi(n =50, p=1/2)") > plot(function() pnorm(, mean = n * p, sd = sqrt(n * p * (1 - + p))), 0, n, lwd = 2, col = "red", add = TRUE) Binomial Distribution Bi(n=50, p=1/2) Binomial Distribution Bi(n =50, p=1/2) PDF CDF

2 2 POISSON DISTRIBUTION 2 2 POISSON DISTRIBUTION The Poisson distribution, when the parameter λ is large can be approimated by a normal distribution with parameters µ = λ and standard deviation σ = λ. The result is not surprising if we recall the relationship of the Poisson distribution with the binomial. The illustration below assumes λ = 20. > lambda <- 20 > a1 <- round(lambda * sqrt(lambda)) > a2 <- round(lambda * sqrt(lambda)) > dma <- ma(dpois(a1:a2, lambda)) > plot(a1:a2, dpois(a1:a2, lambda), type = "h", lwd = 5, ylim = c(0, + dma), lab = "", ylab = "PDF", main = "Poisson Distribution (lambda = 20)") > plot(function() dnorm(, mean = lambda, sd = sqrt(lambda)), + a1, a2, lwd = 2, col = "red", add = TRUE) > plot(a1:a2, ppois(a1:a2, lambda), type = "s", lwd = 2, lab = "", + ylab = "CDF", main = "Poisson Distribution (lambda = 20)") > plot(function() pnorm(, mean = lambda, sd = sqrt(lambda)), + a1, a2, lwd = 2, col = "red", add = TRUE) Poisson Distribution (lambda = 20) Poisson Distribution (lambda = 20) PDF CDF EXPONENTIAL DISTRIBUTION The sum of n independent eponential distributions with parameter λ, when n is large, can be approimated by a normal distribution with parameters µ = n/λ and standard deviation σ = n/λ. This is easily checked because the sum of n independent eponential distributions is a gamma random variable. The illustration below assumes λ = 1 and n = 40. > lambda <- 1 > n <- 40 > mn <- n/lambda > sn <- sqrt(n)/lambda > a1 <- mn * sn > a2 <- mn * sn

3 4 UNIFORM DISTRIBUTION 3 > plot(function() dgamma(, shape = n, rate = lambda), a1, a2, + lwd = 2, lab = "", ylab = "PDF", main = "Sum of 40 Eponential Distributions (lambda = 1)") > plot(function() dnorm(, mean = mn, sd = sn), a1, a2, lty = "dashed", > plot(function() pgamma(, shape = n, rate = lambda), a1, a2, + lwd = 2, lab = "", ylab = "CDF", main = "Sum of 40 Eponential Distributions (lambda = 1)") > plot(function() pnorm(, mean = mn, sd = sn), a1, a2, lty = "dashed", Sum of 40 Eponential Distributions (lambda = 1) Sum of 40 Eponential Distributions (lambda = 1) PDF CDF UNIFORM DISTRIBUTION The sum of n independent uniform distributions R(a, b), when n is large, can be approimated by a normal distribution with parameters µ = n(a + b)/2 and standard deviation σ = n/12(b a). A fairly good approimation is already apparent with n = 12. The illustration below deals with the R(0, 1) distribution and n = 12. Since neither the pdf nor the cdf of the sum of (independent) uniform distributions admit eplicit representations, here we simulate a very large sample from X 1 + X X 12, where the X i are independent R(0, 1) random variables. > set.seed( ) > matr <- matri(runif(24000), 2000, 12) > vett <- rowsums(matr) > hist(vett, freq = FALSE, breaks = 23, lab = "", ylab = "Simulated PDF", + lwd = 2, main = "Sum of 12 R(0, 1) Distributions") > plot(function() dnorm(, mean = 6, sd = 1), 0, 10, lty = "dashed", > plot.ecdf(vett, pch = 20, lab = "", ylab = "Simulated CDF", + lwd = 2, main = "Sum of 12 R(0, 1) Distributions") > plot(function() pnorm(, mean = 6, sd = 1), 0, 10, lty = "dashed",

4 5 OTHER APPLICATIONS 4 Sum of 12 R(0, 1) Distributions Sum of 12 R(0, 1) Distributions Simulated PDF Simulated CDF OTHER APPLICATIONS E1 A fair coin is tossed 120 times. What is the probability that the % of heads is greater than 55%? Solution. Let X i Bi(1, 1/2) be the Bernoulli distribution corresponding to the i-th trial, i = 1,..., 120. Then the total number of heads is S 120 = X i Bi(120, 1/2) and the relative frequency of heads is X 120 = S 120 /120. Therefore and the required probability is X 120 > 0.55 S 120 > = 66 P (X 120 > 0.55) = P (S 120 > 66) = 120 =67 ( ) 120 ( 1 2 )120. The previous probability can be obtained through the R function pbinom. > 1 - pbinom(66, 120, 1/2) [1] An approimate solution is given by the CLT. The first step is to derive epectation and standard deviation of S 120. E(S 120 ) = np = = 60; SD(S 120) = np(1 p) = The second step is to derive the standard value corresponding to 66. (66 E(S 120 ))/SD(S 120 ) = (66 60)/ The final step is to derive the normal approimation of the binomial CDF. P (S 120 > 66) = 1 P (S ) = 1 P (S 120,ST 1.187) 1 F N(0,1) (1.187). The result can be computed through the R function pnorm.

5 5 OTHER APPLICATIONS 5 > 1 - pnorm(1.187) [1] The accuracy of the approimation is good. E2 The time (minutes) spent by a teacher to check a written eamination is a random variable X whose distribution is eponential with rate parameter λ = 1/5 minutes. 1. What is the probability that more than 10 minutes are needed to check an eamination? 2. Suppose that 80 written eaminations have to be checked. What is the epected total time to perform the job? 3) What is the probability that the total time is is between 6 and 7 hours? Solution. 1. P (X > 10) = 1 F X (10) = ep( 2) The total time S 80 is the sum of the times needed to check each eamination. Writing X i for the time required by the i-th eamination, i = 1,..., 80, S 80 = 80 i=1 X i. The standard model is assumed, that is, the random variables X i are independent and have the same eponential distribution Ep(λ = 1/5). It follows E(S 80 ) = ne(x) = 80 5 = 400; SD(S 80 ) = nsd(x) = We use the normal approimation allowed by the CLT, even though we can not epect a very good accuracy here (the eponential distribution is far from the normal shape). Si hours correspond to 360 minutes and seven hours correspond to 420 minutes. The standardized values are ( )/ , ( )/ Finally, P (360 < S 80 < 420) = F S80 (420) F S80 (360) = F S80,ST (0.447) F S80,ST ( 0.894) F N(0,1) (0.447) F N(0,1) ( 0.894) We use R to derive the approimate probability and its true value (from a gamma distribution with shape parameter n equal to 80 and rate λ = 1/5. > pnorm(420, mean = 400, sd = 20 * sqrt(5)) - pnorm(360, mean = 400, + sd = 20 * sqrt(5)) [1] > pgamma(420, shape = 80, rate = 1/5) - pgamma(360, shape = 80, + rate = 1/5) [1] The CLT approimation shows an error of about 1%. E3 A drunkard eecutes a random walk (see Rice, p. 140, E. C, for details) in the following way. Each minute he takes a step north or south, with probability 1/2 each, and his successive directions are independent. His step length is 50 cm. 1) Use the CLT to approimate the probability distribution of his location after 1 hour. Where is he most likely to be? 2) What is the probability that after 1 hour the distance from the starting point is greater than 10 metres? (Rice, 5.13) Solution. Suppose, for the sake of simplicity, that the initial location is the origin of the real line. The drunkard s location after n minutes can be represented as the sum S n = n i=1 X i, where X i is the random variable describing the i-th movement. For i = 1,..., n, X i can assume the values ±0.5 metres with equal probabilities and they are independent. It is easily checked that E(X i ) = 0 and SD(X i ) = 1/2.

6 5 OTHER APPLICATIONS 6 1. The previous discussion implies E(S n ) = ne(x i ) = 0; SD(S n ) = nsd(x) = 1 2 n. By the CLT, the location after 1 hour, S 60 = 60 i=1 X i, is approimately a normal random variable, centered at the origin with standard deviation SD(S 60 ) = 60/ Hence, the most probable location is (a neighbourhood of) the origin! 2. The required probability is P (S 60 < 10) + P (S 60 > 10) = 2F S60 ( 10) = 2F S60,ST ( 2.582) 2F N(0,1) ( 2.582). Observe that, if Z i Bi(1, 1/2), then X i = Z i 1/2 and therefore S n Bi(n, 1/2) n/2. The approimate and eact probabilities are computed below. > 2 * pnorm(-10, mean = 0, sd = sqrt(60)/2) [1] > 2 * pbinom(19, 60, 1/2) [1] E4 Suppose that a measurement has mean µ and standard deviation σ = 5. Let X n be the average of n such independent measurements. How large should n be so that P ( X n µ < 1) = 0.95? (Rice, 5.17) Solution. Observe that, according to the WLLN, the event A n = X n µ < 1 is almost sure when n, because X n has mean µ and standard deviation 5/ n. Hence the problem has a solution n sufficiently high to allow application of the CLT. In other words, we can assume the eistence of a given (high) n so as X n is approimately normally distributed. Therefore, we have to solve a quantile problem about a normal distribution. Let (ST ) denote the quantile of the N(0, 1) distribution of the order The following equation must hold and the solution is (about) n = (ST )