STAT 241/251 - Chapter 7: Central Limit Theorem In this chapter we will introduce the most important theorem in statistics; the central limit theorem. What have we seen so far? First, we saw that for an i.i.d random sample X 1, X 2,..., X n, where E(X i ) = µ and Var(X i ) = σ 2 E( X) = µ X = µ, Var( X) = σ 2 X = σ2 n E(S) = µ S = nµ, Var(S) = σ 2 S = nσ2 Further, in chapter 5 we saw that if the X i N(µ, σ 2 ), then... X N(µ, σ2 n ) S N(nµ, nσ 2 ) (ie) Linear functions of a normal RV are also normally distributed. These are nice results, although often we have a random sample where each of the X i s do not follow a normal distribution. (The distribution we are sampling from is not Normal) (eg) We may take a random sample of new fluorescent lightbulbs and measure the life-time of each bulb. In this case, we might model the life-time of each bulb as an exponential RV. For this random sample, we will still probably look at the mean life-time of all the bulbs or maybe the sum of the life-times. So, what distribution do X and S follow then? 1
The Central Limit Theorem (CLT) If our random sample X 1, X 2,..., X n is i.i.d and comes from any particular distribution (may not be normal) with mean µ and variance σ 2, then when n is large enough (and the samples are independent of one another and are made randomly), the sample mean X and the sample sum S approximately follow a normal distribution. (ie) If X i Almost any shape distribution with mean µ and variance σ 2, and large n then... X N(µ, σ2 n ) S N(nµ, nσ 2 ) This result is very important and is used extensively throughout statistics, as it tells us that no matter what distribution our random sample comes from, that the sample mean and sample sum follow a Normal distribution as long as certain conditions are met. The CLT is an asymptotic or limit result, meaning that when n =, X and S are Normally distributed, but when n <, X and S are only approximately Normally distributed. This raises the question, when is n large enough to say that X and S are Normally distributed? Answer: The more symmetric and light tailed the distribution of the X i s, the quicker that X and S will converge to Normality. Provided that the distribution of the X i s is not too skewed or asymetric, a sample size of n 20 is usually adequate for the CLT to kick in. If the distribution we are sampling from is very skewed, then we need a larger sample size. For the sake of this class, we will use n=20 as the magic number. In reality it depends more on the shape of the distribution that we are sampling from. 2
First we will look at a simulation to explain the central limit theorem. Then I will draw a picture to illustrate the concept. The CLT can also be proven mathematically, although that is beyond the scope of this course. Simulating The Sampling Distribution of a Mean: Below is a picture of histograms of some simulated rolls of dice. I will talk about these in class. Note: I used dice as the example, as these cover many areas. (ie) A dice can lead to proportions, it can be binomial, and it is an ordinal variable, which is similar to a quantitative variable. 10000 Rolls of 1 Die The Mean of 10000 Rolls of 2 Dice The Mean of 10000 Rolls of 3 Dice 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.0 0.1 0.2 0.3 0.4 0.5 0.6 1 2 3 4 5 6 die1 1 2 3 4 5 6 die2 1 2 3 4 5 die3 The Mean of 10000 Rolls of 5 Dice The Mean of 10000 Rolls of 25 Dice 0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.2 0.4 0.6 0.8 1.0 1 2 3 4 5 6 die5 2.5 3.0 3.5 4.0 4.5 5.0 die25 We can see that as the sample size increases (1, 2, 3, 5, 25), the sampling distribution of the mean begins to look like a Normal distribution. Here is another picture interpretation of the CLT... 3
Examples: 1. You have designed a new sattelite that is planned to orbit in space for the next 150 years. It is set up in the following way. It has 25 battery packs. One powers the sattelite, and when it burns out the next battery takes over, and when that one burns out the next takes over, and so on. The lifetime of each battery follows an exponential distribution with a mean of 8 years and a standard deviation of 8 years. What is the probability that the sattelite runs out of batteries before the 150 years is up? 2. A standard bottle of beer advertises that it contains 341mL of beer. In fact, the machine that pours the beer into the bottle pours a mean amount of 343mL with a standard deviation of 2mL. The amount of beer poured follows a normal distribution. (a) What is the probability that a randomly selected bottle of beer is underfilled? (b) If you buy a two-four, what is the probability that no more than 4 bottles are underfilled? (c) If you buy a 6-pack, what is the probability that the average amount of liquid is less than 341mL? 3. An elevator has a limit of 10 people or 2000lbs. If 10 people get on the elevator what is the probability that they surpass the limit? Suppose that the weights of people follow a normal distribution with a mean of 170lbs and a standard deviation of 30lbs. 4. You have developed a new type of concrete that is reinforced with steel fibers. Suppose you know that a concrete block of this type has a true mean breaking strength of 500lbs with a standard deviation of 12lbs. If you take a sample of 25 blocks and test their breaking strengths, what is the probability that the 25 blocks have a mean breaking strength greater than 505lbs? 5. Suppose you will go to the casino and make 81 bets, each of $1, on the colour of the number coming up. What is the mean and variance of the gain/loss you will make from these 81 bets? What is the probability that you leave the casino having made money? 4
So far, we have only dealt with the CLT and continuous distributions. But what happens when we have a random sample where each X i comes from some discrete distribution? We present a few results here that follow from the CLT. Shortly, we will see that under certain conditions, we can use a normal distribution to approximate the binomial and the poisson distributions. Normal Approximation to the Binomial Recall: That if X BIN(n, p), then µ x = np and σ 2 x = np(1 p) When n is large and p is not too close to 0 or 1, then... BIN(n, p) N(np, np(1-p)) The rule-of-thumb for this approximation to work is that min{np, n(1 p)} 10 Continuity Correction: Because we are approximating a discrete distribution with a continuous distribution, we must make a continuity correction. (ie) In the discrete case, P(X x) P(X > x) P (X = k) = P (k 0.5 X k + 0.5) P (a X b) = P (a 0.5 X b + 0.5) P (a < X < b) = P (a + 0.5 X b 0.5) P (X < a) = P (X a 0.5) P (X a) = P (X a + 0.5) P (X > a) = P (X a + 0.5) P (X a) = P (X a 0.5) Note: The continuity correction makes little difference when n is large. I will explain the idea of the continuity correction more in depth during lecture, and the above should make more sense then. 5
Examples: 1. Consider rolling a die 150 times. What is the probability that you get... (a) Exactly 23 6 s? (b) Between 15 and 35 6 s? (c) More than 3 6 s? 2. You go to the casino and make 81 bets, each of $1, on the colour of the number coming up. What is the probability that you leave the casino having made money? 3. It is believed that 4% of children have a gene that may be linked to juvenille diabetes. Researchers are hoping to track 20 or more of these children (with the defect) for several years. they will test 732 newborn babies for the presence of this gene, and if the gene is present, they will track the child for several years. What is the probability that they find 20 or more subjects to be in the study? 4. Suppose you will toss a coin 100 times. Create an interval that you are 95% sure that the number of heads tossed will be in. Center this interval around the expected number of heads. 6
Normal Approximation to the Poisson Recall: That if X POISSON(λ ), then µ x = λ and σ 2 x = λ When λ is large, then... POISSON(λ ) N(λ, λ ) The rule-of-thumb for this approximation to work is that λ 20 Like in the Binomial case, here we are approximating a discrete distribution using a continuous distribution, so we must use the same continuity correction as in the Binomial case. Example: 1. Recall example (1) from chapter 6, in the section on Poisson Processes. We were monitoring the number of earthquakes in California over 6.7, and there were an average of 1.5 per year. (a) What is the probability of having more than 14 large earthquakes in the next 15 years? (b) What is the probability of having exactly 1 large earthquake in the coming year? 2. A factory produces sheet metal. Impurities occur at a rate of 0.1 per meter squared, and are equally likely to occur anywhere on a given sheet. The occurrence of impurities are independent of one another. If you buy 500 square meters of this sheet metal, what is the probability that there is more than 40 impurities on the sheet? 7