Sampling Distributions

Transcription

1 Sampling Distributions This is an important chapter; it is the bridge from probability and descriptive statistics that we studied in Chapters 3 through 7 to inferential statistics which forms the latter part of this book. Here is the link: we are presented with a population about which we would like to learn. And while it would be desirable to examine every single member of the population, we find that it is either impossible or infeasible to for us to do so, thus, we resort to collecting a sample instead. We do not lose heart. Our method will suffice, provided the sample is representative of the population. A good way to achieve this is to sample randomly from the population. Supposing for the sake of argument that we have collected a random sample, the next task is to make some sense out of the data because the complete list of sample information is usually cumbersome, unwieldy. We summarize the data set with a descriptive statistic, a quantity calculated from the data (we saw many examples of these in Chapter 3). But our sample was random... therefore, it stands to reason that our statistic will be random, too. How is the statistic distributed? The probability distribution associated with the population (from which we sample) is called the population distribution, and the probability distribution associated with our statistic is called its sampling distribution; clearly, the two are interrelated. To learn about the population distribution, it is imperative to know everything we can about the sampling distribution. Such is the goal of this chapter. We begin by introducing the notion of simple random samples and cataloguing some of their more convenient mathematical properties. Next we focus on what happens in the special case of sampling from the normal distribution (which, again, has several convenient mathematical properties), and in particular, we meet the sampling distribution of X and S 2. Then we explore what happens to X s sampling distribution when the population is not normal and prove one of the most remarkable theorems in statistics, the Central Limit Theorem (CLT). With the CLT in hand, we then investigate the sampling distributions of several other popular statistics, taking full advantage of those with a tractable form. We finish the chapter with an exploration of statistics whose sampling distributions are not quite so tractable, and to accomplish this goal we will use simulation methods that are grounded in all of our work in the previous four chapters.

2 8.1 Simple Random Samples Simple Random Samples Definition 8.1. If X 1, X 2,..., X n are independent with X i f for i = 1, 2,..., n, then we say that X 1, X 2,..., X n are independent and identically distributed (i.i.d.) from the population f or alternatively we say that X 1, X 2,..., X n are a simple random sample of size n, denoted S RS (n), from the population f. Proposition 8.2. Let X 1, X 2,..., X n be a S RS (n) from a population distribution with mean µ and finite standard deviation σ. Then the mean and standard deviation of X are given by the formulas µ X = µ and σ X = σ/ n. 8.2 Sampling from a Normal Distribution The Distribution of the Sample Mean Proposition 8.4. Let X 1, X 2,..., X n be a S RS (n) from a norm(mean = µ, sd = σ) distribution. Then the sample mean X has a norm(mean = µ, sd = σ/ n) sampling distribution.

3 8.2. SAMPLING FROM A NORMAL DISTRIBUTION The Distribution of the Sample Variance Theorem 8.5. Let X 1, X 2,..., X n be a S RS (n) from a norm(mean = µ, sd = σ) distribution, and let n X = X i and S 2 = 1 n (X i X) 2. (8.2.1) n 1 Then i=1 1. X and S 2 are independent, and 2. The rescaled sample variance (n 1) σ 2 S 2 = has a chisq(df = n 1) sampling distribution. i=1 n i=1(x i X) 2 σ 2 (8.2.2) The Distribution of Student s T Statistic Proposition 8.6. Let X 1, X 2,..., X n be a S RS (n) from a norm(mean = µ, sd = σ) distribution. Then the quantity T = X µ S/ (8.2.3) n has a t(df = n 1) sampling distribution.

4 The code to produce Figure is > curve(dt(x, df = 30), from = -3, to = 3, lwd = 3, ylab = "y") > ind <- c(1, 2, 3, 5, 10) > for (i in ind) curve(dt(x, df = i), -3, 3, add = TRUE) Similar to that done for the normal we may define t α (df = n 1) as the number on the x-axis such that there is exactly α area under the t(df = n 1) curve to its right. Example 8.7. Find t 0.01 (df = 23) with the quantile function. > qt(0.01, df = 23, lower.tail = FALSE) [1]

5 8.3. THE CENTRAL LIMIT THEOREM 185 y x Figure 8.2.1: Student s t distribution for various degrees of freedom 8.3 The Central Limit Theorem In this section we study the distribution of the sample mean when the underlying distribution is not normal. We saw in Section 8.2 that when X 1, X 2,..., X n is a S RS (n) from a norm(mean = µ, sd = σ) distribution then X norm(mean = µ, sd = σ/ n). In other words, we may say when the underlying population is normal that the sampling distribution of Z defined by Z = X µ σ/ (8.3.1) n is norm(mean = 0, sd = 1). However, there are many populations that are not normal... and the statistician often finds herself sampling from such populations. What can be said in this case? The surprising answer is contained in the following theorem. Theorem 8.9. The Central Limit Theorem. Let X 1, X 2,..., X n be a S RS (n) from a population distribution with mean µ and finite standard deviation σ. Then the sampling distribution of Z = X µ σ/ n (8.3.2) approaches a norm(mean = 0, sd = 1) distribution as n.

6 186 CHAPTER 8. SAMPLING DISTRIBUTIONS How to do it with R The TeachingDemos package [79] has clt.examp and the distrteach [74] package has illustrateclt. Try the following at the command line (output omitted): > library(teachingdemos) > example(clt.examp) and > library(distrteach) > example(illustrateclt) The IPSUR package has the functions clt1, clt2, and clt3 (see Exercise 8.2 at the end of this chapter). Its purpose is to investigate what happens to the sampling distribution of X when the population distribution is mound shaped, finite support, and skewed, namely t(df = 3), unif(a = 0, b = 10) and gamma(shape = 1.21, scale = 1/2.37), respectively. For example, when the command clt1() is issued a plot window opens to show a graph of the PDF of a t(df = 3) distribution. On the display are shown numerical values of the population mean and variance. While the students examine the graph the computer is simulating random samples of size sample.size = 2 from the population = "rt" distribution a total of N.iter = times, and sample means are calculated of each sample. Next follows a histogram of the simulated sample means, which closely approximates the sampling distribution of X, see Section 8.5. Also show are the sample mean and sample variance of all of the simulated Xs. As a final step, when the student clicks the second plot, a normal curve with the same mean and variance as the simulated Xs is superimposed over the histogram. Students should compare the population theoretical mean and variance to the simulated mean and variance of the sampling distribution. They should also compare the shape of the simulated sampling distribution to the shape of the normal distribution. The three separate clt1, clt2, and clt3 functions were written so that students could compare what happens overall when the shape of the population distribution changes. It would be possible to combine all three into one big function, clt which covers all three cases (and more).

7 8.4.3 Ratio of Independent Sample Variances Proposition Let X 1, X 2,..., X n1 be an S RS (n 1 ) from a norm(mean = µ X, sd = σ X ) distribution and let Y 1, Y 2,..., Y n2 be an S RS (n 2 ) from a norm(mean = µ Y, sd = σ Y ) distribution. Suppose that X 1, X 2,..., X n1 and Y 1, Y 2,..., Y n2 are independent samples. Then the ratio F = σ2 Y S 2 X σ 2 X S 2 Y (8.4.6) has an f(df1 = n 1 1, df2 = n 2 1) sampling distribution. Proof. We know from Theorem 8.5 that (n 1 1)S 2 X /σ2 X is distributed chisq(df = n 1 1) and (n 2 1)S 2 Y /σ2 Y is distributed chisq(df = n 2 1). Now write F = σ2 Y S 2 X σ 2 X S 2 Y = (n 1 1)S Y 2 (n 2 1)S Y 2 (n 2 1)S 2 Y σ 2 Y / (n1 1) / (n2 1) 1/ σ2 X, 1/ σ 2 Y by multiplying and dividing the numerator with n 1 1 and doing likewise for the denominator with n 2 1. Now we may regroup the terms into / (n 1 1)S 2 X (n σ 2 1 1) X F = /, (n 2 1) and we recognize F to be the ratio of independent chisq distributions, each divided by its respective numerator df = n 1 1 and denominator df = n 1 1 degrees of freedom.

8 8.5. SIMULATED SAMPLING DISTRIBUTIONS 189 Remark For the special case of σ X = σ Y we have shown that F = S 2 X S 2 Y (8.4.7) has an f(df1 = n 1 1, df2 = n 2 1) sampling distribution. This will be important in Chapters 9 onward.