Tutorial 6. Sampling Distribution. ENGG2450A Tutors. 27 February The Chinese University of Hong Kong 1/6

Transcription

1 Tutorial 6 Sampling Distribution ENGG2450A Tutors The Chinese University of Hong Kong 27 February /6

2 Random Sample and Sampling Distribution 2/6

3 Random sample Consider a random variable X with distribution F. Suppose we repeatedly pull n samples from this distribution F, then these n samples can also be considered as n random variables with the same distribution F. We denote these n random variables by X 1, X 2,..., X n. The distribution of each X i is given by f (x). Now we can define the so-called random sample. Definition Random sample. A set of observations X 1, X 2,..., X n constitutes a random sample of size n from the population f (x) if 1 each X i is a random variable whose distribution is given by f (x); and 2 these n random variables are independent of each other. There are n elements in each sample. One can then measure some statistics of each sample. For instance, the average (the arithmetic mean) of the n elements of each sample is a common statistic, which is usually referred to as the sample mean. 2/6

4 Sampling distribution Again, the sample mean, the average of the n elements of a sample, is a random variable. We denote it by X, and there is X = 1 n X i. n If a statistic is considered as a random variable, then we are interested in the distribution of this random variable. A distribution (of a statistic of a sample) is call the sampling distribution of that statistic. For instance, the sample mean X is a statistic of a sample, the distribution of X is called the sampling distribution of the sample mean. To study a distribution, we are first interested in the mean and the variance of it. For instance, if we denote by f X the p.d.f of the sample mean X, then we are interested in the mean µ X and the variance σ 2 X = Var [ X ] of X given, respectively, by µ X = xf X (x)dx and σ2 X = (x µ x ) 2 f x (x)dx. i=1 3/6

5 Properties of the sampling distribution of the sample mean X 4/6

6 Mean of the sample mean X Theorem If a random sample of size n is taken from a population having the mean µ and the variance σ 2, then X is a random variable which has the mean µ X = µ. Motivation. For any random variable X with distribution F, of which we do not know the mean. To estimate the mean µ of F (or to say, of the population) it is intuitive to pull some samples from the population and use their average as an estimate of µ. Thus, we want to study the expected value and the standard error of this estimator, i.e., the mean and the standard deviation of X = 1 n n i=1 X i. This theorem relates the expected value µ X of this estimator X to the mean µ of the population. This theorem states that if we estimate µ by the arithmetic mean x = n i=1 x i of the samples and enough number of samples are taken, then the expected value of the estimator is truly the population mean µ. 4/6

7 Variance of the sample mean X Theorem If a random sample of size n is taken from a population having the mean µ and the variance σ 2, then X is a random variable of the variance σ2 n. Motivation. Similarly, we want to know how reliable the estimate is, then we need to study the standard error of our estimator X to the mean value of the population. Because by Chebyshev s Theorem, if σ X is small enough, then the probability that our estimate X is close to the population mean µ approaches 1, i.e. ɛ > 0, P ( X µ < ɛ ) 1 σ2 X ɛ 2. The standard error SE X is the standard deviation σ X of X. We have two intuitions, 1) if the population has a smaller variance, the estimate should more likely be accurate; and 2) if we take enough number of samples, the estimate should be reliable. This theorem is a justification and a rigorous statement of above intuitions. We can make σ X as close to 0 as possible by making n large enough. 5/6

8 Law of large numbers Theorem Law of large numbers. Let X 1, X 2,..., X n be independent and identically distributed random variables each having the same mean µ and variance σ 2. Then for any ɛ > 0, where X = 1 n n i=1 X i. P ( X µ > ɛ ) 0 as n, In practice, it is usually impossible to calculate the expected value of a random variable since the true probability distribution remains unknown. This theorem shows that the sample mean converges (in probability) to the mean of the underlying distribution of the population. Hence, we can estimate the expected value of a random variable as accurate as possible, by taking enough number of samples, and then calculate sample mean as an alternative. 6/6