χ 2 distributions and confidence intervals for population variance Let Z be a standard Normal random variable, i.e., Z N(0, 1). Define Y = Z 2. Y is a non-negative random variable. Its distribution is known as Chi-square (χ 2 (1)) with one d.f. whose shape is depicted as follows. χ 2 (0.05; 1) denotes the upper 5 percentile of a chi-square distribution with 1 d.f. The sum of m independent squared standard Normal random variables has a Chi-square (χ 2 (m)) with m d.f., i.e., Y = Z 2 1 + Z2 2 + + Z2 m χ2 (m). The mean and variance of a χ 2 (m) random variable equals m and 2m respectively. Let Z1, Z 2 and Z 3 be three independent N(0, 1) random variables. Find c such that P (Z 2 1 + Z2 2 + Z2 3 c) = 0.01. 22S:39 class notes 53
Let X 1,, Xn be a random sample from a N(µ X, σ X 2 ) population. It can be shown that n i=1 (X i X) 2 /σ X 2 χ2 (n 1). Recall that the sample variance S 2 = n i=1 (X i X) 2 /(n 1). What is E(S 2 )? The above distribution result implies that a 95% confidence interval for σ 2 X equals [(n 1)S 2 /χ 2 (0.975; n 1), (n 1)S 2 /χ 2 (0.025; n 1)]. Why? Example. Speed of light. Student t distributions For the case of unknown σ X 2, it is estimated by the sample variance S2 = {(X 1 X) 2 +... + (Xn X) 2 }/(n 1). In this case, instead of standardizing the sample mean, we consider studentizing it: T = X µ is known, then X µ σ X / n S/ n. If σ X N(0, 1). The estimation of the population variance implies that T has a 22S:39 class notes 54
more variable distribution than the standard normal. In fact, its distribution is called a t-distribution with n 1 degree of freedoms. A t-distribution has zero mean and a symmetric pdf about 0. Its tails are thicker than the standard normal. A t-distribution becomes a standard normal when its df becomes very large. Example The upper 2.5 percentile of a t-distribution with 19 d.f. is denoted as t(0.025; 19). It equals When σ 2 X is estimated by S2, the 95 % C.I. of µ becomes ( X t(0.025; n 1) σ X, X + t(0.025; n 1) σ X) Example Speed of light. 22S:39 class notes 55
It can be shown that if Z is N(0, 1) and independent of W χ 2 (m), then Z/ W/m is a t-distribution with m d.f. Using this result and the result that for Normal populations, X is independent of S 2, we can prove that the studentized ratio X µ X S 2 /n has a t-distribution with n 1 d.f., which forms the basis of the above confidence intervals for the mean µ X. Comparing the variances of two populations Let X 1,..., Xn N(µ X, σ X 2 ) and Y 1,..., Y m N(µ Y, σ Y 2 ) be two independent random samples. We want to compare the two population variances and estimate the ratio σ X 2 /σ2 Y by S2 X /S2 Y. To construct C.I. (confidence interval), we need to introduce a new distribution. Let U and V be two independent χ 2 random variables with a and b df. Then the ratio F = U/a has an F (a, b) distribution. V/b Note that F (1 α; a, b) = 1/F (α; a, b). Applying this result, we see that (m 1)S2 Y /{(m 1)σ2 Y } (n 1)S X 2 /{(n 1)σ2 X } = S 2 Y σ2 X S X 2 σ2 Y F (m 22S:39 class notes 56
1, n 1). Therefore, with 95% probability, F (0.975; m 1, n 1) S2 Y σ2 X S 2 X σ2 Y F (0.025; m 1, n 1) After doing some algebra, the above is equivalent to S 2 X F (0.025; m 1, n 1)S 2 Y σ2 X σ 2 Y S2 X F (0.025; m 1, n 1) S Y 2 The latter is the formula for a 95% C.I. of σ2 X σ 2 Y Example. Speed of light.. Comparing two population means Let X 1,..., Xn N(µ X, σ X 2 ) and Y 1,..., Y m N(µ Y, σ Y 2 ) be two independent random samples. The difference µ X µ Y can be estimated by X Ȳ. By independence, σ 2 X Ȳ = σ2 X + σ 2 Ȳ = σ2 X /n + σ2 Y /m. If σ2 X and σ2 Y are unknown, they can be estimated respectively by S X 2 and S2 Y. There are three cases to consider in constructing a confidence interval of µ X µ Y. Case 1: σ X 2 and σ2 Y are known. (If n > 30 and m > 30, S2 X and S2 Y are effectively treated as if they were the true population variances.) Then a 95 % C.I. of µ X µ Y is given by ( X Ȳ 1.96σ X Ȳ, X Ȳ + 1.96σ X Ȳ ). 22S:39 class notes 57
More generally, the (1 α) 100% C.I. of µ X µ Y is given by ( X Ȳ z α/2 σ X Ȳ, X Ȳ + z α/2 σ X Ȳ ). Case 2: σ 2 X and σ2 Y are unknown, but assumed to be identical, i.e., σ2 X = σ2 Y = σ2. In this case, both S 2 X and S2 Y are estimators of σ2, which can be pooled to form a better estimator: S 2 = S 2 p = {(X 1 X) 2 +... + (Xn X) 2 } + {(Y 1 Ȳ )2 +... + (Ym Ȳ )2 }. n 1 + m 1 It can be shown that (n 1 + m 1)S 2 p /σ2 χ 2 (n + m 2). Estimate σ 2 X Ȳ = σ2 X + σ 2 Ȳ = σ2 X /n + σ2 Y /m. by σ X Ȳ = Sp 1/n + 1/m. Let the degree of freedom (df) be r = n + m 2. Then a 95 % C.I. of µ X µ Y is given by ( X Ȳ t(0.025; r) σ X Ȳ, X Ȳ + t(0.025; r) σ X Ȳ ). More generally, the (1 α) 100% C.I. of µ X µ Y is given by ( X Ȳ t(α/2; r) σ X Ȳ, X Ȳ + t(α/2; r) σ X Ȳ ). 22S:39 class notes 58
Example: Speed of light. Case 3. σ X 2 and σ2 Y are unknown, and they need not equal. In this case, we can t pool the two variance estimates. Instead, estimate σ 2 X Ȳ by σ X Ȳ = S2 X /n + S2 Y /m. The formula of the C. I. derived in the preceding case continue to hold except that the df is given by the formula: r = (S X 2 /n + S2 Y /m)2 1 n 1 (S2 X /n)2 + m 1 1 (S2 Y /m)2. Example: Speed of light. Estimating proportions Suppose that members of a population can be classified into two types: type I and non-type I. The population proportion of type I s is p. Let X = 1 if a randomly selected member from the population is of type I and 0 otherwise. The distribution of X is as follows: 22S:39 class notes 59
Then, µ X = p and σ X 2 = p(1 p). Hence, the population variance is a function of the mean. Let X 1,..., Xn be a random sample from the above populations. The sum of the X s equal the number of type I s in the sample. What is the distribution of X? The sample mean equals ˆp = X = {X 1 +... + Xn}/n is the sample proportion of type I in the sample. For large sample size, the CLT implies that X N(p, p(1 p)/n). When sample size is large as in most sample survey, we can estimate the variance by σˆp = ˆp(1 ˆp)/n. Hence, we have the approximate 95% C.I. of p: (ˆp 1.96 σˆp, ˆp + σˆp ). Example It is found that there are 40 defective parts in a sample of 100 parts from a lot of parts. Construct a 95% confidence interval for the true defective rate. Example The management introduced a new manufacturing procedure which is supposed to reduce the defective rate of the computer parts. A second sample was obtained which yields 30 defective parts in a random sample of 100 parts. Does the new manufacturing process represent an improvement over the old process? 22S:39 class notes 60