Point Estimation. Principle of Unbiased Estimation. When choosing among several different estimators of θ, select one that is unbiased.

Transcription

1 Point Estimation

2 Point Estimation Definition A point estimate of a parameter θ is a single number that can be regarded as a sensible value for θ. A point estimate is obtained by selecting a suitable statistic and computing its value from the given sample data. The selected statistic is called the point estimator of θ.

3 Point Estimation

4 Point Estimation Definition A point estimator ˆθ is said to be an unbiased estimator of θ if E(ˆθ) = θ for every possible value of θ. If ˆθ is not unbiased, the difference E(ˆθ) θ is called the bias of ˆθ.

5 Point Estimation Definition A point estimator ˆθ is said to be an unbiased estimator of θ if E(ˆθ) = θ for every possible value of θ. If ˆθ is not unbiased, the difference E(ˆθ) θ is called the bias of ˆθ. Principle of Unbiased Estimation When choosing among several different estimators of θ, select one that is unbiased.

6 Point Estimation

7 Point Estimation Proposition Let X 1, X 2,..., X n be a random sample from a distribution with mean µ and variance σ 2. Then the estimators ˆµ = X = n i=1 X i n and ˆσ 2 = S 2 = n i=1 (X i X ) 2 n 1 are unbiased estimator of µ and σ 2, respectively. If in addition the distribution is continuous and symmetric, then X and any trimmed mean are also unbiased estimators of µ.

8 Point Estimation

9 Point Estimation Principle of Minimum Variance Unbiased Estimation Among all estimators of θ that are unbiased, choose the one that has minimum variance. The resulting ˆθ is called the minimum variance unbiased estimator ( MVUE) of θ.

10 Point Estimation Principle of Minimum Variance Unbiased Estimation Among all estimators of θ that are unbiased, choose the one that has minimum variance. The resulting ˆθ is called the minimum variance unbiased estimator ( MVUE) of θ. Theorem Let X 1, X 2,..., X n be a random sample from a normal distribution with mean µ and variance σ 2. Then the estimator ˆµ = X is the MVUE for µ.

11 Point Estimation

12 Point Estimation Definition Let ˆθ be a point estimator of parameter θ. Then the quantity E[(ˆθ θ) 2 ] is called the mean square error (MSE) of ˆθ.

13 Point Estimation Definition Let ˆθ be a point estimator of parameter θ. Then the quantity E[(ˆθ θ) 2 ] is called the mean square error (MSE) of ˆθ. Proposition MSE = E[(ˆθ θ) 2 ] = V (ˆθ) + [E(ˆθ) θ] 2

14 Point Estimation

15 Point Estimation Definition The standard error of an estimator ˆθ is its standard deviation σ ˆθ = V (ˆθ). If the standard error itself involves unknown parameters whose values can be estimated, substitution of these estimates into σˆθ yields the estimated standard error (estimated standard deviation) of the estimator. The estimated standard error can be denoted either by ˆσ ˆθ or by s ˆθ.

16

17 Example (a variant of Problem 62, Ch5) The total time for manufacturing a certain component is known to have a normal distribution. However, the mean µ and variance σ 2 for the normal distribution are unknown. After an experiment in which we manufactured 10 components, we recorded the sample time which is given as follows: time time

18 Example (a variant of Problem 62, Ch5) The total time for manufacturing a certain component is known to have a normal distribution. However, the mean µ and variance σ 2 for the normal distribution are unknown. After an experiment in which we manufactured 10 components, we recorded the sample time which is given as follows: time time We know that both MME and MLE for the population mean µ is the sample mean X, i.e. ˆµ = X = How accurate is this estimation?

19

20 Assume the other parameter σ is known, e.g. σ = 2.7

21 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n

22 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval?

23 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95

24 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95 A is the 97.5th percentle, which is 1.96.

25 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95 A is the 97.5th percentle, which is ( ) P 1.96 < X µ σ/ n < 1.96 =.95

26 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95 A is the 97.5th percentle, which is ( ) P 1.96 < X µ σ/ n < 1.96 =.95 ) σ P (X 1.96 n σ < µ < X n =.95

27 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95 A is the 97.5th percentle, which is ( ) P 1.96 < X µ σ/ n < 1.96 =.95 ) σ P (X 1.96 n σ < µ < X n =.95 ) σ The interval (X 1.96 n σ, X n is called the 95% confidence interval for µ.

28 Assume the other parameter σ is known, e.g. σ = 2.7 X is normally distributed with mean µ and variance σ 2 /n. Therefore, Z = X µ σ/ is a standard normal random variable. n For the interval [ A, A], how large should A be such that with 95% confidence we are sure Z falls in that interval? P( A < Z < A) =.95 A is the 97.5th percentle, which is ( ) P 1.96 < X µ σ/ n < 1.96 =.95 ) σ P (X 1.96 n σ < µ < X n =.95 ) σ The interval (X 1.96 n σ, X n is called the 95% confidence interval for µ. In our case, 95% confidence interval for µ is (63.28, 66.62).

29

30 Interpretation of Confidence Interval

31 Interpretation of Confidence Interval The 95% confidence interval for µ (63.28, 66.62) doesn t mean P(µ falls in the interval(63.28, 66.62)) =.95

32 Interpretation of Confidence Interval The 95% confidence interval for µ (63.28, 66.62) doesn t mean P(µ falls in the interval(63.28, 66.62)) =.95 It is a long-run effect: if we have 1000 random samples, then for approximately 950 of them, µ falls in the interval (X 1.96 σ n, X σ n ).

33

34 Example (a variant of Problem 62, Ch5) The total time for manufacturing a certain component is known to have a normal distribution. However, the mean µ for the normal distribution is unknown. After an experiment in which we manufactured 10 components, we recorded the sample time which is given as follows: time time We know that both MME and MLE for the population mean µ is the sample mean X, i.e. ˆµ = X = We further assume the standard deviation is known to be σ = 2.7. What is the 99% confidence interval for µ?

35

36 Definition A 100(1 α)% confidence interval for the mean µ of a normal population when the value of σ is known is given by ( x z α/2 σ n, x + z α/2 ) σ n or, equivalently, by x z α/2 σ n

37

38 Graphically interpretation:

39

40 Example (a variant of Problem 62, Ch5) The total time for manufacturing a certain component is known to have a normal distribution. However, the mean µ for the normal distribution is unknown. Thus we decide to do an experiment in which we manufacture n components to estimate the population mean µ. We know that both MME and MLE for the population mean µ is the sample mean X, i.e. ˆµ = X. We further assume the standard deviation is known to be σ = 2.7. If we want a 99% confidence interval for µ with width 3.34, how large should n be?

41

42 Proposition To obtain a 100(1 α)% confidence interval with width w for the mean µ of a normal population when the value of σ is known, we need a random sample of size at least n = ( 2z α/2 σ ) 2 w

43 Proposition To obtain a 100(1 α)% confidence interval with width w for the mean µ of a normal population when the value of σ is known, we need a random sample of size at least n = ( 2z α/2 σ ) 2 w Remark: The half-width w 2 of the 100(1 α)% CI is called the bound on the error of estimation associated with a 100(1 α)% confidence level.

44

45 Example: Extensive experience with fans of a certain type used in diesel engines has suggested that the exponential distribution provides a good model for time until failure. However, the parameter λ is unknown. The following table records the data for a size 10 sample: time time What is a 95% confidence interval for λ?

46

47 Proposition Let X 1, X 2,..., X n i.i.d random variables from an expentional distribution with parameter λ. Then the random variable Y = 2λ n i=1 X i has the chi-squared distribution with 2n degrees of freedom, i.e., Y χ 2 (2n)

48 Large-Sample Confidence Intervals

49 Large-Sample Confidence Intervals Proposition If n is sufficiently large, the standardized variable Z = X µ S/ n has approximately a standard normal distribution. This implies that x ± z α/2 s n is a large-sample confidence interval for µ with confidence level approximately 100(1 α)%. This formula is valid regardless of the shape of the population distribution.

50 Large-Sample Confidence Intervals

51 Large-Sample Confidence Intervals Example (a variant of Problem 16) The charge-to-tap time (min) for a carbon steel in one type of open hearth furnace was determined for each heat in a sample of size 46, resulting in a sample mean time of and a sample standard deviation of Calculate a 95% confidence interval for true average charge-to-tap time.

52 Large-Sample Confidence Intervals

53 Large-Sample Confidence Intervals Example (Problem 19) The article Limited Yield Estimation for Visual Defect Sources (IEEE Trans. on Semiconductor Manuf., 1997: 17-23) reported that, in a study of a particular wafer inspection process, 356 dies were examined by an inspection probe and 201 of these passed the probe. Assuming a stable process, calculate a 95% confidence interval for the proportion of all dies that pass the probe.

54 Large-Sample Confidence Intervals

55 Large-Sample Confidence Intervals Proposition A confidence interval for a population proportion p with confidence level approximately 100(1 α)% has lower confidence limit = ˆp + z2 α/2 2n z α/2 1 + (z 2 α/2 )/n ˆpˆq n + z2 α/2 4n 2 and upper confidence limit = ˆp + z2 α/2 2n + z α/2 1 + (z 2 α/2 )/n ˆpˆq n + z2 α/2 4n 2

56 Large-Sample Confidence Intervals

57 Large-Sample Confidence Intervals Example (Problem 16) The charge-to-tap time (min) for a carbon steel in one type of open hearth furnace was determined for each heat in a sample of size 46, resulting in a sample mean time of and a sample standard deviation of Calculate a 95% upper confidence bound for true average charge-to-tap time.

58 Large-Sample Confidence Intervals

59 Large-Sample Confidence Intervals Example (Problem 19) The article Limited Yield Estimation for Visual Defect Sources (IEEE Trans. on Semiconductor Manuf., 1997: 17-23) reported that, in a study of a particular wafer inspection process, 356 dies were examined by an inspection probe and 201 of these passed the probe. Assuming a stable process, calculate a 95% lower confidence bound for the proportion of all dies that pass the probe.

60 Large-Sample Confidence Intervals

61 Large-Sample Confidence Intervals Proposition A large-sample upper confidence bound for µ is µ < x + z α s n and a large-sample lower confidence bound for µ is µ > x z α s n A one-sided confidence bound for p results from replacing z α/2 by z α and ± by either + or in the CI formula for p. In all cases the confidence level is approximately 100(1 α)%