Chapter 7 - Lecture 1 General concepts and criteria January 29th, 2010
Best estimator Mean Square error Unbiased estimators Example Unbiased estimators not unique Special case MVUE Bootstrap
General Question What do you think the main purpose of Statistics is?
Parameter versus Estimator Parameter refers to a value that represents the whole population. (If we go back to Stat 318 we will see that the distribution of a random variable might depend on some parameters. For example: On which parameter does the Poisson distribution depends?) Estimator refers to a value that represents a specific sample of the population and is used to find the value of the parameter when the parameter is unknown.
s A point estimate of parameter θ is a single number that can be regarded as a sensible value for θ. A point estimate is obtained by selecting a suitable statistic and computing its value from the given data. The selected statistic is called the point estimator of θ. Both the point estimate and point estimator of a parameter θ are denoted with ˆθ. The difference is the that, as a function the estimator uses the capital letters and the estimate uses the small letters.
Example with Poisson Suppose that the number of typos in each of my lectures follow a Poisson distribution with mean λ unknown. I randomly choose 10 lectures and the number of typos I find in them are: 0, 0, 0, 1, 1, 1, 1, 2, 3, 6. Find the estimator and the estimate of λ.
Example with Normal Suppose that the average time a student needs to finish a homework assignment is normally distributed with unknown average. I randomly ask 10 students and I get the following answers for the time it takes for them: 30, 25, 55, 60, 40, 45, 30, 60, 45, 50. Find several estimators and estimates of the population average.
Best possible estimator Best estimator Mean Square error Unbiased estimators Example As we have seen in the last example (see also example 7.2 in your book) there might be several choices to estimate a parameter θ. Which one is the best? In the best case, an estimator that is equal to the parameter always, that is ˆθ = θ, is the best estimator. But every estimator is a statistic. And we know that every statistic has a different value from sample to sample. So the above is impossible. Any solutions?
of Mean Square error Best estimator Mean Square error Unbiased estimators Example The mean square error for an estimator ˆθ is defined as follows: ( ) ) 2 MSE(ˆθ) = E (ˆθ θ It can be shown (how?) that the above can be written as: Why is MSE important? ( ) ) 2 MSE(ˆθ) = var(ˆθ) + E (ˆθ θ
Unbiased estimators Best estimator Mean Square error Unbiased estimators Example ) The value E (ˆθ θ is called bias of the point estimator ˆθ. A point estimator ˆθ is said to be an unbiased estimator of θ if: ) E (ˆθ = θ
Example 7.5 page 332 Best estimator Mean Square error Unbiased estimators Example
Proposition Outline Unbiased estimators not unique Special case MVUE If X 1,..., X n is a random sample from a distribution with mean µ then X is always an unbiased estimator of µ. Moreover if the distribution is continuous and symmetric then the median and the trimmed mean are also unbiased estimators of µ. The above shows that unbiased estimators are not unique. If we have several unbiased estimators which one is better?
of MVUE Unbiased estimators not unique Special case MVUE Among all estimators ˆθ that are unbiased we choose the one with the minimum variance. This estimator is called minimum variance unbiased estimator (MVUE) of θ. MVUEs are important in connection with the MSE. What happens to the MSE if we have an MVUE?
Theorem Outline Unbiased estimators not unique Special case MVUE Theorem If X 1,..., X n is a random sample from a normal distribution with parameters µ and σ 2 ( X i N(µ, σ 2 ) ). Then the estimator ˆµ = X is the MVUE for µ.
Example 7.7 page 335 Unbiased estimators not unique Special case MVUE
Unbiased estimators not unique Special case MVUE Example 7.6 page 335 - The confusing one
of an estimator Bootstrap Every time we report an estimator, we want to know how much inflation there is in our estimation for the parameter of interest. This inflation is called the standard error of the estimator. It is calculated as follows: ) var (ˆθ (1) σˆθ =
Bootstrap Estimated of an estimator Sometimes the standard error involves unknown parameters. In this case the standard error cannot be calculated. If the parameters can be estimated then we can calculate the estimated standard error, which is: ) ˆσˆθ = var (ˆθ (2)
Example Outline Bootstrap Suppose that the average time a student needs to finish a homework assignment is normally distributed with unknown mean and standard deviation equal to 15. I randomly ask 10 students and I get the following answers for the time it takes for them: 30, 25, 55, 60, 40, 45, 30, 60, 45, 50. Find the standard error of the estimator for the mean.
Example Outline Bootstrap Suppose that the average time a student needs to finish a homework assignment is normally distributed with unknown mean and standard deviation. I randomly ask 10 students and I get the following answers for the time it takes for them: 30, 25, 55, 60, 40, 45, 30, 60, 45, 50. Find the standard error of the estimator for the mean.
Bootstrap Bootstrap Sometimes we cannot find a specific formula to estimate the standard error of the estimate. In this case one can use bootstrap to find an estimated standard error of the estimate. Let s assume that we have a random sample from N(µ, σ 2 ) where µ and σ 2 are both unknown. We estimate them by ˆµ = x = 2.2 and ˆσ 2 = s 2 = 8.3. We then use computer to calculate B bootstrap samples from N(2.2, 8.3).
Bootstrap Outline Bootstrap In each sample we calculate a new estimate for the mean ˆµ B and a new estimate for the variance ˆσ 2 B. We average all the means and all the variances to obtain ˆµ = B i=1 ˆµ i B, ˆσ 2 = The bootstrap estimated of the standard error of the two estimators are: Sˆµ = 1 B (ˆµ B 1 i ˆµ ) 2, Sˆσ 2 = 1 B 2 (ˆσ i ˆσ 2 ) 2 B 1 i=1 B i=1 B ˆσ 2 i i=1
Section 7.1 page 340 1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19