BIO5312 Biostatistics Lecture 5: Estimations

Transcription

1 BIO5312 Biostatistics Lecture 5: Estimations Yujin Chung September 27th, 2016 Fall 2016 Yujin Chung Lec5: Estimations Fall /34

2 Recap Yujin Chung Lec5: Estimations Fall /34

3 Today s lecture and some following lectures How to infer the properties of the underlying distribution in a data set. Two types of statistical inferences: Estimation: concerned with estimating the values of specific population parameters. These specific values are referred to as point estimates. Sometimes, interval estimation is carried out to specify an interval which likely includes the parameter values. Hypothesis testing: concerned with testing whether the value of a population parameter is equal to some specific value Yujin Chung Lec5: Estimations Fall /34

4 Point estimations Let X 1,..., X n be a random sample from a probability distribution. That is, X 1,..., X n are independent and identically distributed (iid). If X 1,..., X n N(µ, σ 2 ), what are the point estimations of µ and σ 2, respectively? If X 1,..., X n Bernoulli(p), what is the point estimation of p? If X 1,..., X n P oisson(λ), what is the point estimations of λ? Yujin Chung Lec5: Estimations Fall /34

5 A point estimation of the population mean Consider a random sample X 1,..., X n drawn from a distribution with mean µ = E(X) (unknown). A natural estimator for the population mean µ is the sample mean: µ = Ê(X) = X = 1 n n i=1 X i If X 1,..., X n N(µ, σ 2 ), X is a point estimation of E(X) = µ. If X 1,..., X n Bernoulli(p), X (the proportion of success) is a point estimation of E(X) = p. If Y B(n, p), Ȳ = Y (the number of successes) is a point estimations of E(Y ) = np. That is, ˆp = Y/n = X, where X 1,..., X n Bernoulli(p). If X 1,..., X n P oisson(λ), X is a point estimations of E(X) = λ? Yujin Chung Lec5: Estimations Fall /34

6 Examples Suppose a random sample of 5000 women is selected from this age group, of whom 28 are found to have malignant melanoma. What is the probability of having the disease (prevalence)? Let p be the probability of having the disease. Let the random variable X i represent the disease status for the ith woman, where X i = 1 if the ith woman has the disease and 0 if she does not for i = 1,..., The random variable X i was also defined as a Bernoulli trial. That is, X 1,..., X 5000 Bernoulli(p). Then a point estimation of E(X) = p is ˆp = x = = Yujin Chung Lec5: Estimations Fall /34

7 Properties of X (1) An unbiased estimator ( of ) µ: E(X) = µ 1 n proof) E(X) = E X i = 1 n E(X i ) = 1 n n n i=1 i=1 n µ = µ We consider infinitely many sets of random sample of size n. From each sample, the sample mean X is computed. Then the average value of X over infinitely many sets is µ = E(X). i=1 Average of sample means Sample~N(10,1) Sample size = the number of sets of random sample Yujin Chung Lec5: Estimations Fall /34

8 Properties of X (2) The minimum variance unbiased estimator of µ: If the underlying distribution is normal, then it can be shown that the unbiased estimator with smallest variance is given by X. For example, from a random sample X 1,..., X n N(µ, σ 2 ), we consider two estimators for µ: one is X and the other is the first observation X 1. Both are unbiased estimators: E( X) = µ and E(X 1 ) = µ. However, X has a smaller variance than X1 : V ar(x) = V ar V ar(x 1 ) = σ 2. ( 1 n ) n X i = 1 n n 2 V ar (X i ) = 1 n 2 nσ2 = σ2 n, i=1 i=1 Yujin Chung Lec5: Estimations Fall /34

9 Properties of X (3) A consistent estimator of µ: The estimator X converges to the population mean µ, as the sample size n goes to infinity. Sample mean Sample~N(10,1) 0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Sample size (n) Yujin Chung Lec5: Estimations Fall /34

10 A point estimation for V ar(x) Let X 1,..., X n N(µ, σ 2 ). A point estimation of V ar(x) = σ 2 is 1 n (X i µ) 2. (unbiased) n ( ) i=1 1 n pf) E (X i µ) 2 = 1 n E[(X i µ) 2 ] = 1 n σ 2 = σ 2 n n n i=1 i=1 Let X 1,..., X n N(µ, σ 2 ) but µ is unknown. ˆσ 2 = S 2 = 1 n (X i n 1 X) 2. (unbiased), ˆσ = ˆσ 2 = S. (biased) i=1 Let X 1,..., X n Bernoulli(p). A point estimation of V ar(x) = p(1 p) is V ar(x) = ˆp(1 ˆp) = X(1 X) (biased) Let X 1,..., X n P oisson(λ). A point estimation of V ar(x) = λ is ˆλ = X as the estimation of E(X) = λ. (unbiased) Yujin Chung Lec5: Estimations Fall /34 i=1

11 Consistency of a variance estimator The estimators of variances on the previous slide are Consistent! Let X 1,..., X n Bernoulli(p). As the sample size goes to the infinity, V ar(x) = ˆp(1 ˆp) = X(1 X) converges to p(1 p). Variance estimation random sample~bernoulli(0.7) 0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Sample size Yujin Chung Lec5: Estimations Fall /34

12 Examples LEAD data example What are the point estimations of the mean and standard deviation of the full IQ of children in the exposed group? The point estimation of the mean is x = and the estimation of the standard deviation is s 2 = Yujin Chung Lec5: Estimations Fall /34

13 Interval estimation of µ = E(X): σ known Interval Estimation: specify an interval which likely includes a parameter value of interest. Point estimates do not reflect our uncertainty when estimating a parameter. We always remain uncertain regarding the true value of the parameter when we estimate it using a sample from the population. To address this issue, we can present our estimates in terms of an interval of possible values (as opposed to a single value). Yujin Chung Lec5: Estimations Fall /34

14 Interval estimation: the uncertainty of X Let X = (X 1,..., X n ), where X i N(µ, σ 2 ) for i = 1,..., n. Sample mean X is a point estimator of µ. How is X distributed? X N(µ, σ 2 /n) What is the interval (u(x), v(x)) such that Pr[(u(X), v(x)) µ] =.95? Standardization X µ σ/ N(0, 1) n Consider a constant a = z = 1.96 such that ( Pr a < X ) µ σ/ n < a = 0.95 Yujin Chung Lec5: Estimations Fall /34

15 Interval estimation for µ ( Pr a < X µ ) σ/ n < a ( = Pr [( = Pr = Pr ( a n σ < X µ < a n σ ) ) X + a σ > µ & X a σ < µ n n X a σ, X + a σ ) ] µ = 0.95 n n Therefore, u(x) = X 1.96 σ n and v(x) = X σ n. Yujin Chung Lec5: Estimations Fall /34

16 Confidence interval for µ Let X = (X 1,..., X n ), where X i N(µ, σ 2 ) for i = 1,..., n. Assume σ is known. 100(1 α)% confidence interval for µ: ( X z 1 α/2 σ n, X + z 1 α/2 σ n ) For short hand, X ± z1 α/2 σ n. 1 α: confidence level z 1 α/2 : critical value for confidence level 1-α For example, the 95% confidence interval for µ is ( X 1.96 σ n, X σ n ) Yujin Chung Lec5: Estimations Fall /34

17 Examples LEAD data example The point estimation of the mean of the full IQ of children in the exposed group is x = Assume the full IQ follows a normal distribution with standard deviation is σ = Compute 95% confidence interval for the mean of IQ. There are 46 children in the exposed group. The standard error is σ/ n = Therefore, the 95% CI is ( , ) = (84.494, ) Yujin Chung Lec5: Estimations Fall /34

18 Confidence interval (CI) Interpretations of 95% CI: The probability that the interval contains the true value (parameter) is 0.95 Consider infinitely many sets of random sample of size n and compute the CIs. 95% of the infinitely many CIs will contain the true value. Yujin Chung Lec5: Estimations Fall /34

19 Factors Affecting the Length of a CI the 95% confidence interval (CI) for µ is ( X 1.96 σ n, X σ n ) The length of the CI indicates the precision of the point estimate X. The length of a 100%(1 α) CI for equals 2z σ/ n and is determined by α, the standard error σ/ n. α: as the confidence desired increases (decreases), the length of the CI increases. n: as the sample size (n) increases, the standard error decreases and the length of the CI decreases σ: As the variability of the distribution increases, the length of the CI increases Yujin Chung Lec5: Estimations Fall /34

20 Interval estimation of µ = E(X): σ unknown If X 1,..., X n N(µ, σ 2 ), the 95% confidence interval (CI) for µ is ( X 1.96 σ n, X σ n ) What if we don t know σ? X µ s/ n is distributed as a t distribution with (n 1)df. A 100% (1 α) CI is given by ( X t n 1,1 α/2 S/ n, X + tn 1,1 α/2 S/ n), where t n 1,1 α/2 is the (1 α/2)th percentile of t n 1 distribution If n > 200, use the standard normal distribution instead of t n 1 : ( X z 1 α/2 S/ n, X + z 1 α/2 S/ n) Yujin Chung Lec5: Estimations Fall /34

21 Examples LEAD data example The point estimation of the mean of the full IQ of children in the exposed group is x = There are 46 children in the exposed group. Assume the full IQ follows a normal distribution Compute 95% confidence interval for the mean of IQ. The standard error is ˆσ/ n = The critical value is t 45,.975 = Therefore, the 95% CI is ( , ) = (84.396, ) Note: The CI with unknown σ is wider than the CI (84.494, ) with known σ. Yujin Chung Lec5: Estimations Fall /34

22 Interval Estimation of the Variance of a Distribution Let X 1,..., X n N(µ, σ 2 ). A point estimation of σ 2 is S 2. Using (n 1)S2 σ 2 χ 2 n 1, a 95% CI is (u(x), v(x)) such that Pr ( u(x) < σ 2 & v(x) > σ 2) = 0.95 Find (u(x), v(x)): 0.95 = Pr (χ 2n 1,0.025 < ( (n 1)S 2 = Pr χ 2 > σ 2 & n 1,0.025 (n 1)S2 σ 2 < χ 2 n 1,0.975 (n 1)S2 χ 2 n 1,0.975 ) < σ 2 ) A 100% (1 α) CI for σ 2 is given by ( (n 1)S 2/ ) χ 2 n 1,1 α/2, (n 1)S2/ χ 2 n 1,α/2 Yujin Chung Lec5: Estimations Fall /34

23 Examples LEAD data example The point estimation of the variance of the full IQ of children in the exposed group is s 2 = There are 46 children in the exposed group. Assume the full IQ follows a normal distribution. Compute 95% confidence interval for the variance of IQ. The critical values are χ 2 45,.025 = and χ2 45,.975 = Therefore, the 95% CI is ( /65.41, /28.366) = ( , ). Yujin Chung Lec5: Estimations Fall /34

24 CI for a binomial parameter p Let X be a binomial random variable with parameters n and p. An unbiased estimator of p is given by the sample proportion of events ˆp = X/n. Its standard error is estimated by ˆp(1 ˆp)/n. By the Central limit theorem, ˆp p p(1 p)/n Z, where Z N(0, 1), as n. ˆp p We replace p by ˆp in the standard error: N(0, 1). ˆp(1 ˆp)/n When np(1 p) 5 (that is nˆp(1 ˆp)), an approximate 100% (1 α) CI for the binomial parameter p: ˆp ± z 1 α/2 ˆp(1 ˆp)/n Yujin Chung Lec5: Estimations Fall /34

25 Exact CI for a binomial parameter p When nˆp(1 ˆp) and X = x, an exact binomial distribution to build a CI. As p increase, Pr(X x p) increases, while Pr(X x p) decreases. CI for a binomial parameter p is obtained by (p 1, p 2 ) such that p 1 = min{p Pr(X x p) > α/2} & p 2 = max{p Pr(X x p) > α/2} p=0.2, n=5, X=2 p^=0.4, Exact CI (0.15, 0.85) Probability Pr(X <= 2 p) Pr(X >= 2 p) parameter p Yujin Chung Lec5: Estimations Fall /34

26 Examples Suppose a random sample of 5000 women is selected from this age group, of whom 28 are found to have malignant melanoma. What is the probability of having the disease (prevalence) and the 95% CI? Let p be the probability of having the disease. Let X i = 1 if the ith woman has the disease; 0 otherwise, for i = 1,..., Then a point estimation of E(X) = p is ˆp = x = = Since nˆp(1 ˆp) = , we use the normal approximation to compute a CI for p. The standard error is estimated by ˆp(1 ˆp)/n = and hence the 95% CI for p is ( , ) = (0.0035, ). Yujin Chung Lec5: Estimations Fall /34

27 CI for Poisson Distribution Let X 1,..., X n P oi(λ). A point estimation of λ is ˆλ = X and its standard error is λ/n. By the central limit theorem, an approximate 100%(1 α) CI for λ is ˆλ ± z 1 α/2 ˆλ/n n Let S = X i. Then, S = i=1 (λ 1, λ 2 ) such that n X i P oi(nλ). An exact CI for λ is i=1 λ 1 = min{λ Pr(S s λ) > α/2} & λ 2 = max{λ Pr(S s λ) > α/2} Yujin Chung Lec5: Estimations Fall /34

28 Bootstrap confidence interval Real data has a complex structure and we may be interested in more complex parameters or quantity. We have a large sample(e.g., n = 1000), but its distribution is very skewed. We d like to compute a CI for the population mean, but the Normal approximation may not be good enough. Histogram of a data Frequency n= If we are interested in the median of a data, how to compute a CI for the median? Yujin Chung Lec5: Estimations Fall /34

29 Bootstrap confidence interval Let X 1,..., X n be randomly sampled from an unknown distribution. We are interested in estimating a parameter θ and the estimator of θ is ˆθ = S(X). How can we build a CI for θ? A confidence interval for θ is in the form of (point estimation) ± z 1 α/2 (standard error of the estimation). That is, ˆθ ± z 1 α/2 SE(ˆθ)! However, we do NOT know the standard error of ˆθ. We use the bootstrap method to estimate the standard error. Yujin Chung Lec5: Estimations Fall /34

30 Bootstrap method: resampling method Goal: estimating the distribution of θ and hence the s.d. of θ Estimating the distribution from which the data was sampled: the population is estimated by the sampled data x = (x1,..., xn ) Sample many data sets from the estimated distribution P : x 1,..., x B Compute the estimation of θ: θ (x 1 ),..., θ (x B ) (This forms the distribution of θ ) Yujin Chung Lec5: Estimations Fall /34

31 Bootstrap confidence intervals Now we estimated the distribution of ˆθ: ˆθ(x 1 ),..., ˆθ(x B ). 100%(1 α) CI for θ (normal approximation) is where ˆθ = 1 B i=1 (ˆθ + (ˆθ ˆθ )) ± z 1 α/2 ŝe (ˆθ), B ˆθ(x i ) and ŝe (ˆθ) = 1 B (ˆθ(x i B 1 ) ˆθ ) 2. i=1 A percentile CI is using the 100%(α/2)th and 100%(1 α/2)th percentiles of ˆθ(x 1 ),..., ˆθ(x B ): (q α/2, q 1 α/2 ) Yujin Chung Lec5: Estimations Fall /34

32 Examples LEAD data example The point estimation of the mean of the full IQ of children in the exposed group is x = There are 46 children in the exposed group. Compute the 95% normal- and percentile CIs for the mean IQ. Resample the data and generate 1,000 replicates. The normal-ci is (84.64, 91.35) and the percentile-ci is (84.78, 91.54). Previously with normal assumption, the CIs are (84.494, ) with known σ = s and (84.396, ) with unknown σ. Yujin Chung Lec5: Estimations Fall /34

33 Summary A point or interval estimation of a parameter of interest from a random sample. 1 Identify the data type: continuous? Normal or non-normal? Bernoulli? Poisson? unknown? 2 What parameter or quantity to estimate? Are there any other unknown parameters? 3 What is a point estimation of the parameter of interest? Is the estimator good enough? 4 What is the standard error of the estimate? 5 What is a CI for the parameter? What is the distribution of the point estimation? Normal, Chi-square, t-dist, Binomial dist, etc Normal approximation? Difficult to discover or unknown: Bootstrap Yujin Chung Lec5: Estimations Fall /34

34 Next week Statistical hypothesis testing: concerned with testing whether the value of a population parameter is equal to some specific value. Yujin Chung Lec5: Estimations Fall /34