Chapter 7: SAMPLING DISTRIBUTIONS & POINT ESTIMATION OF PARAMETERS

Transcription

1 Chapter 7: SAMPLING DISTRIBUTIONS & POINT ESTIMATION OF PARAMETERS Part 1: Introduction Sampling Distributions & the Central Limit Theorem Point Estimation & Estimators Sections 7-1 to 7-2 Sample data is collected on a population to draw conclusions, or make statistical inferences, about the population. Types of statistical inference: 1) parameter estimation (e.g. estimating µ) - with a certain level of confidence 2) hypothesis testing (e.g. H 0 : µ = 50) 1

2 Example of parameter estimation (or point estimation): We re interested in the value of µ. We collected data and we use the observed x as a point estimate for µ. µ is the unknown parameter being estimated. NOTATION: ˆµ = X X is the estimator. {We often show an estimator as a hat over its respective parameter.} The observed x estimate is a single value, or a point estimate. Prior to data collection, X is random variable and it is the statistic of interest from the data. 2

3 Sample-to-sample variability The value we get for X (the sample mean) depends on the specific sample chosen. Sample Population This means, X is a random variable! The distribution of the random variable X is called the sampling distribution of X. We expect X to be close to µ (we ARE using it to estimate µ) but there is variability in X before it is observed because we use random sampling to choose our sample of size n. 3

4 The Sampling Distribution of X... Tells us what kind of values are likely to occur for X. Puts a probability distribution over the possible values for X. HINT: It s distribution will be normal when conditions are met. In a simple random sample of n observations from a population, E( X) = µ X is an unbiased estimator of µ. This gives us a measure of center for the sampling distribution for X, but what about the variability of the X random variable? 4

5 Sampling distribution of X Case 1 Original population is normally distributed. f(x) x The x I observe depends on the sample (the particular n observations) I chose from this normal distribution. Let s look at the distribution of x values if I choose a sample of size n and compute x for that sample, and I repeat this process 1000 times... 5

6 f(x) x 1) Choose a sample of size n from a normal distribution 2) Compute x 3) Plot the x on our frequency histogram 4) Do steps times See applet at: sim/sampling dist/index.html 6

7 SKETCH THE PLOTS: Distribution of X for n=2 when original population is normal. Distribution of X for n=25 when original population is normal. 7

8 Turns out, in this case, the random variable X is normally distributed. This normal distribution is centered at µ (the mean of the original population we were sampling from). The variability of X depends on the sample size n, and the variability in the original population. SPECIFICALLY: When X N(µ, σ 2 ), X N(µ, σ2 n ) NOTE: the distribution for X is less variable than the distribution for X. 8

9 X N(µ, σ2 n ) NOTE: X from n = 25 is less variable than X from n = 2. More data (larger n) gives us a better estimate of µ from X. The distribution of our estimator X is squished closer, or is tighter, around the thing we re trying to estimate. Which is beneficial when estimating something. 9

10 Sampling distribution of X Case 2 Original population is NOT normally distributed. f(x) f(x) x x f(x) x Or anything else... 10

11 What does the distribution of X look like? 1) Choose a sample of size n from the distribution 2) Compute x 3) Plot the x on our frequency histogram 4) Do steps times Right-skewed with n =

12 Really non-normal (mass out at the ends) with n = 2. Really non-normal (mass out at the ends) with n =

13 Turns out the random variable X is normally distributed no matter what your original distribution was IF n is large enough... What s large enough? Rule of thumb is n 30 So, what have we learned... if X is normally distributed, then X N(µ, σ 2 /n) for any n. if X is NOT normally distributed, then X N(µ, σ 2 /n) for n 30. if X is not severely non-normal, then X N(µ, σ 2 /n) is close to true for n <

14 Sampling Distributions and the Central Limit Theorem Section 7-2 Sample data is collected on a population to draw conclusions, or make statistical inferences, about the population. NOTATION: A large letter like X represents the random variable X, and X can take on many values. A small letter like x represents an actual observed x from a sample, and it is a fixed quanitity once observed. 14

15 Random Sample The random variables X 1, X 2,..., X n are a random sample of size n if... a) the X i s are independent random variables, and b) every X i has the same sample probability distribution (i.e. they are drawn from the same population). NOTE: the observed data x 1, x 2,..., x n is also referred to as a random sample. 15

16 Statistic A statistic is any function of the observations in a random sample. Example: The mean X is a function of the observations (specifically, a linear combination of the observations). X = ni=1 X i n = 1 n X 1+ 1 n X n X n A statistic is a random variable, and it has a probability distribution The distribution of a statistic is called the sampling distribution of the statistic because is depends on the sample chosen. 16

17 The sampling distribution of the mean is very important. What is the expected value of the sample mean X in a random sample? E( X) = E( 1 n X n X n X n) = 1 E(Xi ) n = 1 nµ µ = n n = µ = µ X Notation: E( X) = µ X = µ where µ is the population mean. (µ is also the expected value of a single X i ) 17

18 What is the variance of the sample mean X in a random sample? (X i s in a random sample are independent.) V ( X) = V ( 1 n X n X n X n) = = = ( ) 1 2 V (Xi ) n ( ) 1 2 σ 2 n ( 1 n ) 2 nσ 2 = σ2 n Notation: V ( X) = σ 2 X = σ2 n where σ 2 is the population variance. (σ 2 is also the variance of a single X i ) 18

19 As we have described earlier, for n 30 X N(µ, σ2 n ) (and this is also true for n < 30 if each X i comes from a normal population). Using this fact, and what we know about standardizing variables, leads to... The Central Limit Theorem If X 1, X 2,..., X n is a random sample of size n taken from a population with mean µ and variance σ 2, the limiting form of the distribution of Z = X µ σ/ n as n is the standard normal distribution, or N(0, 1). 19

20 The approximation of X µ σ/ n N(0, 1) depends on the size of n. Satisfactory approximation for n 30 for any population. Satisfactory approximation for n < 30 for near normal populations. The next graphic shows 3 different original populations (one nearly normal, two that are not), and the sampling distribution for X based on a sample of size n = 5 and size n =

21 The three original distributions are on the far left (one that is nearly symmetric and bell-shaped, one that is right skewed, and one that is highly right skewed). As shown in: Navidi, W. Statistics for Engineers and Scientists, McGraw Hill,

22 Things to notice from the previous graphic: The variability of X decreases as n increases Recall: V ( X) = σ2 n. If the original population has a shape that s closer to normal, smaller n is sufficient for X to be normal. The normal approximation gets better with larger n when you re starting with a nonnormal population. Even when X has a very non-normal distribution, X still has a normal distribution with a large enough n. 22

23 Example: Flaws in a copper wire. Let X denote the number of flaws in a 1 inch length of copper wire. The probability mass function of X is presented in the following table: x P (X = x) Suppose n = 100 wires are sampled from this population. What is the probability that the average number of flaws per wire in the sample is less than 0.5? 23

24 ANS: P ( X < 0.5) =? 24

25 Some Notation: Sampling distribution for sample mean ( X) Suppose we have a random sample of size n drawn from a parent (original) population with an expected value µ and variance σ 2. Then, X N(µ, σ2 n ) is true for sample size n > 30 no matter what the distribution of the parent population, but also true for smaller n when the parent population is normal or near-normal. Notation: E( X) = µ X = E(X) = µ V ( X) = σ 2 X = V (X) n = σ2 n 25

26 Terminology: The term standard deviation refers to the population standard deviation, or V (X) = σ, and... Z = X µ σ The term standard error is a value related to X and is also more fully stated as the standard error of the sample mean and it is the square root of the variance of X, or V ( X) = σ 2 n = σ n And then... Z = X µ σ 2 n = X µ σ/ n 26