Announcements. Unit 2: Probability and distributions Lecture 3: Normal distribution. Normal distribution. Heights of males

Transcription

1 Announcements Announcements Unit 2: Probability and distributions Lecture 3: Statistics 101 Mine Çetinkaya-Rundel First peer eval due Tues. PS3 posted - will be adding one more question that you need to use R for (will be posted this evening). January 31, 2013 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Heights of males Unimodal and symmetric, bell shaped curve Most variables are nearly normal, but none are exactly normal Denoted as N(µ, σ) Normal with mean µ and standard deviation σ The male heights on OkCupid very nearly follow the expected normal distribution except the whole thing is shifted to the right of where it should be. Almost universally guys like to add a couple inches. You can also see a more subtle vanity at work: starting at roughly 5 8, the top of the dotted curve tilts even further rightward. This means that guys as they get closer to six feet round up a bit more than usual, stretching for that coveted psychological benchmark. blog.okcupid.com/ index.php/ the-biggest-lies-in-online-dating/ Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

2 model Heights of females s with different parameters µ: mean, σ: standard deviation N(µ = 0, σ = 1) N(µ = 19, σ = 4) When we looked into the data for women, we were surprised to see height exaggeration was just as widespread, though without the lurch towards a benchmark height blog.okcupid.com/ index.php/ the-biggest-lies-in-online-dating/ Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / Rule Rule Rule For nearly normally distributed data, about 68% falls within 1 SD of the mean, about 95% falls within 2 SD of the mean, about 99.7% falls within 3 SD of the mean. It is possible for observations to fall 4, 5, or more standard deviations away from the mean, but these occurrences are very rare if the data are nearly normal. Describing variability using the Rule SAT scores are distributed nearly normally with mean 1500 and standard deviation % of students score between 1200 and 1800 on the SAT. 95% of students score between 900 and 2100 on the SAT. 99.7% of students score between 600 and 2400 on the SAT. 68% 95% 99.7% 68% 95% 99.7% µ 3σ µ 2σ µ σ µ µ + σ µ + 2σ µ + 3σ Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

3 Rule Number of hours of sleep on school nights % 95 % 75 % SAT scores are distributed nearly normally with mean 1500 and standard deviation 300. ACT scores are distributed nearly normally with mean 21 and standard deviation 5. A college admissions officer wants to determine which of the two applicants scored better on their standardized test with respect to the other test takers: Pam, who earned an 1800 on her SAT, or Jim, who scored a 24 on his ACT? Pam Jim Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Since we cannot just compare these two raw scores, we instead compare how many standard deviations beyond the mean each observation is. Pam s score is = 1 standard deviation above the mean. Jim s score is = 0.6 standard deviations above the mean. Jim Pam (cont.) These are called standardized scores, or Z scores. Z score of an observation is the number of standard deviations it falls above or below the mean. Z scores Z = observation mean SD Z scores are defined for distributions of any shape, but only when the distribution is normal can we use Z scores to calculate percentiles. Observations that are more than 2 SD away from the mean ( Z > 2) are usually considered unusual. Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

4 Percentiles Percentile is the percentage of observations that fall below a given data point. Graphically, percentile is the area below the probability distribution curve to the left of that observation. Approximately what percent of students score below 1800 on the SAT? (Hint: Use the % rule.) Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Calculating percentiles Calculating percentiles Calculating percentiles - using computation There are many ways to compute percentiles/areas under the curve: R: > pnorm(1800, mean = 1500, sd = 300) [1] Applet: htmls/ SOCR Distributions.html Calculating percentiles - using tables Second decimal place of Z Z Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

5 Recap Clicker question Which of the following is false? (a) Majority of Z scores in a right skewed distribution are negative. (b) In skewed distributions the Z score of the mean might be different than 0. (c) For a normal distribution, IQR is less than 2 SD. (d) Z scores are helpful for determining how unusual a data point is compared to the rest of the data in the distribution. Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Evaluating the normal approximation Normal probability plot A histogram and normal probability plot of a sample of 100 male heights. Male heights (inches) Theoretical Quantiles male heights (in.) Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Evaluating the normal approximation Anatomy of a normal probability plot Data are plotted on the y-axis of a normal probability plot, and theoretical quantiles (following a normal distribution) on the x-axis. If there is a one-to-one relationship between the data and the theoretical quantiles, then the data follow a nearly normal distribution. Since a one-to-one relationship would appear as a straight line on a scatter plot, the closer the points are to a perfect straight line, the more confident we can be that the data follow the normal model. Constructing a normal probability plot requires calculating percentiles and corresponding z-scores for each observation, which is tedious. Therefore we generally rely on software when making these plots. Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Evaluating the normal approximation Below is a histogram and normal probability plot for the NBA heights from the season. Do these data appear to follow a normal distribution? Height (inches) Theoretical quantiles Why do the points on the normal probability have jumps? Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

6 Evaluating the normal approximation Construct a normal probability plot for the data set given below and determine if the data follow an approximately normal distribution. 3.46, 4.02, 5.09, 2.33, 6.47 Observation i x i Percentile = i n Corrsponding Z i Since the points on the normal probability plot seem to follow a straight line we can say that the distribution is nearly normal. Evaluating the normal approximation Normal probability plot and skewness Right Skew - If the plotted points appear to bend up and to the left of the normal line that indicates a long tail to the right. Left Skew - If the plotted points bend down and to the right of the normal line that indicates a long tail to the left. Short Tails - An S shaped-curve indicates shorter than normal tails, i.e. narrower than expected. Long Tails - A curve which starts below the normal line, bends to follow it, and ends above it indicates long tails. That is, you are seeing more variance than you would expect in a normal distribution, i.e. wider than expected. Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Six sigma The term six sigma process comes from the notion that if one has six standard deviations between the process mean and the nearest specification limit, as shown in the graph, practically no items will fail to meet specifications. Clicker question At Heinz ketchup factory the amounts which go into bottles of ketchup are supposed to be normally distributed with mean 36 oz. and standard deviation 0.11 oz. Once every 30 minutes a bottle is selected from the production line, and its contents are noted precisely. If the amount of ketchup in the bottle is below 35.8 oz. or above 36.2 oz., then the bottle fails the quality control inspection. What percent of bottles have less than 35.8 ounces of ketchup? (a) less than 0.15% (b) between 0.15% and 2.5% (c) between 2.5% and 16% (d) between 16% and 50% en.wikipedia.org/ wiki/ Six Sigma Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

7 Finding the exact probability - using the Z table Finding the exact probability - using R Second decimal place of Z Z > pnorm(-1.82, mean = 0, sd = 1) [1] OR > pnorm(35.8, mean = 36, sd = 0.11) [1] Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Finding cutoff points Clicker question At Heinz ketchup factory the amounts which go into bottles of ketchup are supposed to be normally distributed with mean 36 oz. and standard deviation 0.11 oz. Once every 30 minutes a bottle is selected from the production line, and its contents are noted precisely. If the amount of the bottle goes below 35.8 oz. or above 36.2 oz., then the bottle fails the quality control inspection. What percent of bottles pass the quality control inspection? (a) 1.82% (b) 3.44% (c) 6.88% (d) 93.12% (e) 96.56% Body temperatures of healthy humans are distributed nearly normally with mean 98.2 F and standard deviation 0.73 F. What is the cutoff for the lowest 3% of human body temperatures? 0.03? Z P(X < x) = 0.03 P(Z < -1.88) = 0.03 obs mean Z = x 98.2 = 1.88 SD 0.73 x = ( ) = 96.8 Mackowiak, Wasserman, and Levine (1992), A Critical Appraisal of 98.6 Degrees F, the Upper Limit of the Normal Body Temperature, and Other Legacies of Carl Reinhold August Wunderlick. Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29

8 Finding cutoff points Clicker question Body temperatures of healthy humans are distributed nearly normally with mean 98.2 F and standard deviation 0.73 F. What is the cutoff for the highest 10% of human body temperatures? (a) 99.1 (b) 97.3 (c) 99.4 (d) 99.6 Statistics 101 (Mine Çetinkaya-Rundel) U2 - L3: January 31, / 29