Announcements. Data resources: Data and GIS Services. Project. Lab 3a due tomorrow at 6 PM Project Proposal. Nicole Dalzell.

Transcription

1 Announcements UNIT 2: PROBABILITY AND DISTRIBUTIONS LECTURE 3: NORMAL DISTRIBUTION PRACTICE STATISTICS 101 Nicole Dalzell Lab 3a due tomorrow at 6 PM Proposal May 21, 2015 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Data resources: Data and GIS Services instructions posted: stat.duke.edu/ nmd16/ courses/ Summer15/ sta / problem sets/ project.pdf guides.library.duke.edu/ stat101 Think about research questions to explore. Decide if you ll be collecting your own observational data, conduct an experiment, or use previously collected data (from a published study or public database). Brainstorm due Tuesday May 25. Proposal due Thurday, June 4. due Thursday, June 18. Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

2 Data resources: Data and GIS Services - office hours For questions related to finding data and getting it into R only, not statistical analysis questions. library.duke.edu/ data/ about/ schedule.html Data resources: Hillary Mason & Bit.ly bitly.com/ bundles/ hmason/ 1 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Data resources: Reddit Data resources: DASL r/ datasets lib.stat.cmu.edu/ DASL/ Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

3 Data resources: Citizen Statistician Data resources citizen-statistician.org/ 2012/ 11/ 07/ data-sets-a-list-in-flux/ and many others, get creative! Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Getting Data in R Participation question In lab you have been given nicely formatted data that can be directly loaded into R using either load or source. This will rarely happen outside of this class, and for your project you will need to convert your data into a format R can read. Ideally use a plaintext format: csv, tab delimited, etc. (read.csv, read.table, read.delim) Avoid proprietary formats (usually doable but require extra work) Programs like Excel are useful to convert and clean up data Find your data early, if you run into trouble ask for help Scores on a standardized test are normally distributed with a mean of 100 and a standard deviation of 20. If these scores are converted to standard normal Z scores, which of the following statements will be correct? (a) Both the mean and median score will equal 0. (b) The mean will equal 0, but the median cannot be determined. (c) The mean of the z-scores will equal 100. (d) The mean of the z-scores will equal 5. Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

4 Approximating percentiles Percentiles Approximately what percent of students score below 1800 on the SAT? The mean SAT score is 1500, with a standard deviation of 300 (Hint: Use the % rule.) Percentile is the percentage of observations that fall below a given data point. Graphically, percentile is the area below the probability distribution curve to the left of that observation Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Calculating percentiles - using computation There are many ways to compute percentiles/areas under the curve: R: > pnorm(1800, mean = 1500, sd = 300) [1] Z-Scores Z-Score The z-score for a data value, x i, is z = x i x s Applet: htmls/ SOCR Distributions.html Values farther from 0 are more extreme. A z-score puts values on a common scale A z-score is the number of standard deviations a value falls from the mean 95% of all z-scores fall between -2 and 2. z-scores beyond -2 or 2 can be considered extreme Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

5 Calculating percentiles - using tables Participation question Which of the following is false? (a) Z scores are helpful for determining how unusual a data point is compared to the rest of the data in the distribution. (b) Majority of Z scores in a right skewed distribution are negative. (c) Regardless of the shape of the distribution (symmetric vs. skewed) the Z score of the mean is always 0. (d) In a normal distribution, Q1 and Q3 are more than one SD away from the mean. Second decimal place of Z Z Z-score = 1 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Example What percent of the standard normal distribution is above Z = 0.82? Choose the closest answer. (a) 79.4% (b) 20.6% (c) 82% (d) 18% (e) Need to be provided the mean and the standard deviation of the distribution in order to be able to solve this problem. The average daily temperature in June in LA is 77 F, with a standard deviation of 5 degrees. Suppose the temperatures in June closely follow a normal distribution. What is the probability of observing a temperature of at most 83 F on a randomly chosen day in June? ) P(T 83) = P T N (mean = 77, sd = 5) ( Z ) = P (Z 1.2) Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

6 Calculating percentiles - using tables Example Second decimal place of Z Z Z-score = 1.2 The average daily temperature in June in LA is 77 F, with a standard deviation of 5 degrees. Suppose the temperatures in June closely follow a normal distribution. What is the probability of observing a temperature of at most 83 F on a randomly chosen day in June? ) P(T 83) = P T N (mean = 77, sd = 5) ( Z ) = P (Z 1.2) The probability of observing a temperature of at most 83 F on a randomly chosen day in June is approximately 0.885, or 88.5%. Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Example (cont) Example The average daily temperature in June in LA is 77 F, with a standard deviation of 5 degrees. Suppose the temperatures in June closely follow a normal distribution. What is the probability of observing a temperature of at least 83 F on a randomly chosen day in June? ) T N (mean = 77, sd = 5) P(T 83) = 1 P(T 83) = The probability of observing a temperature of at least 83 F on a randomly chosen day in June is approximately 0.115, or 11.5%. The average daily temperature in June in LA is 77 F, with a standard deviation of 5 degrees. Suppose the temperatures in June closely follow a normal distribution. What is the probability of observing a temperature of at most 83 F on a randomly chosen day in June? ) P(T 83) = P T N (mean = 77, sd = 5) ( Z ) = P (Z 1.2) The probability of observing a temperature of at most 83 F on a randomly chosen day in June is approximately 0.885, or 88.5%. Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

7 Example (cont) The average daily temperature in June in LA is 77 F, with a standard deviation of 5 degrees. Suppose the temperatures in June closely follow a normal distribution. What is the probability of observing a temperature of at least 83 F on a randomly chosen day in June? ) T N (mean = 77, sd = 5) This study also found that approximately 25% of Facebook users are considered power users. The same study found that the average Facebook user has 245 friends. What is the probability that the average Facebook user with 245 friends has 70 or more friends who would be considered power users? We are given that n = 245, p = 0.25, and we are asked for the probability P(K 70). P(T 83) = 1 P(T 83) = The probability of observing a temperature of at least 83 F on a randomly chosen day in June is approximately 0.115, or 11.5%. P(K 70) = P(K = 70 or K = 71 or K = 72 or or K = 245) = P(K = 70) + P(K = 71) + P(K = 72) + + P(K = 245) This seems like an awful lot of work... Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Histograms of number of successes Hollow histograms of samples from the binomial model where p = 0.10 and n = 10, 30, 100, and 300. What happens as n increases? When the sample size is large enough, the binomial distribution with parameters n and p can be approximated by the normal model with parameters µ = np and σ = np(1 p). In the case of the Facebook power users, n = 245 and p = µ = = σ = = 6.78 Bin(n = 245, p = 0.25) N(µ = 61.25, σ = 6.78) n = n = Bin(245,0.25) N(61.5,6.78) n = n = k Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

8 When the sample size is large enough, the binomial distribution with parameters n and p can be approximated by the normal model with parameters µ = np and σ = np(1 p). In the case of the Facebook power users, n = 245 and p = µ = = σ = = 6.78 Bin(n = 245, p = 0.25) N(µ = 61.25, σ = 6.78) Bin(245,0.25) N(61.5,6.78) What is the probability that the average Facebook user with 245 friends has 70 or more friends who would be considered power users? (a) (b) (c) (d) k Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Low large is large enough? Participation question The sample size is considered large enough if the expected number of successes and failures are both at least 10. np 10 and n(1 p) 10 Below are four pairs of Binomial distribution parameters. Which distribution can be approximated by the normal distribution? (a) n = 100, p = 0.95 (b) n = 25, p = 0.45 (c) n = 150, p = 0.05 (d) n = 500, p = Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

9 Application exercises Finding probabilities // Quality control Application exercises Finding cutoff points // Hot bodies At Heinz ketchup factory the amounts which go into bottles of ketchup are supposed to be normally distributed with mean 36 oz. and standard deviation 0.11 oz. Once every 30 minutes a bottle is selected from the production line, and its contents are noted precisely. If the amount of the bottle goes below 35.8 oz. or above 36.2 oz., then the bottle fails the quality control inspection. What percent of bottles pass the quality control inspection? Body temperatures of healthy humans are distributed nearly normally with mean 98.2 F and standard deviation 0.73 F. What is the cutoff for the highest 10% of human body temperatures? Mackowiak, Wasserman, and Levine (1992), A Critical Appraisal of 98.6 Degrees F, the Upper Limit of the Normal Body Temperature, and Other Legacies of Carl Reinhold August Wunderlick. Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Application exercises Conditional probability // SAT scores Application exercises Finding missing parameters // Auto insurance premiums SAT scores (out of 2400) are distributed normally with mean 1500 and standard deviation 300. Suppose a school council awards a certificate of excellence to all students who score at least 1900 on the SAT. What percent of the students who received this certificate scored above 2100? P(SAT > 2100 and SAT > 1900) P(SAT > 2100 SAT > 1900) = P(SAT > 1900) P(SAT > 2100) = P(SAT > 1900) ( ) P(SAT > 2100) = P 300 = P(Z > 2) = = P(X > 1900) = P(Z > 1.33) = = P(SAT > 2100 SAT > 1900) = % of students Suppose a newspaper article states that the distribution of auto insurance premiums for residents of California is approximately normal with a mean of $1,650. The article also states that 25% of California residents pay more than $1, What is the standard deviation of this distribution? 2. What is the IQR of this distribution? Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37 Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37

10 To Do To Do PS 3 due tomorrow in class Reading Assignment for Friday: Chapter 4 Sections ( A sampling Distribution for the mean) Statistics 101 (Nicole Dalzell) U2 - L2: Normal distribution May 21, / 37