1.2 Describing Distributions with Numbers, Continued

Transcription

1 1.2 Describing Distributions with Numbers, Continued Ulrich Hoensch Thursday, September 6, 2012

2 Interquartile Range and 1.5 IQR Rule for Outliers The interquartile range IQR is the distance between the first and the third quartiles of a distribution: IQR = Q 3 Q 1. The 1.5 IQR Rule for Outliers states that an observation is a suspected outlier if it is smaller than Q 1 (1.5 IQR), or it is larger than Q 3 + (1.5 IQR).

3 Example 1: Call Length Data The call lengths (service times) given in the next silde have the following five-number summary (in seconds): The IQR is = 145.5, so any value below 54.5 ( ) = , or above ( ) = would be considered an outlier. Indeed, there are 8 such outliers in the data (two of which are identical). These outliers can be represented in a modified box plot as shown below. (Note that the box plot is constructed from the data obtained by dropping these outliers.)

4 Example 1: Call Length Data

5 Example 1: Call Length Data

6 Standard Deviation Given a data set x 1, x 2,..., x n, the sum of squared deviations is the sum of the squares of differences of each value from the mean: ss = (x 1 x) 2 + (x 2 x) (x n x) 2 = (x x) 2. the variance is the average sum of the squares of deviations from the mean, and can be computed as s 2 = (x 1 x) 2 + (x 2 x) (x n x) 2 n 1 = ss n 1. (note the expression n 1, and not n in the denominator); the standard deviation is the positive square root of the variance: (x x) 2 s = s 2 = n 1.

7 Example 2: Metabolic Rates The basal metabolic rates of 7 men, in calories per day, are shown here To compute the standard deviation, we follow these steps. 1. Compute the mean: x = = = cal/day. 2. Compute the individual deviations from the mean: Value Mean Deviation/Difference =

8 Example 2: Metabolic Rates 3. Square the deviations: Value Mean Deviation (Deviation) (192.0) 2 = Add the squared deviations: ss = = Divide by 7 1 = 6: s 2 = / Take the square root: s = calories/day.

9 Example 2: Metabolic Rates Calculator Use: We can also enter the data into the TI-83/84 calculator and compute the relevant measures. Step 1: Press the STAT key. Step 2: Enter the data in List 1.

10 Example 2: Metabolic Rates Step 3: Press the STAT key, arrow over to the CALC menu. Step 4: Press ENTER when 1-Var Stats appears. The standard deviation is s =

11 Example 2: Metabolic Rates Step 5: By scrolling down on the last screen we can also find the five-number summary.

12 Remarks about the Standard Deviation The expression n 1 in the denominator of the formula for s 2 and s is called the degrees of freedom. This is because if n 1 of the deviations in the sum of squares are known, then the remaining deviation can be found. The standard deviation, just like the mean, is not resistant; a few extreme values can make s very large. s = 0 precisely when all the values in the data set are the same. Once we get to know about normal distributions (in the next section), we will able to interpret the standard deviation in terms of the distribution of a variable.

13 Choosing a Summary If the distribution is skewed or has strong outliers, the five-number summary should be used to summarize the data. If a distribution is reasonably symmetric, free of outliers and bell-shaped, the mean and the standard deviation can be used.

14 IQ Scores of Fifth-Grade Students What is an appropriate summary for the data? The five-number summary? The mean and the standard deviation?

15 Number of Letters in Shakespeare s Words What is an appropriate summary for the data? The five-number summary? The mean and the standard deviation?

16 Iowa Test Vocabulary Scores of Seventh-graders What is an appropriate summary for the data? The five-number summary? The mean and the standard deviation?

17 Tuition and Fees of a Sample of Colleges What is an appropriate summary for the data? The five-number summary? The mean and the standard deviation?

18 Data Reference Data sets will typically appear in the columns of an Excel spreadsheet; each value is placed in a cell, and referenced to via its horizontal and vertical coordinates. In the picture below, a data set of twelve values resides in the cells A3 through A14.

19 Using Worksheet Functions MS Excel provides a variety of statistical functions. For example, to find the mean of the twelve values above, first click on an empty cell in which the result should appear (e.g. the cell B3). Then, click on the Formulas tab, and select the functions button on the left.

20 Using Worksheet Functions Next, choose Statistical as the function category, select AVERAGE, and press OK.

21 Using Worksheet Functions Then enter A3:A14 into the field with caption Number 1 (or move the box out of the way, and select the cells by dragging the mouse over them with the left mouse button down).

22 Using Worksheet Functions Press OK, and then your spreadsheet should look like this: In effect, we entered =AVERAGE(A3:A14)into the cell B3.

23 Other Worksheet Functions =MEDIAN(A3:A14) returns the median of the values in cells A3 through A14 =MAX(A3:A14) returns the largest value in cells A3 through A14 =MIN(A3:A14) returns the smallest value in cells A3 through A14 =VAR(A3:A14) returns the sample variance of the values in cells A3 through A14 (use if the data set represents a sample) =STDEV(A3:A14) returns the sample standard deviation of the values in cells A3 through A14 (use if the data set represents a sample) =PERCENTILE(A3:A14,0.85) returns the value corresponding to the 85th percentile using the values in cells A3 through A14 =QUARTILE(A3:A14,1) returns the 1st quartile using the values in cells A3 through A14 =COUNTIF(A3:A14,8) returns the number of values in the cells A3 through A14 that are equal to 8