Introduction to R (2)

Transcription

1 Introduction to R (2) Boxplots Boxplots are highly efficient tools for the representation of the data distributions. The five number summary can be located in boxplots. Additionally, we can distinguish outliers using boxplots. Remember the three distributions in the previous handout (figures 6, 7 and 8). Note that these distributions are symmetric, left skewed and right skewed respectively. Here is how I created figure 1 below: > par(mfrow=c(3,2)) > hist(n1) > boxplot(n1) > hist(n3) > boxplot(n3) > hist(n5) > boxplot(n5) Note that the command par(mf row) creates a 3 by 2 grid in the graphic area. The side-by-side boxplots are very helpful in comparing two or more distributions. For example figure 2 shows a side-by-side boxplots for the two skewed distributions of figure 1. > boxplot(n3,n5) 1

2 Histogram of n n1 Histogram of n n3 Histogram of n n5 Figure 1. Histograms along with boxplots for the distributions of the previous handout. 2

3 Figure 2. Side-by-side boxplots for the two skewed-distributions of figure 1. 3

4 Linear Transformations and Their Effect on x and s. To experiment with the idea of linear transformations, let s assume that we have the following sample of observations. > test1 [1] [10] [19] [28] [37] [46] [55] [64] [73] [82] [91] [100] the mean and the standard deviation of these number are: > mean(test1) [1] > sd(test1) [1] Note also that the distribution of these numbers is fairly symmetric as shown in figure 3. Let us perform the following simple linear transformation on these numbers: Y =3+2 X (1) 4

5 Histogram of test test1 Figure 3. Histogram of the data set test1. 5

6 Here is the code: > test2<-2*test1+3 > mean(test2) [1] > sd(test2) [1] > hist(test2) Note that the shape of the distribution remains intact yet the linear transformation affects the mean and the standard deviation: > *2 + 3 [1] > *2 [1]

7 Histogram of test test2 Figure 4. Histogram of the linearly transformed data set or test2. 7

8 Verification of the 68% - 95% % Rule To verify the rule, first I generate 100,000 observations from a normal distribution whose mean is 0 and whose standard deviation is 1. Next, I count the number of simulations whose values are between -1 to 1, -2 to 2 and -3 to 3 respectively. > test3<-rnorm(100000,0,1) > mean(test3) [1] > sd(test3) [1] > length(test3[test3>-1 & test3<1]) [1] > length(test3[test3>-2 & test3<2]) [1] > length(test3[test3>-3 & test3<3]) [1] Standardization By subtracting data points from their mean and dividing them by their standard deviation, we will generate a new data set whose mean is 0 and whose standard deviation is 1. Here is an example: > test<-c(1,2,3,4,5) > mean(test) [1] 3 > sd(test) [1] > mean(test-mean(test)) [1] 0 8

9 Histogram of test test3 Figure 5. Histogram of the 100,000 simulated observations from a Normal(0,1) distribution. 9

10 > sd(test/sd(test)) [1] 1 > z.test<-(test-mean(test))/sd(test) > z.test [1] > mean(z.test) [1] 0 > sd(z.test) [1] 1 10

11 Areas under the Normal Curve We can re-produce the z table using R. The two commands of interest are qnorm and pnorm to calculate the quantiles and the probabilities respectively. For example to find the area associated with Z< 1.3, we read from the z table. We can do the same using the pnorm command: > pnorm(-1.3,0,1) [1] Note that the syntax is such that we input the cut-off value of interest, followed by the mean and the standard deviation of the normal distribution. In that context, we are not bound to the standard normal distribution, but we can calculate the areas under any normal distribution. Conversely, we can calculate the cut-off value below which the area of the standard normal is say We should get 1.3 back. This is done using the qnorm command: > qnorm( ,0,1) [1] -1.3 Normal Quantile Plots or QQ- Assessing Normality: plots These are the most powerful tools for assessing the normality of a data set. The idea is relatively simple. We want to know whether the empirical quantiles of our data will match with the theoretical quantiles of a standard normal. This can be achieved by plotting the data distributions against their associated empirical and theoretical quantiles. Data will form a 45-degree line if normality holds. R does provides QQ-plots by the qqnorm command. Here is the QQ-plot for the data set test3 (figure 1): > qqnorm(test3) In figure 2, we demonstrate the qqplot of a symmetric, a left-skewed and a right-skewed distribution. The code below, shows you how I generated this figure. > par(mfrow=c(3,2)) > hist(n1) > qqnorm(n1) > hist(n3) 11

12 0 2 4 Sample Quantiles 2 4 Normal Q Q Plot Theoretical Quantiles Figure 1. The Normal Quantile plot for test

13 > qqnorm(n3) > hist(n5) > qqnorm(n5) 13

14 Histogram of n1 Normal Q Q Plot Sample Quantiles n Theoretical Quantiles Histogram of n3 Normal Q Q Plot Sample Quantiles n Theoretical Quantiles Histogram of n5 Normal Q Q Plot Sample Quantiles n Theoretical Quantiles Figure 2. The Normal Quantile plots for symmetric and asymmetric distributions. 14

15 Data Relationships Importing Text-files > grades<-read.table(" > grades Verbal Math GPA > dim(grades) [1] > grades[,1] [1] [19] [37] [55]

16 [73] [91] > grades[,"verbal"] [1] [19] [37] [55] [73] [91] > hist(grades[,"verbal"]) > summary(grades[,1]) Min. 1st Qu. Median Mean 3rd Qu. Max > verbal<-grades[,1] > stem(verbal) The decimal point is 2 digit(s) to the right of the

17 Histogram of grades[, 1] grades[, 1] Figure 3. The histogram of the first column of grades generated by R Histogram of grades[, "Verbal"] grades[, "Verbal"] Figure 4. The histogram of the first column of grades (called Verbal ) 17

18 grades[, 3] grades[, 1] Figure 5. The scatterplot for Verbal versus Math Scatterplots and Pearson s Correlation To create scatterplots, the command is simply plot. > plot(grades[,1],grades[,2]) To calculate the correlation coefficient between any two random variables, we can use the cor command: > cor(grades[,1],grades[,2]) [1] > cor(grades[,1],grades[,3]) [1] > cor(grades[,2],grades[,3]) [1]

19 grades[, 2] grades[, 1] Figure 6. The scatterplot for Verbal versus GPA grades[, 3] grades[, 2] Figure 7. The scatterplot for Math versus GPA 19