Written by N.Nilgün Çokça. Advance Excel. Part One. Using Excel for Data Analysis

Transcription

1 Written by N.Nilgün Çokça Advance Excel Part One Using Excel for Data Analysis March, 2018

2 P a g e 1 Using Excel for Calculations Arithmetic operations Arithmetic operators: To perform basic mathematical operations such as addition, subtraction, or multiplication; combine numbers; and produce numeric results, use the following arithmetic operators. Arithmetic operator Meaning (Example) + (plus sign) Addition (3+3) (minus sign) Subtraction (3 1)Negation ( 1) * (asterisk) Multiplication (3*3) / (forward slash) Division (3/3) % (percent sign) Percent (20%) ^ (caret) Exponentiation (3^2) Mathematical functions There are a large number of mathematical functions, most of which you will never use. Find the right function is fairly simple using Help. Search for math functions. Math and Trigonometry Functions Function Description ABS COMBIN EVEN EXP FACT FACTDOUBLE FLOOR GCD INT LCM LN LOG10 ODD PI POWER PRODUCT QUOTIENT Returns the absolute value of a number Returns the number of combinations for a given number of objects Rounds a number up to the nearest even integer Returns eraised to the power of a given number Returns the factorial of a number Returns the double factorial of a number Rounds a number down, toward zero Returns the greatest common divisor Rounds a number down to the nearest integer Returns the least common multiple Returns the natural logarithm of a number Returns the base-10 logarithm of a number Rounds a number up to the nearest odd integer Returns the value of pi Returns the result of a number raised to a power Multiplies its arguments Returns the integer portion of a division

3 P a g e 2 RAND Returns a random number between 0 and 1 RANDBETWEEN Returns a random number between the numbers you specify ROUND Rounds a number to a specified number of digits ROUNDDOWN Rounds a number down, toward zero ROUNDUP Rounds a number up, away from zero SERIESSUM Returns the sum of a power series based on the formula SIGN Returns the sign of a number SQRT Returns a positive square root SQRTPI Returns the square root of (number * pi) SUBTOTAL Returns a subtotal in a list or database SUM Adds its arguments SUMIF Adds the cells specified by a given criteria SUMPRODUCT Returns the sum of the products of corresponding array components SUMSQ Returns the sum of the squares of the arguments SUMX2PY2 Returns the sum of the sum of squares of corresponding values in two arrays TRUNC Truncates a number to an integer The Insert Function dialog box opens. Type in a description of the function you want, and press Go.

4 P a g e 3 The Economic Functions and Analysis Using Excel for Data Analysis Excel has a number of functions for computing probabilities. Binomial Probabilities A binomial experiment consists of a fixed number of trials, n. On each independent trial the outcome is success or failure, with the probability of success, p, being the same for each trial. The random variable X is the number of success in n trials, so x= 0, 1,, n. For this discrete random variable, the probability that X=x is given by the probability function. n! P(X = x) = f(x) = ( x! (n x)! ) px (1 p) n x, x = 0, 1,, n We can compute these probabilities two ways; the hard way and the easy way. Computing Binomial Probabilities Directly Excel has a number of mathematical functions that make computation of formulas straightforward. Assume there are n=5 trails, that the probability of success is p = 0.3, and that we want the probability of x=3 successes. What we must compute is 5! P(X = x) = f(x) = ( 3! (5 3)! ) 33 (1 3) 5 3 Eventually you will learn many shortcuts in Excel, but should you forget how to compute some mathematical or statistical quantity, there is a Insert Function toolbar. Select Math & Trig Fact function In cell A1 type f(3) and in B1 type the formula = (FACT(5)/(FACT(3)*(FACT(2)))*(0.3^3)*(0.7^2) button on the Excel Computing Binomial Probabilities Using BINOMDIST The Excel function BINOMDIST can be used to find either cumulative probability, P(X x) or the probability function, P(X = x) for a Binomial random variable. Syntax for the function is: BINOM.DIST(number_s, trials, probability_s, cumulative) Where number_s is the number of successes in n trials. trials is the number of independent trials (n) probability is p, the probability of success on any one trial cumulative is a logical value. If set equal to 1 (true), the cumulative probability is returned; if set to 0 (false), the probability mass function is returned. Insert function (fx) Statistical BINOM.DIST in the Function name.

5 P a g e 4 Using the values n=5, p=0.3, and x=3 we obtain the probability Alternatively, we can type the function equation directly into a cell. For example, if p=0.2 and n=10, to find the probability that x=4 and x 4, the worksheet would appear as follows: =BINOM.DIST(4,10,0.2,0) =BINOM.DIST(4,10,0.2,1) The Normal Distribution Excel provides several functions related to the Normal and Standard Normal Distributions. 1. The STANDARDIZE function computes the Z value for given values of X, µ, and. The format of this function is STANDARDIZE(X; µ; ) Referring to the example which µ=3 and =3, if we wanted to find the Z value corresponding to X=6, we would enter =STANDARDIZE(6;3;3) In a cell and the value computed would be The NORMDIST function computes the area, or cumulative probability, less than a given Z value. The format of this function is NORMSDIST(Z) If we wanted to find the area below a Z value of 1.0, we would enter =NORMSDIST(1.0) in a cell, and the value computed would be The NORMDIST function computes the area or probability less than a given X value. The format of this function is NORMDIST(X; µ; ; TRUE) True is a logical value, which can be replaced by 1. If we wanted to find the area below an X value of 6, we would enter =NORMDIST(6;3;3;1) in a cell, and the value computed would be

6 P a g e 5 Stdev, Var, Skew, Kurt, Correl, Ttest... STDEV (Standard Deviation): Calculates the standard deviation of a sample. Array provides the location of the sample values. STDEV uses the following formula: n x 2 ( x) 2 n( n 1) STDEV assumes that its arguments are a sample of the population. If your data represents the entire population, then compute the standard deviation using STDEVP. The standard deviation is calculated using the "nonbiased" or "n-1" method. Example: =STDEV(G2:G4) returns the sample standard deviation of the numbers in G2:G4 G G5 =STDEV(G2:G4) returns 7,8102 which is the sample standard deviation. STDEVP (Standard Deviation of Population): Calculates the standard deviation of a population. Array provides the location of the population values. STDEVP uses the following formula: n x 2 ( x) 2 2 n STDEVP assumes that its arguments are the entire population. If your data represents a sample of the population, then compute the standard deviation using STDEV. For large sample sizes, STDEV and STDEVP return approximately equal values. The standard deviation is calculated using the "biased" or "n" method. =STDEVP(G2:G4) returns the population standard deviation of the numbers in G2:G4. G5 =STDEVP(G2:G4) returns 6,3770 which is the population standard deviation.

7 P a g e 6 Examples: 1) Residents of upstate New York are accustomed to large amounts of snow with snowfalls often exceeding 6 inches in one day. In one city, such snowfalls were recorded for two seasons and are as follows (in inches): 8,6; 9,5; 14,1; 11,5; 7,0; 8,4; 9,0; 6,7; 21,5; 7,7; 6,8; 6,1; 8,5; 14,4; 6,1; 8,0; 9,2; 7,1 What are the mean and the population standard deviation for this data, to the nearest hundredth? Snowfall: 8.6, 9.5, 14.1, 11.5, 7.0, 8.4, 9.0, 6.7, 21.5, 7.7, 6.8, 6.1, 8.5, 14.4, 6.1, 8.0, 9.2, 7.1 2) From 1984 to 1995, the winning scores for a golf tournament were 276, 279, 279, 277, 278, 278, 280, 282, 285, 272, 279, and 278. Using the standard deviation for this sample, Sx, find the percent of these winning scores that fall within one standard deviation of the mean. VAR (Variance): Calculates the variance of a sample. Array provides the location of the sample values. VAR uses the following formula: n x 2 ( x) 2 n( n 1) VAR assumes that its arguments are a sample of the population. If your data represents the entire population, then compute the standard deviation using VARP. Logical values such as TRUE and FALSE and text are ignored. If logical values and text must not be ignored, use the VARA worksheet function. Example: =VAR(G2:G4) returns the sample variance of the numbers in G2:G4 G G5 =VAR(G2:G4) returns 61 which is the sample variance. VARP (Variance of Population): Calculates the variance of a population. Array provides the location of the population values. VARP uses the following formula: n x 2 ( x) 2 2 n VARP assumes that its arguments are the entire population. If your data represents a sample of the population, then compute the variance using VAR.

8 P a g e 7 Example: =VARP(G2:G4) returns the population variance of the numbers in G2:G4. G5 =VARP(G2:G4) returns 40,667 which is the population variance. SKEW (Skewness) Returns the skewness of a distribution. Skewness characterizes the degree of asymmetry of a distribution around its mean. SKEW uses the following formula: S = (x i x ) 3 /T σ3 where σ x = 1 T (x x T t=1 i x ) 2 n x j x 3 ( ) ( n 1)( n 2) s The arguments must be either numbers or names, arrays, or references that contain numbers. If an array or reference argument contains text, logical values, or empty cells, those values are ignored; however, cells with the value zero are included. If there are fewer than three data points, or the sample standard deviation is zero, SKEW returns the #DIV/0! error value. A B C D E F G 1 6,5 7,2 6,6 7,4 8,5 8 =SKEW(A1:F1) Returns 0,37943 KURT (Kurtosis) Returns the kurtosis of a data set. Kurtosis characterizes the relative peakedness of flatness of a distribution compared with the normal distribution. Kurtosis is defined as: n( n 19 ( n 1)( n 2)( n 3) x ( J x ) s 4 2 3( n 1) ( n 2)( n 3) where: s is the sample standard deviation. If an array or reference argument contains text, logical values, or empty cells, those values are ignored; however, cells with the value zero are included. A B C D E F G H I J K =KURT(A1:J1) Returns 0,1518 Skewness measures the symmetry of the data, a value of zero indicating perfect symmetry. Kurtosis refers to the peakedness of the distribution, with a value of 3 for a normal distribution. Using these measures, the test statistic for the Jarque-Bera test for normality is JB = T 6 (S2 + (K 3)2 ) 4 Where S is skewness and K is kurtosis. This test statistic follows a chi-square distribution with 2 degrees of freedom. We will now calculate this value for the food expenditures model.

9 P a g e 8 TTEST Returns the probability associated with a Student s t-test. Uses TTEST to determine whether two samples are likely to have come from the same two underlying populations that have the same mean. TTEST(array1; array2; tails; type) Array1 is the first data set. Array2 is the second data set. Tails specifies the number of distribution tails. If tails = 1, TTEST uses the one-tailed distribution. If tails = 2, TTEST uses the two-tailed distribution. Type is the kind of t-test to perform. Example: Returns 0, A B =TTEST(A1:A9;B1:B9;2;1)

10 P a g e 9 The Economic Functions and Analysis The Simple Linear Regression Model For estimating a simple linear regression model we will use the food Expenditure data. We have 2 dataset in file food.xlsx food_exp (y) weekly food expenditure in $ income (x) weekly income in $100 We have 40 observations. In first few chapters we will use data on household food expenditure. Save As food.xlsx file food_exp.xlsx Then we will compute the summary statistics, to make sure they match the ones in food.def. we are looking for Mean, Std.Dev., Min., Max., ext.

11 P a g e 10 For this we have to install Data Analysis File Options Add-ins Go Click to File menu for add Data Analysis

12 P a g e 11 Select Add-ins Click to Go to install Analysis ToolPak From the pop-up dialog box Select Analysis ToolPak and click to OK button.

13 P a g e 12 Descriptive Statistics Excel has a tool called Descriptive Statistics, which can be found on the Tools/Data Analysis menu. You are encouraged to explore this tool. While it is a quick and easy way to obtain statistics about our residuals we will compute skewness and kurtosis directly. After it installs Analysis ToolPak, you will see Data Analysis under Data Menu. On Data Analysis you can see Correlation, Covariance, Descriptive Statistics, F-Test, and Histogram ext. To compute the summary statistics, select Descriptive Statistics. Input Range is for Data sets so we click there and select area of datasets. A1:B41 When you select A1:B41 you will notice that range turn to $A$1:$B$41 that it makes Absolute Cell reference that will not be changed if the datasets are moved.

14 P a g e 13 Tick the box Labels in First Row so that these cells will not be treated as data. We have labels in A1 and B1. Output range will put our results in the same sheet. New worksheet Ply will be best choice for us. So we will use a new sheet name it Des_Stats. Last part is for what we want : Summary Statistics Confidence level for mean OK! Will give us results for summary statistics. You can enlarge cells by double clicking between two column labels after selecting all cells. And also you can copy and paste the results to WORD.

15 P a g e 14 food_exp income Mean Mean Standard Error Standard Error Median Median Mode #N/A Mode #N/A Standard Deviation Standard Deviation Sample Variance Sample Variance Kurtosis Kurtosis Skewness Skewness Range Range Minimum Minimum 3.69 Maximum Maximum Sum Sum Count 40 Count 40 Plotting the Food Expenditure Data We introduce the simple linear regression model and estimate a model of weekly food expenditure. We also demonstrate the plotting capabilities of Excel and show how to use the software to calculate the income elasticity of food expenditure, and to predict food expenditure from our regression results. We will use Chart Wizard to scatter plot the data. Open food.xls file in Excel. Although it will select by itself, our graphic source is $A$1:$B$41.

16 P a g e 15 Cell Reference A reference identifies a cell or a range of cells on a worksheet and tells Microsoft Excel where to look for the values or data you want to use in a formula. With references, you can use data contained in different parts of a worksheet in one formula or use the value from one cell in several formulas. You can also refer to cells on other sheets in the same workbook, and to other workbooks. References to cells in other workbooks are called links. Reference to another worksheet =Sheet2!$B$2:$C$5 Relative vs. Absolute References Relative references: A relative cell reference in a formula, such as A1, is based on the relative position of the cell that contains the formula and the cell the reference refers to. If the position of the cell that contains the formula changes, the reference is changed. If you copy the formula across rows or down columns, the reference automatically adjusts. By default, new formulas use relative references. Absolute references: An absolute cell reference in a formula, such as $A$1, always refer to a cell in a specific location. If the position of the cell that contains the formula changes, the absolute reference remains the same. If you copy the formula across rows or down columns, the absolute reference does not adjust. By default, new formulas use relative references, and you need to switch them to absolute references. For example, if you copy a absolute reference in cell B2 to cell B3, it stays the same in both cells =$A$1.

17 P a g e 16 we click the graphic tools. Design menu- Data Source. when the graphic come to the screen We will edit the data set. Otherwise our graphic will be wrong. The chart will be the Food Expenditure not the income. We need to create FOOD-EXP. Chart, but Excel assumes that the first column is the X variable. You may either change the order of column or change series order. Click on Chart then Select Data command to change the data range included in the chart. Then select Edit to correct the series in chart.

18 P a g e 17 Series name, series X values, and Series Y values must be fill same as in dialog box. Chart Title: Food Expenditure Primary Horizontal Axis Title: Income Primary Vertical Axis Title: Food_Exp. Legend: None Move Chart as a new sheet. When you finish chart you can copy the chart and paste it to Word document.

19 Food_Exp. P a g e 18 Food Expenditure Income Estimating A Simple Regression To estimate the parameters b1 and b2 of the food expenditure equation, place cursor in an empty cell and click to Data menu, Data Analysis command. When Data Analysis dialog box appears, click on Regression. To define the input range, first click on the Input Y range box. Input Y range is food_exp.,($a$1$a$41) Input X Range is income, ($B$1$B$41) If you are including labels, don t forget to check the Label box! Do not check the Constant is Zero box. This would suppress the intercept. Output option will be same as descriptive statistics. New Worksheet Ply: Food_exp. and press OK button.

20 P a g e 19 Since you chose to place the output in a separate worksheet, a new worksheet will appear as a tab in the lower left corner of the work area. If you click on the Food_Exp tab, you will notice that the columns are not wide enough to show the cells completely. Highlight the data, or the entire sheet, and double click between any columns. The output contains many items that you will learn about later. For now, it is important to note that the Coefficients corresponding to Intercept and income are the least squares estimates 1 and 2.

21 food_exp P a g e 20 Plotting a Simple Regression In order to plot the regression function, we must re-estimate the food expenditure equation and choose the Line Fit Plots option in the regression dialog box. Rename chart title to Food Expenditure Regression Food Expenditure Regression food_exp Predicted food_exp income Adding a Trendline Notice that both the actual values of Y and the predicted values are plotted. To include the estimated regression function, place the cursor over one of the predicted Y points (the pink ones) until the caption "Series Predicted Y" appears.

22 Right Click and choose Add Trendline. P a g e 21

23 P a g e 22 Properties of the Least Squares Estimators In addition to the regression output, we now have the RESIDUAL OUTPUT, which gives for each observation the fitted value of y, y = b 1 + b 2 x, and the least squares residual, e t = y t y t. Some of the output is shown on the next page. Estimating the Error Variance, σ 2 In the simple Linear model the variance of the error term is var(et)= 2. To estimate this parameter we divide the sum of squared least squares residuals e t 2 by the degrees of freedom, T-2 so that σ 2 = e t 2 T 2 To compute the sum of squared residuals proceed as follows: In cell D25, square the value in cell C25 by typing =C25^2 then <ENTER>

24 P a g e 23 To copy for all observations, drag and drop formula from black square dot. (Or double click). Then get sum by ( autosum). The result is which represent e t 2 We can then calculate: σ 2 = e t 2 T = This quantity is so important it is reported automatically in the ANOVA (analysis of variance) table when a regression is estimated. The column labeled SS contains various sum of squares, the df column contains degrees of freedom, and the MS column, standing for mean squares, is the ratio SS/df. If we use the numbers in the Residual row, we see that the computed value is σ 2 Furthermore, under Regression Statistics, the Standard Error is the estimated standard error of the model. As you can verify, squaring this value gives, , is the estimated variance of the error term.

25 P a g e 24 Regression Statistics Standard Error σ = e t 2 T 2 The Variances and Covariance of the Least Squares Estimators The estimated variances and covariances of the least squares estimators are not directly reported in Excel. However, in the simple model they are easily obtained. The estimated variance of b2 is va r(b 2 ) = The standard error of the estimated coefficient is σ 2 t t=1(x i x ) 2 σ 2 se(b 2 ) = va r(b 2 ) = (x i x ) 2 In the Excel output we are given the values of the standard errors for the least squares estimates. t t=1 The standard errors are reported in the column next to the coefficient estimates. The estimation variances can be obtained by squaring the standard errors. The estimated covariances of the least squares estimators are not reported by Excel. For the simple regression model, with only one explanatory variable, calculating the covariance is not difficult. The formula for the covariance (or the variance) can be translated into an Excel formula. X cov(b 1, b 2 ) = σ 2 [ (X t X ) 2]

26 P a g e 25 From the regression worksheet, copy the standard error of the model to the worksheet containing the original data. Square the value, and label the cell. To compute the sample mean of the X values we can use Excel s AVERAGE function. Covariance and correlation analysis The covariance and correlation can tell us about the linear relationship between two variables, a primary concern of linear regression. Specifically, the covariance tells us the direction of the linear relationship, while the correlation is a measure of the strength (and direction) of the linear relationship. One should always investigate these measures when considering a linear regression. Multiple R, in the simple regression output, gives us the square root of R2 which is the correlation between x and y. A more general way to calculate covariance and correlation can be achieved by utilizing the DataAnalysis under the Data menu. The sample correlation coefficient, r, measures the direction and strength of the linear relationship between two variables and is between 1 and 1. To obtain the sample correlation coefficient, follow the instructions above for the correlation, except choose correlation from the Tool/Data Analysis menu.

27 P a g e 26 The estimated correlation between food expenditures and weekly income, r, is which is the value given as Multiple R in the regression output summary. Values on the diagonal of the correlation matrix will always equal 1. R 2, the coefficient of determination is equal to r 2, the coefficient of correlation. This is only true in the simple regression model. In an empty cell on the worksheet corr, type =B3^2. The result is which is the value reported for R square in the Summary Output from the regression results. A relationship that is always true, in simple linear regression and in the multiple regression models is that R 2 is the square of the simple correlation between the values of y and their predicted values, y. Including the variable labels is recommended because they appear in the output. If they were not included, the labels would simply be COLUMN1 and COLUMN2, which can be confusing when you don t remember which variable is in which column. The diagonal elements of the covariance matrix are the estimated sample variances. The covariance between food expenditures and weekly income is positive, suggesting a positive linear relationship. The value of the covariance, , does not, however, tell you the strength of that linear relationship. Residual Diagnostics By analyzing the residuals of the fitted model, we may be able to detect model specification problems. A histogram of the residuals can suggest the distribution of the errors, and the Jarque-Bera test statistic can be used to formally test for normality. Both of these functions

28 P a g e 27 are important since hypothesis testing and interval estimations are based on distributional assumptions. In order to create a histogram of the residuals, we need to rerun the food expenditures model and choose the Residuals output option. Testing for Normality When choosing the functional form for a particular model, we want to ensure that the resulting errors are normally distributed. Hypothesis testing and interval estimation are based on this assumption. A histogram of the residuals can suggest the distribution of the errors, and the Jarque-Bera test statistic can be used to formally test for normality. Creating a Histogram of the Residuals If not done previously, run a regression of the food expenditures model, choosing the Residuals Output option. Excel will provide the residuals for each observation, in addition to the standard Regression output.

29 P a g e 28 Examine the values of the residuals, noting the lowest and highest values. Create a BIN column next to the residuals column and determine the category values for the histogram. In this column, enter the values -250, -200, -150, -100, -50, 0, 50, 100, 150, 200 and 250. The Histogram dialog box will appear. Fill in the data ranges, by highlighting the residuals for the Input Range, and highlight the values created in the BIN column for the Bin Range. Place the output on a new worksheet called Histogram. After checking the Chart Output box, click OK. A histogram is a column chart that displays frequency data. In order to use the Histogram tool in Excel, you ll need to organize your data into two columns on the spreadsheet: one column for input data and the other for bin numbers. Input data is the data that you want to analyze. Bin numbers represent the intervals that you want the Histogram tool to use for measuring and analyzing the input Data.

30 Frequency P a g e 29 Histogram Frequency More Bin The Bin values and Frequencies appear, along with the histogram. Format the histogram graph as needed. (Remove legend, resize, rename title, etc). The residuals seem to be centered around zero, but the symmetry of the distribution seems questionable. A formal test of normality, the Jarque-Bera test, uses skewness and kurtosis, which can be easily estimated with Excel. Please repeat every step with Consumption.xls Write a report in Word document.