11/28/2018. Overview. Multiple Linear Regression Analysis. Multiple regression. Multiple regression. Multiple regression. Multiple regression

Transcription

1 Multiple Linear Regression Analysis BSAD 30 Dave Novak Fall 208 Source: Ragsdale, 208 Spreadsheet Modeling and Decision Analysis 8 th edition 207 Cengage Learning 2 Overview Last class we considered the relationship between one independent variable and one dependent variable Referred to as simple linear regression Today, we consider the relationship between more than one independent variable (X s) and a single dependent variable (Y) Referred to as multiple linear regression Example When more than one independent variable can be used to explain variance in Y Assume that you want to develop a model to predict the market value (or price) of houses in your town / city We have access to the following data 3 Y i = b 0 + b X i + b 2 X 2i +. + b n X ni Where it is assumed all b i s are independent 4 Sq. Feet Garage Obs (in 000s) (# cars) # Bedrooms Price (in 000s) $ $ $ $ $ $ $ $ $ $ Source: Ragsdale, 208 Spreadsheet Modeling and Decision Analysis 8 th We want to predict housing price (in thousands of $) using some combination of data we have on square footage, the garage size, and number of bedrooms (3 possible X values) Even with three independent variables, we can create many different regression models A model with the most X s is often not the best model Having access to many different independent variables does not necessarily mean that they all should be part of a regression model Rule of thumb for linear regression: KEEP IT AS SIMPLE AS POSSIBLE Start by looking at scatter plots and correlation 5 6

2 Selling Price. Selling Price. Selling Price. /28/ Square Footage 0 2 Size of Garage Bedrooms Look at correlation coefficient (r) for each X / Y combination Price (in 000s) Sq. Feet (in 000s) Sq. Feet (in 000s) Price (in 000s) Garage (# cars) Garage (# cars) Price (in 000s) # Bedrooms # Bedrooms Source: Ragsdale, 208 Spreadsheet Modeling and Decision Analysis 8 th 8 Given results from scatter and correlation, start with three separate simple linear regression models and compare results Y i = b 0 + b X i (X = square footage) Y i = b 0 + b 2 X 2i Y i = b 0 + b 3 X 3i (X 2 = garage size) (X 3 = # bedrooms) X square footage Multiple R R Square Adjusted R Square Standard Error E-05 Residual E Sq. Feet (in 000s) E X2 garage size Multiple R R Square Adjusted R Square Standard Error Residual Total CoefficientsStandard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% E Garage (# cars) X3 # bedrooms Multiple R R Square Adjusted R Square Standard Error Residual CoefficientsStandard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% E # Bedrooms

3 X X has highest R 2,Adj R 2, and lowest Std. error Reasonable to start with X and build off that variable Combinations of two variables Y i = b 0 + b X i + b 2 X 2i (square footage + garage) Y i = b 0 + b X i + b 3 X 3i (square footage + # bedrooms) 3 4 X + X2 (Sq ft + Garage) X + X3 (Sq ft + Bedrooms) Multiple R R Square Adjusted R Square Standard Error E-05 Regression 2 Residual E X Variable X Variable Multiple R R Square Adjusted R Square Standard Error Regression 2 Residual E X Variable X Variable Change in b 0 and b Notice values of b 0 and b have changed from the model where we just had X Y i = b 0 + b X i (b 0 = 09.5, b = ) Y i = b 0 + b X i + b 2 X 2i (b 0 = 27.68, b = ) Y i = b 0 + b X i + b 3 X 3i (b 0 = 08.3, b = 44.33) X X + X X + X Where X = sq. ft., X2 = garage, X3 = # bedrooms 7 8 3

4 Multicollinearity Not surprising adding X 3 (# of bedrooms) to regression model with X (total square footage) did not improve model Both variables represent similar things a measure of house size (sq ft) These variables appear to be highly correlated Sq. Feet (in 000s) Garage (# cars) Sq. Feet (in 000s) Garage (# cars) Combination of all three Y i = b 0 + b X i + b 2 X 2i + b 3 X 3i (square footage + garage + # bedrooms) As it is not a time consuming undertaking to test all three variables, we also want to examine the FULL (all independent variables) model 9 Sq. Feet (in 000s) # Bedrooms Sq. Feet (in 000s) # Bedrooms X + X2 + X3 Multiple R R Square Adjusted R Square Standard Error Regression 3 Residual E X Variable X Variable X Variable X X + X X + X X + X2 + X Best fit How do we choose? The two variable model with X and X2 has highest adj R 2 and lowest std. error of all models Making the model more complex by adding all three variables doesn t add anything to predictive power We also know that X3 is highly correlated with X (X and X3 not necessarily independent) 24 Making predictions Use our selected model to estimate average selling price of house with 2,00 sq ft and a 2-car garage Y i = X i X 2i 4

5 Making predictions 95% prediction interval for the actual selling price: Problem In-class example problem