Ordinal Multinomial Logistic Regression. Thom M. Suhy Southern Methodist University May14th, 2013

Transcription

1 Ordinal Multinomial Logistic Thom M. Suhy Southern Methodist University May14th, 2013

2 GLM Generalized Linear Model (GLM) Framework for statistical analysis (Gelman and Hill, 2007, p. 135) Linear Continuous data Logistic Binary data Ordered Multinomial Logistic Unordered Multinomial Logistic

3 Logistic Dependent variable is dichotomous Yes or No Apply or Not Apply Pass or Fail Heisman or no Heisman Probability of trait (yes, apply, pass, Heisman) based on independent variables Independent variable does not need to be dichotomous Categorical Integral Dichotomous Nominal Ordinal

4 Logistic Refresher Call: glm(formula = comply ~ physrec, family = binomial(link = "logit")) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-06 *** physrec e-07 *** --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 163 degrees of freedom Residual deviance: on 162 degrees of freedom AIC: Number of Fisher Scoring iterations: 4

5 Logistic Refresher Formula for logit ## logit= (2.2882*physrec) ## ## logit = for no physrec ## ## logit =.4499 for yes physrec ## Probability to comply exp( )/(1+(exp( ))) Probability of comply with no physrec =.137 or 13.7% exp(.4499)/(1+(exp(.4499))) Probability of comply with physrec =.6106 or 61%

6 Logistic Refresher factor(comply) physrec

7 Logistic vs. Ordered Multinomial How are they different? An extension of logistic regression to multiple categories (Gelman & Hill, 2007) Not binary, categorical (but ordered) Decision (Yes, Maybe, No) Order of Finish (1 st, 2 nd, 3 rd ) Likert Scale (Strongly Disagree Strongly Agree) Income ranges (0 25K, 25K-50K, 50K+) Degree (None, Bachelors, Masters, PhD) There is unordered multinomial logistic regression, but that is not for today!

8 A Little More Information Ordinal multinomial logistic regression is an extension of logistic regression using multiple categories that have a logical order. (Gelman & Hill, 2007) Ordinal data are the most frequently encountered type of data in the social sciences (Johnson & Albert, 1999, p. 126).

9 Running a Model in R 1. We are going to use a file from UCLA, but first load your libraries : > library(psych) > library(arm) 2. Now we will read in our data: > suhy<- read.dta(url(" edu/stat/r/dae/ologit.dta"))

10 Running a Model in R 3. Let s examine our data: > head(suhy) apply pared public gpa 1 very likely somewhat likely unlikely somewhat likely somewhat likely unlikely

11 Running a Model in R Defining our variables: apply = Likelihood of college juniors applying to grad school. (Self-reported)(very likely, somewhat likely, unlikely) pared = Does at least one parent have a graduate degree? (no=0, yes=1) public = Undergrad was a private or public institution. (private = 0, public = 1) gpa = Undergrad grade point average

12 Running a Model in R What are our assumptions? Data are case specific iv has a single value for each case No perfect predictors no single predicator variable, iv can determine the outcome of the dv No zero or very small quantities in a crosstab cell Sample size larger than normal OLS regression

13 Running a Model in R 4. Let us check our assumptions: >xtabs(~suhy$pared+suhy$apply) suhy$pared unlikely somewhat likely very likely > xtabs(~suhy$public+suhy$apply) suhy$public unlikely somewhat likely very likely

14 Running a Model in R 5. We are good to go, let s run the model: > summary(m1<-bayespolr(as.ordered(suhy$apply)~suhy$gpa)) Call: bayespolr(formula = as.ordered(suhy$apply) ~ suhy$gpa) Coefficients: Value Std. Error t value suhy$gpa Intercepts: Value Std. Error t value unlikely somewhat likely somewhat likely very likely Residual Deviance: AIC:

15 Running a Model in R A visual: Thank you Pooja Shivraj (2012)

16 Running a Model in R 6. Lets calculate the probabilities for the average gpa > x<-mean(suhy$gpa) x = > coef<-m1$coef > coef suhy$gpa > intercept<-m1$zeta > intercept unlikely somewhat likely somewhat likely very likely

17 Running a Model in R 6. Let s calculate the probabilities for the average gpa (cont.) Remember: > prob<-function(input){exp(input)/(1+exp(input))} > (p0<-prob(intercept[1]-coef*x)) unlikely somewhat likely OR 55% > (p1<-prob(intercept[2]-coef*x)-p0) somewhat likely very likely OR 35% > (p2<-1-(p0+p1)) very likely OR 9% p0+p1+p2 always equal 1 when using 3 categories

18 Running a Model in R 2.5 GPA > (p0<-prob(intercept[1]-coef*2.5)) 3.7 GPA > (p0<-prob(intercept[1]-coef*3.7)) unlikely somewhat likely unlikely somewhat likely > (p1<-prob(intercept[2]-coef*2.5)-p0) > (p1<-prob(intercept[2]-coef*3.7)-p0) somewhat likely very likely somewhat likely very likely > (p2<-1-(p0+p1)) > (p2<-1-(p0+p1)) very likely very likely

19 Running a Model in R Now you tell me the probability for each category if you had a 4.0 GPA. > (p0<-prob(intercept[1]-coef*4.0)) unlikely somewhat likely = 37% > (p1<-prob(intercept[2]-coef*4.0)-p0) somewhat likely very likely = 44% > (p2<-1-(p0+p1)) very likely = 18%

20 Multiple Predictors 1. Let s look at a model with multiple predictors: > summary(m2<-bayespolr(as.ordered(suhy$apply)~suhy$gpa+suhy$pared+suhy$public)) Call: bayespolr(formula = as.ordered(suhy$apply) ~ suhy$gpa + suhy$pared + suhy$public) Coefficients: Value Std. Error t value suhy$gpa suhy$pared suhy$public Intercepts: Value Std. Error t value unlikely somewhat likely somewhat likely very likely Residual Deviance: AIC:

21 Multiple Predictors 2. Let s calculate the probabilities: >(coef<- m2$coef) suhy$gpa suhy$pared suhy$public > (intercept<-m2$zeta) unlikely somewhat likely somewhat likely very likely > mean(suhy$public) [1]

22 Multiple Predictors 2. Let s calculate the probabilities: (cont.) >(x1<-cbind(0:4, 0,.1425)) [,1] [,2] [,3] [1,] [2,] [3,] [4,] [5,] > (x2<-cbind(0:4, 1,.1425)) [,1] [,2] [,3] [1,] [2,] [3,] [4,] [5,]

23 Multiple Predictors Ordered Multinomial Logistic For pared = no (x1) > prob<-function(var){exp(var)/(1+exp(var))} > (p1<-prob(intercept[1]-x1 %*% coef)) [,1] [1,] [2,] [3,] [4,] [5,] > (p2<-prob(intercept[2]-x1 %*% coef)-p1) [,1] [1,] [2,] [3,] [4,] [5,] >p3<-1-(p1+p2) >p3 [,1] [1,] [2,] [3,] [4,] [5,] For pared = yes (x2) > prob<-function(var){exp(var)/(1+exp(var))} > (p4<-prob(intercept[1]-x2 %*% coef)) [,1] [1,] [2,] [3,] [4,] [5,] > (p5<-prob(intercept[2]-x2 %*% coef)-p4) [,1] [1,] [2,] [3,] [4,] [5,] > p6<-1-(p4+p5) > p6 [,1] [1,] [2,] [3,] [4,] [5,]

24 Graphing the Results >library(lattice) >Undergrad.GPA <-0:4 >plot(undergrad.gpa, p1, >type="l", col=1, ylim=c(0,1)) >lines(0:4, p2, col=2) >lines(0:4, p3, col=3) >lines(0:4, p4, col=1, lty = 2) >lines(0:4, p5, col=2, lty = 2) >lines(0:4, p6, col=3, lty = 2) >legend(1.5, 1, >legend=c("p(unlikely)", >"P(somewhat likely)", >"P(very likely)", "Line Type >when Pared = 0", >"Line Type when Pared = 1"), >col=c(1:3,1,1), >lty=c(1,1,1,1,2)) p P(unlikely) P(somewhat likely) P(very likely) Line Type when Pared = 0 Line Type when Pared = Undergrad.GPA

25 Why Not Linear The decision is not always black and white Large categories that are equally spaced could call for a simple linear model However, you must ALWAYS check your assumptions (Gelman &Hill, 2007)

26 Why Not Linear Here is why If we run our model using a simple linear model: >apply2<-as.numeric(suhy $apply) >m3<-lm(apply2~gpa, suhy) >summary(m3) Call: lm(formula = apply2 ~ gpa, data = suhy) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** gpa ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 398 degrees of freedom Multiple R-squared: , Adjusted R- squared: F-statistic: on 1 and 398 DF, p-value: This is what we see when we check our assumptions

27 Why Not Linear Residuals Residuals vs Fitted Standardized residuals Scale-Location Fitted values Fitted values Normal Q-Q Residuals vs Leverage Standardized residuals Standardized residuals Cook's distance Theoretical Quantiles Leverage You Tell me..

28 References Gelman, A. & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. NewYork: Cambridge University Press. Hoelze, B. (2009). analysis with the ordinal multinomial logistic model [PowerPoint slides]. Retrieved from ordered_multinomial.pptx Johnson, V. E. & Albert, J. H. (1999). Statistics for the social sciences and public policy: Ordinal data modeling. New York: Springer. Shivraj, P. (2011). Ordered multinomial logistic regression analysis [PowerPoint slides]. Retrieved from Ordered_ML_Shivraj.pdf. UCLA: Academic Technology Services. (n.d.). Retrieved from