ORDERED MULTINOMIAL LOGISTIC REGRESSION ANALYSIS. Pooja Shivraj Southern Methodist University

Transcription

1 ORDERED MULTINOMIAL LOGISTIC REGRESSION ANALYSIS Pooja Shivraj Southern Methodist University

2 KINDS OF REGRESSION ANALYSES Linear Regression Logistic Regression Dichotomous dependent variable (yes/no, died/ didn t die, at risk/not at risk, etc.) Predicts the probability of a person belonging in that category.

3 QUICK REVIEW: LOGISTIC REGRESSION Values calculated from linear regression are continuous need to be transformed on a 0-1 scale to represent probability since 0 p 1 Logistic regression probability calculated by: ^ e (B 1 x + B 0 ) p = 1 + e (B 1 x + B 0 )

4 CLASS EXAMPLE: LOGISTIC REGRESSION Probability of a person complying for a mammogram, based on whether or not they get a physician s recommendation

5 CLASS EXAMPLE: LOGISTIC REGRESSION ^ e (B 1 x + B 0 ) p = 1 + e (B 1 x + B 0 ) Probability of complying if NOT recommended by physician: (2.29(0) ) ^ e p = (2.29(0) ) 1 + e = 0.14 = 0.61 Probability of complying if recommended by physician: (2.29(1) ) ^ e p = (2.29(1) ) 1 + e

6 ORDERED MULTINOMIAL LOGISTIC REGRESSION ANALYSIS Type of logistic regression that allows more than two discrete outcomes Outcomes are ordinal: Yes, maybe, no First, second, third place Gold, silver, bronze medals Strongly agree, agree, neutral, disagree, strongly disagree

7 ASSUMPTION No perfect predictions one predictor variable value cannot solely correspond to one dependent variable value check using crosstabs.

8 ORDERED LOGISTIC REGRESSION Load libraries: library(arm) library(psych) EXAMPLE Load data: pooj<-read.csv(" stat/r/dae/ologit.csv")

9 ORDERED LOGISTIC REGRESSION Variables: EXAMPLE apply college juniors reported likelihood of applying to grad school (0 = unlikely, 1 = somewhat likely, 2 = very likely) pared indicating whether at least one parent has a graduate degree (0 = no, 1 = yes) public indicating whether the undergraduate institution is a public or private (0 = private, 1 = public) gpa college GPA

10 > str(pooj) 'data.frame': 400 obs. of 4 variables: $ apply : int $ pared : int $ public: int $ gpa : num > table(pooj$apply) > table(pooj$pared) > table(pooj$public)

11 CHECK ASSUMPTION CROSS-TABS > xtabs(~pooj$pared+pooj$apply) pooj$apply pooj$pared > xtabs(~pooj$public+pooj$apply) pooj$apply pooj$public Why is this important?

12 SINGLE PREDICTOR MODEL - GPA > library(arm) > summary(m1<-bayespolr(as.ordered(pooj$apply)~pooj$gpa)) Call: bayespolr(formula = as.ordered(pooj$apply) ~ pooj$gpa) Coefficients: Value Std. Error t value pooj$gpa Intercepts: Value Std. Error t value Residual Deviance: AIC:

13 CUMULATIVE DISTRIBUTION FUNCTION

14 LABELING COEFFICIENTS Coefficients: Value Std. Error t value pooj$gpa Intercepts: Value Std. Error t value Coefficient of the model coef<m1$coef Intercepts of the model intercept <- m1$zeta Let us look at the likelihood of students with an average GPA applying to graduate school. > x<-mean(pooj$gpa) [1]

15 TRANSFORMING OUTCOMES TO PROBABILITIES prob<-function(input){exp(input)/ (1+exp(input))} (p0<-prob(intercept[1]-coef*x)) (p1<-prob(intercept[2]-coef*x)-p0) (p2<-1-(p0+p1))

16 WHY NOT USE LINEAR REGRESSION? > summary(linreg<-lm(pooj$apply~pooj$gpa)) Call: lm(formula = pooj$apply ~ pooj$gpa) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) pooj$gpa ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 398 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 398 DF, p-value:

17 AND OUR ASSUMPTIONS AREN T MET

18 LINEAR REGRESSION VERSUS ORDERED LOGISTIC REGRESSION The decision between linear regression and ordered multinomial regression is not always black and white. When you have a large number of categories that can be considered equally spaced simple linear regression is an optional alternative (Gelman & Hill, 2007). Moral of story: Always start by checking the assumptions of the model.

19 USING MULTIPLE PREDICTORS summary(m2 <- bayespolr(as.ordered(apply)~gpa + pared + public,pooj)) Call: bayespolr(formula = as.ordered(apply) ~ gpa + pared + public, pooj) Coefficients: Value Std. Error t value gpa pared public Intercepts: Value Std. Error t value Residual Deviance: AIC:

20 TRANSFORMING OUTCOMES TO PROBABILITIES (coef<- m2$coef) gpa pared public (intercept<-m2$zeta) (x1<-cbind(0:4, 0,.14)) [,1] [,2] [,3] [1,] [2,] [3,] [4,] [5,] (x2<-cbind(0:4, 1,.14)) [,1] [,2] [,3] [1,] [2,] [3,] [4,] [5,]

21 TRANSFORMING OUTCOMES TO PROBABILITIES prob<-function(var){exp(var)/(1+exp(var))} > (p1<-prob(intercept[1]-x1 %*% coef)) [,1] [1,] [2,] [3,] [4,] [5,] > (p2<-prob(intercept[2]-x1 %*% coef)-p1) [,1] [1,] [2,] [3,] [4,] [5,] > (p3<-1-(p1+p2)) [,1] [1,] [2,] [3,] [4,] [5,]

22 TRANSFORMING OUTCOMES TO PROBABILITIES > (p4<-prob(intercept[1]-x2 %*% coef)) [,1] [1,] [2,] [3,] [4,] [5,] > (p5<-prob(intercept[2]-x2 %*% coef)-p1) [,1] [1,] [2,] [3,] [4,] [5,] > (p6<-1-(p4+p5)) [,1] [1,] [2,] [3,] [4,] [5,]

23 PLOTTING THE RESULTS Undergrad.GPA <-0:4 plot(undergrad.gpa, p1, type="l", col=1, ylim=c(0,1)) lines(0:4, p2, col=2) lines(0:4, p3, col=3) lines(0:4, p4, col=1, lty = 2) lines(0:4, p5, col=2, lty = 2) lines(0:4, p6, col=3, lty = 2) legend(1.5, 1, legend=c("p(unlikely)", "P(somewhat likely)", "P(very likely)", "Line Type when Pared = 0", "Line Type when Pared = 1"), col=c(1:3,1,1), lty=c(1,1,1,1,2))

24 PRACTICE Read in the following table (Quinn, n.d.): practice <- read.table(" nes96r.dat", header=true) Task: Run a regression using the ordered multinomial logistic model to predict the variation in the dependent variable ClinLR using the independent variables PID and educ. ClinLR = Ordinal variable from 1-7 indicating ones view of Bill Clinton s political leanings, where 1 = extremely liberal, 2 = liberal, 3 = slightly liberal, 4 = moderate, 5= slightly conservative, 6 = conservative, 6 = extremely conservative. PID = Ordinal variable from 0-6 indicating ones own political identification, where 0 = Strong Democrat and 6 = Strong Republican educ = Ordinal variable from 1-7 indicating ones own level of education, where 1 = 8 grades or less and no diploma, 2 = 9-11 grades, no further schooling, 3 = High school diploma or equivalency test, 4 = More than 12 years of schooling, no higher degree, 5 = Junior or community college level degree (AA degrees), 6 = BA level degrees; 17+ years, no postgraduate degree, 7 = Advanced degree

25 REFERENCES Gelman, A. & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. New York: Cambridge University Press. Quinn, K. (n.d.). Retrieved from data/nes96r.dat UCLA: Academic Technology Services. (n.d.). Retrieved from