Random Effects ANOVA

Transcription

1 Random Effects ANOVA Grant B. Morgan Baylor University This post contains code for conducting a random effects ANOVA. Make sure the following packages are installed: foreign, lme4, lsr, lattice. library(foreign) library(lsr) library(lme4) Loading required package: Matrix raneff<-read.spss("/users/grant_morgan/box Sync/Teaching/EDP Experimental Design/Data/Ra Warning in read.spss("/users/grant morgan/box Sync/Teaching/EDP Experimental Design/Data/Random Effect.sav", : /Users/grant morgan/box Sync/Teaching/EDP Experimental Design/Data/Random Effect.sav: Unrecognized record type 7, subtype 18 encountered in system file re-encoding from CP1252 First, we should generate some descriptive statistics for the outcome variable, Score, for each of the randomly selected raters. library(psych) describeby(raneff$score, raneff$rater) Descriptive statistics by group group: 1 vars n mean sd median trimmed mad min max range skew kurtosis se X group: 2 vars n mean sd median trimmed mad min max range skew kurtosis se X group: 3 vars n mean sd median trimmed mad min max range skew kurtosis se X group: 4 1

2 vars n mean sd median trimmed mad min max range skew kurtosis se X The null hypothesis in the random effects ANOVA states that the variance of the means is equal to zero. Let s generate some plots to see how the distribution of scores for each of the randomly selected raters overlaps. library(lattice) xyplot(raneff$score ~ raneff$rater, pch = 16, col = "blue") boxplot(raneff$score ~ raneff$rater, pch = 16, col = "blue") 2

3 Both of these plots indicates that the variability seems pretty similar for each rater, but the scores appear to be centered at different locations for each rater. We also see this from the descriptive statistics. Next, let s estimate the random effects ANOVA model. raneff$rater<-as.factor(raneff$rater) raneff.out<-aov(raneff$score ~ Error(raneff$Rater), data=raneff) summary(raneff.out) Error: raneff$rater Df Sum Sq Mean Sq F value Pr(>F) 3

4 Residuals Error: Within Df Sum Sq Mean Sq F value Pr(>F) Residuals raneff.out Call: aov(formula = raneff$score ~ Error(raneff$Rater), data = raneff) Grand Mean: Stratum 1: raneff$rater Terms: Residuals Sum of Squares Deg. of Freedom 3 Residual standard error: Stratum 2: Within Terms: Residuals Sum of Squares Deg. of Freedom 24 Residual standard error: sum((raneff.out$"within"$residuals)**2) [1] sum((raneff.out$"raneff$rater"$residuals)**2) [1] raneff.out$"within"$df.residual [1] 24 raneff.f<-(sum((raneff.out$"raneff$rater"$residuals)**2)/raneff.out$"raneff$rater"$df.residual) (sum((raneff.out$"within"$residuals)**2)/raneff.out$"within"$df.residual) raneff.f [1]

5 raneff.p<-pf(raneff.f,raneff.out$"raneff$rater"$df.residual,raneff.out$"within"$df.residual,low raneff.p [1] raneff.results<-matrix(c((sum((raneff.out$"raneff$rater"$residuals)**2)/raneff.out$"raneff$rate pf(raneff.f,raneff.out$"raneff$rater"$df.residual,raneff.out$"within"$df.residual,lower.tail= colnames(raneff.results)<-c("f","p-value") raneff.results F p-value [1,] For some reason, the aov function in R does provide not the p-value for random effects ANOVA so I have computed it. According to the results, there is sufficient evidence to conclude that the variance of the means of the raters is nonzero. There is no post hoc analysis because we are not dealing with specific (i.e., fixed) values of the raters. If we replicated the sudy, the raters would likely be different. We can get a better understanding of the random effect by computing the variance components and effect size. We can compute the intraclass correlation coefficient. This estimates the proportion of variance in the outcome variable that is explained by the random effect in the population. raneff.vc<-lmer(raneff$score ~ 1 + (1 raneff$rater), data = raneff) summary(raneff.vc) Linear mixed model fit by REML ['lmermod'] Formula: raneff$score ~ 1 + (1 raneff$rater) Data: raneff REML criterion at convergence: Scaled residuals: Min 1Q Median 3Q Max Random effects: Groups Name Variance Std.Dev. raneff$rater (Intercept) Residual Number of obs: 28, groups: raneff$rater, 4 Fixed effects: Estimate Std. Error t value (Intercept) raneff.varexp<-as.numeric(varcorr(raneff.vc)) / (as.numeric(varcorr(raneff.vc))+attr(varcorr(ra raneff.varexp [1]

6 The variance components are 2.85 for the rater effect and 7.81 for the residual. The ICC is computed by dividing the rater effect by the sum of the two variance components: ICC = σ 2 Rater σ 2 Rater + σ2 Residual = The ICC for this study is.27, which indicates that the raters explained 27% of the variability in the scores in the population. 6