Outline. Assessing the precision of estimates of variance components. Modern practice. Describing the precision of parameters estimates

Size: px

Start display at page:

Download "Outline. Assessing the precision of estimates of variance components. Modern practice. Describing the precision of parameters estimates"

Ruth French
6 years ago
Views:

1 Outlie Assessig the precisio of estimates of variace compoets Douglas Bates Departmet of Statistics Uiversity of Wiscosi Madiso U.S.A. LMU, Muich July 16, 9 Estimates ad stadard errors Summarizig mixed-effects model fits A simple (but real) example A brief overview of the theory ad computatio for mixed models Profiled deviace as a fuctio of θ Summary Describig the precisio of parameters estimates I may ways the purpose of statistical aalysis ca be cosidered as quatifyig the variability i data ad determiig how the variability affects the ifereces that we draw from it. Good statistical practice suggests, therefore, that we ot oly provide our best guess, the poit estimate of a parameter, but also describe its precisio (e.g. iterval estimatio). Some of the time (but ot early as frequetly as widely believed) we also wat to check whether a particular parameter value is cosistet with the data (i.e.. hypothesis tests ad p-values). I olde days it was ecessary to do some rather coarse approximatios such as summarizig precisio by the stadard error of the estimate or calculatig a test statistic ad comparig it to a tabulated value to derive a /1 respose of sigificat (or ot) at the 5% level. Moder practice Our ability to do statistical computig has chaged from the olde days. Curret hardware ad software would have bee uimagiable whe I bega my career as a statisticia. We ca work with huge data sets havig complex structure ad fit sophisticated models to them quite easily. Regrettably, we still frequetly quote the results of this sophisticated modelig as poit estimates, stadard errors ad p-values. Uderstadably, the cliet (ad the referees readig the cliet s paper) would like to have simple, easily uderstood summaries so they ca assess the aalysis at a glace. However, the desire for simple summaries of complex aalyses is ot, by itself, eough to these summaries meaigful. We must ot oly provide sophisticated software for statisticias ad other researchers; we must also chage their thikig about summaries.

2 Summaries of mixed-effects models Commercial software for fittig mixed-effects models (SAS PROC MIXED, SPSS, MLwi, HLM, Stata) provides estimates of fixed-effects parameters, stadard errors, degrees of freedom ad p-values. They also provide estimates of variace compoets ad stadard errors of these estimates. The mixed-effects packages for R that I have writte (lme with José Piheiro ad lme4 with Marti Mächler) do ot provide stadard errors of variace compoets. lme4 does t eve provide p-values for the fixed effects. This is a source of widespread axiety. May view it as a idicatio of icompetece o the part of the developers ( Why ca t lmer provide the p-values that I ca easily get from SAS? ) The 7 book by West, Welch ad Galecki shows how to use all of these software packages to fit mixed-effects models o 5 differet examples. Every time they provide comparative tables they must add a footote that lme does t provide stadard errors of variace compoets. The Dyestuff data set The Dyestuff, Peicilli ad Pastes data sets all come from the classic book Statistical Methods i Research ad Productio, edited by O.L. Davies ad first published i The Dyestuff data are a balaced oe-way classificatio of the Yield of dyestuff from samples produced from six Batches of a itermediate product. See?Dyestuff. > str(dyestuff) data.frame : 3 obs. of variables: $ Batch: Factor w/ 6 levels "A","B","C","D",..: $ Yield: um > summary(dyestuff) Batch Yield A:5 Mi. :144 B:5 1st Qu.:1469 C:5 Media :153 D:5 Mea :158 E:5 3rd Qu.:1575 F:5 Max. :1635 What does a stadard error tell us? Typically we use a stadard error of a parameter estimate to assess precisio (e.g. a 95% cofidece iterval o µ is roughly x ± s ) or to form a test statistic (e.g. a test of H : µ = versus H a : µ based o the statistic x s/ ). Such itervals or test statistics are meaigful whe the distribuio of the estimator is more-or-less symmetric. We would ot, for example, quote a stadard error of σ because we kow that the distributio of this estimator, eve i the simplest case (the mythical i.i.d. sample from a Gaussia distributio), is ot at all symmetric. We use quatiles of the χ distributio to create a cofidece iterval. Why, the, should we believe that whe we create a much more complex model the distributio of estimators of variace compoets will magically become sufficietly symmetric for stadard errors to be meaigful? Dyestuff data plot Batch E C B A D F Yield of dyestuff (grams of stadard color) The lie jois the mea yields of the six batches, which have bee reordered by icreasig mea yield. The vertical positios are jittered slightly to reduce overplottig. The lowest yield for batch A was observed o two distict preparatios from that batch.

3 A mixed-effects model for the dyestuff yield > fm1 <- lmer(yield ~ 1 + (1 Batch), Dyestuff) > prit(fm1) Liear mixed model fit by REML Formula: Yield ~ 1 + (1 Batch) Data: Dyestuff AIC BIC loglik deviace REMLdev Radom effects: Groups Name Variace Std.Dev. Batch (Itercept) Residual Number of obs: 3, groups: Batch, 6 Fixed effects: Estimate Std. Error t value (Itercept) Fitted model fm1 has oe fixed-effect parameter, the mea yield, ad oe radom-effects term, geeratig a simple, scalar radom effect for each level of Batch. Re-fittig the model for ML estimates > (fm1m <- update(fm1, REML = FALSE)) Liear mixed model fit by maximum likelihood Formula: Yield ~ 1 + (1 Batch) Data: Dyestuff AIC BIC loglik deviace REMLdev Radom effects: Groups Name Variace Std.Dev. Batch (Itercept) Residual Number of obs: 3, groups: Batch, 6 Fixed effects: Estimate Std. Error t value (Itercept) (The extra paretheses aroud the assigmet cause the value to be prited. Geerally the results of assigmets are ot prited.) REML estimates versus ML estimates The default parameter estimatio criterio for liear mixed models is restricted (or residual ) maximum likelihood (REML). Maximum likelihood (ML) estimates (sometimes called full maximum likelihood ) ca be requested by specifyig REML = FALSE i the call to lmer. Geerally REML estimates of variace compoets are preferred. ML estimates are kow to be biased. Although REML estimates are ot guarateed to be ubiased, they are usually less biased tha ML estimates. Roughly, the differece betwee REML ad ML estimates of variace compoets is comparable to estimatig σ i a fixed-effects regressio by SSR/( p) versus SSR/, where SSR is the residual sum of squares. For a balaced, oe-way classificatio like the Dyestuff data, the REML ad ML estimates of the fixed-effects are idetical. Evaluatig the deviace fuctio The profiled deviace fuctio for such a model ca be expressed as a fuctio of 1 parameter oly, the ratio of the radom effects stadard deviatio to the residual stadard deviatio. A very brief explaatio is based o the -dimesioal respose radom variatio, Y, whose value, y, is observed, ad the q-dimesioal, uobserved radom effects variable, B, with distributios (Y B = b) N ( Zb + Xβ, σ I ), B N (, Σθ ), For our example, = 3, q = 6, X is a 3 1 matrix of 1s, Z is the 3 6 matrix of idicators of the levels of Batch ad Σ is σ b I 6. We ever really form Σ θ ; we always work with the relative covariace factor, Λ θ, defied so that Σ θ = σ Λ θ Λ θ. I our example θ = σ b σ ad Λ θ = θi 6.

4 Orthogoal or uit radom effects We will defie a q-dimesioal spherical or uit radom-effects vector, U, such that U N (, σ I q ), B = Λθ U Var(B) = σ Λ θ Λ θ = Σ θ. The liear predictor expressio becomes where U θ = ZΛ θ. Zb + Xβ = ZΛ θ u + Xβ = U θ u + Xβ The key to evaluatig the log-likelihood is the Cholesky factorizatio L θ L θ = P ( U θ U θ + I q ) P (P is a fixed permutatio that has practical importace but ca be igored i theoretical derivatios). The sparse, lower-triagular L θ ca be evaluated ad updated for ew θ eve whe q is i the millios ad the model ivolves radom effects for several factors. Profilig the deviace with respect to β Because the deviace depeds o β oly through r (θ, β) we ca obtai the coditioal estimate, β θ, by extedig the PLS problem to r (θ) = mi u,β [ y Xβ U θ u + u ] with the solutio satisfyig the equatios [ U θ U θ + I q U θ X ] ] [ [ũθ U X U θ X = θ y ] X β θ X y. The profiled deviace, which is a fuctio of θ oly, is [ ( πr l(θ) = log( L θ )] (θ) ) log The profiled deviace The Cholesky factor, L θ, allows evaluatio of the coditioal mode ũ θ,β (also the coditioal mea for liear mixed models) from ( U θ U ) θ + I q ũθ,β = P L θ L θ P ũ θ,β = U θ (y Xβ) Let r (θ, β) = y Xβ U θ ũ θ,β + ũ θ,β. l(θ, β, σ y) = log L(θ, β, σ y) ca be writte l(θ, β, σ y) = log(πσ ) + r (θ, β) σ + log( L θ ) The coditioal estimate of σ is σ (θ, β) = r (θ, β) producig the profiled deviace l(θ, β y) = log( L θ ) + [ 1 + log Profiled deviace ad its compoets ( πr )] (θ, β) For this simple model we ca evaluate ad plot the deviace for a rage of θ values. We [ also plot its compoets, )] log( L θ ) (ldl) ad 1 + log (lprss). ( πr (θ) lprss measures fidelity to the data. It is bouded above ad below. log( L θ ) measures complexity of the model. It is bouded below but ot above deviace ldl θ lprss

5 The MLE (or REML estimate) of σ b ca be Compoets of the profiled deviace for Dyestuff For some model/data set combiatios the estimate of σ b is zero. This occurs whe the decrease i lprss as θ is ot sufficiet to couteract the icrease i the complexity, log( L θ ). The Dyestuff data from Box ad Tiao (1973) show this deviace ldl lprss θ Batch C A E D B F 5 1 Simulated respose (dimesioless) For this data set the differece i the upper ad lower bouds o lprss is ot sufficiet to couteract the icrease i complexity of the model, as measured by log( L θ ). Software should gracefully hadle cases of σb = or, more geerally, Λ θ beig sigular. This is ot doe well i the commercial software. Oe of the big differeces betwee ifereces for σ b ad those for σ is the eed to accomodate to do about values of σ b that are zero or ear zero. Profiled deviace ad REML criterio for σ b ad σ Profilig with respect to each parameter separately deviace 99.9% REML 345 σ 8 34 σ 6 95% 9% 99.9% 5% Deviace % 8% 99% 8% 9% 95% 99% 99.9% The cotours correspod to -dimesioal margial cofidece regios derived from a likelihood-ratio test. The dotted ad dashed lies are the profile traces. These curves show the miimal deviace achieveable for a value of oe of the parameters, optimizig over all the other parameters.

6 Profiled deviace of the variace compoets Recall that we have bee workig o the scale of the stadard deviatios, σ b ad σ. O the scale of the variace, thigs look worse σ Square root of chage i the profiled deviace The differece of the profiled deviace at the optimum ad at a particular value of σ or σ b is the likelihood ratio test statistic for that parameter value. If the use of a stadard error, ad the implied symmetric itervals, is appropriate the this fuctio should be quadratic i the parameter ad its square root should be like a absolute value fuctio. Deviace 335 The assumptio that the chage i the deviace has a χ 1 distributio is equivalet to sayig that LRT is the absolute value of a stadard ormal If we use the siged square root trasformatio, assigig LRT to parameters to the left of the estimate ad LRT to parameter values to the right, we should get a straight lie o a stadard ormal scale. Plot of square root of LRT statistic Siged square root plot of LRT statistic σ σ Profile z Profile z

7 Summary Summaries based o parameter estimates ad stadard errors are appropriate whe the distributio of the estimator ca be assumed to be reasoably symmetric. Estimators of variaces do ot ted to have a symmetric distributio. If aythig the scale of the log-variace (which is a multiple of the log-stadard deviatio) would be the more appropriate scale o which to assume symmetry. Estimators of variace compoets are more problematic because they ca take o the value of zero. Profilig the deviace ad plottig the result ca help to visualize the precisio of the estimates.

Parametric Density Estimation: Maximum Likelihood Estimation

Parametric Density Estimation: Maximum Likelihood Estimation Parametric Desity stimatio: Maimum Likelihood stimatio C6 Today Itroductio to desity estimatio Maimum Likelihood stimatio Itroducto Bayesia Decisio Theory i previous lectures tells us how to desig a optimal