Technical Appendix. The Behavior of Growth Mixture Models Under Nonnormality: A Monte Carlo Analysis

Transcription

1 Monte Crlo Technicl Appendix 1 Technicl Appendix The Behvior of Growth Mixture Models Under Nonnormlity: A Monte Crlo Anlysis Dniel J. Buer & Ptrick J. Currn 10/11/2002 These results re presented s compnion to the mnuscript: Buer, D. J. & Currn, P. J. (in press). Distributionl Assumptions of Growth Mixture Models: Implictions for Over-Extrction of Ltent Trjectory Clsses. Forthcoming in Psychologicl Methods. Further detil on the derivtion of the hypotheses, design of the Monte Crlo, nd ll references my be found in this mnuscript.

2 Monte Crlo Technicl Appendix 2 Tble of Contents Hypotheses nd Method... 3 Figure 1. Popultion Model... 4 Tble 1. Convergence of 1 Clss Unconditionl Model... 5 Tble 2. Convergence of 2 Clss Unconditionl Model with Clss-Invrint Vrince & Covrince Prmeters... 5 Tble 3. Convergence of 2 Clss Unconditionl Model with Clss-Vrying Vrince & Covrince Prmeters... 5 Tble 4. Likelihood Rtio Test of Invrince Constrints on Vrince nd Covrince Prmeters in 2 Clss Unconditionl Model (Proper Solutions Only)... 5 Tble 5. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Invrint Vrince & Covrince Prmetes: Proper Solutions Only (of 500 smples) t N= Tble 6. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Invrint Vrince & Covrince Prmetes: Proper Solutions Only (of 500 smples) t N= Tble 7. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Vrying Vrince & Covrince Prmetes: Proper solutions Only (of 500 smples) t N= Tble 8. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Vrying Vrince & Covrince Prmetes: Proper solutions Only (of 500 smples) t N= Tble 9. Expected Vlue for Prmeter Estimtes Compred With the Men Vlue of the Model Prmeter Estimtes (Empiricl SE, Men Estimted SE): Proper Solutions Only (of 500 Smples) t N= Tble 10. Expected Vlue for Prmeter Estimtes Compred With the Men Vlue of the Model Prmeter Estimtes (Empiricl SE, Men Estimted SE): Proper Solutions Only (of 500 Smples) t N= Tble 11. Popultion Vlues of Model Prmeters Relting Predictor to the Intercept nd Slope Fctors Compred With the Men Vlue of the Prmeter Estimtes (Empiricl SE, Men Estimted SE) Obtined From 1- nd 2-Clss Models: Proper Solutions Only (of 500 Smples) t N= Tble 12. Popultion Vlues of Model Prmeters Relting Predictor to the Intercept nd Slope Fctors Compred With the Men Vlue of the Prmeter Estimtes (Empiricl SE, Men Estimted SE) Obtined From 1- nd 2-Clss Models: Proper Solutions Only (of 500 Smples) t N= Tble 13. Evlution of the effect of the covrite when treted s within-clss predictor of individul vribility in intercepts nd slopes Tble 14. Evlution of the effect of the covrite when treted s clss predictor in two clss model... 13

3 Hypotheses nd Method Monte Crlo Technicl Appendix 3 Hypotheses: (1) Obtining convergence nd proper solution for two clss growth mixture model would be more difficult if dt were drwn from single group multivrite norml distribution thn if they were drwn from single group multivrite nonnorml distribution. (2) Conventionl model fit sttistics would support the estimtion of two (or more) trjectory clsses if the dt were drwn from single group multivrite nonnorml distribution but not if they were drwn from single group multivrite norml distribution. (3) Estimting ltent clsses tht do not correspond to true groups in the popultion could obscure the role of significnt predictors of individul chnge, or identify spurious effects. Method: Dt were generted to be consistent with the single group model in Figure 1. Five hundred smples t ech of two smple sizes, N=200 nd N=600, were generted for three distributionl conditions. In the first condition, the dt were generted to be normlly distributed (i.e., with univrite skew 0 nd kurtosis 0). The other two conditions involved trnsformtions of the repeted mesures dt using Fleishmn's (1978) method for generting nonnorml rndom vribles, s extended by Vle nd Murelli (1983). Specificlly, in these conditions the repeted mesures dt were trnsformed to hve univrite skew 1 nd kurtosis 1, nd skew 1.5 nd kurtosis 6, respectively. (In conditionl models the covrite ws generted from norml distribution in ll conditions). All models were estimted in Mplus 2.01, employing the EM estimtor with the MLR option to obtin robust stndrd errors (Muthén & Muthén, 1998). A modified version of the RUNALL utility ws used to compile the results (Nguyen, Muthén & Muthén, 2001). Finite norml mixture models re known to hve poorly behved likelihood functions (McLchln & Peel, 2001). For this reson, two clss models were estimted both with nd without cross-clss equlity constrints on the vrince components (e.g., Ψ k =Ψ nd Θ k =Θ) though these constrints re often not optiml from the stndpoint of substntive theory. Second, to void obtining locl solutions, ll two clss models were estimted with six sets of strt vlues. One set of strt vlues ws derived from the prmeter estimtes obtined from single group models (per Muthén & Muthén, 1998, p. 132). The single group popultion prmeter estimtes were used s strt vlues for ll of the prmeters except the growth fctor mens, which were set higher in one group thn the other for both growth fctors (µ α =1.50 nd µ β =1.60 for Clss 1 nd µ α =.00 nd µ β =.00 for Clss 2). The other five sets of strt vlues were generted rndomly by tking for ech prmeter rndom drw from norml distribution with men equl to the single-group popultion vlue for the prmeter nd stndrd devition set to provide brod coverge of the surrounding prmeter spce. Our use of rndom strt vlues is consistent with other simultion studies on finite norml mixtures (e.g., Bierncki, Celeux & Govert, 1999; McLchln & Peel, 2000, p. 217). The model ws llowed 1000 itertions to converge. We dopted the following lgorithm for selecting solutions for nlysis: (1) When given repliction filed to converge with ny of the six sets of strt vlues, the solution ws lbeled nonconvergent. (2) When more thn one set of strt vlues led to convergence for given repliction, the solution with the mximum (best) log-likelihood ws selected. This gin follows stndrd prctice in studies of finite norml mixtures (Bierncki, Celeux & Govert, 1999; Everitt & Hnd, 1981; McLchln & Peel, 2000, p. 217). (3) The solution selected from Step 2 ws considered "improper" if ny of the prmeter estimtes fell outside of their permissible boundries (i.e., negtive vrinces, or correltions greter thn one). Unless convergence ws of explicit interest, nonconverged nd improper solutions were excluded from the nlyses since such solutions re rrely interpreted in prctice (Chen et l., 2001). Additionl nlyses including improper solutions did not show meningful differences from the results reported here. (Further detil nd ll references my be obtined from the originl mnuscript)

4 Popultion Model Monte Crlo Technicl Appendix 4 Figure 1. Pth digrm of single group ltent trjectory model. Displyed numbers re the popultion vlues of the prmeters used in the simultion study y 0 y 1 y 2 y 3 y α µ α =1.0 ψ α = β µ β =.80 ψ β =.20 ψ αβ =.11

5 Monte Crlo Technicl Appendix 5 Tble 1. Convergence of 1 Clss Unconditionl Model Distribution Converged N Skew Kurtosis Filed to Converge Improper Solution Proper Solution (1%) 495 (99%) (1%) 496 (99%) (2%) 492 (98%) (100%) (100%) (100%) Tble 2. Convergence of 2 Clss Unconditionl Model with Clss-Invrint Vrince & Covrince Prmeters Distribution Converged N Skew Kurtosis Filed to Converge Improper Solution Proper Solution (1%) 193 (39%) 301 (60%) (46%) 268 (54%) (30%) 350 (70%) (5%) 108 (22%) 369 (74%) (42%) 289 (58%) (15%) 426 (85%) Tble 3. Convergence of 2 Clss Unconditionl Model with Clss-Vrying Vrince & Covrince Prmeters Distribution Converged N Skew Kurtosis Filed to Converge Improper Solution Proper Solution (1%) 450 (90%) 46 (9%) (34%) 330 (66%) (32%) 338 (68%) (6%) 380 (76%) 89 (18%) (6%) 471 (94%) (3%) 485 (97%) Tble 4. Likelihood Rtio Test of Invrince Constrints on Vrince nd Covrince Prmeters in 2 Clss Unconditionl Model (Proper Solutions Only) Distribution Likelihood Rtio Test N Skew Kurtosis Pirs vilble Men χ 2 p <.05 p > (38.89%) 22 (61.11%) (100%) (100%) (38.16%) 51 (66.23%) (100%) (100%) 0 Likelihood Rtio χ 2 test hs 8 df.

6 Monte Crlo Technicl Appendix 6 Tble 5. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Invrint Vrince & Covrince Prmetes: Proper Solutions Only (of 500 smples) t N=200. Fit Sttistic % of time fvors 2-clss model Men Difference Men % Chnge in Fit Stt Skew 0, Kurtosis 0 (301 of 500 Smples) AIC 25.58% % CAIC.33% % BIC.66% % Smple Size Adjusted BIC 21.59% % CLC 5.98% % NEC 5.98% % ICL-BIC 0% % Skew 1, Kurtosis 1 (265 of 500 Smples) AIC 70.57% % CAIC 62.26% % BIC 64.15% % Smple Size Adjusted BIC 69.43% % CLC 36.98% % NEC 62.64% % ICL-BIC 24.91% % Skew 1.5, Kurtosis 6 (343 of 500 Smples) AIC 70.85% % CAIC 65.60% % BIC 67.06% % Smple Size Adjusted BIC 70.55% % CLC 67.06% % NEC 95.04% % ICL-BIC 60.06% % Men difference clculted s Fit1-Fit2 where Fit1 nd Fit2 re the vlues of the sttistic for the 1- nd 2-clss models. Percent chnge clculted s (1-Fit2/Fit1)*100. Positive vlues indicte tht the fit sttistic decresed (e.g., improved) by moving to the 2-clss model. Negtive vlues indicte worse fit of the 2-clss model reltive to the 1-clss model.

7 Monte Crlo Technicl Appendix 7 Tble 6. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Invrint Vrince & Covrince Prmetes: Proper Solutions Only (of 500 smples) t N=600. Fit Sttistic % of time fvors 2-clss model Men Difference Men % Chnge in Fit Stt Skew 0, Kurtosis 0 (369 of 500 Smples) AIC 25.93% % CAIC 0% % BIC 0% % Smple Size Adjusted BIC 4.88% % CLC 1.90% % NEC 1.90% % ICL-BIC 0% % Skew 1, Kurtosis 1 (289 of 500 Smples) AIC 82.01% % CAIC 75.78% % BIC 77.16% % Smple Size Adjusted BIC 80.97% % CLC 16.96% % NEC 34.26% % ICL-BIC 10.73% % Skew 1.5, Kurtosis 6 (426 of 500 Smples) AIC 75.59% % CAIC 73.00% % BIC 73.94% % Smple Size Adjusted BIC 75.12% % CLC 68.54% % NEC 92.02% % ICL-BIC 63.85% % Men difference clculted s Fit1-Fit2 where Fit1 nd Fit2 re the vlues of the sttistic for the 1- nd 2-clss models. Percent chnge clculted s (1-Fit2/Fit1)*100. Positive vlues indicte tht the fit sttistic decresed (e.g., improved) by moving to the 2-clss model. Negtive vlues indicte worse fit of the 2-clss model reltive to the 1-clss model.

8 Monte Crlo Technicl Appendix 8 Tble 7. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Vrying Vrince & Covrince Prmetes: Proper solutions Only (of 500 smples) t N=200. Fit Sttistic % of time fvors 2-clss model Men Difference Men % Chnge in Fit Stt Skew 0, Kurtosis 0 (46 of 500 Smples) AIC 32.61% % CAIC 0% % BIC 0% % Smple Size Adjusted BIC 26.09% % CLC 0% % NEC 0% % ICL-BIC 0% % Skew 1, Kurtosis 1 (329 of 500 Smples) AIC 100% % CAIC 99.70% % BIC 99.70% % Smple Size Adjusted BIC 100% % CLC 98.48% % NEC 98.48% % ICL-BIC 69.60% % Skew 1.5, Kurtosis 6 (334 of 500 Smples) AIC 100% % CAIC 100% % BIC 100% % Smple Size Adjusted BIC 100% % CLC 99.10% % NEC 99.10% % ICL-BIC 92.51% % Men difference clculted s Fit1-Fit2 where Fit1 nd Fit2 re the vlues of the sttistic for the 1- nd 2-clss models. Percent chnge clculted s (1-Fit2/Fit1)*100. Positive vlues indicte tht the fit sttistic decresed (e.g., improved) by moving to the 2-clss model. Negtive vlues indicte worse fit of the 2-clss model reltive to the 1-clss model.

9 Monte Crlo Technicl Appendix 9 Tble 8. Reltive Fit of 1-Clss v. 2-Clss Unconditionl Model With Clss-Vrying Vrince & Covrince Prmetes: Proper solutions Only (of 500 smples) t N=600. Fit Sttistic % of time fvors 2-clss model Men Difference Men % Chnge in Fit Stt Skew 0, Kurtosis 0 (89 of 500 Smples) AIC 21.35% % CAIC 0% % BIC 0% % Smple Size Adjusted BIC 0% % CLC 0% % NEC 0% % ICL-BIC 0% % Skew 1, Kurtosis 1 (471 of 500 Smples) AIC 100% % CAIC 100% % BIC 100% % Smple Size Adjusted BIC 100% % CLC 98.73% % NEC 98.73% % ICL-BIC 91.08% % Skew 1.5, Kurtosis 6 (485 of 500 Smples) AIC 100% % CAIC 100% % BIC 100% % Smple Size Adjusted BIC 100% % CLC 100% % NEC 100% % ICL-BIC 99.18% % Men difference clculted s Fit1-Fit2 where Fit1 nd Fit2 re the vlues of the sttistic for the 1- nd 2-clss models. Percent chnge clculted s (1-Fit2/Fit1)*100. Positive vlues indicte tht the fit sttistic decresed (e.g., improved) by moving to the 2-clss model. Negtive vlues indicte worse fit of the 2-clss model reltive to the 1-clss model.

10 Monte Crlo Technicl Appendix 10 Tble 9. Expected Vlue for Prmeter Estimtes Compred With the Men Vlue of the Model Prmeter Estimtes (Empiricl SE, Men Estimted SE): Proper Solutions Only (of 500 Smples) t N= Clss Model Prmeter Popultion 1 Clss Model Clss 1 Clss 2 Skew 0, Kurtosis 0 (495 Smples) (46 Smples) µ α (.09,.09) 1.14 (.40, 51).70 (.38, 34) µ β (.05,.05).97 (.21, 21).62 (.22, 19) ψ α (.20,.20).94 (.57,.61).81 (.39,.49) ψ β (.05,.05).18 (.13,.13).15 (.09,.13) ψ αβ (.08,.07) -.07 (.20,.23) -.09 (.48,.16) CORR αβ % Cses 100% 100% 48.3% 51.7% Skew 1, Kurtosis 1 (496 Smples) (330 Smples) µ α (.09,.09) 1.48 (.25,.19).22 (.23,.19) µ β (.05,.05).98 (.16,.10).53 (.15,.10) ψ α (.22,.22).79 (.35,.36).24 (.15,.16) ψ β (.05,.05).20 (.09,.09).05 (.03,.04) ψ αβ (.08,.08) -.06 (.13,.14) -.03 (.04,.06) CORR αβ % Cses 100% 100% 58.8% 41.2% Skew 1.5, Kurtosis 6 (492 Smples) (338 Smples) µ α (.09,.09) 1.85 (.47,.38).63 (.10,.12) µ β (.05,.05) 1.07 (.24,.19).75 (.20,.06) ψ α (.28,.26) 1.22 (.81,.83).39 (.14,.17) ψ β (.06,.06).35 (.21,.24).09 (.03,.04) ψ αβ (.09,.09) -.16 (.31,.35).01 (.05,.06) CORR αβ % Cses 100% 100% 26.7% 73.3% Estimted With Clss-Vrying Vrince nd Covrince Prmeters.

11 Monte Crlo Technicl Appendix 11 Tble 10. Expected Vlue for Prmeter Estimtes Compred With the Men Vlue of the Model Prmeter Estimtes (Empiricl SE, Men Estimted SE): Proper Solutions Only (of 500 Smples) t N= Clss Model Prmeter Popultion 1 Clss Model Clss 1 Clss 2 Skew 0, Kurtosis 0 (500 Smples) (89 Smples) µ α (.05,.05) 1.14 (.41,.42).74 (.42,.42) µ β (.02,.03).92 (.23, 28).68 (.21,.19) ψ α (.11,.12).86 (.46,.51).85 (.38,.52) ψ β (.03,.03).19 (.09,.12).18 (.10,.12) ψ αβ (.04,.04).05 (.18,.19).06 (.15,.16) CORR αβ % Cses 100% 100% 48.6% 51.4% Skew 1, Kurtosis 1 (500 Smples) (471 Smples) µ α (.05,.05) 1.50 (.12,.12).17 (.13,.13) µ β (.03,.03).99 (.06,.06).51 (.06,.07) ψ α (.12,.13).79 (.21,.21).19 (.08,.09) ψ β (.03,.03).21 (.05,.06).05 (.02,.02) ψ αβ (.05,.05) -.06 (.08,.08) -.01 (.03,.03) CORR αβ % Cses 100% 100% 60.7% 39.3% Skew 1.5, Kurtosis 6 (500 Smples) (485 Smples) µ α (.05,.05) 1.99 (.25,.22).65 (.06,.06) µ β (.03,.03) 1.16 (.12,.11).69 (.09,.03) ψ α (.16,.15) 1.19 (.48,.50).40 (.08,.08) ψ β (.04,.04).36 (.13,.13).09 (.02,.02) ψ αβ (.05,.05) -.15 (.21,.20).02 (.03,.03) CORR αβ % Cses 100% 100% 25.3% 74.7% Estimted With Clss-Vrying Vrince nd Covrince Prmeters.

12 Monte Crlo Technicl Appendix 12 Tble 11. Popultion Vlues of Model Prmeters Relting Predictor to the Intercept nd Slope Fctors Compred With the Men Vlue of the Prmeter Estimtes (Empiricl SE, Men Estimted SE) Obtined From 1- nd 2-Clss Models: Proper Solutions Only (of 500 Smples) t N= Clss Model With Equlity 2 Clss Model Without Constrints b Prmeter Popultion 1 Clss Model Constrints Clss 1 Clss 2 Skew 1, Kurtosis 1 (473 smples) (281 Smples) (265 Smples) γ (.029,.028).097 (.027,.027).140 (.043,.045).074 (.034,.046) γ (.015,.015) (.012,.014) (.023,.025) (.018,.020) % Cses 100% 100% 58.6% / 41.4% 59.6% 40.4% Skew 1.5, Kurtosis 6 (465 Smples) (291 Smples) (267 Smples) γ (.029,.029).096 (.024,.024).181 (.086,.086).085 (.027,.028) γ (.015,.014) (.013,.012) (.049,.048) (.014,.014) % Cses 100% 100% 26.7% / 73.3% 27.4% 72.6% Prmeters γ 1 nd γ 2 constrined to be equl cross clsses, vrince nd covrince prmeters permitted to vry over clsses. All prmeters permitted to vry over clsses. Tble 12. Popultion Vlues of Model Prmeters Relting Predictor to the Intercept nd Slope Fctors Compred With the Men Vlue of the Prmeter Estimtes (Empiricl SE, Men Estimted SE) Obtined From 1- nd 2-Clss Models: Proper Solutions Only (of 500 Smples) t N= Clss Model With Equlity 2 Clss Model Without Constrints b Prmeter Popultion 1 Clss Model Constrints Clss 1 Clss 2 Skew 1, Kurtosis 1 (499 smples) (453 Smples) (437 Smples) γ (.017,.016).096 (.015,.015).139 (.024,.025).070 (.019,.021) γ (.009,.008) (.007,.007) (.014,.013) (.010,.011) % Cses 100% 100% 59.5% / 40.5% 60.0% 40.0% Skew 1.5, Kurtosis 6 (499 Smples) (473 Smples) (458 Smples) γ (.017,.017).098 (.015,.014).189 (.049,.049).087 (.016,.016) γ (.009,.008) (.007,.007) (.029,.028) (.008,.008) % Cses 100% 100% 25.1% / 74.9% 25.5% 74.5% Prmeters γ 1 nd γ 2 constrined to be equl cross clsses, vrince nd covrince prmeters permitted to vry over clsses. All prmeters permitted to vry over clsses.

13 Monte Crlo Technicl Appendix 13 Tble 13. Evlution of the effect of the covrite when treted s within-clss predictor of individul vribility in intercepts nd slopes: Tble gives the percent of replictions converging on proper solution where the effect of the covrite on individul intercepts (γ 1 ) nd slopes (γ 2 ) ws significnt t p <.05 (nd in the sme direction s the effect in the popultion). N=200 N=600 γ 1 γ 2 γ 1 γ 2 One-Clss Model Skew 1, Kurtosis 1 99% 55% 100% 94% Skew 1.5, Kurtosis 6 99% 54% 100% 93% Two-Clss Model Skew 1, Kurtosis 1 Clss 1 88% 49% 100% 90% Clss 2 57% 27% 93% 55% Skew 1.5, Kurtosis 6 Clss 1 59% 23% 97% 61% Clss 2 87% 42% 100% 89% Men Tble 14. Evlution of the effect of the covrite when treted s clss predictor in two clss model. Distribution Proper Solutions % of Replictions N Skew Kurtosis (of 500 Smples) Men Logit Odds-Rtio Effect ws NS (67%).11 (.083,.080) % (66%).11 (.077,.081) % (95%).11 (.044,.044) % (98%).10 (.044,.044) % Numbers in prentheses correspond to the empiricl stndrd error nd verge estimted stndrd error of the logit.