Staying On Track with Growth-to-Standards for English Learner Students. Joni Lakin, Ph.D. Auburn University

Size: px

Start display at page:

Download "Staying On Track with Growth-to-Standards for English Learner Students. Joni Lakin, Ph.D. Auburn University"

Edith Pearson
6 years ago
Views:

1 Staying On Track with Growth-to-Standards for English Learner Students Joni Lakin, Ph.D. Auburn University

2 Growth models poised to maintain or increase relevance under new assessment systems Many states have policies of status-plusgrowth accountability models Need to evaluate model performance for key subgroups

3 Same issues as status-based models: Heterogeneity Identification and reclassification Test Validity New issues: Data matching Changes in accommodations Low baseline scores Consistency and validity of models 3 6/5/2014

4 4 Compared behavior of four growth-based accountability models Value Table Model Trajectory Model Projection Model Student Growth Percentile Model Are EL students classified similarly to non-el students by each model? Are the classifications similarly accurate? Does the stringency of the proficiency cut score impact these results? 6/5/2014

5 Grade 6 is set as target (status-only) year Grade 3 (post-test) is first growth-eligible year Current plus one prior year of scores for SGP and Projection Designate students as on-track to future proficiency 5

6 Value Table: Positive transition made between proficiency categories Trajectory: Meet growth targets set by distance from current score to proficient score

Projection: Regression model predicts future scores above proficiency cut off in target year Student Growth Percentiles: Student s relative growth exceeds the rate needed to reach proficient by

7 Projection: Regression model predicts future scores above proficiency cut off in target year Student Growth Percentiles: Student s relative growth exceeds the rate needed to reach proficient by target year Images: Betebenner (2011). Retrieved March 29, 2012, from webprogram/session2199.html

8 Using cohorts, grades pre-3 to 6 English Language Arts (ELA) Tests Ethnicity Non-ELs (207k) ELs (15k) White 59% 3% Hispanic 5% 83% African American 29% 2% Asian 2% 10% Student with disabilities 13% 10% Free or reduced lunch 45% 82% Migrant 0.1% 2% 8 6/5/2014

9 Cut score (z scale) Non EL Proficient EL Proficient Similar to 0.5 (low) 65% 33% >>State reports >>Hoffer et al % 18% 0.5 (stringent) 31% 4% >>NAEP results >>Lakin & Young (CA data) 9 6/5/2014

10 Percent not proficient, but on track (out of all non-proficient) 50% 40% 30% 20% 10% 2016 cohort 46% 40% 31% 32% 23% 27% 25% 23% 20% 20% 17% 20% 14% 15% 15% 12% 16% 12% 14% 13% 15% 10% 10% 9% 0% grade 3 grade 4 grade 5

11 2016 cohort Percent not proficient, but on track (out of all non-proficient) 50% 40% 30% 20% 10% 0% 34% 28% 22% 33% 44% 37% 20% 16% 13% 12% 10% 7% 7% 5% 7% 7% 8% 5% 6% 4% 3% 0% 1% 1% grade 3 grade 4 grade 5

12 2016 cohort Percent not proficient, but on track (out of all non-proficient) 50% 40% 30% 20% 10% 0% 44% 37% 34% 33% 28% 22% 20% 16% 13% 8% 12% 7% 5% 7% 6% 6% 5% 5% 5% 0% 1% 2% 1% 3% grade 3 grade 4 grade 5

13 2016 cohort Grade 3 on-track classification vs. Grade 6 proficiency 100% 80% 60% 40% 20% 0% 82% 86% 86% 89% 86% 76% 79% 71% 72% 65% 27% 15% 14% 13% 10% 13% 8% 14% 14% 11% 11% 5% 8% 7% 7% 7% 5% 7% 8% 6% Status only VT Traj Proj SGP

14 2016 cohort Grade 3 on-track classification vs. Grade 6 proficiency 100% 80% 60% 77% 73% 76% 77% 64% 66% 81% 80% 76% 76% 40% 20% 20% 10% 26% 24% 14% 9% 15% 9% 15% 9% 7% 9% 13% 10% 11% 14% 8% 10% 10% 9% 0% Status only VT Traj Proj SGP

15 2016 cohort Grade 3 on-track classification vs. Grade 6 proficiency 100% 80% 76% 90% 67% 69% 66% 69% 90% 84% 78% 78% 60% 40% 20% 0% 27% 28% 16% 25% 27% 14% 9% 8% 7% 8% 10% 11% 6% 4% 4% 3% 6% 4% 6% 10% Status only VT Traj Proj SGP

16 VT+ TRAJ VT+ PROJ VT+ SGP TRAJ+ PROJ TRAJ +SGP PROJ +SGP On track by both models 22% 0% 6% 0% 7% 0% On track by ONE model 6% 37% 21% 38% 18% 19% Not on track by both 71% 63% 74% 62% 74% 81% On track agreement (row 1/1+2) 78% 0% 22% 0% 28% 0%

17 Average 6th grade gain (in SDs) given 3rd grade classification SDs (Cohen's d) Status VT Traj Proj SGP Non EL Not on track Non EL On track Non EL Proficient EL Not on track EL On track EL Proficient

18 If goal is to identify students who will meet proficient cut-score in the future Growth models show little improvement over a naïve status-only model For stricter cutoffs, growth models worse than nothing predictively If goal is to identify students making larger gains Some success for all models Larger effects for Projection and SGP On-track determinations NOT interchangeable

19 Overall results Value Table and Trajectory optimistic, especially for ELs, less accurate at high cutoff Projection and SGP pessimistic, but accurate ELs on track make bigger gains than nonels Percent of proficient students DOES impact model performance Lower cutoffs result in more false negatives for ELs for all models, lower accuracy Although ELs are less likely to be on track with projection and SGP, those that are show large gains by 6 th grade

20 Lakin & Young found much smaller differences in % on track and accuracy of models for ELs CA district had large Hispanic population (50%) Here, just 5% ELs Proficiency rate close to L&Y(high cut score) led to pessimistic projection model observed in CA data, replicated here Lower cut scores in this study led to more similarity in the number of students ontrack across ELs and non-els

21 Further research Do growth classifications predict real achievement outcomes? Do they help plan changes to instruction? Explore differences in growth trajectories across groups Monitor match rates for ELs Track differential accuracy by key subgroups 21 6/5/2014

DR. MAYA ANGELOU COMMUNITY HIGH

DR. MAYA ANGELOU COMMUNITY HIGH UPDATED - 03/28/16 Public school 1248 students CDS code: 19647330124479 SD: 65% AA: 11% FI: LAUSD-C EL: 41% AI/AN :0% PI: 0% 300 East 53rd Street SWD: 14% AS: WH: 0% Los