Overview of the MEISR Pilot

Transcription

1 Virginia MEISR Pilt: Findings & Final Reprt Prepared by the Infant & Tddler Cnnectin f Virginia Part C Lead Agency Virginia Department f Behaviral Health and Develpmental Services Purpse f the MEISR Pilt The MEISR pilt was cnducted t explre whether r nt the Measure f Engagement, Independence and Scial Relatinships (MEISR) 1 culd be used t supprt Virginia s bjective f implementing a cnsistent child utcme summary (COS) prcess t facilitate and ensure: Families being included as full participants thrughut the early interventin prcess; Inclusin f each family s cultural values, beliefs and practices; Cllectin f substantial functinal infrmatin t infrm assessment fr service planning, aid in the selectin f IFSP utcmes, and help t determine child utcme ratings (i.e., initial, nging and exit ratings); and A high measure f inter-rater reliability. Overview f the MEISR Pilt Thirteen (13) lcal early interventin systems (with representatin frm each f the six early interventin regins) in Virginia were selected t partner with the Part C Lead Agency and the SSIP Functinal Assessment Team (SSIP-FA) t explre whether r nt the MEISR culd be used t supprt the bjective (see abve) recmmended by the SSIP-FA and apprved by the Virginia Interagency Crdinating Cuncil (VICC). In February f 2017, seventy-seven (77) participants representing the thirteen participating lcalities gathered in Richmnd t be trained n use f the MEISR by ne f its A number f assessment tls were reviewed prir t identificatin f the MEISR as mst prmising ntably due t the fact that the MEISR has been specifically develped t be cmpleted by bth the family and multidisciplinary team tgether at initial, nging and exit assessments. Using the MEISR with fidelity prmtes family participatin and allws fr cnsideratin f the family s cultural values, beliefs and practices each a key cmpnent f Virginia s verall bjective. c-creatrs, Nami Yunggren, Ph.D. Then, frm March t August, pilt participants met mnthly nline. These mnthly nline cmmunity f practice webinars included a learning prtin led by Dr. Yunggren and an pprtunity fr designated lead spkespersns frm each lcality t ask questins and share feedback. S as t facilitate the participatin f all individual team members, pprtunities fr feedback were prvided t all pilt participants mnthly by way f nline surveys and questinnaires. Each participating lcal system identified at least ne multi-disciplinary assessment team t participate in the pilt. Teams and their individual members were expected t participate fully in a number f activities including: Cmpletin f nline pre- and pst-pilt surveys which wuld invlve: Respnding t a variety f survey questins; 1 R. A. McWilliam and Nami Yunggren (2012) 1

2 Determining (as a team) child utcme ratings using narratives prvided; Determining the amunt f time it tk t get frm intake t written IFSP fr five (5) children using lcal billing and activity nte dcumentatin; and Submitting the mst recent three (3) IFSPs develped by each pilt system t I&TCVA (IFSPs selected shuld reflect the members f the pilt team members, as clsely as pssible); Attending ne in-persn MEISR training (presented by Nami Yunggren in Richmnd n February 10 th f 2017); Participatin n six ne-hur mnthly (March-August) fllw-up calls with Nami; MEISR Excel Versin Regular and nging use f the MEISR 2 ; and Offering at pilt s end a recmmendatin n whether r nt t recmmend use f the MEISR statewide. Pre- and Pst-pilt Data Analysis Highlights Of the 77 pilt participants, frty-seven (47) cmpleted bth the pre- and pst-pilt individual and team surveys. The data that fllws is based n these 47 participants. The MEISR pilt called fr teams t use the MEISR alng with the Decisin Tree thrughut the early interventin assessment prcess. Only nine (9) 47 participants respnded that they Always use bth the MEISR and Decisin Tree t determine the child utcme statements and ratings. Three questins asked participants t identify the utcme area in which they wuld put a specified skill that was taken directly frm the MEISR (see charts 4, 5 and 6). The MEISR clearly defines in which utcme area any ne specific functinal behavir A cmpleted MEISR becmes a child s prfile; this prfile can be cmpleted with pen and paper ( paper versin ) r by using an Excel wrkbk ( Excel versin ). Bth versins assist with determining IFSP utcmes and gals as well as child utcme ratings; and, when implemented with fidelity and accmpanied by use f the Decisin Tree, the majrity f respndents reprted an increased rate f inter-rater reliability culd likely be achieved. The Excel versin f the MEISR ffers an advantage by being able t quickly calculate the percentage f items a child has mastered cmpared t his/her peers in the fllwing areas: Furteen (14) typical rutines; Five (5) develpmental dmains (A, CG, CM, M, and S); Three (3) functinal dmains (E, I, S); and The three (3) OSEP glbal utcme areas (S, K, A). Prir t the start f the MEISR pilt, ITCVA staff tk the MEISR 2012 Excel versin and imprved its srting, filtering and calculating capabilities. Midway thrugh the pilt, with input frm pilt participants, Dr. Yunggren further imprved the MEISR 2012 by: adding Transitins as the 14 th typical rutine; adjusting sme f the starting ages in the 5 develpmental dmains (A, CG, CM, M, S); and rewrded items as actin statements. This revised versin was disseminated t pilt participants in July 2017 as the MEISR Excel v belngs; there is and can be nly ne crrect respnse. A number f participants answered incrrectly suggesting that the MEISR was nt referenced when respnding t these questins. 2 Over the 6-mnth-lng pilt, lcalities were asked t cmplete at least twelve (12) MEISRs and as many additinal as pssible. As needed, the Part C Lead Agency negtiated with lcal leads t re-evaluate the number f times the tl wuld be used during the pilt perid, giving cnsideratin t, fr example, system size and number f mnthly assessments cmpleted. 2

3 Pilt participants were expected t cmplete (at minimum) twelve MEISRs within the 6-mnth pilt perid. 16 f 47 participants (34%) met the expectatin. During the six (6) mnth pilt, participants cmpleted the fllwing number f MEISRs reprted in the chart belw: INSERT: Numbers f MEISRs cmpleted CHART Data in Relatin t Pilt Objectives 1. Families being included as full participants thrughut the early interventin prcess: a. Chart 10 indicates an increase in family participatin in determining child ratings. b. Chart 15 indicates ne (1) pilt participant ALWAYS refers back t infrmatin frm the MEISR t prmpt family participatin thrughut the early interventin prcess. c. Chart 16 indicates 68% (33 f 47) agreement that the MEISR encurages family participatin and engagement thrughut the early interventin prcess. d. Chart 20 indicates 11% (5 f 47) ALWAYS srt the cmpleted MEISR and discuss the child prfile t assist the family t identify IFSP Outcmes. Data indicates significant agreement that the MEISR encurages family invlvement. Data als reprts limited implementatin. 2. The inclusin f the family s cultural values, beliefs and practices: Chart 17 indicates that apprximately half the pilt participants agree the MEISR helped the team cnsider the family s culture, values, beliefs and practices in assessment, IFSP develpment and service delivery. 3. Substantial functinal infrmatin that will infrm the assessment fr service planning, aid in the selectin f IFSP utcmes and help t determine child utcme ratings (i.e., initial, nging and exit ratings): a. Chart 14 indicates an increase in the strength f agreement frm pre- t pst-pilt that I understand the difference between functinal behavirs and discreet skills. b. Chart 18 indicates that 85% f pilt participants smetimes r always feel cnfident using the MEISR t cllect rutines-based functinal infrmatin. c. Chart 19 indicates that 92% (43 f 47) agree the MEISR cllects substantial rutine-based functinal infrmatin that can be used t infrm the assessment fr service planning. d. Chart 21 indicates 70% (32 f 47) agree the MEISR aids in the indicatin f IFSP utcmes. e. Chart 22 indicates 19% (9 f 47) f pilt participants always use bth the MEISR and Decisin Tree t determine the child utcme (indicatr) statements/ratings. f. Chart 23 indicates 64% (29 f 47) agree the MEISR helps in determining INITIAL child utcme (indicatr) statements/ratings. g. Chart 24 indicates 74% (31 f 47) agree the MEISR helps in determining ONGOING child utcme (indicatr) statements/ratings. 3

4 h. Chart 25 indicates 67% (28 f 47) agree the MEISR helps in determining EXIT child utcme (indicatr) statements/ratings. Data indicates significant agreement that the MEISR prvides substantial functinal infrmatin that will infrm the assessment fr service planning, aid in the selectin f IFSP utcmes and help t determine child utcme ratings (i.e., initial, nging and exit ratings). Data als reprts limited implementatin using the MEISR in this capacity. 4. A high measure f inter-rater reliability. Chart 26 indicates 30% (13 f 47) agreement that based n my experience using the MEISR, I believe it will help increase inter-rater reliability acrss teams thrughut the state. It is difficult t assess the impact f the MEISR n inter-rater reliability due t limited implementatin. Time Cmparisn using the MEISR Lcal systems reprted the amunt f time it tk t get frm intake t written IFSP fr 5 children using lcal billing and activity nte dcumentatin pre- and pst- pilt. The MEISR was used fr pst-pilt minutes nly. The purpse f this activity was t determine if the MEISR added a substantial amunt f time t mve a child and family frm intake t written IFSP. times are listed in the chart belw. Less time =6 Mre time = 3 Same time = 3 INSERT: Time CHART MEISR Pilt Limitatins In-persn training in February 2017 did nt allw fr sufficient training n hw t use the MEISR. The MEISR underwent a number f revisins thrughut the pilt prcess, and the final Excel versin was nt available until July. This said, the final tl represents significant imprvements in functinal behavir descriptins, the additin f a typical rutine (transitins), and adjustment fr sme develpment starting age levels. The abve-mentined factrs cntributed t limited implementatin by pilt participants. MEISR Pilt Successes The use f the MEISR significantly increased parent engagement and participatin in the early interventin prcess especially as it relates t their child s functinal participatin in rutines and activities. There was a significant increase in many respndents level f cnfidence in determining utcme ratings in all three child indicatr ratings. 4

5 Althugh the increase in participant s use f the Decisin Tree was nt what we had hped, there was an increased awareness and use when cmparing pre t pst survey results. Despite the challenges with implementatin, theretically the MEISR did help shift a small percentage (10%) f prviders perceptin n the subjectivity f the child utcme prcess. State Decisins Based n Pilt Findings and Recmmendatins Based n the MEISR pilt findings, as described abve and detailed in the Attachment (r the fllwing charts, depending n hw we cmpile the dcument) and the pst-pilt recmmendatins f participants, the State ffice has determined the fllwing next steps: The MEISR as mdified thrughut the curse f the pilt and in its current (yet unpublished) frm des each f the things identified in the bjective and mre. Hwever, the Part C Lead Agency will nt mandate its use at this time. A chrt f 2-4 early adpter lcalities will be identified and Part C Lead Agency staff will prvide extensive technical assistance and supprt t them fr a perid f mnths t supprt their all-in implementatin f the mdified MEISR ( vameisr ). In the interim, all lcal systems are invited t experiment with the vameisr and will have access t any training materials (vides/mdules and any written resurces) develped by the state n hw t use the MEISR. 5