SDM Danger and Safety Assessment and. Assessment Case Reading. Florida Department of Children and Families

Transcription

1 The Structured Decision Making System for Child Protective Services SDM Danger and Safety Assessment and SDM Caregiver Risk Assessment Strengths Case and Reading Barriers Assessment Case Reading June 2017 August 2016 Florida Department of Children and Families

2 TABLE OF CONTENTS BACKGROUND... 1 SDM RISK ASSESSMENT CASES... 2 CONCLUSION Children s Research Center is a nonprofit social research organization and a center of the National Council on Crime and Delinquency. Structured Decision Making and SDM are registered in the US Patent and Trademark Office by NCCD, All Rights Reserved

3 BACKGROUND In June 2017, NCCD Children s Research Center (CRC) staff conducted a case reading of Structured Decision Making (SDM) risk assessments for the Florida Department of Children and Families (DCF). The CRC case reading aimed to assess the use and appropriate completion of the risk assessment, as well as identify strengths and opportunities in DCF s practical application of the assessment. DCF provided CRC with a sample of child protection cases along with remote access to their electronic case management system, Florida Safe Families Network (FSFN). CRC evaluated 201 cases, 191 of which had a risk assessment completed. This case reading memo will present findings based on the 191 completed risk assessments. One limitation of this case reading is the small sample size. Consequently, CRC s findings should not be viewed as representative of the entire DFPS caseload; they should be regarded as a general trend extrapolation. Florida s SDM risk assessment was updated in December 2016 during the course of the case reading period. All risk assessments that were reviewed were completed using the October 2016 Update version of the risk assessment. Therefore, case readers used this version of the risk assessment to complete the case reading and evaluate the accuracy of the completed risk assessments. It is important to note the changes to the SDM risk assessment made in December 2016, as they were intended to address some of the issues present in this evaluation. The changes to the manual included the following. Clarifying the SDM definition of who to include in the household. This clarification will help workers know which prior investigations to include when completing the risk assessment. Clarifying the SDM definition of previous child protective services. This clarification will help workers better understand what to count as previous child protective services, making scoring this item easier. Risk Assessment Considerations 1. Appropriate Completion: Was the risk assessment completed according to policy and CRC recommendations? 2. Narrative Support: Did the corresponding case narrative and documentation support the items selected on the risk assessment and the final risk level? by NCCD, All Rights Reserved

4 3. Action: Did the case action that was documented in narrative match the recommended action based on the final risk level? If the recommended action was not followed, did the documentation adequately reflect why and describe what action was taken instead? 4. Additional Considerations: Is there evidence in the narrative that the worker discussed the risk assessment with the family? Is there evidence in the record that the worker reflected on the risk assessment results when making decisions? SDM RISK ASSESSMENT CASES Appropriate Completion The risk assessment identifies families with low, moderate, high, and very high probabilities of subsequent referral and/or substantiation within the next 12 to 18 months. Initial risk assessments are completed on all CPS investigations, including new investigations of families currently receiving ongoing services. Risk assessments are completed prior to the conclusion of the investigation after the safety assessment has been completed. Workers should complete the risk assessment before deciding to open a case for post-investigation services or close the referral with no additional services. Findings Out of the 201 cases that CRC evaluated, 10 cases had no risk assessment completed. Five of the 10 cases had documentation explaining why the risk assessment was not completed, which included duplicate cases, investigations taking place in other jurisdictions, and allegations on non-caregivers. The remaining five cases had no supporting documentation regarding why risk assessments were not completed. Therefore, CRC reviewed the remaining 191 cases that included risk assessments. Of these, 134 (70%) were completed according to policy. All (100%) were completed on time. This indicates that workers understand that the risk assessment should be completed after determining whether any safety factors are present and before case closure. Of the 57 risk assessments that were not completed according to policy, two were completed on the incorrect household, 23 identified incorrect primary and/or secondary caregivers, and 41 included or excluded household members incorrectly on the risk assessment by NCCD, All Rights Reserved

5 Workers completed the risk assessment on the correct household 99% of the time, or for 189 of the 191 cases that included a risk assessment (Figure 1). Workers should always assess the legal parent/caregiver s household that is the subject of the investigation. If the alleged perpetrator is part of the child s household, that household is assessed. Figure 1 Completed on Correct Household No 2 (1%) Yes 189 (99%) N = 191 Another measure of accurate assessment completion according to policy is whether primary and/or secondary caregivers were accurately identified on the risk assessment. Case readers reviewed whether workers identified and listed the correct primary and secondary caregivers on the risk assessment and marked this as incorrect if the worker did not identify a primary and/or secondary caregiver who should have been identified or the worker assigned the primary and/or secondary caregiver incorrectly per the definitions in the risk assessment training workbook by NCCD, All Rights Reserved

6 Case readers found that of the 191 cases where a risk assessment was completed, workers correctly identified the primary and/or secondary caregivers in 168 (88%) cases. In 23 (12%) cases, workers misidentified the primary and/or secondary caregiver (Figure 2). Figure 2 Primary/Secondary Caregivers Identified Correct caregivers not identified 23 (12%) Correct caregivers identified 168 (88%) N = 191 The final measure of accurate risk assessment completion per policy is whether the worker included or excluded household members correctly on the risk assessment, which is separate from whether the worker completed the risk assessment on the correct household overall. The worker should consider all household members when completing the risk assessment. The October 2016 update of the SDM risk assessment defines a household member as any person who resides in a household, including the caregiver and other family members, additional relatives, visitors expected to stay an indefinite length of time, college students expected to return to the household as a primary residence, and all persons who have significant in-home contact with the child, including anyone with an intimate relationship with any person in the home or a household member s boyfriend or girlfriend who frequents the home by NCCD, All Rights Reserved

7 Workers accurately included or excluded household members in 150 (79%) cases where a risk assessment was completed. In 41 (21%) cases, workers either included individuals who were not a part of the household or did not include actual household members (Figure 3). Figure 3 Household Members Identified Correctly included or excluded household members 150 (79%) N = 191 Incorrectly included or excluded household members 41 (21%) Takeaways Most workers successfully completed the risk assessment on time per policy and on the correct household. Often, workers accurately identified the correct primary and secondary caregivers. There is some opportunity for improvement in identifying the correct household members. As previously noted, since this review period, CRC worked with DCS to clarify the SDM definition of household. This should increase clarity for workers on who to include as household members when completing the risk assessment. Next Steps for Workers Continue to complete a risk assessment before case closure, as stated in policy, or thoroughly document the reason for breaking with policy. Continue to ensure that all assessments are completed on the correct household. Review the definitions of primary and secondary caregiver and ensure that the correct caregivers are identified on the risk assessment by NCCD, All Rights Reserved

8 Review the current definition of household and ensure that all household members are included and that non-household members are not included on the risk assessment. Narrative Support Workers assess risk by considering the presence or absence of several items that increase the likelihood of future abuse or neglect. Evidence for the presence or absence of items on the risk assessment should be present in the narrative support (i.e., documentation) the worker provides. CRC case readers examined whether risk assessment items were supported by the accompanying narrative, whether selected or not. In addition to risk assessment items, case readers looked at the narrative support for the selection of overrides. Policy overrides change the final risk level to very high, while a discretionary override may only increase the risk level by one. Workers must have adequate documentation to support any override selected. Case readers also determined whether narrative information supported the final risk level after overrides. Findings Among the 191 cases for which a risk assessment was completed, 157 (82%) cases had at least one assessment item that was not supported by information in the case file or information in the case file supported at least one item that was not selected (Figure 4). Figure 4 All Selected Items Supported by Case File Yes 34 (18%) N = 191 No 157 (82%) by NCCD, All Rights Reserved

9 Of the 157 cases not supported by the narrative: A total of 108 (69%) cases included narrative support for an item that was not selected; and An item was selected but not supported by narrative in 75 (48%) cases. 1 The two-stream risk assessment includes separate indices for neglect and abuse. For some items on the risk assessment, neglect and abuse answers should match. In 31 cases, case readers found that items should have had consistent answers across indices but did not. For example, a worker might indicate that there was at least one prior abuse investigation on the neglect index but indicate that there were no prior investigations on the abuse index. This issue can be easily resolved by adjusting to a single-stream tool, which can take place when the risk assessment is validated. Another finding relates to item A2, Number of prior abuse investigations. For this item, if there are two or more prior abuse investigations, the worker is required to record the actual number. In 16 cases, workers correctly selected Two or more but the case file did not support what they listed under actual number. Although this did not change the outcome of the abuse risk score, this indicates that workers were not correctly identifying the number of prior abuse investigations. As priors are based on household members, CRC and DCS s clarification of the SDM definition of household should clarify for workers which prior investigations to include when scoring the risk assessment. 1 Some risk assessments fell into both categories. For example: for item N10 Housing, narrative supported current housing is physically unsafe and it was not marked AND worker selected homeless but narrative did not support the definition by NCCD, All Rights Reserved

10 Case readers also evaluated the number of overrides applied to the risk assessment and whether they were supported by the narrative (Figure 5). Of the three (2%) cases with overrides, one was a policy override and two were discretionary overrides. Additionally, two of the overrides that were selected were not supported by information in the narrative. Yes 3 (2%) Figure 5 Was an override applied? No 188 (98%) N = by NCCD, All Rights Reserved

11 After overrides, case readers evaluated whether the final risk level was correct and supported by the narrative. In cases where the final risk level was not correct, case readers also determined whether this risk level changed the final recommendation. CRC case readers found that the final risk level was supported by narrative for 123 (64%) risk assessments. For 27 (14%) cases, the final risk level was not supported by the narrative but the final recommendation remained the same. For 41 (21%) cases, the final risk level was not supported by the narrative and the final recommendation would have been different if completed correctly (Figure 6). Figure 6 Is the final risk level correct? No, but did not change recommendation 27 (14%) No, and changed final recommendation 41 (21%) Yes 123 (64%) N = 191 Takeaways Workers should ensure that selected items are supported by information in the case file. Furthermore, documentation should include what workers see in the field and provide evidence as the basis for selecting various items on the risk assessment. Documentation is important as it is the only way the case file can reflect what the worker s decisions were based on. Even if a worker selects the correct item on the risk assessment based on the information gathered, it will not appear correct if the selection is not supported by subsequent documentation. Only 3% of cases had a policy or discretionary override. It is generally expected that 5 8% of cases will have an override (either policy or discretionary), so it is important to ensure that overrides are applied in all appropriate cases by NCCD, All Rights Reserved

12 Next Steps for Workers Ensure that items selected on the risk assessment are supported by information in the case file. Correctly identify items on the risk assessment and appropriately document the justification for item selection. Continue to apply policy and discretionary overrides accurately and appropriately. Action Case readers determined whether the final tool recommendation of whether to offer services or not matched whether a worker subsequently offered or did not offer services in a case. Generally, if the risk level is low or moderate and the safety decision is safe, the recommendation is that services should not be offered. If the risk level is high or very high OR at least one safety factor remains, the recommendation is that services be offered. Occasionally a worker s decision may not match the recommendation, in which case the worker should document a clear, supportable reason in the narrative by NCCD, All Rights Reserved

13 Findings For most cases, the recommended action was the same as the action taken by the worker (Figure 7). Of the 24 (13%) cases where the final tool recommendation did not match the action taken, nine cases had a risk level of low or moderate with no safety factors and services were provided, but no adequate explanation was included. For one case, the risk level was low or moderate with safety factors, but the case was not offered services and no adequate explanation was provided. For the remaining 14 cases, the risk level was high or very high but the case was not offered services and no adequate explanation was provided (not shown). Figure 7 Does final recommendation match the action taken? No 24 (13%) N = 191 Yes 167 (87%) Takeaways The risk level is used to determine whether services should be offered. As risk level increases, more cases are offered services with the goal of reducing maltreatment recurrence. DCF should ensure workers document and provide rationale for why they do not offer services for high or very high risk cases. Due to the small sample size, it is not clear whether the high- or very high-risk cases that were not offered services are proportional statewide. However, the data show this action occurring in many instances, so it is important to track this and ensure that workers follow recommendations by NCCD, All Rights Reserved

14 Next Steps for Workers Ensure that workers are offering and not offering services based on CRC guidelines and recommendations. Continue to ensure that the action taken on a case either matches the recommended action or documents a clear justification for not doing so. Additional Considerations CRC case readers also evaluated whether there was evidence in the narrative that the worker discussed risk assessment results with the family and reflected on those results when deciding whether to offer services. Of the 191 cases where a risk assessment was completed, case readers found some evidence in the narrative that the worker discussed the risk assessment with the family in 22 (12%) cases. In 167 (87%) cases, there was no evidence that the worker discussed the risk assessment with the family. For 63 (33%) cases, some documentation appeared in the narrative that the worker reflected on the risk assessment results when deciding whether to offer services. CONCLUSION This case reading was done to analyze the quality of practical implementation of the SDM risk assessment after a period of use. While workers demonstrated timely completion of the tool, a large portion of assessments were completed inaccurately or lacked documentation in support of workers item selections. CRC recommends that DCF workers using the risk assessment focus on correctly identifying household members and caregivers, reviewing the risk assessment item definitions and thresholds, and understanding the purpose of the risk assessment classification to provide services to families with higher likelihood of future maltreatment recurrence. In addition, CRC can only determine what workers did or did not do based on information documented in the case file, which speaks to the importance of workers completing accurate and thorough documentation of conversations and information that they gather by NCCD, All Rights Reserved