Staging options and resourcing for implementation of the four-level analysis framework for the Investment Approach

Transcription

1 Hon Amy Adams, Minister of Justice, Minister for Courts Hon Christopher Finlayson, Attorney-General Hon Michael Woodhouse, Minister of Police, Minister Responsible for the Serious Fraud Office Hon Simon Bridges, Associate Minister of Justice Hon Peseta Sam Lotu-Iiga, Minister of Corrections Implementation options for the Investment Approach Staging options and resourcing for implementation of the four-level analysis framework for the Investment Approach Date 29 July 2015 File reference Action Sought key recommendations Agree to a two-stage process for implementation that begins with a segmentation model and basic Return on Investment analysis now, and considers the need for further resource as the road map for Social Investment emerges Approve, subject to final approval from the Minister of Finance, initial one-off funding from the JSF of $2m for 2015/16 to build an actuarial model that shows the concentration of crime among cohorts of offenders, victims and locations Agree that the first stage of work will be conducted using the Integrated Data Infrastructure, so that the broader social outcomes of high-risk cohorts of offenders, victims and locations can be observed, as well as the broader outcomes generated by crime prevention investments Forward this paper to the Minister of Finance for final approval of the Justice Sector Fund proposal Timeframe/ Deadline

2 Contacts for telephone discussion (if required) Name Position (work) Telephone (a/h) 1st contact Paul O Connell GM Sector Investment Y Tim Hughes Minister s office to complete Principal Advisor Sector Investment Noted Approved Overtaken by events Referred to: Seen Withdrawn Not seen by Minister Minister s office comments Minister s feedback on quality of briefing note: 1 = Was not satisfactory 2 = Fell short of my expectations in some respects 3 = Met my expectations 4 = Met and sometimes exceeded my expectations 5 = Greatly exceeded my expectations Implementation of the Investment Approach 2

3 Implementation of the Investment Approach Purpose 1. This paper is to support a discussion at the 29 July meeting of Justice Sector Ministers about implementation of the Investment Approach. Executive summary 2. This paper seeks agreement to: a staged approach to implementation of the Investment Approach funding of $2m from the Justice Sector Fund in 2015/16 to execute stage one develop the initial model on the Integrated Data Infrastructure. 3. The Social Investment model is a conceptual framework using four interconnected levels of analysis: outcomes (level 1), cohorts of people/places (level 2), investments/services (level 3), and practitioners/providers (level 4). The Social Investment model is being implemented in a number of agencies as the Investment Approach. 4. The Justice sector has completed its feasibility work and is ready to begin implementing the Investment Approach. We propose starting with two pieces of work in a first stage of implementation. The first piece of work will use existing resource to describe the Return on Investment being generated by the sector s crime prevention activity, in terms of both reduced crime and other improved social outcomes. This will provide a broad overview of where there are opportunities to reinvest to more effectively reduce crime and achieve broader social outcomes. 5. The second piece of work would require new funding to build an actuarial segmentation model. This model would predict the long-term crime likely to be experienced or committed by cohorts of victims, locations and offenders. For each cohort, we would examine: the proportion of future crime expected to be committed or experienced by the cohort the broader social outcomes being experienced by the cohort (welfare receipt, educational achievement, ill health etc) the demographic features and personal profiles of the people and places in the cohort the pattern of service use in the Justice sector (family court interactions, collections engagement, probation experience etc) for the people in the cohort the views of people in the cohort about the Justice sector the perspectives of frontline Justice sector staff about the people in the cohort. 6. We propose developing our analysis on the Integrated Data Infrastructure to allow the broader social outcomes of high-crime cohorts to be better understood. 7. In combination, the segmentation model and ROI analysis will provide good insight about the best opportunities to achieve our BPS results. In addition, this modelling capability will be useful for policy simulations and evaluations, such as understanding the effect that Implementation of the Investment Approach 3

4 changes such as the Public Protection Orders and Three Strikes legislation are having or are expected to have on crime through incapacitation of high-risk offenders. 8. A segmentation model will require new funding to contract specialist support from actuaries or other data scientists. To implement the Investment Approach to Welfare, MSD received ongoing funding of $5m per year, of which about $1m per year is used to pay for external actuarial support. 9. We are seeking out-of-cycle funding of up to $2m from the Justice Sector Fund to pay for the initial model development, although we hope to secure a provider for less than this amount. We consider $1m may be insufficient because: unlike MSD, we do not have an in-house actuarial team the initial development costs of the model are higher than the costs of maintaining/updating it the Justice sector datasets are spread across multiple agencies, whereas the welfare dataset was for a single agency there are likely to be additional costs associated with using the Integrated Data Infrastructure, particularly for the stronger Australian providers that we wish to attract. 10. To reduce risk to the crown, we intend to structure the contract in a way that places a portion of the revenue at risk for the provider, contingent on achieving stretch objectives. 11. We propose placing remaining work to build the Investment Approach into a second stage of implementation because of uncertainty in a number of key dependencies. 12. The most important dependency is with the Social Investment model. Although there is broad agreement across the social sector as to the basic concept, the specifics of the operating model for Social Investment have not yet been developed. As such, we can not yet have certainty about the final role of the Justice sector within the Social Investment model. 13. Other dependencies include the future of the New Zealand Crime and Safety Survey, the investment case for a sector-level data platform, the project plan for Policing Excellence: the Future, and the long-term measure for the harms associated with crime. 14. Next steps 15. Following any funding decision, we will begin the procurement process for the segmentation model. We will report back to sector Ministers for their August meeting with a progress update on the segmentation work and on the Return on Investment work. Implementation of the Investment Approach 4

5 Background Context: Social Investment 16. The Social Investment model is a conceptual framework that describes an integrated analytical function across the social sector. The purpose of this integrated analysis is to provide Ministers and other decision-makers with robust and consistent information about: those cohorts of individuals, families and communities that are at risk of multiple poor outcomes across the domains of health, education, welfare, justice and others the effectiveness of those services and frontline practices delivered to these high-risk, high-need cohorts the full Return on Investment generated by these services across multiple outcome domains. 17. The Social Investment model is ultimately about thinking about social services from the perspective of those cohorts of people who need support and providing services that meet the full needs of these cohorts, rather than operating from the narrow perspective of each agency s service channel. 18. The Social Investment model is moving from conception to reality. There are two types of work underway to implement Social Investment. On the one hand, there are short-term initiatives and demonstration projects consistent with the Social Investment framework, such as the Budget 2016 process, the review of services for youth, the Vulnerable Children s work, and Social Sector Trials. 19. On the other hand, there is long-term work underway to build capability within and between agencies, and achieve an enduring change in the way investments are made in the social sector. This more enduring application of the Social investment model is generally described by the term Investment Approach. The Social Investment model has been applied in this way in ACC and PHARMAC, with the Investment Approach to Welfare, and is being applied in this way with the Investment Approach to Social Housing. 20. There is currently no road map for when, if and how this model will be rolled out across the remaining parts of the social sector, or what the final operating model will be. We append some thoughts on the decisions that will be needed in the development of this road map. We have shared these thoughts with MSD and Treasury, as we consider it is their role to lead this discussion and seek decisions from Ministers. 21. This paper focuses on the challenge faced by the Justice sector, to build the capability to contribute to the Social Investment model by implementing the Investment Approach. It focuses on concrete steps that will build the sector s capability to contribute and that can be taken in the absence of a clear road map on a no regrets basis. The Justice sector s contribution to Social Investment 22. On 26 June 2015, Justice sector Ministers discussed the paper Advancing an Investment Approach. Implementation of the Investment Approach 5

6 23. Ministers agreed that the appropriate scope of the Investment Approach in the Justice sector is the crime prevention expenditure, and commissioned further advice for their July meeting covering off implementation options. You are now reading this advice. 24. Proposed approach: a two-stage implementation 25. We propose a two-stage approach to implementation of the Investment Approach. The first stage would begin immediately with support from the Justice Sector Fund. 26. We favour a two-stage approach because we are not currently in a position to plan out the entire implementation project, but want to make initial progress wherever possible. We are not able to plan out the entire project now because of some important dependencies: the operating model for Social Investment is still being developed, so the expected role of the Justice sector in supporting this model in its endstate is still unclear the future of the New Zealand Crime and Safety Survey is being considered in a project that is not yet complete the investment case for a sector-level data platform (the Intelligent Justice Sector concept) is still being developed the investment case for Policing Excellence: the Future is still being developed options for a long-term measure for the harms associated with crime are still being considered. 27. These dependencies are discussed in more detail further below in the paper. 28. This paper first discusses the decisions needed to implement the first stage of implementation, and then identifies the areas we consider are best implemented during the second stage. Outputs of stage one 29. With reprioritised analytical resource from the sector, we will produce over the next 18 months: selected pieces of descriptive analysis from the Integrated Data Infrastructure about the broader outcomes experienced by high-risk groups of offenders and victims, such as the welfare receipt of released offenders selected findings about the effect that Corrections rehabilitation programmes have on broader social outcomes selected pieces of one-off data analysis using the IDI to support priority pieces of work such as the domestic violence initiative a basic view of the cost-effectiveness of the investments used to reduce crime, including their effect on broader social outcomes where relevant. Implementation of the Investment Approach 6

7 30. With additional funding from the Justice Sector Fund, we would also produce an actuarial model that would divide the New Zealand population into segments based on their risk of future crime. For each segment, we would analyse: the proportion of future crime expected to be committed or experienced by the segment the proportion of crime prevention expenditure being targeted at the segment the broader social outcomes being experienced by the segment (welfare receipt, educational achievement, ill health etc) the demographic features and personal profiles of the people and places in the segment the pattern of service use in the Justice sector for the people in the segment (family court interactions, collections engagement, probation experience etc) the views of people in the segment about the Justice sector the perspectives of frontline Justice sector staff about the people in the segment. 31. This segmentation model could also be used to analyse the effects of policy changes on crime, as described in the inset box. Example of a policy simulation using the segmentation model The segmentation model would be based on predicting the future crime of victims, offenders and locations. These models could be used to understand the effect of policy changes on crime. For example, if asked to consider a modification to the three strikes legislation, policy advice developed with the Investment Approach infrastructure would be able to inform Ministers about: the expected future crime profile of those offenders who would be captured by the legislative change, and how this risk profile compares to those offender segments with the highest risk profile the expected number of crimes that would be prevented by the policy change through incapacitation, and therefore the return on investment of this approach to reducing crime Under the status quo, these pieces of analysis would be challenging to undertake, and would take substantially longer to produce. 32. To clarify what the first stage would not produce, refer to the section of this paper that details what would be produced in the second stage of implementation. Decision: Funding for the first stage of work 33. A segmentation model will require new funding to contract specialist support from actuaries or other data scientists. We are seeking this funding from the Justice Sector Fund. Given a funding round has just been completed we propose seeking funding now, out-of-cycle, rather than waiting for the next scheduled bid round. 34. To implement the Investment Approach to Welfare, MSD received permanent ongoing funding of $5m per year, of which about $1m per year is used to contract external actuaries. Owing to the greater complexity of the Justice sector datasets, the higher costs associated with initial development, and the fact we want to use the Integrated Data Infrastructure, we think it is prudent to allow for funding of up to $2m in the first year. However, we hope to Implementation of the Investment Approach 7

8 secure a provider for less than this amount, and we expect the outyear funding requirements to be substantially less than this. 35. This amount will also help us attract the stronger Australian providers, who could incur additional costs associated with using the Integrated Data Infrastructure, which can only be used by people who are physically located in New Zealand. 36. In Work and Income and ACC, external actuaries are still contracted every year to support the in-house capability they have built, and the same model is likely to apply in the Justice sector. As such we intend to structure the tender for a single-year contract with rights of renewal for two further single-year contracts. We will also endeavour to structure the contract in a way that places a portion of the value of the contract at risk, subject to delivery of specific outcomes. 37. Exercising the rights of renewal would be contingent on securing ongoing funding for outyears through the Budget 2016 process or some other means. Decision: Data platform for first stage of work 38. We propose undertaking the initial development of the Investment Approach on the Integrated Data Infrastructure (IDI). The IDI has the widest set of data currently available. This will allow us to identify which pieces of information from other sectors are useful to help predict future concentrations of crime, and will allow us to understand where crime prevention activities are effective at improving broader social outcomes such as welfare receipt. 39. In the medium-term, we may wish to take our actuarial models from the IDI and apply them to a non-anonymised, operational dataset that includes the relevant information from other sectors. This would allow the insights generated by the Investment Approach to be fed directly to the frontline to support tactical decision-making in real time. 40. The choice about how and when to do this depends on the operating model for social investment. Broadly, if a centralised operational data platform emerges for the whole social sector then we may wish to use this platform. If a partially decentralised model is chosen then we may wish to build our own data platform for this purpose, but in a way that is coordinated with and does not duplicate other data platforms in the social sector. The options for data co-ordination under social investment are discussed in the appendix. 41. Although the IDI is the best short-term option, it does have some limitations: The IDI can only be used within NZ, meaning it may be more difficult/expensive to attract the higher-quality providers that are based in Australia Where data is not collected by agencies (e.g. information about the individuals receiving contracted services from MSD), this information is not in the IDI Much of the data is quite raw and unfamiliar to many data analysts, which will add to development time The IDI is anonymised, meaning it can not currently be used to develop tactical-level tools for the frontline, although the frontline will benefit from the strategic insights generated by the IDI Implementation of the Investment Approach 8

9 42. However, we note that the anonymity of the IDI may not be a major barrier to operational use, given the intelligence capabilities in Police and Corrections. It may be sufficient to provide general profiles of high-risk offenders, victims and locations to intelligence units in these agencies, and have them use these anonymous profiles to identify specific people and places of interest. 43. While the IDI does have limitations, the alternative approaches in the short term are undesirable: Using MSD s sector-level platform (the IAP) is undesirable because it contains only a subset of the information that is in the IDI, and some of the information they have from other agencies and others may not be accessible to us (because in many cases the information was shared with MSD for a specific purpose that does not include crime prevention) Building a new data platform immediately as a prerequisite is undesirable because it may be inconsistent with the emerging social sector approach to data integration, because it would delay the development of the Investment Approach by at least 12 months, and because we do not yet know what data from other agencies we really need. 1 Using only Justice data is undesirable, because it would not allow us to observe the effect of crime prevention investments on broader social outcomes, and would lead to an isolated approach unconnected to the broader Social Investment work. Components of the second stage of work 44. For each area of work, this section also explains the rationale for not including it in the first stage of implementation. Permanent data platform 45. As discussed earlier, the Integrated Data Infrastructure is appropriate in the short-term but may not be the best medium-term option for a data platform to support the Investment Approach. Permanent analytical capability 46. In the same way that the endstate for a social sector data platform(s) is unclear, so too is the desired endstate for data analysis. There will inevitably need to be some balance of capability and function between centralised teams (such as in Treasury and Stats NZ), line agencies (such as the Ministry of Justice), and external providers (such as consultant actuaries). Without knowing the intended balance, it is difficult to understand how much 1 At MSD, they spent a year negotiating an MOU with Health to gain access to their data to build a model to predict child maltreatment, only to find that the health data did not add to the predictive validity of the model. The IDI did not exist at this time, but if it did then developing the tool on the IDI before seeking the data from Health would have saved time and effort. Implementation of the Investment Approach 9

10 analysis will be provided for us by a centralised team for the whole social sector, how much we will be expected to produce ourselves, and what constraints there might be on the use of external providers (such as if all agencies are expected to use a forward liability approach, calculated by a single provider). 47. It is also unclear the extent to which we will be able to redeploy existing analytical resource to fill the new analytical positions, and therefore the need for new resource. In any case, the Justice Sector Fund can only provide one-off funding, so a Budget bid would be needed for any ongoing funding. A response cost model 48. The stage one implementation would produce models to understand crime: where and among whom it is concentrated, and the effectiveness of crime prevention activity at reducing it. Understanding the costs that crime imposes on the Justice sector in its response, and therefore the fiscal return generated by crime prevention activity, is a separate analytical exercise. 49. Cost modelling of this type was not needed in the welfare system because the fiscal costs of benefit receipt are fixed in legislation in the form of standard benefit payments. To understand the fiscal costs of crime requires understanding the way in which different crimes flow through to the activities in the Justice sector such as investigation, prosecution, appeal, and sentence management, and the costs of these activities. 50. Although this exercise may sound straightforward, it is fairly complex in practice. There are many decision points through the Justice pipeline, and the decisions taken at these points are usually taken with a high degree of practitioner discretion, such as where a police officer decides whether to prosecute an offence or use an alternative resolution such as diversion. The use of discretion tends to change over time with policy and practice changes, making the system quite dynamic and therefore challenging to model accurately. 51. Previous efforts to model the criminal response pipeline at this level of detail have met with limited success. In particular, a comprehensive model to predict movements through the pipeline started development about ten years ago and was abandoned six years later, still incomplete, as it was struggling to adequately model the complexity of the system. 52. Given the challenge of this modelling exercise, it would add risk to the first stage to attempt to build this model at the same time as the segmentation model, which is also challenging in itself Advice on this issue is being presented for Ministerial discussion in September. Implementation of the Investment Approach 10

11 Additional work on Return on Investment 55. The stage one implementation allows for the redeployment of existing analytical resource to do basic, data-driven effectiveness evaluations of the crime prevention expenditure. This approach will be suitable for some, but not all, of the roughly $500m portfolio of prevention activity. In particular, some types of policing activity will not be amenable to this type of effectiveness analysis. 56. In the second stage, we may wish to expand the amount of resource being used to understand the effectiveness of the crime prevention portfolio. The amount of additional resource that may be needed depends on how successful we are at evaluating the portfolio using existing resource, and the extent to which Police seek resource to undertake this work as part of Policing Excellence: the Future. Monitoring of unreported crime 57. Any use of recorded crime as a performance measure creates a perverse incentive not to report crime, or to treat a minor crime as an infringement offence. It also means that changes in crime rates are difficult to interpret without information about how reporting rates are changing. 58. As noted in earlier papers, the infrequency of the New Zealand Crime and Safety Survey (NZCASS) currently limits our ability to understand movements in recorded crime, as we do not closely track whether reporting rates are changing over time. 59. The design of the NZCASS is being reconsidered over the next few months. As part of this process, options will be considered to increase the frequency of the NZCASS. Any increase in frequency will require ongoing funding, and therefore the Justice Sector Fund is not the appropriate source of funding. Funding and Accountability 60. As noted in earlier briefings, the Investment Approach to welfare was also accompanied by changes to funding and accountability arrangements. The need for similar changes in the Justice sector depends partly on decisions about the future of the Justice Sector Fund, and on decisions on the operating model for Social Sector. The latter issue is being considered in the context of the Productivity Commission s report into social services. 61. Owing to these dependencies, we consider this issue is best considered in the second stage of work. Next Steps 62. Following any funding decision, we will begin the procurement process for the segmentation model. We will report back to sector Ministers for their August meeting with a progress update on the segmentation work and on the Return on Investment work. Options for a long-term measure of the harm from crime will be presented for consideration by Ministers at their September meeting. Implementation of the Investment Approach 11

12 Financial implications 63. The resourcing needs for the first stage of work are summarised in the following table: 2015/ / /18 TOTAL $ 2,000,000 $ - $ - $ 2,000, This funding is a maximum amount; the precise amount spent will depend on the contract negotiations with an external provider. 65. Allowing for the most recent round of funding decisions, there is currently $43.186m in the Justice Sector Fund. The amount that would remain in the Justice Sector fund after this proposal would be $41.186m. Consultation 66. This paper reflects the position of the Ministry of Justice, NZ Police, and Department of Corrections. Treasury has been consulted. Recommendations 67. It is recommended that you: 1. Note that to support the Social Investment model as it emerges, the Justice sector needs to improve its capability to analyse the risk levels and wider social needs of cohorts of victims, offenders and locations, and its capability to analyse the Return on Investment of its crime prevention services 2. Agree to a two-stage process for implementation that begins with a segmentation model and basic Return on Investment analysis now, and considers the need for further resource as the road map for Social Investment emerges 3. Approve, subject to final approval from the Minister of Finance, initial one-off funding from the JSF of $2m for 2015/16 to build an actuarial model that shows the concentration of crime among cohorts of offenders, victims and locations YES / NO YES / NO Implementation of the Investment Approach 12

13 4. Agree that the first stage of work will be conducted using the Integrated Data Infrastructure, so that the broader social outcomes of high-risk cohorts of offenders, victims and locations can be observed, as well as the broader outcomes generated by crime prevention investments YES / NO 5. YES / NO 6. Forward this paper to the Minister of Finance for final approval of the Justice Sector Fund proposal 6.1. Minister of Finance: Approve the Justice Sector Fund proposal in recommendation 3 Jean-Pierre De Raad Acting Deputy Chief Executive, Sector Group Date: / / YES / NO YES / NO Implementation of the Investment Approach 13

14 Justice Sector Fund approval for Implementation of the Investment Approach Hon Bill English Minister of Finance Date: APPROVED / SEEN / NOT AGREED Hon Christopher Finlayson Attorney-General Date: APPROVED / SEEN / NOT AGREED Hon Peseta Sam Lotu-Iiga Minister of Corrections Date: APPROVED / SEEN / NOT AGREED Attachments: A3 on Implementation of the investment Approach Hon Amy Adams Minister of Justice, Minister for Courts Date: APPROVED / SEEN / NOT AGREED Hon Michael Woodhouse Minister of Police Date: APPROVED / SEEN / NOT AGREED Implementation of the Investment Approach 14