OUTCOME-BASED MANAGEMENT: A PLANNING TOOL FOR HIGH LEVEL MANAGEMENT Roger Lucero Sanchez* E XECUTIVE S UMMARY

Transcription

1 OUTCOME-BASED MANAGEMENT: A PLANNING TOOL FOR HIGH LEVEL MANAGEMENT Roger Lucero Sanchez* E XECUTIVE S UMMARY INTRODUCTION Outcome-based management (OBM) is a planning tool for high level management. Traditional program management is based upon an individual approach to program performance and outcome measures. Outcome-based management binds multiple programs together to achieve higher levels of planning. The OBM process usually involves a communitywide visioning and planning process with goal development and outcomes measures. Programs and budget resources are aligned under OBM to contribute towards desired results. Outcome measures are then developed and implemented through a system of data collection and application. BACKGROUND San Mateo County Human Service Agency (HSA) implemented outcome-based management and budgeting agency-wide in FY The OBM process engaged the community to develop a county-wide vision of the future and develop goals along with measurable outcomes. Community and stakeholders identified three major categories which contribute to the future of San Mateo County: Economic self-sufficiency for individuals and families, Supportive services for families and children, and Community capacity-building. San Mateo County then developed a plan to align programs under these general service areas and outcome measures were developed along with budget. Santa Clara County Social Services Agency (SSA) piloted performance based budgeting in FY Under the Santa Clara County model, program and services are aligned with the county s yearly budget priorities. Outcome measures are designed to address multiple needs within the organization and apply at various levels, including individual, program, and policy levels. FINDINGS OBM is an excellent system of management when built upon realistic goals and outcomes measures. OBM should be client-driven and not create new superstructures or drain existing community and program needs. Measures should be flexible to account for variables in human services and must be applicable at various levels. Data collection and interpretation must be valid and reliable. OBM must be able to adapt to changes in individual, program, policy, and economic circumstances which require human service systems to adjust and refine their efforts to meet new challenges. RECOMMENDATIONS San Mateo County HSA and Santa Clara County *Roger Lucero Sanchez is a Social Work Supervisor with Santa Clara County Social Services Agency. 177

2 BASSC Executive Development Training Program SSA have implemented outcome-based management county-wide. Both plans are still new and the fine points of implementation are still being refined. The following recommendations are points of review for both County departments to consider in the implementation process. OBM should be utilized as a planning and management tool. OBM must be practical and not strain existing community needs and services. OBM must be built upon realistic goals and desired outcomes. OBM measures must be flexible to address variables in human services. OBM measures must be applicable at various levels and address program performance, management, and policy needs. Data collection and interpretation must be valid and reliable. Outcome measures should be incorporated in all service contracts. Agencies should incorporate a method to redefine OBM outcomes as needed. 178

3 OUTCOME-BASED MANAGEMENT: A PLANNING TOOL FOR HIGH LEVEL MANAGEMENT Roger Lucero Sanchez INTRODUCTION Outcome-based management (OBM) represents a serious challenge for human services. Efforts to measure the effectiveness of programs and services for the most part have fallen short in application and validity. Unlike units of measurement used in production, human service outcomes are much more difficult to capture. Families and individuals are subject to multiple changes which contribute to their personal and economic wellbeing. Variables in human services are difficult to control and disregard any attempt to measure outcomes. Traditional human service measures are based on service delivery rather than outcome measures. State and federal programs require delivery of specific services as incentive for funding. Foster care and block grant funding have very specific service requirements. While the Adoptions and Safe Families Act requires child welfare agencies to create concurrent plans for children under three removed from their parents, the Workforce Investment Act requires an integration of employment related services as one of its main components. Most contracts with community-based agencies are based on service delivery rather than outcome measures whether it is in parenting education, domestic violence, or employment-related services. Human service effectiveness and accountability has become more prominent in recent decades as agencies struggle to meet the growing demand for human services and compete for limited resources. Service providers are now required to justify their efforts and make optimal use of resources. Outcome-based management, however, is still in its early stages and outcome measures in human services must still be designed to provide useful and meaningful results. This report will examine two systems of outcomebased management, San Mateo County s Outcomebased Management and Budgeting, and Santa Clara County s Performance-based Budgeting. BACKGROUND San Mateo County recently introduced an ambitious plan to measure service outcomes based on a county-wide vision for the future. Outcome-based Management and Budgeting is an effort to align budget resource allocation with service outcomes based on goal commitments undertaken by the county to achieve an overall vision for the future. In October 2000, the Board of Supervisors engaged the public in the planning process for OBM by hosting a series of community meetings to develop a county-wide vision for the future, and establish goals along with measurable outcomes. To ensure broad representation, the Board of Supervisors scheduled three forums which targeted youth, the technology industry, and the Spanish speaking population. In addition, the board held a series of townhall style meetings and encouraged people to provide on-line input over a six-month period. During this goal setting and planning process, communities in San Mateo County were challenged with the task to develop of a shared vision for the future 179

4 BASSC Executive Development Training Program with clearly defined goals to be used to measure outcomes. Community workgroups identified three major categories that contribute to the future of San Mateo County and can be used to measure service outcomes: Economic self-sufficiency for individuals & families, Supportive services for families and children, and Community capacity-building. METHODOLOGY In order to facilitate the development of outcome measures towards these goals, the county government developed a remarkable plan to incorporate services into these categories (see Appendix A). The development of outcomes measures towards these goals was an equally daunting task. Outcome measures needed to be compatible with existing data collection systems in order to avoid duplication of efforts. Information on outcome measures was drawn from existing data available to agencies. Measures were also limited to the scope of services provided by the agencies and contract service providers. The County Manager and Board of Supervisors offered leadership and support in this area to ensure outcome measures were meaningful and practical. Assistance included a contract with the Fiscal Policy Studies Institute to develop a framework of methodology based on The Four-Quadrant Model. The Four-Quadrant Model generally describes how efforts lead to outcome. The model focuses on Activities (efforts) and Outcomes (effects) which are then split by Quantity (how much #) and Quality (how well %). Technical assistance also included a contract to develop SHARP data warehouse and web intelligence to increase the managers ability to use OBM measures and guide program activities. SHARP data warehouse basically extracts data from existing computer systems employed by San Mateo County agencies including, Case Data System (CDS), Child Welfare System (CWS/CMS), Job Training and Automation System, Service Management Access Resource Tracking System (SMART), and GAIN Information System. Data extracted is transferred to a SHARP data warehouse for analysis & reporting. SHARP questions can be designed to meet OBM measures. SHARP can provide managers with access to adhoc reporting, broadcast and distribution, and geographical analysis (i.e. charts, alerts, etc). The SHARP data warehouse is limited, however, to information stored in the above mentioned computer systems. Outcome measures were extracted from existing data and carefully interpreted for validity and reliability. A series of analytical questions, designed to take into account variables such as economic and policy changes, were used for interpretation. Managers were trained in the field of descriptive statistics to properly interpret data results. OBM budget reporting required agencies to structure fiscal reports along commitments developed during the county wide visioning process. Services must demonstrate contributions towards goals of Economic Self-Suffiency, Family Strength, and Community Capacity. Agencies were provided OBM budget planning worksheets to structure and report service outcome measures towards the goals undertaken in the OBM process. Budget planning worksheets included the following descriptions: 180

5 Program Area Priority Strategy Description of Activities Support Needs Costs Funding Sources Program Relation to OBM Measures Budgeting Information. Agencies were also required to provide Strengths, Limitations, Opportunities, and Threats assessments (SLOT s) in relation to OMB measures and commitments. SAN MATEO COUNTY HUMAN SERVICES AGENCY San Mateo County Human Services Agency (HSA) implemented Outcome-Based Management and Budgeting agency-wide in FY An HSA steering committee was created to develop policy and make decisions guiding OBM implementation. The infrastructure to support and implement OBM included a project director and implementation teams in each of the OBM service areas (see Appendix B). The Steering Committee moved quickly to develop an OBM implementation plan for the first year with the following objectives: Draw upon existing agency resources and structures; Involve program staff and community in the process; Use OBM backfill positions to promote staff and succession planning; and Accommodate short-term and long-term goals. The plan was divided into four sections: implementation activities, contextual considerations, resource allocation, and time guidelines. The second year OBM implementation plan was structured to support budget planning, data development and reporting, training, technical assistance, and information feedback. The plan outlined detailed strategies for each of these areas and recommended agency-wide implementation. A tiered approach was used to support the parallel and simultaneous developments required to build the information and communications infrastructure essential to OBM. The objectives of the second year plan were as follows: Involve staff and community in budget process; Phase MIS construction to match program priorities for data; Provide training and technical assistance; Develop program specific outcome measures with contract providers; and Manageable and achievable implementation activities. HSA estimated that OBM would take several years to implement and developed recommendations and time guidelines for a phase-in process. Modifications to the plan were later introduced as a result of September 11th and the subsequent economic down turn which required HSA to re-deploy resources and re-assess the feasibility to undertake OBM activities. HSA piloted OBM within the Department of Alcohol and Other Drug Services (AOD). AOD s treatment and assessment components include contract services with 20 community based agencies and 30 AOD employees. The pilot project included the application of budget templates through planning meetings with staff and contract service providers to develop program performance mea- 181

6 BASSC Executive Development Training Program sures. Outcome measures were based on data already collected and negotiated with communitybased agencies for practical application purposes. Outcome measures were developed to be consistent with state reporting requirements already utilized for funding purposes, in effect, making the Human Services Agency the liaison of information between the state and local agencies. The pilot success was attributed in part to existing data collection tools and flexibility to modify outcome measures for practical purposes. OBM measures were incorporated in all HSA contracts. SANTA CLARA COUNTY SOCIAL SERVICES AGENCY At the direction of the Board of Supervisors, and with the oversight of the finance and & government operations committee, the County of Santa Clara began work to develop and implement Performance- Based Budgeting (PBB). Initial pilot projects for PBB were presented to the County Executive s FY 2001 and FY 2002 Recommended Budgets. The County Executive s Office of Budget Analysis (OBA) engaged the services of PVK Management Consulting, Inc. (PVK) to assist in the development and implementation efforts. In December 2000, PVK presented a Readiness Assessment and Initial Action Plan for Performance Based Budgeting report to the Board of Supervisors. The report introduced the use of a program logic model to standardize the approach to performance-based budgeting across departments and provided recommendations for PBB implementation. The program logic model applies outcome-based measurements at strategic, program, and operational levels and contributes to consistency throughout the organization to assure that efforts are generally focused on similar levels of results. Initial efforts to develop PBB have been focused on training all county departments on the program logic model tool provided by PVK and on developing a consistent, standardized format for presentation in the FY2003 budget document. The budget document format includes the following elements: Public Purpose: Similar to departmental mission statements. Desired Results: Defined through use of program logic model. Performance Indicators: Provide data on desired results. Description of Services and Programs: Narrative description of products and services. County Executive Funding Recommendations: Narrative summary of recommendations. Under this format, public purpose and desired outcomes are linked to the Board of Supervisor s budget priorities published in January of each year and are based on the mission of the Santa Clara County Social Services Agency. Although public purpose and desired outcomes are not likely to change, PBB provides flexibility for refinement of performance indicators. The Department of Family and Children s Services (DFCS), the Agency Office (AO), the Department of Employment Services (DEBS), and the Department of Aging and Adult Services (DAAS) all presented under this new budget document format. Simultaneously, Santa Clara County began Cross Systems Evaluation (CSE) in May 1998 to develop a common outcome evaluation system to determine the effectiveness of services provided by the county, by county funded agencies, and by partner agencies. CSE, together with county departments and Information Services Department (ISD), designed and launched a centralized database and delivered 182

7 profiles of county programs serving children and families. ANALYSIS Outcome-based management is a tool that can be utilized for high-level planning. Program management is often driven by specific outcomes to individual programs. OBM binds individual programs to achieve higher levels of planning. In San Mateo County, OBM was implemented to align program resources and budget to achieve a vision developed by the community for its own future. In Santa Clara County, PBB was implemented to align programs and services with the county s yearly budget priorities. Regardless of desired results, OBM is a useful tool for high-level planning and management of multiple programs. In order to be successful, OBM must be practical and reliable. Most OBM models focus on the development of outcome measures and provide some type of methodology for the collection and application of data towards desired results. In order to be practical, desired results must be realistic and achievable with the resources available. OBM must not be so ambitious as to require the creation of new superstructures to achieve its desired outcomes. Available resources must be able to contribute to OBM without additional strain to existing community needs and programs. OBM must be client driven and built upon realistic goals and outcome measures. This requires OBM planning to take careful consideration of community needs and resources in the development of desired outcomes. OBM measures must also be valid and reliable to truly reflect the desired outcomes. Measures must be flexible to account for variables inherent in human services and must be applicable at various levels. Data collection and interpretation should be applicable at program performance, management, and policy levels. Outcome measures work best when applied towards a desired result, but rigid measures can not address multiple variables. Although desired results do not have to be compromised, outcome measures must be refined to remain valid and reliable. RECOMMENDATIONS San Mateo County Human Services Agency and Santa Clara County Social Services Agency have implemented outcome-based management countywide. Both plans are still new and the fine points are still being refined. The following recommendations are points of review for both plans to consider in the OBM implementation process. OBM should be used as a planning and management tool. OBM must be practical and not strain existing community needs and services. OBM must be built upon realistic goals and desired outcomes. OBM measures must be flexible to address variables in human services. OBM measures must be applicable at various levels and address program performance, management, and policy needs. Data collection and interpretation must be valid and reliable. OBM outcome measures should be incorporated in all service contracts. ACKNOWLEDGEMENTS I would like to acknowledge Madelyn Martin, Deputy Director, San Mateo County Human Services Agency, for her commitment and dedication to the professional development of BASSC interns. 183

8 BASSC Executive Development Training Program APPENDIX A OUTCOME BASED MANAGEMENT AND BUDGETING HSA PROGRAMS Economic Self-Sufficiency Family Strength Community Capacity Building California Work Opportunity and Responsibility to Kids (Cal- Works) CalLearn General Assistance Work First Refugee Cash Assistance Workforce Investment Act Workforce Development Employment Services Youth Services Career Advancement through Supported Hours (CASH) Vocational Rehabilitation Work Center Independent Living Program Moving to Work Family Self-Sufficiency Program Family Loan program Homeless Job Link Benefits Assistance Advocacy Individual Self Self Sufficiency Planning (ISSP) Supplemental Security Income/ State Supplemental Payments (SSI/SSP) Cash Assistance Programs for Immigrants (CAPI) Alcohol & Other Drug Treatment Child Welfare Case Management Emergency Response Family Maintenance Family Reunification Long Term Foster Care Adoptions Foster Family Agency (FFA) Emergency Shelter Care Family Self-Self Sufficiency Team (FFST) Program /Strategy Behavioral Health Services Linkages Out-of-Home Placement Payment Rental Assistance Vouchers Transitional/Emergency Shelter Transitional Housing for Adolescents-Foster Youth Housing Opportunities for People with HIV/AIDS Second Harvest Food Bank Medi-Cal Food Stamps Child Care Transportation Assistance Moving to Work Family Resource Centers Social Capital Alcohol and Other Drugs Prevention Foster Adoptive Home Recruitment Family Resource Centers Core Service Information and Referral Workforce Development: Employer Services Work Force Investment Board Peninsula Works Supportive Housing for families and people with special needs Individual Development Account (IDA) Physical Capital Housing Rehabilitation Public Housing Community Development Block Grant (CDBG) Programs Home Ownership Program Home Ownership Program Affordable Housing/Home Grant Transportation Transitional Housing- Emancipated Youth 184

9 APPENDIX B Steering Committee Executive Team AOD Representative Financial Managers Economic Self Sufficiency for Individuals & Families Team Project Manager Admin. Assistant Research Consultant Data Consultant Financial Manager *Program Managers *Program Staff *Contract providers Project Director Supportive Services for Families & Children Team Project Manager Admin. Assistant Research Consultant Data Consultant Financial Manager *Program Managers *Program Staff *Contract providers Community Capacity Building Team Project Manager Admin. Assistant Research Consultant Data Consultant Financial Manager *Program Managers *Program Staff *Contract providers Cross Program Implementation Supports Human Services Manager I Program Support Information Technology Cost Accounting 185