Monitoring and Evaluation of Budget Performance CPA John Kauta Partner, Ariska Associates March 31, 2017
Outline Introduction Monitoring Budget Performance Evaluating Budget Performance Conclusion 2
Introduction
Definitions A government budget: shows government's proposed revenues and spending for a financial year that is often passed by the legislature, approved by the Head of State/President and presented by the Finance Minister to the nation. Could be a balanced budget (when government revenue and expenditure are equal), surplus budget (when anticipated revenues exceed expenditure) or a deficit budget (when anticipated expenditure is greater than revenues). Budget Performance: is measured to ascertain whether public entity or activity is achieving its objectives and if progress is being made toward attaining policy or organizational goals. is a quantifiable expression of the amount, cost, or result of activities that indicate how much, how well, and at what level, products or services are provided to clients/customers/the public during a given time period. 4
Why do we Measure Budget Performance? Promotes credibility and public confidence by reporting on the results of programs Helps formulate and justify budget requests Provides crucial information about public sector performance Provides a view over time on the status of a project, program, or policy Permits managers of public resources to identify and take action to correct weaknesses Focuses attention on achieving outcomes important to the public entity and its stakeholders Provides timely and frequent information to government and the public 5
Good budget performance measures Understandable - are clear, concise, and easy for a non-specialist to comprehend. Timely - have information available frequently enough to have value in making decisions. Comparable - have enough data to tell if performance is getting better, worse or staying about the same. Reliable - have data that is verifiable, free from bias, and an accurate representation of what it is intended to be. Cost effective - justify the time and effort to collect, record, display, and analyze the data given the measure s value. Another aspect of costeffectiveness is feasibility. Useful - help people doing the work understand what is happening with 6 their business process, and how to get better results for customers.
The PFM Cycle Planning /Policy Scrutiny and Audit Optimum utilization Budgeting Budget implementation. Accounting, recording and reporting 7
Key areas of Budget Performance that are Measured? Budget Realism: Is the budget realistic, and implemented as intended in a predictable manner? Comprehensive, Policy-based, budget: Does the budget capture all relevant fiscal transactions, and is the process, giving regard to government policy? Six core objectives of PFM system Accountability and Transparency : Are effective external financial accountability and transparency arrangements in place? Control : Is effective control and stewardship exercised in the use of public funds? Comprehensive fiscal risk oversight : Is oversight of fiscal risk arising from public enterprises and sub-national governments adequate? Information: Is adequate fiscal, revenue and expenditure information produced and disseminated to meet decision-making and management purposes? 8
Comparison between Monitoring and Evaluation Monitoring Evaluation Focus Continuous. Clarifies program objectives Periodic. Analyzes why intended results were or were not achieved Links activities and their resources to objectives Translates objectives into performance indicators and set targets Routinely collects data on these indicators, compares actual results with targets Reports progress to managers and alerts them to problems Time Focus Present Past - Future Main Question What needs to happen now to reach our goal? Attention Level Details Big Picture Assesses specific causal contributions of activities to results Examines implementation process Explores unintended results Provides lessons, highlights significant accomplishment or program potential, and offers recommendations for improvement Have we achieved our goal? 9
Monitoring Budget Performance
Definition of Monitoring Is a continuous process of collecting and analyzing information to compare how well a project, program or policy is performing against expected results Is a planned/systematic process of observation that closely follows a course of activities and compares what is happening with what is expected to happen Is the periodic collection and review of information on programme implementation, coverage and use for comparison with implementation plans. Allows for modifying original plans during implementation Identifies shortcomings before it is too late. Provides elements of analysis as to why progress fell short of 11 expectations
Types of Monitoring 1. Traditional/Implementation monitoring: This involves tracking inputs, activities (what actually took place) and outputs (the products or services produced) This approach focuses on monitoring how well a project, program or policy is being implemented Often used to assess compliance with work plans and budget 2. Results-based monitoring: the regular collection of information on how effectively is performing demonstrates whether a project, program, or policy is achieving its stated goals Focus is on outcomes and Impact 12
The results chain under Program Based Budgeting Inputs (Resources) Activities (Processes) Outputs Intermediate Outcomes High-level (Ultimate) Outcomes INPUTS are used in carrying out ACTIVITIES in order to produce OUTPUTS and thereby achieving OUTCOMES. 13
14
Example of Budget Performance Indicators Type of Indicator Definition Example Outcome - Effectiveness Indicator Degree to which the program objective is being met Increase in immunisation coverage Increased life expectancy % decrease in infant mortality Increased deliveries in health facilities Output Quantity Indicator Quantity of service provided No. of children immunised No. of births at health facilities Output - Equity Indicator Participation of target group Proportion of girls accessing reproductive health services % of people from poor households accessing the health facilities Output Quality Indicator Quality of the service provided Client satisfaction Proportion of health facilities accredited Output Efficiency Indicator Cost per unit of output Cost per vaccination Cost per medical doctor educated Activity Indicator Indicator of internal work processes Number of positions filled No of policy statements developed Input Indicator Measure of resources employed Number of doctors per patient 15
Evaluating Budget Performance
Definition of Evaluation A process that assesses an achievement against preset criteria. A systematic process to determine the extent to which service needs and results have been or are being achieved and analyse the reasons for any discrepancy. Attempts to measure service s relevance, efficiency and effectiveness. It measures whether and to what extent the program s inputs and services are improving the quality of people s lives. 17
Types of Evaluations Retrospective Evaluation Often used when programs have been functioning for some time. Determines what actually happened (and why) Prospective Evaluations Conducted when a new program within a service is being introduced. Identifies ways to increase the impact of a program on clients; It identifies how to improve delivery mechanisms in order to be more effective. Determines what ought to happen (and why) 18
Evaluation Matrix 19
Conducting Evaluations Ongoing program evaluation End of program evaluation Impact evaluation Spot check evaluation Desk evaluation Internal evaluation (self evaluation) - in which people within a program sponsor, conduct and control the evaluation. External evaluation - in which someone from beyond the program acts as the evaluator and controls the evaluation. 20
Qualitative Evaluation Tools Unobtrusive seeing - involving an observer who is not seen by those who are observed; Participant observation - involving an observer who does not take part in an activity but is seen by the activity s participants. Interviewing - involving a more active role for the evaluator because she /he poses questions to the respondent, usually on a one-on-one basis Group-based data collection processes such as focus groups Content analysis - which involves reviewing documents and transcripts to identify patterns within the material 21
Evaluation Tools Surveys/questionnaires Registries Activity logs Administrative records Registration forms Case studies Attendance sheets Cost benefit analysis Cause-effect diagram 22
Conclusion
24 Conclusion Developing effective budget monitoring and evaluation systems requires welldefined formulation and implementation strategies for setting up performance indicators These strategies vary depending on a country s priority for measuring results and on the scope and pace of its performance management reform objectives Some countries have followed an incremental method for developing performance indicators, (for example, Canada, the United Kingdom, and Colombia), while others have taken big bang approach (for example, Mexico and the Republic of Korea) Either way there is need to continuously improve the quality and meaningfulness of the indicators to inform government processes Its important to note that Prospective evaluations can produce monitoring strategies. Also Monitoring strategies can inform retrospective evaluations.
Thank you