IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA and MULT-16540) ON A CREDIT IN THE AMOUNT OF SDR MILLION (US$95.11 MILLION EQUIVALENT)

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR Public Disclosure Authorized IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA and MULT-16540) ON A CREDIT IN THE AMOUNT OF SDR MILLION (US$95.11 MILLION EQUIVALENT) AND A Public Disclosure Authorized GRANT IN THE AMOUNT OF US$1.585 MILLION TO THE UNITED REPUBLIC OF TANZANIA FOR A BASIC HEALTH SERVICES PROJECT (BHSP) Public Disclosure Authorized April 21, 2017 Health, Nutrition and Population Global Practice Africa Region

2 CURRENCY EQUIVALENTS (Exchange Rate Effective February 2017) Currency Unit = Tanzania Shilling (TShs) TShs 1.00 = US$ US$ 1.00 = SDR0.725 FISCAL YEAR July 1 June 30 ABBREVIATIONS AND ACRONYMS AF ANC ASC BFC BHSP BRN CAG CAS CBG CDH CCHPs CHF CHMT CHSB DbyD DHIS DHS DLI DMO DPs FM GDP GFATM GOT GP HBF HBS HCWM HFGC HIV/AIDS HMIS HRH Additional Financing Antenatal Care Audit Sub-Committee Basket Financing Committee Basic Health Services Project Big Results Now Controller and Auditor General Country Assistance Strategy Capacity Building Grant Council District Hospital Comprehensive Council Health Plans Community Health Fund Council Health Management Team Community Health Service Board Decentralization by Devolution District Health Information System Demographic and Health Survey Disbursement Linked Indicator District Medical Officer Development Partners Financial Management Gross Domestic Product Global Fund for AIDS, Tuberculosis and Malaria Government of Tanzania Global Practice Health Basket Fund Household Budget Survey Health Care Waste Management Health Facility Governing Committee Human Immuno - Deficiency Virus/Acquired Immune Deficiency Syndrome Health Management Information System Human Resources for Health i

3 HRITF HSDG HSDP HSSP ICR IDA IFR IO IMR ISR JAHSR LGAs LGDG LGSP M&E MDG MESI MMR MNCH MOF MOHSW MOU MRP MSD MTR NHIF OC ODI OIG OPD P4P PAD PBF PDO PE PFM PforR PHC PMO-RALG PPA PPRA QER RBF RHMT RMNCH Health Resources Innovation Trust Fund Health Sector Development Grant Health Sector Development Project Health Sector Strategic Plan Implementation Completion and Results International Development Association Interim Unaudited Financial Report Intermediate Outcome Infant Mortality Rate Implementation Status and Results Joint Annual Health Sector Review Local Government Authorities Local Government Development Grant Local Government Support Project Monitoring and Evaluation Millennium Development Goal Monitoring and Evaluation Strengthening Initiative Maternal Mortality Ratio Maternal Neonatal and Child Health Ministry of Finance Ministry of Health and Social Welfare Memorandum of Understanding Maximum Retail Price Medical Stores Department Mid Term Review National Health Insurance Fund Other Charges Overseas Development Institute Office of the Inspector General Outpatient Department Pay for Performance Project Appraisal Document Performance Based Financing Project Development Objective Personnel Emolument Public Financial Management Program for Results Primary Health Care Prime Minister s Office Regional Administration and Local Government Public Procurement Act Public Procurement Regulatory Authority Quality Enhancement Review Results Based Financing Regional Health Management Team Reproductive, Maternal, Neonatal and Child Health ii

4 SDG SDI SDR/XDR SIM SOE SPHCRP SWAp TB TF TFDA TFR TTL TShs TWG U5MR VAH UNFPA UNICEF US$ Service Delivery Grant Service Delivery Indicator Special Drawing Rights Sector Investment and Maintenance Loan Statement of Expenditure Strengthening Primary Health Care for Results Program Sector Wide Approach Tuberculosis Trust Fund Tanzania Food and Drug Authority Total Fertility Rate Task Team Leader Tanzanian Shillings Technical Working Group Under-five Mortality Rate Voluntary Agency Hospital United Nations Population Fund United Nations Children's Fund United States Dollar Senior Global Practice Director: Timothy Grant Evans Country Director: Bella Bird Practice Manager: Magnus Lindelow Project Team Leader: Gayle Martin ICR Team Leader: Gayle Martin ICR Author: Iffat Mahmud iii

5 COUNTRY Tanzania Basic Health Services Project CONTENTS Datasheet... v 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Summary of Borrower's ICR Annex 6. Comments of Co-financiers and Other Partners/Stakeholders Annex 7. Disbursement mechanism and allocation conditions of HBF Annex 8. Issues with results framework indicators Issues with the PDO indicators Issues with data reported in the ISRs Lack of quality data: Annex 9. (A) Systemic Deficiencies of MSD and (B) Steps initiated by MSD to address stock-outs Annex 10. Achievement of indicators against target Annex 11: Data used for the results chain Annex 12. BHSP s fund flow arrangement Annex 13. List of Supporting Documents and People Consulted iv

6 Datasheet A. Basic Information Country: Tanzania Project Name: Project ID: P L/C/TF Number(s): IDA ICR Date: 04/21/2017 ICR Type: Core ICR Lending Instrument: SIM Borrower: Original Total Commitment: Revised Amount: XDR 63.10M Environmental Category: B Implementing Agencies: Ministry of Health and Social Welfare Project Coordinator Regional Administration and Local Government Cofinanciers and Other External Partners: UNICEF Canada High Commission UNFPA KFW Entiwicklungsbank DANIDA Irish Aid Swiss Development Cooperation B. Key Dates Tanzania - Basic Health Services Project GOVERNMENT OF TANZANIA XDR 63.10M Disbursed Amount: XDR 63.08M Process Date Process Original Date Revised / Actual Date(s) Concept Review: 03/08/2011 Effectiveness: 03/02/2012 Appraisal: 06/27/2011 Restructuring(s): 05/13/ /29/2016 Approval: 12/20/2011 Mid-term Review: 09/30/ /30/2014 Closing: 06/30/ /30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Moderately Unsatisfactory Moderate Unsatisfactory v

7 Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory Quality of Supervision: Unsatisfactory Implementing Moderately Agency/Agencies: Unsatisfactory Overall Bank Overall Borrower Moderately Unsatisfactory Performance: Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project No at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes Moderately Unsatisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Major Sector/Sector Public Administration Public administration - Health 3 3 Sub-National Government (Historic)Health and other social services Health Major Theme/Theme/Sub Theme Human Development and Gender Health Systems and Policies Health System Strengthening Public Sector Management Public Administration Municipal Institution Building E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili vi

8 Country Director: Bella Deborah Mary Bird Mercy Miyang Tembon Practice Manager/Manager: Magnus Lindelow Jean J. De St Antoine Project Team Leader: Gayle Martin Dominic S. Haazen ICR Team Leader: ICR Primary Author: Gayle Martin Iffat Mahmud F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) Assisting the Government of the United Republic of Tanzania in improving the equity of geographic access and use of basic health services across districts and enhancing the quality of health services being delivered. Revised Project Development Objectives (as approved by original approving authority) Not applicable (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Births attended in health facility broken down by LGA (percent) Actual Value Achieved at Completion or Target Years Value quantitative or Qualitative) 58.4% 69% 62.4% Date achieved 12/31/ /30/ /31/2015 Comments (incl. % achievement) Indicator 2 : 38% -- target not achieved Average outpatient attendances per clinical staff broken down by LGA (number) Value quantitative or Qualitative) 1,395 1,600 2,259 Date achieved 12/31/ /30/ /31/ % -- target surpassed. It is, however, not clear if the increase in Comments average outpatient attendances increased efficiency of human resources or (incl. % achievement) it increased the workload of staff. vii

9 Indicator 3 : Average outpatient attendances per capita broken down by LGA (number) Value quantitative or Qualitative) Date achieved 12/31/ /30/ /31/2015 Comments (incl. % Target not achieved; actual value less than baseline which indicates that the number of outpatient attendances per person has decreased. achievement) Health facilities with any stockouts of tracer medicines and vaccines Indicator 4 : according to CCHP (percent) Value quantitative or Qualitative) 28% 5% 53% Date achieved 12/31/ /30/ /31/2015 Comments (incl. % Target not achieved. Stock-out level at end of the project was higher than that at the baseline achievement) Ratio of the 10 best performing LGAs to the 10 worst performing LGAs Indicator 5 : in indicators 1 through 3 (ratio) Value quantitative or Qualitative) 1: 8.1 2: : % decrease 1: 4.1 2:15.4 (figure of 2014) 3: 8.1 Date achieved 12/31/ /30/ /31/2015 1: 329% -- target surpassed 2: target not achieved as the ratio did not Comments decrease. 3: target not achieved. The ratio may decrease due to the worst (incl. % achievement) performing LGAs doing better or best performing LGAs doing worse (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Children immunized (number) - core indicator Formally Revised Target Values Actual Value Achieved at Completion or Target Years Value (quantitative or Qualitative) 0 6,142,347 8,016,240 Date achieved 12/31/ /30/ /15/2016 Comments (incl. % 131% -- target surpassed achievement) viii

10 Indicator 2 : Children receiving a dose of Vitamin A (number) - core indicator Value (quantitative or Qualitative) 0 28,978,389 29,730,482 Date achieved 12/31/ /30/ /15/2016 Comments (incl. % achievement) Indicator 3 : Value (quantitative or Qualitative) 103% -- target surpassed Direct project beneficiaries (of which female) (number and percent) - core indicator 0 37,118, % 48,605, % Date achieved 12/31/ /30/ /15/2016 Comments 131% and 102% respectively -- target surpassed (incl. % achievement) Pregnant women receiving antenatal care during a visit to a health Indicator 4 : provider (number) - core indicator Value (quantitative or Qualitative) 0 6,471,889 10,884,475 Date achieved 12/31/ /30/ /15/2016 Comments (incl. % achievement) Indicator 5 : Value (quantitative or Qualitative) 168% -- target surpassed Health centers in each LGA that provide emergency obstetrical care (EMOC) (percent) Hospitals: 64.5% Health Centers: 5.5% 20% increase Health Centers: 48% Date achieved 12/31/ /30/ /15/2016 Comments (incl. % Health centers: 873% -- target surpassed. Since the indicator measures EMoC in health centers, data for hospitals has not been included. achievement) Indicator 6 : District Readiness for Service Delivery (score) Value (quantitative or Qualitative) TBD, available December 31, % increase 55.3% Date achieved 12/31/ /30/ /31/2015 Comments (incl. % Baseline was not defined. Achievement, therefore, cannot be calculated as there is no benchmark for comparing end of project achievement. ix

11 achievement) Indicator 7 : Use of performance and equity indicators in LGA allocations and use of the approach developed under this project for service delivery grants in other sectors (degree) Value (quantitative or Qualitative) None Use of SDG approach by other sectors Formula review on equity indicators was completed and the new equity based formula was used for allocation of HBF to LGAs. 17 LGAs selected by PMO-RALG for support based upon performance and capacity criteria Date achieved 12/31/ /30/ /30/2015 Comments (incl. % SDG not established. Equity based allocation revised in 2012 and since then used only for HB achievement) Indicator 8 : LGAs with unqualified audit reports (percent) Value (quantitative or Qualitative) 32% 48% 86% Date achieved 12/31/ /30/ /30/2016 Comments (incl. % 338% -- target surpassed achievement) Indicator 9 : LGAs which have implemented the new DMIS system (percent) Value (quantitative or Qualitative) 0 70% 100% Date achieved 12/31/ /30/ /31/2015 Comments (incl. % achievement) Indicator 10 : 143% -- target surpassed Council Health Services Boards (CHSB) that meet on a quarterly basis to discuss issues related to budget, CHF, rehabilitation, schedule of visits to lower facilities, staff houses, etc. (percent) Value 60% 86% 85% x

12 (quantitative or Qualitative) Date achieved 12/31/ /30/ /31/2015 Comments (incl. % achievement) Indicator 11 : 96% -- target achieved LGAs which have implemented one or more community or social accountability mechanisms (score-card, posting financial or performance information, complaint line, enhanced community participation in health facility governance or public opinion survey) Value (quantitative or Qualitative) 0 15% 21% Date achieved 12/31/ /30/ /23/2016 Comments (incl. % 140% -- target surpassed achievement) Health personnel receiving training (number) - core indicator Indicator 12 : Value (quantitative or Qualitative) 0 25,500 32,945 Date achieved 12/31/ /30/ /31/2015 Comments (incl. % achievement) Indicator 13 : 129% -- target surpassed Regional Health Management Teams that have implemented performance monitoring and verification initiatives (percent) Value (quantitative or Qualitative) 0 35% 100% Date achieved 12/31/ /30/ /31/2015 Comments (incl. % 286% -- target surpassed achievement) xi

13 G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 04/23/2012 Satisfactory Satisfactory /17/2012 Satisfactory Satisfactory /11/2013 Satisfactory Moderately Satisfactory /29/2013 Moderately Satisfactory Moderately Satisfactory /22/2014 Moderately Satisfactory Moderately Satisfactory /29/2014 Moderately Satisfactory Moderately Satisfactory /29/2015 Moderately Satisfactory Moderately Satisfactory /31/2015 Moderately Satisfactory Moderately Satisfactory /30/2016 Unsatisfactory 10 10/28/2016 Moderately Unsatisfactory Moderately Unsatisfactory Moderately Satisfactory H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO Amount Disbursed at Restructuring in USD millions 05/13/2015 N MS MS IP Reason for Restructuring & Key Changes Made i) 10-month no-cost extension of the closing date from June 2015 to April 2016; ii) Amendment of the Financing Agreement to include provision for procurement through United Nations Agencies 04/29/2016 MS MS month no cost extension of the closing date from April 30, 2016 to October 30, 2016 xii

14 I. Disbursement Profile xiii

15 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The Project was prepared during a period when Tanzania documented good economic growth (at an average of 7 percent between 2002 and 2008) although commensurate gains in poverty reduction were not achieved (poverty incidence declined from 35.7 percent in to 28.2 percent in 2012). 2. Sector-wide approach and funding arrangement: In 1999, the Government of Tanzania (GOT) adopted a Sector Wide Approach (SWAp) in the health sector which provided the framework for collaboration among the various stakeholders including development partners (DPs), private sector, civil society, the Ministry of Health and Social Welfare (MOHSW) 1, the Prime Minister s Office Regional Administration and Local Government (PMO-RALG) 2, and the Ministry of Finance (MOF). Health services were being delivered by Local Government Authorities (LGAs) through public dispensaries, health centers and district hospitals 3. DP funds that were pooled with the Government were calculated as per capita allocations and flowed to the Health Basket Fund (HBF). DP funds additional to the HBF also financed the SWAp. 3. Political context: A key element of the reforms was decentralization, which increased LGAs responsibilities for health service delivery. The GOT s decentralization process 4 aimed to improve intergovernmental fiscal transfers, strengthen revenue generation for the LGAs, and make service delivery more efficient and equitable. The GOT s overarching policy, Decentralization by Devolution (D by D), was enacted in 1998 and shifted greater political, administrative, and financial powers to the LGAs. 4. Health outcomes and challenges: Significant progress was made in some of the health-related indicators and Tanzania was on track to achieve the health-related Millennium Development Goals (MDGs). Between 1999 and 2010, infant mortality rate was reduced by almost one half; under-five child mortality rate by 45 percent; and maternal mortality ratio by 14 percent. There was, however, little reduction in total fertility rate (TFR of 5.4 in 2010) and contraceptive prevalence rate was low at 27 percent in Despite the reduction of 14 percent between 1999 and 2010, maternal mortality ratio was high at 454 deaths per 100,000 live births in Significant variations existed in health status, access to services, and funding across regions and LGAs. There were challenges, in particular relating to the availability of skilled human resources for health (HRH), availability and distribution of medicines, public financial management (PFM) and general management at the LGAs that affected service delivery. There was not enough 1 The Ministry has since been renamed as the Ministry of Health, Community Development, Gender, Elderly and Children (MOHCDGEC). The ICR uses MOHSW. 2 It has been renamed as the President s Office Regional Administration and Local Government (PO-RALG). In the ICR, PMO-RALG is used. 3 Zonal hospitals, specialized hospitals and national hospitals were under the remit of MOHSW. Regional Referral Hospitals as well as the dispensaries, health centers and district hospitals were under the PMO-RALG 4 Guided by the Local Government Reform Act 1982, second Poverty Reduction Support Program, and 1998 Policy Paper on Local Government Reform 1

16 funding from the General Budget Support for service delivery (World Bank Report TZ). 5. Financing sources for the LGAs: The LGAs were financed mainly by the GOT and the DPs through block grants and HBF 5. Health Block Grants were allocated between personnel emoluments (PE, i.e. salaries), other charges (OC) and development. The HBF mainly supported OC, although it had provisions for supporting renovation/construction of facilities 6. The HBF did not pay for salaries. Disbursement mechanisms for the Health Block Grants and HBF followed government procedures (details provided in Annex 7). 6. Rationale for Bank involvement: During preparation, the perception was that there was inadequate engagement between DPs and PMO-RALG/LGAs. The GOT and DPs wanted to use the Local Government Development Grant (LGDG) system as the main vehicle for intergovernmental fiscal transfers. Between 2000 and 2010, the HBF represented 15.8 percent of the overall health sector and the World Bank provided almost one fifth of the HBF financing during the period. These resources were a majority of the discretionary funding available for the health sector as most of the GOT funds were earmarked for salaries and various vertical programs. It was widely recognized that the SWAp arrangement in general, and the HBF in particular, were effective and resulted in improvements in health outcomes as well as quality of health services at the community level. The Bank had a comparative advantage in: (i) the development and management of the HBF; (ii) providing fiduciary oversight based on which the DPs and GOT requested the Bank to provide procurement oversight for the international tenders financed by the HBF; (iii) strong technical expertise in the areas of PFM and capacity building as well as tailoring cost-effective innovative approaches; and (iv) the Bank s experience in supporting the two phases of Health Sector Development Project (HSDP). It was recognized that the SWAp worked well: donor harmonization and alignment to GOT priorities; provision of non-salary discretionary funding through the HBF assisted in attaining results and made service delivery more effective; strengthening of joint reviews and inter-ministerial coordination; greater sectoral coherence and consistency as well as reduction of transactions costs. 7. Links to strategies: The project aligned fully with: (i) the Country Assistance Strategy (CAS) and contributed to its third objectives which included outcomes relating to improvements in access to and quality of health services; (ii) the World Bank s Africa Strategy Pillar 2 Vulnerability and Resilience and Pillar 3 Governance and Public- Sector Capacity; and (iii) the GOT s priorities as outlined in the Third Health Sector Strategic Plan (HSSP III). 1.2 Original Project Development Objectives (PDO) and Key Indicators 5 Complementary sources such as user fees, Community Health Fund (CHF), National Health Insurance Fund (NHIF) and Tiba Kwa Kadi (TIKA, urban form of CHF) were also available for spending by the facilities. 6 Funds for rehabilitation/construction were provided mainly using Local Government Development Grant (LGDG) mechanism 2

17 8. The PDO for this project is to assist the Government of the United Republic of Tanzania in improving the equity of geographic access and use of basic health services across districts and enhancing the quality of health services being delivered. This would be achieved by introducing innovative financing mechanisms for health service delivery which encourage both effective and efficient management of health services at the local level and a focus on quality improvement, and would be accomplished within the framework of HSSP III. 9. In doing so, the project would seek to improve the performance of districts...by changing the way in which HBF are allocated to districts. This would include moving from the current unconditional grant approach.to one which also considers both performance and equity aspects at the LGA level. Once done, the project would then look at transforming the HBF into a more generalized Service Delivery Grant (SDG) mechanism. The aim was to integrate all basket funds into a multi-sectoral SDG mechanism. 10. The following indicators were listed in the PAD for measuring improvements in accessibility, use and performance: i. Births taking place in a health facility (percent, national and broken down by LGA access to and use of health services); ii. Average outpatient attendances per clinical health worker by LGA (number, national and broken down by LGA efficiency in delivery of health services); iii. Average outpatient attendances per capita by LGA (number, national and broken down by LGA access to and use of health services); iv. Health facilities with any stock-outs of tracer medicines and vaccines (percent, national and broken down by LGA quality of health services delivered); v. Ratio of the ten best performing LGA s to the ten worst performing LGA s in indicators (i) through (iii) (variation in access, use and quality of health services). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification none 1.4 Main Beneficiaries 11. All Tanzanians, using publicly financed health services, were identified as beneficiaries. With the Bank s contribution to the HBF expected to be around 20 percent of the total HBF, it was estimated that the project would benefit 8 9 million people per year. Activities aimed at strengthening capacity of LGAs, and at transforming the HBF to an SDG mechanism would benefit the MOHSW, PMO-RALG, LGAs as well as other Ministries using basket funding mechanism. 1.5 Original Components 12. Component 1 Support to Local Government Service Delivery (US$80 million) disbursing to the HBF and used for service delivery at the district-level using an average allocation to the LGA s of approximately US$0.30/capita per year. 3

18 13. Component 2 Capacity Building in Local Governments (US$11 million) through the Capacity Building Grant (CBG) 7 mechanism of LGDG to support councils with Capacity Building Plan for improving the management of health services. The grant would finance technical assistance, training and systems strengthening interventions, to help improve overall performance at the LGA and health facility levels. 14. Component 3 Central Programs to Support Local Service Delivery (US$9 million). Provided funding through the HBF (US$4 million) and non-pooled funding (US$5 million) for central level interventions. 15. In summary, component 1 used pooled modality disbursing to the HBF 8 (district health services) which financed a slice of the Comprehensive Council Health Plans (CCHPs) 9. Each year, the LGAs prepared their individual CCHPs based on the budgets made available to them, program priorities and planned activities. Component 2 used the CBG mechanism and disbursed to the PMO-RALG in parallel for capacity building of LGAs. While component 3 used a combination of pooled and parallel mechanism disbursing to the HBF (central level) for use by PMO-RALG and the MOHSW and also disbursing to the MOHSW in parallel for project management and other non-pooled activities. From the HBF, a portion of the funds was provided to the Medical Stores Department (MSD) for procurement of medicines which were then supplied to the districts and health facilities. The proportion of HBF to be allocated between the central level activities and the district health services were ascertained each year and documented in the annual Side Agreements 10. Annex 12 depicts the project s fund flow arrangement. 1.6 Revised Components none. 1.7 Other significant changes 16. The following two changes were made through a Level-Two Restructuring of the project in May 2015, approved by the Country Director: i. Project closing date extended twice: first, a 10-month no-cost extension of the project closing date from June 30, 2015 to April 30, 2016; and second, a 6-month no-cost extension of the closing date from April 30 to October 30, These extensions were processed to provide time to MOHSW to complete some of the planned health care waste management and procurement activities under the non-pooled portion. ii. Procurement arrangement modified: Inclusion of a clause to allow procurement through United Nations agencies. This was done to allow the United Nations Office 7 The PAD for BHSP Report No TZ states CBG is a new mechanism and it was expected that the use and distribution of funds between CBG and centrally managed mechanisms would be reviewed and revised during project implementation. 8 The DPs contributed their funds to an account maintained at the MOF. 9 Main sources of funds for CCHPs were GOT block grant, HBF, Health Sector Development Grant (HSD), in-kind receipts, council own resources, LGDG, cost sharing, the Global Fund for AIDS Tuberculosis and Malaria (GFATM), community contribution, and others 10 The HBF was guided by a Memorandum of Understanding (MOU) between the GOT and the contributing DPs as well as annual Side Agreements that defined the priorities for the following year and level of contribution of the DPs. Details provided in section 2.3 4

19 for Project Services (UNOPS) to procure vehicles for the MOHSW using project funds Additionally, the following changes were made: iii. Disbursement arrangement modified: in 2013 expenditure reporting format was changed from Interim Financial Report (IFR) for all components to a combination of reports based on Statement of Expenditure (SOE) for the non-pooled portions and IFR for the HBF. Also, a fourth designated account 12 was added for the project for channeling funds relating to component 3. iv. Reallocation of funds between components: Due to losses from exchange rate fluctuation between Special Drawing Rights (SDR) and United States Dollar (US$), funds were re-allocated between components as shown in table 1. This reallocation did not require a restructuring as the allocation by disbursement categories was unchanged. Table 1: Project s allocation, by component in US$ million Component Original allocation as per PAD Revised allocation in 2014 Revised allocation in 2015 Actual expenditure at end of Project 1: Basket Fund : LGA Capacity Building : Central-level Capacity Building Project Management Exchange rate loss 4.9 Total v. Proposed Additional Financing (AF) of US$40 million not processed. An AF of US$25 million IDA credit 13 and US$15 million from the Multi-Donor Trust Fund for Health Results Innovation (HRITF) was proposed. This would have supported a results-based financing (RBF) component and community-based nutrition intervention. The AF, however, did not proceed as the RBF interventions were included in the follow-on operation (Strengthening Primary Health Care for Results Program, SPHCRP) which is using Program for Results (PforR) as its instrument. Nutrition is a priority under the PforR and one of the co-financiers is Power of Nutrition (US$20 million). vi. Co-financing added: in 2014/15, US$1.585 million from HRITF was added to the project for preparation of the proposed AF. Although the AF did not proceed, the 11 GOT policy requires procurement of vehicles to be done by government entity. However, the procuring capacity of the government entity that does vehicle procurement has not been assessed by the Bank. Hence, it could not procure for Bank-financed projects. As there is a provision in the Procurement Act that procurement guidelines of international agencies will prevail for donor-funded projects, UNOPS was chosen to do the procurement for MOHSW using Bank procurement guidelines. 12 Four designated accounts A) for HBF holding account operated by the MOF associated with disbursement category 1, financing for components 1 and 3; B) for CBG operated by PMO-RALG associated with disbursement category 2, financing for component 2; C) for MOHSW associated with disbursement category 3, financing for components 2 and 3 non-pooled/non-cbg portions; and D) for PMO-RALG associated with disbursement category 3, financing for components 2 and 3 non-pooled/non-cbg portions. 13 Planned Board date was second quarter of FY2015 5

20 HRITF supported the MOHSW for developing RBF manual and implementation guidelines and implementing RBF activities in Pwani region and in Kishapu district of Shinyanga region, as explained in section Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 18. The project was prepared in 9.6 months 14, which is much less than the average time taken in the Africa region of 14 months and in Tanzania of 18 months. The team included a public health specialist, a public sector specialist and fiduciary experts. While the team composition was pretty standard, the team would have benefitted from the expertise of an Economist/ monitoring and evaluation (M&E) expert for formulating an appropriate set of indicators for the results framework. Comments were obtained from a Quality Enhancement Review panel of the World Bank on the design of the project prior to the appraisal. The comments, however, could not be retrieved during the time of the Implementation Completion and Results report (ICR). The preparation team consulted with the GOT and DPs during the design phase. 19. Design of component 1 (i.e. contribution to the pool) was informed by the successful implementation of the SWAp as well-documented in the Project Appraisal Document (PAD). The rationale for having the non-pooled portions was credible to ensure availability of technical assistance to support the project, which could not be channeled through the HBF. There was no broad mechanism to address the LGA capacity issues. The CBG mechanism, therefore, was the appropriate choice as it was already operational and the PMO-RALG was accustomed with the functionality of the mechanism. Central-level capacity needed to be strengthened and the project allocated funds for it through component 3. The SDG was conceptualized based on interest in all sectors to integrate the various baskets into one platform, as the transactions costs of the ongoing arrangements were high. 20. At appraisal, there was an ongoing IDA-financed Accountability, Transparency and Integrity Project that was addressing PFM issues under the GOT s PFM Reform Program. This intervention could have been used to strengthen the capacity of the LGAs. The Bank s preparation team, however, felt that the capacity building component should be housed in health as the sector needed this initiative and a cross-sectoral PFM program would have not realized the intended objective. The CBG mechanism was set-up as part of the Local Government Support Project (LGSP), a separate project led by Bank s Governance Global Practice (GP) which channeled funds through the LGDG 15. The health sector was added to the LGDG mechanism in The implementation arrangements for components 2 and 3 were not adequately thought through prior to project effectiveness. The lack of readiness is 14 Project prepared between March and December 2011 Concept Review meeting was held on March 8, 2011; Quality Enhancement Review on May 18, 2011; Decision meeting on June 21, 2011; Appraisal completed on June 27, 2011; Negotiations conducted on November 8, 2011; and Board approval obtained on December 20,

21 evident from the need to change the reporting mechanism from IFR based reporting to a combination of SOE and IFR. At appraisal, the GOT s financial management (FM) capacity should have been adequately assessed to ensure that a suitable disbursement mechanism was instituted rather than changing it six months into implementation. 22. The clear and compelling rationale for Bank involvement was adequately reflected in the PAD. It stated that a 2008 Lancet article attributed a large proportion of the child survival gains in Tanzania to the investments in health systems and scale up of decentralized interventions. The PAD also noted the SWAp has contributed to improvements in the health outcome and to improvements in the quality of health services at the community level. 23. Although the PDO is aligned to the GOT s priorities to a large extent, it is rather complex and overambitious trying to cover aspects (e.g. quality of services) that are difficult to measure, especially given the weak health management information system (HMIS) and district health information system (DHIS). During appraisal, it was expected that the HMIS would be strengthened rapidly through the Monitoring and Evaluation Strengthening Initiative (MESI) supported by the Governments of Netherlands and the United States as well as the Global Fund for AIDS, Tuberculosis and Malaria (GFATM). 24. To what extent background analyses were done for the project is not clear, apart from the evidence on the success of the HBF modality which is well documented. While it could be argued the design of CBG mechanism was done based on informed choices made under the Bank-financed LGSP and aligning with the GOT s D by D policy 16, the aim of the project to establish an SDG mechanism was premature. The SDG mechanism was conceived as an innovative financing mechanism; however, there were no GOT policy to back it up nor specific plan in the project to facilitate the integration of the various baskets. 25. The project built on the lessons learned from implementation of the two phases of HSDP. One of the lessons learnt is that the SWAp arrangement has clear advantages donor harmonization, alignment with national priorities, etc. However, some of the other successes of HSDPII were not continued with and the reason(s) for it are not apparent. For example, HSDP used a set of indicators used by the GOT but BHSP selected a stand-alone set of indicators, which was problematic as explained in section 2.3. Also, HSDP relied on surveys to triangulate data as the HMIS weaknesses were known; a practice that was not continued with under the project, possibly because efforts were being put in to strengthen the HMIS. 15 The Capacity Development Grant (CDG) and CBG were collectively referred to as the LGDG Core. New sector windows (LGDG non-core) were added over time as the GOT rationalized and decentralized funding for development needs in particular sectors where LGAs have core responsibilities. The LGDG included LGDG Core established in 2004 and LGDG Non-Core added between 2007 and 2009 and included: (i) Agriculture Sector Development Grant; (ii) Water Sector Development Grant; (iii) Health Sector Development Grant (HSDG) set up in 2009; and (iv) Urban Performance Grant. 16 As explained in section 1.1, D by D was enacted in 1998 which shifted greater political, administrative, and financial powers to the LGAs. Accordingly, the LGAs responsibilities for service delivery in education, health, agriculture, and infrastructure also increased. 7

22 26. An internal decision meeting of the World Bank that authorized the team to proceed with appraisal of the project carefully considered whether the risks had been appropriately assessed and addressed. Key risks were identified except for a few: (i) the risk of HMIS not providing routine and reliable data; (ii) turnover of staff at the LGA level; (iii) weak capacity of the regional and council health management teams; and (iv) unsatisfactory procurement practices as indicated in the FM assessment but not included in the Operational Risk Assessment Framework (ORAF) of the PAD. 2.2 Implementation 27. Although the broad concept of the project was well-intended, implementation arrangements were complex caused by a lack of readiness and realism of the project further compounded by the over-ambitious targets it set for itself. The three components followed different kinds of implementation modality and disbursement arrangements. The design of and expectations from the project were not intuitive as there was no operational manual, which is why the implementation team struggled as the MOHSW team that was present during preparation was changed right after project approval. The following paragraphs assesses implementation in two parts 17 : i) the pool, i.e., the HBF which used SWAp modality; and ii) the non-pooled portions. 28. Implementation of the HBF (i.e. the pooled portion). There was overall good coordination and harmonization between the HBF partners, which contributed to strengthened implementation, as reflected through the extensive and regular use of the joint monitoring platforms including the Joint Annual Health Sector Review (JAHSR). This reduced the burden on the GOT agencies while avoiding duplicative reporting system, saving time for the implementing agencies, and aligning DP support to the national priorities and needs. As is typical with a SWAp, the HBF used country systems and institutional settings for implementation. This meant that implementation started as soon the project was made effective without any delay in setting up the building blocks. Moreover, the HBF was fully aligned with the HSSP III and, therefore, was fully owned by the GOT. This ensured smooth implementation of the project activities. 29. One of the major challenges was stock-out of essential/tracer medicines in public health facilities across the country. During the project period, stock-out levels increased substantially (from 28% of health facilities having any stock-out at appraisal to 53% at end of the project, as measured by PDO indicator # 4). Although various reform attempts and initiatives were taken in the last decade to resolve this issue, the Service Delivery Indicator (SDI) survey 2014 noted a decrease in the availability of tracer medicines (World Bank Group and African Economic Research Consortium, 2014). In 2014, on average 60 percent of the health facilities had tracer medicines, compared to 76 percent in Less than a tenth of the facilities (8.4 percent) had all 14 tracer medicines available in 2014 and virtually no rural public facility (1.2 percent) had all the tracer medicines in 17 During the first year (FY2011/12) of implementation, the project disbursed US$10 million of IDA credit to the HBF for procurement of medicines by MSD. It did not finance any other activities in FY2011/12. By June 2015 (FY2014/15), 99 percent of IDA credit was disbursed. In FY2015/16, the project proceeds were, therefore, used for completing activities that were initiated earlier mainly under component 3. For components 1 and 2, in FY2015/16 IDA support was channeled through the follow-on intervention (SPHCRP). 8

23 stock and unexpired during the same period. The MSD, which is responsible for bulk procurement of medicines, has initiated some steps to address this issue, as detailed out in Annex There were several other factors that adversely affected implementation of the HBF, some of which were beyond the purview of the health sector. One, decentralization in the health sector did not fully materialize during project implementation period. Health facilities had limited financial autonomy to utilize their own funds. Until recently, most primary health care (PHC) facilities did not have a bank account 18. Funding for PHC was channeled to LGAs, which often served as a major bottleneck preventing resources to reach lower levels. Also at that time, the PMO-RALG did not have the technical expertise required for the health sector. 31. Two, accountability for results was low at all levels, especially between (a) the central government and LGAs; (b) the LGAs and the facilities; and (c) the facilities and the communities. More than half of the health workers were either absent or late during work hours (as noted in the SPHCRP PAD, Report No TZ). On average, 14.3 percent of health providers were absent from the facilities (World Bank Group and African Economic Research Consortium, 2014). There was a perception among community members that petty corruption existed (MOHSW, 2014). 32. Three, annual pledges for the HBF were finalized in September October every year, i.e., after one quarter of each fiscal year was over. There were further delays in fund release (including HBF) from the MOF to LGAs 19. As a result, implementation slowed down districts (after they received their funds) sent their requisitions for medicines to MSD and then MSD 20 consolidated requests of the districts, delaying the entire procurement process. Four, insufficient resources were available to address the burden of diseases. The 2013/14 JAHSR noted In 2011/12 HIV/AIDS, tuberculosis and malaria together account for 45 percent of Total Health Expenditure, while Reproductive and Child Health accounts for 12 percent. There is a need to ensure that there is a balance in distribution of resources across diseases while increasing the focus on non-communicable diseases. 33. Five, there were wide variations in the distribution of HRH, with shortages in some areas 49 percent of doctor positions and 46 percent of nurses were empty in 2014/15 (MOHSW and PMO-RALG, 2015). CCHP funds had to be used for strengthening HRH 21. Moreover, existing workforce was not well-distributed. Health workers density ranged from 4 to 10 per 10,000 population (MOHSW, Human Resource for Health and Social Welfare Strategic Plan ). Six, overall weak capacity of the LGAs in facility 18 All PHC facilities now have bank accounts that meet the Treasury guidelines (i.e. not simply commercial bank accounts) and this is considered a major achievement. 19 This affected all sectors, not just health 20 MSD procured medicines using HBF allocated to them, funds from other vertical programs, and GOT resources. 21 Almost half (52.5 percent) of the CCHP funds was allocated for strengthening HRH while 9.8 percent was allocated for maternal, newborn and child health; 8.6 percent for medicines, equipment and consumables; 1.9 percent for noncommunicable diseases (MOHSW and PMO-RALG, 2015). 9

24 management, PFM, and M&E presented challenges for implementation. Seven, frequent turnover of leadership of the Council Health Management Teams (CHMTs) that manage health care and social welfare services at the Council level. Eight, overall weak capacity of the Regional Health Management Teams (RHMTs) that oversaw the work of the Regional Referral Hospitals and the CHMTs. For these multiple factors, implementation support was weak and the health outcomes relating to reproductive, maternal, neonatal and child health (RMNCH) suffered as explained in section Implementation of component 2, capacity building of the LGAs. This component supported both the MOHSW and the PMO-RALG. The MOHSW coordinated with all the 167 LGAs to monitor district health services. The MOHSW with the PMO- RALG identified the low performing LGAs and built capacity of these LGA staff. PlanRep 22 was improvised and CHMT members trained on it by MOHSW. Other achievements are listed in Annex 2. In FY2013/14, funds 23 were provided to 17 LGAs (out of the total of 167) for capacity development. These LGAs were selected based on the review of the CCHPs by the PMO-RALG and MOHSW. Subsequently, it was realized that all LGAs needed CBG funds and in FY2014/15, all 167 LGAs were given CBG grants 24. Component 2 also supported the MOHSW for M&E (detailed out in section 2.3 and Annex 2). On the PMO-RALG side, initiation of project implementation was delayed due to insufficient capacity with only three people dealing with the health sector. Also, it was not clear to the implementation team what this component aimed to do. Gradually, more people were posted and a working committee was set-up for better coordination, which helped implementation. Although the PMO-RALG was conversant with the CBG mechanism, there were teething problems of the newly included health window Component 3 supported: (i) the PMO-RALG in overall project management through the central basket (i.e. the pool) by supporting the RHMTs and the PMO-RALG in monitoring activities. The non-pooled portions of component 3 supported the MOHSW in: (ii) ensuring CCHPs are adequately analyzed and monitored; and (iii) implementation of health care waste management (as described in section 2.4). 36. Two factors contributed to improving implementation of the non-pooled portions. Appointment of the Project Coordinator improved coordination and establishment of the Working Group facilitated implementation of the CBG mechanism. 37. Factors that adversely affected implementation of non-pooled portions: (i) Disagreements between the MOHSW and PMO-RALG often presented a situation where the Bank had to mediate for reaching agreements. For such disagreements, it was difficult also to reallocate funds between designated accounts. 22 PlanRep is a tool used by the MOHSW for planning and budgeting purposes, mainly for CCHP preparation 23 TShs 506 million was spent (Source: PMO-RALG) 24 TShs 3.34 billion was provided (Source: PMO-RALG) 25 Health included in LGDG in

25 (ii) (iii) There were competing demands on the time of the Project Coordinator and the project management team and they could not provide adequate supervision support. Complicated implementation arrangements for the non-pooled activities slowed down implementation progress. Moreover, having five Task Team Leaders of the World Bank over the five-year period affected continuity. 38. P4P pilot financed by HRITF. The GOT initiated a pay-for-performance (P4P) mechanism in 2008 to help accelerate improvements in maternal, infant and child mortality and morbidity. Through this initiative, health facilities and its supervisors were rewarded for attaining population coverage targets for activities related to child and maternal health. This pilot was designed with funding from Norway and was implemented in all health facilities in the Pwani region. After the completion of four cycles of incentive payment in June 2013, the funding from Norway ended. This is why US$1.58 million from HRITF was provided to the MOHSW to sustain the incentive payment in Pwani between January 2014 and June It also supported the P4P activities in 45 health facilities in Kishapu district. Progress is discussed in Annex 2. Main achievements under the Pwani pilot include increase in: institutional deliveries, coverage of antenatal care (ANC), provider kindness, patient satisfaction, staff motivation and overall efficiency. The Pwani P4P model used a fee-for-service approach and hence a lot of the funds were used for paying staff. Based on the experiences of other countries as well as the implementation challenges faced in Pwani, the P4P design has been revised and is now being implemented in Shinyanga region with support from the follow-on PforR. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 39. There are two parts to M&E: i) the monitoring arrangements of the SWAp; and ii) monitoring of the project s results framework. These are separately assessed in this section. 40. M&E design of the project: The PDO aimed to improve three aspects of service delivery equity of geographic access, utilization, and quality. Although the PAD clearly identified the indicators that will measure each of these three aspects, there were problems with defining and monitoring of these (detailed out in Annex 8). There were significant shortcomings in the architecture as well as monitoring of the results framework which set some unrealistic and aspirational targets and did not include SMART 26 indicators: (i) improbable targets set based on incorrect baseline values at appraisal (at least for two of the PDO indicators). For example, facility-based delivery was 50.2 percent in 2010 (National Bureau of Statistics of Tanzania and ICF Macro, 2011) but the project used a baseline of 58.9 percent; (ii) indicators were loosely defined for example, PDO indicator # 5 used for tracking performance of the 10 best and worst performing LGAs. The indicator aimed to decrease the ratio of the best and worst performing LGAs by 15 percent without any clear definition. A decrease in ratio can be either for best performing LGAs doing worse or worst performing LGAs doing better. Also, the 10 best and worst performing 26 SMART is a mnemonic acronym, giving criteria to guide in the setting of objectives, for example in project management. The letters S and M usually mean Specific and Measurable with the other letters having different meanings depending on the field of study. For public health, the following is usually the case: A is for Achievable, R is for Relevant, and T is for Time-bound. 11

26 LGAs were not identified so it was not possible to compare interim results; (iii) selfreported data from the Health Management Information System, which were unreliable, were used for reporting interim results with no third-party verification and/or quality audit done 27 ; and (iv) key aspects were not monitored, for example the pro-poor impacts of the project (data by wealth quintile were not tracked). 41. Some M&E design decisions of the project can be questioned. For component 1, the project used a set of indicators that was different from that used by the GOT to monitor HBF (except for one indicator which was common). The project s results framework was not able to capture all of the important aspects. Income inequity in health outcomes was an issue as per the Demographic and Health Survey (DHS) 2010 data 28 but the results framework did not track this. High level of TFR was highlighted in the PAD as one of the challenges in the sector but the results framework did not include any indicator on improving family planning. Many of the indicators used population data as the denominator. With the National Census revising the population estimates for Tanzania every year, the data provided for these indicators became unstable and un-comparable over time. 42. The Bank s ISRs consistently documented some of the challenges regarding the results framework, particularly the lack of reliable data. While processing an extension of the closing date, there was a proposal to revise the results framework as part of project restructuring. This, however, was not done as majority of project proceeds were disbursed and the revision would have had negligible impact on the outcomes. The team could have restructured the project to correct the baselines as soon as the errors were identified. Alternatively, the Mid-Term Review (MTR) would have been an opportune time. But because an MTR of HSSP III was done jointly with the DPs and GOT which covered the HBF (i.e. the pooled portion of the project), the Bank team did not undertake a separate MTR for the non-pooled portions of the project as only a small portion of IDA credit was left undisbursed for the non-pooled portions. Also, the planned AF was not processed, which would have provided a basis for revising the results framework. 43. M&E design of the SWAp: Continuing with and building on the practices in place for M&E of the SWAp arrangement, the SWAp Committee 29 was agreed to be the body for overall policy dialogue. The PAD specified two meetings for discussing implementation progress and sectoral issues: the JAHSR, and six-monthly interim progress meetings. Other dialoguing structures included the DP Group for Health 30 ; 27 For interim results, the project relied heavily on the HMIS/DHIS, which had weak capacity at the time of appraisal. The project supported strengthening of HMIS and transitioning to the second phase of DHIS (DHIS2, rolled out in October 2015). Progress in improving HMIS/DHIS2, however, was less than satisfactory and quality of data was a concern as discussed in Annex 8 sub-section titled Lack of quality data. 28 Data: (i) facility-based delivery: lowest quintile 33.1 percent and highest quintile 89.6 percent; (ii) percent of children underweight: lowest 21.5 percent and highest 9.3 percent; (iii) prevalence of anemia: lowest 6.7 and highest 4.8; (iv) TFR: lowest 7.0 and highest 3.2; (v) use of any modern method of contraception: lowest 19.2 and highest 37.7 (National Bureau of Statistics of Tanzania and ICF Macro (2011), Demographic and Health Survey (DHS) 2010 ) 29 The SWAp committee included representatives of the MOHSW, PMO-RALG, MOF, NGOs, DPs and the private sector 30 Included 17 bi-lateral and multi-lateral agencies supporting the health sector in Tanzania 12

27 Basket Financing Committee (BFC) 31 ; Technical Committee of the SWAp; and several sub-committees of the Technical Committee. The HBF was guided by an MOU 32 between the GOT and the contributing DPs as well as annual Side Agreements that defined the priorities for the following year and level of contribution of the DPs. Additionally, the MOHSW, PMO-RALG, MOF and the DPs agreed on a Code of Conduct for the Health SWAp in Tanzania. These provided platforms for effective monitoring of and dialoguing in the sector. 44. Implementation: For the HBF (i.e. the pooled portion), the Bank used M&E arrangements under the SWAp to monitor progress. For the non-pooled portions, the Bank fielded six-monthly missions for monitoring progress and providing implementation support. A Bank team based in the country office provided implementation support. 45. Using project proceeds, 20 M&E officers were placed at the regional level to accelerate implementation of the DHIS version 2, which helped the MOHSW immensely. The DHIS2 software was customized and successfully introduced to all 167 LGAs. Also, the MOHSW has developed a Hospital Management Information System to track nonhealth and non-hrh related indicators as the existing management information systems track health outcomes, budget and expenditure data, and HRH. This system has been connected to 2 regional hospitals. Indicators were harmonized across the 16 Health Management Information System (MTUHA) tools. With the aim of integrating the various information systems, the MOHSW has plans to establish a unified system titled Health Sector Resource Warehouse, which will be a one-stop shop for information relating to the health sector. A feasibility study has been completed but further work could not be initiated as the project came to a close. This is being continued with support of the followon interventions. Capacity building initiatives included training of more than 28,000 health workers and 2,300 CHMT/RHMT on use of revised HMIS tools and training of 800 CHMT and RHMT staff on use of DHIS2. Annex 2 has details of the outputs. 46. The SWAp M&E arrangements worked reasonably well with the committees meeting frequently 33. The HSSP IV noted the Technical Working Groups (TWGs) are an asset for joint planning and implementation but are not always functioning as desired. Attendance varies and there is overlap with each other and with other committees or task groups. The coordination or exchange of information across TWGs is not optimal and weak linkages are maintained with the management structure of the MOHSW. 47. Fragmented systems were used for monitoring the SWAp and performance of LGAs 34. Multiple parallel systems for monitoring the SWAp included: (i) HMIS, a paperbased tool feeding into the digitized DHIS2, measured health outcomes; (ii) PlanRep used by the MOHSW for planning and budgeting purposes, mainly for CCHP preparation; (iii) EPICOR used by PMO-RALG for tracking expenditures; (iv) Human Resources Information System (HRIS) of PMO-RALG tracked staff; (v) Star Rating assessment done 31 Basket Financing Committee included representatives of the MOHSW, PMO-RALG, MOF, and basket DPs 32 The MOU provided a legal platform for cooperation of the signatories, setting out terms and procedures, but it is not legally binding or enforceable 13

28 under the Big Results Now (BRN) 35 initiative of the GOT, assessed readiness of facilities for providing services; and (vi) Hospital Management Information System of the MOHSW that tracks non-hrh issues. These parallel systems over-burdened the LGAs and facilities that were responsible for feeding information into it. 48. Utilization: Many of the management tools were not used effectively and to their full potential, particularly PlanRep. Due to a lack of reliable data, the DHIS2 could not contribute effectively to decision-making processes. The project s results framework was not useful either due to the various issues discussed. At appraisal, there was widespread recognition of the successes of the SWAp; this realization may have taken attention off supervision. Also limited use of data in decision-making contributed to it. 49. Sustainability: Coordination mechanisms of the SWAp are sustainable, being continued with under HSSP IV. Overall M&E systems of the MOHSW are donordependent, with each vertical program instituting a new information system. The HMIS/DHIS2 was supported by the donor funded MESI phase I 36 which was aligned to HSSP III. Currently, MESI phase II is under implementation, financed by the DPs, and aligned with HSSP IV. This raises concerns regarding sustainability of: one, the investments made in M&E; and two, the capacity created by the project. Twenty M&E officers were placed at the regional levels but could not be retained after project closure as the GOT was not able to take these people on board. The project s results framework is unsustainable as it includes a set of stand-alone indicators and is not being monitored after project closure. Important lessons learned from this project provided useful inputs to the design of the follow-on operation. 2.4 Safeguard and Fiduciary Compliance 50. Safeguards: The project triggered one of the Bank s Operational Policies, 4.01 relating to Environmental Assessment due to the potential impact caused by medical waste. Component 3 supported the MOHSW for implementing health care waste management (HCWM) activities, mostly at the regional and national referral hospitals. Major achievements included revision of various HCWM guidelines and procedures including HCWM Regulations signed by the Minister of Health (as detailed out in Annex 2). The 33 The Bank participated actively in many of these committees. 34 There are three fragmented mechanisms for tracking performance of the LGAs. The President s Delivery Bureau assesses the readiness of facilities (i.e. the supply side) through the BRN Star Rating. The MOHSW measures performance of the LGAs by tracking progress in health outcomes based on DHIS2 data, hence assists in monitoring. While the PMO-RALG assesses capacity of the LGAs through analyses of CCHPs done jointly with the MOHSW, which helps in planning. These three mechanisms produce three different tracking lists. At present, it is not clear if all three are used for planning and policy decisions. It would be useful to consolidate findings of the three tracking streams for effective use of the systems. 35 The BRN in Health program aims to accelerate the reduction of maternal and neonatal mortality through improving performance, governance and accountability in PHC. It was developed as part of Tanzania s Development Vision 2025 and has four national key results areas: (i) Performance Management; (ii) Human Resources for Health; (iii) Health Commodities; and (iv) Maternal, Neonatal and Child Health. The first area on performance management includes a stepwise accreditation scheme for all PHC facilities in the country (also known as the Star Rating Assessment ) which has both nation-wide assessment and a subsequent facility improvement program (including RBF incentives) to help facilities improve their performances and star ratings. 36 Financed by GFATM, Netherlands, CDC, Norway and the World Bank 14

29 PMO-RALG had limited/no role in HCWM activities of the project. During implementation, cross-check was not done to assess if HCWM activities were included in the CCHPs and implemented at the district and below-level facilities. In the absence of any review of HCWM activities at the field level, it is difficult to ascertain if HCWM activities were adequately practiced by the facilities. Other challenges relating to HCWM included: (i) initial delays in procuring HCWM equipment by the MOHSW; and (ii) lack of modern incinerators at the regional hospitals. 51. Social accountability: under component 2, district services were strengthened for better social accountability. A community score-card monitoring tool was developed and rolled-out to 1,069 facilities in 40 councils. Supportive supervision was provided for 51 low performing Council Health Service Boards (CHSBs) 37 and Health Facility Governing Committees (HFGCs) 38 that were selected based on the analysis of CCHPs. Also, the MOHSW followed-up with the PMO-RALG to ensure that all CHSBs were given the legal instrument (Hati Rasmi) 39 for operating and by February 2016, 97 CHSBs out of 167 were provided with the instrument. Other achievements under this component are listed in Annex 2. The CHSBs, however, have not been very effective in providing oversight of CCHP implementation in spite of the capacity building initiatives undertaken by the MOHSW. 52. Procurement arrangements for HBF: Procurement was done by MSD 40 as well as the LGAs. However, the procurement capacity of the LGAs and MSD was not assessed by the Bank. Although the PAD states that the Bank will provide procurement oversight for the international tenders financed by the HBF (based on the request of the GOT and the DPs) 41 ; in reality, this was not practiced in tenders by MSD. The Bank only reviewed procurement packages of the non-pooled portions. For procurement done using the HBF, the Bank relied on the audits done by the Controller and Auditor General (CAG) of GOT. The Bank team did not undertake any other audits on the use of the HBF, particularly relating to procurement 42, and used the country systems for auditing purposes. The annual procurement plans were reviewed and endorsed by the Audit Sub-Committee (ASC) 43. Quarterly progress report on procurement were reviewed by the ASC and follow-up discussions were held on CAG reports. 37 CHSBs are legal entities being part of the Local Government. They include political representatives and 7 volunteers from the communities. These meet 3 times annually (statutory meetings) and can also have 2 adhoc meetings. The minutes of CHSB meetings are kept by the District Medical Officers. CHSBs monitor facility activities, for example, when MSD delivers medicines to a facility a CHSB member is present to ensure medicines are properly received. The CHSBs also provide oversight in the implementation of the CCHPs. 38 HFGCs meet quarterly. The facility in-charge is the Secretary of HFGC who is also part of the CHSBs. 39 Hati Rasmi is the legal instrument provided to CHSBs so that these have the authority to operate with supremacy 40 The Annual Side Agreements specified the amount of funds to be transferred to MSD for procuring medicines centrally. It also stated the total funds that should be allocated to the districts/councils, of which a certain percentage could be spent on procuring medicines as per CCHP guidelines. 41 PAD page 3, para # 9. Also in page 29, para # 98 of the PAD, MSD is identified as a procuring entity but detailed description of the methods to be followed and/or the arrangements required not included. 42 Post-review audits, which are a standard practice of the Bank for any investment lending operation, were not done for the pooled funds. 43 The Bank was part of the Audit Sub-Committee 15

30 53. Procurement using HBF: Between FY2012/13 and FY2014/15, MSD procured medicines worth TShs 91.3 billion (approximately US$42.5 million) using HBF proceeds. A special audit done by CAG on drugs availability at MSD on request of the HBF partners in December 2011, revealed systemic weaknesses and inefficiencies at MSD (National Audit Office of Tanzania, 2011). The ASC followed-up on the CAG findings but not much progress was made in mitigating the issues, as documented in the audit by the Office of the Inspector General (OIG of GFATM) in February 2016 and systemic review of MSD done by Deloitte in December Consequently, the share of HBF allocated to MSD was reduced from 17 percent in FY2012/13 to 14 percent in FY2014/15 and more funds allocated to the LGAs for procuring medicines. Some of the challenges at MSD included: i) no evidence of drugs delivered by MSD reaching the end users; ii) stock of expired drugs at MSD; iii) MSD accepted drugs that had less than 80 percent shelf life; and iv) possible leakages. Additionally, audit trail was not properly maintained in MSD and anyone could delete information relating to procurement and inventory. Details provided in Annex Procurement for non-pooled activities: Initial delays in procurement at the MOHSW level were overcome during implementation through capacity building. Postreview audits done by the Bank did not reveal significant issues. Equipment for information, communication and technology were procured as well as 9 motor vehicles and other furniture and equipment (details provided in Annex 2). 55. Financial management: As noted in the previous section, the financial reporting arrangement was changed to IFR for the pooled portion and SOE for the non-pooled portions due to insufficient capacity in the MOHSW and PMO-RALG 44. The Bank s FM assessment at appraisal did not identify this and it was expected that IFR could be prepared. During implementation, the risk mitigation measures instituted resulted in modest improvements in the FM capacity as evident from: i) IFRs prepared accurately by the LGAs for HBF; ii) no significant financial irregularity found in CAG audits 45 ; iii) the Bank found internal control mechanisms and internal audit arrangements to be adequate. The software used for tracking expenditure, EPICOR, was not fully utilized 46 in processing financial reports and manual records were maintained (MOHSW, 2013c). This manual processing caused delays in finalizing reports. Capacity building initiatives included professional qualification of accountants, training conducted by the World Bank, workshop for MOHSW accountants, training on accruals basis of training, etc. (details provided in Annex 2). 56. Legal covenants: Twelve out of the fourteen covenants (PAD, page x-xii) were complied with. Two remaining covenants were external monitoring of LGA compliance to performance indicators and annual verification of these indicators. These could not be 44 This change was done during first six months of implementation 45 Except for two issues identified in the audit of FY2014 which were subsequently resolved. 46 Data inserted in EPICOR were not categorized by cost center and CCHP line item. Hence, expenditure (tracked by EPICOR) could not be automatically linked to budget (tracked by PlanRep) as PlanRep disaggregated data by cost center and CCHP line item. 16

31 completed as the HBF contributing partners did not agree on having performance indicators. 2.5 Post-completion Operation/Next Phase 57. The follow-on PforR operation became effective in September 2015, i.e., prior to project closure. This ensured continuity of funding and engagement of the Bank. The PforR incorporated important lessons learned from the project and the necessary mitigation measures have been instituted in the PforR. Examples (many of which have been included as foundational activities under the PforR) are: Increased attention to performance: Performance is being incorporated at the macro level (by use of PforR instrument), at the regional and LGA levels (through performance indicators that have for the first time been added to the HBF), and at the facility-level under the RBF intervention. M&E and data emphasis: Use of data has been amplified substantially due to the strong performance orientation and results-based focus of the PforR. The Program s LGA Scorecard and the RBF program utilize routinely generated data. District Medical Officers (DMOs) are more proactive in use of data to track their own performance. There is a close link between data and management of performance. For example, the Star Rating Assessment has identified quick wins that would allow facilities to improve their Star Rating and has been a strong incentive for DMOs to focus on facility performance data. Moreover, a Data Quality Audit tool has been generated to track improvement in data completeness and timeliness to address issues relating to data quality and because payment is based on the data. Capacity strengthening of LGAs: Capacity weaknesses are being addressed through a capacity building plan in the PforR, that is linked to key areas of delivery. This includes inter alia improving Comprehensive Emergency Obstetric capability as well as HMIS and M&E related capacity. Focus on MNCH services: The PforR is strongly grounded in the One Plan II (the RMNCH strategic plan), and with the use of the Global Financing Facility there has been strong DP alignment around the RMNCH agenda. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 58. The relevance of the project s objectives is rated as high, design substantial and implementation modest. Therefore, the overall rating for relevance is substantial. 59. Objectives: The project is fully aligned with two of the four objectives set out in the CAS FY , which was extended to : (i) strengthening human capital and safety nets, which aims to improve, inter alia, access to and quality of health service; and (ii) promoting accountability and governance, which is a crosscutting objective to improve accountability and efficiency of public management. 47 The June 2014 CAS Progress Report proposed a 12-month extension to the CAS, until 2016 (PAD of SPHCRP Report No TZ), which was granted 17

32 60. The World Bank s Africa Strategy has two broad pillars, which the project is aligned to: (i) competiveness and employment, addressing poor business environment, poor infrastructure, and the need for a healthy and skilled workforce; and (ii) vulnerability and resilience, addressing the high risk of idiosyncratic shocks, including those to individuals health (HIV/AIDS, malaria, Ebola, maternal mortality, etc.). Governance and public sector capacity is a foundation of the Africa Strategy. 61. The project is also aligned with the overall mission of GOT s HSSP IV, which is to ensure the provision of basic health and social welfare services that are of good quality, equitable, accessible, affordable, sustainable, and gender sensitive. 62. Design: the concept was proven to be effective at appraisal, i.e. continued use of the SWAp modality through HBF financing. To this two smaller components were to strengthen capacity of the LGAs, MOHSW, and PMO-RALG. The successful implementation of the previous SWAp was beneficial for DP harmonization as well as alignment of program to national priorities. The choice of Sector Investment and Maintenance loan (SIM) was appropriate. 63. Implementation: Implementation arrangements were complicated. There was no effective mechanism to track progress towards PDO or specific activities to ensure achievement of the PDO. It turned out as two projects (pooled and non-pooled portions) weaved into one. The HBF was not able to accommodate the much needed attention required for critical areas, for example RMNCH. In hindsight, it appears that more emphasis should have been placed in strengthening technical capacity of PMO-RALG in providing implementation support to the LGAs and strengthening fiduciary aspects. The functionality of the HBF could have been re-assessed to make it more responsive to the changing needs (as detailed out in section 3.3 and Annex 3), perhaps if the performance indicators for the HBF were agreed as was planned, it may have improved efficiency of the project. 64. HBF partner engagement: Over time, some of the DPs discontinued their financing of the HBF due partially to changing political and shifting priorities for engagement. Another reason was the diminishing focus of the HBF on equity 48. The discussions in BFC increasingly shifted to processes and the needed strategic focus was lost. Overall relevance Objectives Design Implementation Substantial High Substantial Modest 3.2 Achievement of Project Development Objectives 65. For the purposes of the ICR, five aspects of the PDO are assessed separately: improving equity of geographic access (rated modest); improving utilization of services 48 Netherlands and KfW did not contribute to the HBF in FY2014/15. The Netherlands stopped financing of the health sector of Tanzania. Funding from Canada and Denmark to the HBF declined significantly between FY2012/13 and FY2014/15. 18

33 (rated substantial); improving quality of services (rated modest); establishment of SDG (rated negligible), and improving capacity of LGAs (rated modest). Overall, achievement of PDO/efficacy is rated as modest. Table 2: Summary of achievements of results framework indicators Rating PDO indicator Intermediate Outcome Indicator Total Surpassed (100%+) Achieved/Substantially (85%+) Partially achieved (65%-84%) Not achieved (less than 64%) Total % surpassed and achieved 28.5% 84.6% 65% 66. Based on the results framework data, 29 percent of the PDO indicators achieved its target as shown in the table above (further details provided in Annex 10). With two out of the seven 50 PDO indicators achieved (surpassed) the target, the PDO was not achieved. PDO indicator # 2 measured average outpatient department (OPD) consultation per clinical staff and this indicator surpassed its target. At appraisal, HRH was not being efficiently used 51 which is why PDO indicator # 2 aimed to increase efficiency by increasing average OPD consultation per clinical staff from 1,300 (baseline) to 1,600 (target). In 2015, average OPD consultation per clinical staff was 2,259, representing 421 percent achievement compared to the target. In the absence of more detailed information, it is difficult to assess if this increase in average OPD consultation per clinical staff represents an increase in efficiency as envisaged. The other indicator that surpassed its target is the ratio of 10 best and 10 worst performing LGAs relating to PDO indicator # 1 (monitoring births attended in health facility), which decreased from 8.1 (baseline) to 4.1, representing an achievement of 329 percent. The target for this ratio was 15 percent decrease from baseline, which comes to Although the ratio has surpassed its target, it is not clear if this is due to an increase in facility-based delivery of the 10 worst performing LGAs or due to fall in facility-based delivery at the 10 best performing LGAs. In the absence of detailed information, it is difficult to assess the actual achievement. 67. With respect to the intermediate outcome (IO) indicators, 85% of the indicators surpassed/achieved the end targets. For component 1, 5 out of the 7 indicators surpassed the end target while the other 2 did not achieve it. For component 2, all four indicators (IO 49 Intermediate outcome indicator # 3 has two parts. Hence treated as two indicators for the purpose of this calculation. 50 PDO indicator #5 measures the ratio of 10 best and 10 worst performing LGAs relating to indicators 1 to 3. Hence, these are treated as three indicators instead of one, for the purpose of the ICR percent of clinical staff members of selected sample of health centers and dispensaries saw fewer than 10 patients per day, (Haazen, D., (2012) Making Health Financing Work for Poor People in Tanzania, the World Bank) 19

34 indicators 8 to 11) achieved the end targets. The IO indicators 12 and 13 that tracked progress of component 3 surpassed the end targets. 68. Did equity of geographic access and utilization of health services improve? Although the PAD does not specify the regions where access was considered to be an issue, it is understood that access was particularly problematic in at least three regions Mara, Mwanza and Shinyanga. Three indicators are considered here for the ICR analysis: (i) percentage of deliveries at a health facility, (ii) the number of pregnant women who availed one ANC visit, and (iii) the number of OPD visits per person per year. The figure below shows that proportion of deliveries at facilities in these three regions has improved although it did not reach the PDO target of 69 percent. For Shinyanga, the improvements are substantial as institutional deliveries were on a decline before the project start (45 percent in 2004 to 33 percent in 2010) but increased to 57 percent in Figure 1: Institutional deliveries (percentage) in Mara, Mwanza and Shinyanga 52 47% 48% 46% 53% 45% 57% 32% 33% 33% Mara Mwanza Shinyanga Trend of OPD visits indicates an overall decline during the project period, as shown in the figure below, which implies a decline in utilization of services. Increase in OPD visits from 0.4 to 0.53 in Shinyanga between 2012 and 2015, may be attributed to the pilot P4P mechanism, whereby providers were given incentivizes for improving service delivery. Figure 2: OPD visits per capita per year in Mara, Mwanza and Shinyanga in 2010, 2012, 2015 (MOHSW, 2013b) Mara Mwanza Shinyanga 52 Data taken from Tanzania DHS , 2010 and

35 70. ANC coverage has declined, as shown in the figure below. It was not possible to analyze the reason for declining ANC when institutional deliveries have increased due to lack of supporting data. The MTR of HSSP III noted: service access and general utilization appear unchanged and as a measure of quality, readiness of services generally not improved, except for few items (e.g. rapid testing for malaria), and many gaps remain (MOHSW, 2013b). Based on the evidence presented, it can be concluded that geographic access and utilization of services have not improved substantially. Figure 3: Antenatal care in Mwanza, Mara and Shinyanga in 2011 and , ,994 86, ,171 87,450 85,071 Mwanza Mara Shinyanga # of 1st ANC # of 1st ANC 71. A 2015 Lancet article concluded that Tanzania s progress in RMNCH is mixed, with substantial advances in child survival but slower progress in maternal and newborn survival, and family planning. Tanzania has attained the MDG target 4 (Hoviyeh, et. al., 2015) relating to child mortality by drastically reducing under-five child mortality to 67 deaths per 1,000 live births in 2015 (National Bureau of Statistics of Tanzania and ICF Macro, 2016) compared to 81 deaths in 2010 (National Bureau of Statistics of Tanzania and ICF Macro, 2011). Reductions in child mortality that was documented between 2004 and 2010, continued during the project period as shown below. Insufficient progress, however, was made towards attainment of MDG 5 relating to maternal health with the maternal mortality ratio of 410 deaths per 100,000 live births in 2013 (Lancet 2015) compared to 454 maternal deaths in Slow progress was noted in reducing neonatal mortality which in 2013 accounted for 40 percent of deaths in children aged less than 5 years (Lancet 2015). Figure 4: Trends in child mortality (deaths per 1,000 live births), compared to HMIS 54 Project Appraisal Document of BHSP, Report No TZ, page 2 55 Data taken from Tanzania DHS 2010 and

36 (project baseline) (project end) Under-5 child mortality (probability of a child dying between birth and the 5th birthday) Infant mortality (probability of a child dying before the 1st birthday) 72. Did the utilization and quality of health services improve? The result is mixed and the evidence points towards modest improvements in quality. Given the significant issues related to the design of the project s results framework, a set of proxy indicators is used to further assess the outcomes of the project as presented in the results chain below. The data have been derived from SDI 2014 and DHS and are, therefore, not directly comparable. Details of the indicators are provided in Annex 10. Overall, the proportion of deliveries attended by skilled personnel has increased, as per DHS data. This includes both institutional and home deliveries. It is likely that more home-based deliveries are now being performed by skilled personnel. Institutional deliveries have increased by 10 percentage points from 50.2 percent in 2010 to 60.2 percent in as per DHS data (although the end-of-project target of 69 percent was not achieved). For ANC coverage, the overall proportion has increased albeit some geographic variations. With regards to child health, maintaining the level at 75 percent 56 for child immunization is considered an achievement. 73. Project s contribution to the PDO: US$85 million was channeled through the HBF to finance the CCHPs. In doing so, the project ensured basic health services for 47 million people. In 2014/15, 1.2 million women went for delivery at health facilities; 31 million outpatient cases were attended at health facilities; 29.7 million children were provided vitamin A supplements; and 10.8 million pregnant women were provided ante-natal care. Also, 32,945 health personnel were trained on various technical aspects. 56 Tanzania DHS 2010 and

37 74. The two indicators diagnostic accuracy and adherence of staff to clinical guidelines indicate a modest improvement in the quality of services, although the rates are still very low. A review of clinicians ability to manage maternal and neonatal complications in the 2014 SDI survey showed only a 30.4 percent compliance rate with clinical guidelines among providers and 35.7 percent among doctors. The MTR of HSSP III 57 noted: low attendance rates of outpatient departments and maternity wards can be an indication of the population s dissatisfaction with health services. 75. Two salient aspects of the project are the capacity improvements of the LGAs and formation of the SDG, although the PDO does not reflect these. For the purposes of the ICR, these two aspects are being separately assessed. By extension of the PDO, one of the aims of the project was to establish an SDG, which could not be done. The various basket funds within the health sector as well as the basket funds in other sectors continue to operate separately. Moreover, the concept of the SDG as stated in the PAD did not include any specific activity/intervention for it to materialize. Figure 5: Results chain of the project MOHSW, (2013) Midterm Analytical Review of Performance of the Health Sector Strategic Plan III, Data for inputs taken from SDI 2014 survey, data for output taken from Service Provision Assessment 2016; and data for Outcomes taken from DHIS2. Details provided in Annex

38 INPUTS: 16%-point decrease in drug availability 5%-point increase in equipment availability 31%-point increase in infrastructure availability 7%-point decrease in absenteeism of clinical staff OUTPUTS: 1%-point increase in availability of all basic client services* OUTCOMES: ACCESS AND UTILIZATION 13%-point increase in deliveries attended by skilled personnel 10%-point increase in institutional deliveries 8%-point increase in ANC coverage No change in child immunization 10%-point decrease in vitamin A supplementation QUALITY 8%-point increase in adherence to clinical guidelines** 3%-point increase in diagnostic accuracy** * measured as percentage of facilities offering child curative care, child growth monitoring, vaccination, any modern family planning method, ANC, and services for sexually transmitted infections ** relating to acute diarrhea, pneumonia, diabetes mellitus, pulmonary tuberculosis, malaria with anemia (SDI 2014) 76. Did capacity of the LGAs improve? Over the life of the project, LGA capacity relating to PFM has improved 92 percent of LGAs with unqualified audit opinion of CAG in FY2013/14 compared to 78 percent in FY2011/ Almost half (54 percent) of the LGAs was able to use EPICOR in FY2013/14 to provide financial information compared to none in FY2011/ A challenge was slow implementation of the CAG recommendations. In spite of the improvements in PFM, the capacity constraints of LGAs posed challenges, particularly in M&E. The 2013/14 Summary Analysis of CCHPs by the MOHSW and PMO-RALG noted that half of the councils faced challenges in reporting the implementation status of their activities as they were not able to quantify them; 64 percent of LGAs fully implemented their annual plan of activities; 52 percent complied with the standard supervision practices; and 72 percent performed standard outreach services (MOHSW and PMO-RALG, 2014). 77. Project s contribution to the PDO: For capacity building, TShs 3.34 billion out of the project proceeds were provided to all 167 LGAs (each LGA receiving TShs 20 million) in FY 2014/15. Ninety health workers were trained on National Plan of Action for HCWM. Equipment for improving PFM information technology worth US$25,500 were 59 Report of the Audit Sub-Committee of the health sector 2015/16, (Updated December 2015) 60 In FY2013/14, 43 percent of the CAG recommendations was implemented by the LGAs and 35 percent by the MOHSW. Also, the LGAs audit and internal control functions were weak with 67 percent of the LGAs not having an effective audit committee in FY2013/14 (Report of the Audit Sub-Committee of the health sector 2015/16, updated December 2015). Also, the LGAs audit and internal control functions were weak with 67 percent of the LGAs not having an effective audit committee in FY2013/14 (Report of the Audit Sub-Committee of the health sector 2015/16, updated December 2015). 24

39 procured and used by project team. Thirty-three MOHSW accountants passed professional examination. To improve M&E, 28,750 health care workers and 2,395 CHMT/RHMT were trained on use of the revised HMIS tools. The DHIS2 software was customized and successfully introduced to all councils. Further details provided in Annex Efficiency 78. Overall rating for efficiency is modest. With the HBF supporting the SWAp, transactions costs associated with DP coordination were reduced. Reliance on existing structures avoided the creation of new set-ups and the associated costs. The HBF (to which 90 percent 61 of the project proceeds went) played an important role in health care financing and has generally been pro-poor in its allocation. The HBF funds offered the LGAs assured opportunities for implementing service delivery. As noted in the operational review of HBF (MOHSW, 2013c), the broad earmarking of funds to LGA level has helped ensure that funds reached primary health care level; traceability has allowed follow-up to monitor that resources actually reached. However, allocative efficiency of the project (spending on right things) is low/negligible given the limited progress made in achieving project outcomes. There was an apparent lack of flexibility in the use of HBF resources which prevented its effective use and somewhat limited autonomy of the LGAs. A resource allocation matrix (with thirteen priority areas and six allocation ceilings as detailed out in Annex 7) had to be followed when preparing the CCHPs, which complicated the planning process and once approved, the districts could not reallocate funds between categories. Many of the needs of the facilities remained unmet as this restricted the use of funds 62. At appraisal, this matrix may have been introduced to ensure that funds are used for priority areas but during implementation it created inflexibility in the use of funds. 79. During the life of the project, the proportion of Bank funding of HBF compared to the other DPs increased from 9.6 percent of the HBF to 37.8 percent. Netherlands and Germany pulled out of the HBF 63. Overall, total HBF fell by 36 percent. Table 3: DP contribution to the HBF to (Side Agreements for FY2011/12 to FY2015/16) Source of fund World Bank (US$ million) Total DP contribution to HBF including WB (US$ million) WB contribution as % of total HBF 9.63% 24.20% 31.09% 37.82% 80. With the HBF shrinking, non-basket funds were the major source of DP funding for HSSP III 65. The HBF as a proportion of total on-budget sources declined 61 Due to exchange rate losses, IDA credit of US$100 million at appraisal became US$95.1 million at end of the project. As US$85 million was disbursed to HBF, it accounted for 89.4 percent of total IDA allocation 62 Reported by the LGAs in Shinyanga visited during field trip 63 In FY2011/12, HBF partners were Canada, Denmark, Ireland, Netherlands, Switzerland, Germany, World Bank, UNFPA and UNICEF. In FY2013/14 Netherlands and Germany stopped financing the HBF. 64 Although the Side Agreement indicated Bank s pledge of US$22 million and hence a total of US$115.81m total in HBF, the Bank only disbursed US$10 million as the project became effective in March Total figure for FY2011/12 have, therefore, been adjusted accordingly 25

40 TShs billion (for bars) TShs billion (for line) from 12 percent in FY2012/13 to 6 percent in FY2014/15, as shown in the following figure. Figure 6: Changes in on-budget financing source, FY2012/13 to FY2014/ ,000,000 2,000,000 1,000,000 1,000, Central Govt funds National Health Insurance Fund Health Basket Fund Foreign Funded Projects and Programs- non basket funds Total on-budget 81. Compared to the total district budget (for the non-salary portions), the HBF accounted for 19 percent in FY2014/15, down from 30 percent in FY2012/13(MOHSW and PMO-RALG, 2015). Vertical programs represented a major share of financing of nonsalary activities. For essential health interventions, the largest financier was GFATM (34 percent) followed by other DPs (19 percent) and HBF (14 percent) in FY2014/15 (MOHSW and PMO-RALG, 2015). The HBF s relevance declined as it possibly served as additional funding for the sector (Operational Assessment of the Health Basket Fund, 2014). 82. For these two factors total HBF shrinking over the years and the non-basket portions growing larger the HBF s effectiveness and efficiency were limited. While the vertical programs may have made services more easily accessible, there were concerns that these have created inequality in resource allocation (Mujinja and Kida, 2014). The MTR of HSSP III suggested for the coming years, the focus should be on improving value for money by making optimal use of available resources for better quality (MOHSW, 2013b). 65 MTR presentation by MOHSW, October 23, Data provided by MOHSW 26

41 83. Efficiency of project execution: With the HBF supporting the SWAp, transactions costs associated with DP coordination were reduced. Reliance on existing structures avoided the creation of new set-ups and the associated costs. There were, however, delays in initiating the non-pooled portions as the designated accounts were not set-up timely. Low utilization of project funds was noted for the non-pooled portion 52 percent of allocation, primarily due to delays in procurement (National Audit Office, 2015) but improvements were noted in 2016 when 99 percent of the funds were used. Moreover, the fact that the pooled and the non-pooled portions operated like two projects in parallel, limited implementation efficiency of the project. Annex 3 provides further details. 3.4 Justification of Overall Outcome Rating 84. Based on the modest ratings for efficacy and efficiency, the overall outcome is rated moderately unsatisfactory. Overall rating Relevance Efficacy Efficiency Moderately Unsatisfactory Substantial Modest Modest 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 85. It is likely that the project has benefitted the poorer segments of the population as an increasing share of the HBF was allocated to the LGAs (further discussed in Annex 3). The project design, however, did not specifically take into account its impacts on poverty reduction and hence, no data is available for it. Benefits to females surpassed its target of 50.5 percent 67. Although all health services provided by the public facilities were available to all people, the reproductive and maternal services particularly benefitted the women/girls. As part of project component 3, the HCWM procedures and guidelines were revised and HCWM catalogue was developed which strengthened HCWM practices in select hospitals. (b) Institutional Change/Strengthening 86. CCHP process was strengthened through capacity building and supportive supervision of RHMTs. Also, capacity of the LGAs improved in PFM as well as M&E capacity of the regions improved through deployment of M&E officers due to the project. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops no survey/workshop done. 4. Assessment of Risk to Development Outcome Rating: Moderate 67 Approximately 48.6 million direct project beneficiaries of which 51.4 percent were female 27

42 87. Some of the risks that were there at appraisal are still relevant. These include the capacity constraints of the LGAs, and HMIS/DHIS2 data quality issues, although some progress has been made in these areas. Additionally, the following risks exist: Vertical programs account for a larger portion of the district budgets. The GOT will need to reduce its reliance on external financing as DP funds are expected to decline with Tanzania transitioning to a middle income country. Systemic inefficiencies at MSD resulting in medicine stock-outs. This is being mitigated as facilities can use RBF proceeds under the PforR to meet immediate requirements of medicines. MSD is setting-up outlets in regional hospitals as detailed out in Annex Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 88. As mentioned, the project was strategically relevant, being well-aligned with the GOT s priorities and the Bank s strategies. The pooled portion built on the implementation successes under the Bank-financed projects supporting the previous SWAps. The concept of the project was logical, but the design was complex with overambitious targets and unclear implementation arrangements, particularly for the nonpooled portions. The choice of indicators for the results framework was not appropriate and many of the indicators were not adequately defined. Procurement capacity assessment was not done for MSD and LGAs although these entities used project proceeds to procure medicines. By design, HCWM was practiced only at selected facilities (regional and national level hospitals) and no assessment was done to check if the LGAs (i.e. facilities at the district and below levels) practiced HCWM activities. (b) Quality of Supervision Rating: Unsatisfactory 28

43 89. Continuing with the earlier practice, the Bank led the BFC for most of the project duration, which proved to be advantageous for donor coordination purposes as well as leading the dialogue with the GOT. Supervision missions were conducted every six months with reasonably staffed teams. Procurement of medicines by MSD (worth US$42 million allocated from the HBF) mostly using international tenders were not prior or post reviewed by the Bank although the PAD stated that the Bank will provide procurement oversight for such tenders. No reviews were undertaken to assess compliance of LGAs to the HCWM practices. Due attention was not paid to the outcomes of the project and the trends of indicators. The results framework was not revised to address the design problems. Interim results fed into the ISR reports were self-reported data extracted from the weak HMIS/DHIS2 for which no quality audit and/or third-party verification were done. Although many of these problems were noted in the ISRs, the PDO and implementation progress were consistently rated moderately satisfactory, signaling good project performance which was not the case. The Bank should have put in more effort in extracting reliable results to justify the investments. Turnover of Bank staff with five Task Team Leaders over the five-year period may have affected continuity of the project. (c) Justification of Rating for Overall Bank Performance 90. Overall rating for the Bank s performance is unsatisfactory as overall outcome of the project and quality at entry are rated as moderately unsatisfactory, while supervision is rated as unsatisfactory. Overall rating Quality at Entry Quality of Supervision Unsatisfactory Moderately Unsatisfactory Unsatisfactory 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 91. HSSP III being a government document represents full ownership of the GOT of the SWAp, to which the project contributed. This ensured harmonization of DP funding to national priorities and continued commitment of the GOT. The decline in share of GOT s budget to the health sector from 8.94 percent of total budget in FY2011/12 to 8 percent in FY2014/15 68, can indicate a reliance on DP funding for the sector. The GOT funds were mostly used for paying salaries and hence, the districts had to rely more on DP financing for operational costs. Delays in fund release by MOF affected project implementation. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 92. The main implementing agencies were the MOHSW and PMO-RALG. The team at the MOHSW present during project design was replaced by an entirely new team during implementation with no transitioning and/or hand-over. This adversely affected the nonpooled portions in particular. Appointment of the Project Coordinator in the second year of the project greatly benefitted implementation. Synergies between the pooled and nonpooled portions were limited due to lack of clarity of the project concept and 68 Source: Author s calculation based on data provided in the Citizens Budget Edition for Financial Year, , , , Ministry of Finance 29

44 disagreements between the MOHSW and PMO-RALG. Weak technical expertise of PMO-RALG meant that supportive supervision provided to the LGAs was not sufficient. This was aggravated by the weak institutional capacity of the LGAs and improper distribution of HRH. There were insufficient institutional and technical capacities to handle environmental and social issues. Due to a large number of ongoing interventions, the MOHSW could not provide adequate technical input to the districts and HBF suffered from lack of oversight. (c) Justification of Rating for Overall Borrower Performance 93. Overall rating for borrower performance is moderately unsatisfactory as overall outcome of the project is rated as moderately unsatisfactory. Overall rating Government Implementing Agencies Moderately Unsatisfactory Moderately Satisfactory Moderately Unsatisfactory 6. Lessons Learned 94. Projects should continue supporting SWAps when there is evidence of the modality working well, as in the case of Tanzania. However, in a SWAp with funding from various sources, there should be an effective mechanism for dialogue between government and all DPs including the off-budget sources, if possible. When off-budget sources grow larger than the on-budget sources, there are competing demands on the human resources that result in inefficiencies. It would be beneficial to have measures in place for consolidating the parallel projects under the SWAp umbrella, or for greater harmonization with the off-budget sources to ensure complementarity. 95. Efforts should be made to make the Bank s support to SWAps results-oriented. A criticism in many SWAp reviews indicates focus on results have tended to suffer due to strong emphasis on processes and transactions. Future engagements in pooled funds/basket arrangements should have a stronger results-orientation. 96. In a SWAp arrangement where funds are pooled, it is important to have agreement upfront amongst the pooling DPs on the key aspects of the project. Without this agreement in principle, it should not be included in the design. In the project, for example, there was a legal covenant of externally monitoring LGA compliance to performance indicators. The pooling DPs did not agree to have the performance indicators and the covenant was not met. 97. In countries where a decentralized arrangement exists whereby health project activities are implemented by districts, design should be done in consultation with the district authorities and sectoral/line ministry can provide technical inputs to it. In this case, the project was designed in consultation with the MOHSW but most of the implementation responsibility was with the PMO-RALG and the districts. Also, the service delivery managers need to be involved along with the planning departments of the line ministries. 98. It is necessary to undertake political economy analyses of reforms at appraisal; otherwise the project can fail during implementation. Also, managing institutional changes 30

45 is difficult and it involves taking into account feedback of all stakeholders. For this project, the SDG did not materialize due to many of these issues. The key stakeholders may have felt the need to have an integrated basket; the project, however, did not include any realistic strategy to establish it, neither was a feasibility analysis done. 99. M&E should be focused on from start of project rather than towards the end. While this project diverted significant efforts in M&E, it came quite late during implementation. Bank teams should proactively revise results framework if problems are identified. While relying on country systems, additional measures should be instituted that provide adequate assurance to the Bank of its investments. In this case, the project could have factored in alternative mechanisms for validating/triangulating data, for example through regular surveys, instead of relying fully on the HMIS. Similarly, third-party verifications administered jointly with the government would have been useful It is critical to ensure that strong implementation support is provided by the Bank to ensure that the PDO is on track, and if not, necessary changes are proactively made to ensure value-for-money of Bank investments. Projects should include SMART indicators for which regular updates are available. Wherever possible, the results framework should include indicators that are already being tracked by the government to avoid the need to rely on additional player(s) for reporting on project s outcomes. For critical project indicators, relying on other DP-driven projects/initiatives to strengthen HMIS is not advisable The concept behind a project design should be easy to understand. During design, rigorous assessment and/or in-depth analytical work should be undertaken to institute implementable activities. This project suffered as the pooled and non-pooled portions continued as two parallel interventions. Also, it would have been useful to have a detailed project operation manual so that subsequent teams could understand the rationale of concept and implementation arrangements Adequate attention must be placed to ensure realism in institutional assessment. For the project, the preparation period was relatively short, and there seems to have been a great degree of optimism regarding the government s capacity to meet the operation s institutional requirements. The project preparation could have benefitted from a more formal assessment of institutional requirements for implementation of the operation and government s capability to meet those requirements With changes of Bank task team leaders (TTLs), it is important that there is detailed handover. The change in TTLs happens for a variety of reasons, but when there are relatively frequent changes as was the case for this project, detailed TTL handover is needed, as well as management oversight of operational risks posed by frequent turnover. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Co-financiers 31

46 (c) Other partners and stakeholders 32

47 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent) Components Appraisal Estimate Actual/Latest Estimate Percentage of Appraisal 1: Basket Fund % 2: LGA Capacity Building % 3:Central level Capacity Building % Project Management 0.5 Exchange rate loss 4.9 Total Financing Available % (b) Financing of the Program (in US$ Million equivalent) Source of Funds Appraisal Estimate Borrower 1, Actual/Latest Estimate Percentage of Appraisal International Development Association (IDA) % Pooled fund % Non-pooled fund Total 2,

48 Annex 2. Outputs by Component Component 1: supported the implementation of CCHPs. The CCHPs were financed through multiple sources of funds. HBF was one of the sources, to which the Bank contributed. Some of the outputs achieved are: Basic health services provided to approximately 47 million people 69. Medicines and medical supplies worth US$42 million procured by MSD and used by the health facilities across the country. 1.2 million women went for delivery at health facilities in FY2014/ million outpatient cases were attended by health facilities in FY2014/15. 8 million children were immunized with DPT3 vaccine in FY2014/ million children were provided vitamin A supplements in FY2014/ million pregnant women were provided ante-natal care in FY2014/15. 32,945 health personnel were trained. Component 2: 17 LGAs in Tanga, Pwani, Mara, Mbeya, Mwanza, Kigoma, Singida, Arusha and Katavi regions, were provided grants for capacity building in FY2013/14 for a total amount of TSHs million. Each LGA received TShs million. In FY2014/15, TShs 3.34 billion were provided to all LGAs for capacity building activities. Each LGA received TShs 20 million. Component 3: District health services: The MOHSW coordinated with all 167 LGAs to monitor district health service delivery. The MOHSW along with the PMO-RALG analyzed the technical and financial reports of all 167 councils every year to identify the low-performing LGAs. Supportive supervision and training were provided to these LGAs. Guidelines and operations tools for RMSS printed and distributed. Annual summary analysis of CCHPs and quarterly progress reports printed. Improvements made to PlanRep and CHMTs trained on it. M&E: Indicators were harmonized across the 16 Health Management Information System (MTUHA) tools. These were printed and distributed to all 25 regions for in FY2013/2014 and 2014/2015. Training of 28,750 health care workers and 2,395 CHMT/RHMT done on using the revised HMIS tools. DHIS2 software was customized and successfully introduced to 167 councils. HMIS training manual integrated into Health Training Institutions curricular to ensure that all graduates from health colleges have knowledge on HMIS. Trainings were conducted and manuals printed and distributed to these institutions. Training of 888 CHMT and RHMT staff done on using DHIS2. 69 MOHSW used this population figure for allocating HBF 34

49 Annual Health Sector Performance Profile Reports were produced for 2011 and 2013, with the Mid Term Review Report for Reproductive and Child Health scorecard launched in May 2014 which makes use of routine HMIS data to monitor performance of districts in these areas. Hospital Management Information System developed and 2 regional hospitals connected to it. Feasibility study done for Health Sector Resource Warehouse. Social accountability: Community scorecard developed and citizen engagement initiated for improving social accountability. Training imparted to 174 local government leaders of the 29 newly established districts to ensure that CHSBs and HFGCs were set-up. By February 2016, 23 CHSBs were set-up (79 percent) out of the 29 districts. 16,000 copies of guidelines on CHSBs and HFGCs were printed and distributed to all districts. HCWM: HCWM Regulations developed, signed by the Minister of Health. This will now be disseminated to the facilities and key stakeholders and its implementation will be monitored through the National HCWM Program. National Guidelines for HCWM, initially developed in 2004, revised in National Standards and Procedures for HCWM revised. National Policy Guidelines for HCWM revised. HCWM Monitoring Plan revised. National Catalogue for HCWM developed. This catalogue provides guidance on how to implement HCWM practices as well as a list of the equipment needed for HCWM along with price ranges and standardized technical specification. 90 health workers have been trained on National Plan of Action for HCWM. Financial management: 8 accountants of the MOHSW qualified in CPIA exams. 33 accountants of the MOHSW passed professional examination. Short training courses conducted by the World Bank. IT equipment for PFM worth US$25,500 procured and used by project team. National Guidelines for Cost Sharing reviewed and signed. Expenditure report books printed and distributed. A workshop done for the MOHSW accountants and PFM secretaries on PFM required for PforR. The MOHSW has migrated from IPSAS cash basis to IPSAS accrual. Training done for accountants and procurement staff on accruals basis of accounting. Procurement by the MOHSW: 9 motor vehicles procured worth US$143,783. ICT equipment (computers, laptops, printer, scanners, servers, photocopier etc.) worth US$138,691 procured for use by project team and working group. Office furniture and equipment worth US$20,020 procured. Other: Manuals and training materials developed for non-financial staff. Supported implementation of the Health Financing Strategy. Project completion report prepared. 35

50 P4P pilot supported by HRITF in Pwani region and Kishapu district of Shinyanga region: Pwani region: 104. The Pwani pilot documented some positive results (Binyaruka, et. al.): A 10.3 percent increase in the share of women receiving two doses of intermittent preventive treatment for malaria during antenatal care (ANC). An 8.2 percent increase in the share of women having institutional delivery. Increased provider kindness reported by patients during delivery. Overall ANC coverage increased by 3.3 percent. Increased patient satisfaction The pilot also contributed to health systems strengthening (MOHSW P4P Assessment Team, 2013): Staff were more motivated. Facilities were more proactive at resolving bottlenecks including improving medicine availability. Supervision was more strategic. Distribution and retention of HRH improved. Improved accountability at multiple levels. Overall efficiency improved Some of the challenges that affected the P4P intervention included (MOHSW, 2015): Stock-out of RMNCH commodities adversely affected service delivery. Delays in paying incentives. Shortage of HMIS tools that delayed reporting and compromised data quality The Pwani P4P model used a fee-for-service approach and hence a lot of the funds were utilized in payment of staff. Based on the experiences of other countries as well as the implementation challenges faced in Pwani, the P4P design has been revised which is now being implemented in Shinyanga region with support from the follow-on project (SPHCRP). A wider roll-out of P4P is planned once the Shinyanga pilot is successfully completed and lessons drawn from implementing the two designs initial design implemented in Pwani and the revised design in Shinyanga. Kishapu district, Shinyanga region: 108. The HRITF also supported P4P in 45 health facilities in Kishapu district of Shinyanga region. Progress of the RBF indicators are provided in the table below: 36

51 Sl. Table 4: Achievement of P4P in Kishapu district 70 RBF Indicators Achievement Quarter Ending June 2015 Achievement Quarter Ending September 2015 Achievement Quarter Ending December New outpatient consultations 41,178 44,132 34,686 2 Clients initiated to counsel and test for HIV 3,051 2,283 2,523 3 Under-five children receiving Vitamin A supplements 4,066 1,651 5,469 4 New users using modern family planning methods 2,131 1,957 2,408 5 Pregnant mothers who received mebendazole 1,744 1,801 2,453 6 Institutional deliveries 1,826 1,717 1, Pregnant women attending ANC at: least 4 times during pregnancy Pregnant women receiving two doses of (IPT2) Mothers receiving Post Natal Services within 3-7 days after delivery Children under one year immunized against measles Poor individuals (from TASAF register) who received outpatient care Children given mebendazole every Six months ,453 1,073 1, ,020 1,238 1,000 2, ,690 1,325 3, First antenatal visits with gestation age < 12 weeks Tuberculosis suspect referred (already screened) HIV positive pregnant women receiving anti-retrovirals (ARVs) HIV exposed infants receiving ARVs Data provided by MOHSW 37

52 Annex 3. Economic and Financial Analysis Efficiency in the utilization of the HBF: 109. The HBF was allocated using an equity based formula 71. In the first two years, the weights used were population (70 percent), poverty count (10 percent), district medical vehicle route (10 percent), and under-five mortality (10 percent) which were revised to 60, 20, 10 and 10 respectively in FY2014/ There were, however, inconsistencies in statistics applied by the MOHSW for HBF allocation formula. The population figures used by the MOHSW were under-stated, much lower than the figures projected by the National Bureau of Statistic for the same years. Further, the population figures used in the formula were different from that reported in the MOHSW Annual Health Sector Statistics for the same years. For under-five child mortality, the figures used by the MOHSW for the formula were significantly different from the figures reported in the CCHPs (MOHSW, 2013c) As stated earlier, a larger share of HBF was allocated for use by the councils. The table below shows the utilization of HBF over the years. Allocations to the councils for service delivery increased by 17 percentage points between FY2012/13 and FY2014/15. Allocation to the regions and to PMO-RALG for supervision also increased while allocations to the MOHSW and MSD decreased by 18 percentage points. This was done to ensure that adequate funds were available to the districts for service delivery and strengthening health systems and to address the issue of medicine stock-outs (as councils could procure medicines with their own funds when supplies from MSD were delayed and often resulted in stock-outs). The funds available to the districts, however, could not be efficiently used due to the expenditure ceilings associated with HBF as well as diversion of attention and HRH for the vertical projects. Also, the total share of HBF was falling as explained in the following paragraphs. Table 5: Use of HBF, FY2012/13 to 2014/15 72 Allocated to 2012/ / /2015 TShs million % TShs million % TShs million % Councils: resource allocation formula (average US$1.05/capita) 89, , , Regions: RHMT supportive supervision to 4, , , councils PMO-RALG: Supportive Supervision Performance based formula was not designed as the HBF partners did not agree 72 Data provided by MOHSW 38

53 Allocated to 2012/ / /2015 TShs million % TShs million % TShs million % MOHSW & MSD: (US$0.20/capita for medicines and supplies) 63, , , Total 157, , , As discussed in section 3.3, the on-budget sources of funds increased by 55 percent between FY2011/12 and FY2014/15. The increases in on-budget sources were primarily from Government sources (Local Government Block Grant), which mostly paid for salaries. For operational costs, foreign funded projects were the main sources of funding. Over this period, the HBF declined as a share of total district budget, as shown in the figure below. The rationale at appraisal for continuing to finance the HBF as it served as a major source of fund for districts diminished over time. Figure 7: HBF as a share (percentage) of total district budget, FY13 to FY % 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% 29.2% 14.7% 20.3% 10.8% 18.8% 9.0% % of budget including salaries (PE) % of budget excluding salaries (PE) Health financing issues: 113. The total health expenditure per capita has increased in spite of a growing population, as shown in the figure below. The GOT has rebased the GDP estimates recently. The health expenditure figures stated are not related to the rebased GDP. The graph on the right indicates that although total health expenditure per capita has increased, it has been financed mostly from private sources. Health expenditure from private sources as percentage of GDP has increased from 4.38 percent to 4.66 percent while health expenditure from public sources has fallen from 2.82 percent to 2.65 percent of GDP. 39

54 Tshs billion (for the bars) Figure 8: Total health expenditure (left) and Health expenditure as a % of GDP (right) 73 Health Expenditure, per capita Health Expenditure, as a % of GDP Health exp. per capita (current US$) Health exp. per capita, PPP Health expenditure, total (% of GDP) Health expenditure, public (% of GDP) Health expenditure, private (% of GDP) 114. As discussed in section 3.3, non-basket funding was the major tool of DP funding for HSSP III 74. Over the same time period, the GOT s budget allocation to the health sector declined as shown in the figure below. Figure 9: Government s financing of the health sector 75 25,000 20,000 15,000 10,000 5, % 8.48% 8.16% 8.00% % 9.00% 8.50% 8.00% 7.50% Allocation to Health % allocation to health Total budget 115. The significant increase in vertical programs funded by DPs, raises the concern of increased donor dependence and the associated sustainability risks. Complete data on offbudget sources are not available which is why it is not possible to quantify the increases over time. One of the largest contributor to the off-budget source of funds is the United States Government President s Emergency Program for AIDS Relief (PEPFAR). Between FY2011 and FY2015 (excluding FY2014), Tanzania spent approximately US$1.2 billion from PEPFAR allocations. With Tanzania transitioning to a middle-income country, DP financing is expected to decline in the coming years and the Government would need to strategize on how to sustain the recurrent expenditures. 73 World Bank Databank. GDP figures are not the rebased GDP 74 MTR presentation, October 23, Source: Author s calculation based on data provided in the Citizens Budget Edition for Financial Year, , , , Ministry of Finance 40

55 Economic analysis: 116. Given the lack of data and the inability to undertake an impact evaluation, it is not possible to measure the real impact of the project. For the purpose of economic analysis, potential benefits of the project are estimated from the economic benefits gained due to reductions in infant and maternal mortality between 2010 and The analysis presented in the PAD calculated an internal rate of return (IRR) of 8.76 percent based on the assumption that the target for MDG 4 would have been achieved. This would mean that infant mortality rate (IMR) would have reduced from 51 deaths per 1,000 live births in 2010 to 38 by 2015 and under-five mortality rate (U5MR) from 81 deaths per 1,000 live births in 2010 to 64 by The IRR was calculated using average value addition of US$864 per year which was taken from the Integrated Labor Force Survey. The benefits were calculated by multiplying lives saved by the average annual value addition and a labor participation rate of 89.6 percent. The analysis also assumed that 3.67 percent of the benefits would be attributed to the project. The analysis presented in the PAD is not conclusive as life expectancy and healthy life years have not been considered. Therefore, for this ICR disability adjusted life years (DALYs) are used, which provide more realistic estimates Regarding attribution, as discussed in section 3.2, it is not possible to attribute results by financing source in a SWAp with pooled financing mechanism. For the purpose of this analysis, however, a rate of 2.5 percent is used for attributing the economic benefits to IDA instead of 3.67 percent used in the PAD. The HBF accounted for an average of 10 percent of the on-budget financing of the Government between and and the Bank s contribution to the HBF averaged 25 percent over the same period of time. Hence, a proximate rate of 2.5 percent is used for this analysis As per DHS 2015 data (National Bureau of Statistics of Tanzania and ICF Macro, 2016), U5MR declined from 81 deaths per 1,000 live births in 2010 to 67 in 2015 and IMR declined from 51 deaths per 1,000 live births in 2010 to 43 in Hence, the MDG target relating to child mortality has not been achieved. As per data of the World Bank 76, the MMR declined from 514 deaths per 100,000 live births to 398 in 2015 (estimated) The table below summarizes the estimated economic benefits from reducing IMR, U5MR and MMR. DALYs gained have been calculated by multiplying the number deaths averted by the average life expectancy 77. The economic value has been calculated by DALYs gained multiplied by GDP per capita (conservative approach) and by three times the GDP per capita as recommended by WHO Commission on Macroeconomics and Health. Table 6: Economic Benefit Analysis from Reduced IMR, U5MR and MMR IMR U5MR MMR For IMR and U5MR, life expectancy at birth of 60 has been used. For MMR, a 30 year life expectancy has been used. 41

56 IMR U5MR MMR Death averted 16,169 28,295 30,658 Attributable to IDA (2.5%) DALYs averted 24,253 42,443 22,994 Economic value (US$) DALY = one time GDP per capita 23,161,602 40,532,804 19,323,903 DALY = three times GDP per capita 69,484, ,598,412 57,971,709 Benefit-Cost Ratio DALY = one time GDP per capita DALY = three times GDP per capita The results indicate that for US$1 invested by the project, benefits generated are in the range of US$0.23 and US$0.69 when IMR is considered; US$ for U5MR; and US$ for MMR. The project has generated economic benefit of US$23 69 million from reduction of IMR, and between US$19 57 million from reducing MMR Around 48.6 million people directly benefited from the project, of which 51.4 percent were female; 8 million children were immunized and 29.7 million children received a dose of vitamin A. The project also contributed to capacity building of RHMTs as well as training of 32,945 health personnel. During the life of the project, improvements were made in the MIS with the roll-out of DHIS2. The benefits from these initiatives have not been quantified due to lack of data. Clearly, the analysis presented here underestimates the overall benefits of the project to a large extent. On the other hand, multiple factors within and beyond the health sector have contributed towards achievements of the economic benefits. For example, health systems strengthening have been funded by both on-budget and off-budget sources (e.g. PEPFAR) together with non-hnp factors such as education. The benefit-cost ratio presented here, therefore, is acceptable given the lack of available data. Equity: 123. An increasing share of the HBF was allocated to the LGAs 74 percent of HBF budget allocated to councils in FY2014/15 compared to 54 percent in FY2012/13. This has benefitted the poorer segments of the population more, although there were no targeted interventions for the people of the lower income quintiles. This conclusion is derived from the fact that a larger proportion of the poor people resided in rural areas where widespread poverty existed and half of the poor people accessed services provided by the public health facilities. According to a World Bank assessment, poverty is particularly pervasive in the rural areas, where around 70 percent of the Tanzanian population lives. About 10 million people in the rural population live in poverty, and 3.4 million live in extreme poverty, compared to less than 1.9 million living in poverty and 750,000 people in extreme poverty in the urban sector (World Bank, 2015). As per DHS , 58.3 percent of people in the lowest income quintile and 72 percent of the second 42

57 lowest income quintile sought treatment for child diarrhea from health facility or provider (National Bureau of Statistics of Tanzania and ICF Macro, 2016). Hence, with a greater share of the HBF budget made available to the districts, it may be concluded that the HBF was pro-poor in nature In the figure below, four indicators are analyzed to track improvements in equitable access to services between 2010 and The evidence is mixed and points to modest gains in equitable access of services. Institutional deliveries increased by 6 percent in the lowest income quintile within this time period whereas for the richest quintile, the rate has fallen by 1 percentage point. With respect to the proportion of children who were stunted (i.e. short for age, which is a measure for long-term undernutrition), in the lowest wealth quintile a 9 percentage point reduction was achieved compared to 7 percentage point in the richest quintile. When contraceptive prevalence rate is considered, there was no change between 2010 and 2015 for the lowest quintile whereas for the richest quintile it dropped by 2.5 percentage point. Figure 10: Measures of equity, 2015 compared to Lowest quintile Highest quintile Contraceptive prevalence rate (any modern method) Lowest quintile Highest quintile Births delivered at a health facility Lowest quintile Highest quintile All basic vaccinations for children under 5 Lowest quintile Highest quintile Children stunted (short for age) 125. Centrally administered resources were not equitably allocated. With the Local Health Block Grant 79 accounting for half of total budgets available for funding the CCHPs, there are variations in resource allocation to LGAs as only the HBF used the more equitable formula which is not used for allocating Local Health Block Grant. Regions such as Singida, Tabora, Shinyanga, Rukwa, Kigoma, Kagera, Katavi and Simiyu, identified as the most underserved regions and in need of additional resources and support 78 Data taken from Tanzania DHS 2010 and Majority of this fund is used for paying employee emolument 43

58 through the BRN 80 process, all received below the equitable share of resources as shown in the figure below. Figure 11: Actual allocation of block grant (in TShs billion) in FY2014/15, compared to the proxy equitable allocation (using the HBF allocation formula) (MOHSW and PMO-RALG, 2015) 126. The variations in the share of budget allocation each region received was influenced by a number of factors including, decision-making around councils own resources, councils ability to raise funds through cost-sharing mechanisms and user fees, and the range of DPs and NGOs active in the LGA (MOHSW and PMO-RALG, 2015). There were factors that affected credibility of the budget and practical usefulness for facility and program managers, in spite of significant progress. Rigidity in the budgeting process, the late definition of budget ceilings and poor linkages with programmatic priorities or actual population needs reduce its usefulness as a planning and managerial tool. The approved budget was often modified substantially, and irregular disbursement patterns over the year make it generally a poor reflection of the original plan developed by LGAs and facilities Out-of-pocket expenditures were 23 percent of total health expenditure 81. With widespread medicine stock-outs and relatively high out-of-pocket payments, the poorer segments of the population may have been adversely impacted. This, however, cannot be confirmed due to lack of data. 80 The BRN in Health program aims to accelerate the reduction of maternal and neonatal mortality through improving performance, governance and accountability in PHC. It was developed as part of Tanzania s Development Vision 2025 and has four national key results areas: i) Performance Management; ii) Human Resources for Health; iii) Health Commodities; and iv) Maternal, Neonatal and Child Health. The first area on performance management includes a stepwise accreditation scheme for all PHC facilities in the country (also known as the Star Rating initiative) which has both nation-wide assessment and a subsequent facility improvement program (including RBF incentives) to help facilities improve their performances and star ratings. 81 World Health Organization, Global Expenditure Database: 44

59 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Sanjeev Ahluwalia Consultant GGODR Maryam Salum Ahmed Team Assistant AFCE1 Denis Maro Biseko Senior Public Sector Specialist GGODR Bella Lelouma Diallo Senior Financial Management Specialist GGODR Dominic S. Haazen Lead Health Policy Specialist GHNDR Jane A. N. Kibbassa Senior Environmental Specialist GENDR Emmanuel G. Malangalila Consultant GTIDR Donald Paul Mneney Senior Procurement Specialist GGODR Helen Z. Shahriari Senior Social Scientist GSURR Supervision/ICR Dominic S. Haazen Lead Health Policy Specialist GHNDR Emmanuel G. Malangalila Consultant GTIDR Donald Paul Mneney Senior Procurement Specialist GGODR Raymond Mbishi Consultant GGODR Janneke Blomberg Nutrition Specialist GHNDR Yvonne Nkrumah Senior Operations Officer WBIHD Julia Mensah Consultant WBIHD Michael Okuny Senior Financial Management Specialist GGODR England Maasamba Financial Management Analyst GGODR Mwanaisha Kassanga Program Assistant AFCE1 Gisbert Kinyero Senior Procurement Specialist GGODR Neema Mwakalinga Program Assistant AFCE1 Rekha Menon HD Sector Leader GHNDR Olusoji Adeyi Sector Manager AFTHN Sangeeta Raja Senior Health Specialist GHNDR Yahya Abdallah Ipuge Senior Health Specialist GHNDR Christophe Rockmore Senior Economist GHNDR Agnes Nderakindo Mganga Team Assistant AFCE1 Eva K Ngegba Program Assistant GHN01 Son Nam Nguyen Lead Health Specialist GHNDR Bernard Olayo Health Specialist GHNDR Gayle Martin Program Leader AFCE1 Helen Z Shariari Senior Social Scientist GSU07 Jane A N Kibbassa Senior Environment Specialist GEN07 Evelyn Anna Kennedy Senior Operations Officer GHN01 Paulina Proches Shayo Temporary AFCE1 Iffat Mahmud Operations Officer GHN19 45

60 (b) Staff Time and Cost Bank budget (in US$): Year No. of staff weeks Staff cost Travel Other Total FY , , FY , , , , FY , , , , FY , , , FY , , , FY , , , , FY2017 5, , Total 559, , , , Trust fund #TF01484 (in US$): Year No. of staff weeks Staff cost Travel Other Total FY , , , , FY , , , , Total 64, , , ,

61 Annex 5. Summary of Borrower's ICR The summary includes excerpts from the Project Completion Report of the MOHSW, dated April 13, 2016, prepared by Dr. Daudi Simba. 1. Context at Appraisal With a 2010 population of approximately 43 million, Tanzania is the seventh most populous country in Africa. The annual population growth rate is high (2.9 percent), and it is projected to remain high into the foreseeable future, dropping only to 2.7 percent by The National Bureau of Statistics population projection attributes this stable population growth rate to a combination of a significant decline in the Total Fertility Rate, and a reduction in the crude death rate, largely resulting from declining infant and underfive mortality. As a result of these dynamics, Tanzania s total population is projected to increase to 65.3 million by Despite the solid economic growth, the 2007 Household Budget Survey (HBS) poses a mixed picture of the country s progress in poverty reduction over the past eight years. Lower relative prices from imported goods and increased government spending led to tangible improvements in some areas, including ownership of consumer durables, housing quality, and some social indicators such as under-five mortality and enrollment in primary education. On the other hand, progress was limited in the areas of improvements in basicneeds income poverty, ownership of productive assets in rural areas, as well as some other social indicators, such as maternal mortality or access to safe water. Further, while both income-poverty incidence and the depth and intensity of poverty declined, the size of these decreases is small. The HBS indicates that per capita consumption increased by only 5 percent in real terms between and 2007, and in rural areas the reduction in poverty was not statistically significant. There was also a marked urban-rural gradient in poverty incidence, with the poverty head-count ratio in Dar es Salaam calculated at 16.4, compared to 24.1 in other urban centers and 37.6 in rural areas. Rationale for the Project: The project contributes to the third objective in the World Bank country assistance strategy, which includes outcomes related to improvements in the access and quality of health care delivery. There is inadequate engagement between DP and PMO-RALG/LGA s on sectorial issues: no complementary mechanism exists for grants to finance direct service delivery. The rationale for the Bank involvement in the SWAp was to mobilize the Bank's comparative advantage vis-à-vis the other partners. The Health Basket pooled funds from 11 DPs was selected in order to reduce transaction costs and strengthen government systems. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Justification for the Design: The Project had the objective to assist the Government in improving equity in geographic access and use of basic health services across districts and 47

62 enhancing the quality of health services by supporting LGSD; Capacity Building in Local Governments and Central Programs Supporting Local Service Delivery. The design did not follow the classical logical framework common with many Projects. In addition to having broad objective(s), classical Projects are usually guided by measurable specific objectives that are logically linked with strategies and activities for attaining them. Each specific objective has appropriate output(s) and indicators to specifically measure each of the respective objectives. The design adopted in BHSP was likely to have been influenced by the nature of support that the Government requested from the Bank. The Bank supported the Government to fill in the financial gap in the provision of health services. Almost 90 percent of BHSP funds were channeled through HBF and MSD (procurement of medicines and supplies). Since the impact of this kind of support was overarching and covered the whole country, this justifies the choice of the design. The BHSP support, however, constituted only a quarter of the total CCHP funding, which also contributed less than 10 percent of the overall Councils operational costs. This relatively small contribution, therefore, compromises the level of attribution to the project. It is from the nature of the design that the Project adopted HSSP III indicators to measure its outputs because it can be correctly argued that the funding was contributing to this strategy. Justification for the Indicator Selection: The Project had one main objective (Project Development Objective) and three intermediate objectives. Selected indicators for measuring the objective (PDO) are not Project specific. This can be argued, from the Project design point of view, to be appropriate because the support, however small, covered all the health services delivered in the Councils. Similarly, Intermediate indicators for Component 1 cover a wide spectrum of services; only Component 2 and 3 are Project specific. Much as the use of HSSP III indicators was appropriate, but these did not reflect on the actual Project implementation due to small level of attribution. The alternative could be to limit assessment to input and process indicators. But this would defeat the purpose of the financial support which was to assist the Government in provision of health services rather than implementing the project as an intervention per se. Selection of the indicators was biased towards MNCH services; HRH and commodities. Selection of MNCH indicators was justifiable given the level of priority given by Government on improving these services. The selection of HRH and availability of medicines indicators was justified because the two contribute significantly to quality of services and that they are important health systems building blocks. Justification for HBF: The Project was planned to utilize HBF to channel funds to the LGAs. This was based on a number of advantages including, (i) long term experience accrued by the HBF DPs and Government in operating the fund; (ii) reaching all LGAs without additional costs; (iii) ensuring sustainability of the performance-based financing (PBF) approach since HBF uses Government systems; and (iv) building the capacity of the existing systems (PFM and M&E) in administering and monitoring the fund. Alternatively, funding could be disbursed to the LGA through a standalone Project. Although this could 48

63 ensure efficiency this would be with additional costs and risk of collapse once the Project came to an end. Another alternative was to send the fund directly to the Councils; this carried the risk of the fund being misused and failure to achieve objectives due to weak financial and monitoring systems. The decision to use HBF to initiate PBF approach did not, however, come without its cost. The challenge was to get members of the HBF to agree with the arrangement. As a mitigation strategy, the Bank adopted an evolutionary approach to introduce PBF by maintaining existing approach until when stakeholders are ready for a change. Although there was no better alternative to this approach, it took the whole Project s life to come to a consensus and in the end PBF approach could not be implemented within BHSP. 2.2 Implementation Project disbursements: Halfway through, the Project had received and disbursed about 60 percent of the funds and by end of June 2015 almost all the Project s fund (99 percent) had been received and disbursed by the Bank. Disbursement of funds to Implementing Agencies went smoothly, throughout the Project s life, for Component 1. The fact that this Component carried the bulk of the funds (90 percent) rendered the flow of funds satisfactory throughout the implementation period. The smooth disbursement of funds was attributed to timely production of CCHP thanks to the strong and constant supportive supervision, coaching and mentoring from RHMTs; and central level supervisors. The use of PlanRep helped to a large extent to expedite planning process thus ensuring timely completion and reporting. Several problems were experienced in the disbursement of funds for Component 2 and 3. Disbursement of funds was delayed for more than six months due to delay in completion of the Project Implementation Plan which was one of the trigger conditionality for disbursement of funds. Serious delay occurred in the disbursement of Capacity Building Grant to the Council; this was as a result of funds being withheld for failure to comply with the Legal Covenants. It was agreed in the Legal Covenant framework that CBG will be disbursed upon the production of proposal to the satisfaction of the Bank. Unfortunately, the process of proposal preparation took longer than anticipated time and in the process had to be modified to avoid further delays. Adequacy of Government involvement and commitment: Evidence on Government commitment with the BHSP is depicted through the signing of the Project s MOU. Signing the document affirmed understanding and acceptance of the relevance and appropriateness of the Project to national interests. The mere action of requesting for extension in order to accomplish the remaining activities signifies Government s intention to have a successful end for the Project. Furthermore, in order to make the Project more viable, the Government requested for additional funding to scale up P4P that is also based on PBF approach. The frequent delays in implementing some of the activities may be interpreted as inadequate Government commitment. But, on the other side, there is adequate evidence on the Government position towards attaining the objective espoused in the BHSP. 49

64 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization: Linking Implementation to results Framework HSSP III indicators are reported through the Annual Health Sector Performance Profile Report, presented and discussed during the JAHSR. Due to limited analysis, many intriguing questions do not get answered. Several long standing problems in the health sector require gathering of adequate data so as to perform in-depth analysis and arrive at plausible solutions. In most cases this is not what happens during decision making meetings. As a result of superficial analysis sometimes simple non-committal answers are provided in resolving complex issues. A comprehensive M&E Strengthening Initiative (MESI) exists at the Ministry of Health that aims to improve M&E by ensuring data timeliness and accuracy. MESI also aims to strengthen the routine HMIS and improving data management so as to improve information access using information and communication technology. During implementation of the BHSP, efforts to integrate internet-based data warehouse information subsystems was initiated with the support from the University Computing Centre. But due to some technical challenges the process was delayed and the outputs are unlikely to be realized within the Project s timeframe. Nevertheless, the integrated system will still be useful in the monitoring and evaluation of HSSP IV and other ongoing programs. Challenges: Understanding of some of the BHSP indicators posed a challenge to some central level staff thus necessitating members of the Mission to take time to make clarification. This might have been attributed to complexity of the said indicators or inconsistent use of different indicators to mean the same thing. For example, births at health facility can easily be mixed up with Skilled Birth Attendance since they all reflect on access and quality of delivery services. But while the latter includes home deliveries by skilled personnel the former does not. Some central level staff had difficulties calculating indicators prompting the Bank mission to recommend training on Excel. Therefore, the need to have simple and consistently used indicators is of paramount importance, especially, in the wake of RBF/PBF implementation at the health facility level where the level of understanding of indicators is relatively lower. Failure of staff at the health facilities to understand and interpret the indicators may bring unnecessary complaints of unfair judgment by the verifiers. Some of these challenges partly contributed to delays in completing the results framework. Staff reported spending long time to address data inadequacies in the attempt to make them intelligible. Wide in between-year variations that had no plausible explanations were not uncommon. Lack of reliable denominator from the National Bureau of Statistics also contributed to the variations because of continued change of the projected country population in a year. CCHP summaries consistently raised concerns of incomplete filled in tables, making trend analysis difficult. Overdependence on HMIS data resulted into system overload that overstretched it beyond its ability to produce quality data. The health facilities are mostly affected because it absorbs the whole load in a manual data collection system. Due to the colossal amount of resource required for capacity strengthening at this level, it will take a considerable period of time before this level is computerized. 50

65 Activities to improve data quality: A number of activities were conducted during the Project s life that contributed towards improving data quality. Most of these occurred at the central and Councils levels. These activities included strengthening of the computerized systems DHIS 2 and integrating it with other routine information system (HRH Information System, PlanRep and EPICOR). Apart from supplying HMIS registers and books, not much was reported to have been done to improve data quality, processing and use at the health facility level. Yet this is the level where most data inaccuracies and incompleteness originates. Conclusion: Despite some limitations, the HMIS remains to be the best option among the available data sources for monitoring health care services. While efforts to link information systems are exemplary, it is high time now to focus on linking routine data with surveys and operational/implementation research. This will enable the M&E team to obtain in-depth data that will provide evidence-based responses to major health systems questions and concerns raised by policy and decision makers. 2.4 Safeguard and Fiduciary Compliance Fiduciary: The Project was implemented within the Governments systems and the Permanent Secretaries of the Ministry of Health and PMO-RALG took full responsibility as accounting officers. The Chief Accountants of the two Ministries were responsible for the day-to-day financial management. The Project involved pooled funds for the LGA HBF in Component 1 and Central level HBF and non-pooled funds for Component 2 and 3 activities, respectively. Prior to Project commencement, an initial assessment was conducted to establish the level of compliance to fiduciary issues. It reported that unaudited IFRs were submitted on time and were reviewed and found to be satisfactory. The Ministries got clean audit reports for the year ending June, 2010 for out-of-basket funded activities. HBF reports prepared by the Ministries and the LGAs were up to date. Assessment rated the fiduciary compliance as satisfactory. In the course of BHSP implementation, the overall rating of financial management of the Project s funds was rated satisfactory. All IFRs were reported by the Ministry of Health quarterly, albeit with some delays, and were rated satisfactory. All legal covenants were complied with. Audit reports for non-pooled funds received unqualified audit reports for the whole period of Project implementation and were submitted to the Bank on time. The CCHP teams adhered to the guidelines and the quality of CCHPs improved with time with the proportion that qualified in the first round increasing from 31 percent to 57 percent. Reported problems were mainly on the capacity to use computer system (PlanRep) and technical problems with the software; and the level of commitment in the preparation of the documents. Majority of the LGA received unqualified audit opinions (150/163) and very few had adverse audit opinion (only one in 2013/14). The main reasons for qualified audit opinions were related to inability to use the PlanRep software; delays and poor documentation. 51

66 Procurement: Risk assessment prior to project commencement reported the Ministry of Health had insufficient capacity in terms of number of staff and level of training. Staff were not experienced in procurement of goods through international competitive bidding and contract management. There were also inadequacies in procurement planning, procurement filing and record keeping. Procurement capacity at the LGA level was reportedly unsatisfactory. This led to the overall risk in procurement to be rated high. But given mitigation measures that included staff training planned to be conducted during Project implementation through Component 2 and 3; the risk was reduced to moderate. Currently, the number of staff has increased to almost 30, with the majority of them being graduates. In the course of implementation need arose for a procurement officer to work on full time basis in the Project in order to expedite procurement and optimize contract management. One procurement officer from the Project Management Unit was assigned to work with the Project on full-time basis. Environmental safeguard: Environmental assessment conducted prior to commencement of the project highlighted the potential for health care waste to pose a risk to patients as well as health service providers. The Ministry of Health through the HCWM program has the responsibility to standardize HCWM practices so as to reduce health risks. The Project supported the Program on a number of issues as outlined in the National Action Plan. Assessment done in 2014 reported a number of challenges in HCWM including wide variation on HCWM and standards across health facilities. And although some best practices were observed in some few hospitals such as Mawenzi hospital in Moshi, in general, standards were yet to be achieved. The assessment concluded with a call for more attention and supervision support by both regional and national HCWM. 3. Assessment of Outcomes Achievement of Project Development Objectives The PDO of BHSP was moderately achieved and rated moderately satisfactory. The Project achieved only one out of five of the outcome targets. This was outcome 5 in which only one of the six targets were met, that is, the ratio of the 10 best performing LGA s to the 10 worst performing LGA s in birth attendance. Of the 15 targets for the intermediate indicators, 8 were achieved. While only 3 out of 9 intermediate indicators for component 1 were achieved, almost all component 2 and 3 indicator targets were achieved. One of the indicator targets was not achieved in component 2, this was on conducting CHSB quarterly meetings to discuss, among others, issues related to budget, CHF, rehabilitation, schedule of visits to lower facilities and staff houses. The indicator on the use of performance and equity indicators in LGA allocations and use of the approach developed under this project for service delivery grants in other sectors measures the achievement of performance based approach, the underlying concept behind the Project. It is through the successful implementation of this approach that equity and quality of services, the cornerstone of the Project, could be attained. A total of 17 LGAs 52

67 were selected by PMO-RALG for support in 2013, based on performance and capacity criteria. In view of this achievement, and almost all other intermediate indicators, the level of achievement of the Program is rated as moderately satisfactory. Health Structure and implementation: The trends in child mortality and infant mortality are downwards; and Tanzania met the targets of the MDGs in The trends in neonatal mortality and maternal mortality are also downwards, but less, and did not meet MDG targets. The rate of deliveries attended by skilled attendants is increasing, slowly, while figures on ANC care are hardly improving and ANC fourth visit even reduced. Health services in Tanzania are provided through a decentralized policy environment, with Local Government Authorities running public dispensaries, health centers and district hospitals, while referral and tertiary hospitals are run through the Ministry of Health. At the central government level, the PMO-RALG oversees the LGA s and regions. LGA s have very limited own revenue sources, so funding flows to local governments are mostly through the national budget. These flows include per capita allocations by DPs through the HBF. Births attended in health facility: Tanzania has achievement significant success in reducing under five mortality; increasing vaccination coverage; increasing ANC attendance and increase in number of health facilities providing MNCH. However, neonatal mortality rate remains quite high with limited availability of equipment for newborn. Although a slight increase in skilled birth attendants was noted, the pace was slow. The overall percent HBF has been below the target in the Project implementation period. This implies that many women continue to deliver under unconducive environment that exposes them to higher morbidity and mortality. Although availability of basic and comprehensive emergency and obstetric care showed a significant improvement, the health facilities are still faced with inadequate performance; shortages of skilled staff and supplies; and inadequate access and quality improvement. Most of the challenges are more pronounced in rural and poor communities. Outpatient attendance per clinician: The average number of OPD attendance per clinician declined steadily from 3,130 (2012) and 2,259 (2014) but fell short of meeting the target set for 2015 (1600 OPD attendance per clinician). The anticipated achievement was, however, not uniform due to observed wide gap (three-fold) among regions. OPD attendance per clinician measures the quality of services provided which is mainly influenced by the number of clinicians. The country has been experiencing shortage of health staff in particular medical doctors whose ratio was reported to be 5.4 per 10,000 population with a four-fold variation between regions. Despite a tremendous increase in students output in colleges, employment of graduates remained a major bottleneck. OPD utilization: OPD attendance per capita in the 4-year period of BHSP implementation did neither reach the target nor surpass the baseline. The trend was the same for four years prior to BHSP. Most of the regions (21/25) did not meet the target in all the four years of BHSP implementation. In fact, three regions performed below half of the target during the whole implementation period. Interestingly, some regions did well despite having less 53

68 than half of their population living within 5 kilometers from a health facility. These differences contributed to the six-fold gap observed among regions between 2012 and OPD per capita is also known as outpatient utilization measures access to health services. The indicator is influenced mainly by financial, geographical and the perceived quality of services. Financing of health sector has not performed well in the country. Government contribution on health reached the Abuja target in 2008/9 only to drop thereafter. Out-ofpocket expenditure on health accounts for about one-third of total health expenditure putting a large proportion of patients, especially, those from low income bracket to the risk of catastrophic spending. Although the country has achieved much success in bringing health services closer to the people (73 percent population lives within 5 kilometers) there has been a slow pace in the increase in number of health facilities; about 22 percent in three years. At this pace, it will take another six years to achieve the national target of 10,000 health facilities. Availability of Medicines and Medical Supplies: Several definitions exist for stock out rate. But, irrespective of the definition used, stock out rates have been high before and during BHSP implementation period with most reports revealing stock outs of 50 percent or more. An assessment done by the Ministry of Health in 2008 using 20 trace medicines found half of the health facilities had stock outs of one or more medicines ranging between days. The HMIS / DHIS use 10 tracer commodities and define stock out as any event of missing a tracer medicine within one month of observation. Using the HMIS definition, high rates of stock outs ranging between 69 percent and 95 percent were observed during the period of BHSP implementation. Not only did performance fail to meet the target but the stock-out rates were unacceptably higher (two-three fold) compared to the baseline. Like many other indicators, a wide variation (nine-fold) between regions was observed. Despite a good reputation that MSD has amassed on quality management systems, still the organization faces several challenges in ensuring availability of medicines at the health facility level. Major challenges are frequent delays in disbursement of funds that disrupts flow of commodities; and few local pharmaceutical companies that do not meet national requirements. Weak inventory systems at the HF level contributes to stock outs in the facilities and poor forecasting that also leads to disruption in the flow of commodities at the MSD. The high level of stock outs may explain the low per capita OPD attendance because patients are likely to practice self-medication by buying medicines from accredited drug dispensing outlets that are allowed to sell, in addition to over-the-counter drugs, a selected number of prescription-only medicines. Progress towards Results Framework indicators: The Project had five PDO outcomes each with one indicator except outcome 5 that had six. Indicators for outcome 5 measures achievement in equity that is, the extent of closing the gap between the best and the least performing LGAs as well as regions. Targets were achieved in one of the six equity indicators that is, reducing the ratio between the 10 best performing LGA s and the 10 worst performing LGAs in indicator 1 (increasing proportion of births at health facilities). 54

69 Evidence is lacking on factors prevailing in the LGAs that may explain the differences in performance. There is therefore need to establish factors contributing to the differences in performance among LGAs in order to design more focused intervention strategies. Moreover, the findings were not unexpected since the Project design linking achievement of equity with the implementation of PBF approach started in 2013 through the PforR Program only in 17 LGAs. Although the target for indicator 2 (OPD attendance per clinician) was not met, the observed trend towards achievement clearly indicates good progress towards meeting the target. Several strategies are being undertaken by the government to increase HRH in the LGAs. This pattern is an indication that these strategies are working albeit in a slow pace. About half of the intermediate indicators were achieved. Almost all targets for component 2 and 3 were achieved except one. This was on the functioning of CHSBs. Towards the end of the Project, 64 percent of CHSBs were reported functioning thus falling short by 22 percent from the set target (86 percent). In addition to not achieving the target the trend toward achievement has stagnated throughout the Project implementation period. There is need to establish factors influencing the slow progress in order to inform strategies for enhancing the speed in future. Contributions to efficiency: The Project channeled funds to LGA through HBF thus minimizing operational costs. Efficiency in performing day to day operations was improved by training accountants in PFM and Bank procedures; and CHMTs and RHMTs on M&E. through the Project s support provided to the CCHPs, supervisory teams were facilitated; thus contributing significantly to the improvement of quality of CCHPs and the quarterly and annual reports. The overall Project outcome is rated moderately satisfactory. Implementation of the project contributed, irrespective of the proportion, to country s achievement in reducing IMR and U5MR. Although the PBF approach could not take-off within BHSP timeframe, the consultations and dialogue that ensued contributed significantly to the consensus that led to introduction and implementation of the PforR (SPHCRP) in Since it was not possible to realize the outcomes within BHSP, the overall outcome is rated moderately satisfactory. 4. Assessment of Bank Performance Quality of entry: The Bank made some efforts to involve stakeholders in the efforts to obtain a consensus on the performance based approach to financing health services. This included consultations with the Health Development Partners Group. Apparently, consensus was not reached and the idea was shelved for future discussion. At this juncture, the MOU for implementing BHSP using PBF approach had already been signed by the Government. Evidence is lacking on whether consultations and dialogue were conducted prior to signing the MOU. This gives an impression that the project might have been introduced in haste. The same style of introduction new programs is reported by some Ministry official to have been used during the introduction of the PforR. The Program is 55

70 reported to have started before conclusion of existing Project (BHSP) which is reported by some of the Ministry officials as unusual. The problem with this approach is, it creates a vacuum between the time the project is signed and actual implementation. Thus by the time implementing agencies organizes themselves a considerable amount of time would have passed. This is reported to have befallen the introduction of BHSP in which officials involved during designing were not the ones involved in initiating the project. By the time the Project team was constituted, a significant delay has occurred. Would be beneficiaries of the CBG were required to write proposals for capacity building. This was new arrangement to them thus required sensitization on the rationale and procedures. In the absence of prior sensitization delay was inevitable. It is evident that there was inadequate (if any) stakeholders engagement prior to signing of the MOU with the Government; and that minimal efforts were put to engage them during implementation. This contributed to failure to implement PBF within the framework of BHSP. The emergence of RBF thereafter came as an opportunity for the Bank to pursue the PBF dialogue to fruition. In view of the unsatisfactory quality at entry and satisfactory quality of supervision overall Bank performance is rated moderately satisfactory. 5. Lessons Learned Conceptual Framework: The route to change (PBF) was long, tortuous and expensive. It took about four years of advocacy and dialogue for stakeholders to reach a consensus. This underscores the need for high level Government commitment and substantial financial support from the onset of the Project. It also emphasizes the importance of perseverance in soliciting for stakeholders consensus, allowing for adequate space for dialogue and flexibility for changes to evolve over time. Project s Take-off: Inadequate consultation with stakeholders and sensitizing implementing agencies resulted into delays in Project take-off and implementation of several activities thereafter. Delays are common when new approaches and procedures are introduced. There is need to ensure adequate consultations with stakeholders, especially those involved in the day to day operations in order to facilitate smooth take-off. There is also need for prior sensitization of responsible officers in the implementing agencies when new procedures and approaches are introduced. This will enable them to understand the rationale for the changes and the implementation procedures. Parallel Funding Systems: Experience during implementation of the project proved HBF to be an effective approach to disburse funds to LGAs by creating synergy and avoiding duplication. Moreover, HBF funds are closely monitored and the funds are adequately accounted for. Thus parallel financing mechanisms created by other DPs results into duplication hence loss of value for money, due to inefficiency; and disrupting PBF approach. Apparently, the PMO-RALG has taken up the challenge and is currently planning to do mapping of DPs. Effective coordination: Successful implementation of health interventions is dependent on key Ministries Ministry of Health to guide on standards and regulations; PMO-RALG 56

71 to coordinate implementation; Ministry of Finance to provide necessary funding to facilitate the availability of human resource. Achievement can only be realized if each Ministry plays its part. But the most crucial is having strong coordination and mechanisms spearheaded by equally strong institution. Findings from BHSP implementation clearly underscored the importance of PMO-RALG to effectively assume its coordination position. The fact that PMO-RALG has realized this fact and started working taking the driver s seat is a good start. Quality of Routine Data: The HMIS is the most relied upon source of data for monitoring and evaluating health system performance especially in the absence of frequent surveys. Many strategies have been successfully implemented aimed to improve data quality. However, data completion, delayed reporting and data inaccuracies are still common thus limiting data analysis and interpretation. Most of the strategies for improvement have been directed at the national and Council levels while the biggest challenge is at the health facility level where data are generated. Investment on strengthening the routine data systems at HF level might seem to be expensive, due to large numbers of health facilities and personnel, but the opportunity cost is comparatively more expensive. The first step in addressing this challenge has been undertaken, that is, the introduction of the data quality assessment tool and using it as one of the criteria for assessing achievements in RBF. This step needs to be followed now by the strengthening of HF workers capacity to analyze; interpret and use of data to improve quality of service. Evidence based decision making: Concerted efforts have been directed to strengthening routine information systems (HMIS, HR Information System, etc.) that enabled identifying problems in various health systems areas. However, routine data are inadequate in responding to why questions that normally requires in-depth information to address chronic problems. To date, less priority has been given to strengthening HSR that incorporates operational and implementation research thus M&E does not take advantage of the potential synergy obtained through the combination of routine and HSR data. The two sources synergize to respond to the why questions that are common in policy and decision making meetings. In the absence of evidence based responses it took several decades to restructure and strengthen HRH capacity in PMO-RALG with the goal to accelerate decentralization. In the absence of in-depth analyses, the inadequate PMO- RALG structural and capacity weaknesses to spearhead changes escaped the eye of decision makers for years to the detriment of the decentralization progress. 57

72 Annex 6. Comments of Co-financiers and Other Partners/Stakeholders 58

73 Annex 7. Disbursement mechanism and allocation conditions of HBF Source: MOHSW and PMO-RALG, 2015 Disbursement Mechanisms for Health Basket Funds to Councils a) PMO RALG in collaboration with MOHSW recommends to the BFC, CCHPs and/or in progress reports for approval and funding. b) BFC will approve transfer of funds from the Holding Account to Councils through Treasury, based on the recommended list of Councils compiled by PMO-RALG and MOHSW. c) The PMO RALG prepares a request to the Accountant General to transfer the approved amount of funds from the Health Basket Funds holding account into the Exchequer Account. MOF will notify PMO-RALG and the BFC on the released of funds to Councils in writing. d) The Accountant General facilitates the transfer of funds from Exchequer Account to the respective Council s Development account. The Council Treasurer must immediately inform the Council Medical Officer in writing that funds is available in the Councils account. Auditors have to verify compliance to this requirement. e) PMO RALG notifies Councils with a copy to MOHSW, RS and Basket Partners on the amount of Health Basket Funds transferred for records, follow up and comparison with the approved budgets. f) The Disbursement of 1st quarter (July September) funds depends upon submission of satisfactory January - March technical and financial progress reports and approved Comprehensive Council Health Plan for the current year. g) The disbursement of 2nd quarter (October December) funds is subject correct and timely submission of the Annual Council s technical and financial reports of the previous year (July June). h) Disbursement of the 3rd quarter (January March) funds to the Council subject to correct and timely submission of the first quarter (July September) technical and financial report that also reports on progress made to attain expected outputs for that quarter. i) The disbursement of 4th quarter (April June) funds will depend on October December technical and financial reports. j) Once the Council has received funds should publish on the Council and Health facility Notice Boards according to cost center for transparency and accountability. 59

74 Resource allocation formula for Health Block Grants and Health Basket Funds 128. The annual allocation of funds from Central Government to the LGAs is based on a resources allocation formula that is used to distribute the Health Basket Funds and Health Block Grant. It takes into account the allocation factors: population (70 percent); poverty count (10 percent); district medical vehicle route (10 percent); and under-five mortality (10 percent). a) In recognition of the individual as the main client-recipient of health and social welfare services, 70 percent of the Health Block Grant and the Health Basket are distributed in proportion to the population of each council. In addition to the overall population, districts receive additional resources for the special needs of a poor population (10 percent of the grant resources), the special needs of rural population and the needs of local governments with a higher Burden of Disease. b) The formula recognizes the higher expenditure needs of rural areas by directing 10 percent for the route mileage regularly travelled by health sector vehicles. As such, the formula takes into account the higher operational cost of delivering health services to a rural population and to sparsely populated areas; including higher costs faced in drug distribution, immunization and supervision. c) The formula also aims at directing resources (10 percent) to places with high burden of diseases; here the under-five mortality is considered as an appropriate proxy for burden of diseases. d) The funds allocated through the resources allocation formula is translated into action through the CCHP and its six cost centers. It is important to note that the resource allocation formula and the cost centers have no direct relationship. 13 priority interventions: (i) medicines, medical equipment, medical and diagnostic supplies management system; (ii) maternal, newborn and child health; (iii) communicable diseases; (iv) non-communicable diseases; (v) treatment and care of other common diseases of local priority within the council; (vi) environmental health and sanitation; (vii) strengthen social welfare and social protection services; (viii) strengthen HRH and social welfare management capacity for improved health services delivery; (ix) strengthen organizational structures and institutional management at all levels; (x) emergency preparedness and response; (xi) health promotion and behavior change communication; (xii) traditional medicines and alternative healing; and (xiii) construction, rehabilitation and planned preventive maintenance of physical infrastructures of health facilities. 6 allocation ceilings for the Health Basket Fund and Block Grant (OC) 60

75 Annex 8. Issues with results framework indicators Issues with the PDO indicators PDO Indicator Main Remark issue 1. Births attended in health facility broken down by LGA (percent) Unreliable data 2. Average outpatient attendances per clinical staff broken down by LGA (number) 3. Average outpatient attendances per capita broken down by LGA (number) Unstable indicator. Data broken down by LGA not tracked. Unreliable data. Data broken down by LGA not tracked. Description in the PAD: total number of expected births divided by the number taking place in health facilities it should have been the other way round. Baseline data and interim results are national averages, not broken down by LGA as the description states. And without data available by LGA, changes in inequity of access cannot be measured. Baseline of 58.4 percent does not match with DHS 2010 data of 50.2 percent. Interim results reported from HMIS may be incorrect as the percentages indicate achievement against target 82. Baseline was 1,395 and the target 1,600, i.e., increasing the patient load. In 2008, performance audit by the Comptroller and Auditor General concluded that at the local level health facilities are not efficiently managed and are funded without proper consideration of service demands and performance. Specifically, it found that 80 percent of clinical staff members of the selected sample of health centers and dispensaries see fewer than 10 patients per day. Hence, the need to increase productivity and efficiency. In retrospect, the aim to increase the patient load appears to be impractical due to the shortages of HRH for which implementation was affected. In the ISRs filed in June and December 2015, the baseline is indicated as 4,197 as the TTL who prepared the ISR observed that the PAD baseline was incorrect. Also, the TTL remarked that the aim should have been to decrease patient load over the project period, not increase it. The source of the revised baseline data, however, is not indicated and since this was not formally documented through project restructuring, the ICR is not taking the revised baseline into account. Baseline in the PAD is 0.83 whereas the HSSP III results framework has baseline of 0.68 in 2008 and target as 0.80 in If the target is set at 0.80 for 2015 by HSSP III, it is highly unlikely that the baseline in 2011 was Baseline data and interim results are national averages, not broken down by LGA as the description states. And without data available by LGA, changes in inequity of access cannot be measured. 4. Health Problem The ISRs highlighted that it was not possible to report back on this 82 Percentages were calculated as actual number of births in facilities divided by the projected/targeted number of births at facilities 61

76 PDO Indicator facilities with any stock-outs of tracer medicines and vaccines according to CCHP (percent) 5. Ratio of the 10 best performing LGAs to the 10 worst performing LGAs in indicators 1 through 3 (ratio) Main issue with definition. Data broken down by LGA not tracked. Problem with definition. Remark indicator as the description did not specify which medicines to track. And it is impossible to ensure all health facilities have all tracer medicines and vaccines all the time. Data reported in the ISRs were not consistent with nationally representative survey data. Due to unclear definition and lack of clarity on the method of calculating the ratio, interim results are unstable and unreliable. Description in the PAD did not specify the 10 best performing and the 10 worst performing LGAs. The target was to decrease the ratio by 15 percent for all three PDO indicators. Any decrease could have been because, either the best performing ones are doing worse or the worst performing ones are doing better. And unless these are monitored separately, a declining ratio is meaningless. The aim should have been to ensure the best performing ones continue to do well while the worst performing ones improve and indicators monitored for specific LGAs. The ISR of June 2015 changed the baseline and target values without any formal restructuring process. Hence, these cannot be considered for the purpose of the ICR. Issues with data reported in the ISRs Significant variations in reported data. For example, for PDO indicator 1, ISRs filed in June and December 2015 reported an increase in births at facilities from 58% to 77.6% and then ISR of June 2016 reported a rate of 65.9%. The changes in reported data were not explained. Baseline values changed without project restructuring. For example, for PDO indicator 2, ISRs filed in June 2015, December 2015 and June 2016 used baseline of 4,197 instead of 1,395 (as indicated in the PAD). There was no project restructuring to revise the baseline. The ISR of October 2016 changed the baseline back to 1,395. No explanation provided in the ISRs for these changes. Similarly, for PDO indicator 5, ISRs filed in June 2015 and December 2015 revised baselines without a project restructuring. No explanation provided in the ISRs for these changes. Sudden increases in reported data not explained. For example, for intermediate outcome indicators 3 and 4, reported data increased by two to three folds within a span of six months. It could have been that the actual figures reported were changed from absolute to cumulative numbers but the ISRs do not include any explanation. Lack of quality data: 129. During project implementation, DHIS was in transition to DHIS2 which was rolled out across the country in October HMIS, a paper-based tool, was used by the facilities to collect data. These were then compiled at the district-level by data entry operators and inputted into DHIS2. With no third-party verification and/or quality audit 62

77 done of the HMIS/DHIS2, interim results were based on self-reported data. There were sudden spikes in the trends of the interim results documented in the Implementation Status and Results reports (ISRs) of the Bank, without clear justification/explanation (detailed out in Annex 8). Given the weak capacity of the LGAs to report back properly and the multiple sources of errors (at the data collection points, i.e., the facilities, as well as the LGA level during compilation), data generated by the DHIS2 were variable. The percentages reported for various indicators in the DHIS2 are calculated against targets/projected figures and hence, could not be used effectively. For example, the data on percentage of ANC visits of pregnant women in the DHIS2 has been calculated as actual number of pregnant women who availed ANC visits divided by the expected number of ANC visits, and not divided by the (projected) total number of pregnant women. Similarly, the percentage of births at health facilities was calculated as actual number of births at facility divided by the number of births at facility that they expected, rather than total number of births at facility divided by the total number of births. Therefore, some figures are more than 100 percent which is not possible. The proportion of institutional deliveries in Arusha City council was reported more than 100 percent by the DHIS2 data for 2015 but as per DHS , it was 54.8 percent (National Bureau of Statistics of Tanzania and ICF Macro, 2016). 63

78 Annex 9. (A) Systemic Deficiencies of MSD and (B) Steps initiated by MSD to address stock-outs 130. As per their website, the Medical Stores Department (MSD) was established by the Act of Parliament No.13 of 1993 as an autonomous department under the MOHSW to develop, maintain and manage an efficient and cost effective system of procurement, storage and distribution of approved medicines and medical supplies required for use by the public health services as the MOHSW may from time to time approve. The Board comprises of 9 members appointed every three years. The Chairman is appointed by the President of the United Republic of Tanzania and 8 members are appointed by the Minister responsible for MOHSW MSD procures medicines and medical supplies in bulk using the funds from the GOT and the HBF, while the GFATM donates through Vertical Programs following the GOT/MOHSW donation guidelines In 2014, the Overseas Development Institute (ODI) undertook a political economy analysis of the medicine stock-outs in Tanzania (Wales, 2014). It noted 41 percent of patients were not able to get the medicines they needed directly from a public health facility 83. In urban areas this usually meant paying a premium for essential medicines and in rural areas, where private facilities were fewer, it often indicated having to incur additional expenses for transport and medicine costs or alternatively, not getting the needed medicines at all (ODI 2014). The ODI report concluded that there is a general lack of oversight within the health sector as well as unclear lines of accountability, which facilitates the wastage of medicines and allows corruption to occur, including leakages of both funds and medicines. (A) Systemic Deficiencies of MSD 133. Some of the major findings of the CAG audit in 2011 were: MSD officials and the LGAs approved orders without verifying whether adequate budgets were available. MSD central warehouse was not able to maintain continuous delivery to its zonal offices. High percentage of stock-outs at all MSD zonal offices, averaging 35 percent. There was no evidence to indicate that drugs and medical supplies intended to be delivered by MSD reached the end users as record keeping at LGA and facility levels were poor. MSD zonal offices delivered drugs and medical supplies not ordered by the facilities. Significant increase in stock of expired drugs at MSD, 572 percent between FY2009/10 and FY2010/11. MSD accepted drugs that had less than 80 percent shelf life. 83 Based on a survey conducted by Twaweza which is an NGO in Tanzania 64

79 134. Many of these issues continued to persist as documented by the OIG audit in 2016 and Deloitte s review in Some of these issues were: Despite an average stock-out rate of 24 percent for essential items, the value of expired goods has grown by 37 percent over the past one year. In addition the Dormant, Obsolete and Slow Moving (DOS) items grew by 136 percent over the same period. As at June 2015, the MSD had over 11.7Bn worth of expired and DOS stock stored in its warehouse. (Deloitte, 2015). On average, 29 percent of goods were overstocked in June 2015 (Deloitte, 2015). Expired stock grew by 36.8 percent between FY2013/14 and FY2014/15 (Deloitte, 2015). Stock-outs of health products of varying magnitudes were documented at different levels of the supply chain caused by inadequate planning and coordination by key stakeholders (e.g. the MSD, MOHSW, and facilities) in making appropriate distribution decisions (Office of Inspector General, GFATM, 2016). Unexplained stock differences throughout the supply chain (OIG-GFATM, 2016): - Differences between stock dispatched by the GFATM Pooled Procurement Mechanism and recorded as received at MSD valued at US$1.55 million - Unexplained differences ranging between percent between the quantity of malaria and HIV test kits and anti-malaria medicines reported as delivered by the MSD and accounted for by the district medical offices/facilities. Distribution decisions by the Medical Stores Department are not always informed by requests from facilities (Office of Inspector General -GFATM, 2016): - Only 25 percent of zonal warehouse requests for antimalarial medicine were honored while Dar es Salaam had received 300 percent of its request in one period. Expired health commodities were noted at the different levels of the supply chain. While the magnitude of the expired drugs cannot be quantified due to the lack of required data, the value of expired drugs related to vertical programs at MSD increased by 220 percent (TShs 750 million to TShs 2.4 billion) in the years 2013/14 to 2014/15. TShs million of this amount relates to GFATM commodities (Office of Inspector General -GFATM, 2016). (B) Steps initiated by MSD to address stock-outs To address the issue of stock-outs, various measures have been taken by MSD. One, they have has started using framework agreements for procurement of essential medicines. Due to the requirements of the Public Procurement Act (PPA) 2011, MSD cannot issue a tendering document unless they have assurance of funds for procurement. This restricted MSD in initiating the procurement process before funds were allocated to them. Most of the essential drugs were procured using international competitive bidding process and supplies mostly were from Indian and Chinese manufacturers that had license issued by the Tanzania Food and Drug Authority (TFDA). Without a license, an international manufacturer cannot supply to Tanzania. Manufacturers can apply for TFDA license online which are then processed relatively quickly. The TFDA also has ascertained maximum retail prices (MRP) for the essential drugs (for each manufacturer) and the prices paid by MSD are within the MRP limits. The framework agreement will provide 65

80 flexibility to MSD as it is not obligated to procure quantity stated in the agreement. MSD is following a one-stage framework agreement process whereby they invite bids from qualified bidders for a certain quantity of medicines. The bidders quote the unit price and technical specification for the medicines. The bidders that are technically responsive enter into a framework agreement with MSD and the prices remain valid for 2 years. As and when MSD receives requests from facilities/lgas, MSD can issue a call-off for the required quantity at the agreed unit price. If there are no call-offs, MSD is not obligated to buy any quantity from the bidders. Given that MSD procures most of the essential medicines from international manufacturers, sufficient time needs to be considered for the shipments to arrive. Also, if the manufacturers do not have enough stock at the time of the call-off, additional time has to be considered for manufacturing the drugs Second, alongside the bulk procurement done by MSD centrally, they have initiated the process of setting up MSD outlets in the regional hospitals and zonal referral facilities. The health facilities will be able to directly buy essential drugs from these outlets to meet temporary shortages Third, the PPA 2011 is being revised by the GOT and specific provisions for procurement of health sector goods will be included. This is expected to streamline the processes and procedures. The PPA 2011 considers procurement of health sectors goods the same as procurement of all other goods. The bidding document is also the same for all goods. With the revision of PPA 2011, it is expected that a separate bidding document will be developed for procurement of health sector goods. 66

81 Annex 10. Achievement of indicators against target Source of data (for latest values): MOHSW PDO Baseline Target Latest Value Achievement Indicator 1: Births attended in health facility broken down by LGA (percent) Data 58.40% 69% 62.40% 38% Not achieved (<64%) Indicator 2: Average outpatient attendances per clinical staff broken down by LGA (number) Data 1,395 1,600 2, % Surpassed (>100%) Indicator 3: Average outpatient attendances per capita broken down by LGA (number) Data % Not achieved (<64%) Indicator 4: Health facilities with any stockouts of tracer medicines and vaccines according to CCHP (percent) Data 28% 5% 53% -109% Not achieved (<64%) Indicator 5: Ratio of the 10 best performing LGAs to the 10 worst performing LGAs in indicators 1 through 3 (ratio) Data For 1: % decrease For 1: % Surpassed (>100%) For 2: 15.0 For 2: % Not achieved (<64%) For 3: 8.0 For 3: 8.1-8% Not achieved (<64%) IO Baseline Target Latest Value Achievement Indicator 1: Children immunized (number) - core indicator Data 0 6,142,347 8,016, % Surpassed (>100%) Indicator 2: Children receiving a dose of Vitamin A (number) - core indicator Data 0 28,978,389 29,730, % Surpassed (>100%) Indicator 3: Direct project beneficiaries (of which female) (number and percent) - core indicator Data 0 37,118,980 48,605, % Surpassed (>100%) 50.50% 51.40% 102% Surpassed (>100%) Indicator 4: Pregnant women receiving antenatal care during a visit to a health provider (number) - core indicator Data 0 6,471,889 10,884, % Surpassed (>100%) Indicator 5: Health centers in each LGA that provide emergency obstetrical care (EMOC) (percent) 67

82 IO Baseline Target Latest Value Achievement Data Hospitals: % increase Hospitals: % Surpassed (>100%) Health Centers: 5.5 Health Centers: % Surpassed (>100%) Indicator 6: District Readiness for Service Delivery (score) Data TBD 20% increase 55% Cannot be assessed Indicator 7: Use of performance and equity indicators in LGA allocations and use of the approach developed under this project for service delivery grants in other sectors (degree) Data None Use of SDG approach by other sectors SDG not established. Equity based allocation revised in 2012 and since then used only for HBF Not achieved Indicator 8: LGAs with unqualified audit reports (percent) Data 32% 48% 86% 338% Surpassed (>100%) Indicator 9: LGAs which have implemented the new DMIS system (percent) Data 0% 70% 100% 143% Surpassed (>100%) Indicator 10: Council Health Services Boards (CHSB) that meet on a quarterly basis to discuss issues related to budget, CHF, rehabilitation, schedule of visits to lower facilities, staff houses, etc. (percent) Data 60% 86% 85% 96% Achieved (>85%) Indicator 11: LGAs which have implemented one or more community or social accountability mechanisms (score-card, posting financial or performance information, complaint line, enhanced community participation in health facility governance or public opinion survey) Data 0% 15% 21% 140% Surpassed (>100%) Indicator 12: Health personnel receiving training (number) - core indicator Data 0 25,500 32, % Surpassed (>100%) Indicator 13: Regional Health Management Teams that have implemented performance monitoring and verification initiatives (percent) Data 0% 35% 100% 286% Surpassed (>100%) 68

83 Annex 11: Data used for the results chain Data taken from SDI survey 2014 Indicator Drug availability (% drugs) 76.00% 60.30% Equipment availability (% facilities) 78.00% 83.50% Infrastructure availability (% facilities) 19.00% 50.00% Absence from facility (% providers) 21.00% 14.30% Adherence to clinical guidelines (% clinical guidelines) 35.00% 43.80% Diagnostic accuracy (% clinical cases) 57.00% 60.20% Other data used: Sl. Indicator Year Source of Data Pregnant women attending 4 or more 42.8% 50.7% DHS DHS ANC visits (note i) 2 Institutional deliveries 50.2% 60.2% DHS DHS 3 Deliveries by skilled attendant 50.6% 63.7% DHS DHS 4 Contraceptive prevalence rate (any 27.4% 45.0% DHS DHS modern method) 6 Children immunized (all basic 75.2% 75.2% DHS DHS vaccinations) 7 Availability of all basic client services (notes iii and iv) 73.0% 74.0% ( ) Author's estimate TSPA Notes: i) ANC coverage was 62% in ii) Vitamin A supplementation for 2010 is for children aged 6-59 months iii) Basic client services include outpatient curative care for sick children, child growth monitoring, facility-based child vaccination services, any modern method of family planning, ANC, and services for STI iv) Author's estimate is based on linear curve plotted on data from TSPA 2006 (72%) and (74%) Source of data: DHS is Demographic and Health Survey 2010 TSPA is Tanzania Service Provision Assessment done by MOHSW, National Bureau of Statistics and other organizations. 69

84 Annex 12. BHSP s fund flow arrangement 11 STRATEGIES Health Sector Strategic Plan III ( ): uses a sector-wide approach, implemented by public institutions (MOHSW, PMO-RALG, LGAs, other ministries) as well as non-government stakeholders at the central, regional & district levels 1: District health services 2: Referral hospital services 3: Central support 4: Human resources for health 5: Health care financing 6. Public private partnerships 7: Maternal, newborn and child health 8: Disease prevention and control 9: Emergency preparedness and response 10: Social welfare and social protection 11: Monitoring & evaluation and research IMPLEMENTATION CCHP: All councils prepare annual CCHPs in line with the HSSPIII, which are assessed and approved by MOHSW and PMO-RALG and then funds are released to the LGAs. Sources of funds include: Fund flowing through the Councils: (i) Health Block Grants/Council Grants; (ii) Health Basket Funds; (iii) Cost sharing money user fees, TIKA/CHF, NHIF, etc.; (iv) Local Government Development Grant; (v) Health Sector Development Grant; (vi) Council-own resources Other Funding: (i) Block Grant from MOHSW to Council Designated Hospitals; (ii) Bed and staff grant to VA Hospitals; (iii) MOHSW-Fund for medicines, equipment & supplies through MSD; (iv) Global Fund;( v) National Programs (EPI, NACP, etc.); (vi) NGO (Plan International, World Vision, etc.); (vii) Bi-lateral and multi-lateral DPs; (viii) Donations (cash/receipts in-kind) INSTITUTIONS LGAs: responsible for delivering public services (including health), funded and regulated by the central govt but for which provision is devolved by the sector ministries to the local govt WORLD BANK District Health Services: Funds running costs, for District and Municipal Council health services based on annual action plans. It provides a stable and predictable resource base for local councils, complementing the District Health Block Grant of GOT. Also provides funds for PMO-RALG and RHMTs to oversee implementation of the funds PMO-RALG: directly supervises the work of the LGAs Central Regulatory and Support Services: Provides funds for the MOHSW headquarters and other central organizations with central support functions MOHSW: responsible for the overall stewardship of the health sector, particularly policy development, strategic planning, resource mobilization, and M&E HBF is a joint funding mechanism for DPs, created in 1999, as part of the SWAp. An HBF account was maintained at the MOF to which DP contributions were deposited. Initial DPs were: Canada, Denmark, Ireland, Netherlands, SDC, KfW, WB, UNFPA, and UNICEF Direction of fund flow 70

85 Annex 13. List of Supporting Documents and People Consulted Binyaruka et. al., Effect of paying for performance on utilization, quality, and user costs of health services in Tanzania: a controlled before and after study Boak, E., and Ndaruhutse, S., (2011), The impact of sector-wide approaches: where from, where now and where to?, CfBT Education Trust Deloitte, (2015) Strategic Review of the Medical Stores Department of Tanzania Directorate of Policy and Planning, MOHSW, (2016) "Documentation of the Monitoring and Evaluation Strengthening Initiative (M&ESI) " Financing Agreement, (2012) Cr TZ, Basic Health Services Project between United Republic of Tanzania and International Development Association Haazen, D., (2012) Making Health Financing Work for Poor People in Tanzania, the World Bank Hoviyeh, et. al. (2015), Tanzania s Countdown to 2015: An analysis of two decades of progress and gaps for reproductive, maternal, newborn, and child health, to inform priorities for post-2015, The Lancet Global Health, June 2015 Implementation and Completion and Results Report (IDA IDA IDA IDA-H0710) on a Credit in the amount of SDR27.9 million (US$40 million equivalent); SDR40 million (US$60 million equivalent); SDR25.2 million (US$40 million equivalent) to the United Republic of Tanzania for Health Sector Development Project Phase II, Report No. ICR Implementation and Completion and Results Report (IDA IDA-40031) on a Credit in the amount of SDR102.7 million (US$155 million equivalent) to the United Republic of Tanzania for a Local Government Support Project, Report No. ICR International Development Association, (2004) Project Appraisal Document on a Proposed Credit in the amount of SDR35.6 million (US$52 million equivalent) to the United Republic of Tanzania for the Local Government Support Project, Report No TZ International Development Association, (2011) Project Appraisal Document on a Proposed Credit in the amount of SDR63.1 million (US$100 million equivalent) to the United Republic of Tanzania for a Basic Health Services Project, Report No TZ International Development Association, (2012) Program Appraisal Document on a Proposed Credit in the amount of SDR167.6 million (US$255 million equivalent) to the United Republic of Tanzania for an Urban Local Government Strengthening Program, Report No TZ International Development Association, (2015) Program Appraisal Document on a Proposed Credit in the amount of SDR 145 million (US$200 million equivalent) to the United Republic of Tanzania for the Strengthening Primary Health Care for Results Program, Report No TZ Memorandum of Understanding Between the Partners (Government of Tanzania and Development Partners) participating in the pooled fund ("Basket") of the Health Sector concerning The pooled funding for the Government of Tanzania s Health Sector Program based on the Second & Third HSSP, The Health Sector Medium Term Expenditure Framework of the MOHSW and PMO-RALG, and the CCHPs, July 1, June 30,

86 MOF, "Government Budget for Financial Year 2012/13", Citizens' Budget Edition MOF, "Government Budget for Financial Year 2013/14", Citizens' Budget Edition MOF, "Government Budget for Financial Year 2014/15", Citizens' Budget Edition MOHSW and PMO-RALG, (2011), Comprehensive Council Health Planning Guidelines 4th edition MOHSW and PMO-RALG, (2014), Summary Analysis of Comprehensive Council Health Plans Annual Performance Report 2013/14 MOHSW and PMO-RALG, (2015), Summary Analysis of Comprehensive Council Health Plans Annual Performance Report 2014/15 MOHSW P4P Assessment Team, (2013), Tanzania P4P Assessment Report MOHSW, "Health Sector Performance Profile Report 2010 Update" MOHSW, "Health Sector Performance Profile Report 2011 Update" MOHSW, "Health Sector Strategic Plan July 2015-June 2020 (HSSP IV)" MOHSW, (2013a), Annual Health Sector Performance Profile 2011/12 MOHSW, (2013b), Mid-term Analytical Review of Performance of the Health Sector Strategic Plan III MOHSW, (2013c), Special Audit in the use of Health Basket Grant by Districts in the years 2010/11 and 2011/12 MOHSW, (2014) Annual Joint Health Sector Review for , Summary Report of the Technical Review MOHSW, (2015) Pwani Pay for Performance (P4P) Pilot Report for January 2014-June 2015 MOHSW, Human Resource for Health and Social Welfare Strategic Plan MOHSW, National Bureau of Statistics, ICF International (2007) "Tanzania Service Provision Assessment Survey 2006" MOHSW, National Bureau of Statistics, ICF International (2016) "Tanzania Service Provision Assessment Survey " MTR presentation by MOHSW, October 23, 2013 Mujinja and Kida, (2014), Implications of Health Sector Reforms in Tanzania: Policies, Indicators and Accessibility to Health Services, HDR 2014: Background Paper No. 8, ESRF Discussion Paper 62 National Audit Office of Tanzania, (2011), Report of the Controller and Auditor General on Special Audit on Drugs Availability at Medical Stores Department for the Period From 30 June 2009 to 30 June 2011 National Audit Office, (2012) Report of the Controller and Auditor General on the Financial Statements of Basic Health Services Project IDA Credit 5029-TZ for the Year Ended 30th June

87 National Audit Office, (2013) Report of the Controller and Auditor General on the Financial Statements of Basic Health Services Project IDA Credit 5029-TZ for the Year Ended 30th June 2013 National Audit Office, (2014a) Annual General Report of the Controller and Auditor General on the Financial Statements of the Local Government Authorities for the Year Ended 30th June 2014 National Audit Office, (2014b) Report of the Controller and Auditor General on the Financial Statements of Basic Health Services Project IDA Credit 5029-TZ for the Year Ended 30th June 2014 National Audit Office, (2014c) Report of the Controller and Auditor General on the Financial Statements of the Ministry of Health and Social Welfare for the Year Ended 30th June 2014 National Audit Office, (2014d) Report of the Controller and Auditor General on the Financial Statements on Prime Minister's Office Regional Administration and Local Government (Vote 56) for the Year Ended 30th June 2014 National Audit Office, (2015) Report of the Controller and Auditor General on the Financial Statements of Basic Health Services Project IDA Credit 5029-TZ for the Year Ended 30th June 2015 National Audit Office, the Controller and Auditor General, (2013a) "Management Letter on the Financial Statements of the Health Basket Fund for the Year Ended 30th June 2013 National Audit Office, the Controller and Auditor General, (2013b) "Management Letter on the Financial Statements of the Medical Stores Department for the Year Ended 30th June 2013 National Bureau of Statistics of Tanzania and ICF Macro (2011), Demographic and Health Survey (DHS) 2010 National Bureau of Statistics of Tanzania and ICF Macro (2016), Demographic and Health Survey and Malaria Indicator Survey Key Indicators Office of the Inspector General, GFATM, (2016) Audit of Global Fund Grants to the United Republic of Tanzania (mainland) Operational Assessment of the Health Basket Fund with Recommendations for improved mechanisms in light of the need for a new MOU beyond 2015, June 2014 Pearson (2010), Impact Evaluation of the Sector Wide Approach (SWAp), Malawi, Ukaid PMO-RALG, (2012) "Local Government Development Grant System: Annual Report (July 2011-June 2012)" PMO-RALG, (2013) "Local Government Development Grant System: Annual Report (July 2012-June 2013)" Report of the Audit Sub-Committee of the health sector 2015/16, (Updated December 2015) Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2010/11 Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2011/12 Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2012/13 73

88 Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2013/14 Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2014/15 Side Agreement between Health Basket Financing Partners and the Government of Tanzania, Fiscal Year 2015/16 Wales, J., et.al. (2014), Stock-outs of essential medicines in Tanzania: A political economy approach to analyzing problems and identifying solutions, Overseas Development Institute (ODI) World Bank Group and African Economic Research Consortium, (2014) Tanzania 2014 Service Delivery Indicators Health Technical Report World Bank, (2011) Project Appraisal Document on a Proposed Credit in the Amount of SDR63.1 million to the United Republic of Tanzania for a Basic Health Services Project, Report No TZ World Bank, (2015) Tanzania Mainland Poverty Assessment People consulted Name Mariam Ally Fadhila Mkomy Richard Noah Daniel Mhando Azma Simba Scholastica Ndilanha Anna Nswille Regina Mutabuko Levina Kimaro Claud John Kumalija Laurean Rugambwa Bwanakunu Joseph Tesha Deo Mtasiwa Nassoro Shemzigwa Emmanuel Malangalila Shannon Young Nadia Hamel Kira Thomas Eimear McDermott Karen Kirsten Haveman Gregers Juel Jensen Olivier Praz Jacques Mader Agency BHSP Manager, MOHSW Social Accountability, MOHSW Social Accountability, MOHSW HCWM, MOHSW PFM, MOHSW Epidemiology, MOHSW PMU (Central), MOHSW District Health Services, MOHSW Accountant, MOHSW HBF Coordinator, MOHSW Head of HMIS, MOHSW Director General, Medical Stores Department Director of Finance and Planning, MSD Deputy Permanent Secretary, PO-RAGL HBF Coordinator, PO-RAGL Public Sector Strengthening (PS3) HSS Advisor, USAID Deputy Director (Children & Youth) Global Affairs Canada Health Advisor, Global Affairs Canada Irish Aid Irish Aid Danish Embassy Counsellor Finance, Danish Embassy Swiss Development Cooperation Swiss Development Cooperation 74

89 Name Carol Hannon Dominic Haazen Rekha Menon Yahya Ipuge Gisbert Joseph Kinyero Michael Okuny Petronella Vergeer Sangeeta Raja Agency Irish Aid Lead Health Specialist, World Bank Practice Manager, HNP, World Bank ETC, World Bank Senior Procurement Specialist, World Bank Senior FM Specialist, World Bank Senior Health Specialist, World Bank Formerly with the World Bank 75

90 76