Indonesia: Capacity Building in Urban Infrastructure Management Project

Transcription

1 Validation Report Reference Number: PCV: INO Project Number: Loan Number: 1572-INO May 2009 Indonesia: Capacity Building in Urban Infrastructure Management Project Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank CMS career mapping system CPCO central project coordination office EA executing agency MOHA Ministry of Home Affairs MPW Ministry of Public Works PBME project benefit monitoring and evaluation system PIU project implementation unit PCR project completion report PPMS project performance management system NOTE In this report, $ refers to US dollars. Key Words adb, benefit monitoring and evaluation, capacity building in urban infrastructure management Indonesia, human resources development, implementation support, improved human resources, institutional development, policy development, urban infrastructure, urban sector development Director R. B. Adhikari, Independent Evaluation Division 1, Independent Evaluation Department (IED) Team Leader J. S. Bayley, Evaluation Specialist, Independent Evaluation Division 1, IED Team Members M. O. Nuestro, Evaluation Officer, Independent Evaluation Division 1, IED B. Q. Cafirma, Evaluation Assistant, Independent Evaluation Division 1, IED In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT COMPLETION REPORT VALIDATION A. Basic Project Data PCR Validation Date: May 2009 Project Number: Loan Number: INO Approved Actual Project Name: Capacity Building in Urban Total Project Costs Infrastructure Management ($M): Project (SDR equivalent) Country: Indonesia Loan/Grant ($M): Sector: Multisector Total Cofinancing 0 0 ($M): ADB Financing ADF: 0 Borrower ($M): ($M): OCR: Beneficiaries ($M): 0 0 Cofinancers: None Others ($M): 0 0 Approval Date: 4 November 1997 Effectiveness Date: 4 March March 1998 Signing Date: 5 December 1997 Closing Date: 31 October November 2004 Project Officers: Location (HQ or RM): From (yr) To (yr) Appraisal A. Valsvik HQ November 1997 December 1998 Implementation K. Choe HQ December 1998 December 1999 C. Andrews RM December 2000 February 2003 C. Gregory HQ March 2003 August 2003 M. Senapaty HQ September 2003 December 2004 PCR P. van Klaveren HQ Validator: E. Breckner, Consultant Director: R. B. Adhikari, IED1 Quality Control Reviewer/Peer Reviewer: J. S. Bayley, Evaluation Specialist, IED1 ADB = Asian Development Bank, ADF = Asian Development Fund, HQ = headquarters, OCR = ordinary capital resources, IED1 = Independent Evaluation Division 1, INO = Indonesia, M = million, PCR = project completion report, RM = resident mission, SDR = special drawing rights, yr = year. B. Project Description (summarized from the RRP) (i) Rationale. At the time of project preparation, the demand for urban services was outstripping the sector s ability to deliver and sustain adequate levels of municipal services. Urban sector development, although occurring within a supportive policy framework, was constrained by local governments (a) lack of experience in urban infrastructure project preparation, funding, and implementation; (b) limitations in their organization, staffing, and operations; (c) shortage of professional manpower; and (d) lack of experience with decentralized approaches to service provision, public private partnerships, and financial management. Some cities, however, had been pioneering the development of new approaches that could serve as models. To support these new approaches, the Capacity Building in Urban Infrastructure Management Project (the Project) focused on human resources development, institutional development, and policy development. (ii) Impact. The expected impact of the Project was improved (a) living conditions, (b) public health standards, (c) urban environments, and (d) economic opportunities for Indonesia s urban population.

4 2 (iii) Objectives. The objectives of the Project were to help improve the ability of (a) local and provincial governments to deliver urban infrastructure services, (b) nongovernment institutions to participate in urban management, and (c) national agencies to provide advisory services and training programs. (iv) Components. The project had four components: (a) Component 1 focused on improving human resources through provision of (1) a career mapping system (CMS); and (2) post-graduate, undergraduate, and short-course education and training for urban managers, particularly at the local level. (b) Component 2 sought to strengthen sector institutions through (1) pilot projects for developing and testing improved operational arrangements in about 11 project cities, and (2) preparation and implementation of institutional development programs for selected agencies. (c) Component 3 prepared and implemented sector policy action plans. (d) Component 4 featured implementation support, including project performance monitoring and evaluation. C. Evaluation of Design and Implementation (PCR assessment and validation) (i) Relevance of design and formulation. The PCR discussed (a) the Government s strategy for urban sector development as stated in its Sixth Five-Year Development Plan, (b) the Government s current urban sector policy, and (c) the Asian Development Bank s (ADB) operational strategy for Indonesia at the time of appraisal. It did not, however, explicitly link these to the project design or address the relevance of design and formulation. In this context, the Validator notes that a more detailed analysis of the original project design should have been conducted since the Project was eventually rated partly successful. Such an analysis would have exposed the Project s overly complex design that entailed nine implementing institutions; a project area of 11 pilot cities spread throughout the archipelago; and work conducted at three levels of government (central, provincial, and local). Therefore, while the Project may have been relevant vis-à-vis national and ADB priorities, the design was not relevant to the challenges that the Government faced in its implementation. The PCR should have also noted that the project design did not take into account that career mapping should have occurred within the context of the entire civil service not just a portion of it. (ii) Project outputs. The PCR described the achieved outputs as follows: (a) Component 1. Under this component, (1) a CMS was established and applied in the pilot districts, but it has yet to be institutionalized for countrywide application; and (2) the education and training programs were satisfactorily implemented, significantly exceeding the number of targeted beneficiaries. (b) Component 2. Under this component, (1) urban development management information systems were developed and installed in all pilot cities; (2) institutional support was provided for central and provincial government agencies involved in urban management to monitor local government operations, prepare feasibility studies, and improve financial management systems; and (3) the new CMS was tested but not institutionalized, as it required a radical reform of the existing civil service structure. (c) Component 3. Under component 3, (1) the National Urban Development Strategy II was developed; (2) spatial planning methods, technical guidelines, and urban management information systems were developed and tested in Semarang, one of the pilot cities, and then disseminated to the other pilot cities; and (3) financial management and accounting systems were improved for local governments and local government water supply enterprises, and installed in water supply enterprises in all pilot cities. (d) Component 4. This component resulted in (1) the preparation of city profiles in six pilot provincial governments and 14 pilot cities (as opposed to 11 targeted in the original design in the report and recommendation of the President [RRP]), providing basic data related to

5 3 urban infrastructure services and outlines for preliminary capacity-building programs; and (2) preparation of a project performance management system (PPMS) to monitor key indicators of project performance at local and central levels as well as to assess the impact on the target institutions and degree of achieving end-of-project objectives. Monitoring reports were prepared annually. The Validator notes these achieved outputs but also that the PCR did not clearly address planned outputs that were not achieved. For example, under component 2, the Project was supposed to (a) pilot test systems, programs, and activities in the areas of organization, public private partnerships, staffing and career mapping, spatial planning, feasibility studies, urban infrastructure services, transport management, and financial management in the 11 cities selected; (b) establish management systems and programs for central training units of the Ministry of Public Works (MPW) and the Ministry of Home Affairs (MOHA); and (c) provide assistance to central and provincial government agencies for monitoring local government operations, monitoring and preparing feasibility studies, and financial management. The PCR failed to mention outputs for items a and b and only noted the monitoring activities in item c. It did not mention outputs related to public private partnerships or the central training units of MPW and MOHA. (iii) Project cost, disbursements, borrower contribution, and conformance to schedule (as relevant to project performance). (a) Project cost. The PCR noted that at appraisal, the total project cost was estimated at $70.00 million. In July 1998, the Project s cost was reduced to $58.00 million, with ADB s share estimated at $35.00 million and the Government s share at $23.00 million, due to the limited availability of counterpart funds and increased project costs in local currency terms. At project completion in December 2003, the Project s actual cost was $44.22 million, of which ADB financed $25.60 million (58%) and the Government $18.63 million (42%). The reasons for the lower project costs were (1) the Asian financial crisis, which resulted in substantial depreciation of the rupiah; and (2) nonachievement of some activities, such as finalization of National Urban Development Strategy II, and institutionalization of capacity-building products due to lack of counterpart funds. There was a large cost overrun in foreign consulting services (150% of the original allocation), while all the other categories were lower than estimated. (b) Disbursement. Total disbursements of the ADB loan at project completion amounted to $25.6 million, of which $11.9 million (46.5%) was for education and training and $8.2 million (32.2%) for consulting services. Delays in disbursements occurred over the entire project implementation period. Disbursements picked up in 2001 when institutional support activities commenced at the local level upon the establishment of local project implementation units (PIUs). The delays in disbursement were also attributed to insufficient coordination and cooperation among the various parties involved. (c) Project schedule. The Project was approved on 5 December 1997 and the loan declared effective on 4 March The loan closing date was originally set for 31 October 2002, but ADB extended this to 31 December In fact, the Government had an extension of the loan closing date by 22 months until 31 August 2004 to ensure implementation of the remaining activities. Major reasons for delays in project implementation were (1) counterpart fund constraints after the financial crisis; (2) dramatic changes in the national- and local-level political and administrative environment after enactment of decentralization laws of 1999; (3) a complicated coordination and decision-making process among the multiple agencies involved; and (4) a lack of clear operational procedures and definition of roles and responsibilities of each agency, including the central project coordination office (CPCO), as well as the project and subproject managers. Although the Midterm Review Mission proposed changes in the implementation arrangements, the Government s financial problems, compounded by the introduction of new decentralization initiatives, continuously hindered achievement of action plans and eventually resulted in a 1-year delay in completing the Project.

6 4 The Validator notes the Project s tumultuous implementation issues during a tumultuous time, but also notes that some problems faced by the Project, such as a complicated coordination and decision-making process, were inherent in the project design. The Validator also notes that a 1-year delay in closing is not unusual, but it may have been better to keep the loan open for a longer period in order to achieve a greater number of intended outputs and outcomes. (iv) Implementation arrangements, conditions and covenants, related technical assistance, and procurement and consultant performance. (a) Implementation arrangements. The PCR noted that project implementation arrangements designed during appraisal were basically followed. MPW was the executing agency (EA), and MPW and MOHA were the principal implementing agencies. The central PIUs were created by the main implementing offices at the central level to manage their activities under the Project. Local PIUs were established in the participating provincial and local governments to manage project activities in the pilot areas. However, the PCR also noted that the local PIUs in the selected pilot areas were not established until the beginning of 2001, a delay from January 1999, because necessary decrees could not be issued by the mayors and district heads in the absence of instructions from MOHA and of counterpart funds. Furthermore, provincial PIUs were never established due to an unclear definition of the role of provincial governments under the new decentralized structure. The PCR further related that given the number of participating agencies and project management units involved in the Project, the CPCO encountered difficulties in coordination and decision making. To ensure integration of project activities among the principal implementing agencies, the project management structure was changed based on the October 2000 Midterm Review Mission s recommendations. Following changes in government administrative arrangements, the EA was changed three times. These frequent changes and a lack of well-defined policies and operating procedures, particularly with regard to financial management, adversely affected project management and coordination across the various implementing agencies. The Validator notes that the PCR s statement on implementation arrangements is not consistent with what actually transpired. It is clear that the project implementation arrangements designed during appraisal were not workable and had to be revised. Even after revision, they did not work particularly well. The Validator also notes that the new decentralized government changes caused further inefficiencies. (b) Conditions and covenants. From a total of 31 covenants, 22 covenants were fully complied with, 8 were partly compiled with, and the remaining 1 was waived. Covenants partly complied with were those affected by issues related to counterpart funding, establishment of project implementation offices, implementation of the CMS, pilot testing of institutional programs, and policy support. Those covenants were influenced by coordination difficulties among implementing agencies due to administrative changes, and delays in allocating annual project budgets on the part of central and local governments. The Validator notes that compliance with these covenants was central to the achievement of the Project objectives. The Validator agrees with the PCR assessment mentioned above and notes that (1) students in project schools did not receive core textbooks until mid-2006; and (2) although some complied-with covenants were delayed, the PCR did not mention this information. (c) Technical assistance. No advisory technical assistance was associated with the Project during implementation. (d) Procurement and consultant performance. In terms of procurement, the PCR noted that a total of 50 packages of consulting services were approved, consisting of 1,742

7 5 person-months. All packages were engaged in accordance with ADB s Guidelines on the Use of Consultants and government procedures. However, there were many delays in engaging consultants, mainly due to the government tender committee s limited understanding of how ADB s guidelines differed from the Government s regulations. In particular, problems occurred when ADB s guidelines were revised in April 2002, and the revised version was applied retroactively. Moreover, only a very small amount ($100,000) was allocated for civil works, while slightly more was allocated for equipment and materials ($1.1 million), although there does not seem to have been any problem with this procurement. In this context, the Validator points out that a large procurement item costing $11.9 million (about 46.5% of project cost) consisted of education and training. The PCR was not clear about how these services were procured and if there was any particular problem or issue involved. The PCR also noted that the consultants performance was generally satisfactory. The turnover of consulting experts was less than 20%. A common problem that affected overall project activities and effectiveness was lack of integration and coordination among activities under different contracts. While this would not be unusual for projects or programs implemented by several different institutions through different packages, each consultant should have related more closely with the others to achieve the common project objectives in a more coordinated manner. (v) Performance of the Borrower and Executing Agency. The PCR assessed the performance of the Borrower and EA as partly satisfactory, noting that it was affected by various factors that were mostly beyond their control. The EA was changed three times during the project period, following successive government administrative restructuring. All these changes also involved changes in key personnel within the EA. Steering committee members also changed whenever the EA was changed, adversely affecting project implementation. Moreover, the large number of implementing institutions as many as nine with a corresponding number of project managers and technical teams for monitoring the performance of consultants and providing guidance, made coordination more difficult. This was particularly true for the CPCO, which was primarily responsible for coordination among all the parties concerned. The difficulty was compounded by a lack of clear procedures relating to responsibilities and decision-making authority among the CPCO, project managers, and technical teams. In 2000, the CPCO s structure was changed to strengthen its coordination and decision-making authority. The multiplicity of implementing agencies and especially since financial and implementation management activities were the responsibility of different administrations was one of the main factors causing lack of motivation and accountability, and resulted in project implementation delays. In addition, the financial crisis made it difficult for the Government to provide sufficient counterpart funds for the Project. The Government, in the end, however, financed Rp158.3 billion, far exceeding the Rp83.5 billion originally estimated as the counterpart portion. It is difficult for the Validator to assess the performance since it appears that the main reason for implementation issues was an inappropriate and overly ambitious project design and implementation arrangements as well as exogenous factors such as the financial crisis and the Government s decentralization drive. Certainly, the Government could have taken a stronger initiative to resolve all of the administrative and implementation matters, but a large part of the problem evolved from the nature of the Project itself and factors beyond the Government s control. Indeed, at a certain level, the Government seems to have shown a willingness to resolve these issues and make the Project more productive by (a) suggesting a further project extension of 1 year, and (b) funding nearly double its original estimated counterpart allocation. On balance, however, the Validator cannot find a compelling reason to change the partly satisfactory rating. (vi) Performance of the Asian Development Bank. The PCR rated the performance of ADB as satisfactory. In giving this rating, it noted that from March 1997 to June 1998, a total of 10 ADB

8 6 missions (226 person-days) were fielded. However, the Validator notes that this statement does not correspond with the data provided (PCR, section D, p. v), which indicates 11 missions between March 1997 and December 2005 including fact-finding, appraisal, special project administration, and PCR missions. Therefore, supervision missions from inception to final review only numbered seven over a 6-year period, while the accepted norm is two review missions per year. In addition, many missions were very short; four of the seven were for less than 1 week and two of these were only for 4 days. This does not seem adequate for a complex project located throughout the Indonesian archipelago at three different levels of government. Nonetheless, the PCR noted that ADB review missions identified major issues related to the country situation and governmental restructuring and recommended adjustments needed for project implementation. In addition, difficulties encountered by the Project were addressed by the ADB review missions in a timely manner. The PCR further noted that the October 2000 Midterm Review Mission discussed with the EA the overall program and implementation schedule and agreed to a time-bound action plan to minimize delays in implementation and other problems. (This was done in the space of a mere nine days). The PCR also noted that in addition to these missions, continued support and assistance to the EAs to assist them in complying with ADB procedures and procurement regulations (mainly through the Indonesia Resident Mission) were effective. The Validator notes ADB s flexibility in the presence of a changing political and financial environment and the support provided by the resident mission. However, on balance, the Validator finds ADB s performance only partly satisfactory due to (a) the overly complex and ambitious project design that should have been identified during the project processing phase and appropriate changes made to ensure implementability; (b) inadequate project implementation supervision for such a complex and important project; and (c) retroactively applied revised consultant guidelines, which caused delays and confusion in recruiting consultant services. D. Evaluation of Performance (PCR assessment and validation) (i) Relevance. The PCR rated the Project as relevant for its consistency with the Government s sector strategy and ADB s strategy for Indonesia, including its urban sector strategy. The PCR then referred to the Government s current policy and strategies for urban development, as given in the midterm development plan and strategic plan, as well as ADB s current country strategies, given in the country strategy and program for The Validator notes that while the guidelines ask for relevance in both the pre- and post-project situations, a supportive argument for a project approved in 1997 cannot be based on a document for the period A more substantive argument would have been to relate the relevance of the project to the Government s strategy for urban sector development as stated in its Sixth Five-Year Development Plan, which it did in paras. 4 to 8 earlier in the report. On this basis, the Validator agrees with the PCR rating as relevant. (ii) Effectiveness in achieving outcomes. The PCR rated the Project less effective. The reasons given are that while the education and training programs were satisfactorily implemented, many of the expected capacity-building programs were not adopted and institutionalized by the Government. However, the PCR then failed to justify a less effective rating by providing specific details. Instead, it noted (a) improved capacity of provincial and local governments for urban management, and increased institutional effectiveness and efficiency in the pilot project areas; (b) notable improvements in urban infrastructure and services investments in the pilot areas; involvement of public private and community partnerships; resource revenues; maintenance budgets for roads, drains, and other urban infrastructure; and cost recovery for solid waste; (c) the newly-established financial management system being utilized by the participating local governments and water service companies; (d) enhanced private sector and community participation in planning, policy making, and budgeting through the Project s urban development coordination forums; and (e) increased participation of nongovernment institutions.

9 7 The Validator notes that no concrete evidence was given for these listed achievements but also notes that many of the expected outputs, especially in components 2 and 3, have not been institutionalized within the agencies involved. As such, the expected outcomes are unlikely and the Validator thus agrees with the less effective rating. (iii) Efficiency in achieving outcomes and outputs. The PCR rated the Project efficient in financial and economic terms. Almost all activities were carried out, and the outputs, as revised in July 1998 and during the midterm review, were attained at a cost substantially less than estimated at appraisal ($44.2 million rather than $70.0 million). The implementation of incountry education and training programs was particularly cost-effective. Urban management master's degrees provided at domestic institutions, for example, required an average of $200 per student-month, compared with $600 per student-month originally envisaged. The use of senior government officials as trainers for training was also cost-effective. The Validator agrees with this analysis and the rating as efficient but notes that a more accurate original cost estimate might have saved considerable effort involved in redesigning and re-costing the Project. The Validator also notes that the Government provided a significantly larger contribution to the project cost than was originally envisaged, funding in the end 42% rather than 28%. (iv) Preliminary assessment of sustainability. Overall, the PCR rated the Project as less likely to be sustainable. The Project achieved one of its primary objectives to improve the ability of 14 pilot local governments to deliver urban infrastructure services through strengthening human resources and institutional arrangements. However, practices and new approaches are yet to be disseminated and replicated to other local governments, and they are unlikely to be since the capacity-building instruments developed under the Project contain functions similar to those in other guidelines, causing confusion for local governments. Based on the reasons given, the Validator agrees with the less likely sustainability rating. (v) Impacts (both intended and unintended). The PCR noted that the Project, through its human resources development, training, and pilot-testing activities, will eventually benefit the population (23.4 million) of the 14 pilot cities through improved provision of urban services. It has improved environmental management and service provisions in these cities and helped expand urban infrastructure to the urban poor, women, and children. In respect to the Project s objective to put in place a countrywide, long-term capacity-building strategy for these institutions, the results were mixed, as the capacity-building initiatives developed under the Project are yet to be institutionalized and applied countrywide and are unlikely to be in the foreseeable future. The Validator agrees with this assessment and considers that the Project had a modest, but positive, impact. E. Overall Assessment, Lessons, and Recommendations (validation of PCR assessment) (i) Overall assessment. Based on the assessment of relevance, effectiveness, and efficiency, the PCR rated the Project partly successful. It noted that the Project achieved its main objective of improving the ability of the 14 pilot local governments to deliver improved urban infrastructure services through human resources development programs. However, given a fluid political environment and frequent changes in government administrative systems, the capacity-building models developed under the Project are yet to be fully institutionalized and applied countrywide. The sustainability of project facilities and instruments appears weak, as it depends on the capacities of individual leaders at the central and local levels. Accordingly, the Project s long-term goal to improve the living conditions, public health standards, urban environments, and economic opportunities for the urban population through improved institutional capacity of provincial and local governments will not be achieved to the extent envisaged at appraisal. The Validator concurs with the partly successful rating. (ii) Lessons. The PCR provided a long list of lessons learned. The most notable are (a) the project design and implementation schedule should be detailed and logical to avoid confusion;

10 8 (b) clear operating procedures and instructions relating to responsibilities and decision-making authority among all the parties concerned should be issued before project implementation; (c) sufficient time should be allowed for implementing a capacity-building project, particularly when the so-called new paradigm of decentralization is to be established in a dynamic and still unsettled environment; (d) the roles and responsibilities of various offices and committees temporarily established for project implementation should be formally authorized to ensure they are able to fulfill their mandates without resistance from the established cadre; and (e) project implementation arrangements should be carefully prepared, as the large number of implementing institutions and corresponding project managers and technical teams for monitoring and guiding the consultants, made coordination more difficult. The Validator considers these lessons to be relevant and adds that (a) project proposals should be prima facie viable and implementable the complex and overly ambitious design of this Project calls into question the viability of its original formulation, and (b) difficult and ambitious projects should be provided adequate support and supervision the sparse attention provided this Project by ADB (based on the number and length of review missions) raises questions about the institution s commitment to project quality. (iii) Recommendations. The most critical recommendation in the PCR relates to the need for further assistance to the sector. The PCR considered that continued assistance to the Government in its effort to spread capacity building in urban infrastructure management activities and processes to other local governments is required to achieve the Project s longterm objective, which remains valid. Any delays in ensuing initiatives will halt the momentum built by the Project (and especially in some committed local governments). That momentum would support other urban development programs and projects that have been prepared. The Validator supports this recommendation and adds that any further initiatives should be realistic, feasible, and have the ability to be implemented. F. Monitoring and Evaluation Design, Implementation, and Utilization (PCR assessment and validation) The RRP included a project framework with monitoring indicators. It also provided for the establishment of PPMS under the CPCO and central project offices. The PCR noted that under the Project, both a project benefit monitoring and evaluation system (PBME) and the PPMS were established before implementation and that these were used to monitor achievement of project outputs and objectives. In this regard, the PCR reported that monitoring indicators were developed under the PPMS and used regularly, albeit with a late start. It also noted that results from the PBME and PPMS were used for the preparation of the PCR. In this context, the Validator notes that the updated project framework given in Appendix 1 of the PCR is useful and informative. G. Others (e.g., safeguards, including governance and anticorruption; fiduciary aspects; government assessment of the Project, as applicable) (PCR assessment and validation) The main issue raised by the PCR concerns governance and ownership. As noted, local PIUs in the selected pilot areas were not established until the beginning of 2001, a delay from January 1999, because necessary decrees could not be issued by mayors and district heads in the absence of instructions from MOHA and of counterpart funds. Provincial PIUs were never established. This situation was complicated from a financial point of view due to the financial crisis and the subsequent shortage of counterpart funds on the one hand, and the political upheaval and decentralization of government functions on the other. These factors need to be taken into account when considering the rating of the Project and any further assistance.

11 9 H. Ratings PCR IED Review Reason for Disagreement/Comments Relevance: Relevant Relevant Effectiveness in Less Effective Less Effective Achieving Outcome: Efficiency in Achieving Efficient Efficient Outcome and Outputs: Preliminary Less Likely Less Likely Assessment of Sustainability: Borrower and EA: Partly Satisfactory Partly Satisfactory Performance of ADB: Satisfactory Partly Satisfactory Impact: Overall Assessment: Quality of PCR: Modest (not specifically mentioned but implied) Partly Successful Modest Partly Successful Satisfactory I. Comments on Project Completion Report Quality The original project design was overly ambitious and complex, which should have been recognized during the approval process and remedied. In addition, supervision was inadequate, amounting to only seven missions of which two were for only 4 days each. In addition, revised guidelines on the recruitment of consultants were applied retroactively. The PCR is considered satisfactory. It clearly and frankly described the Project s history and the salient events and issues in line with the requirements of the PCR Guidelines. In some cases, however, it is not entirely internally consistent (e.g., in the discussion on effectiveness) but in general, it is clear and concise. Two weaknesses are noted: (i) a lack of a comprehensive analysis as to why the project was only partly successful, which, in the view of the Validator, was in large part due to its original overambitious design; and (ii) a critical assessment of ADB s performance, which, again in the view of the Validator, was partly satisfactory rather than satisfactory. The lessons identified were useful and the recommendations germane to the prevailing situation. J. Recommendations for Operations Evaluation Department Follow-Up No follow-up is necessary. The lessons learned from this Project should be taken into account if further activities are being considered in capacity building in the urban infrastructure sector in Indonesia or other developing member countries. K. Data Sources for Validation Data sources for validation included the PCR, project files, and RRP.

12 REGIONAL DEPARTMENT'S RESPONSE TO THE PROJECT COMPLETION REPORT VALIDATION REPORT On 17 November 2008, the Independent Evaluation Department (IED) received the following comments from the Southeast Asia Department: "We have reviewed the validation and discussed this with colleagues who were involved in PCR. We have no further remarks."