Completion Report. Project Number: Loan Number: 1467 December Pakistan: Bahawalpur Rural Development Project

Transcription

1 Completion Report Project Number: Loan Number: 1467 December 2008 Pakistan: Bahawalpur Rural Development Project

2 CURRENCY EQUIVALENTS Currency Unit Pakistani rupee (PR) At Appraisal At Project Completion (15 August 1996) (12 October 2007) PR1.00 = $ $ $1.00 = PRs PRs ABBREVIATIONS ADB Asian Development Bank BME benefit monitoring and evaluation BRDP Bahawalpur Rural Development Project CDA Cholistan Development Authority CWD Communication and Works Department DOWM Directorate of On-farm Water Management EIRR economic internal rate of return FIRR financial internal rate of return FMR farm-to-market road IsDB Islamic Development Bank LGRDD Local Government and Rural Development Department LSRS large-scale reconnaissance survey MEPCO Multan Electric Power Company M&E monitoring and evaluation NGO nongovernment organization O&M operation and maintenance P&DD Planning and Development Department PCC project coordination committee PCR project completion review PMU project management unit PSC project steering committee RRP report and recommendation of the President SDR special drawing rights SSI small-scale infrastructure VOC vehicle operating cost WAPDA Water and Power Development Authority WDC women's development center WUA water users association

3 NOTES (i) (ii) The fiscal year of the Government of Pakistan ends on 30 June. In this Report, "$" refers to US dollars. Vice-President X. Zhao, Operations 1 Director General J. Miranda, Central and West Asia Department (CWRD) Director D. Walton, Officer-in-Charge, Agriculture, Environment and Natural Resources Division, CWRD Team leader Team member S. Teoh, Natural Resources Economist, CWRD E. de Castro, Associate Project Analyst, CWRD

4 CONTENTS Page BASIC DATA MAP I. PROJECT DESCRIPTION 1 II. III. IV. EVALUATION OF DESIGN AND IMPLEMENTATION A. Relevance of Design and Formulation B. Project Outputs C. Project Costs D. Disbursements E. Project Schedule F. Implementation Arrangements G. Conditions and Covenants H. Consultant Recruitment and Procurement I. Performance of Consultants, Contractors, and Suppliers J. Performance of the Borrower and the Executing Agency K. Performance of the Asian Development Bank EVALUATION OF PERFORMANCE A. Relevance B. Effectiveness in Achieving Outcome C. Efficiency in Achieving Outcome and Outputs D. Preliminary Assessment of Sustainability E. Impact OVERALL ASSESSMENT AND RECOMMENDATIONS A. Overall Assessment B. Lessons Learned C. Recommendations APPENDIXES 1. Performance Indicators and Achievements Summary Evaluation of Rural Roads Project Costs ADB Disbursements by Year Project Implementation Schedule / Achievement of Targets by Year Status of Compliance with Loan Covenants Consultancy Inputs List of Equipment and Vehicles Summary Financial and Economic Analysis 39 i SUPPLEMENTARY APPENDIXES (available upon request) A. Detailed Evaluation of Rural Roads B. Detailed Financial and Economic Analysis

5 i BASIC DATA A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number B. Loan Data 1. Appraisal Date Started Date Completed 2. Loan Negotiations Date Started Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness In Loan Agreement Actual Number of Extensions 6. Closing Date In Loan Agreement Actual Number of Extensions 7. Terms of Loan Interest Rate Maturity (number of years) Grace Period (number of years) Pakistan 1467 Bahawalpur Rural Development Project Islamic Republic of Pakistan Planning and Development Department SDR26,070,000 PCR:PAK June July August August September January April June June October % per annum 35 10

6 ii 9. Disbursements a. Dates Initial Disbursement 23 September 1997 Effective Date 11 June 1997 Final Disbursement 12 October 2007 Original Closing Date 30 June 2003 Time Interval 122 months Time Interval 74 months b. Amount (in SDR 000) SDR26,070 US$38,000 Category or Subloan Original Allocation Last Revised Allocation Amount Canceled Amount Disbursed 01A Civil Works - Rural Roads 13,310 12,118 (1,192) 12,118 01B 01C Civil Works Watercourse Improvement Civil Works Small-Scale Infrastructure 2,538 3, ,449 1,990 2, ,026 01D Rural Electrification 2,381 2,381 2,381 02A 02B 02C 02D 02E Institutional Strengthening Equipment & Vehicles Institutional Strengthening Contract Management Staff Institutional Strengthening NGOs Institutional Strengthening Consulting Services Institutional Strengthening Studies (111) 369 2, (1,548) (230) Service Charge 1, (572) Unallocated 3,430 (3,430) 99 Imprest Fund Total SDR 26,070 22,906 (3,164) 22,906 Total US$ Equivalent 38,000 31,816 4,709 31, Local Costs (Financed) - Amount ($ million) Percent of Local Costs Percent of Total Cost 40.5

7 iii C. Project Data 1. Project Cost ($ million) Cost Appraisal Estimate Actual Foreign Exchange Cost Local Currency Cost Total Financing Plan ($ million) Cost Appraisal Estimate Actual Implementation Costs Borrower-Financed ADB-Financed Beneficiaries IsDB Total ADB = Asian Development Bank, IsDB = Islamic Development Bank. 3. Cost Breakdown by Project Component ($ million) Component Appraisal Estimate Actual Rural Roads Watercourse Improvement Small-Scale Infrastructure Rural Electrification Institutional Strengthening Total Base Cost Contingencies Service Charge During Construction Total Project Cost

8 iv 4. Project Schedule Item Appraisal Estimate Actual A. Date of Contract with Consultants Project Implementation Consultants March 1999 Benefit Monitoring and Evaluation Consultants March 1999 Contract Agreement Consultant January 1999 Engineering Design Consultant May 1999 B. Civil Works 1. Main Road a. Date of Award February 2001 b. Completion of Work December Farm-to-Market Road a. Date of Award August 1999 b. Completion of Work March Link Road a. Date of Award January 2000 b. Completion of Work March Watercourse Improvement a. Date of Award December 1998 b. Completion of Work June Rural Electrification a. Date of Award January 1999 b. Completion of Work October Small-scale Infrastructure a. Date of Award December 1998 b. Completion of Work February 2007 C. Equipment and Supplies 1. First Procurement September Last Procurement February 2007 D. Vehicles 1. First Procurement September Last Procurement October 2003 E. Other Milestones Reallocation of Loan Proceeds a. First September 2002 b. Second March 2004

9 v 5. Project Performance Report Ratings Implementation Period Development Objectives Ratings Implementation Progress From June 1998 to December 1998 Satisfactory Satisfactory From January 1999 to December 1999 Satisfactory Satisfactory From January 2000 to December 2000 Partly Satisfactory Satisfactory From January 2001 to December 2001 Satisfactory Satisfactory From January 2002 to December 2002 Satisfactory Satisfactory From January 2003 to December 2003 Satisfactory Satisfactory From January 2004 to December 2004 Satisfactory Satisfactory From January 2005 to December 2005 Satisfactory Satisfactory From January 2006 to December 2006 Satisfactory Satisfactory From January 2007 to October 2007 Satisfactory Satisfactory D. Data on Asian Development Bank Missions Name of Mission Date No. of Persons No. of Person-Days Specialization of Members Loan Fact-Finding Mission 7 19 Apr a, b, c, d, e Appraisal Mission 17 Jun 2 Jul a, f, g, h, i Inception Mission 9 15 Jul j, i Review Mission 1 23 Mar 1 Apr a, k Review Mission Nov a, l Special Administration Mission & 27 Jul m, k Special Administration Mission 3 12 Apr m Midterm Review Mission 9 24 Nov m, l Review Mission Sep m Review Mission May m Review Mission Nov m, k Special Administration Mission 5 6 Jul m Review Mission 6 20 Feb 2 Mar n, k, o, Project Completion Review Mission 7 21 July p, k, q, r a = senior project economist, b = social development specialist, c = young professional, d = civil engineer (consultant), e = rural development planner (consultant), f = project specialist, g = programs officer, h = senior counsel, i = project implementation officer, j = project administration unit head, k = project analyst, l = economist, m = senior project specialist, n = rural development specialist, o = disbursement analyst, p = natural resources economist, q = rural roads engineer (consultant), r = economist/financial analyst (consultant).

10 63 o 00'E 73 o 00'E o 71 00'E o 73 00'E 35 o 00'N AFGHANISTAN PAKISTAN Quetta Tarbela Dam 35 o 00'N ISLAMABAD Peshawar Mangla Dam Indus R. Jhelum R. Sargodha Lahore P U N J A B Bahawalpur NORTH-WEST FRONTIER Multan Sutlej R Jhelum Chenab R. Ravi R. o 30 00'N PAKISTAN BAHAWALPUR RURAL DEVELOPMENT PROJECT (as completed) Akuke Bahawalnagar Minchinabad Laleka o 30 00'N IRAN 26 o 00'N ARABIAN SEA B A L O C H I S T A N 63 o 00'E Karachi Indus R. S I N D H Hyderabad PROJECT LOCATION INDIA 73 o 00'E 26 o 00'N Indus River Uch Sharif Jhelum R. Lodhran Bahawalpur Samasata P U N J A B Khairpur Sutlej R Chistian Mandi Hasilpur Quraish Chak BAHAWALNAGAR Harunabad Faqirwali I N D I A HR B A L O C H I S T A N o 28 00'N S I N D H N Kilometers Indus River Chacharan Serdargarh Khan Bela Firoza Daihri Lianquatpur Miawali Khanpur Rajanpur Kot Samabad Rahimyar Khan Khairgarh Fort Rahimabad Sadiqabad Mithra Sanjarpur Bhagla Nawa Kot Walhar Nawankot RAHIMYAR KHAN Islagarh o 71 00'E Derawar Fort Hakra River Bed Ahmadpur East Miruwali Khangarh Dhori Bijnot Yazman BAHAWALPUR Ranhal Gaddul Nagra Rukanpur Chak Mirgah Marot Fort Abbas Newly Settled Area National Capital Provincial Capital District Capital City/Town Village Rural Electrification Schemes Small Scale Infrastructure Schemes Water Course Improvement Schemes Ahmadpur East-Yazman Main Road Upgrading Rural Road (Link Road and Farm to Market Road) River District Boundary Provincial Boundary International Boundary Boundaries are not necessarily authoritative. o 73 00'E o 28 00'N

11 I. PROJECT DESCRIPTION 1. The Bahawalpur Rural Development Project was designed for the now defunct 1 Bahawalpur Division of Punjab Province. At appraisal, this Division was considered one of the least developed divisions in Pakistan, with more than 50% of the population living below the poverty line. 2 The Project intended to target poor communities in newly settled areas within the division that were least served by infrastructure services and most isolated from social services. The poverty level in the newly settled areas was much higher approximately 80% and therefore needed urgent investment. The expected impact of the Project was poverty reduction in the project area. The expected outcome was increased rural incomes, employment, and quality of life through improvements in infrastructure services, agricultural production, and economic/market activities. The project scope, components, and targets in the report and recommendation of the President (RRP) were as follows: (i) Rural roads: improvement of about 100 existing roads totaling an estimated 400 kilometers (km), with emphasis on roads linking agricultural areas with markets. Targets were the widening of about 60 km of the existing road from Ahmadpur East Town to Yazman Town from a single-lane to a double-lane paved carriageway, upgrading of 10 existing farm-to-market roads (FMR), and upgrading of 90 existing link roads or tracks; (ii) Watercourse improvement: organization of about 480 water users associations (WUA), and improvement of 1,440 km of watercourses with an irrigated area of about 52,800 hectares (ha); (iii) Small-scale infrastructure (SSI): implementation of an estimated 450 works 3 identified by the communities; (iv) Rural electrification: connection of about 150 selected villages to existing transmission lines; and (v) Institutional strengthening: capacity building and institutional strengthening of government agencies and community organizations of project beneficiaries to (a) prepare an investment program based on participatory approaches, (b) implement the Project, (c) update and maintain databases relevant to the monitoring and evaluation (M&E) of the Project, and (d) establish an M&E system. 2. The Project was approved on 26 September 1996 for the amount of SDR26,070,000. It became effective on 11 June At appraisal, the total project cost was estimated at $64.8 million, of which Asian Development Bank (ADB) funding covered $38.0 million and an estimated $7.0 million was in parallel cofinancing from the Islamic Development Bank (IsDB). The IsDB cofinancing was to support the widening of the main road from Ahmadpur East Town to Yazman Town, and rural electrification activities. The planned closing date was 30 June 2003 and the actual closing date was 12 October Four extensions were requested by the Government and granted by ADB. 1 The "Division" level of administration was abolished during the local government reforms of Administrative powers at the division level were devolved to the district governments. There are three districts in the defunct Bahawalpur Division: Bahawalpur District, Bahawalnagar District, and Rahim Yar Khan District. 2 ADB Report and Recommendation of the President to the Board of Directors on a Proposed Loan to Pakistan for the Bahawalpur Rural Development Project. Manila. 3 Such as community buildings, simple drinking water systems, wells, tube-wells, ponds, feed processing units, small flour mills, oil pressing units, saw mills, fruit and forest nurseries, road drainage structures, and social facilities.

12 2 II. EVALUATION OF DESIGN AND IMPLEMENTATION A. Relevance of Design and Formulation 3. At appraisal, the Project s design in terms of expected impact (then goal) and outcome (then purpose), as stated in para. 1, was consistent with ADB s strategy in Pakistan and with the Government s objectives on rural poverty reduction. The location of the Project within Bahawalpur Division appears to have been appropriate as the poverty incidence for the Punjab Cotton/Wheat Zone, within which the project area falls, was 56% the second highest in the country. 4 In particular, the RRP made the case for targeting newly settled areas (primarily in Bahawalpur and Rahim Yar Khan Districts) between the more settled area adjacent to the Sutlej River to the north and the Cholistan Desert to the south. 5 However, there was also only limited analysis of the proposed investment mix and the extent to which investments would contribute toward the expected poverty reduction impact Targeting. At the start of project implementation, the project management unit (PMU), in consultation with the Executing Agency, the Planning and Development Department (P&DD) of the Government of Punjab, concluded that targeting the newly settled areas was not necessary as the poverty levels in the newly settled areas did not significantly differ from the division average poverty level of approximately 55%. 7 Therefore, the PMU did not specifically focus its activities on the newly settled areas. 5. Lessons. The RRP comprehensively documented lessons from similar agricultural and rural development projects and incorporated them into the design of the Project, especially lessons on beneficiary participation and operation and maintenance (O&M). B. Project Outputs 6. Project output achievements and indicators are presented in Appendix 1 using the project monitoring and development framework in the RRP as a basis for comparing planned and actual achievements. ADB revised the output targets twice, in September 2002 and in June The Government also revised output targets twice. The first revised Proforma-I 8 was approved in October 2000 and the second in December In most areas, physical project targets were surpassed. Details of outputs at completion, broken down by component, are as follows: 1. Rural Roads 7. The rural roads component upgraded 126 individual roads, with a total length of 574 km 9 against a planned total of 400 km. This comprised 89 (268 km) link roads, 36 (246 km) FMRs, 4 Household Consumption-Expenditure Survey of May As defined in Article 1 of the Loan Agreement dated 8 January For example, there is little discussion on the poverty reduction rationale for including a significant cost item in the Project, the widening of the 60 km existing farm-to-market road (FMR) from a single-lane to double-lane road. 7 PMU communication during Project Completion Review (PCR) Mission. No official ADB documentation records ADB concurrence to the change in targeting approach. 8 Proforma I of the Planning Commission of the Government of Pakistan approved by the Executive Committee of the Government of Pakistan s National Economic Council for the Project. 9 Discrepancies appear in the project management unit (PMU) PCR in respect of the total length of road constructed. Appendix 2 therein refers to the length of roads for which contracts were awarded as km, and Appendix 9 (Table 13) uses 587 km. However, the actual construction specified in Appendix 1 indicates that 574 km were completed.

13 3 and the widening of one (60 km) main road (between Yazman Town and Ahmadpur East Town) from one lane to two lanes. The largest number of upgraded roads were in Bahawalpur District (46), followed by Bahawalnagar District (43) and Rahim Yar Khan District (37). The main road is located in Bahawalpur District. Community organizations were formed only for link roads. 10 Requests from communities for an additional 93 roads were rejected because of problems with land acquisition (20 roads), the need to fell a large number of trees (7 roads), or insufficient poor households living in the subproject area (the selection criteria established at appraisal required 66% of households to be poor). Systematic analysis of poverty level of households is not available. Documentation to show consistent application of the selection criteria and systematic poverty targeting across all target villages is limited The quality of both link roads and FMRs built under the Project is considered superior to those built by other government agencies. Appendix 2 contains a summary of the rural roads evaluation, which includes technical observations on the quality of the road. 12 Farmers interviewed during the project completion review (PCR) Mission field visits reported tangible and intangible benefits from the road construction. No impact evaluation was carried out for the main road. A key aspect of the component that has not been accomplished is road maintenance. Neither routine nor periodic maintenance has taken place. 2. Watercourse Improvement 9. The Project improved 684 watercourses and established 684 WUAs, against the original target of 480. A total of 301 watercourses (44%) were in Bahawalpur District, 189 (28%) were in Bahawalnagar District, and 194 (28%) were in Rahim Yar Khan District. 10. Targeting and selection of watercourse improvement schemes, as with other schemes, was not done as rigorously as originally intended. However, for this component each scheme application prepared by the WUA showed a breakdown of WUA members by landholding size, which gave a general indication of the distribution of households. However, available data does not link poverty to landholding size. 13 Poverty data specific to the project area was not available and no assessment was made of the breakdown of WUA households by income. The PMU rejected 111 applications for watercourse improvement (representing 14% of all applications). The reasons for rejection included: (i) too few i.e., less than 66% potential beneficiaries were "smallholders" with less than 12.5 acres of land, (ii) the WUA failed to deposit the required share of construction costs, and (iii) the watercourse concerned was already lined and the application was for an extension The PCR Mission observed farmers reporting increases in yields and switching to higher-value crops such as sugarcane as a result of the investment. Most farmers interviewed still rely on other water sources, such as tube-wells. There is no routine O&M. The PMU views the watercourse improvement as successful, in particular with regard to the implementation 10 The widening of the main road was funded by the Islamic Development Bank. 11 Anecdotal evidence suggests that political connections partly influenced some of the selection of rural roads investment; however, there is no documentation to substantiate this. 12 Supplementary Appendix A contains the full evaluation and accompanying photo-documentation. 13 Farm income depends on several factors other than landholding size alone, including the landholding's proximity to reliable irrigation, soil fertility, drainage, and proximity to roads and markets. Use of 12.5 acres as poverty-based selection criteria is too large a landholding, and therefore more households would be considered poor. 14 However, the PCR Mission noted examples where project-improved watercourses successfully applied for extensions of watercourse improvements already carried out under the Project.

14 4 approach, 15 which ensured construction quality. This approach has since been adopted as the modus operandi of the National Program of Improvement of Watercourses in Pakistan. 3. Small-scale Infrastructure 12. A total of 564 SSI schemes were established against a planned target of 450. A total of 236 schemes (42%) were in Bahawalpur District, 148 (26%) were in Bahawalnagar District, and 180 (32%) were in Rahim Yar Khan District. Investments under the SSI component were largely commercially-oriented enterprises owned, managed, and operated by the community. These include agroprocessing, agricultural production, engineering workshops, and small-scale factories engaged in the manufacture of products such as leather goods and household goods. Together, these enterprises and manufacturing units number 400 (71%) of the 564 SSI schemes developed under the Project. The second major category of scheme is women's development centers (WDCs), which account for 19% of all schemes. WDCs provide skills training to community women mainly in sewing and embroidery, and operate on a commercial basis in the production and marketing of clothing, carpets, and other products. Schemes categorized as infrastructure by the PMU (water supply, culverts, and bridges) account for 10% of all schemes. 13. SSI selection criteria were: (i) majority of beneficiaries willing to form a formal community organization, (ii) community organization to agree in writing to contribute to the cost of the improvement and to carry out all subsequent O&M, and (iii) at least two thirds of the beneficiaries with incomes below the rural poverty line. The PMU rejected 2,425 of the scheme applications submitted for appraisal. Of those, 228 were rejected on technical grounds, and 2,197 were rejected on "social" grounds. As noted in paras. 8 and 11 above, there is inadequate documentation to show that the poverty criterion was consistently applied. In contrast to the selection criteria of the first two components (rural roads and watercourse improvement), there is no environment-related criterion. 14. The benefits from this component are varied, given the diverse nature of SSI schemes. Most schemes appear to be operated as small-scale enterprises, with the exception of the infrastructure schemes. Each of the enterprise-like SSIs has a limited number of active community organization members. Profits are shared amongst the community organization members. The main clients appear to be neighboring farmers and communities. A few of the WDCs have been successful (some through middlemen, some directly) in reaching markets in Lahore or Karachi for their products. 4. Rural Electrification 15. Under this component, 428 villages or settlements were electrified, against an initial target of 150 villages. This component was cofinanced by IsDB. The component electrified 168 villages in Bahawalpur District, 110 villages in Bahawalnagar District, and 150 villages in Rahim Yar Khan District. Electrification was carried out in two main phases, the first from and the second from 2003 to the end of the Project. These subprojects were carried out by the Multan Electric Power Company (MEPCO). The PMU rejected 78 applications for rural electrification on the basis that the households concerned were scattered or numbered less than 15 The disbursement of funds for watercourse improvement took place through installments tied to assessments of ongoing construction quality. The PMU also retained 10% of the total watercourse improvement cost until final certification of satisfactory construction by field teams. This system is considered to have adequately addressed the problems of poor construction quality in previous on-farm water management projects identified in the lessons learned at appraisal.

15 5 25. No villages appear to have been rejected for failing to meet the poverty criteria. It appears that poverty targeting for this component was not carried out effectively. The PMU coordinated with MEPCO to ensure that villages that had an opportunity to be electrified from government or other sources of funds would not be covered under the Project. In this way, the PMU deduced that the villages ultimately electrified through the Project did meet the poverty criteria. 16. One of the main benefits from electrification cited by communities visited 16 was the increase in the level of household comfort brought about by the use of electric fans, refrigerators, and kitchen appliances. 17 Electric lighting made it possible for some women to work longer hours outside the home during the day in addition to doing household chores in the evening. 18 Other benefits include extended hours for children s studies, extended use of productive machines such as fodder-cutting machines, and increased security in the villages. Communities visited by the PCR Mission showed a high level of appreciation for the subprojects as many settlements had been waiting a long time for electrification Institutional Strengthening 17. Institutional strengthening appears to have received the least attention of all the components. As originally envisaged, this component was supposed to build the capacity of government agencies and community-based organizations at the divisional and village levels to (i) prepare an investment program based on participatory approaches, (ii) implement the Project, (iii) update and maintain databases relevant to project M&E, and (iv) establish an M&E system. These activities were largely in the scope of work of the contracted nongovernment organization 20 (NGO). After the termination of the NGO's contract in June , the PMU took over the majority of these tasks. 18. An estimated total of 25, beneficiaries at the village and community organization levels were trained under the Project. Institutional strengthening of government line agencies was limited. In Bahawalnagar district, the Project provided training to Communications and Works Department (CWD) staff in laboratory and site testing. There is little evidence that the institutional strengthening component has enabled villages to prepare investment programs and access financing, as originally envisaged in the RRP. Output under this component is not considered satisfactory. C. Project Costs 19. The actual final cost of the Project was $48.2 million, compared with $64.8 million estimated during appraisal. The total cost comprised $12.3 million in foreign currency cost (compared with $16.3 million estimated at appraisal) and $35.9 million equivalent in local currency cost (against $48.5 million at appraisal). 16 PCR Mission in July The degree to which such appliances are used depends largely on income levels and the availability of funds to purchase relevant household appliances. 18 This suggests an increase in women s workloads. There is insufficient data to conclude whether women were better or worse off as a result of the extended work hours. 19 Villagers interviewed were generally of the view that larger villages in the constituency of influential persons were usually given priority for electrification. 20 The NGO contracted was the National Rural Support Program. 21 See para Approximately 1,215 trainees were trained in more formal courses; approximately 5,000 as part of community organization mobilization, and approximately 19,000 women were trained either directly by the Project or indirectly by master trainers trained by the Project. (PMU estimates).

16 6 20. In spite of the increase in component costs resulting from the expansion of project output targets of rural road construction, watercourse improvement, SSI, and rural electrification, the project cost at completion remains lower than the appraisal cost estimates. This was due to: (i) the depreciation of the Pakistan rupee (PR) against the US dollar ($) as shown in Table 1 below; (ii) a decrease in the beneficiary contribution by 50% because the communities were not requested to provide unpaid labor to help with the construction of the roads; and (iii) cancellation by ADB of potential loan savings of $3.0 million in 2004, and $1.7 million upon closure of the loan account in October Table 1: $:PR Exchange Rate during Project Implementation 15 Aug 96 (Appraisal) Sep 2002 Jun 2003 Apr 2004 Oct 2007 (Completion) $1.00 = PRs35.62 PRs60.15 PRs57.8 PRs PRs60.71 Source: Asian Development Bank estimates. 21. Loan proceeds were reallocated in September 2002 because of increases in (i) the length of link roads from 180 km to 230 km, (ii) the number of villages for rural electrification from 150 to 350, and (iii) the number of watercourses from 480 to 600. Another reallocation was done in March 2004 as the amounts of awarded contracts had exceeded the allocated amounts for two categories, rural electrification and consulting services. The actual costs by project component are compared to the cost estimate at appraisal in Appendix 3. D. Disbursements 22. Total loan disbursements were $31.8 million, or 84% of the approved loan amount. The undisbursed balance of $1.7 million at the loan closing date was cancelled on 12 October The foreign currency amount disbursed was $12.3 million, and the local currency amount disbursed was $19.5 million. The breakdown of ADB's yearly disbursements is in Appendix An imprest account was established at the National Bank of Pakistan with an initial advance of $1.2 million. The imprest account ceiling was increased to $2.2 million in November 1998, then reduced to $1.5 million in January 2001 and increased again to $2.0 million in September The turnover ratio measuring the imprest account s utilization reached an average of 3.09 in The imprest account was liquidated before loan closing. 24. A change in ADB financing percentages for the civil works' components (Rural Roads, Watercourse Improvement, and SSI) and Rural Electrification was approved in October 2002 to make them consistent with the project implementation arrangements and Government-approved financing in Proforma-I. 25. According to the Government project completion report, 23 the average length of time between the sending of a withdrawal application and ADB replenishment was 62 days; in six cases the interval was more than 3 months. One delay lasted 606 days. There were 3 months where the imprest account balance was negative and 8 months where it was less than $100,000. ADB records show that the delayed replenishment of the funds was caused by incomplete documentation accompanying the submitted withdrawal applications, such as summary sheets, supporting documents, bank statements, and reconciliation statements. 23 PMU Project Completion Report, para 10.

17 7 E. Project Schedule 26. The loan was approved on 26 September 1996 and became effective on 11 June It was designed as a 6-year project and was originally due for closure on 30 June Progress during the initial implementation stage in 1998 and 1999 was slow. This was mainly due to the delayed staffing of the PMU; delayed recruitment of consultants; and inconsistencies found in the original Proforma-I documents, which necessitated a revision and resubmission of the Proforma-I. Four loan extensions were requested by the Government and approved by ADB. 24 The actual implementation period was 8 years and 9 months. See Appendix 5 for the implementation schedule. F. Implementation Arrangements 27. Implementation arrangements at design are described in paras. 51 to 63 of the RRP. The PMU was established in Bahawalpur with the appointment of the project director in June The PMU established project offices in Bahawalnagar and Rahim Yar Khan. P&DD authorized the Project Director to use the power of Administrative Secretary. The PMU dealt directly with P&DD and was relatively isolated from other local government departments. This was viewed by PMU in a positive light as it facilitated efficient implementation of the Project. However, this meant that line agencies were only marginally involved, with the exception of the District s Department of On-farm Water Management, which was involved in the watercourse component. 28. In 2001, the Punjab Local Government Ordinance came into effect and devolved administrative, political, and fiscal powers to the grassroots level. 25 The immediate effect of the Ordinance on the Project was that the divisional administration led by the commissioner disappeared. The Project had to coordinate with the three new district governments separately. The Project remained operated centrally, in very close coordination with the district administrations Changes in implementation arrangements. Work on rural roads, the largest component of the Project, could not start until changes to the original implementation arrangements were made. The three implementing agencies specified i.e. CWD, the Cholistan Development Authority, and the Local Government and Rural Development Department (LGRDD) were unable to fulfill their specified roles. This, plus the failure to negotiate consulting packages for rural roads design and supervision (see para. 33 below), led to the establishment of the Mini-Highways Division within the PMU in February 1999, which thereafter undertook the functions of road design and supervision of the contractors. Other changes to implementation arrangements were necessitated by the termination of the main implementing NGO s contract. G. Conditions and Covenants 30. Most covenants of the Loan Agreement have been complied with in a timely manner (see Appendix 6 for the status of compliance) with the exception of the following: 24 The four extensions were granted on 31 June 2003, 2 April 2004, 17 February 2005, and 15 March Under this modality, the roles and responsibilities of different local actors changed and new functions, structures, and hierarchies were created in which local political aspirations were represented through the locally elected representatives. 26 Communication from ADB project officer.

18 8 (i) (ii) (iii) Late compliance. Appointment of agreed PMU staff took place after the targeted deadline of within 6 months of the effective date. The submission of the PCR which was due on 30 September 2007 was also late as it was only received by ADB in 27 May 2008; Partial compliance. Key covenants that were partly complied with relate to: (a) the steering committee which did not meet at least two times per year; (b) changes affecting contract management staff without prior concurrence of ADB as required; (c) incomplete submission of reports and Benefit Monitoring and Evaluation (BME) surveys after the departure of the BME consultant in 2005; and (d) the incomplete application of selection criteria of rural roads, watercourses, rural electrification, and SSI schemes; and (e) the implementation of the institutional strengthening without the assistance of the NGOs. With regard to the reporting requirements, the annual report for FY2005/2006 and quarterly and monthly reports for 2006 were not submitted. The PMU made efforts to apply selection criteria for the schemes, but the process is not sufficiently documented and seems to have been based more on a subjective reasoning than on objective targeting. 27 Non-compliance. The covenants pertaining to the establishment of the project coordination committee, and the maintenance of rural roads were not complied with. The project coordination committee was established but it never met. Budgetary funds were not allocated for the O&M of the roads. The only exception is found in , when the district CWD received an allocation for this purpose. However, according to the PMU, the funds were not used for the maintenance of rural roads constructed under the Project. Non-compliance of the road maintenance covenant has a direct impact on the sustainability and life of the rural roads constructed. 28 No maintenance has been carried out to date. H. Consultant Recruitment and Procurement 31. At appraisal, 840 person-months of domestic NGO inputs were to provide services in community organizations, institutional strengthening, and training. A summary of consultancy inputs is provided in Appendix In addition, about 124 person-months of domestic consulting packages for rural roads design and supervision, implementation, and BME were envisaged at appraisal, and a package to prepare simple standard contract agreements was also envisaged. A major problem in the recruitment of the rural roads design and supervision consultants was that the appraisal cost estimate was too low. The short-listed firms all submitted cost proposals three to five times greater than the cost allocation, and contract negotiations failed to lower these proposed costs. In September 1999, a change in implementation arrangements was approved to recruit individual consultants. The Mini-Highways Division created within the PMU was staffed with fulltime individual consultants Vehicles, equipment, and materials were procured by direct purchase in packages not exceeding $50,000. The NGO contract provided for the procurement of seven jeeps, which were procured through international shopping procedures. Vehicles and equipment procured through 27 The PMU and Planning and Development Department maintain that, in spite of the political pressure in terms of the selection of some schemes, the selection criteria have been consistently and objectively applied. 28 The reduction of the economic life of rural roads has been factored into the economic analysis (see Appendix 9). 29 Total of 470 person-months.

19 9 the NGO contract were turned over to the PMU after termination of the contract. Civil works for rural roads were awarded on the basis of local competitive bidding from among prequalified local contractors. The list of equipment and vehicles procured is in Appendix 8. I. Performance of Consultants, Contractors, and Suppliers 34. The services rendered by the domestic NGO were deemed unsatisfactory by the PMU, mainly because of differences in opinion on the approach for community mobilization. The contract was terminated in June The PMU s community development cell took over the training of village volunteers. However, it is not clear if the PMU had the capacity to take over all aspects of the NGO s original scope of work, particularly in respect of poverty targeting, application of selection criteria, and the formation of effective, sustainable community organizations. 35. The work done by the consultants staffing the PMU Mini-Highways Department is considered highly satisfactory. The work surpassed physical targets of rural road construction and the roads constructed were of superior quality. 36. Of the 44 contractors involved in the construction of 127 roads, 16 performed satisfactorily, 21 performed partly satisfactorily, and 7 performed unsatisfactorily. Nine contracts awarded to five contractors were terminated for various reasons. These contracts were retendered. Work on watercourses carried out by the District s Department of On-farm Water Management staff was satisfactory. MEPCO s performance was satisfactory. J. Performance of the Borrower and the Executing Agency 37. Supervision of P&DD relied on the mechanism of project steering committee (PSC) meetings in Lahore. Over the course of the Project, P&DD chaired eight PSC meetings. 30 PSC guidance focused on the physical outputs and financial management aspects of the Project. P&DD carried out a field visit in 2004 for project evaluation. All ADB review missions had interactions with P&DD in Lahore. 38. One of the few differences in ADB's otherwise smooth relationship with P&DD concerned the employment status of the senior contract management staff. Termination of the first project director in September 2001 and appointment of his successor was not done in consultation with ADB, and ADB duly registered its concerns as this was in violation of the Loan Agreement (see para. 31 above). The problems associated with selection and contract arrangements of senior contract management staff continued into This caused ADB in June 2002 to consider suspending the loan the only time during loan implementation that ADB considered such a step. However, it cannot be said conclusively that progress in achieving project outputs suffered because of these frequent senior staff changes. 39. The performance of the Executing Agency is rated "partly satisfactory" as a result of non-compliance or partial compliance of pertinent loan covenants, especially the covenant on the maintenance of rural roads. The performance of the PMU is rated "highly satisfactory" based on four out of five project components surpassing physical output targets. 30 PSC meetings were held on annual basis with the exception of 2004, when no meeting was held, and 2006, when the committee met twice.

20 10 K. Performance of the Asian Development Bank 40. Eleven ADB review missions were carried out during project implementation, starting from the inception mission in July 1997 to the review focusing on loan closure in February March Review missions focused on charting the progress of physical output targets and solving PMU implementation issues. The PMU recorded high appreciation of ADB review missions and ADB actions to solve problems encountered by the PMU. It seems that ADB's tracking of physical targets was effective, but it paid insufficient attention to the quality of the poverty targeting, subproject selection process and implementation, and the component on institutional strengthening. Notably, there was little mention of oversight on safeguard issues and limited investigation on the eventual poverty impact of the Project. Consistent scrutiny of compliance with loan covenants, especially in application of scheme selection criteria and O&M of rural roads, was not demonstrated. 41. The overall performance of ADB is rated "partly satisfactory", mainly because of ADB's failure to address and resolve non- or partial compliance with some pertinent loan covenants, including the maintenance of rural roads. III. EVALUATION OF PERFORMANCE A. Relevance 42. The Project is considered relevant. At completion, the Project s expected impact of rural poverty reduction is still relevant. However, the modality and approach of the Project would need to be revisited. Continuous grant-based livelihood projects, without adequate attention to markets, may encourage rural communities to continue to rely on the government s hand-outs rather than creating opportunities to participate in private sector growth. When both Government and ADB strategies in Pakistan are emphasizing increased participation of the private sector in infrastructure delivery and cost recovery from beneficiaries, the use of grant financing for between 80% and 100% of project investments with no subsequent cost recovery would likely be considered an unsustainable strategy. B. Effectiveness in Achieving Outcome 43. Overall the Project has been effective. The outcome (formerly purpose ) statement is to increase rural incomes, employment, and quality of life through improvements in infrastructure services, agricultural production, and economic/market activities. The main investment category of the Project was civil works. The agriculture production improvement and economic and market activities appear to have been rather sparsely and sporadically covered under the Project. 44. Targeting. 31 As described in para. 4, the Project did not narrow its focus on the newly settled areas. The PMU delegated responsibility for the selection of villages to the NGO responsible for community mobilization. The NGO focused on the 1,082 villages that had been covered by the large-scale reconnaissance survey (LSRS), 32 and for which the PMU had already obtained basic data including the number of households, existing infrastructure, farming 31 Since the Project was designed to target poor communities that were least served by infrastructure services and most isolated from social services, a discussion of the outcome also needs to include an analysis of the effectiveness in terms of targeting. 32 Assumed to be the poorer villages.

21 11 activities, and perceived community needs. At project completion, only 209 (31%) of the total of 684 watercourse improvement projects, 68 (16%) of 428 rural electrification schemes, and 115 (20%) of 564 SSI schemes were located in the LSRS villages. The location of schemes outside the LSRS villages was determined according to formal applications received by the PMU from villagers, resulting from the dissemination of information on the Project by the PMU's media campaign and word of mouth. There is an indication that some villages were selected on the basis of personal recommendation of local influential people and political representatives, but there is no documented evidence supporting these views. All non-lsrs villages were, however, assessed for eligibility by the PMU. There is a concern that some poorer communities may not have been aware of the Project at all (and hence did not make an application for a scheme) because of their isolation, lack of information, or lack of political connections. Hence, even if the selection criteria were applied by the PMU, there was no clear and consistent poverty targeting strategy. 45. Contribution of outputs to achieving the outcome. In general, the outputs contributed to the outcome. The rationale for widening a 60-km section of the Yazman to Ahmadpur East road is unclear from a poverty perspective. 33 While transport operator savings from link and FMRs may be passed on to the users of their services in the form of increased output and lower input prices and/or costs, the likelihood of the savings resulting from the upgrading of this main road being passed on is low. The road also appears to serve a relatively small section of the project area. 34 Women's groups involved in carpet making and sewing visited by the Mission reported a daily take-home income from the activity from PRs100 to PRs While some marketing links have been established, the women are clearly in a disadvantageous position. They have limited capacity to negotiate a better rate with the middlemen. In spite of this, the clean, dignified working conditions (the workshop building was financed from the Project funds), and the social aspect of working together are clearly non-tangible benefits. C. Efficiency in Achieving Outcome and Outputs 46. The Project is efficient. The analysis of the Project as a whole is based on the estimation of the weighted average of the economic internal rate of return (EIRR) of individual components. The weighting is based on the proportion of each component's economic costs within total project economic costs. The result is shown in Table 2. Table 2: Overall Project EIRR (%) Component EIRR (%) Rural Roads 15.8 Watercourse Improvement 14.8 Small Scale Infrastructure 8.8 Rural Electrification 12.9 EIRR (weighted average) 14.5 EIRR = economic internal rate of return. Source: Asian Development Bank estimates. 33 The PMU maintains that the widening of the main road has had a poverty impact, but it does not have any impact analysis to demonstrate this. PMU states that the main road generated a lot of economic opportunities and is being served for easy transport of agriculture inputs and outputs 34 Based on PMU Project Completion Report estimates of an average cost of PRs2.1 million per km of link road and FMR, the funds allocated to the widening of this main road could have financed the construction of approximately 115 km of additional roads. This represents an increase of 22% over the total length of link roads and FMRs completed under the Project, and is equivalent to a further 28 roads. 35 As a comparison, the average daily unskilled agricultural daily wage for men is approximately PRs However, the PMU highlights that many women in this area do not have many opportunities or permission from the male members of the household for outside wage labor employment besides the cotton picking season.