US$M): Sector Board : Transport Cofinancing (US$M (US$M US$M): US$M): Closing Date : 06/30/ /31/2011.

Transcription

1 Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR Project Data: Date Posted : 04/23/2013 Public Disclosure Authorized Public Disclosure Authorized Country: Brazil Project ID : P Project Name : Sao Paulo Metro Line 4 Project Appraisal Actual Project Costs (US$M US$M): ,26 L/C Number: L4646; L7536 Loan/Credit (US$M US$M): Sector Board : Transport Cofinancing (US$M US$M): Cofinanciers : Japanese Bank for Board Approval Date : 01/22/2002 International Cooperation (JBIC) Closing Date : 06/30/ /31/2011 Sector(s): General transportation sector (100%) Theme(s): Urban services and housing for the poor (40% - P); Infrastructure services for private sector development (40% - P); Pollution management and environmental health (20% - S) Prepared by : Reviewed by : ICR Review Group: Coordinator : Victoria Alexeeva Alain A. Barbu Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The statement of objectives in the Project Appraisal Document (PAD) and the Loan Agreement is identical: " (a) to improve the quality and long-term sustainability of urban transport in the São Paulo Metropolitan Region (SPMR) by interconnecting the existing subway, commuter rail and bus networks through the construction of METRO's Line 4; (b) to improve the accessibility of the low-income population of the areas served by Line 4 to employment centers and health and education facilities; and (c) to seek private sector participation in the development of Line 4. The project development objective (PDO) was not altered during restructuring of the project (upon approval of the additional financing in 2008), while several PDO- related targets were revised, as follows: Public Disclosure Authorized Increasing the target percentage of Line 4 Phase 1 stations integrated with buses to 100%. Reduction of average generalized travel costs between Vila Sonia and Luz from 104 to 82 minutes against original target of 51 minutes with Line 4 Phase 1 in operations. The revised target value reflects higher than expected growth in traffic and an updated methodology for value of time. Adjustment of the target Metro share of urban transport motorized trips to 17.5% from 23% in 2010 due to a revised methodology to calculate the base mode share for Metro in Clarifying METRO s working ratio (operating costs over operating revenues ) to remain less than one, excluding depreciation, cost of capital and the operations of Line 4. b.were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 06/12/2008 c. Components:

2 Component A : Infrastructure and Equipment Investment (appraisal cost US$758.2million; actual cost US$1,509.39million). Construction of, provision of equipment for, and operation and maintenance of, the line of the SP Metro which will link the SP Metro s Vila Sonia yard facility to the SP Metro s Luz station (Line 4), consisting of about 12,800 meters of double underground track, 5 stations and 4 station shells, acquisition and installation of system-wide facilities (including fixed installations for electrification ), and acquisition and operation of 16 train sets. Component B : Technical Assistance (appraisal cost US$33.00million; actual cost US$15.84million). Provision of technical assistance for: (1) the management oversight and supervision of the carrying out of Part A of the Project; (2) the carrying out of a financial management and cost recovery review (including recommendations on tariff structure) of the SP Metro operations; and (3) for a follow-up project finance study on alternatives for private sector participation in the carrying out of Part A of the Project. Revisions were made to the components between approval in 2002 and project restructuring in 2008: Inclusion of a sixth station, Faria Lima, in Phase 1; Change in the construction method from Shields to NATM in parts of Lot 2 (was done by the project with all risks assumed by the turnkey contractors ); Delivery of 14 train sets instead of 16 (was done by the operating concessionaire ); Inclusion of additional works (advanced engineering design) for supply and implementation of platform door system (PSD) for the Phase 1 and Phase 2 stations; Inclusion of additional works for the production, supply and implementation of data transmission system (STD); Inclusion of additional works for the production, supply and implementation of the passenger fare collection system (SCAP) Line 4, phases 1 and 2, including Vila Sonia yard; Inclusion of additional works for the supply and implementation of moving walkways. The main reasons for revisions were the appreciation of the Brazilian Real against the US dollar, inflation, and implementation delays. The project was restructured during approval of the additional financing in d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Financing : The World Bank Group contribution consisted of a loan in the amount of US$ 209 million. It was also co-financed by the Japanese Bank for International Cooperation (JBIC) in the same amount as the Bank s financing (US$209million. In April 2008, the Bank approved an additional loan (IBRD-75360) of US$95.0 million to compensate for the effect of the US$ devaluation against the Brazilian currency and inflation over the turnkey contract costs as well as the respective annual price readjustment clause. The same co-financing ratio for the Bank and JBIC was maintained for the additional financing. At closure, Bank's total financing was US$304million. Borrower contribution : The contribution of the State of São Paulo was US$308.40million at appraisal, which increased to US$ by the project closure. This variation was primarily due to: (1) changes in the scope or schedule of the project including amendments and change in construction methods; (2) variation in the exchange rate given that the financing was in US Dollars while main contracts were in Brazilian Real; and (3) increase in prices due to the annual readjustment formula (inflation) in the contracts. In addition, a private consortium, Consórcio Via Amarela, made a contribution to the project in the total amount of US$246.10million (raised from the appraisal amount of US$207.50million). The private consortium was competitively awarded the concession in 2006 to operate the system for 30 years in exchange for the provision of rolling stock and systems. Dates: The original closing date was June 30, 2007; the actual closing date was March 31, The closing date of the original loan was extended two times. The first extension was from June 30, 2007 to June 30, 2009 occurred in March A second extension to June 30, 2010 occurred in conjunction with the approval of the additional loan. The original loan was closed and fully disbursed by June 30, A delay of three years in the completion of Line A Phase 1 was caused by the implementation problems that included a combination of procurement litigation and a delay in expropriation due to a protracted strike of the official evaluators of property values, the unavailability of counterpart funds in the first year, and a fatal construction accident in 2007 slowing the project schedule and disbursement by almost a year. In May 2010, the Bank approved a 9-month extension of the closing date of the additional financing loan from June 30, 2010 to March 31, 2011 to allow for the completion of civil works and systems for full operations and full disbursement of funds. 3. Relevance of Objectives & Design: a. Relevance of Objectives:

3 The project development objective (PDO) was consistent with the State of São Paulo's development priorities within the Integrated Urban Transport Plan (Plano Integrado de Transporte Urbano ) for the the São Paulo Metropolitan Region. The PDO remained relevant to the World Bank Group's FY Country Partnership Strategy for Brazil (current at project closure), the key goals of which included competitiveness and equity through improving access to infrastructure/ urban services. The PDO was also in line with the main priorities of the Country Strategies at appraisal and during implementation, which supported targeted interventions (a) to reduce poverty through the provision of urban services to the poor, (b) sustainable fiscal adjustment through reform of public enterprises, and (c) renewed growth through private sector participation in infrastructure provision, among others. Overall, relevance of the project objective is rated high. b. Relevance of Design: The Project design appropriately targeted the highest priority issues of the urban transport sector in the S ão Paulo Metropolitan Region: in particular, the need to address an increasing traffic congestion and atmospheric pollution, as well as a lack of access to job opportunities. Despite an existing 270 km rail network, the lack of integration between the Metro and the suburban trains discouraged more rail trips in favor of buses and the automobile, thereby creating heavy congestion during peak hours and significantly increasing travel times. The project results framework provided a logical link between the activities to be financed by the project and the outputs and outcomes. The project design was relevant throughout the life of the loan, though it underwent a number of amendments to the project components at the request of the State of S ão Paulo as a result of the exchange rate appreciation and inflation. The currency risk was not explicitly considered among the causes of potential cost increases; this risk could not have been credibly forecast at preparation. The planned implementation period of 5 years was appropriate given the risks identified during preparation but turned out to be too short given the complexity of the project and unforeseen delays regarding resettlement and the construction accident. The relevance of the project design is rated substantial. 4. Achievement of Objectives (Efficacy): The objectives of the project for the São Paulo Metropolitan Region were achieved to the following extent: a) Improved quality of urban transport : High Outputs 100% of planned infrastructure was completed : construction, provision of equipment, and operation and maintenance of Phase 1 of Line 4, consisting of double underground track, 6 stations and 4 station shells, acquisition and installation of system-wide facilities (including fixed installations for electrification ), and acquisition and operation of 14 train sets (revised from 16). Outcomes Line 4 Phase 1 improved travel times, convenience, and reliability : a 30% improvement was achieved in generalized travel costs (travel time plus fare plus reliability) from the baseline value of 104 to 73 minutes, exceeding the revised target. 6 stations of Phase 1 are fully operational. Line 4 is an essential integration line of the São Paulo metropolitan public transport system. The Phase 1 portion of Line 4 was moving over 600,000 passengers per day after 6 months in operation with a strong growth trend. This ridership meets the projections at the time of preparation and during implementation. According to the TTL, the daily ridership is currently approaching 700,000 (as of October 2012). In the last decades, very few new metro lines in the world have achieved this level of ridership, which validates the significant projected benefits. The target of integrating municipal and inter -municipal bus lines was achieved in all six stations of Phase 1 allowing for the efficiency benefits of transferring passengers from the surface roads to rail -based modes in a new underground line. The share of Metro trips among motorized trips increased from 16% estimated for 2001 to 19.3% exceeding the revised target value of 17.5%. The increase in metro share of urban transport was achieved despite the rapid growth in motorization during this period. The other rail ridership and mode share of São Paulo Metropolitan Train Company (CPTM) also increased significantly in this period. This positive trend is attributable to a portfolio of investments made

4 by the SSP, including the implementation of Line 4, improved reliability and service frequencies on CPTM lines, and extensions to Lines 1, 2 and 3. Improved long -term sustainability of urban transport : Substantial Outputs Financial management and cost recovery studies (including recommendations on tariff structure ) were completed by They were designed to propose cost cutting measures and revenue maximization to improve METRO s working ratio. A follow up of the project finance studies was also completed. These included a report by financial advisor on private participation in the operation of Line 4, a review of PPP studies developed by METRO, and a study of urban operations (including financial implications). Outcomes The financial management and cost recovery studies were concluded and recommendations were implemented. Metro s working ratio (defined as operating costs divided by operating revenues) improved from the baseline value of 0.98 to 0.79, and below the covenanted 1.0, with the introduction of Line 4. While the financial impact cannot be fully assessed because detailed operating cost data is not publically available from the concessionaire, this result suggests that the public sector s burden has remained the same or decreased with Line 4. The review also resulted in the development and introduction of the Single Integrated Fare Ticket (BUI) in b) Improved accessibility of the low -income population : High Outputs Transfer stations, platforms, and extension of the network Line 4 were completed. Line 4 is integrated with Lines 1, 2 and 3 of Metro and Lines 7, 9, and 12 of CPTM. Fare integration was implemented (see above). Outcomes At least 50% of Metro users are considered low income and live in the suburbs of the greater S ão Paulo Metropolitan Region. A long-term study of the travel conditions of the poor population in the areas influenced by the Line 4 project began in 2010 with a survey before the project entered operations. Preliminary results from the initial survey and the Bank s analysis suggest that low-income commuters from the periphery of the region benefited the most from the combination of the single integrated fare (BUI) and Line 4. The fare integration (Bilhete Único) started in 2006 providing free transfers between Metro, CPTM, municipal buses, and some inter -municipal buses by way of specially branded contactless farecards. The majority of accessibility benefits (proximity to land uses that generate social and economic opportunities) and mobility benefits (generalized travel savings) accrued to low-income families in the periphery of the São Paulo Metropolitan Region who now use Line 4 as one of the legs of journey to cross the city or region. Line 4 Phase 1 has increased the accessibility of low -income population to public transport and to employment centers, health, education, and leisure facilities. The main beneficiaries of Line 4 were the populations of Embu, Taboão da Serra, Cotia, Embu Guaçu, Itapecerica da Serra, Juquitiba, São Lourenço e Vargem Grande Paulista, whose accessibility to the Central Business District (CBD) and other areas of employment, health and education facilities was greatly enhanced. c) Private Participation : High The project sought private participation in the new urban rail project development and operations in Brazil. The objective was achieved. Line 4 was the first Metro project developed under federal and state public -private partnership (PPP) laws. Metro Line 4 introduced an innovative structure for Brazil including the owner (METRO), operating concessionaire established by a consortium of private companies, and a turnkey contract signed in 2003 for the construction works. Infrastructure and part of the rolling stock were financed by the State of S ão Paulo (with the support of loans from the

5 World Bank and JBIC) and by private partners at 80% and 20% respectively. The 30-year concession contract includes the operation of Metro Line 4, as well as the investment and installation for rolling stock, signs, track connections and data transmission with the train networks. The concessionaire s operations using new technologies such as driverless train operation (Communications- Based Train Control) and station platform doors suggest greater operating efficiencies (lower costs) and improved safety performance. According to KPMG s publication Infrastructure 100 World Cities Edition issued in July 2012, Line 4 is one of the 100 most innovative infrastructure projects in the world. The selection was made by international specialists based on project scale, feasibility, complexity, innovation, and impact on society. 5. Efficiency: For the project at closure, the revised Net Present Value (NPV) of benefits is US$364million and an Economic Internal Rate of Return (EIRR) is estimated to be13.8% as compared with an NPV of US$554million and EIRR of 17.7 percent at appraisal. The results are slightly lower primarily due to an increase in costs and updated assumptions. A conventional cost-benefit analysis for the project was carried out using the latest cost and benefits estimates available. The situation with the project was compared against the situation without the project. The following conservative assumptions were made in the analysis with respect to the situation at appraisal : 1. Updated investment cost stream to reflect the changes to the components and timing. 2. Updated benefits stream to reflect the approximately 4-year delay in the start of Phase 1 operations since the original appraisal. 3. Maintained the reduction in wages and the incremental increase in operating and maintenance cost which were consistent with appraisal. 4. Updated value of time coefficients to account for the differences in time -savings for home to work, business and other trip types. 5. Updated travel purpose distribution based on latest estimates. The project experienced significant cost increases and delays, however the ex -post assessment still confirms a positive economic efficiency of the project. The project team subsequently stated that the cost overrun and implementation delay were considered in the analysis, and were largely compensated by an increase in travel time savings. As mentioned in Annex 3 of the ICR, updated value of time coefficients for different trip types were used in the travel time savings estimate. These new coefficients are from economic analysis done for the Line 4 Phase 2 PAD in 2009/2010. These updated values of time are higher than were projected in 2001 due largely to the economic boom and income growth in Brazil in the past decade. Overall, the almost 4 year delay due to procurement litigation, unavailability of counterpart funds, accident caused by engineering defects resulting in 7 deaths, and resettlement problems suggesting inadequate social assessment point to Modest efficiency, a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 17.7% 81% ICR estimate Yes 13.8% 85% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project significantly contributed to the improved quality, accessibility, and long -term sustainability of urban transport in the São Paulo Metropolitan Region achieving all the stated objectives. It remains highly relevant to the needs of the users of urban transport in the SPMR. Significant cost increases and delays during implementation -- almost 4 year delay due to procurement litigation, unavailability of counterpart funds, accident caused by engineering defects resulting in 7 deaths, and resettlement problems suggesting inadequate social assessment -- point to modest efficiency, According to KPMG s publication Infrastructure 100 World Cities Edition issued in July 2012, Line 4 is one of the 100 most innovative infrastructure projects in the world. a. Outcome Rating : Moderately

6 7. Rationale for Risk to Development Outcome Rating: The main risks to development outcome are as follows: The risk of the operating concessionaire deferring maintenance of the fleet and facilities or not meeting the performance standards of the contract, thereby affecting the long -term sustainability of Line 4. The risk that the continued expansion of the Metro and CPTM network under the current fare structure (flat fares with BUI) could upset the equilibrium of operating revenues in the metropolitan network and negatively impacting METRO or Line 4 operations in the future. This can be mitigated by studying the possibilities of making the fare structure less rigid, for example by gradually introducing zonal fares or more targeted subsidies designed to minimize any negative impact on low-income passengers in the periphery that currently benefit from flat fares. There is modest risk of the achievements in metropolitan and institutional coordination (Integrated Transport Coordination Commission- CDTI) being undone due to political pressures or other circumstances in the S ão Paulo Metropolitan Region. The risk of macroeconomic or fiscal conditions adversely affecting the planned expansion and future operations is low. However, significant additional investments in rolling stock, stations, and system upgrades are needed to increase operating capacity in the São Paulo Metropolitan Region and alleviate the continued growth in congestion. These investments in transport infrastructure are among the highest priorities for municipal and state governments in SPMR. A Bank loan for US$130 million was approved in 2010 for Line 4 Phase 2, which includes the civil works and systems to complete the 12.8 km subway with 11 stations. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The Task Team designed an urban transport operation that entailed features that are still valid and appropriate today. These included: (i) capacity building and support for the development of a successful PPP, promoting progressive private sector participation in the investment and operations management of urban transportation systems; (ii) institutional development to promote metropolitan transport coordination and integration, and (iii) improved and increased accessibility of low -income populations to the public urban transport system. The Line 4 project consolidated and deepened previous efforts from the World Bank, initiated under the S ão Paulo Metropolitan Transport Decentralization Project (Ln BR) and the Barra Funda- Roosevelt project (Ln BR), which assisted respectively in the decentralization of the federally - owned CBTU to the State to allow for more effective modal and tariff integration, and in connecting the two suburban rail systems. The Bank and the Borrower prepared the project by : (i) carrying out a solid background analysis; (ii) evaluating lessons learned from previous operations and appropriate alternatives; (iii) holding detailed missions with specialized teams of consultants; (iv) designing a technically well- conceived project; (v) paying attention to institutional aspects, such as ensuring that the project had support from the METRO Employee Union as well as from all São Paulo political parties; and (vi) ensuring the availability of the necessary funding for the project. A range of foreseeable risks were evaluated during project preparation although greater attention to them could have mitigated at least some of the subsequent delays. Among the most relevant risks were cost increases, delays in procurement and resettlement, limited private sector interest in the concession, and tariffs too high for low income users after private sector participation. Intrinsic risks such as construction accidents were not itemized in the original PAD. Quality-at at-entry Rating : b. Quality of supervision: Supervision was carried out frequently and the teams were well -balanced. In addition, there were separate financial management, safeguards, and procurement supervision missions, which took place at least once every year. 26 ISRs were filed between August 2002 and December 2011 (more than 2 per year).

7 The Task Team identified implementation problems such as delays and the construction accident in a timely way. It took appropriate measures, including additional reporting and supervision requirements from the Borrower with the support of a Project Management Oversight Consultant (PMOC). It also agreed to increase Bank financing due to Brazilian R$ appreciation and inflation. High-quality advice and support to the Borrower was provided during the entire process of developing and implementing the turnkey and concession contracts. The Task Team showed deep knowledge of the client s needs, engagement, and appropriately adapted the project to ensure maximum relevance and achievement of PDOs in light of changing conditions and delays. The Mid-Term Review (MTR) was thorough. Between December 2004 and May 2005, the main concerns were issues largely out of control of the Bank : the expropriation delay that impacted the start of construction and the interest of the private sector in investing in the project considering the market and economic conditions. The Bank team kept assisting Metro in all of those aspects, providing the international experience required for METRO to design locally acceptable options. However, two issues detract from the Quality of Supervision. First, supervision could have addressed some design issues -- project complexity, and procurement and other procedural issues -- and could thereby have prevented at least some of the delays (ICR pages 11, 18, 20, and 21). Second, regarding safety issues and fatalities, the Bank clearly has a duty to supervise a project in general and the Borrower in particular, including workplace safety commitments embedded in the Bank's safeguard provisions, although that does not necessarily imply a causal link between Bank supervision and the fatalities. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately 9. Assessment of Borrower Performance: a. Government Performance: The State of São Paulo was effective in implementing the necessary fare integration as well as physical/operation integration between Line 4 and the other lines of the Metro, São Paulo Metropolitan Train Company (CPTM) and bus systems. However, delays and the initial unavailability of counterpart funds impacted the scope and implementation in the first years of the project, while improving later on. Moreover, the Regulatory Commission was not properly staffed in the beginning with a clear mandate as recommended by the Bank. One of the requirements of the loan was that the Borrower should establish a Regulatory Agency or Transitional Regulatory Entity independent of the State Secretary for Metropolitan Transport no later than one year after signing. This was partially achieved with the implementation of the Regulatory Commission in Legal and contractual expertise was needed in addition to technical skills offered by METRO. Although, their performance has significantly improved, the Commission was not transformed into a more independent Regulatory Agency approved by the State legislature (as required by the loan agreement). Government Performance Rating Moderately b. Implementing Agency Performance: The METRO is an experienced organization with a department dedicated to supervising Line 4 implementation and an appropriate Project Management Unit (PMU) focused on institutional, financial, fiduciary, reporting and safeguards issues of the Bank loan. The PMU continues this role for the Line 4 Phase 2 Project. In addition, the PMU is supported by a Project Management Oversight Consultant (PMOC) financed by the loan, which issues monthly and ad hoc reports to the State Secretary of Metropolitan Transport (STMSP) and to the Bank. Finally, there was always an appropriate Project Coordination Unit (PCU) reporting to the STMSP Secretary and to the Governor on strategic aspects of the project. While the project was successfully implemented, the schedule, cost, and disbursements were significantly

8 impacted by problems of early litigation delays and the construction accident. Although METRO tried to follow sound procedures, neither local litigation nor the international litigation worked well. Having all documents and litigation in English was a burden and time-consuming exercise for METRO. On the other hand, national litigation could suffer from the potential influence of contractors in the domestic market. These problems could have been better anticipated by METRO in their procedures and schedules. The construction accident that involved 7 deaths happened due to controllable factors. These included weaknesses in project engineering, civil works implementation (particularly supervision of tunnel settlement ), and overall risk management (see Section 11). Implementing Agency Performance Rating : Unsatisfactory Overall Borrower Performance Rating : Moderately 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project's indicators were appropriately linked with the PDO and included baseline data and measurable targets set at appraisal: a) Two indicators (percentage of integrated stations, the share of Metro in urban transport motorized trips ) addressed the first part of the PDO, which is related to the integration and quality of services in the region. b) The third indicator (generalized cost of travel in minutes) addressed the second PDO related to the accessibility of low-income population from areas served by Line 4 to employment centers and health and education centers since a majority of these trips were made by low income people (those earning less than the equivalent of 4 minimum salaries). With regard to the targeting evidence on the low -income population, the project team subsequently suggested that when the project was prepared in 2001, it was believed that direct monitoring of the project s impact on low-income people was difficult because it would not be measurable for months or years after operations and required an in-depth before-after study that might not be available at project closing. The PAD contained a baseline study on the impacts of Line 4 on poverty, prepared jointly by the World Bank team and the Borrower (Annex 11). It forecasted beneficial impacts of the project on low -income population and confirmed the project s potential to improve accessibility of the low-income population of the areas served by Line 4 to employment centers and health and education facilities. The approach taken in the ICR was to report on daily ridership 6 months after the start of operations (600,000 passengers) as a rough proxy for the impact on the low -income population since previous studies had shown that a majority of Metro users are considered low income people. An update of this study by Metro is underway. c) The project had also the objective of seeking private sector participation in the development of Line 4, which was achieved through a concession agreement signed with Consórcio Via Amarela (CVA). Regarding the M&E of outputs, four indicators were defined for Part A (% of infrastructure completed, percentage of stations completed, percentage of systems works completed, and percentage of rolling stocks delivered ). Two indicators were defined to monitor the institutional development promoted by the project - Part B (financial management and cost recovery review of the Metro, and a follow -up study on private sector participation ). The indicators (provided subsequently by the project team ) used to monitor the concessionaire's performance included the following: (i) Headway (time interval between trains) availability of the service; (ii) Average trip duration between terminal stations speed of the service; (iii) Level of use during peak-hour (standing passengers/square meter during peak-hour) comfort of users; (iv) Supply level (monitoring the availability of trains according to estimated offer ) availability of the service; (v) Number of accidents with users - safety of users; (vi) Occurrence of crime in the system - safety of users; (vii) Time spent by users in queues to buy tickets; (viii) Time spent by users to access restricted areas after fare payment; (ix) Number of complaints by users.

9 b. M&E Implementation: The project team appropriately used the outcome and output indicators to supervise the progress of the objectives and implementation. Annual or semi-annual monitoring reports included useful updates on institutional objectives such as integrated modal tariffs (Bilhete Único Integrado), creation and maintenance of a regulatory commission, and environmental management programs. The ISRs were timely and usually included an Aide Memoire as an attachment. In the first few years of the project, the ISRs and monitoring reports included updates on the two output indicators related to Part B of the loan agreement : Financial management and cost recovery review and a Followup study on private sector participation that were completed and implemented. c. M&E Utilization: The M&E utilization helped inform decision making on the restructuring of the project. Some of the original targets for the outcome indicators were revised in 2008 to reflect the changing conditions of the project. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: This was a Category A project that triggered two safeguards policies OP4.01 Environmental Assessment, and OP 4.12 Involuntary Resettlement. Environmental Safeguards and Management : The project was expected to have a net beneficial impact on society and the environment. The new subway link would help relieve congestion in major transport corridors and central areas, resulting in lower emission of pollutants per vehicle - kilometer. Junction improvements and pedestrian overpasses would improve safety and quality of life. The Line 4 Phase 1 Project had a full environmental assessment, which was submitted by the Board prior to its approval. The project has been successful in preventing or mitigating direct environmental and social impacts (ICR, page 10). According to the project team, OP4.01 and OP4.12 were complied with, even considering the 2007 construction accident. The procedures and supervision given to the project were adequate. Subsequent to the accident at Pinheiros station (see below) and as a prerequisite for the Additional Financing (loan 7536-BR), the Bank required a detailed social assessment, periodic monitoring of those affected by the accident, and an environmental assessment of the accident. On January 12, 2007 a construction accident involving the collapse of the shaft at the future Pinheiros station caused seven fatalities. The people killed were passengers in one informal minivan, one truck driver and one pedestrian walking on a nearby street. No construction workers were killed or hurt. Several homes and other structures were affected. Six homes were demolished shortly after the accident and others were evacuated by Civil Defense authorities. The Bank monitored the review of the construction safety standards with recommendations for their enhancement and better enforcement. The causes of the accident were thoroughly investigated by the relevant authorities and a report was produced by the renowned S ão Paulo Technological Research Institute (Instituto de Pesquisas Tecnológicas ). The investigation concluded that a number of factors contributed to the accident including weaknesses in project engineering, civil works implementation (particularly supervision of tunnel settlement), and overall risk management. The accident caused an unforeseeable delay in the project, as works had to be stopped. The Bank also monitored and documented the mitigation of all social and environmental impacts of the accident. In the immediate aftermath, all the evacuated families were accommodated at five-star hotels at the expense of the construction consortium, Consórcio Via Amarela (CVA). The Bank requested the Borrower to submit detailed monthly reports which confirmed that all required compensation and pending cases were being resolved appropriately. Settlement negotiations involved the claimants, CVA, Unibanco AIG and METRO, with oversight by São Paulo Public Defender and the State Justice Department, which ratified all decisions. Involuntary Resettlement : "The Line 4 project included a significant amount of resettlement that was conducted by METRO in a manner consistent with the agreed plan and Bank policy " (ICR, page 13). The following additional information was obtained from the project team : the social assessment conducted at appraisal (2001) for the full extension of Line 4 (Phases 1 and 2) estimated 274 properties would be resettled (77 residential and 197 commercial). In the 77 residential units approximately 227 people would be affected. In the end, the resettlement for Phase 1 was concluded in 2005/2006 with 36 properties resettled and less than 227 people were affected. The remaining resettlement was conducted under the Phase 2 Project.

10 As is common for underground civil works, minor damage to structures along the alignment were recorded during implementation and appropriately repaired or compensated by the turnkey contractor under the supervision of METRO. b. Fiduciary Compliance: Procurement: The procurement component was well designed and supervised. The implementing agency had good procurement capacity, as assessed during preparation, and was knowledgeable of Bank procurement procedures. Financial management. Financial management was in compliance with Bank procedures. Both the State and METRO had previous experience with Bank loans and were able to handle all aspects related to the financial management of the project including accounting, disbursement and auditing functions. Auditing was carried out annually by independent consultants selected on a competitive basis according to Bank procurement guidelines. Financial management reporting and auditing were satisfactory. c. Unintended Impacts (positive or negative): d. Other: 12. Ratings: ICR IEG Review Outcome: Moderately Risk to Development Moderate Moderate Outcome: Reason for Disagreement /Comments The almost 4 year delay points to modest efficiency, Bank Performance : Borrower Performance : Moderately Quality of ICR : Moderately Moderately There are suggestions in the ICR that a better risk analysis (especially given that metro projects typically face safety issues, a less complex project design, and upfront attention to procurement and other procedural issues, could have prevented at least some of the delays (ICR pages 11, 18, 20, and 21). Some of these design issues could have been addressed during supervision, but were not. Furthermore, regarding safety issues and fatalities, the Bank clearly has a duty to supervise a project in general and the Borrower in particular, including workplace safety commitments embedded in the Bank's safeguard provisions, although that does not necessarily imply a causal link between Bank supervision and the fatalities. NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

11 13. Lessons: The ICR has identified a number of specific lessons in relation to PPPs and the most important ones are listed below: For any PPP, regulatory commissions and strong supervision by the State or its representative is needed during construction and operation. A formal agency should be given a clear mandate or established early enough to ensure that it is properly staffed. The agency must be considered sufficiently independent and credible to conduct its duty. PPP contracts must have very stiff penalties for delays in construction as well as bonus for earlier completion. They must also have very stiff penalties for non -compliance during operations and key performance indicators must be carefully designed and enforced. When a turnkey is used, mechanisms to handle changes in the project design, either requested by the State or in view of unanticipated geological or construction reasons, must be clearly defined between the parties. This will ultimately protect public interest and facilitate the acceptance of these changes by overseeing Accounts Tribunals. 14. Assessment Recommended? Yes No Why? To verify the ratings and document lessons learned including on safety issues. 15. Comments on Quality of ICR: The ICR is well-written and concise. The discussion of achievements is results - oriented. The quality of evidence and analysis is adequate. However, while covering well the social aspects, the ICR does not provide good detail with regard to the safeguard policy triggered by the project on involuntary resettlement and related delays (some more information was subsequently provided by the region ). The ICR could also have included more details on the actual financial performance of the São Paulo Metro (complete financial forecasts were included in the PAD ), as well as more information of the terms and conditions of the concession, including indicators used to monitor the concessionaire's performance (related information was subsequently provided by the region ). a.quality of ICR Rating :