Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR13819 1. Project Data: Date Posted : 06/12/2012 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Country: Vietnam Project ID : P082627 Appraisal Actual Project Name : Vn-second Payment Project Costs (US$M US$M): 112.00 110.78 System And Bank Modernization Project L/C Number: C4041 Loan/Credit (US$M US$M): 105.00 92.77 Sector Board : Cofinancing (US$M US$M): Cofinanciers : Board Approval Date : 03/10/2005 Closing Date : 06/30/2010 06/30/2011 Sector(s): Banking (76%); Payments settlements and remittance systems (24%) Theme(s): International financial standards and systems (67% - P); Other financial and private sector development (33% - S) Prepared by : Reviewed by : ICR Review Group: Coordinator : Mariluz Cortes-Gorman Kris Hallberg Ismail Arslan IEGPS2 2. Project Objectives and Components: a. Objectives: The specific Objectives of the Second Payment System and Bank Modernization Project (PSBM 2), as stated in the PAD (page 4), are to assist Vietnam s banking sector to: (i) meet the demand of the economy for fast, reliable and safe payment services; (ii) broaden access to finance to facilitate the achievement of the Government s poverty reduction targets; and (iii) improve the operations of the participating banks through expansions of the Inter-bank Payment System (IBPS) and the Core Banking Operational and Management Control Systems (CBS). The statement of project objectives in the PAD and in the Credit Agreement (p.17) are basically consistent. The definition of objectives used in this review is that in page 4 of the PAD. The PSBM2 was a repeater project, expanding and enhancing the systems that were introduced on a pilot basis under the PSBM1. b.were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components: The project had two main components. It also financed its implementation support activities:
Component 1: Interbank Payment System Expansion (appraisal $24.6 million and actual $21.4 million): Expansion of the pilot Inter-bank Payment System (IBPS), a centralized, single-current account based, inter-bank clearing and settlement system, managed by the State Bank of Vietnam (SBV). The activities and investments financed included: (i) upgrading hardware and installing new software to expand processing capability and strengthen interconnection, business continuity, and network and security system; (ii) consultant support of project implementation; and (iii) staff training. The implementing agency of this component was the Informatics Technology Department (ITDB) of the SBV. Component 2: Core Banking Solution Expansion (appraisal $82.2 million and actual $76.7 million) : Expansion and enhancement of the pilot international standard core banking solutions (CBS) and other modules of banking operational systems (BOS) including financial management and improved management information systems (MIS). The activities and investments financed included: (i) upgrading hardware and installing new software to expand processing capability and strengthen interconnection, business continuity, and network and security system; (ii) consultant support of project implementation; and (iii) staff training. The implementing agencies of Component 2 were the four participating banks: Bank for Investment and Development of Vietnam (BIDV), Vietnam Bank for Agricultural and Rural Development (VBARD), Maritime Joint Stock and Commercial Bank (MSB), and VietinBank. The project also financed its implementation activities and investments included: (i) Project Management Unit (PMU) office equipment, (ii) Project audits (6 years). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The outputs of the project required heavy capital investment in imported capital goods. At the time of appraisal, the estimated project cost was USD 112.98 million of which USD105 million would be financed by an IDA credit and the rest with Borrower s counterpart funds. At project completion, the government had provided USD18.01 million in counterpart funds, more than double the amount estimated at appraisal. The government also provided project preparation funding (mainly financed by a Japanese Government PHRD grant) for the review and costing of the technical needs for system expansions. At project completion, USD12.23 million were cancelled from the IDA credit, due to several factors including USD5.0 million in savings from the competitive procurement processes, changes in priorities in the implementing agencies for the use of some funds, and lack of time in the case of one bank to implement a sub-component before the project closing date. The USD depreciated slightly with respect to the SDR between appraisal and last disbursement. As a result, the actual Bank financing in SDRs at the time of the last disbursement was somewhat lower in SDRs than if the exchange rate would have remained constant. The project closing date was extended once from June 3, 2010 to June 30, 2011 mainly to allow time for implementation completion by three participating banks (BIDV, MSB and VietinBank). 3. Relevance of Objectives & Design: a. Relevance of Objectives: The relevance of the project s objectives is rated as high. This assessment is based on the relevance of all three project objectives, although the second objective--to broaden access to finance--was not monitored during implementation because the monitoring and evaluation framework (M&E) did not include indicators to measure the achievement of this objective. The objectives of assisting Vietnam s banking sector to meet the demands for reliable and safe
payment services, broaden access to finance and improve the operations of participating banks remain highly relevant for the country s current economic conditions. Vietnam s economy has begun to stabilize after the impact of the global financial crisis, but it is vulnerable to external shocks due to lower global growth and increased financial market volatility caused by the Eurozone crisis. Under these circumstances, the authorities need to demonstrate that they are taking pro-active steps to limit the degree of vulnerability in the banking and corporate sector through tightening prudential standards and supervision in the banking system, and improving governance and financial discipline in both state and private corporations. The improvements in financial infrastructure supported by the project are crucial to achieving these objectives. Also, increasing access to finance to underserved sectors of the economy, such as small and medium enterprises, remains important to achieve the government's objectives of employment generation and poverty reduction. The project s objectives are also highly relevant to the Bank Group s new Country Partnership Strategy (CPS FY12-FY16), presented to the Board on December 15, 2011. The CPS supports the thrust of the Government s 2011-2020 Socio-Economic Development Strategy through investments, programs and advisory services organized into a strategic framework of three pillars and three cross-cutting themes. The pillars are: (i) strengthening Vietnam's competitiveness in the regional and global economy; (ii) increasing the environmental sustainability of its development; and (iii) broadening access to economic and social opportunity. Underpinning the pillars are two cross-cutting themes: (i) strengthening governance and (ii) improving resilience in the face of external economic and natural hazards and shocks and the impact of climate change. The Bank continues to support financial sector strengthening in the context of the Tenth Poverty Reduction Support Credit approved by the Board on December 15, 2011. b. Relevance of Design: The relevance of the project s design is rated as substantial based on the following considerations: The project activities and components were well designed to achieving the project s objectives. The two components, the expansion and improvement of the pilot IBPS and CBS systems were crucial for meeting Vietnam's need for fast, reliable and safe payment services, broaden access to finance and improve efficiency of the banking sector. The Results Framework presented a clear and convincing causal chain between project activities, outputs and and outcomes with respect to the first and third project objectives. However, the Results Framework failed to establish a clear link between activities, outputs and outcomes with respect to the second project objective of broadening access to finance, and as a result, there was no follow up of the achievement of this objective. The design of the project components incorporated lessons learned from the implementation of PSBM1 and other Bank IT-based projects, as well as the need to strengthen the capacity of other implementing agencies including commercial banks in payments system related projects. By providing funding and technical support to three of the country s four largest banks, and a fast expanding joint-stock bank, the PSBM2 design ensured a critical mass of transaction coverage and a balanced representation of national payments system members. However, as some participating banks mentioned in their comments of the ICR, the project implementation schedule was overly optimistic given the technical complexity of the systems, the unforeseen technical difficulties unavoidable in new IT-based project and the long review process for procurement. 4. Achievement of Objectives (Efficacy): (i) To assist Vietnam s banking sector to achieve the objective of meeting the demands of the economy for faster, reliable and safe payment services.
There is a clear causal link between the project inputs including the financing of goods and consulting services contracts and the outcome of faster, reliable and safer payment services in Vietnam. At project completion, 27 contracts financed by the IDA credit had been successfully implemented to enable the expansion and upgrading of the IBPS and CBS systems and the implementation of one contract continued after project closure per original contract terms. Also, all systems acquired were accepted. The outcome indicator of a steady increase of the number of financial institutions/bank branches connected to IBPS and CBS and of transaction volume and value processed by the new system was fully achieved. At the beginning of the project 49 financial institutions were connected to the pilot IBPS and 401 bank branches were connected to the CBS system. At project completion all the banks (97) were members of the IBPS and 3,091 bank branches were connected to the CBS/BOS system, including VBARD s vast branch network of over 2000 business units. The enhanced/expanded capacity of the systems has contributed to the exponential increases of the transactions processed. The targets of 80 percent growth of transaction volume and value over 5 years have been surpassed. Data from BIDB shows that daily transactions increased by 8 times in volume and by 90 times in monetary terms. Data from MSB shows that daily transactions increased by 760 times and by 16 times in monetary terms. The expanded IBPS has enhanced the SBV s capacity to better monitor and manage inter-bank market liquidity, thanks to direct transfer of funds between banks and within a bank across the country, on-line processes, standardized procedures, and single settlement accounts. The efficacy is rated as high. (ii) To broaden access to finance to facilitate the achievement of the government s poverty reduction targets. As indicated, there were no outcome indicators specified for this objective. However, there is evidence that access to finance improved during project implementation. The expansion of the systems enabled the banks to provide to their customers one-stop services and new products such as credit and debit cards and internet banking, and ATM networks. An example of increase in access is the jump in the number of VBARD customers from 7 million to more than 17 million after the completion of their card management system. In a more general view, access to finance and poverty reduction improved in Vietnam during project implementation, although it is unclear to which extent these outcomes were connected to this operation. Credit as percentage of GDP jumping from 75.4 percent in 2006 to 135.8 percent in 2010 (level comparable to that of highly developed countries). Also, the percentage of Vietnam's population living in poverty fell from 19.5 percent in 2004 to 14.5 percent in 2008. The reduction in poverty was probably due to other factors in addition to this project. The efficacy is rated as substantial. (iii) To improve operations of the participating banks through the expansion of the IBPS and the CBS services. Rating high, based on project's achievement of the following outcomes 1) Continuing high level of performance of expanded/upgraded systems in terms of speed, certainty, reliability and convenience. All the project entities confirmed good performance of the systems in the areas highlighted by the indicator. Inter-bank and intra-bank payments can now be settled in real time as compared to days in the past (speed and certainty). The systems reliability has been significantly strengthened with the backup and disaster recovery centers and network security capacity and the use of user-friendly web-based technology interface. The expanded CBS/BOS have improved the banks competitiveness and operational efficiency as well as strengthened their capacity for better risk management. The technologies adopted are in line with international standards in terms of capacity, speed, application software, security and business continuity. 2) Improved frequency and coverage of management information reports of participating banks. The upgraded MIS systems at BIDV and VBARD send overnight financial and operational information to the desk of senior management based on data from the CBS and the centralized databases. VietinBank completed the implementation of a MIS module before the project closing date. MSB uses an in-house developed MIS which has benefited from the CBS and the data warehouse.
The centralized databases also enable the banks to respond to the SBV s information requirements in a timely manner. All the MIS modules procured/enhanced under the two projects are IAS (International Accounting Standards) compatible. The management information modules of the systems are able to provide timely and holistic information which also strengthens risk management at the participating banks. The efficacy for achievement of third objective is rated as high. 5. Efficiency: The efficiency of the project in achieving the project s objectives is rated as substantial although notice is made of the lack of information in the ICR on project costs. No ERR/NPV analysis was carried out to quantify the economic benefits of the project at appraisal. Also, no standard financial analysis was conducted at appraisal, as cost-accounting was not yet adopted on a widespread basis in Vietnam s banking sector, and the financial results of IT systems development are difficult to quantify in the short life cycle of the project. In the ICR the efficiency of the project is assessed based on the fact that under the project, systems were successfully implemented. The project was completed in a relatively short time for similar IT-based projects (6 years), with all the expected outcomes fully or largely achieved. Concerns over low disbursement at the early phase of project implementation proved to be premature as the final disbursement rate was 91 percent of the original baseline cost (Bank financing portion of total cost). Another evidence of efficiency is that savings from the competitive procurement processes led to the cancellation of US$5.0 million in funding under SBV, BIDV and VBARD sub-components. Implementation was not always smooth and at times suffered major delays in: a) the credit effectiveness (270 days), b) engagement of implementation support consultants (12 months), c) procurement processes for goods, and d) payments to contractors. The delays were due to a number of factors some of which were subject to the implementing agencies control such as staff changes in management units; and others were subject to government control such as confusion about the gaps between government and Bank procurement guidelines, and long and tedious government internal processes for reviewing and approving payments to contractors. These issues were eventually overcome by changes in procedures following Bank recommendations and gains in experience by the implementing agencies. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The outcome of the project is rated as satisfactory because this project significantly improved the speed and reliability of inter-bank payments facilitating the exponential growth of financial transactions nationwide. The participating banks are able to settle customer funds transfers within minutes. Centralized funds and customer management have strengthened the banks' risk management capacity. Also, the project has strengthened SBV s capacity to monitor and manage inter-bank market liquidity, thanks to direct transfer of funds between banks and within a bank across the country, on-line processes, standardized procedures, and single settlement accounts. The rating of satisfactory also takes into account the following considerations: (i) the relevance of objectives and design, including the quality of the results framework which are rated respectively as high and
substantial; (ii) the efficacy in meeting the project development objectives which is rated as high for objectives one and three and substantial for the second objective; and (iii) the efficiency of the project which is rated as substantial. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The development outcomes of the project are likely to be maintained. The systems are technically sustainable thanks to steady system performance; existence of multi-year maintenance contracts; and availability of backup and disaster recovery facilities. All the project entities have dedicated staff to monitor and maintain system performance on a real-time and/or daily basis. Also, all the project entities have secured budget from their institutions for the maintenance of the acquired systems, and are investing their own funding for continuous system improvements. Finally, the increase in the use of the systems and the adequacy of the designed capacity are an important factor ensuring post-completion business sustainability of the IBPS and CBS systems, as the financial deepening of the economy continues. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: The Bank correctly identified the strategic relevance for Vietnam s financial sector of expanding/upgrading the IBPS and the two CBS systems developed on a pilot basis under the PSBM 1 project. The appraisal team ensured quality at entry by incorporating in the design lessons learned from the first project and similar projects elsewhere in terms of the modular technical architecture, the flexible approaches towards specific expansion needs and the institutional arrangements. The scope and cost of the project and of each component was based on extensive reviews of the pilot systems and the overall operational environment and practical experience of the project entities. The design maintained the project management units at the SBV and the implementing agencies to ensure that the institutional memories gained were maintained. Procurement plans were carefully evaluated at project appraisal. Also, the appraisal team reviewed the project s financial management and procurement capacity and took into consideration the capacity building needs in the accounting and financial reporting requirements of the project. Quality at entry was negatively affected by a too broad PDO with expected indirect impacts beyond a project's control. This was recognized by the task team, which took the decision of not to change the PDO, but to take into account a more realistic scenario in designing the outcome indicators. The result is that no performance indicators were established for the second project objective of broadening access to finance. Quality-at at-entry Rating : Satisfactory b. Quality of supervision: The Bank team was technically knowledgeable on processes of payments systems and banking IT projects and the review of procurement documents not only checked the compliance with Bank guidelines but also offered solutions to problems. The Bank team was focused on the development impact of the project and adjusted the M&E indicators once it became clear that measurability of those indicators would need to be enhanced. The team used the Implementation Status Reports (ISRs) to report progress in achieving output indicators and main implementation issues in a candid manner and rated the project according to the progress (or lack of it) in each review period.
The Bank team responded appropriately to major issues causing implementation delays. When staff changes in the project management units (and to a lesser extent the technical teams) threatened to delay project implementation, the Bank team worked out an action plan with the client, arranged for several rounds of training on Bank guidelines, and supported the hiring of implementation consultants to supplement the technical weaknesses of some implementing agencies. In addition, the Bank worked to secure a PHRD implementation support grant to hire a consulting firm to support the implementing agencies on system quality assurance and payments systems oversight. The Bank also addressed the slow approval of contracts and payments (see below) dealing with the problems at the project level and through the country management s dialogue with the relevant government agencies. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government was strongly committed to the project and supported the implementing agencies request for Bank support of the systems expansion. It provided project preparation funding (mainly financed by a Japanese Government PHRD grant) for the review and the costing of the technical needs for system expansions. The government ownership of the project was strong, as reflected in the provision of sizeable counterpart funding. However, some problems under government control became substantial causes of delay in project implementation. Lack of clarity on the part of the government about which procurement regulations, the Bank s or the government s, applied to the project resulted in confusion and multiple rounds of review and revisions. Also, internal processes for reviewing and approving payments to contractors were long and tedious at the State Treasury and the Vietnam Development Bank (VDB). However, once identified, the government accepted the Bank s recommendations and took actions to address these problems. The government also carried field visits at project completion of a large number of project sites to learn about the experience under the project. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The performance of the five implementing agencies at appraisal was satisfactory. All the project entities carried out detailed review of their pilot systems and the business requirements for system expansion and worked closely with the preparation consultants on the costing of the expansion needs. At completion, the performance of ITDB, BIDV and VBARD was rated in the ICR as satisfactory. The management of those implementing agencies remained strongly committed to the development objectives throughout the life of the project and maintained experienced project management units and capable technical teams. However, the performance of the two other implementing agencies, MSB and VietinBank, was rated as moderately satisfactory due to the delays experienced in the initial years of project implementation. Eventually, both agencies replaced their project management teams and adjusted their procurement plans. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory
10. M&E Design, Implementation, & Utilization: a. M&E Design: There was a failure in the M&E design to specify outcome indicators for the second project objective of broadening access to finance. This failure was not corrected during project implementation because the task team considered that this objective was too broad and would be an indirect result of the project. The outcome indicators as originally approved were somewhat repetitive, and during implementation were revised and consolidated to make them more measurable. The revised outcome indicators were designed to provide information for appropriate evaluations given project objectives and available data. b. M&E Implementation: The SBV s Project Management Unit (PMU) and the banks Project Implementation Units (PlUs) were responsible for collecting the outcome indicators data and evaluating project implementation status on a regular basis. They for the most part met these expectations. Progress in achieving the outcome indicators was periodically reported in the ISRs. That information was used by the PMU to prepare monitoring and evaluation reports that were periodically sent to the Bank task team. c. M&E Utilization: The Bank team used the revised outcome indicators regularly for project supervision purpose and to inform decision making and resource allocation. The project entities also used the indicators to assess the project s achievement particularly when the initial tangible outputs became available. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project did not trigger safeguards review. b. Fiduciary Compliance: Once the initial procurement problems were solved, procurement processes under the project were in line with Bank guidelines. Regarding financial management, all the annual audits of the project were unqualified and carried out in accordance with international auditing standards by a local accounting firm acceptable to the Bank. c. Unintended Impacts (positive or negative): A positive unintended impact is the experience gained by SBV in turning six Payment Processing Centers (PPCs) into regional centers instead of creating new PPCs as originally planned. This was in line with the international trend wherein inter-bank payment systems are increasingly centralized for better efficiency, enhanced reliability and reduced cost. Based on this experience, SBV intends to follow a similar approach in the re-organization of its branch network.
d. Other: Other positive impacts of the project are the long-term capacity building and institutional development of the agencies and banks involved. These include operational policies for the acquired systems issued by SBV; new organizational structures established in participating banks for card and IT centers; human capital development through training and learning by doing processes in the project s entities; and behavioral changes in the management of participating banks triggered by on-line management tools. 12. Ratings: ICR IEG Review Outcome: Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome: Reason for Disagreement /Comments Bank Performance : Satisfactory Borrower Performance : Satisfactory Quality of ICR : Satisfactory Moderately Satisfactory Satisfactory The ICR rated the borrower performance as S after rating the Government performance and that of 2 of the 5 implementing agencies as MS. The ICR would have come with the same overall rating of MS if it had followed the OPCS /IEG Harmonized Evaluation Criteria for ICRs. NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: IEG concurs with the ICR that the project demonstrates the importance of the following factors for the success of IT projects: (i) flexibility in overall solution design and project implementation; (ii) careful implementation planning, particularly the packaging of IT systems for procurement and the sequencing of the acquisition; (iii) changes in management measures and organizational structures of the implementing agencies to support the new systems; (iv) stepwise approach starting from pilot systems under the first PSBM project to upgraded and expanded systems under the second PSBM; (v) strong collaboration between the task team leader, the technical advisor in charge of quality assurance and the procurement specialist for reviewing the complex IT procurement packages; (vi) high technical capability of the staff in both the Bank team and the implementing agencies; (vii) commitment to mobilize adequate resources (staff and funds) in both the implementing agencies and the Bank (for supervision); and (viii) understanding that complex IT projects tend to be rear-loaded for disbursement. 14. Assessment Recommended? Yes No
15. Comments on Quality of ICR: The ICR follows the guidelines and is clearly written. It provides ample evidence of project implementation and achievement of objectives. However, the ICR has some shortcomings. First, there is a lack of efficiency analysis and the data on project costs in Annex I is incomplete. Second, the ICR failed to discuss the achievement of the second project objective, which although had no defined outcome indicators in the M&E framework, had not been formally dropped and had to be evaluated. Finally, it rated the borrower performance as satisfactory, after rating the Government performance and that of 2 of the 5 implementing agencies as moderately satisfactory. a.quality of ICR Rating : Satisfactory