Lao People s Democratic Republic Ministry of Planning and Investment Lao PDR Project for Establishing Public Investment Plan under NSEDP (PCAP3) Second Year Mid-Term Progress Report February 2014 Japan International Cooperation Agency IC Net Limited
JICA Technical Cooperation Project for Establishing Public Investment Plan under NSEDP Second Year Mid-Term Progress Report Abbreviations ADB AMP DOF DPI DPO DPWT JCC JICA MAF MOF MPI MPI-DIC MPI-DOE MPI-DOP MPWT MTEF MTPIP MTR NSEDP ODA PAFO PCAP1 PCAP2 PCAP3 PDM R/D SEDP SPES SPIC SPIS TOT UNDP Asian Development Bank Aid Management Platform (Provincial) Department of Finance (Provincial) Department for Planning and Investment District Planning Office (Provincial) Department of Public Works and Transportation Joint Coordinating Committee Japan International Cooperation Agency Ministry of Agriculture and Forestry Ministry of Finance Ministry of Planning and Investment Department of International Cooperation, MPI Department of Evaluation, MPI Department of Planning, MPI Ministry of Public Works and Transportation Medium-Term Expenditure Framework Medium-Term Public Investment Plan Mid-Term Review National Socio-Economic Development Plan Official Development Assistance Provincial Agriculture and Forestry Office Project for Capacity Building in Public Investment Program Management Project for Enhancing Capacity in Public Investment Program Management Project for Establishing Public Investment Plan under NSEDP Project Design Matrix Record of Discussions Socio-Economic Development Plan Simplified Project Evaluation Sheet Simplified Project Information Checklist Simplified Project Information Sheet Training of Trainers United Nations Development Programme
Contents 1. Project Background... 1 1.1 Background... 1 1.2 Overall Results of First-Year Project Implementation... 2 2. Results of Second-Year Activities until Present (September 2013 to February 2014)... 4 2.1 Plan of Operation for the Second Year (September 2013 to June 2014)... 4 2.2 Activities Related to the Project as a Whole... 5 2.3 Output 1... 5 2.4 Output 2... 7 2.5 Output 3... 8 2.6 Output 4... 10 3. JICA Mid-Term Review for PCAP3... 13 3.1 Outline... 13 3.2 Current Achievement Levels of Outputs... 13 3.3 Current Achievement Level of the Project Purpose... 16 3.4 Five Criteria Evaluation... 16 3.5 Conclusion... 18 4. Issues, Approaches and Lessons Learned... 20 5. Outlines of Activities for the Second Half of the Second Year... 22 5.1 Activities Related to the Project as a Whole... 22 5.2 Output 1... 24 5.3 Output 2... 24 5.4 Output 3... 24 5.5 Output 4... 25 Annexes Annex 1: Project Design Matrix version 1.0 (initial version dated April 30, 2012) Annex 2: Project Design Matrix version 2.0 (recommendation from Mid-Term Review dated February 28, 2013) Annex 3: Plan of Operation
1. Project Background 1.1 Background In the Lao People s Democratic Republic, the Ministry of Planning and Investment (MPI) is vested with both responsibility and authority in all matters related to the management and supervision of public investment projects. The MPI verifies the appropriateness of projects in each field, and reviews the development budget for approval in the National Assembly. The MPI is also assigned to regularly monitor and evaluate projects, and to report the results to the National Assembly. However, the MPI lacks project management ability, as do its local agencies, the Department of Planning and Investment (DPI) in each province, and the district-level District Planning Office (DPO). Therefore, public investment projects that use the domestic development budget are not implemented effectively. Further, the degree to which these projects contribute to the achievement of the targets set out in the government s five-year National Socio-Economic Development Plan (NSEDP) is unclear, resulting in major discrepancies in planning and implementation. Given this situation, a Japan International Cooperation Agency (JICA) technical cooperation project was implemented between November 2004 and October 2007. The project was entitled Project for Capacity Building in Public Investment Program Management (hereinafter PCAP1 ). This was followed by another JICA technical cooperation project from March 2008 to August 2011, entitled Project for Enhancing Capacity in Public Investment Program Management (hereinafter PCAP2 ). PCAP1 and PCAP2 improved the capacity of the MPI and DPI to manage public investment projects by developing, revising, and disseminating forms, methods, and tools for assessing, monitoring, and evaluating projects. They also conducted training programs. The government passed the Public Investment Law in December 2009, prescribing the obligation to assess public investment projects, and positioning the methods and tools developed through PCAP1 and PCAP2 as national systems under the law. A new technical cooperation scheme has been agreed on to address the remaining issues in public investment management, and to utilize outputs from PCAP1 and PCAP2. Entitled Project for Establishing Public Investment Plan under NSEDP (hereinafter PCAP3 ), this scheme is to be implemented from March 2012 to September 2015. To realize the primary objective of sound project management that aligns with NSEDP targets, PCAP3 will develop methods and tools for public investment project management from a mid-term perspective, and further enhance the capacity of the MPI, DPI, and DPO nationwide. More specifically, PCAP3 employs the following four approaches to achieve its primary objective: i) establish a midterm public investment framework and financial management guidelines; ii) develop a process which ensures that project effects will be sustainable after completion of projects; iii) improve the Official Development Assistance (ODA) project information management method; and iv) strengthen districtlevel public investment project management. PCAP3, with the duration of three years and seven months, is divided into three project years. The periods were divided with consideration of the Lao fiscal year (October to September) and the timings for activities that PCAP3 plans to conduct. Below are the three project years applied to PCAP3. First Year: March 2012 to June 2013 (16 months) Second Year: September 2013 to June 2014 (10 months) Third Year: September 2014 to September 2015 (13 months) 1
1.2 Overall Results of First-Year Project Implementation The following table is the implementation schedule of PCAP3 first-year project activities from March 2012 to June 2013. The numbers given to each activity are those of the overall Plan of Operations, as attached in Annex 3. Table 1 Activity Plan and Results, March 2012 June 2013 The table shows that all activities were conducted as originally planned, although there were minor delays in completing the Baseline Survey. 2
For Output 1, the project has developed the prototype of mid-term public investment, which was shared at the Seminar for Mid-Term Public Investment Plan and Financial Management in February 2013. Validation studies of the methods and tools were conducted at Champassak and Oudomxay provinces, and in the Ministry of Agriculture and Forestry (MAF) and the Ministry of Public Works and Transport (MPWT). For Output 2, drafts of terminal and ex-post evaluation workflow and reporting formats were completed. In this process, initial discussions with the Budget Department, Ministry of Finance (MOF) were conducted to clarify the current status of operation and maintenance budget resources. To assess what further modifications were required, the methods and processes of terminal and ex-post evaluation were then tested through two irrigation projects, one in Champassak Province and the other in Vientiane Capital. For Output 3, the workflow for obtaining ODA information from counterpart organizations on the Lao Government side through the Simplified Project Information Sheet (SPIS) was refined. This was done in the expectation that donor project information from the United Nations Development Programme (UNDP)-supported Aid Management Platform (AMP) will be open as a public portal website. For Output 4, two handbooks were drafted, one for the DPO and another for district-level sector offices and district-level project owners. These handbooks will be modified as district demarcation in the 3-Build policy becomes more detailed. Validation of methods and processes was carried out at Oudomxay, Bolikhamxay, and Champassak provinces. 3
2. Results of Second-Year Activities to Date (September 2013 to February 2014) The following table shows the second year activity results by output from September 2013 to February 2014. 2.1 Plan of Operation for the Second Year (Sept. 2013 to June 2014) The following table shows the activity plans by output for the second year of PCAP3. The numbers given to each activity are those of the overall Plan of Operations as attached in Annex 3. Activity Details 2013 2014 9 10 11 12 1 2 3 4 5 6 All 2 3 Holding Joint Coordinating Meeting Formulating Progress Report and Work Completion Report Second Year 31 Implementation of Counterpart Training Second Year - Receiving JICA Mid-Term Evaluation Team [Output 1] A framwwork for Mid-Term Public Investment Plan and Financial Management Guidelines is established Second Year [Output 2] Second Year 19 20 21 2223 Review of the contents of the mid-term public investment plan and financial management guidelines based upon the monitoring results in (7) and revising processes and contents as required Formulation of the Manual and handbooks on the establishment process of the mid-term public investment plan and financial management guidelines for 2016-2020 reflecting the issues in (19) Preparation for seminar on the establishment of proposed mid-term public investment plans and financial management guidelines for 2016-2020 Measures for effective operation and maintenance of public investment projects after its completion are established Updating the methods and process in the Manuals and Handbooks based on the revised contents as referred to in (11) Preparations for training on project evaluation study [Output 3] ODA project information management is improved 24 Revision of processes based on the review results in (14) Second Year 25 Updating the methods and processes in the Manuals and Handbooks based on the revised contents in (24) 26 Preparations for training on ODA project information management methods [Output 4] 27 A mechanism of management and capacity development for district-level public investment projects is established Implementation of training on managing district-level public investment projects at the monitoring organization Second Year 28 Review of the status of district-level public investment project management at the monitor organization 29 Updating manuals and handbooks based on the review results in (28) 30 Preparations for TOT for the purpose of establishment of district-level public investment projects and capacity building Activities Training and Seminars Completion of Manuals and Handbooks 4
The following subchapters explain the activity results from September 2013 to February 2014, based on the initial Plan of Operations. 2.2 Activities Related to the Project as a Whole The results and measures for overall activities that were originally scheduled between September 2013 and February 2014 are as follows: 1) Holding the JCC Meeting The Third Joint Coordinating Committee (JCC) Meeting was planned for January 2014, after completion of the Mid-Term Review (MTR) studies of PCAP3. However, because of a change in the schedule of the MTR studies, and changes in the schedules of JCC members, the meeting was moved to March 6, 2014. Details of this JCC Meeting are provided in Section 5.1. 2) Mid-Term Implementation Report It was agreed between JICA and PCAP3 that the Second Year Mid-Term Progress Report will be submitted at the end of February 2014. 3) Receiving Mid-Term Review Studies PCAP3 received the MTR Study team from January 15 to February 4, 2014. Details of the study and its results are provided in Chapter 3. 2.3 Output 1 (1) Activity Contents The activities necessary to accomplish Output 1 are listed from 4) to 33) in the table below. Table 2: Output 1 Activity Plan [Output 1] A Framework for Mid-Term Public Investment Plan and Financial Management Guidelines is Established 4) Implementation of the baseline survey, selection of pilot and monitor organizations First Year 5) Establishment of the draft mid-term public investment plan and financial management guidelines 6) Holding a joint seminar on the proposed mid-term public investment plan and financial management guidelines 7) Formulation of the proposed mid-term public investment plan and financial management guidelines by the monitor organization Second Year 19) 20) 21) Review of the contents of the mid-term public investment plan and financial management guidelines based upon the monitoring results in 7) and revising processes and contents as required Formulation of the Manual and handbooks on the establishment process of the mid-term public investment plan and financial management guidelines for 2016-2020 reflecting the issues in 19) Preparation for seminar on the establishment of proposed mid-term public investment plans and financial management guidelines for 2016-2020 Third Year Preparation for and implementation of seminars on establishment of 2016-2020 proposed mid-term public 32) investment plan and financial management guidelines for central government organizations and provinces Support for establishment of 2016-2020 proposed mid-term public investment plan and financial 33) management guidelines 5
19) Review of the contents of the five-year public investment plan and financial management guidelines based upon the monitoring results in 7) and revising processes and contents as required In the first six months of the second year, based on discussions with pilot and monitoring organizations in the first year, PCAP3 has focused on developing the contents of the five-year public investment plan, including the financial management method. PCAP3 experts have had continuous discussions with their counterparts in the Department of Planning, MPI (MPI-DOP) during this development process. In November 2013, the MPI and the Project held a seminar on formulation of the mid-term public investment plan (MTPIP) at the MPI. The aims of the seminar were as follows. To understand the concept of the five-year MTPIP To validate the proposed methods and tools To identify necessary action to be taken towards the formulation of the five-year MTPIP To clarify the roles of MPI staff members Both directors of the MPI-DOP and the Department of Evaluation, MPI (MPI-DOE) and more than 30 staff members of the MPI took part in this seminar and discussed the concept and contents of the five-year MTPIP proposed by the Project. The participants presented their counterproposals to the Project proposals. These inputs were taken into consideration when drafting the Manual for Medium-Term Public Investment Management The seminar participants discussed how to improve the feasibility of public investment plans. They appreciated the importance of selecting priority public investment programs and projects within the allotted budget for the next three to five years. The MPI is expected to discuss the development budget; the budget allocated for public investment projects; for the next three to five years with the MOF prior to finalization of the five-year MTPIP. 20) Formulation of the manual and handbooks on the establishment process of the midterm public investment plan and financial management guidelines for 2016 2020 reflecting the issues in 19) After the seminar in November 2013, the project personnel started drafting the Manual and guidelines for financial management. The major contents of the Manual are as follows. Definition of a public investment plan Relationship between public investment and development plans Objectives of the five-year MTPIP Concept and contents of the five-year MTPIP Formulation process of the five-year MTPIP The MPI is expected to work with the MOF to decide the development budget for upcoming years, along with expected annual growth in the budget. PCAP3 experts and their MPI counterparts had approached the Department of Fiscal Policy and the Department of Budget of the MOF in November 2013 to discuss the possibility of collaboration. Both departments of the MOF are willing to work with the MPI to improve the feasibility of public investment. The MPI is expected to take proactive action to realize this collaboration. For the first time, the Lao Government is going to formulate the five-year public investment plan in parallel with the NSEDP and Socio-Economic Development Plans (SEDPs). Guidelines for the formulation of the NSEDP 2016 2020 therefore need to refer to the linkages between the NSEDP and the five-year MTPIP. Project personnel have submitted draft NSEDP guidelines to the MPI. 6
(2) Development Progress of Output 1Results The Project has been drafting the manual and handbook for the five-year MTPIP and guidelines for financial management; they are expected to produce the drafts in February 2014. The manual and handbooks are to be finalized by the end of the second year through discussions with the MPI, the provinces, and sector ministries. 2.4 Output 2 (1) Activity Contents The activities necessary to accomplish Output 2 are listed from 8) to 35) in the table below. At this stage, the Project will have an updated manual and handbooks, as well as training materials for the third implementation year that incorporate the evaluation workflow and reporting formats prepared in the first implementation year. The activities and results are shown below, itemized according to Project implementation headers. Table 3: Output 2 Activity Plan [Output 2] Measures for effective operation and maintenance of public investment projects after its completion are established First Year 8) Discussions with the Ministry of Finance regarding the operation and maintenance costs after the completion of the public investment projects 9) Establishment of methods and processes for the terminal and ex-post evaluations of public investment projects 10) Pilot project review based on project evaluation studies 11) Revisions based upon the results of 10) above Second Year Third Updating the methods and process in the Manuals and Handbooks based on the revised contents as 22) referred to in 11) 23) Preparations for training on project evaluation study 34) Implementation of training on the methods for project evaluation study Year 35) Implementation of meta-evaluation (self-evaluation of evaluation tasks), confirmation of project evaluation performance 22) Updating the Methods and Processes in the Manuals and Handbooks Based on the Revised Contents Progress: This activity was conducted according to the original plan. The Project started updating the manual and handbooks, incorporating the workflow and methodology of terminal and ex-post evaluation that had been already developed in the first year. After two review workshops, the final drafts of the manual and handbooks are currently in the editing process. Developing the process of the Manual and Handbooks: MPI officials, along with the PCAP3 experts, prepared the first draft of the manual and handbooks in September 2013. From October 9 to October 11, 2013, the Project hosted the first Review Workshop to decide on modifications to the second draft. To develop the final draft, the second Review Workshop was held from December 23 to 25, 2013. At present, the final drafts of the manual and handbooks are being edited, incorporating comments from workshop participants and their respective organizations. 7
Counterpart officials: The participants of the development process were regularly assigned counterpart officials from the MPI and the pilot organizations. There were 30 of these officials in all, from the DPI, the Department of Public Works and Transportation (DPWT), the Provincial Agriculture and Forestry Office (PAFO), the Department of Finance (DOF), the MPWT, the MAF and the MOF. From the outset, these officials have regularly participated in a series of Project activities, namely, the baseline survey, workshops, evaluation practices, and seminars with regular members. They have thus learned how to be more effective leaders in evaluating issues. The joint activities have also deepened their understanding and knowledge about the importance of an institutional framework, universal principles, and the methodology and procedure of evaluation. They are now very well equipped to take on key roles as core trainers at nationwide training courses to be conducted in the third implementation year. Table; Composition of the counterpart officials from MPI and Pilot Organizations Category Organization Number of officials MPI officials MPI-DOE 3 section chief level and 3 Pilot Organization officials DPI, PAFO, DPWT, DOF of Champassak Province and Vientiane Capital MPWT, MAF, MOF staff level officials 6 officials in all From 2 to 4 officials of director/section chief level or staff level Approx. 30 officials in all 23) Preparations for Training on Project Evaluation Study The second draft of training materials was prepared, incorporating contents of the updated manual and handbooks. They will be the main materials for the nationwide training courses to be held in the third year of implementation. (2) Development Progress of Output 2 Results The Manual for Public Investment Project Evaluation has been prepared in an independent volume so as to be convenient for users. The major contents of the Manual are as follows: i) background of evaluation; ii) methodology of evaluation; iii) how to implement evaluation studies; iv) required items in the evaluation reports; and v) blank formats and examples. In other words, everything essential for officials who implement evaluative studies, from basic principles to concrete know-how, has been compiled into one volume. The handbooks are being edited to be consistent with the Manual. The training materials consist of slide shows and pre- and post-tests. The slide shows are in four parts, and they explain the contents of the Manual in simple, user-friendly terms. They also include photos of previous field practices and actual project sites, so that trainees can easily visualize what to do during actual evaluations. The pre- and post-tests have already been prepared and updated after verification trials at the second Review Workshop. All these materials incorporate the workflow and reporting formats for terminal and ex-post evaluation that were prepared in the first implementation year. 2.5 Output 3 (1) Activity Contents The activities necessary to accomplish Output 3 are listed from 12) to 37) in the table below. The final drafts of the improved workflow and SPIS formats have already been prepared and verified, except for one part, which has been held up because of the delay in AMP public portal website disclosure. The 8
draft focuses on using information in the public portal website and is consistent with key terminologies on that site. Training materials have already been updated with the drafts. The activities and results are shown below, itemized according to Project implementation headers. Table 4: Output 3 Activity Plan [Output 3] ODA project information management is improved 12) Discussions with donors on standardization of ODA projects within the scope of public investment First 13) Establishment of an improvement process regarding ODA information management 14) Pilot review of new ODA projects 24) Revision of processes based on the review results in 14) Second 25) Updating the methods and processes in the Manuals and Handbooks based on the revised contents in (24) 26) Preparations for training on ODA project information management methods Third Implementation of training on information management methods for ODA projects in consultation with 36) donors 37) Monitoring information management performance in respect of ODA projects 24) Revision of Processes Based on the Review Results in 14) This activity has already been completed with the collaboration of the Disbursement Division, Road Department of MPWT. The following key issues were raised for improvement of the process. Is the workflow supported by a functioning mechanism? Does the format match the capabilities of the users who will actually implement the project? What is the most important factor that will make the workflow and formats work? In the previous workflow of the process, it was mandatory for the project owner agencies to prepare and submit SPIS; no particular divisions in the agencies were specified for this task. The improved workflow requires the cooperation of disbursing divisions within the agencies, which can provide most of the information requested in the SPIS. As pilot cases, project personnel have verified the effectiveness of the improved workflow for completion of SPIS formats with the Disbursement Division, Road Department of MPWT on two road construction ODA projects. Some minor improvements recommended by the MPWT were incorporated into the workflow and SPIS. 25) Updating the Methods and Processes in the Manuals and Handbooks Based on the Revised Contents in 24) The contents of the improved workflow and SPIS have been incorporated into the present version of the Manual for Public Investment Project Management and handbooks. This consolidates all available information on the AMP public portal website for SPIS, and ensures consistency in key terminologies between the AMP and SPIS. 26) Preparations for Training on ODA Project Information Management Methods The final drafts of the training materials are under completion, incorporating the improved contents of the Manual and handbooks. They will be completed by March 2014, despite the 9
delay in the disclosure of the AMP public portal website. There are no critical obstacles to completion of this activity. (2) Development Progress of Output 3 Results The major products of Output 3 are the updated versions of the chapters incorporated in the Manual for Public Investment Management and related handbooks. As stated above, the only remaining task is that soft copies of the relevant parts are to be uploaded on the AMP public portal site. 2.6 Output 4 (1) Activity Contents The activities necessary to accomplish Output 4 are listed from 15) to 39) in the table below. This series of activities aims to develop district-level public investment management methods, along with effective training methods for diffusion to districts nationwide. Among the activities, those scheduled to be implemented in the second Year of the Project are listed from 27) to 30). The status of the activities conducted in the second Year is as follows. Table 5: Output 4 Activity Plan A mechanism for management and capacity development for district-level public investment projects [Output 4] is established 15) Discussions within the MPI and with relevant authorities on establishment and management of district-level public investment projects First 16) Implementation of a baseline study to select pilot and monitoring organizations Year 17) Development of processes and training methods for establishment and management of district-level public investment projects in the pilot organizations 18) Preparation of training in monitoring organizations on managing district-level public investment projects 27) Implementation of training on managing district-level public investment projects at the monitoring organization Second 28) Review of the status of district-level public investment project management at the monitor organization Year 29) Updating manuals and handbooks based on the review results in 28) Third 30) 38) Preparations for TOT for the purpose of establishment of district-level public investment projects and capacity building Implementation of TOT for each DPI for the purpose of establishment and management of district-level public investment projects Year 39) Evaluation of training implemented by the DPIs at district level 10
27) Implementation of training on managing district-level public investment projects at the monitoring organization A workshop on the development of the handbook and Training of Trainers (TOT) materials was conducted for MPI and Oudomxay Province (pilot province) officials from September 23 to 26, 2013 in Vientiane Capital. Five DPI officials from Oudomxay Province who had undergone TOT training by MPI officials in the first year participated in the workshop. Two officials of the MPI-DOE and one official of the MPI-DOP attended. The two handbooks, Public Investment Project Management Handbook for District Planning Office and Public Investment Project Management Handbook for District Sector Office and District Project Owner, as well as PowerPoint materials for the TOT have been developed based on a series of discussions through the workshop. The drafts of the Simplified Project Information Checklist (SPIC) formats that will be newly introduced into district-level public investment project management were examined. The curriculum and agenda for the TOT sessions in October and November 2013 were also prepared. Objectives of TOT at the monitor provinces are as follows: Better understanding of Public Investment Law and its draft Implementation Decree (draft) Clearer perception of the new roles of the DPO in public investment project management at district sector offices Clearer demarcation between the responsibilities of the province and the district Enhancement of the capacities of provincial and district authorities in terms of public investment management The TOT curriculum is divided into a theoretical part and a practical part. As trainers, Oudomxay provincial officials delivered theoretical lectures to officials from Bolikhamxay and Champassak provinces. Summary of TOT (theoretical part) for monitor provincial officials 1) Date: October 23 25, 2013 2) Venue: International Cooperation and Training Center in Vientiane Capital 3) Main components and duration: TOT from Oudomxay provincial officials to Bolikhamxay and Champassak provincial officials: two and a half days Evaluation of training program / trainers performance and preparation for TOT (practice part): Half a day 4) Contents TOT is composed of 8 units according to the table of contents of the draft handbook. Each unit includes a lecture, a Q and A session, and a content check session. 5) Participants Seven (7) Bolikhamxay provincial officials Eight (8) Champassak provincial officials Photo 1 TOT on District-Level Public Eight (8) MPI officials Investment Project Management Four (4) Oudomxay provincial officials For the latter part of the TOT course, Bolikhamxay and Champassak provincial officials delivered lectures to DPO officials from 3-Build pilot districts in each province. 11
Summary of TOT (practical part) for monitor provincial officials 1) Date: November 5 7, 2013 in Bolikhamxay province, November 12 14, 2013 in Champassak province 2) Venue: Paksan DPO meeting room in Bolikhamxay province, DPI meeting room in Champassak province 3) Main components and duration: Practical Training from DPI officials to DPO officials in 3-Build pilot districts, including a pre-test and completion test: two and half days Formulation of future action plan for capacity building among DPI and DPO officials: Half a day 4) Contents: The TOT course is composed of eight units according to the table of contents of the draft handbook. Each unit includes a lecture and a Q and A session. TOT has a practical exercise using a case from the actual proposed public investment project, so that DPO officials might learn how to fill out the draft of the SPIC format. 5) Participants Monitor Province MPI DPI 3 Build pilot district Bolikhamxay 3 DOE officials (2 technical officials and 1 coordinator) 7 officials 13 DPO officials Champassak 3 DOE officials (2 technical officials and 1 coordinator) 8 officials 15 DPO officials Photo 2: TOT on District-Level Public Investment Project Management in Bolikhamxay Province Photo 3: TOT on District-Level Public Investment Project Management in Champassak Province (2) Development Progress of Output 4 Results The drafts of the two handbooks and the TOT training materials have been developed through the activities described above. However, the Prime Minister s Decree on the Implementation of Public Investment Law, which will be the key source document for drafting the handbooks, is still not officially approved. Moreover, the 3-Build pilot implementation is still in progress. Possible changes in policies are therefore anticipated, and the draft handbooks and TOT training materials would need to be modified according to such changes. The handbooks and TOT contents will also be revised based on the results of reviews in monitor organizations during the second half of the second 12
implementation year. The main manuals will be revised accordingly after the contents of the draft handbooks are determined. As for trained trainers, 20 DPI officials (five in Oudomxay Province, seven in Bolikhamxay Province, and eight in Champassak Province) are now capable of training DPO and district sector department officials. 3. JICA Mid-Term Review for PCAP3 This chapter summarizes the results of the MTR Studies of PCAP3 conducted by JICA. Specific details of the Studies should be available from the JICA Laos Office and the JICA main office in the form of the Mid-term Report. 3.1 Outline JICA conducted an MTR to evaluate the current status of PCAP3. The objectives of the MTR were to examine the achievements and process so far by reviewing the PCAP3 Project Design Matrix (PDM) and the Plan of Operations, checking the current status of inputs and activities, and assessing achievement of outputs and the Project Purpose. Any issues that may affect the achievement of outputs and the Project Purpose were compiled, and recommendations were forwarded on how to overcome these. The MTR studies were conducted from January 15 to February 4, 2014. The MTR team consisted of members from both Japan and Laos. The following table shows the team members: Japanese Side Lao Side MTR Team Members Name Designation Title Mr. Koichi Takei Leader Chief Representative, JICA Laos Office Dr. Nobuhisa Takeda Government Project Planning JICA Senior Advisor Ms. Yu Nagaiwa Cooperation Planning JICA Laos Office Ms. Makiko Fujita Evaluation Analysis Asuka World Consultants Co., Ltd. Name Designation Title Senior Consultant Dr. Khamlien PHOLSENA Project Director Vice Minister, MPI Mr. Sengmany KEOLANGSY Project Manager Acting Director General, MPI-DOE Mr. Ounheaune CHITTHAPONG Deputy Project Manager Deputy Director General, MPI-DOP Mr. Houmphanh SOUKPRASITH Deputy Project Manager Deputy Director General, MPI-DIC The team reviewed documents related to the Project such as the Record of Discussions, (R/D), the latest PDM, the Plan of Operations, progress reports, and minutes of meetings. The team conducted a questionnaire survey and a series of interviews and discussions with the officials and organizations concerned. The team also undertook site visits and interviews at Vientiane Capital, Champassak, and Oudomxay provinces. These results were used to ensure consistency with the interviews of PCAP3 experts and their counterpart officials from the MPI. 3.2 Current Achievement Levels of Outputs The following tables indicate the MTR team s analysis of achievement levels for each output based on the current status of each indicator. 13
(1) Output 1 The following table indicates the current achievement levels of the objectively verifiable indicators for Output 1 as analyzed by the MTR team [Output 1] A framework for Mid-Term Public Investment Plan and Financial Management Guidelines is established. Objectively Verifiable Indicators 1-1 A draft Mid-Term Public Investment Plan 2011-2015 is prepared in the pilot organizations. 1-2 A draft Mid-Term Public Investment Financial Guideline 2016-2020 is prepared by the Ministry of Planning and Investment 1-3 Methods to develop and monitor the Mid-Term Public Investment Plan and the Mid-Term PIP Financial Guideline are included in the manuals and handbooks. Achievement level Satisfactory Satisfactory Satisfactory Output 1 has not been achieved, but the likelihood of achievement by the end of the Project is high. Although the Output 1 indicators have not been achieved at the time of the MTR, both the draft MTPIP 2011-2015 and the Mid-Term Public Investment Financial Management Guideline 2016-2020 have been already shared with the MPI and the pilot and monitor organizations. All parties have been in a productive dialogue, and have been working for the past few months to ensure the most effective utilization of the Project s resources within the Lao policy framework. (2) Output 2 The following table indicates the current achievement levels of the objectively verifiable indicators for Output 2 as analyzed by the MTR team: [Output 2] Measures for effective operation and maintenance of public investment projects after its completion are established. Objectively Verifiable Indicators 2-1 The Ministry of Planning and Investment and the Ministry of Finance agrees on budget conditions including costs for maintenance and operation of completed public investment projects. 2-2 Methods and procedures for terminal and post evaluation studies are developed and reflected in the manuals and handbooks. 2-3 Method and procedures for terminal and post evaluation studies are known through training etc. to ministry province and district organizations concerned. 2-4 The Ministry of Planning and Investment announces public investment projects that terminal and post evaluations will be conducted. 2-5 Number of public investment projects that submit completion report; with operations and maintenance plan attached; increases. Achievement level Challenging Satisfactory Satisfactory Fair Fair The likelihood of achieving Output 2 by the end of the Project is medium. Of the five indicators under Output 2, only one indicator (indicator 2-2) has been achieved at the time of the MTR. The achievement of indicators 2-3 and 2-4 is likely. However, the achievement of indicators 2-1 and 2-5 will be challenging. 14
Indicator 2-1 is hard to achieve because of the difficulties and complexity of obtaining the budgets required for operation and maintenance of completed public investment projects, whether from the development budget (sourced from the MPI) or from the recurrent budget (sourced from the MOF). Achieving indicator 2-5 is possible, but actual implementation of the operation and maintenance plans would be challenging because of the limitation of the Lao Government s overall budget allocation. (3) Output 3 The following table indicates the current achievement levels of the objectively verifiable indicators for Output 3 as analyzed by the MTR team. [Output 3] ODA project information management is improved. Objectively Verifiable Indicators 3-1 Development partners and the Lao Government are able to read ODA project information inclusive of counterpart funds. 3-2 A guideline on ODA project information management is included in the manuals and handbooks 3-3 Methods and procedures of ODA project information management are disseminated to ministries, government organizations, provinces and district organizations concerned. Achievement level Achieved Satisfactory Satisfactory Output 3 is likely to be achieved by the end of the Project. Of the three indicators, one indicator has been achieved, and two have shown satisfactory progress at the time of the MTR. (4) Output 4 The following table indicates the current achievement levels of the objectively verifiable indicators for Output 4 as analyzed by the MTR team. [Output 4] A mechanism of management and capacity development for district-level public investment projects is established. Objectively Verifiable Indicators Achievement level 4-1 The Ministry of Planning and Investment develops and announces roles and responsibilities of district-level public investment project Fair management. 4-2 Methods and procedures for district-level public investment project Satisfactory management are included in the manuals and handbooks. 4-3 More than 5 provincial Department of Planning and Investment officials are capable of conducting training to district planning offices and district Satisfactory sector departments. The likelihood of achieving Output 4 by the end of the Project is medium. Of the three indicators, two indicators show satisfactory progress, and one shows fair progress at the time of the MTR. However, external conditions, particularly political influence in authority levels of public investment management, might impact adoption of the methods and tools in the future. Therefore, the direction of management of public investment projects is somewhat uncertain. 15
3.3 Current Achievement Level of the Project Purpose The following table shows the MTR team s analysis of achievement levels for each indicator output based on the current status of each indicator: [Project Purpose] The Ministry of Planning and Investment and its subordinate organizations manage public investment projects through a Mid-Term Public Investment Plan and a comprehensive framework. Objectively Verifiable Indicators P-1 A guideline to draft Mid-Term Public Investment Plan 2016-2020 that is related to the next NSEDP2016-2020 is officially announced from the Ministry of Planning and Investment to ministries, government organizations, provinces and districts. P-2 Terminal evaluation studies for completed public investment projects are conducted. Achievement level Satisfactory P-3 Post evaluation studies for completed public investment projects are conducted. Fair P-4 Accuracy of information is confirmed through ODA counterpart fund request documents that the Lao side applies, and the information is shared Satisfactory with the donor projects. P-5 Increased number of projects that are managed in the district levels. Fair Fair The Project Purpose is considered achieved when there is satisfactory management of public investment projects by MPI and its subordinate organizations through an MTPIP and a comprehensive framework. Based on the progress of activities towards the achievement of five indicators of the Project Purpose as well as the progress of the four Outputs, it is currently difficult to judge if the Project can achieve the Project Purpose in the given timeframe. Meanwhile, the MTR team noticed that the current indicators P-2 and P-3 are not clearly described to measure the achievement level of the Project Purpose. The Project is expected to specify those indicators with measurable targets. 3.4 Proposal of Change in Indicators The current indicators of the Project Purpose and Output 1 are not clearly described; therefore, measuring their respective achievement levels is difficult. The Project is expected to specify these indicators with measureable targets. The MTR team proposes the following modifications in the indicators: Current Indicators Project Purpose P-2: Terminal evaluation studies for completed public investment projects are conducted (specific indicative figures to be determined through project baseline studies) P-3: Post evaluation studies are conducted (specific indicative Issues It is almost impossible to conduct terminal evaluation studies for all ongoing public investment projects before the end of the Project, because of the large number of public investment projects near completion. However, it would be possible to do so if the number of projects are limited based on the appropriate scope of target public investment projects. It is almost impossible to conduct post evaluation studies for all public investment Newly Proposed Indicators P-2: Evaluation studies for completed public investment projects are conducted for projects to be selected by the relevant authorities with operation & maintenance plans attached 16
figures to be determined through project baseline studies) P-4: Accuracy of information is confirmed through ODA counterpart fund request documents that the Lao side applies, and the information is shared with the donor projects P-5: Increased number of projects that are managed in the district levels projects before the end of the Project, because of the large number of public investment projects at completion. However, it would be possible to do so if the number of projects are limited based on the appropriate scope of target public investment projects. P-3 (old) is combined with P-2 (old), thus P-3 (old) becomes P-2 (new). No revision in the indicator but becomes P- 3 (new) instead of P-4 (old) because of the integration of P-2 and P-3(old) as P- 2(new). No revision in the indicator but becomes P- 4 (new) instead of P-5 (old) because of the combination of P-3 and P-4 (old) as P-3 (new). P-3: Accuracy of information is confirmed through ODA counterpart fund request documents that the Lao side applies, and the information is shared with the donor projects P-4: Increased number of projects that are managed in the district levels Current Indicator Output 1 1-2: A draft Mid-Term Public Investment Financial Guidelines 2011-2015 is prepared by MPI. Issues The Mid-Term Public Investment Financial Management should be prepared in close coordination with the MOF and should follow the government policies on financial discipline. New Proposed Indicators 1-2: A draft Mid-Term Public Investment Financial Guidelines 2016-2020 is prepared by MPI in line with the government policies on financial discipline. 3.5 Five Criteria Evaluation The MTR team provided the following evaluation results based on the five evaluation criteria: (1) Relevance The Project is highly relevant to the national, international, and Japanese policy agenda. The design, counterpart team composition, model selection, and Japanese expert team are well elaborated, given the Project Purpose. (2) Effectiveness At the time of the MTR, the effectiveness of the project implementation is moderate. This rating is based on assessment of progress towards achievement of the Project Purpose, logical relations between the Project Purpose and outputs, and important assumptions under the current PDM ver. 1.0. While the indicators to measure progress towards achievement of the Project Purpose are available, they lack measurable targets. Therefore, it is difficult to judge the effectiveness of implementation. It is strongly recommended that the MPI and PCAP3 set specific and measureable targets to be achieved by the end of the Project. (3) Efficiency 17
The efficiency of the Project is considered high based on assessment of the inputs and the level of achievement of the outputs at the time of the MTR. (4) Impact It is very difficult to judge the impact of the Project outcomes at this stage of implementation. This is due to the unavailability of data on the indicators of progress towards the Overall Goal. However, the following positive impacts of outputs and activities so far have been observed: The Asian Development Bank (ADB) is willing to work closely with PCAP3 for financial framework reform, particularly through its support to the MOF in developing a Medium- Term Expenditure Framework (MTEF), which has a strong linkage with the Project s multiyear rolling public investment plan. As indicated in the Mid-Term Review Report of the 7th NSEDP (2011 2015), the UNDP is willing to closely work with PCAP3 for a formulation of the MTPIP that includes mid-term financial management. The MPI has already set up a committee that selects priority public investment projects based on assessment of approval requests using tools developed in PCAP1 and PCAP2. DPIs are willing to continue providing capacity building training to districts and sector departments from their own domestic budgets. For instance, technical promotion project requests are being funded out of the development budget. Provincial DPI officials are willing to work with sector departments and the DOF to conduct efficient and effective evaluation studies on completion of public investment projects. (5) Sustainability The Project sustainability is viewed as moderate at the time of this MTR, because the MTR team was unable to confirm the MPI s commitment to sustain the Project from the political, financial, and technical aspects. The MTR team is encouraged that the Project takes serious measures to ensure sustainability with concrete exit strategies. In terms of rolling out the Project achievement and lessons learned, more serious commitment is expected from the MPI in the latter half of the implementation period. It is necessary to develop proper exit strategies and promote self-reliant development of the managerial systems implemented by the Project. 3.5 Conclusion Based on the above results, the conclusions of the MTR Report are as follows: The Midterm Review Team found that the activities of the Project Purpose and each Output which can work inside of MPI- DOE have made good progresses. Meanwhile, regarding Output 1 and Output 2, there is little progress on the discussion between MPI-DOP and MOF Fiscal Policy department and budget department which have the authority of budgeting policy. In Lao PDR, MPI which is responsible for and authorized to manage NSEDP, and MOF which unifies the national revenue and expenditure are involved in public investment plan and its implementation from the different aspects. Public investment plan is approved by MPI, DPI or/and DPO, however the approval is not always linked with the budget allocation of MOF. Therefore, the budget has not often allocated to the approved public investment plan. Moreover, the operations and maintenance costs for completed public investment projects also have not been secured. These situations have hindered the expressions of development effects from the public investment. Thus, in order to make the Project more effective on the mid-term public investment management and the operation and maintenance costs, it is necessary to have careful consideration to securing budgets on public investment. Therefore, it is essential to build up stronger coordination and information sharing between MPI-DOP and MOF. Based on the above aspects, with respect to Output 1 and Output 2 which are fundamental elements of public investment 18