Public Disclosure Authorized IEG ICR Review Independent Evaluation Group Report Number : ICRR13740 1. Project Data: Date Posted : 05/31/2012 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Country: Tunisia Project ID : P088929 Appraisal Actual Project Name : Tunisia Information Project Costs (US$M US$M): 15.37 10.518 And Communication Technologies Sector Development Project L/C Number: L7244 Loan/Credit (US$M US$M): 13.13 10.518 Sector Board : Global Cofinancing (US$M US$M): 0 0 Information/Communi cations Technology Cofinanciers : Board Approval Date : 07/01/2004 Closing Date : 06/30/2009 12/31/2010 Sector(s): Central government administration (45%); Telecommunications (40%); Law and justice (10%); General education sector (5%) Theme(s): Regulation and competition policy (29% - P); Infrastructure services for private sector development (29% - P); Education for the knowledge economy (14% - S); Other social development (14% - S); Access to law and justice (14% - S) Prepared by : Reviewed by : ICR Review Group: Coordinator : Melvin P. Vaz Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The project development objectives (PDOs) as stated in the Project Appraisal Document (PAD, pg 3) and Loan Agreement (LA, pg 13) are identical: to assist the Borrower in promoting the development of its ICT sector by : (a) supporting ICT institutional and sector reforms; (b) improving its e-security mechanisms; (c) developing e-government applications; and (d) promoting the participation of the private sector in the ICT sector. b.were the project objectives/key associated outcome targets revised during implementation? No c. Components: There were four components. 1. Information and Communications Strategy, Policy, and Regulation (US$2.96 million at appraisal, US$2.039 million at closure): This component would assist the Government on further opening of the ICT sector to private participation and competition. It included the following activities: elaboration of a strategic long-term plan for developing the ICT sector, including a strategy for promotion of universal access; assistance for the award of additional licenses; and assistance to the National Telecommunications Agency (Instance Nationale des Telecommunications or INT), the sector regulatory agency. 2. E-government Strategy, Services, and Applications (US$5.61 million at appraisal, US$4.189 million at closure): It included the following activities: formulate an e-government benchmark and strategy (coordinated by the State Secretariat for Administrative
Reform); develop e-government applications and on-line services in government ministries and agencies; provide solutions to equip selected schools and social service centers with specialized Information and Technology (IT) solutions for the disabled under the direction of Ministry of Social Affairs and Solidarity; assist Tunisian Internet Agency in the development of an Arabic Domain Name System (DNS); and increase the ICT training capacity of Institut Superieur des Etudes Technologiques en Communications de Tunis (Institute for Higher Studies in Communications Technologies of Tunis) (ISET'Com) by providing a distance-learning platform and content for building ICT capacity in Tunisia. 3. E-security (US$2.92 million at appraisal, US$3.188 million at closure): It included the following activities: develop and implement a computer risk-prevention strategy based on the Computer Emergency Response Team (CERT); strengthen e-security research and training capacity; and establish a data backup center to ensure continuity of government's critical computer applications. 4. Project Management and Support to the Preparation of the World Summit on the Information Society 2005 (US$1.64 million at appraisal, US$1.101 million at closure). This component would provide technical assistance to the government to implement a complex and fast-paced project with emphasis on results by World Summit on the Information Society in 2005. It included the following activities: technical assistance to strengthen the Project Management Unit; support an ICT statistical and economic analysis unit with the Ministry of Communication Technologies and Transport (MCT) to develop and implement Monitoring and Evaluation (M&E) activities, measure progress in the sector, publish a regular M&E report, and provide input to policy -makers; and preparatory activities for the World Summit in 2005. The following changes that were made to the project components on December 9th, 2009 (five years after approval and about a year before project closing ) did not impact the PDO: Three project activities were cancelled: the Virtual Library and Distance Learning Platform in Component 2 (e-government) and the IT Security Lab in Component 3 (e-security); and Three project activities were added: Women in ICT in Tunisia and ICT Outsourcing in Component 1 and the E-Disabled Pilot that belonged to Component 2. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The total estimated cost of the project at appraisal was US$ 15.37 million, of which US$2.24 million was to be financed by the government and US$13.13 million from IBRD. The total cost of the project of US$10.518 million was completely financed by IBRD. Neither the ICR nor subsequent communications from the project team contain any information on Borrower contribution. A significant portion of the loan amount was cancelled, bringing down the loan from US$13.1 million at appraisal (from the PAD) to US$10.5 million at closure (from TTL). The project was appraised on May 20, 2004, approved on July 01, 2004 and the loan became effective within six months after Board approval on January 18, 2005. The original closing date (June 20, 2009) was extended by 18 months to December 31, 2010 due to delays in local procurement clearance processes and lengthy validation of the deliverables (ICR pg 5). The project received a disbursement delay flag due to a disbursement lag of 45 months during implementation. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial : The objectives were in line with Tunisia's 10th Development Plan 2002-2006 (the central objective of which was to foster the emergence of a knowledge-based economy). One of the goals of the 10th Plan was to increase ICT revenues from 3.3 percent of GDP in 2001 to 7 percent in 2006 and to generate one out of every four jobs in the ICT sector by end 2006. The project s objectives were consistent with the Bank s 2009-2012 Country Assistance Strategy for Tunisia, current at closure. The CAS emphasizes support for the development of Tunisia s knowledge economy, centering on the further expansion of the ICT sector as part of a strategy to make the services sector overall more competitive. The objectives were also relevant to the 2005-2008 CAS, which identified modernization of infrastructure services to increase quality and efficiency, and opening of investment to the private sector. b. Relevance of Design: Substantial : The activities and components of the project in the Project Appraisal Document (PAD) were well designed and consistent with the PDO of assisting government in promoting the development of its ICT sector. The lending instrument (a technical assistance loan) was appropriate because there was agreement between the Bank and the government on both the necessity for, and the modalities of, ICT sector reform. The PDO and the four sub-objectives (Private Sector, e-government, e-security and ICT Policy) of the project were clearly stated in the PAD
and in the Credit Document. The results framework showed clear linkages between the activities, components and intermediate outcomes of the project. The indicator for the PDO (to increase the share of the ICT sector in the GDP) was not appropriate for a relatively small-scale technical assistance project because of problems in attributing the results to the contributions made by the project. Exogenous factors which could affect the outcome of the project (for example, the government's inherent bias towards public sector service provision, decentralization of project activities to too many agencies, restrictive government policies on internet which may restrict the expansion of internet service providers) were clearly identified and addressed. 4. Achievement of Objectives (Efficacy): Overarching Objective : Promoting the development of Tunisia's ICT sector : Substantial The project-supported outputs, intermediate and final outcomes intended to lead to the achievement of the overarching objective are discussed below : Outputs: 1) Supporting ICT institutional and sector reforms : Training was provided to INT on topics ranging from tariff regulation to new 3G technologies and cost accounting. There were workshops and technical assistance on competition in the telecommunication sector, local loop unbundling, and interconnection tariffs. Assistance was provided in grating eight licenses to three operators (two fixed, two VSAT, 2 GSM, 3 Wimax and 1 global), one of which was a global licence awarded to Orange Tunisie, the first of its kind in the region. A study on Women and ICT, which was added in December, 2009. This analyzed the interaction between Tunisian women and the ICT sector and allowed the government to share statistics with international organizations. The study found that women's access to ICT services was not significantly lower than men's and that socio-economic factors were a more important determinant than gender. A strategic long-term plan for developing the ICT sector, including a strategy for promotion of universal access which was based on 6 strategic orientations was presented to the government shortly before project closing. 2) Developing e -security mechanisms : The construction and operationalization of a national back-up center was also achieved. It offered the Government its first cyber security infrastructure and a large backup capacity (4 terabytes). A Computer Emergency Response Team (CERT) has been created with 16 specially trained team members. A total of 178 auditors were certified by the National Agency for Computer Security (Agence Nationale de Sécurité Informatique) by the end of 2010, below the target of 220. 3) Developing e -government applications : Two e-government portals were established, one for the Justice Ministry and the other for the Culture Ministry. Of these, the e-justice portal was operational by project closure (although both were intended to be). The e-disability activity which provides digital courses to disabled students (particularly children with mental or hearing disability) was fully operational by December, 2010. The number of disabled students benefiting from access to specific education services through the project increased from 600 in 2006 to 3,201 in 2009, and 8,370 in 2010. The e-accessibility activity assured the compatibility of all major web sites of the Government with the international standards to enable disabled citizens to access e -government services. Assistance to Tunisian Internet Agency in establishing a solution in creating domain names in Arabic characters. As of January 2011, 28 domains names in Arabic has been registered. Intermediate Outcome : Promoting the participation of the private sector in the ICT sector : Increased competition was fuelled by private sector investment, which rose from 35.2% of sector investment in 2003 to 72% in 2010 (against a target of 65%). Increased competition translated into enhanced consumer access. Fixed and mobile penetration reached 118% in 2010, up from 32% in 2003, and well in excess of the original target of 60% (which had already been surpassed in 2008. The result was also larger than the revised target for 2010 of 109%. Outcomes: Litigators and lawyers have the opportunity to monitor reported cases and obtain information on matters affecting them in the last 5 years. A judicial map providing on-line information about the courts in different regions of the country is available to litigators and lawyers. The exchange of views and information has been improved within public administration at the central, regional and local levels, thereby strengthening the capacity of public servants. The Tunisian CERT was the first African team to become a member of the Forum for Incident Response Team
(FIRST) initiative. The ICR provides no indicators of the impact of the activities and outputs on the promotion of the ICT sector or on sector policy or institutional capacity. The indicator chosen for measuring the development of the ICT sector was its share in GDP. This rose from 5.9% in 2004 to 10% in 2008, exceeding the target of 8.5%. Forecasts indicate a further rise to 12% in 2010 (ICR, page 8). The methodology for computing the indicator was changed during the last year of the project by Tunisia s Institut d'économie Quantitative to come closer to the International Standards Industrial Classification (ISIC). Using the new methodology, ICT s share in GDP rose from 4.3% in 2005 (data for 2004 are unavailable) to 6.3% in 2009, and a forecast 6.8% in 2010. Under both methodologies, the ICT sector s contribution to GDP rose by more than 50% over the life of the project. However, the extent to which this result can be attributed to this relatively small-scale technical assistance operation is limited. A variety of other factors, such as the broader sector legal, regulatory and policy frameworks, as well as national income growth, are key determinants of the share of ICT-related activities in GDP. However, it is reasonable to assume that the measures supported by the project made a contribution to the result, even if its magnitude cannot be determined. 5. Efficiency: The ICR does not estimate the project s economic rate of return on the grounds that it would be difficult to separate the benefits from those of the development of the ICT sector in general. Project costs were lower than anticipated at appraisal (for instance, the ICR reports that E-Disabled equipment was purchased for about $ 0.5 million, instead of the original budget of $1.3 million). This, together with the substitution of three activities by lower cost alternatives, led to the cancelation of about 20 percent of the loan amount The lower than anticipated cost was, however, partly offset by delays in local procurement processes, lengthy validation of the deliverables, and the two years taken in processing the project restructuring. The closing date had to be extended by 18 months. In the case of the E-Justice and E-Culture activities, the Bank underestimated the time needed between design and implementation, resulting in delays. The project involved ten implementing agencies whose lack of capacity combined with coordination difficulties among them also resulted in slow implementation. Efficiency is rated modest. a. If available, enter the Economic Rate of Return (ERR ERR)/Financial Rate of Return (FRR FRR) at appraisal and the re-estimated estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal ICR estimate No No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project's activities contributed to the promotion of Tunisia's telecommunications sector, though the magnitude of the contribution in relation to other causal factors cannot be gauged. The efficacy of three out of the four specific objectives is assessed as substantial, including increased private sector involvement through supporting the issuance of new licenses, thereby leading to greater competition and enhanced consumer access. Relevance of both objectives and design is rated substantial, though efficiency was modest. Overall outcome is assessed as moderately satisfactory. a. Outcome Rating : Moderately 7. Rationale for Risk to Development Outcome Rating: The new legal framework and increased private sector participation have taken telecommunications in Tunisia to a new level of competitiveness and connectivity that is unlikely to be reversed even if the risk of policy reversals exists.there is also a the growing pervasiveness of ICT use in both public and private sectors, and the importance of the sector as a source of both fiscal revenue and foreign direct investment is increasing. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry:
The background analysis completed for the project (see Annex 1 of the PAD: "Country and Sector or Program Background") was partly based on the 2002 ICT Policy Note which was prepared by the Bank in close cooperation with the government. The project preparation team was the same as that which prepared the Policy Note, and therefore benefited from in -depth knowledge of the sector (ICR pg 14). The PAD correctly identified and rated risks, one of which (too many participating agencies) materialized. The project was innovative, being one of the first Bank operations supporting e-applications to government work and improved ICT access for the disabled. However, insufficient allowance was made for the lack of ICT knowledge (especially in rolling out new applications), project management experience, and procurement capacity in some of the implementing agencies. Although measures were taken to address these limitations -- such as additional staffing with expertise in procurement, ICT and project management, and provision of specialized training in procurement -- they still slowed implementation down. Quality-at at-entry Rating : Moderately b. Quality of supervision: According to the ICR (pg 15), the Bank team provided quality policy advice, shared knowledge and best practices, and communicated its concerns on delays in the liberalization process to the Government. Project implementation was supported by an average of between one and two supervision missions per year comprising sector specialists and fiduciary experts (ICR pg 16). There was continuity of staff in that the Task Team Leader (TTL) did not change during the life of the project. In the final years of the project, the TTL was based in Tunisia and held weekly meetings with the Government which helped accelerate the implementation of the e -government applications (Implementation Status Reports 10 & 11). However, the ratings on Implementation Progress in the Implementation Status Reports were not fully reflective of reality, since they remained throughout the life of the project despite considerable delays. Quality of Supervision Rating : Overall Bank Performance Rating : Moderately 9. Assessment of Borrower Performance: a. Government Performance: The Government's commitment and ownership towards the achievement of project objectives was strong throughout the life of the project. However, there were some issues that caused delays during implementation : Frequent changes in Ministers (3 during the life of the project) meant that the new Ministers had to be briefed and become familiar with their new portfolio. Due to changes in government personnel, the Tunisian team involved in project preparation was not the same as the one implementing it, resulting in a slow start (ICR pg 16). There was a two year delay in approving a new sector law, which the government had announced at the 2005 WSIS (ICR pg 17). Insufficient support and empowerment provided to INT in its mission to enforce the new telecommunication law, impeded wider liberalization (ICR pg 17). Government Performance Rating Moderately b. Implementing Agency Performance: The Project Management Unit (PMU) consisted of a strong team that provided high quality interface with the Bank and provided intense support to the implementing agencies throughout the project (ICR pg 7). However, the project involved a total of 10 implementing agencies and delays in implementation were mainly due to insufficient capacity in several of them. Although the engineers in the implementing agencies had good technical skills, there were delays while rolling out new applications because the administrative staff had insufficient skills in managing the technological changes in the new e-government applications. During the last 2 years of implementation, the PMU successfully kept the under-performing implementation agencies under pressure so that the last activities could be completed by project closure (ICR pg 17).
Implementing Agency Performance Rating : Moderately Overall Borrower Performance Rating : Moderately 10. M&E Design, Implementation, & Utilization: a. M&E Design: The results framework (Annex 3) in the PAD established a clear linkage between activities, components, intermediate indicators and outcomes. Baseline and Target values were defined for most of the indicators with three exceptions (number of disabled students having benefited from access to specific education services through the project, number of Portal access put in place and number of downloads per year ). However, the overall outcome indicator (increased share of ICT sector in the GDP) was not suitable for measuring the contributions made by this project because of attribution difficulties. b. M&E Implementation: Output, intermediate outcome, and outcome indicators were regularly collected and updated; their progression is presented in detail in Annex 6 of the ICR. The indicators relating to the number of requests submitted in the e -Justice and e-culture portals were not updated owing to the fact that the e -Justice portal only became operational during the last year of implementation, and the e-culture portal, while technically ready, was not launched before project closure. c. M&E Utilization: The ICR (page 9) reports that utilization of the performance indicators : (i) raised the profile of the sector and increased its visibility throughout the government; (ii) helped in revising the methodology for computing the share of the ICT sector in GDP; and (iii) illustrated the pervasiveness of ICT in the Tunisian economy. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: No safeguard policies were triggered by this Category C project. b. Fiduciary Compliance: There is relatively little discussion in the ICR on financial management and audits. However, according to subsequent information provided by the project team, the Borrower maintained a satisfactory financial management system and provided timely and reliable financial information required for managing and monitoring project implementation.the Borrower submitted semi-annual financial monitoring reports and annual audit reports in a timely manner and in a format and content acceptable to the Bank. The audit opinions were unqualified. According to the ICR (pg 8), the procurement plan became more complex as the project progressed -- in 2005, there were a total of 29 technical assistance activities identified in the procurement plan, whereas in 2010, a total of 90 contracts were signed. The delays in local procurement processes and lengthy validation of the deliverables resulted in the extension of the closing date by 18 months. c. Unintended Impacts (positive or negative): None reported. d. Other: None 12. Ratings: Outcome: ICR IEG Review Moderately Reason for Disagreement /Comments Efficiency is rated modest, indicating moderate shortcomings..
Risk to Development Outcome: Negligible to Low Negligible to Low Bank Performance : Moderately Borrower Performance : Moderately Quality of ICR : NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. Moderately Moderately. 13. Lessons: The following lessons are taken from the ICR with some adaptation : A government that is embracing the concept of the information society may find it difficult to take the policy decisions necessary to make the concept a reality, that is, ICT sector liberalization, private sector participation, level playing field competition, freedom of expression, and unrestricted access to information and the Internet. ICT for disabled children and adults is an important issue deserving more attention. The key elements that led to the success of this initiative in this case were the existence of a policy of targeting these groups, the availability of excellent statistics on the target groups and leveraging of the government effort through NGOs. Successful implementation of e-government applications requires availability of technical capabilities in the implementing agencies, available competencies in the local ICT sector, and sustained and motivated leadership. At the time of appraisal, the time line from design to roll-out of E-applications should be realistic and take full account of local capacity limitations. Given that ICT is a rapidly evolving sector, annual re-evaluation of the main project components would help to plan changing funding requirements in advance. The number of implementing agencies should be limited, and their readiness for implementation fully assessed. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clearly written and contains most of the elements needed to evaluate the project. However, there are some shortcomings. There were discrepancies in the reporting of project costs. The information under Annex1 (pg 20) shows incorrect actual cost of $8.1 million in Table 1 and shows actual project cost of $ 0 in Table 2. On page 23 of the ICR, it is mentioned that "Annex 9b provides a detailed account of the achievement under this activity " and "the national backup center was eventually delivered and operational in the last month of the project implementation (Annex 9c);" however, there is no Annex 9b or 9c in the ICR. The ICR provides little information or analysis on which to judge efficiency and the treatment of fiduciary compliance is insufficient. a.quality of ICR Rating :