Allocation and management of risk in Ministry of Defence PFI projects

Transcription

1 Allocatio ad maagemet of risk i Miistry of Defece PFI projects REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 343 Sessio October 2008

2 The Natioal Audit Office scrutiises public spedig o behalf of Parliamet. The Comptroller ad Auditor Geeral, Tim Burr, is a Officer of the House of Commos. He is the head of the Natioal Audit Office which employs some 850 staff. He ad the Natioal Audit Office are totally idepedet of Govermet. He certifies the accouts of all Govermet departmets ad a wide rage of other public sector bodies; ad he has statutory authority to report to Parliamet o the ecoomy, efficiecy ad effectiveess with which departmets ad other bodies have used their resources. Our work saves the taxpayer millios of pouds every year: at least 9 for every 1 spet ruig the Office.

3 Allocatio ad maagemet of risk i Miistry of Defece PFI projects LONDON: The Statioery Office Ordered by the House of Commos to be prited o 28 October 2008 REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 343 Sessio October 2008

4 This report has bee prepared uder Sectio 6 of the Natioal Audit Act 1983 for presetatio to the House of Commos i accordace with Sectio 9 of the Act. Tim Burr Comptroller ad Auditor Geeral Natioal Audit Office 24 October 2008 The Natioal Audit Office study team cosisted of: Mark Wyiatt, Ady Nichols ad Doug Neal, assisted by Gareth Hartley, David Jackso, Shiree Khattak, Lydia Kidma, Tom McBride ad Rya Wakefield uder the directio of David Filay This report ca be foud o the Natioal Audit Office web site at For further iformatio about the Natioal Audit Office please cotact: Natioal Audit Office Press Office Buckigham Palace Road Victoria Lodo SW1W 9SP Tel: equiries@ao.gsi.gov.uk Natioal Audit Office 2008

5 cotets summary 4 Part ONE Deliverig Miistry of Defece services 12 through PFI cotracts Part TWO Maagig risks 22 Appedices 1 The case studies 49 2 Methodology 68 3 Timelie of Miistry of Defece 70 PFI Projects 4 Allocatio of risks betwee the 71 cotractual parties i PFI cotracts GLOSSARY 74 Photographs o pages 56; 58; 60 ad 62 courtesy of Crow Copyright/MOD, image from Reproduced with the permissio of the Cotroller of Her Majesty s Statioery Office

6 SuMMARy 1 The Miistry of Defece (the Departmet) udertakes a wide rage of activities, icludig frot lie military capability ad essetial support fuctios. To deliver these activities efficietly, the Departmet uses various forms of relatioship with the private sector, with differig approaches to sharig risk. The Private Fiace Iitiative (PFI) represets oe importat form of relatioship. I PFI projects the Departmet usually defies the outputs it requires ad ivites the private sector to desig ad deliver the required service outputs for a agreed aual price. 2 Risk maagemet is importat to all forms of procuremet udertake by the Departmet. Effective risk allocatio ad maagemet is particularly importat to deliverig value for moey i PFI cotracts. These cotracts seek to optimise value for moey by agreeig a allocatio of risks betwee the public ad private sectors over the course of a log-term cotract. This is ot the sole cotributor to value for moey but it is very ulikely that the best possible PFI outcome will be achieved without effective risk allocatio ad maagemet. 4 ALLOcATION AND MANAGEMENT OF RISk IN MINISTRy OF DEFENcE PFI PROJEcTS

7 summary 3 This study examies whether there has bee effective allocatio ad maagemet of risk i the Departmet s PFI projects. Our fidigs are based o a detailed examiatio of eight PFI case study projects. This aalysis is supported by a cesus of all the Departmet s PFI cotracts let i 2007 ad cosultatio with the Departmet s staff, cotractors ad advisers. We selected the case studies to reflect the diverse ature of the Departmet s PFI portfolio (Figure 1). Appedix 1 cotais detailed fidigs from each case study. Our examiatio has also draw o our previous experiece of examiig risk maagemet i PFI projects (see Methodology, Appedix 2). 1 The eight case study projects The case study projects, other tha the cacelled Armoured Vehicle Traiig Service PFI project, were selected to preset a rage of the Departmet s PFI projects i full service i Project Year cotract let Year full service commeced Capital Value millios Type Descriptio Heavy Equipmet Trasporter (HET) Equipmet Service to move battle taks ad other heavy equipmet durig peacetime ad o operatios. Field Electrical Power Supplies (FEPS) Equipmet Provisio of 1347 geerator sets to support operatioal electrical requiremets i the field. Medium Support Helicopter Aircrew Traiig Facility (MSHATF) Traiig Helicopter traiig facilities for Chiook Mk 2/2A, Merli Mk III ad Puma Mk 1 aircraft. Traiig icludes iitial coversio to type, cotiuatio, pre deploymet ad missio rehearsal. Armoured Vehicle Traiig Service (AVTS) Cotract ot let as procuremet was ot completed Services to be provided uder other cotractig arragemets N/A Traiig Provisio of live ad sythetic traiig of guery ad specialist drivig for Armoured Fightig Vehicles. To provide crew traiig for a wide rage of Armoured Fightig Vehicles, icludig Challeger II taks, Warrior Ifatry Fightig Vehicles ad Combat Recoaissace vehicles. Mai Buildig Refurbishmet (MBR) Accommodatio Developmet ad refurbishmet of the mai Miistry of Defece buildig i Whitehall ad temporary accommodatio to other buildigs, as well as upkeep of the Old War Office buildig. Defece Aimal Cetre (DAC) Accommodatio Redevelopmet of ew office ad residetial accommodatio, aimal husbadry ad traiig support. Defece Fixed Telecommuicatios System (DFTS) Other Support Provisio of secure ad survivable Wide Area Network voice, data ad video telecommuicatio services across the Miistry of Defece to users (Miistry of Defece staff ad i some cases its cotractors) based i the Uited Kigdom ad Overseas. Tidworth Water ad Sewerage Nil Other Support Provisio of fresh water ad sewerage services i Tidworth Garriso. Total capital value 902 Source: Natioal Audit Office review of case studies Allocatio ad maagemet of risk i Miistry of Defece PFI projects 5

8 summary 4 Our aalysis of these projects focussed o te key risks. I doig so we recogised two importat features of the Departmet s portfolio of PFI cotracts. Firstly, the Departmet s approach to procurig PFI deals has developed over time. Our case study projects were relatively early cotracts etered ito betwee 1997 ad These were chose to provide a good level of operatioal experiece to examie. Sice these cotracts were let the Departmet has achieved a greater stadardisatio of cotractual terms ad coditios. It has also improved its iteral scrutiy of PFI deals ad all large projects i the light of previous experieces. Secodly, we recogise that the Departmet has faced particular challeges arisig from the fact that its PFI portfolio is less homogeous tha those of other departmets. 5 The Departmet s portfolio of more tha 50 PFI projects represets a small, but importat, part of the Departmet s aual spedig. The total private sector capital ivestmet i Defece is over 9 billio. I the Departmet spet 1.3 billio i PFI service charges. The projects rage from small projects with a capital value of uder 50 millio to larger projects with a capital value of over 1 billio. The larger projects iclude the service accommodatio cotract Alleby ad Coaught let i March 2006 (capital value 1.3 billio, total cotract value 8 billio) ad the Future Strategic Taker Aircraft, the Departmet s largest PFI deal completed i March 2008 (capital value 2.6 billio, total cotract value 13 billio). The capital value of the case study projects we examied, focussig o projects i full service i 2007, was just over 900 millio (Figure 1, page 5) Fidigs 6 The Departmet has achieved a good service delivery o a broad ad diverse portfolio of PFI projects. Across its whole PFI portfolio of more tha 50 projects most have reached full service delivery o time, for the cost set out i the cotract ad are deliverig services satisfactorily. These ew projects have eabled the Departmet to achieve cosiderable beefits from a rage of services. Some of the projects are deliverig ew equipmet ad traiig which are cotributig to improvig the effectiveess of military persoel. Others are providig support services which are helpig the Departmet to carry out its work more efficietly. 7 I the case study projects we examied most of the risks were beig well maaged by the Departmet with the projects deliverig value for moey but there were exceptios. We cocluded that i ie out of the te risk categories we examied there was either a low risk to value for moey or moderate risk to value for moey (Figure 2). I oe risk category, the specificatio of the asset or service, we cocluded there was sigificat risk to value for moey to the procuremet phase although ot to the subsequet maagemet of the projects i their operatioal phase. The specificatio issues had cotributed to problems o two procuremets: the Armoured Vehicle Traiig Service project which was cacelled durig its procuremet ad the Defece Aimal Cetre where the cotract will eed to be reegotiated. I the other six case study projects we examied the risks had geerally bee well maaged cotributig to value for moey. The Defece Aimal Cetre has a capital value of 11 millio ad is therefore, uder curret Treasury policy, below the threshold of projects which would ow eed to be procured uder PFI. Sice these deals the Departmet has take a umber of steps to address the risk of iadequate specificatio of assets or services (Figure 3). 6 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

9 summary 2 Summary of the NAO risk assessmet NAO risk assessmet Low risk to value for moey Moderate risk to value for moey Sigificat risk to value for moey Source: Natioal Audit Office 3 Nature of risks Costructio or service implemetatio; delivery of the ogoig service; eviromet ad safety; service user ad stakeholder satisfactio; relatioships with cotractors. The delivery of a PFI solutio durig the procuremet phase; the specificatio affectig the maagemet of the cotracts; techology ad latet defects; performace moitorig ad maagemet; resources ad skills; risk maagemet processes. Specificatio of the asset or service affectig the procuremet phase. Examples of chages itroduced by the Departmet to address the risk of iadequate specificatio of assets or service 1 The Departmet s strategic plaig assumptios were chaged after deploymets to Afghaista ad Iraq to iclude wider operatig parameters tha existed previously (particularly relevat for the Heavy Equipmet Trasporter project); 2 There have bee several improvemets to systems ad guidace built aroud a idetified idividual i the project teams actig as a Requiremets Maager ad a cetral specialist fuctio with resposibility for specifyig ad deliverig equipmet capability; ad 3 The Departmet has rolled out a web based Acquisitio Operatig Framework available o lie, icludig guidace o producig a statemet of User Requiremet Priciples ad a aide-memoire o how to defie systems requiremets ad produce a Statemet of Need. 8 The Departmet has developed commercial disciplies for scrutiisig the value for moey of its PFI procuremets ad has exteded these ito other projects. The Departmet has over time developed appropriate processes for scrutiisig PFI procuremets. The Departmet has a experieced Private Fiace Uit which provides support ad guidace to PFI projects. Large PFI projects also eed to satisfy the Departmet s Ivestmet Approvals Board. Recetly the Departmet has used its PFI experiece to establish a further review process for its large projects uder all forms of procuremet. The ew review process draws o the checkig, kow as due diligece, which the private sector fuders carry out o PFI deals. It aims to improve assurace that the proposed commercial arragemets will eable the project to meet operatioal requiremets with appropriate cotract terms. 9 The Departmet is usig these disciplies to take tough decisios o some PFI projects although these decisios could have bee made o a more timely basis. Although most of the Departmet s PFI projects have delivered the services required the Departmet is prepared to take decisios to abado iappropriate PFI project proposals or to reegotiate or termiate PFI cotracts which are ot deliverig the required services. These decisios are take i the iterests of achievig value for moey, but i some cases the Departmet has take a log time to decide o the appropriate actio. For example, the Departmet took the right decisio ot to proceed with a proposed PFI procuremet for Armoured Vehicle Traiig but oly after spedig six years i developig the project. The Departmet is rightly seekig to reegotiate its Defece Aimal Cetre cotract but has also take six years to reach this positio despite ogoig dissatisfactio with the service delivery. Source: Miistry of Defece Allocatio ad maagemet of risk i Miistry of Defece PFI projects 7

10 summary 10 The Departmet took o average 37 moths to procure the projects we surveyed, where data was available, but large projects ofte took loger. The Departmet s methods for overseeig PFI procuremets aim to avoid iappropriate deals beig completed. It is right that time is spet o udertakig such assessmet. But it is also importat that procuremets are efficiet so that service delivery is ot uduly delayed ad bid costs, which are likely to be factored ito cotract prices i the log term, are kept withi reasoable limits. The average procuremet time i the Departmet s PFI projects we surveyed, i the 77 per cet of projects where the Departmet held such data, was 37 moths. These statistics compare with the average PFI experiece across govermet of 34 moths i a 2006 NAO survey (Improvig the PFI Tederig Process HC ). Larger projects ofte took loger to procure, the average for the Departmet s PFI projects with a capital value of over 50 millio, where data was available, beig 45 moths. This loger time reflects: the special requiremets of the Departmet s projects compared with repeat projects such as hospitals or schools; the rage of the Departmet s i-house stakeholders who are ivolved i decisios about the projects ad the assessmet of the deals; ad some scope, despite these special cosideratios, for the Departmet to improve the speed at which it closes larger deals, which it is seekig to do through improvemets to the oversight of its capital procuremets. 11 The Departmet s efforts to allocate ad maage risk at the outset of the projects that we examied were ofte hampered by a lack of data o the services required. Lack of data represeted a sigificat risk to value for moey i the eight case study projects we examied. Isufficiet service iformatio creates a risk because the Departmet may ot get the service which matches its eeds or the cotractors may icrease the pricig of deals because of the resultig ucertaity. Procuremet times have also bee affected by the eed to clarify specificatio issues. I particular lack of data o the Departmet s service requiremets was a factor i the cacelled Armoured Vehicles Traiig PFI project. Some issues, which could have bee idetified earlier, oly emerged after cotract lettig. For example, i the Field Electrical Power Geerators project the Departmet had to pay the cotractor 7.3 millio without competitio to modify the geerators whe the Departmet foud that some of its vehicles, whe pullig the geerators, had maoeuvrability problems ad could ot tur corers safely. 12 The Departmet s PFI cotracts have flexibility to deal with chages but there are risks to maitaiig value for moey where chages are required. The Departmet works i a fast chagig, ofte upredictable, eviromet. It ofte fids that chages are eeded to its projects either i the ru up to cotract lettig or oce the project is i service because: the operatioal eeds of its military staff may chage, particularly if they eed to be deployed i ew territories; ew techology used i equipmet or traiig may be developed; or i some cases, chages are eeded because the Departmet s eeds are iitially difficult to defie precisely across its large complex orgaisatio. Ay chage after appoitig a cotractor creates a risk to value for moey as it may ivolve added costs i a situatio where competitio is abset. The NAO report Makig Operatioal Chages i PFI Projects (HC 205, ) aalysed the risks arisig from cotract chages, ad described how Departmets ad agecies ca best maage these risks. The capacity for the Departmet s eeds to chage, sometimes at short otice, does ot mea that the PFI is a iappropriate form of procuremet: well-desiged PFI cotracts are flexible ad ca adapt the services provided as circumstaces chage. It does mea that chages, ad the pricig of them, have to be carefully egotiated betwee the Departmet ad their cotractors. 8 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

11 summary 13 There are istaces where cotract maagemet could be improved, especially i assessig performace. The Departmet s PFI projects i service are maaged by teams who are geerally addressig cotract maagemet issues i a appropriate maer ad buildig effective workig relatioships with their cotractors. The Departmet s Private Fiace Uit has also already made a useful step i evaluatig the Departmet s overall experiece of usig PFI through its 2005 review of the operatioal experiece of its PFI projects. Our examiatio has, however, idetified the followig areas for improvig project maagemet: User satisfactio feedback had ot bee obtaied i 25 per cet of the projects i our cesus. Risk maagemet processes such as risk registers were ot used cosistetly. I some projects post cotract evaluatios, to cosider the costs ad efficiecy of the procuremets, were either ot carried out or were ot doe o a timely basis. I the Defece Fixed Telecommuicatios project, the Departmet had to recover 1.3 millio from the supplier BT after it became apparet that employees of BT had iflated the umber of calls beig aswered withi the required time limit by callig each other. This fraud was ot detected iitially because there was o adverse cosequece for the Departmet s staff usig the telecommuicatios services. The Departmet ad BT have take steps to address the circumstaces which led to this fraud. The Departmet ad BT have ow imposed a ew maagemet structure ad goverace arragemets o the cotract. There is more detailed performace reportig by BT which is subject to audit by the Departmet. BT has also made chages to the staffig of the project. Value for moey coclusio 15 I six of the eight case studies we examied the Departmet has procured ad maaged successful PFI solutios. Oe project was cacelled before cotract lettig ad oe other cotract we examied will eed to be reegotiated. I the PFI projects we examied the Departmet has therefore geerally achieved effective allocatio ad maagemet of risk. The allocatio ad maagemet of risk, which has bee the focus of this examiatio, is however ot the sole cotributor to value for moey. Effective competitios, or strog bechmarkig processes where there is a sigle supply source, are also eeded to obtai the best prices for the risks trasferred. The case study cotracts we examied were all procured through competitive tederig. 14 Appropriate skills are required for maagig PFI cotracts so that value for moey is ot eroded durig the cotract s life. There is a particular eed for the kowledge of the cotract ad the aims of the project to be trasferred to those who will maage the cotract. There was a lack of staff cotiuity o some of the case study projects we examied, but other case study projects had beefited from retaiig at least oe member of the cotract egotiatio team i post for the first year or two of the operatioal phase. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 9

12 summary Recommedatios These recommedatios relate to the Departmet s portfolio of existig PFI projects ad ay further PFI procuremets they may udertake. The recommedatios may also have relevace to other forms of public/private parterships which the Departmet may develop i the future. i ii A lack of robust data for project teams to specify their requiremets ad the risks beig trasferred to cotractors has bee a issue for a umber of the case studies examied. Project teams should esure that the iitial plaig stage of each project icludes the productio of suitable data o ay existig use of the required service, forecast usage ad the coditio of assets beig trasferred to the private sector. The Departmet s Private Fiace Uit should check that this iformatio is available before biddig competitios commece. PFI cotracts are log term cotracts which have the potetial flexibility to deal with the Departmet s chagig requiremets. But chage requires egotiatios betwee the parties which eed to be maaged effectively to esure value for moey is ot eroded durig the cotract period. The Departmet s PFI project teams should assess the likely impact of future chagig circumstaces o the PFI cotracts they propose to eter ito. I particular, they should: be fully coversat with the prescribed processes for dealig with chage, icludig Treasury ad Departmetal guidace; oly eter ito cotracts that set out clearly how chages will be made to the project icludig processes for bechmarkig the price of variatios; cotiue, durig the service period, to udertake regular assessmets (at least yearly) of factors that could affect future requiremets ad the impact these may have; actively maage supplier relatioships with the private sector to successfully icorporate chagig requiremets o terms which are value for moey; ad whe cosiderig a possible chage i the use of a asset delivered by the private sector, remember that the Departmet may become liable for repairs arisig from the chage. iii iv Some of the Departmet s projects with early PFI cotracts may experiece difficulties i eforcig the performace they require because of a lack of clarity i the draftig of the cotract. Where the Departmet s projects experiece service problems because the cotract does ot set out their requiremets clearly they should either reegotiate the cotract or seek, through their relatioship with the cotractor, a mutually agreed workig arragemet to overcome the cotract deficiecies. Termiatig a o-performig cotract represets a extreme optio. It imposes sigificat trasactio costs o the Departmet, but these costs may be lower tha the ogoig costs of poor performace. The Departmet should ot rule out termiatio of o-performig cotracts o the grouds of trasactio costs aloe. Although it is ormal practice i PFI projects for the private sector to record performace ad the public sector to moitor performace levels there is a risk that cotractors might misrecord performace to avoid paymet deductios. The Departmet s project teams should assess the appropriateess of the systems used to validate cotractors service performace. I particular they should be alert to the risk that the extet of successful service delivery could be overstated without ay adverse effect o users of the service that would draw their attetio to the situatio. Project teams should carry out audit work o the performace moitorig systems, cosider whether the performace data beig provided is adequate ad carry out spot checks of the autheticity of the uderlyig data. 10 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

13 summary v vi Good cotract maagemet ivolves retaiig kowledge about the project, moitorig risks ad carryig out regular evaluatios. There was a lack of staff cotiuity o some of the case study projects we examied, which meat that it was harder to achieve a high stadard of cotract maagemet. To improve the maagemet of PFI projects the Departmet s PFI project teams should: keep at least oe seior member of the team i post for the first year after the cotract has bee let, so there is a suitable trasfer of kowledge to the team who will maage the cotract; have staff with appropriate cotract maagemet skills acquired through either previous experiece or appropriate traiig; capture project risks o formal risk registers i both the procuremet ad i-service phases; udertake user satisfactio assessmets o a systematic basis; ad carry out post cotract evaluatios ad subsequet aual reviews of overall cotract performace. The NAO framework for evaluatig PFI projects may assist this process 1. The Departmet s Private Fiace Uit has carried out valuable work i reviewig the procuremet ad service experiece of its PFI projects. The Departmet s Private Fiace Uit should exted its review of PFI projects by: idetifyig ad dissemiatig lessos from its project teams post cotract evaluatios ad subsequet aual reviews; aalysig iformatio o the iteral ad exteral costs of procurig recet ad curret PFI deals to idetify actio poits for improvig the efficiecy of the procuremet of future deals; ad recordig ad moitorig the mai risks affectig the successful delivery of services across the Departmet s PFI portfolio takig accout of the project evaluatios ad the issues idetified i this report. 1 Available from the NAO website Allocatio ad maagemet of risk i Miistry of Defece PFI projects 11

14 Part Oe Deliverig Miistry of Defece services through PFI cotracts This part of the report describes the Departmet s portfolio of PFI projects ad the oversight which it provides to the procuremet ad maagemet of these projects. The Departmet s portfolio of PFI projects 1.1 I relatio to the total defece budget, PFI represets a small, but importat, part of the Departmet s aual spedig. I the Departmet paid 1.3 billio i PFI service charges, approximately four per cet of its total outtur of aroud 36 billio The Departmet is the fourth largest user of PFI i cetral govermet i terms of the umber of cotracts siged. But these are ofte large value cotracts ad the Departmet s aual paymets to PFI cotractors of over 1 billio are projected to be greater tha those of ay other cetral govermet departmet over the ext three years (Figure 4). 4 Projected aual PFI cotract paymets for the four major Cetral Govermet users of PFI millios per aum 2,000 1,500 1, Fiacial Year Departmet for Childre, Schools ad Families Departmet for Trasport Health Miistry of Defece Source: Treasury Iformatio take from the list of siged PFI deals take from the Treasury website 2 Source: Miistry of Defece Resource accouts ( ) This figure icludes amouts paid o a small umber of projects that the Departmet s Private Fiace Uit cosiders to be Public Private Partership (PPP) deals. 12 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

15 part oe The services procured by the Departmet through PFI 1.3 The Departmet s PFI portfolio of operatioal projects i 2007, which was the subject of our cesus, comprised 47 cotracts 3 (Figure 5) coverig four broad categories: accommodatio, traiig, equipmet ad other support projects. Other support projects icluded the provisio of utilities ad iformatio ad commuicatio techology (ICT). Key facts from the 47 projects are at Figure 6 o pages 14 to The Departmet s PFI projects are critical to maitaiig operatioal capability. Several projects directly support frot lie military operatios, such as the Strategic Sea Lift Roll-o Roll-off Ferries ad the Heavy Equipmet Trasporter projects. Other projects provide idirect support to frot lie operatios, for example by providig traiig for frot lie troops. These iclude simulator traiig projects such as those for the Lyx Aircrew Traiig Service, ad the Medium Support Helicopter Aircrew Traiig Facility. 1.5 The Departmet has also used PFI to provide other services such as accommodatio (for example, office accommodatio ad barracks) ad ifrastructure (for example, water ad sewerage services to defece sites, telecommuicatios ad iformatio techology). The size ad value of the Departmet s PFI projects 1.6 I our cesus the capital values of the Departmet s PFI projects (the cost of buildig the asset as opposed to deliverig the service) varied from zero (where the PFI cotractor is takig over the operatio of a existig asset) to 1.3 billio (Figure 7 o page 20). Over half of the 47 projects had a capital value of 50 millio or less but the remaiig larger cotracts cotributed to aroud 83 per cet of the Departmet s PFI paymets of 1.1 billio i Balace Sheet Treatmet of the Departmet s PFI Projects per cet of the MOD PFI projects by umber (62 per cet by capital value) have bee accouted for off the Departmet s balace sheet. Across cetral govermet as a whole, we have cosidered that most PFI projects, predomiatly govermet buildigs, should be o balace sheet because departmets have the risks ad rewards of owership. The MOD s PFI portfolio, however, icludes a rage of other projects icludig a umber of traiig ad iformatio techology (IT) projects. Most of these projects have bee accouted for by the Departmet off balace sheet because the risks ad rewards of owership lie with the private sector cotractors. For example, i traiig projects the cotractors remueratio ofte varies i relatio to the volume of staff usig the traiig or the success of the traiig. 1.8 Cetral govermet will implemet Iteratioal Fiacial Reportig Stadards (IFRS) from 1 April It is likely that whe these ew accoutig rules are itroduced, each project will udergo a fresh detailed assessmet of its accoutig treatmet. As the assessmet criteria will be differet from those used whe the projects were first assessed, it is possible that some cotracts curretly categorised as off balace sheet may come o to the Departmet s balace sheet. The Departmet as a mature user of PFI 1.9 The first PFI cotract siged by the Departmet covered the provisio of o-combat vehicles (kow as the white fleet ) i The Departmet has sice siged a steady stream of deals. As well as the 47 projects i our cesus of the Departmet s PFI operatioal projects i 2007 (Figure 8 o page 20) the Departmet has ow siged further projects. These iclude the Future Provisio of Marie Services ad the 13 billio cotract for Future Strategic Taker Aircraft. It is also curretly procurig further PFI projects such as the helicopter Search ad Rescue project (SAR) ad tri-service Defece Traiig Ratioalisatio project. A timelie for the Departmet s PFI projects is at Appedix 3. 5 Other 12 projects Traiig 11 projects The four categories of the Departmet s PFI cotracts Source: Natioal Audit Office aalysis of Miistry of Defece projects database Accommodatio 13 projects Equipmet 11 projects 3 The list of 47 PFI projects was agreed with the Departmet s Private Fiace Uit. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 13

16 part oe 6 The Miistry of Defece PFI portfolio Project descriptio OJEU/ C cotracts Bulleti Date Accommodatio 1 Alleby/Coaught Redevelopmet of Aldershot Garriso, ad the garrisos i the. Feb 02 Salisbury Plai Area. Costructio of ew barracks will give military staff brad ew/refurbished livig ad workig accommodatio. 2 Bristol Bath Ad Portsmouth Serviced accommodatio for 317 service families. Jul 99 Family Quarters 3 Cetral Scotlad Family Quarters Services accommodatio for 164 houses. Ja 97 4 Colchester Garriso Redevelopmet ad refurbishmet to provide accommodatio ad Feb 97 associated costs. 5 Defece Aimal Cetre Redevelopmet of ew office ad residetial accommodatio, aimal Nov 96 husbadry ad traiig support. 6 Devoport Support Services Armada Provisio of Support Services ad Fleet Accommodatio Cetre Sep 00 services at Devoport Naval Base (HMS Drake) for 25 years. 7 Mai Buildig Redevelopmet (MBR) Developmet ad refurbishmet of the mai MOD buildig, ad Dec 96 temporary accommodatio to other buildigs, as well as upkeep of Old War Office Buildig. 8 Northwood Headquarters Project Itegrated provisio of support services, facilities maagemet ad Exempted capital works improvemet, to meet requiremets at the Northwood headquarters. 9 Portsmouth Housig 2 Proposal for 148 Officers houses. Mar RAF Cosford RAF Shawbury Accommodatio for 145 service families. Ju 97 Family Quarters 11 RAF Lossiemouth Family Quarters Redevelopmet ad provisio of 279 married quarters. Dec Wattisham & Woodbridge Provisio of serviced accommodatio for 250 service families. Sep 99 Married Quarters 13 Yeovilto Family Quarters Accommodatio at Yeovilto for 88 aircrew of 2 Lyx squadros moved Ja 97 uder Project Movit from HMS Osprey at Portlad. Equipmet 14 C Vehicles Procuremet, maiteace, repair ad maagemet for approx 4,000 Ju 99 items of earthmovig plat machiery ad material hadlig equipmet. 15 Commercial Satellite Replacemet of the INMARSAT termials ad airtime cotract for Oct 98 Commuicatio Service INMARSAT Royal Navy ships. 16 Field Electrical Power Supplies (FEPS) Provisio of geerator sets to support operatioal electrical requiremets Oct 96 i the field. 17 Heavy Equipmet Trasporters (HET) Service to move battle taks ad other heavy equipmet durig Feb 97 peacetime ad o operatios. Reserves makig up oe third of the mapower required to deliver the service. 18 Marie Support To Rage To brig the maagemet, operatio ad maiteace of air support N/A 3 & Aircrew Services (MSRSS) crew ad rage safety craft ito oe cotract. 19 Materiel Hadlig Equipmet Provisio of tri-service materiel Pathfider hadlig equipmet for Oct 00 (MHE) (Follow O) Army, Navy ad RAF storage depots. Oe of the MOD s Pathfider PFI projects. 14 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

17 part oe date of Iitial Full Capital Term Ed Uitary Balace Public Sector Private Sector Case fiacial service service Value (Years) Date Charge Est sheet Survey Survey Study close date date ( m) ( m) treatmet received received Mar 06 Jul 06 Ja 16 1, O 4 4 Nov 01 Jul 02 Oct Off 4 Aug 99 Ja 01 Mar Off 4 Feb 04 Ju 04 Oct Off 4 Aug 00 Ja 02 Ja O Jul 04 Oct 04 Mar O 4 4 May 00 Sep 04 Sep O Jul 06 Oct 06 Oct O 4 Oct 05 May 06 Ju Off 4 Mar 99 Dec 99 Ju Off 4 4 Ju 98 Apr 99 Sep O 4 May 01 May 02 Mar Off 4 Jul 98 Ju 99 Mar Off 4 4 Ju 05 Ju 05 May Off 4 4 Apr 01 Apr 01 Novated Off 2 May 2005 Ju 02 May 03 Apr O Dec 01 Jul 03 Jul O 4 4 Dec 01 Apr 02 Apr Off 4 4 May 02 Aug 02 Aug Off 2 Allocatio ad maagemet of risk i Miistry of Defece PFI projects 15

18 part oe 6 The Miistry of Defece PFI portfolio cotiued Project descriptio OJEU/ C cotracts Bulleti Date Equipmet cotiued 20 Roll-O/Roll-Off (RORO) Provisio for a 6 ship trasport service. Ja 97 Strategic Sealift 21 Skyet 5 Rage of satellite services, icludig maagemet of existig N/A 3 Skyet 4 satellites. 22 Tri Service Materials Provisio of equipmet, maiteace, traiig ad IS for MHE fleet, N/A 3 Hadlig Service comprisig of 2,200 commercial lift trucks ad associated hadlig equipmet located worldwide. 23 Tri Service White Fleet Stadard commercial productio vehicles used for o-operatioal Dec 98 admiistrative ad support fuctios. 24 VLF Naval Commuicatio Service Provisio of commuicatios service for submarie fleet. Exempted Traiig 25 Army Foudatio College (AFC) Provisio of facilities ad services icludig vocatioal educatio. Ju Astute Class Traiig Services (ACTS) Provisio of a 36 year service to deliver operator ad maitaier Mar 97 traiig for Astute Class submaries. 27 Attack Helicopter Traiig Service To provide traiig services to support the British Army s Apache AH1 Exempted Attack helicopter. 28 Defece Sixth Form College (DSFC) Desig ad costructio of the Sixth Form college which will help i the Oct 01 future to recruit Armed Forced ad MOD Civil Service. New livig accommodatio will cater for 340 MOD selected studets ad teachig staff. Will be a pricipal source i Techical Officer recruitmet to all UK armed forces ad MOD civil service. 29 Fire Fightig Traiig Uits (FFTU) Provisio of fire fightig traiig facilities for Naval Recruitig & N/A 3 Traiig Agecy (NRTA). 30 Hawk Sythetic Traiig Facility Sythetic traiig service o Hawk T1 aircraft for refresher ad Dec 95 ab-iitio fast jet pilots. 31 Joit Services Commad Ad Project merger betwee three MOD seior defece colleges ito a Mar 96 Staff College (JSCSC) sigle facility combiig educatio ad accommodatio. 32 Lyx Mk 7/9 Aircrew Traiig Aircrew traiig for Lyx pilots at a purpose built facility kow as Apr 98 Service (LATS) School of Army Aviatio. 33 Medium Support Helicopter Sythetic traiig service for aircrew of Chiook, Merli Mk3 ad Aug 95 Aircrew Traiig Facility (MSHATF) Puma helicopters. 34 RAF Setry E3D Aircrew Sythetic traiig service for flightdeck aircrew of Setry E3D aircraft. N/A 3 35 Torado GR4 Sythetic Sythetic traiig service for aircrew of Torado GR4 aircraft. Dec 96 Traiig Service 16 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

19 part oe date of Iitial Full Capital Term Ed Uitary Balace Public Sector Private Sector Case fiacial service service Value (Years) Date Charge Est sheet Survey Survey Study close date date ( m) ( m) treatmet received received Ju 02 Aug 02 Mar Off 4 4 Oct 03 Mar 07 Mar 09 1, Off 4 4 May 00 Feb 01 N/A Off 2 Ja 01 Apr 01 Sep Off 4 4 Ju 00 Apr 04 Apr O 4 Feb 00 May 00 Nov Off 4 Nov 01 Nov 06 Ja Off 4 4 Aug 98 Jul 00 Apr Off 4 4 May 03 Sep 05 Sep Off 4 Apr 99 Ja 01 Ja Off 4 Dec 97 Dec 98 Feb potetially 2.9 Off yrs to 2015 potetial extesio Ju 98 Aug 00 Sep O Jul 00 May 02 Oct potetially 3.7 Off yrs to 2025 potetial extesio Oct 97 Aug 99 Mar potetially 19.9 Off yrs to 2037 potetial extesio Jul 00 Oct 01 Oct potetially 1.3 Off 4 4 5yrs to 2030 potetial extesio Ju 99 Nov 02 Ja potetially 9.7 Off yrs to 2031 potetial extesio Allocatio ad maagemet of risk i Miistry of Defece PFI projects 17

20 part oe 6 The Miistry of Defece PFI portfolio cotiued Project descriptio OJEU/ C cotracts Bulleti Date Other 36 Defece Fixed Telecommuicatios A fully maaged telephoe service desiged to service 200,000 plus Mar 95 Service (DFTS) telephoe users makig 2.5 millio calls a day, across 2,445 UK sites. Facilities iclude voic , data switchig ad video coferecig. 37 Defece Housig Executive Provisio of IT/IS ifrastructure services to aroud 200 DHE sites. Ju 98 Iformatio Systems (DOMIS) 38 Electroic Messagig Project to provide coectivity betwee MOD IS etworks. Sep 96 System (Armymail) 39 Hazardous Stores Iformatio Systems Provisio of a Hazardous Stores iformatio system across whole Mar 96 MOD for 10.5 years. 40 MOD Wide Water & Wastewater Provisio of water ad wastewater services at over 4,000 MOD Mar 00 (Project Aquatrie) Package A sites i 3 geographic packages. Covers Wales, Midlads ad South West Eglad. 41 MOD Wide Water & Wastewater Provisio of water ad wastewater services at over 4,000 MOD Dec 00 (Project Aquatrie) Package B sites i 3 geographic packages. Covers Scotlad. 42 MOD Wide Water & Wastewater Provisio of water ad wastewater services at over 4,000 MOD Dec 00 (Project Aquatrie) Package C sites i 3 geographic packages. Covers North & East of Eglad. 43 RAF Fyligdales Power Statio Provisio of guarateed power supply to the missile early N/A 3 warig system. 44 RAF Lyeham Sewage Treatmet Refurbishmet of existig facilities to meet regulatory stadards. N/A 3 Populatio served 7, RAF Mail Iformal messagig services for RAF. Exempted 46 TAFMIS (IT) Traiig admiistratio ad fiacial maagemet iformatio system. Aug Tidworth Water & Sewerage Provisio of water ad sewage services i Tidworth Garriso. May 95 NOTES 1 The cotract ed date is calculated from the date that the service provided uder the cotract becomes fully operatioal. 2 Survey returs were ot received from these project despite the itervetio of the Departmet s PFU. 3 Iformatio ot available from the Miistry of Defece. 4 Part of this project is off balace sheet. 5 A survey was ot set to these projects as the cotracts were beig subsumed ito aother cotract at the time of our study. 6 Actual ot available, cotractual target was five moths after cotract sigature. 18 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

21 part oe date of Iitial Full Capital Term Ed Uitary Balace Public Sector Private Sector Case fiacial service service Value (Years) Date Charge Est sheet Survey Survey Study close date date ( m) ( m) treatmet received received Jul 97 Dec 97 Jul Off Sep 01 Oct 01 Oct Off 4 Apr 97 Oct 97 Oct Off 5 (Novated durig 2006) Feb 97 Dec 97 Dec Off 5 Apr 03 Dec 03 Dec Off 4 Sep 04 Mar 05 Mar Off 4 4 Oct 04 Mar 05 Mar Off 4 4 Dec 98 Apr 99 Apr O 4 Jul 98 Jul 98 Jul Off 4 Nov 98 Nov Off 2 Aug 96 Mar 97 Dec Off 4 Feb 98 Sep 98 Sep potetially 0.7 Off yrs to 2023 optioal extesio Allocatio ad maagemet of risk i Miistry of Defece PFI projects 19

22 part oe 7 The capital value of the Departmet s curretly operatioal PFI cotracts Number of projects ,000 1,000-1,500 Capital Value Rage ( millios) Source: Natioal Audit Office aalysis of Miistry of Defece projects database 8 The volume of curret operatioal PFI cotracts siged by the Departmet Number of projects siged i year Fiacial year startig Source: Natioal Audit Office aalysis of Miistry of Defece projects database 20 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

23 part oe The Departmet s oversight of its PFI projects 1.10 The Departmet has become a mature user of PFI. It has developed a approach to overseeig the procuremet ad maagemet of these projects i the light of lessos leared from its early PFI projects. The Departmet has a umber of ways i which it provides oversight to the procuremet ad maagemet of its PFI projects. These procedures are all aimed at obtaiig value for moey ad esurig that the risks of the projects are properly idetified ad maaged The Departmet s PFI projects are ofte procured ad maaged withi itegrated project teams (IPTs). For example, the Flight Simulators ad Sythetic Traiers (FsAST) IPT is resposible for the acquisitio ad itegratio of air eviromet sythetic traiig for all three armed services. These IPTs allow idividual projects to beefit from the Departmet s accumulated specialist kowledge of a particular service area Before a large PFI project ca be procured, it must i lie with other Miistry of Defece projects satisfy the Departmet s Ivestmets Approval Board that there is a value for moey case for proceedig with the procuremet. The Ivestmet Approvals Board also establishes whether the project is cosistet with the Departmet s strategic ivestmet plas, well-fouded i relatio to delivery of the customer s requiremets, takes accout of all key cost-drivers ad is deliverable To provide the Departmet s PFI projects with oversight ad advice o the specific issues relevat to PFI procuremet the Departmet established a Private Fiace Uit i the late 1990s. I 2005 the Private Fiace Uit was stregtheed ad the maagemet of the Uit was cetralised i Lodo. The Private Fiace Uit curretly comprises aroud 12 idividuals, based i both Lodo ad Abbey Wood ear Bristol The Departmet s Private Fiace Uit provides specialist commercial expertise to both operatioal ad procuremet project teams. It provides this by issuig Departmetal PFI policy ad guidace ad by offerig its PFI project teams assistace, advice ad a project iformatio resource. It also provides assurace to the Departmet s seior maagemet that PFI projects are properly maaged, soudly based ad i compliace with Treasury requiremets. The Uit coducted a review of operatioal projects i 2005 which cocluded that the Departmet s PFI projects are substatially delivered o time ad to budget ad the projects the deliver the services required. Our fidigs geerally cofirmed this coclusio I 2007 the Departmet s Commercial Directorate itroduced a ew process whereby seve additioal staff provide commercial scrutiy ad due diligece o busiess cases for all major projects that go to the Departmet s Ivestmet Approvals Board. This process seeks to replicate the checkig process that baks typically apply to the risks ad commercial terms of a project before agreeig to make fudig available. Its aim is to provide a higher level of assurace tha previously existed, esurig that the proposed commercial arragemets will meet the approved operatioal requiremet ad to reduce errors ad omissios i cotracts before they are siged. It is also iteded to help develop commercial policy based o the experiece of actual deals which have bee completed. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 21

24 Part two Maagig risks This part of the report aalyses the risks that the Departmet s PFI projects may face i successfully deliverig projects both i procuremet ad the i service phase. 2.1 Previous Natioal Audit Office reports o PFI projects have established that the appropriate allocatio of cotractual risks durig the procuremet phase ad the effective maagemet of those risks durig the operatioal phase are key aspects of obtaiig value for moey. 2.2 We examied the eight case study projects for te key risks to the delivery of the PFI service for the ed user (Figure 9). The risks may relate to both the procuremet ad cotract maagemet of the project. 2.3 We have assessed our fidigs i relatio to the risks to value for moey of the eight case study projects usig the classificatio set out i Figure 10. Value for moey is a compariso with other forms of procuremet ad with the best possible PFI outcome takig ito accout the deal as a whole ad the particular features of the project. This is cosistet with Treasury guidace o assessig the value for moey of PFI deals. 2.4 A summary of our assessmet of risk performace for the eight case studies is icluded as Figure 18 o page 46 ad 47. We cocluded that i ie out of te risk categories we examied there was either a low or moderate level of risk to value for moey. I oe risk category, the specificatio of the asset or service, we cocluded there was sigificat risk to value for moey to the procuremet phase although ot to the subsequet maagemet of the projects i their operatioal phase. The specificatio issues had cotributed to problems o two procuremets: the Armoured Vehicle Traiig Service project which was cacelled durig its procuremet ad the Defece Aimal Cetre where the cotract will eed to be reegotiated. The Defece Aimal Cetre has a capital value of 11 millio ad is therefore, uder curret Treasury policy, below the value threshold of projects which would ow be procured uder PFI. Sice these deals the Departmet has take a umber of steps to address the risk of iadequate specificatio of assets or services (Figure 3, page 7). I the other six of the eight case study projects we examied the risks had geerally bee well maaged with value for moey beig delivered for the taxpayer. Our detailed fidigs o each of the risks is set out startig at page Allocatio ad maagemet of risk i Miistry of Defece PFI projects

25 part two 9 Te key risks to project delivery Type of risk Desig ad delivery of the project Risk Category 1. Specificatio of the asset or service 2. Delivery of a PFI solutio Defiitio of risk Failure to adequately specify the asset or the service to be performed so that the service does ot correctly match the eed ad demads of users. Failure to deliver a PFI solutio i the procuremet phase or, after a cotract has bee let, failure to deliver the implemetatio of the required service i terms of the specificatio, the cotractual timescales or the expected Departmet budget. Delivery of the service throughout the lifetime of the cotract 3. Delivery of the ogoig service 4. Service user ad stakeholder satisfactio with delivery 5. Techology ad Latet Defects 6. Eviromet ad Safety Failure to deliver the service to the required stadards as set out i the cotract. Users are ot satisfied, or satisfactio is ot assessed i order to esure that value for moey is beig achieved from a user perspective. The techology beig used to provide the asset or service may ot be fit for purpose throughout the life of the cotract, due to iadequate desig, or uexpected use or misuse by the user. Security of physical assets ad iformatio may be compromised by iadequate project desig or maagemet. Expected or uexpected evirometal issues (such as military cotamiatio like fuel spillages or cotamiatio from the use of certai weapos) may affect the safe ad effective delivery of the service. Cotract maagemet processes 7. Performace moitorig ad maagemet regimes 8. Relatioships with cotractors 9. Resources ad skills 10. Risk maagemet processes Iadequate cotract performace mechaisms may ot provide sufficiet icetive to the cotractor to deliver the service specified i the cotract. Poor relatioships betwee the cotractor ad the public sector ca impact the service beig delivered ad thereby reduce value for moey. Isufficiet resources, icludig huma resources (staff capacity, skills, recruitmet ad retetio) or iadequate fiacial capacity to maage or procure the cotract effectively. Failure to put i place robust systems to moitor ad maage ogoig risks to the delivery of the service. Source: Natioal Audit Office 10 Classificatio of risk level based o the evidece from the case study projects Red meas that there is a sigificat risk to value for moey from this aspect of these projects. Amber meas that there is a moderate level of risk there are some risks to value for moey from this aspect of these projects. Gree meas that there is curretly a low risk to value for moey from this aspect of these projects. Source: Natioal Audit Office Allocatio ad maagemet of risk i Miistry of Defece PFI projects 23

26 part two RISK 1: Specificatio of the asset or service Defiitio Failure to adequately specify the asset or the service to be performed so that the service does ot correctly match the eed ad demads of users. Overall assessmet of this risk category i the eight case study projects Procuremet phase Sigificat risks to value for moey Maagemet of the cotracts Some risks to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Procuremet of these projects: Sigificat risks to value for moey. Effective specificatio of the required service is critical to successful project delivery. Issues were idetified i all of the case study projects we examied. There was a lack of data, particularly o existig services or the coditio of assets beig trasferred to the cotractor. Oe project did ot reach cotract lettig followig delays caused i part by a lack of iformatio o the service to be delivered. Our case studies also idetified examples where the asset was ot capable of providig the full service required, or where the actual level of usage has bee differet tha that expected whe the project was developed. Maagemet of these cotracts: Some risks to value for moey. The Departmet s aalysis of i-service projects shows that most projects, oce procured, are receivig the service they require from the cotractor. Our evidece suggests that, evertheless, o some existig cotracts actio may be eeded where service delivery problems arise from deficiecies i the cotractual specificatio. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio. 24 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

27 part two Data to quatify risk ad to support plaig 2.5 I order to allocate risk effectively, both parties eed to uderstad the size ad ature of that risk. For specificatios to be effective departmets eed to be clear o what they require, which i tur depeds o accurate data relatig to ay existig service provisio, the coditio of assets beig trasferred to cotractors ad the future eeds of the service users. The Departmet ofte experieces challeges i these areas because of the complexities of some of its projects ad the large umbers of potetial users of the services required. 2.6 I our case study projects we foud that robust data was ot always available. Collectig robust data may be difficult if the project is providig a ew service. We foud, however, that the data o existig services was frequetly isufficietly robust, both for performace iformatio ad o the coditio of the assets beig proposed to be trasferred to the private sector. Isufficiet iitial defiitio of the service requiremets Armoured Vehicle Traiig Service project: The complexities of this traiig project were uderestimated ad the requiremet was ot defied sufficietly clearly before egagemet with the cotractors i the biddig process. Advacig the procuremet took a log time as requiremets were idetified ad developed as the project progressed. It was ot util six years after the project was advertised to bidders that the Departmet uderstood the risks ad decided to abort the pla to procure services uder the PFI. Some of the requiremet has already bee delivered uder a alterative covetioal programme which the Departmet is proceedig with. Further details are at paragraph 2.15.The costs associated with the Armoured Vehicle Traiig Service are discussed i paragraphs 2.16 ad Chages eeded to make equipmet fit for purpose Field Electrical Power Supplies project: Whe the geerators were deployed i October 2003 the Departmet discovered that they were icompatible with certai vehicles. The vehicles, whe pullig the geerators, suffered maoeuvrability problems ad could ot safely tur corers. The Field Electrical Power Supplies Project Team had evertheless accepted the geerators as suitable for their eeds followig validatio testig ad user trials i 2002 ad The Departmet decided to modify the geerators ad paid the subcotractor 7.3 millio to exted the A frame that attaches the geerator to the vehicle. This cotractual chage was made without competitio i order to get the geerators operatioal withi the required timeframe. The project team does ot believe it maximised value for moey from this chage. Further details of the issues ad results of problems experieced by the case study projects arisig from isufficiet robust data are set out i Figure I some of our case studies, actual usage has tured out to be differet from the Departmet s expectatios whe developig the project. Some of these problems arose because the Departmet s operatioal eeds proved to be differet tha those iitially aticipated i the specificatio. Iitial usage lower tha expected Medium Support Helicopter Aircrew Traiig Facility project: The Departmet guarateed paymets to the cotractor equivalet to a level of usage of 80 per cet of the facility s capacity, across the first 20 years of the cotract. The aim was to esure that the cotractor could recover the capital cost of the facility. Actual usage of the facility has bee, ad is curretly, lower tha the usage guarateed to the cotractor (Figure 12 o page 27). It is difficult to idetify a sigle cause for the lower tha expected usage. Factors that may have caused a discrepacy betwee the expected ad actual usage of the facility may iclude: the Chiook Mk3 aircraft has ot bee itroduced ito service; chages to the timig ad umbers of Chiook, Puma ad Merli helicopters procured by the Departmet; ad higher tha expected operatioal deploymet durig the cotract period, which meas that aircrew are ot available for traiig. Overall usage to December 2007 was 64 per cet of the hours guarateed i the cotract. As a result the Departmet has cotracted for guarateed capacity that it has ot eeded of 34,000 hours. However usage icreased durig 2007 to 84 per cet of the guarateed hours level. If this tred cotiues the facility will be utilised, i future, at or above the quarterly level relatig to the Departmet s agreed paymets to the cotractor. If the Departmet eeds additioal traiig above this level it will have to buy additioal hours from the cotractor at a cost of at least 250 per hour. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 25

28 part two 11 The result of ot havig sufficiet robust data Project Armoured Vehicle Traiig Service Issue A key part of the deal structure was the trasfer to the cotractor of the risk of studets ot passig the traiig course. Iformatio critical to assessig the risks icluded historical data o studet pass rates ad the amout of live resource used i traiig (such as the usage of ammuitio ad hours spet i armoured vehicles). The Departmet did ot have sufficiet data available or the systems to collect data of sufficiet quatity, quality ad relevace to eable a robust value for moey PFI deal to be agreed. However, the project team advaced the discussios with PFI bidders i the absece of this data. Result There were misuderstadigs betwee the public ad private sector about the scale of the risk beig trasferred. The private sector s ability to commit to a structure ad price i their proposed solutio was limited. After six years the Departmet decided that a PFI solutio would ot give value for moey. Defece Aimal Cetre Mai Buildig Tidworth Water ad Sewerage Detailed iformatio o the coditio of the estate was ot available to bidders. The Departmet expected bidders to carry out their ow surveys durig the biddig process, or at the preferred bidder stage as part of due diligece. The project agreemet stated that the cotractor should, before the commissioig date, ispect the old equipmet ad project facilities to esure that it complied with the availability criteria. The Departmet was etitled to carry out this ispectio joitly. The ispectio was ot, however, carried out. The cotractor ad the Departmet failed to agree the coditio of the assets to set the baselie to determie whether the asset was fit for purpose ad deemed available i the future. The Departmet did ot carry out additioal surveys of the Mai Buildig premises. They were carried out by Modus after their appoitmet as preferred bidder. The surveys foud that further buildig work would be required because of the coditio of the buildig. Our report o the redevelopmet of MOD Mai Buildig recommeded that Departmets cosider the merits of makig a detailed survey available to all bidders to eable redevelopmet buildig risks to be priced competitively by all bidders. The covetioal optio used to compare the PFI proposals icluded ivestmet i ew sewerage treatmet facilities. As the facility was actually i a better coditio tha the Departmet had evisaged, Thames Water chose to repair ad maitai the existig ifrastructure rather tha build a ew waste treatmet plat. All maps, data, drawigs ad records should have bee made available i advace of the ivitatio to teder phase. Much of this iformatio was, however, retaied by the Departmet s Establishmet Works Cosultat (EWC). This iformatio would have helped bidders i their pricig by reducig the ucertaity i the pricig. The cotractor was exposed to uexpected cost which cotributed to the project s difficulties. As o baselie was set it icreased difficulties for the Departmet i eforcig the performace regime. There was a price icrease i the regio of 37 millio. Although the Departmet was satisfied that the work was ecessary, the opportuity to beefit from competitio betwee bidders o these additioal elemets was lost. The public sector comparator uderstated the good coditio of the facility ad overstated the ivestmet required by the Departmet. This iformatio would have helped bidders i their pricig. Heavy Equipmet Trasporter Defece Fixed Telecommuicatios System Some data o usage of the existig fleet was ot available as it had ever bee recorded. However, i this case the bidder had sufficiet iformatio based o the kowledge ad experiece of their ow bid team. Our previous report o the Defece Fixed Telecommuicatios System project idetified shortcomigs with the iformatio the Departmet preseted to bidders, particularly o volume of usage ad details of assets. 1 This shortcomig was also idetified i a earlier report o the Maagemet of Telephoes i the Miistry of Defece. 2 This iformatio could have helped bidders i their pricig. Bidders foud it difficult to price tariff charges accurately. Source: Social Fud Aual Report NOTES 1 HC The Private Fiace Iitiative: The Cotract for the Defece Fixed Telecommuicatios System. 2 HC The Maagemet of Telephoes i the Miistry of Defece. 26 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

29 part two Late chage i procuremet to reflect chage i operatioal requiremets Heavy Equipmet Trasporter project: The requiremet was chaged from 120 to 92 vehicles at a advaced stage i the procuremet. This reductio resulted from a chage i army structures ad a associated reductio i the umber of vehicles that the Heavy Equipmet Trasporter fleet would eed to trasport at ay oe time. The project team could ot have bee expected to foresee this chage as the iformatio was ot made available to them util a late stage i the procuremet process. Late chages of this magitude could affect the bid process, although i this case the procuremet was completed successfully. Equipmet beig used outside of its defied operatioal parameters due to ew operatioal eeds Both the Field Electrical Power Supplies project ad the Heavy Equipmet Trasporter project operate i eviromets outside of their defied cotractual operatioal parameters. I Iraq temperatures are above the level specified i the cotracts (45 o C was specified but actual temperatures exceed 50 o C). The dusty eviromet which the Field Electrical Power Supplies project works i ad the coditio of roads used by the Heavy Equipmet Trasporters have also bee outside cotractual parameters. These factors could impact performace ad, i certai situatios, the chage i use may mea the Departmet has to bear the risk of performace lapses or the cost of repairs. 12 Usage of the Medium Support Helicopter Aircrew Traiig Facility compared to expected ad guarateed levels Hours 1,200 1, Actual usage Guarateed usage 0 Mar 2000 Mar 2001 Mar 2002 Source: Miistry of Defece Mar 2003 Mar 2004 Mar 2005 Mar 2006 Mar 2007 Mar 2008 Mar 2009 Moth of operatio Mar 2010 Mar 2011 Mar 2012 Mar 2013 Mar 2014 Mar 2015 Mar 2016 Mar 2017 Allocatio ad maagemet of risk i Miistry of Defece PFI projects 27

30 part two RISK 2: Delivery of a PFI solutio Defiitio Failure to deliver a PFI solutio i the procuremet phase or, after a cotract has bee let, failure to deliver the implemetatio of the required service i terms of the specificatio, the cotractual timescales or the expected Departmet budget. Overall assessmet of this risk category i the eight case study projects Procuremet phase Some risks to value for moey Costructio or service implemetatio phase Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Procuremet of these projects : Some risks to value for moey Although the Departmet has achieved a very good delivery record from PFI procuremet times ca be legthy. I oe project, the Armoured Vehicle Traiig Service project, it took the Departmet six years from advertisig the project to potetial suppliers to coclude that it was ot possible to achieve a value for moey outcome usig PFI. Reachig a prompt decisio o the appropriate form of procuremet, ad executig it efficietly, are importat aspects of deliverig services o time ad o terms which are value for moey, particularly as the private sector will, over time, seek to recover bid costs i their pricig of cotracts. Costructio or service implemetatio phase of these projects: Low level of curret risk to value for moey The projects examied were mostly delivered o time ad to budget followig cotract sigature, which has bee a positive feature of the Departmet s use of private fiace. The exceptios, although sigificat at the time, did ot cause ay log term service delivery problems for the Departmet. The cotract geerally served to protect the Departmet from delivery failure. O oe project, the Defece Aimal Cetre, the cotract procured has ot produced the clarity of performace requiremets that would be expected from a PFI cotract. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio. 28 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

31 part two 2.8 The Departmet s 2005 review of its PFI projects i operatio foud that its PFI projects delivered substatially to time ad budget. Our cesus i 2007 foud that the majority of projects cotiued to report good experiece i the delivery of the projects ito service. (Figure 13). Oe case study project, Heavy Equipmet Trasporter, was delivered early by the cotractor i order to allow the equipmet to be used o frot lie operatios i Iraq. The refurbishmet of the Mai Buildig was also completed ahead of schedule. Other projects were geerally delivered o time. I four cases the curret members of the project team did ot kow whether the project has bee delivered o time ad to budget. This partially reflects the fact that may project teams are ow maagig mature projects, so service maagemet, rather tha asset delivery, is their curret focus. But it also reflects a lack of staff cotiuity. If staff with kowledge of procuremet or early operatios are ot retaied, it is harder to retrieve routie, but importat, iformatio such as performace o asset delivery. 2.9 I the Medium Support Helicopter Aircrew Traiig Facility ad Defece Aimal Cetre projects there were some delays to the implemetatio of the required services. I both cases the delays did ot cause ay serious disruptio to traiig ad the followig appropriate cotractual fiacial pealties were imposed o the private sector providers: Although the Medium Support Helicopter Aircrew Traiig Facility project was delivered o time, idividual delivery milestoes were missed because the cotractor experieced difficulties i deliverig the agreed visual system. Although the realism of the simulator was affected, the simulator was able to be used for traiig the Departmet s staff. I lie with the cotract, the Departmet charged the PFI cotractor liquidated damages of 2.98 millio for the late achievemet of those milestoes. Despite the iitial difficulties the early problems were subsequetly fully overcome. The accommodatio elemet of the Defece Aimal Cetre was delivered 11 weeks late. As a result the cotractor icurred pealties of 29,000. Procuremet times 2.10 The average procuremet time of the seve case study projects which reached cotract lettig was 43 moths. (Figure 14 overleaf). For example, the Field Electrical Power Supplies Project took over 68 moths from the iitial advertisemet to fiacial close ad the Defece Aimal Cetre, a less complicated project, took almost 44 moths. By compariso, our 2007 report o Improvig the PFI Tederig Process 4 foud that the average procuremet time for all the PFI deals across govermet with a capital value of over 20 millio ad that closed betwee 2004 ad 2006, was 34 moths. For earlier deals, closed betwee 2000 ad 2003, the comparable figure was 33 moths. 13 Delivery to time ad budget NOTES 1 I four projects the curret members of the project team did ot kow whether their project was delivered to time ad/or budget O time ad to budget 6 O budget, but ot o time O time, but ot to budget Neither o time or to budget 2 Oe of the six projects (Heavy Equipmet Trasporter) was delivered early by the cotractor. Source: Natioal Audit Office cesus of the Departmet s PFI projects 4 Improvig the PFI Tederig Process HC /07. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 29

32 part two 2.11 I the larger populatio of projects we surveyed, where data was available, the Departmet took o average 37 moths to procure its PFI projects. I 77 per cet of the projects we surveyed (36 of the 47 projects) the Departmet held data o procuremet times. I the other projects, maily because of chages i project staff sice the procuremets, such data was ot available. Larger projects ofte took loger to procure, the average for the Departmet s PFI projects with a capital value of over 50 millio, where data was available, beig 45 moths. These larger projects were aroud half of the projects supplyig data o procuremet times Some of the Departmet s PFI projects, icludig some of the case study projects, are complex or deal with specialist equipmet. These factors may add to the time eeded i procuremet compared with hospitals or schools, for which there is geerally a fairly stadard procuremet approach based o may similar previous PFI projects. The rage of the Departmet s i-house stakeholders who are ivolved i decisios about the projects ad the assessmet of the deals is also a factor. But some of the procuremet time icurred o the Departmet s PFI projects is due to chages i the specificatio, or difficulties i obtaiig data about service requiremets, durig the procuremet period. It is i the Departmet s iterest to be as efficiet as possible i its procuremet as the projects ofte eed to be delivered promptly for operatioal purposes, ad i the log ru bidders are likely to build the cost of time spet o biddig ito their pricig of PFI deals. Decisio that a PFI solutio is ot appropriate 2.13 The Departmet has established processes for determiig the procuremet method which it expects to deliver the best value for moey, ad for decidig whether the project should proceed to cotract lettig as a approved project. As a result of these processes the Departmet expects some projects to be abadoed or the form of procuremet to be chaged. I some cases however, a log period of time elapsed before these decisios were take o projects which the Departmet had advertised to the market as potetial PFI projects. Further time may be eeded if alterative procuremet optios are the to be pursued. These delays ca result i sigificat costs for both the Departmet ad private sector bidders. They may also affect bidders strategies for biddig for ad pricig future PFI cotracts. 14 Average procuremet times for the seve operatioal case study projects Projects i order of advertised date Defece Fixed Telecommuicatios Services (DFTS) Tidworth Water & Sewerage Medium Support Helicopter Aircrew Traiig Facility (MSHATF) Field Electrical Power Supplies (FEPS) Defece Aimal Cetre Mai Buildig Redevelopmet (MBR) Heavy Equipmet Trasporters (HET) Moths (from advertised date to fiacial close) Average procuremet times for all PFI deals from our report Improvig the tederig process Average procuremet times for the seve projects Source: Natioal Audit Office aalysis of case studies 30 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

33 part two Abadoig oe of the case study projects after six years 2.14 The Departmet ackowledges that it was usatisfactory for it to have take six years to reach a decisio i 2005 to cacel the Armoured Vehicle Traiig Service project which it had commeced i The project was, however, cacelled prior to a ivestmet decisio ad approval to proceed to cotract award, i lie with its process for esurig oly projects offerig value for moey proceed. The six years to reach a decisio to termiate the procuremet was despite substatial effort by the Departmet ad the iitial private sector bidders. The Departmet had also advaced the project to the appoitmet of a provisioal preferred bidder. However the critical lack of project maagemet ad maagemet iformatio cotributed to the legth of time before the Departmet decided to cacel the project The Departmet subsequetly proceeded with a alterative covetioal procuremet, the Ehaced Capability Armoured Traiig System (ECATS). Achievemets so far uder this project iclude the delivery of a ew device for Challeger II guery traiig, which has bee i service sice Jue 2006 ad a improved Warrior guery turret traier which etered service i May Together the Departmet estimates these will produce savigs of some 3 millio per year i live ammuitio The Departmet is ot able to idetify the total cost icurred o the abortive Armoured Vehicle Traiig Service procuremet as relevat records were ot retaied. The Departmet spet 5 millio o advisers i the bid evaluatio assessmet phase, but has ot retaied records showig spedig o advisers for the early phase of the procuremet prior to Furthermore, although the Departmet ow expects iteral costs such as staff time to be recorded, at the time this iteded project was beig developed, iteral staff cost recordig for each project was ot a requiremet Followig the cacellatio of this PFI procuremet, the Departmet paid a total of 10.6 millio to bidders. Of this, 7.7 millio was paid to secure itellectual property rights for material produced durig the procuremet. The itellectual property rights have ot bee utilised to date, although the Departmet remais cofidet that it will use these i due course. The Departmet also paid a additioal sum of 2.9 millio to oe of the bidders as a ex-gratia paymet i full ad fial settlemet of its ivolvemet with the cacelled project. Other decisios ot to proceed with a PFI solutio 2.18 The Departmet drew attetio to at least six other PFI projects that have bee advertised to idustry ad abadoed prior to cotract sigature. These iclude the Airfield Support Service Project which was cacelled three ad a half years after it was advertised to idustry. I additio, the Departmet has recetly aouced it will ot be pursuig a PFI solutio for oe of the two packages of the Defece Traiig Ratioalisatio Project, five ad a half years after the project was advertised to idustry. Other projects may also have bee abadoed i the early phase of developmet or have bee put o hold. Fluctuatios i fiacig costs prior to cotract lettig 2.19 A risk that may affect the delivery of a PFI solutio withi the Departmet s expected budget is that fiacig costs may fluctuate prior to cotract lettig. The curret ucertaities i the fiacig markets have also led to icreases i the fiacig costs of some of the Departmet s recet private fiace projects i procuremet. I some cases the scope of the PFI solutio may eed to be adjusted so that the project cost remais withi the Departmet s origial budget. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 31

34 part two RISK 3: Delivery of the ogoig service Defiitio Failure to deliver the ogoig service to the required stadards as set out i the cotract. Overall assessmet of this risk category i the eight case study projects Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Low level of curret risk to value for moey We foud that of our case studies oly the Defece Aimal Cetre has suffered serious performace limitatios to date, ad this was o a early private fiace cotract of a type which is ulikely to be replicated. However ay performace failures eed to be addressed promptly ad the Departmet was still addressig this particular issue at the time of our audit. The overall view that specified services were beig delivered is corroborated by the results of our cesus of all projects. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio. 32 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

35 part two 2.20 Oce cotracts reach their full i-service capability, we foud that project maagers mostly cosider that cotractors deliver the service specified i the cotract (Figure 15). 31 out of 39 project maagers who respoded to our cesus stated that the cotractor had delivered the specified service as specified i the cotract I the eight projects where the project maagers reported some shortfall i service delivery, i all but oe project, the Defece Aimal Cetre, project teams cofirmed that the service is curretly beig delivered satisfactorily. We cofirmed with the project maagers that the other egative resposes referred to either performace lapses that were ot cosidered to be serious, or early problems such as late delivery of the asset. I the Defece Aimal Cetre case study, the Departmet cosiders that the PFI cotractor Realm has performed badly o estate maagemet ad the aimal husbadry service. A particular problem has bee the Idoor Ridig School facilities which were deemed usuitable ad ot fit for purpose by the British Equie Associatio, despite the cotractor s attempts to repair it. The Departmet the asked Realm to replace the existig Idoor Ridig School with a much larger oe that meets the moder traiig best practice ad safety eeds at a cost to the Departmet of 705,000. As a result of the historic problems with the idoor ridig school facilities ad other problems with the service delivery the Departmet s employees at the facility have ot bee satisfied with the cotractor s performace. These problems have arise from a poorly drafted cotract, which is ot sufficietly precise i specifyig the cotractor s resposibilities. It has a ieffective paymet ad performace mechaism, which does ot provide effective icetive. The Departmet has cocluded that this project requires reegotiatio The majority of the Departmet s PFI projects reported that they are deliverig the specified service Oe of the projects who retured a survey respose did ot respod to this particular questio Yes No Do t kow Source: Natioal Audit Office cesus of all the Departmet s PFI projects i operatio Allocatio ad maagemet of risk i Miistry of Defece PFI projects 33

36 part two RISK 4: Service user ad stakeholder satisfactio with delivery Defiitio Users are ot satisfied, or satisfactio is ot assessed i order to esure that value for moey is beig achieved from a user perspective. Overall assessmet of this risk category i the eight case study projects Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Low level of curret risk to value for moey User satisfactio with services was geerally good both i the case studies ad i our cesus ad therefore there is a low level of curret risk to value for moey. There were some exceptios, for example the ogoig problems at the Defece Aimal Cetre ad some iitial user dissatisfactio with the late delivery of part of the Medium Support Helicopter Aircrew Traiig Facility. We also foud that because of the diverse ature of the projects, approaches to assessig user satisfactio varied i both the method used ad the regularity of assessmet. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio. 34 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

37 part two 2.22 The users of the PFI services which the Departmet has procured iclude both its military ad civilia persoel. I some cases users may be from outside the Departmet. For example, the Medium Support Helicopter Aircrew Traiig Facility provides traiig services for aircrew from allied coutries as well as the RAF. The Tidworth Water ad Sewerage project provides services to both the Departmet s persoel ad the residets of Tidworth User satisfactio with service delivery has geerally bee good i the case study projects. A particular example is the Defece Fixed Telecommuicatios System project which has cosistetly recorded user satisfactio i excess of target levels. A exceptio was the Defece Aimal Cetre where, because of the problems with service delivery, users have ot bee satisfied I some projects iitial issues over user satisfactio were overcome: The late delivery of the agreed visual system o the Medium Support Helicopter Aircrew Traiig Facility iitially impacted adversely o user satisfactio but this problem has bee resolved ad users are ow satisfied with the service. O the Heavy Equipmet Trasporter project some of the Departmet s stakeholders were opposed to the use of Sposored Reserves. 5 Though there were some iitial problems, the Departmet cosiders that these have bee overcome ad that Sposored Reserves have bee a success. The formal assessmet of user satisfactio 2.25 Assessmet of user satisfactio is a importat part of assessig overall performace. Twety five per cet of the projects respodig to our cesus (10 out of 40 projects) had ot udertake a formal assessmet of user satisfactio. I other projects we foud that the approach of the Departmet s project teams to assessig user satisfactio varied i both the method used ad the regularity of assessmet. I additio, user satisfactio was ot always formally built ito the performace maagemet system for the project There were some good examples i the case study projects of obtaiig regular ad systematic user satisfactio feedback. The Defece Fixed Telecommuicatios Service project uses a regular questioaire to assess user satisfactio. The Medium Support Helicopter Aircrew Traiig Facility uses a Balaced Scorecard as part of the overall assessmet of cotractor performace. User satisfactio is regularly assessed ad traslated ito formal measures, which feed ito the Balaced Scorecard. 6 5 Sposored reserves are idividuals whose employers are cotractually committed to provide specific operatioal capability to the Services. Durig frot lie operatios these persoel are called up ad serve as Army Reservists alogside regular Army persoel. 6 A Balaced Scorecard is a itegrated framework for describig strategy through the use of liked performace measures i four, balaced perspectives - Fiacial, Customer, Iteral Process, ad Employee Learig ad Growth. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 35

38 part two RISK 5: Techology ad latet defects Defiitio The techology beig used to provide the asset or service may ot be fit for purpose throughout the life of the cotract, perhaps because of iadequate desig or uexpected use or misuse by the user. There is also the risk of whether value for moey will be achieved where chages are required. Latet defects (iheret faults i the operatio of a asset which come to light after the asset is i use) may also impact services. Chagig the use of a asset may impact o the ability to idetify that latet defects have occurred. Overall assessmet of this risk category i the eight case study projects Some risks to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Some risks to value for moey The Departmet relies o specialist techology o a umber of its projects, for example i equipmet, traiig ad commuicatios. There are risks that the techology might fail or, give the fast chagig eviromet i which the Departmet operates, it might eed updatig i a situatio where competitio might ot be possible because of reliace o the origial supplier. There is also evidece from the projects we examied that if the Departmet has chaged the way a asset is beig used, it may be difficult for the Departmet to prove whether a defect is a latet defect (for which the cotractor is liable) as opposed to a defect which has arise from the chage of use. Whilst these are issues that could arise uder ay form of procuremet project teams eed to be alert to the risk that the value for moey expected whe lettig a PFI cotract is eroded as projects are adapted to chagig eeds. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio. 36 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

39 part two 2.27 The Departmet operates i a fast movig ad ofte upredictable eviromet. What it eeds from its projects may, therefore, chage more rapidly tha other govermet Departmets. May defece projects, icludig PFI deals such as Heavy Equipmet Trasporter, Field Electrical Power Supplies ad the Strategic Sea Lift Roll-o Roll-off Ferries (RORO) project, are used to support the frot lie operatios. The usage of equipmet may, therefore, eed to chage ad adapt to operatioal circumstaces at short otice The Departmet relies o specialist techology o a umber of its projects, for example i equipmet, traiig (icludig flight simulatio) ad commuicatios. The Departmet faces geeral risks that the techology might fail or, give the fast chagig eviromet i which the Departmet operates, it might eed updatig i a situatio where competitio might ot be possible because of reliace o the origial supplier. These are risks that would arise i ay form of procuremet. The followig issues arose i the case study projects. Medium Support Helicopter Aircrew Traiig Facility project: A umber of chages have bee made to reflect alteratios to the aircraft which the traiig simulator seeks to replicate. Chages of this type are a risk of usig traiig simulators, whether they are procured covetioally or through PFI. Further chages to the simulators may be eeded, but for the foreseeable future, these chages should be mostly capable of beig replicated o the simulators without the eed for a full techological update. Users will however expect the simulator to be kept up to date over the life of a cotract that may ru util Defece Fixed Telecommuicatios Service: The services beig delivered by this cotract have bee successfully chaged over time, i respose to the Departmet s requiremets for ehaced methods of commuicatio through ew techology sice the cotract was let i The Departmet s project team has used bechmarkig procedures to test the value for moey of additioal services ad has achieved some price reductios as a result As oted i the fidigs relatig to Risk Oe o the developmet of projects the Field Electrical Power Geerators ad Heavy Equipmet Trasporters have bee used i coditios outside those evisaged by the cotract specificatio. If there are subsequetly faults with these projects where there has bee a chage i use the Departmet may fid it difficult to show whether the fault is a latet defect relatig to the origial provisio of the equipmet. The cotractor is liable for latet defects whereas remedyig a fault arisig from the Departmet s chage i the use of the equipmet is likely to be a cost to the Departmet. The Departmet guidace expects project teams to take accout of the risk of ay additioal costs i remedyig defects i their value for moey case for makig chages to a project. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 37

40 part two RISK 6: Eviromet ad safety Defiitio Security of physical assets ad iformatio may be compromised by iadequate project desig or maagemet. Expected or uexpected evirometal issues (such as military cotamiatio like fuel spillages or cotamiatio from the use of certai weapos) may affect the safe ad effective delivery of the service. Overall assessmet of this risk category i the eight case study projects Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Low level of curret risk to value for moey Overall there is a low level of curret risk to value for moey i the case study projects from eviromet ad safety issues. These are, however, risks the Departmet eeds to cotiue to be aware of i its projects. Some of these risks may be best maaged by the cotractor, others by the Departmet, for example depedig o who ca most effectively cotrol the risk. Oly i the Field Electrical Power Supplies project have eviromet ad safety issues arise that affected risk maagemet. There have bee some istaces of the geerators catchig fire. The Departmet s use of the asset ad the desig of the geerators were both possible factors. After takig legal advice, the Departmet agreed that, sice it specified the desig of the geerators ad had tested ad fully accepted them, there should be o liability to the cotractor. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio There was a low level of curret risk to value for moey from eviromet ad safety issues i the eight case study projects. A uderlyig factor was that these risks have bee geerally allocated to the party best able to maage them. Sometimes this party was the cotractor, but i other cases the Departmet was better able to maage these risks. For example i the Tidworth Water ad Sewerage project, the Departmet was best placed to maage risks arisig from its previous actios, which had icluded World War II bullets beig placed i the drais. The Departmet s explosive ordace disposal teams are able to deal with ay icidets ad the Departmet ca educate its persoel to avoid iappropriate use of the water ad sewerage system. 38 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

41 part two RISK 7: Performace moitorig ad maagemet regimes Defiitio Iadequate performace mechaisms may ot provide sufficiet icetive to the cotractor to deliver the service specified i the cotract. Overall assessmet of this risk category i the eight case study projects Some risks to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Some risks to value for moey I the case studies examied performace mechaisms were mixed i terms of their effectiveess. O four projects the paymet mechaisms appeared effective. I oe case, Defece Fixed Telecommuicatios System, the risk of icorrect performace recordig materialised so that the Departmet had to subsequetly recover 1.3 millio from the cotractor BT. I the Defece Aimal Cetre ad Mai Buildig Refurbishmet projects the Departmet has bee addressig specific issues about the paymet mechaism. I these projects previous disagreemets o the operatio of the mechaisms had adversely affected the quality of the workig relatioships. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio All of the case study projects we examied had performace maagemet arragemets that set out madatory stadards of performace ad detailed what deductios ca be made if those stadards are ot met. We also foud through our cesus that all 41 of the projects who respoded had performace maagemet arragemets i place as part of their cotracts Our cesus foud that 36 of the 41 project teams strogly agreed or agreed that the performace mechaism icetivised the cotractor to deliver the specified service (Figure 16 overleaf). 7 Oly two project teams disagreed ad oe strogly disagreed that the performace mechaisms icetivised the cotractor to provide the service required by the cotract. I these cases, project teams felt that the performace mechaisms were ot proportioate. Project teams felt that pealties were ot sufficiet, or the burde of performace moitorig outweighed the beefit. 7 Out of 39 resposes to this specific part of our survey. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 39

42 part two 2.33 Our case studies showed that project teams felt that a parterig ethos ad good workig relatioships were a key part of the overall icetivisatio of the cotractor to perform at levels above the stadard required by the cotract Performace mechaisms i the case study projects varied, however, i their cosistecy ad quality (Figure 17). The risk of icorrect performace 2.35 I PFI projects the recordig of performace is usually udertake by the cotractor s staff to eable them to remedy service problems ad to reduce the burde of performace recordig o the public sector. As a cotrol, the public sector should ormally obtai user feedback, ad discuss this ad the cotractor s performace data with the cotractor o a regular basis. These methods of moitorig performace ormally work well. I the case of the Mai Buildig Redevelopmet project the Departmet ad the cotractor have ot yet foud the right balace betwee self-reportig by the cotractor ad audit by the Departmet ad are workig o a solutio. As we oted i our earlier report o this project (Miistry of Defece Redevelopmet of MOD Mai Buildig, HC ) it is appropriate for the Departmet to keep performace reportig systems uder review to esure the itegrity of the performace iformatio they receive Performace mechaisms effectiveess at icetivisig the cotractor The performace mechaisms we have i place icetivises the cotractor to provide the service required by the cotract. Strogly Agree Agree Disagree Strogly Disagree Source: Natioal Audit Office cesus of all Miistry of Defece PFI projects 2.36 These methods of reportig performace also carry a risk that private sector staff might ot record lapses i performace where the lapse is ulikely to be detected by the public sector but, if recorded, would result i paymet deductios I the Defece Fixed Telecommuicatios System project, we foud a specific istace of the private sector frauduletly recordig performace i such circumstaces. The cotractor, BT, was required to meet targets for the percetage of calls aswered withi specified time limits ad they would be liable for paymet deductios if the targets were ot met. BT staff at a call cetre i Ketterig, ow closed, artificially iflated the umber of calls beig aswered withi the required time limit by callig each other This fraudulet recordig of performace was ot detected through the performace maagemet system ad oly came to light subsequetly withi BT. BT seior maagemet have iformed us that they were ot iitially aware of the error i performace reportig which they cosider arose from the miscoduct of a small umber of staff workig at the call cetre. As the iflatio of calls beig aswered did ot result i a adverse experiece for the Departmet s users of the telecommuicatio services there were o complaits or adverse commets i customer feedback surveys to trigger a ivestigatio. This type of fraud ca oly be idetified by audits of the performace moitorig systems ad spot checks The Departmet ad BT reviewed the problems ad have agreed the compesatio due to the Departmet. As a result BT has made retrospective service paymets to the Departmet of 1,021,000 ad has reimbursed the Departmet for the exteral cost icurred by the ivestigatio totallig 122,000 excludig VAT ad for the value of the overstated calls totallig 197,000 excludig VAT Followig the fraud the Departmet ad BT have imposed a ew maagemet structure to the Defece Fixed Telecommuicatios project icludig ew goverace arragemets. BT is ow required to provide more detailed reportig ad the Departmet will carry out regular detailed audits of the ew BT reportig system to esure its itegrity. This will be supplemeted by regular service audits by the Departmet. Certai BT staff ivolved i the activities of Ketterig call cetre have bee released by BT. 40 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

43 part two 17 Case study projects performace mechaisms Project Medium Support Helicopter Aircrew Traiig Facility Commet The paymet regime appears effective but the performace ad paymet mechaisms are very complex. Field Electrical Power Supplies Defece Fixed Telecommuicatios System There were specific problems regardig the reportig of performace o the Defece Fixed Telecommuicatios System project with the private sector frauduletly recordig performace. Mai Buildig Redevelopmet The paymet regime geerally appears effective. However the Departmet believes that the pealties icurred by the cotractor to date for the occasioal service lapses that have occurred were too small. The Departmet also recogised that its ivolvemet i service moitorig caused atagoism betwee the public ad private sector. The Departmet therefore suspeded its service audits i September Both the cotractor ad the Departmet are determied to maitai ad improve performace stadards ad to itroduce a more effective audit methodology. This had ot bee implemeted by the time of our audit i autum 2007 although the Departmet ad the cotractor are cotiuig to work together o this issue. Heavy Equipmet Trasporter Tidworth Water ad Sewerage Few performace deductios, if ay, have bee ecessary to date. Paymet mechaisms have some idicators, such as those goverig reliability ad availability of the asset, which will become more challegig as the cotract progresses ad the assets age. Defece Aimal Cetre The performace mechaism was poorly desiged. There are a large umber of potetial performace idicators, ot all of which are relevat to the delivery of the core service. I additio, deductios for the asset ot beig available are set at a low level ad are much less tha the fiacial cost to the authority of service failure. As a result, the paymet ad performace mechaism lacks credibility ad has bee the cause of dispute betwee the authority ad the cotractor. Source: Natioal Audit Office aalysis of case study projects Allocatio ad maagemet of risk i Miistry of Defece PFI projects 41

44 part two RISK 8: Relatioships with cotractors Defiitio Poor relatioships betwee the cotractor ad the public sector ca impact the service beig delivered ad thereby reduce value for moey. Overall assessmet of this risk category i the eight case study projects Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Low level of curret risk to value for moey Good workig relatioships are recogised as a importat factor i securig good overall performace. We foud that, although there had bee problems i some projects over the operatio of paymet mechaisms, workig relatioships were geerally good betwee the Departmet ad its PFI providers. Poor relatios did however cotribute to overall poor performace o the Defece Aimal Cetre project. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio We foud that the relatioships betwee the Departmet ad the cotractors deliverig services are geerally good ad were makig a positive cotributio to cotract maagemet. Oly at the Defece Aimal Cetre, where there have bee major chages i the staffig both o the Departmet ad cotractor side, were there poor relatioships. These have arise because of the Departmet s frustratio with the ieffective paymet mechaism ad ambiguous cotract terms ad dissatisfactio with cotractor performace. Disagreemets over service moitorig had led to relatioship problems o the Mai Buildig Refurbishmet project but these problems have ow bee resolved. 42 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

45 part two RISK 9: Resources ad skills Defiitio Isufficiet resources, icludig huma resources (staff capacity, skills, recruitmet ad retetio) or fiacial capacity is isufficiet to maage or procure the cotract effectively. Overall assessmet of this risk category i the eight case study projects Some risks to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Some risks to value for moey Lack of skill was idetified as a barrier to success i two projects ad shortages were reported i two others. We also foud that there was sometimes a lack of cotiuity of staffig from the procuremet to operatioal phases meaig that ew staff who were ot ivolved i egotiatig the cotract had to take over the maagemet of the project. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio We foud that, whilst project teams had dealt with the procuremet process well i geeral, the level of staff skills ad cofidece i areas such as budgetig, forecastig ad risk allocatio varied from team to team. I two of the eight case studies, a lack of skill i the procuremet phase was idetified by project teams as a factor that had cotributed to value for moey risks: Despite the use of legal advisers, elemets of the Defece Aimal Cetre cotract are vague ad it has a ieffective paymet mechaism that has led to the service delivery problems. If the project teams had had the right skills ad had made proper use of advisers the problems might have bee idetified prior to the lettig of the cotract. The Departmet has also idetified lack of procuremet ad project maagemet skills as a major factor i the failure of the cacelled Armoured Vehicle Traiig Service PFI project. The Departmet s post project evaluatio cocluded that the project team was ot origially adequately resourced, or sufficietly experieced i dealig with PFI procuremets. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 43

46 part two Staff cotiuity 2.43 Staff cotiuity is a issue that affects all PFI projects. Staff ivolved i egotiatig a PFI project may ot ecessarily stay with the project oce the cotract has bee let. They may move o to egotiate other PFI projects or be trasferred to other duties. Also, some staff have particular skills i cotract maagemet ad will joi the project to carry out that fuctio. Some elemet of staff cotiuity after the lettig of the cotract is, however, desirable to esure that kowledge of the cotract ad the project s aims is trasferred to staff egaged i maagig the cotract O the Heavy Equipmet Trasporter ad Tidworth Water ad Sewerage projects staff cotiuity was idetified by the Departmet s project teams as a key driver of performace. I the MOD Mai Buildig Refurbishmet the Departmet s project director for the procuremet remaied i post for a year after the cotract had bee let to esure the smooth hadover to the cotract maagemet team Lack of cotiuity i core areas was, however, idetified as a area of potetial risk o the Defece Fixed Telecommuicatios System project. There has also bee lack of cotiuity o the Field Electrical Power Supplies project, where better trasfer of kowledge betwee the procuremet ad i-service phases may have bee useful i maagig the subsequet issues with the desig ad specificatio of the asset. 44 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

47 part two RISK 10: Risk maagemet processes Defiitio Failure to put i place robust systems to moitor ad maage ogoig risks to the delivery of the service. Overall assessmet of this risk category i the eight case study projects Some risks to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service 1 N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Reaso for overall assessmet of this risk category i the eight case study projects: Some risks to value for moey Effective risk maagemet is importat i deliverig value for moey i private fiace projects which are log term ad ofte complex projects. We foud some examples of good practice i adoptig a risk-based approach to procuremet i the case studies ad our cesus idetified that most of the Departmet s PFI projects also made some use of processes desiged to help moitor ad maage risks. There were, however, shortcomigs o four case study projects. I our cesus some project maagers did ot kow their risk maagemet process, ad processes i some istaces were isufficiet. NOTE 1 This project did ot proceed to cotract ad is ot part of the curret Departmet portfolio We foud some examples of good practice i adoptig a risk-based approach to potetial threats to value for moey durig procuremet. O the Tidworth Water ad Sewerage project the iitial preferred bidder, Wessex Water, was replaced because risk allocatio could ot be agreed, particularly aroud the extet of cotractor liabilities ad potetial paymets to the cotractor i the evet of breach or termiatio. The Departmet s project team was uwillig to retreat from its iteded risk positio ad approached the reserve bidder, Thames Water, to replace Wessex Water. The Heavy Equipmet Trasporter project was a pathfider for the use of Sposored Reserves employed as drivers ad mechaics. This practice was idetified as a area of potetial risk ad three stadard bid optios were developed, with differig proportios of the service delivered by Army persoel ad Sposored Reserves. The bidders addressed all three optios separately i their bids. By icludig differet optios, the project team could assess the impact of this risk o the evetual cotract price. It decided that a workforce made up of oe third Sposored Reserves offered best value for moey. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 45

48 part two Risk processes i the i-service phase 2.47 We foud that the Departmet s PFI projects made some use of processes desiged to help moitor ad maage risks. Twety-ie out of 41projects respodig to this sectio of our cesus stated that they had a risk register i place durig the i-service phase of the project. For oe of our case study projects, the Tidworth Water ad Sewerage project, the Departmet s project team did ot have a risk register. The cosistecy ad robustess of risk processes 2.48 Although our cesus foud that most of the Departmet s PFI project teams had some risk maagemet processes, eight of the Departmet s project maagers ackowledged that there was ot a clear process outliig how to maage risks o their project. Five of the 29 projects who stated that a risk register was i place did ot cosider that the risk register was a useful documet for maagig the cotract I our case study projects formal risk registers were ot well utilised durig the procuremet phase of the Field Electrical Power Supplies project. We also foud that the mai risk registers curretly used i the Field Electrical Power Supplies ad Heavy Equipmet Trasporter projects did ot have amed owers, as they oly idetified whether the risk lay with the Departmet or the cotractor Commuicatio with cotractors ad stakeholders is a key part of a robust risk maagemet process. Our cesus foud that, of the 29 projects who stated that they had a risk register i the i-service phase, 24 were shared with the cotractor. Maagig risks: summary 2.53 The Departmet has geerally maaged risks well i the PFI projects we examied. Our aalysis above shows that risk has also bee maaged well i six out of the eight case study projects evaluated ad that out of te risk categories ie show either a low or some potetial risks to value for moey (moderate risk). Figure 18 below summarises our assessmet: 18 Assessmet of Risk Performace for the Eight Case Studies Risk 1 Specificatio of the asset or service Procuremet phase Maagemet of the cotracts Risk 2 Delivery of a PFI solutio Procuremet phase Costructio or service implemetatio phase Risk 3 Delivery of the ogoig service Overall assessmet of curret risk to the Departmet: Sigificat risks to value for moey Some risks to value for moey Some risks to value for moey Low level of curret risk to value for moey Low level of curret risk to value for moey Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service N/A Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Source: NAO review of case studies 46 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

49 part two The Departmet s oversight of the PFI portfolio Although the Departmet s Private Fiace Uit keeps abreast of issues affectig the developmet ad performace of the Departmet s PFI projects, there is o cetral risk register that summarises risks across the Departmet s PFI portfolio. Such a register would eable the Private Fiace Uit to idetify commo risks ad esure that appropriate mitigatios are i place. For example a risk register could idetify commo risks, such as difficulties with obtaiig robust data about the service required, the risk of isufficiet skills i the project team or problems with staff retetio Other aspects of cotract maagemet were ot implemeted cosistetly across projects: we foud that post-project evaluatios had take place o may, but ot all projects. Post-project evaluatios are commissioed by the project s Seior Resposible Ower ad the Private Fiace Uit provides iput as the sector specialist. These reviews are importat i uderstadig how issues have arise ad how risks ca be maaged more effectively i the future. I oe case we examied, Field Electrical Power Supplies, a formal evaluatio had ot bee udertake. I other cases, such as the Defece Aimal Cetre, a formal evaluatio of the project might have bee more valuable if it had bee udertake earlier. Risk 4 Risk 5 Risk 6 Risk 7 Risk 8 Risk 9 Risk 10 Service user ad stakeholder satisfactio with delivery Techology ad latet defects Eviromet ad safety Performace moitorig ad maagemet regimes Relatioships with cotractors Resources ad skills Risk maagemet processes Low level of curret risk to value for moey Some risks to value for moey Low level of curret risk to value for moey Some risks to value for moey Low level of curret risk to value for moey Some risks to value for moey Some risks to value for moey N/A N/A N/A N/A N/A N/A Allocatio ad maagemet of risk i Miistry of Defece PFI projects 47

50 part two 48 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

51 Appedix oe The case studies This part provides more iformatio about each of the eight case study projects that we examied ad explais how they were selected. 1 As part of our review of PFI i the Miistry of Defece, we examied a selectio of eight case study projects, which illustrate the diverse ature of the Departmet s portfolio ad provide a isight ito the types of challeges preseted by their PFI projects. We chose two projects from each category of service: equipmet, traiig, accommodatio ad other support projects. 2 As show i Figure 20, we also selected the eight projects to give variety accordig to maturity (how log the project has bee operatioal), the type of service provided, the proximity of the service to the frot lie of operatios ad also the size of deal (based o the capital value of the project). 3 The eight case study projects also spet varyig amouts of time i procuremet. All the operatioal cotracts will ru util at least 2012 ad, i the case of the Medium Support Helicopter Aircrew Traiig facility may ru util The eight case study projects Type of service Equipmet Traiig Accommodatio Other Support Project Heavy Equipmet Trasporter Field Electrical Power Supplies Medium Support Helicopter Aircrew Traiig Facility Armoured Vehicle Traiig Service Miistry of Defece Mai Buildig Refurbishmet Defece Aimal Cetre Defece Fixed Telecommuicatios System Tidworth Water ad Sewerage Source: Natioal Audit Office aalysis of case study projects 20 The eight case study projects illustrate the diverse ature of the Departmet s PFI project portfolio, based o size, project maturity ad the proximity of the service to the frot lie Project Type Maturity Support vs. Frot Lie Size of capital value Heavy Equipmet Trasporter Field Electrical Power Supplies Service Medium Support Helicopters Aircrew Traiig Facility Armoured Vehicles Traiig Service Defece Aimal Cetre Mai Buildig Redevelopmet Tidworth Water ad Sewerage Defece Fixed Telecommuicatios Systems Equipmet Equipmet Traiig Traiig Accommodatio Accommodatio Other support Other support Early Operatio Early Operatio Mature operatio Recetly cacelled PFI procuremet Mature operatio Mature operatio Mature operatio Mature operatio Close to the frot lie Close to the frot lie Support but ifluetial o the frot lie Support but ifluetial o the frot lie Support Support Support Support Medium Medium Medium Medium Small Large Small Large Source: Natioal Audit Office aalysis of case study projects Allocatio ad maagemet of risk i Miistry of Defece PFI projects 49

52 appedix oe 21 Timelie showig the idicative procuremet ad operatioal phases of the eight case studies 01/01/ Heavy Equipmet Trasporters Advertised Field Electrical Power Supplies Advertised Medium Support Helicopter Aircrew Traiig Facility Cotract Siged Publicatio of PFI Case Study by OGC ISD FSD Exempted from OJEU Armoured Vehicle Traiig System Advertised ITN Defece Aimal Cetre Advertised Cotract Siged Mai Buildig Refurbishmet Advertised Cotract Siged Defece Fixed Telecommuicatios Advertised Cotract Siged ISD NAO Report Published FSD Tidworth Water ad Sewerage Advertised Cotract Siged ISD & FSD Procuremet Operatio (post Cotract Sigature) Natioal Audit Office Study ISD Iitial Service Data FSD Full Service Data Source: Natioal Audit Office aalysis of case studies. NOTES 1 Optio to exted to Optio to exted to Allocatio ad maagemet of risk i Miistry of Defece PFI projects

53 appedix oe /12/2007 Fieldwork Cotract Siged ISD FSD MOD Lessos Leart Issued To 2021 Cotract Siged ISD FSD To 2022 To Fial Bids Received BAFO Iteded Pref. Bidder Project Cacelled MOD Lessos Leart Issued ISD & FSD MOD Lessos Leart Issued To 2025 NAO Report Published ISD & FSD To 2030 Cotract exteded for 5 years Origial Date of the ed of the cotract To 2012 To Allocatio ad maagemet of risk i Miistry of Defece PFI projects 51

54 appedix oe Case study 1 Heavy Equipmet Trasporter (HET) The UK Heavy Equipmet Trasporter fleet comprises 89 Oshkosh 1070F Heavy Equipmet Trasporter vehicles (pictured) ad 3 Hitch Recovery Systems. Fact Box Category Equipmet Siged December 2001 Term 20 years Iitial service date July 2003 Full service date July 2004 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c 65.0 millio c 15.1 millio O Balace Sheet FASTTRAX Ltd Service to move battle taks ad other heavy equipmet durig peacetime ad o operatios. Sposored reserves make up oe third of the mapower required to deliver the service The cotract is deliverig the specified service ad the equipmet has provided a frot lie capability 4 The cotract for the Heavy Equipmet Trasporter project was siged i December The service was subsequetly delivered ahead of schedule i order to meet operatioal deadlies for usage i Iraq. The Heavy Equipmet Trasporter ad its operators have also served i frot lie operatios i Afghaista. 5 The Heavy Equipmet Trasporter project was a pathfider for the use of Sposored Reserves. Sposored Reserves are idividuals whose employers are cotractually committed to provide specific operatioal capability to the Services. Durig frot lie operatios these persoel are called up ad serve as Army Reservists alogside regular Army persoel. This was idetified as a area of potetial risk ad three stadard bid optios were developed, with differig proportios of the service delivered by Army persoel ad Sposored Reserves i each. The bidders addressed all three optios separately i their bids. By icludig differet optios, the project team could better assess the impact of this risk o the evetual cotract price ad decide which offered best value for moey. 6 There are curretly 85 Sposored Reserves employed (68 Heavy Equipmet Trasporter operators, ie maitaiers ad a additioal eight i reserve). Although there were some iitial misgivigs about the use of Sposored Reserves ad problems with their itegratio ito regular squadros, it is felt that these have bee overcome ad that Sposored Reserves have bee a success. The Departmet has had some difficulties meetig its cotractual obligatios ad the scope of the cotract may eed to chage to meet future eed. 7 I some istaces the Departmet has used the equipmet i a maer that has led to it acceptig additioal risk I the curret operatioal theatres, temperatures fall outside of the desig specificatio ad the roads are ot always of the coditio specified i the cotract. Whe the Heavy Equipmet Trasporter is used outside of cotractual specificatios, the Departmet is liable for ay damage icurred, or repairs required. Future heavy vehicles that were ot i the origial scope of the project such as Troja, Tita ad the up-armoured Challeger II may fall outside of the cotractual weight limit of 72 toes if carried by the Heavy Equipmet Trasporter. Although the Heavy Equipmet Trasporter is likely to be capable of hadlig such loads, the impact o the vehicle will eed to be assessed ad risks cosidered. The weight capacity is also restricted at the lower ed with a miimum weight limit of 25 toes. This is because light vehicles are curretly carried by a separate fleet of Light Equipmet Trasporters. This does ot mea that risk will be trasferred back to the Departmet but it does reduce flexibility. It would ot be cost effective or beeficial to the eviromet for usig Heavy Equipmet Trasporter to move small loads but there are some at the margis of the 25 toe limit that would be practical to be moved i such a way. 52 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

55 appedix oe 8 It is ot possible to say i these cases that the cotract should have icluded provisio for this ucertaity, sice this would have icreased the price of the cotract. I the case of usage i operatioal theatres, it is appropriate that the Departmet should retai complete cotrol over how the asset is used. However this does illustrate that the Departmet eeds to be aware as far as possible of future developmets whe selectig the most appropriate procuremet route although these ca be difficult to predict. It also illustrates the eed for effective chage mechaisms withi PFI cotracts. Case study 2 Field Electrical Power Supplies (FEPS) Risk trasfer is successful but the Departmet eeds to cotiue to maage future threats to value for moey 9 Risk maagemet is udertake o a joit basis, although the curret risk register origiated from the cotractor ad was added to by the Departmet s project team. The project team should regularly review these risks to esure that they accurately reflect their assessmet of the curret risk profile ad that mitigatios are i place to protect the Departmet s iterest. 10 Oe specific risk that will eed to be maaged is the impact of latet defects arisig as a result of usage outside the cotract specificatio. These may ot materialise util a later stage i the cotract. At the curret time latet defect risks lie with the cotractor, however disputes may arise if the cotractor ca show that defects were directly caused by usage outside of the origial cotractual costraits. I ay evet, ay factor that affects the ability of the cotractor to provide the service will impact o the user, regardless of who bears fiacial resposibility. The project team ad cotractor will eed to work together to esure that this risk does ot impact o the provisio of the service as the cotract matures. 11 The project teams will also have to maitai the curret effective workig relatioships whe the Departmet s team maagig the project moves from Adover to Abbey Wood ear Bristol. This has occurred as part of the formatio of the Defece Equipmet ad Support orgaisatio (DE&S) from the Defece Logistic Orgaisatio (DLO) ad Defece Procuremet Agecy (DPA). Oe of the 1347 geerator sets provided by Geistics. Fact Box Category Equipmet Siged Jue 2002 Term 20 years Iitial service date May 2003 Full service date April 2005 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c 73.4 millio c 12.1 millio O Balace Sheet Geistics, a special purpose compay established by Vickers Specialist egieerig who i tur are owed by Rolls Royce Provisio of 1347 geerator sets to support operatioal electrical requiremets i the field The cotract was ecessary as the existig equipmet was approachig the ed of its useful life ad did ot meet health ad safety legislative requiremets 12 The Field Electrical Power Supplies cotract was let to provide geerator sets to support operatioal electrical requiremets i the field. The cotract was ecessary as the existig equipmet was approachig the ed of its useful life ad did ot meet health ad safety legislative requiremets. I additio the existig system was outdated. Its thermal (heat) ad acoustic (soud) sigatures were high compared to more moder military equipmet, makig it more liable to detectio by eemy forces. It was also ureliable ad had poor mobility. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 53

56 appedix oe 13 The project started i Two bidders were short listed at the prequalificatio stage i April 1997 ad Geistics was selected as preferred bidder i July Ulike other PFI projects the Departmet specified the techical requiremet i detail i a maer similar to a covetioal procuremet. They produced a detailed requiremets specificatio idicatig exactly how may geerator sets were required ad the stadard that they were to be built to. Typically i PFI the customer specifies the outputs ad the suppliers the choose the best solutio to meet the specified criteria. By specifyig the iputs the opportuity for iovatio is reduced, which could impede value for moey. A preferred bidder was appoited while the commercial arragemets had ot bee fully agreed 15 Geistics was appoited preferred bidder before the commercial arragemets had bee fully agreed ad subsequetly the commercial agreemet had to be egotiated. As a result, the preferred bidder stage took 23 moths to reach a acceptable agreemet. These egotiatios were made without the beefit of competitio. PFI projects should have a mature commercial solutio agreed i priciple before a preferred bidder is appoited. Uder the ew system of competitive dialogue this will be a eve more importat cosideratio. 16 Durig the preferred bidder stage, Aggreko, the subcotractor who was proposed to provide the fleet maagemet services of the cotract, withdrew from the cotest. ABRO, who are part of the Miistry of Defece, but are a tradig fud, stepped i ad a sub-cotract was agreed with Geistics. Problems with the desig were idetified two years after the prototype had bee accepted by the Departmet 17 Oce the cotract was siged a prototype was made available to the Departmet for trials. Validatio testig ad user trials took place i 2002 ad After acceptace Geistics started the mass productio of the 1347 Geerator sets. The deploymet of the geerators bega i May 2003 however i October 2003 it was idetified that, for certai Miistry of Defece vehicles, there was a iterface issue that made them icompatible. This made the vehicles usafe whe turig corers. The solutio was to exted the A frame coectig the geerator to the towig hitch at the expese of mobility. As these particular vehicles are widely used by the Departmet this was a problem which eeded to be resolved, particularly as the geerators were eeded o frot lie operatios. 18 The cotractor i a gesture of goodwill agreed to waive their exclusivity clause to allow the Departmet to use their existig stock of geerators whilst the whole fleet of geerators was modified. A acceptable desig had bee obtaied by extedig the A frame by 650mm. Oce the modificatio was agreed the geerators were rectified. Betwee April 05 ad March 2006, 80 per cet of the fleet were modified with the last 20 per cet beig modified by March The cost of the modificatio was 7.3 millio. 19 Although the modificatio solved the majority of the Departmet s problems with the iterface with the existig fleet of vehicles, the Departmet are curretly replacig may of its vehicles. The project to replace the 16,000 fleet of four, eight ad 14 toe cargo ad recovery vehicles is well uderway with the first shipmet of 1285 vehicles eterig service for traiig i Autum 2007 with full service beig achieved i Further deliveries will be made betwee 2008 ad These vehicles are ot fully compatible with the Field Electrical Power Supplies as the height of the towig eye could impede safety ad the wirig will eed modificatio. It is therefore possible that the Field Electrical Power Supplies fleet will have to be further modified to address this issue. The equipmet is beig used outside of its defied operatioal parameters 20 The Field Electrical Power Supplies equipmet is beig used outside of its defied operatioal parameters. I Iraq temperatures are above the level specified i the cotract (45 0 C was specified but actual temperatures exceed 50 0 C). The dusty eviromet is also outside of the specified parameters. These factors may impact performace idicators ad may mea that the supplier will be able to claim that latet defects have arise from the assets use outside of the specified eviromet. The Departmet will have to bear the cost of the additioal wear ad tear. 21 There are other costs that have bee icurred by the Departmet arisig from damage caused to the geerators. These rage from traffic accidets to mior icidets such as losses of tools ad keys. The cost to the Departmet approximates to 0.5 millio per aum. These icidets would have arise i the past ad would also likely arise uder a covetioal procuremet however uder PFI the costs are more visible. Uder the cotract, the Departmet oly has the optio to esure that the geerators are fully repaired because the geerators always have to be i the coditio specified i the 54 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

57 appedix oe cotract. This has the beefit of esurig capability is maitaied but reduces the ability of the Departmet to defer expediture. 22 I additio to the repairable damage to the geerator, seve sets have bee destroyed some due to hostile actio i Afghaista. Whe a geerator is destroyed the Departmet has to pay the cotractor for that geerator util the ed of the cotract or util the Departmet purchases a replacemet geerator. As it is iefficiet to replace the geerators o a piecemeal basis, the Departmet has decided to delay the replacemet util 2008/09 (five years ito the cotract) ad the purchase 15 sets. This will esure that the Departmet has a stock of geerators to call o whe ad if other geerators are lost. Both sides have had to bear further uexpected risks 23 The geerators have, i a umber of isolated cases, caught fire. The cause of the fire is partly dow to misuse ad ot followig documeted procedures, exacerbated i the view of the Departmet, by the desig of the geerators. This was a area for dispute with the cotractor. However, havig take legal advice, it was agreed that sice the Departmet specified the desig of the geerator ad had tested ad fully accepted the geerators there would be o liability to the cotractor. 24 Some risks have remaied with the cotractor: a fire at oe of the premises used to store spares resulted i a paymet deductio of 41,000. I additio there have bee other paymet deductios for ot meetig Key Performace Idicators (KPIs), however the reliability KPIs have ot bee triggered ad the umber of call-outs are lower tha aticipated. However reliability may become more importat i the later stages of the cotract. 25 The cotractor has also had to bear the cost of storig geerators, as the usage has bee lower tha the expected level of 1189 (just uder 90 per cet of the total 1347). I additio, the umber of moves where ABRO collects or delivers geerators is higher tha plaed, which may idicate that the equipmet is beig used differetly tha expected by Army uits. Both moves ad holdigs have also icreased sice the cotract became fully operatioal. Risk maagemet procedures are i place 26 The Departmet did ot use formal risk procedures i the procuremet phase of the cotract. However i the operatioal phase the cotractor ad the Departmet use a joit risk approach to maagig the cotract. Risk is tabled at regular project review meetigs ad cosidered a importat way of maagig the cotract. ABRO also maitais its ow risk register. 22 Both the umber of moves ad the umber of geerators held i store have icreased Jul 05 Sep 05 Nov 05 Ja 06 Mar 06 May 06 Jul 06 Sep 06 Nov 06 Ja 07 Mar 07 May 07 Jul 07 Holdigs Deliveries Source: Miistry of Defece Allocatio ad maagemet of risk i Miistry of Defece PFI projects 55

58 appedix oe Case study 3 Medium Support Helicopter Aircrew Traiig Facility The Medium Support Helicopter Traiig project icludes the provisio of a umber of flight simulators based at RAF Beso. Pictured above is oe of the Chiook simulators. Fact Box Category Traiig Siged October 1997 Term 20 years plus optio to exted 20 years Iitial service date August 1999 Full service date March 2001 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c millio c 20.1 millio Off balace sheet cae Aircrew Traiig Services plc Helicopter aircrew traiig facilities for Chiook Mk2/2A ad Mk3, Merli Mk3 ad Puma Mk1 aircraft. Traiig icludes coversio to type traiig, cotiuatio traiig, ad missio rehearsal The project is deliverig its iteded service to users, overcomig some early problems 27 The Medium Support Helicopter Aircrew Traiig cotract, siged i October 1997, was oe of the first operatioal Miistry of Defece PFI projects. The project has bee fully operatioal sice March 2001 ad is still recogised as a leadig facility of its type. It is used ot oly by RAF aircrew but also by aircrew from allies icludig the Dutch, Australia ad Sigaporea air forces. The cotract could ru for 40 years but the Departmet has the right to break, without compesatio to the cotractor, at the 20 year poit i The Medium Support Helicopter Aircrew Traiig Facility project experieced some problems with the delivery of the PFI project. Overall the project was delivered o time, but idividual delivery milestoes were missed, due to the cotractor experiecig difficulties i deliverig the agreed visual system. I lie with the cotract the Departmet charged the PFI delivery compay liquidated damages of 2.98 millio o the missed milestoes. These costs were passed dow to the compay s subcotractor. 29 The difficulties meat that a iterim visual system had to be put i place with limited capability to reproduce special effects. This impacted o the realism of the simulator ad as a result user satisfactio was reduced. The early problems with the Medium Support Helicopter Aircrew Traiig Facility project were subsequetly fully overcome; however, it ivolved a lot of effort to maage the expectatios of the users durig this difficult period. Overall usage of the facility is ot as high as expected 30 A level of usage, equivalet to 80 per cet of the facility s capacity, is guarateed to the cotractor across the first 20 years of the cotract. Actual usage of the facility has bee, ad is curretly, lower tha the level that is guarateed to the cotractor. Overall usage over the i service phase to December 2007 has bee 64 per cet of the level guarateed i the cotract, with 34,000 uused hours. The Departmet has cotracted ad therefore paid for capacity that it has ot subsequetly eeded. Expected usage icluded Chiook Mk3 traiig, but that aircraft has still to be itroduced to service. However usage is icreasig; i 2007 usage was 84 per cet of the guarateed level ad i the later stages of the cotract it may exceed the guarateed level. This would result i the Departmet havig to buy additioal hours usage at a cost of at least 250 per hour. I such cases, the Departmet will eed to esure that usage of the service is optimised i the remaiig period of the cotract ad that the project team is workig actively with the cotractor to esure that this happes. 31 The Departmet origially aticipated that the facility would be i use at least at the guarateed level for much of the time. Usage of the facility as a whole has steadily icreased throughout the cotract term. However to date it has, geerally, bee lower tha the guarateed level. As a result of this, the Departmet may have retaied a greater level of risk o the usage of the facility tha it expected. Whilst the Departmet may sometimes require spare capacity to cope with surges i demad, the Departmet has cotracted for capacity that it has ot subsequetly eeded. Overall value for moey will be depedet o esurig that the usage of the facility is 56 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

59 appedix oe optimised; i the loger term there may be additioal cost to the Departmet if the usage goes above the guarateed level. If a covetioal solutio had bee used it is likely that the same umber of simulators would have bee provided ad therefore its value for moey would have bee similarly affected by lower tha expected utilisatio. 32 It is difficult to idetify a sigle cause for the lower tha expected usage. Factors that may have caused a discrepacy betwee the expected ad actual usage of the facility may iclude: the Chiook Mk3 aircraft has ot bee itroduced ito service; chages to the timig ad umbers of Chiook, Puma ad Merli helicopters procured by the Departmet; ad higher tha expected operatioal deploymet durig the cotract period, which meas that aircrew are ot available for traiig. always bee possible to match updates to the aircraft o a like-for-like basis. This is a issue o the Chiook simulator, which is ow diverget from the actual machie, particularly o the versio of the Chiook used by Special Forces. 34 A dedicated, seior user represetative is based o site ad is workig with the project team who maage the cotract, the private sector cotractor ad the user commuity i order to promote use of the facility. The project team moitor user satisfactio formally as part of its performace moitorig, via the project s ow balaced scorecard. 35 The cotractor has also made some ivestmet i the facility, from which the Departmet ad other third party customers should beefit. A cotractual value for moey review due i 2008 will provide a opportuity to discuss the log term strategy for the facility, particularly i view of the potetial o compesatio break-poit due i The structure of the cotract is workig well but there are ogoig risks that eed careful maagemet 33 I traiig programmes such as the Medium Support Helicopter Aircrew Traiig Facility, the quality ad relevace of the traiig to users depeds sigificatly o whether the traiig eviromet is faithful to the aircraft or other platform that is beig simulated. The Departmet is resposible for fudig this type of modificatio ad, whilst there has bee ivestmet i the facility, it has ot 23 Usage of the facility has bee below the expected level ad the level at which paymet to the cotractor is guarateed Hours 1,200 1, Actual usage Guarateed usage 0 Mar 2000 Mar 2001 Mar 2002 Source: Miistry of Defece Mar 2003 Mar 2004 Mar 2005 Mar 2006 Mar 2007 Mar 2008 Mar 2009 Moth of operatio Mar 2010 Mar 2011 Mar 2012 Mar 2013 Mar 2014 Mar 2015 Mar 2016 Mar 2017 Allocatio ad maagemet of risk i Miistry of Defece PFI projects 57

60 appedix oe Case study 4 Armoured Vehicle Traiig Service (AVTS) The Armoured Vehicle Traiig Service was desiged to provide a balace of live ad sythetic traiig for a umber of Armoured Fightig Vehicles icludig the Challeger II tak pictured above. Fact Box Category Traiig Siged cacelled i Jue 2005 Term Iitial service date Full service date Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective Potetially 30 years N/A N/A N/A N/A N/A N/A Provisio of live ad sythetic traiig of guery ad specialist drivig for Armoured Fightig Vehicles. To provide crew traiig for a wide rage of Armoured Fightig Vehicles, icludig Challeger II taks, Warrior Ifatry Fightig Vehicles ad Combat Recoaissace vehicles The project was ambitious ad had the potetial to use PFI to improve value for moey 36 The Armoured Vehicle Traiig Service was a procuremet project that was iteded to deliver a PFI cotract to trai the army s armoured vehicle crews. The proposed service icluded the provisio of istructors, traiig services, traiig equipmet ad buildigs, desig of the course syllabus ad developmet of traiig devices. This project had a total cotract value of over 1 billio ad the traiig would have take place across multiple sites ad traied Army persoel o a variety o f armoured vehicles, icludig the Challeger II Mai Battle tak. Similarly to the Medium Support Helicopter Aircrew Traiig Facility project, the ratioale was to use simulatio ad other measures to produce efficiecies. These efficiecies would arise out of reducig the umber of miles that vehicles were used i the field for basic traiig ad out of reducig the amout of live firigs. The deal was cacelled i Jue 2005 ad the Departmet made paymets to the bidders of 10.6m. A additioal 5m had bee spet o the Departmet s advisers 37 The OJEU was issued i October 1999 ad the Outlie Busiess Case was approved i December Three cosortia were biddig for the project ad i July 2004 the Departmet appoited a iteded preferred bidder. After further egotiatio the parties were uable to reach a acceptable agreemet resultig i the aoucemet i Jue 2005 that the PFI project would ot go ahead. 38 The cost of the abadoed PFI project icluded 5 millio spet by the Departmet o advisers i the bid evaluatio phase, though the sped o advisers prior to 2000 i the early phase of the procuremet is ukow as records were ot retaied. The Departmet s ow iteral cost of resourcig this major procuremet over the six years of the project was ot recorded, as this was ot a requiremet. I additio, the bidders are likely to have icurred substatial costs, both i terms of direct costs ad the opportuity cost of ot beig able to use their bid resources o other projects. The experiece of the cacellatio may also have affected market cofidece with bidders ot willig to bid o other projects. 58 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

61 appedix oe 39 The Departmet s procuremet team were redeployed with may of them workig o a alterative to Armoured Vehicle Traiig Service through a covetioal procuremet programme. The project (re amed ECATS) has delivered capability i the form of the Live Fire Crew Traiig System (LFCTS) for Challeger II guery (a removable barrel isert with a smaller bore). The Departmet decided to secure the itellectual property rights produced by the bidders for Armoured Vehicle Traiig Service for possible use i this ew programme. 40 Followig the cacellatio of the deal a total of 10.6 millio was paid to bidders by the Departmet. Of this 7.7 millio was paid to secure itellectual property rights (IPR) for material produced durig the procuremet. The IPR has ot bee utilised to date, although the Departmet is cofidet that it will add value to the ECATS programme i due course. I additio to the IPRs, a sum of 2.9 millio was paid to oe of the bidders as a ex-gratia paymet i full ad fial settlemet of the cacelled project. The procuremet was ot well maaged 41 The Departmet has idetified lack of procuremet ad project maagemet skills as a major factor i the failure of the cacelled Armoured Vehicle Traiig deal. The Departmet s post project evaluatio cocluded that the project team was ot adequately resourced, or sufficietly experieced i dealig with PFI procuremets. The team was also ot well supported, sice it was based i a differet physical locatio, Bovigto, from the mai Defece Procuremet Agecy (DPA) base at Abbey Wood. The project did ot have a Seior Resposible Ower, though this was ot a requiremet at the time. Later o i the deal, a ew experieced team was put i place. The appoitmet of the ew team led to the reaalysis of the project ad the decisio to abado the PFI strategy. 42 As a result of the problems idetified the procuremet process took a log time; requiremets were ot adequately specified ad had to be developed as the team gaied a uderstadig of the project. This was achieved by havig a umber of biddig rouds. The iteded trasfer of risks was ot supported by adequate maagemet iformatio, ad additioal iformatio was ot collected 43 I additio to the issue of defiig the requiremet the project team were drive by the desire to esure that the asset would be off balace sheet to address the Departmet s affordability cocers. This meat that there had to be sufficiet risk trasfer to the private sector ad oe of the key risks was the amout of live resources (such as ammuitio ad hours spet i armoured vehicles) required by cadidates to pass the various tests they would face durig their traiig. 44 A key part of the deal structure was the trasfer of course pass rate risk ad this iflueced the amout of live resources to be maaged ad used. However, historical data o live resource use ad studet pass ad failure rates were either reliable or complete. The project team decided to proceed i the absece of this data; however they did ot seek to collect the data durig the course of the procuremet. This led to fudametal misuderstadigs betwee the public ad private sector about the scale of the risk beig trasferred. 45 At the time whe bidders were submittig their Best ad Fial Offers (BAFO), a ew team, based i Abbey Wood, were put i place together with ew legal advisers. Although the Miistry of Defece ad its advisers had misgivigs they decided to press ahead with the BAFO roud to esure that the project did ot suffer ay further slippage. 46 After the BAFO roud there were still cocers ad the Miistry of Defece decided to appoit a provisioal preferred bidder for a risk reductio exercise. Further problems ad cocers over the value for moey led to the decisio beig made to abado the PFI strategy ad to seek a covetioal procuremet, which will provide improvemets to existig capability over a umber of years. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 59

62 appedix oe Case study 5 Mai Buildig Redevelopmet (MBR) 48 The cotract covered the redevelopmet of Mai Buildig ad limited refurbishmet ad provisio of support to other buildigs where staff would be accommodated durig the redevelopmet. It also icluded the provisio to 2030 of maiteace ad facilities maagemet at Mai Buildig ad the Old War Office followig completio of the redevelopmet. The project was complicated by the eed to decat 3,000 staff ito other cetral Lodo accommodatio durig refurbishmet. 49 The cotract allocated the majority of the risks associated with the project to the cosortium, with oly the risks relatig to delivery of Departmet operatioal objectives ad the iformatio techology systems remaiig with the Departmet. The refurbished Miistry of Defece Mai Buildig as see from the air. Fact Box Category Accommodatio Siged May 2000 Term 30 years Iitial service date September 2004 Full service date September 2004 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c millio c 79.0 millio O balace sheet Modus Service Limited Developmet ad refurbishmet of the Miistry of Defece Mai Buildig, ad temporary accommodatio to other buildigs, as well as upkeep of Old War Office buildig The existig buildig i Whitehall required improvemets to the accommodatio ad workig eviromet 47 The Departmet requires secure, moder ad efficiet workig accommodatio i cetral Lodo from which to direct defece operatios. The existig buildig i Whitehall required improvemets to the accommodatio ad workig eviromet. To this ed the Departmet siged a PFI cotract with Modus a cosortium comprised of Iisfree, Laig ad Amey 10 i May 2000 to redevelop the Miistry of Defece Mai Buildig. The redevelopmet also eabled the Departmet to ratioalise staff i Lodo ad dispose of five of its sites resultig i estimated savigs of up to 18 millio a year. The decisio to pursue the PFI solutio was based o the assessmet of additioal beefits that would flow from PFI 50 The Departmet estimated that the cost of the PFI deal would be similar to covetioal procuremet. The decisio to pursue the PFI solutio was based o the assessmet of additioal beefits that would flow from PFI. The Departmet cosidered these to be the followig: Greater price certaity. Icetives to complete o time uder the cotract, Modus would lose 1 millio for each moth of delay after the 30 November 2004 deadlie for reoccupatio. It would also have to meet the cost of maitaiig alterative accommodatio. Icetives to deliver the service to the required stadard uder the cotract, Modus oly receive the full aual fee oce the accommodatio has bee provided to the specified stadards, eve if the Departmet chooses to use the accommodatio. Furthermore, up to 20 per cet of Modus facilities maagemet charges are at risk if the stadard of service provisio is ot satisfactory. A sigle cotract for the desig, maiteace ad operatio of the buildig ecouragig whole-life costig. 51 The Modus bid was selected as preferred bidder i Jauary 1999 as its bid was 42 millio lower tha that from its competitor ad it more closely matched the Departmet s output specificatio ad commercial requiremets. 10 Amey were subsequetly bought out by other members of the cosortium. 60 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

63 appedix oe The price of the deal icreased durig the preferred bidder stage 52 The deal was closed 16 moths after Modus became preferred bidder. Durig this period the price icreased by 99 millio (at 2000 price levels). This was due to iterest rate icreases, idetificatio of additioal capital expediture ad other movemets i the fiacial markets. The project was the subject of a NAO value for moey report 53 The Mai Buildig project was the subject of a NAO value for moey report (HC 748) i April The report cocluded that: the cotract gives the Departmet what it set out to procure specifically, it requires Modus to deliver the Departmet s physical requiremets ad it has the appropriate features of a PFI deal icludig risk trasfer; the costs uder the cotract will be similar to the forecast cost of covetioal procuremet but the balace was tipped i favour of PFI by other factors; the procuremet was effective specifically, the deal was selected oly after a wide rage of optios had bee cosidered ad the cost of the deal icreased maily due to survey work ad icreased fiacig cost icludig icreased iterest rates, the risk of which ormally rests with the public sector util cotract sigature; ad the maagemet of the cotract has bee good specifically, the Departmet has coducted parterig workshops, adopted a joit missio statemet with Modus, maitaied the cotiuity of well-traied, appropriately skilled staff ad adopted appropriate chage procedures. The refurbishmet was completed ahead of schedule 54 Sice the NAO report was published the refurbishmet was completed ahead of schedule ad staff were moved back. Geerally the project has bee a success with oly mior issues arisig. These issues have largely related to poor performace i the delivery of soft services. The problems were also exacerbated by chages i the cotractor s maagemet but sice the the maagemet team has bee stregtheed. The Departmet is resurrectig its risk maagemet procedures 55 Risk maagemet had bee particularly thorough durig the procuremet phase but had lapsed i its thoroughess durig the i-service phase. The Departmet s cotract maagemet team was i the process of re-establishig its full risk maagemet processes. I November 2006, Modus itroduced a shared risk register to cover the geeric risks shared by both parties. The Departmet were i the process of updatig their risks ito this register durig the course of our study. We were told that the risk register will be formally reviewed by the Facilities Board that meets o a mothly basis. There have bee issues with the paymet mechaism 56 May PFI projects require the cotractor to carry out performace moitorig. However i the case of the Mai Buildig Redevelopmet project, this task fell to the Departmet cotract team. The aual cost of carryig out the service audits i 2002 was some 75,000 a year. Service delivery deductios betwee August 2000 ad August 2006 totalled 221,000, a average of 37,000 per aum. 57 The cotract s performace maagemet mechaism geerates low value fiacial deductios i respect of mior shortfalls i performace delivery; larger deductios would arise i respect of ay part or aspect of the buildigs or their services that are uavailable for use by the Miistry of Defece. As there have bee o uavailability deductios levied to date ad oly relatively mior weakesses i performace delivery, the pealties icurred to date have bee small. The service audits have ot addressed the mior shortfalls i performace. 58 The cotract maagemet team decided to susped the service audits i September Both Modus ad the Departmet state that they are determied to maitai ad improve performace stadards ad are workig together to itroduce a more effective audit methodology. This had ot bee implemeted at the time of our audit. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 61

64 appedix oe Case study 6 Defece Aimal Cetre (DAC) Eightee Dogs ad eightee horses paraded before they were marched off to their ew home at the Defece Aimal Cetre at Melto Mowbray. Fact Box Category Accommodatio Siged August 2000 Term 25 years Iitial service date Jauary 2002 Full service date Jauary 2002 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c 11.2 millio c 3.6 millio O balace sheet Realm Services (DAC) Ltd Redevelopmet of ew office ad residetial accommodatio, aimal husbadry ad traiig support The Defece Aimal Cetre (DAC) is a mixed military ad civilia agecy resposible for traiig ceremoial horses ad dogs for search ad security duties 59 The Defece Aimal Cetre is a mixed military ad civilia agecy resposible for traiig ceremoial horses ad dogs for search ad security duties. It also trais service persoel i equitatio, dog hadlig ad aimal welfare. Traiig of Military Workig Aimals ad associated service persoel is Defece Aimal Cetre s core busiess ad is udertake by the Army ad the Royal Air Force. 60 The Defece Aimal Cetre estate icorporates idoor ad outdoor caie ad equie traiig facilities, keels ad stablig, grazig lad, forge facilities, a veteriary hospital, offices, messes, leisure facilities ad sigle livig accommodatio. Util Jauary 2002, the headquarters facilities ad the livig accommodatio were woode buildigs costructed i 1936 ad iteded as temporary accommodatio. By the early 1990 s these facilities were i very poor repair requirig heavy maiteace expediture. I 1995, the Departmet decided to explore the PFI optio for private sector ivolvemet i the delivery of accommodatio ad services to the Defece Aimal Cetre. 61 The project required a ew headquarters facility comprisig office, messig ad sigle livig accommodatio, stores, motor trasport garagig ad leisure facilities to be costructed o Defece Aimal Cetre lad. The vacated headquarters site was to be demolished for the private sector parter to sell for redevelopmet. The services to be delivered covered all o-core activities except physical security duties which the Departmet would cotiue to provide. The Departmet would retai resposibility for the traiig of Miistry of Defece persoel ad aimals. 62 The competitive process started i late 1996 with a cotract beig siged i August 2000 with Realm Services (DAC) Ltd. However whe the cotract was siged it was cosidered to offer oly margial value for moey. The deal would be cosidered too small for a PFI project today (as its capital value is less tha 20 millio). It is geerally thought that the cost of procuremet ad settig up the deal structure outweighs ay potetial beefits to be gaied from such projects. The facilities were delivered 11 weeks late 63 Realm delivered the ew facilities 11 weeks after the Scheduled Commissioig Date for which it paid a performace deductio of 29,000. Realm also took over resposibility for the existig facilities ad equipmet. However they did ot agree with the Departmet the coditio of those assets to set the base lie to determie whether the asset was fit for purpose ad deemed available i the future. 62 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

65 appedix oe Aspects of the cotract are vague ad the paymet mechaism is ieffective at trasferrig risk to the cotractor 64 With respect to the support services the cotract specificatios were loosely drafted ad ope to iterpretatio. For example, the cleaig output specificatios for furiture ad fittigs are that they are to be free from dirt or dust which i the reasoable opiio of the moitor has bee i place for a uacceptable period of time. The requiremet is subjective ad difficult to measure ad the moitor is the cotractor. 65 The cotractor is resposible for measurig performace. It does this through geeratig a radom sample of Support Service Evets from aroud some 56,000 evets. The radom sample is 0.7 per cet of the total, which meas that the cotractor eeds to check some 390 support service evets o oe day each moth. As checks oly have to be made oce a moth oly a small umber of performace idicators are tested. I additio, certai key aspects of the service are ot subject to ispectio i the mothly moitorig process whilst the relevace of others is questioable. For example, oe measure is to esure that the saitary ware i the veteriary hospital X-ray viewig/developig room was regularly cleaed. There is o saitary ware i that room ad so the service is automatically deemed to have bee delivered. 67 I practice the Departmet accepted the deficiecies of the Ridig School ad made use of the facilities. However, by usig the facilities the Departmet loses its right to impose the pealties. Realm had made some attempt to repair the existig Idoor Ridig School facilities but it had bee deemed usuitable ad ot fit for purpose by the British Equie Associatio. I the evet the Departmet replaced the existig Idoor Ridig School with a much larger oe that meets moder traiig best practice ad safety eeds. The total cost of the ew facility to the Departmet was millio. Relatioships have historically bee poor ad the cotract weakesses eed to be addressed 68 At the Defece Aimal Cetre, there have bee major chages i the staffig both o the Departmet ad cotractor side. Combied with the problems i determiig the service levels this has cotributed to relatioships which have historically bee poor. However the Departmet have made some progress by carryig out a post project evaluatio of the project which was completed durig the course of our study. The NAO agrees with the Departmet s coclusio that the cotract eeds to be reegotiated. If agreemet caot be reached the Departmet should cosider the optio of termiatig the cotract. 66 I additio to the service deductios there are pealties for whe assets are ot deemed available. To date availability deductios have totalled c 14k. However the pealties are small, for example if oe of the Juior Raks Accommodatio uits failed to meet the availability criteria for a whole moth, the Realm would face a deductio of oly 88. More critically the pealty for ot makig the Ridig School available is oly c 800 per moth. This is disproportioate to the cost of providig a alterative facility. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 63

66 appedix oe Case study 7 Defece Fixed Telecommuicatios System (DFTS) Fact Box Category Siged July 1997 Term Other Telecommuicatios 15 years (Origially 10 years, exteded i April 2005 for 5 additioal years util 2012) Iitial service date December 1997 Full service date July 2000 Capital value Aual Uitary Charge (07/08) Balace sheet treatmet Cotractor Objective c millio c millio at 07/08 prices Off balace sheet BT Provisio of secure ad survivable Wide Area Network voice, data ad video telecommuicatio services across the Miistry of Defece to users based i the Uited Kigdom ad Overseas The mai aim of the cotract was to improve efficiecy 69 The Defece Fixed Telecommuicatios System cotract with BT was iitially siged for 10 years i July This was exteded by five years i April 2005, securig the services util July The cotract with BT is maaged by the Defece Fixed Networks (DFN) Itegrated Project Team. The project is based i Corsham i Wiltshire. 70 The mai aim of the cotract was to ratioalise ad improve the efficiecy of existig fixed telecommuicatios services, to esure cotiued provisio of fixed telecommuicatios services ad to deliver fiacial savigs of aroud 30 millio a year. A review i November 2002 idetified further estimated savigs of 400 millio over the life of the project. The scope of the project has grow cosiderably 71 The Defece Fixed Telecommuicatios System is a fully maaged telecommuicatios service. It is desiged to serve 200,000 plus telephoe users, makig over 2.5 millio telephoe calls daily, across 2,445 UK sites. A rage of maaged facilities icludes voice, with data switchig, wide area etwork coectivity, iteret ad itraet, video coferecig ad ISDN. Mobile commuicatio services providers (Vodafoe) have also bee brought ito the Defece Fixed Telecommuicatios System cotract to explore the beefits from emergig mobile capabilities. 72 The services provided uder the cotract are vital to the daily operatios of the Miistry of Defece ad the armed forces. The cotract chage rate is high ad the cotract has grow cosiderably. The service was obtaied at a good price ad a reegotiatio ad bechmarkig has geerated beefits to the Departmet 73 The Defece Fixed Telecommuicatios System cotract was the subject of a NAO report (HC 328) i The report cocluded geerally that the Departmet obtaied the cotract at a good price although the scope of the project may ot have maximised value for moey. This is because there are iterrelatioships betwee the Departmet s various commuicatio systems, ad rapidly chagig techology requires fast ad frequet reassessmet of the most effective form of service delivery. 74 Havig decided to procure a ew fixed telecommuicatios system, the Departmet did ot assess the potetial advatages ad disadvatages of expadig or reducig the scope of the project. I additio, although the Departmet were geerally protected by the cotract, they could have made better use of exteral advice. 75 The cotract allows for ew services ad techology to be icorporated durig the cotract period ad prices for services are adjusted periodically i lie with movemets i agreed price idices. But the origial cotract mechaism limited the opportuities for the Departmet to challege the service prices. I fact the Departmet have oly made oe challege to date o the Restricted Local Area Network Itercoect (RLI) Service which was costig the Departmet 40 millio per aum. 76 The bechmarkig exercise idetified that the cost was 30 to 40 per cet above the expected cost. BT iitially rejected the proposed decrease. However, i 2003, after two years of egotiatio, a agreemet was reached with a price reductio i exchage for extra service growth to be supplied by BT. A savig of 37.3 per cet of the aual cost of the RLI service, represetig 15 millio per 64 Allocatio ad maagemet of risk i Miistry of Defece PFI projects

67 appedix oe 24 The actual reveue geerated by BT o the Defece Fixed Telecommuicatios System cotract has icreased Reveue ( millios) Source: Miistry of Defece aum, was achieved. The Defece Fixed Telecommuicatios System bechmarkig was examied i more detail i the NAO Report: Bechmarkig ad market testig the ogoig services compoet of PFI projects (HC 453) which was published i Jue Durig the cotract extesio discussios the Departmet took the opportuity to chage the cotract terms so that they were more favourable to them. They also secured additioal savigs of c 200 millio ad improved the visibility of the costs ad reveues that BT expects to make through the provisio of a cotractual fiacial model. The Departmet also icluded additioal measures for the parties to examie the prices for services if it moved beyod agreed thresholds. Risk maagemet is geerally good but the project has experieced a sigificat lapse i service performace reportig 78 The cotract is workig well, with the busiess objectives of the Departmet ad BT aliged. As previously stated i our earlier report o this project, BT will be i a strog positio to wi future cotracts ad extesios to Defece Fixed Telecommuicatios System due to its experiece o the project ad the relatioship that has bee established. 79 A sophisticated joit risk maagemet approach is i place ad risk maagemet is itegrated ito the cotract. However, oe of the major risks idetified by the project team was the issue of skills ad staff cotiuity. I particular, it was oted that there were difficulties i retaiig specialist staff i post due to limited opportuities for promotio withi post. I additio, there was reliace by the Departmet o full time exteral cosultats. A umber of these have bee i post sice the cotract was procured i 1997 ad have provided essetial cotiuity of kowledge ad expertise. 80 Recetly, a sigificat lapse i service performace reportig occurred, whe it traspired that staff i call cetres operated uder the cotract were artificially iflatig call umbers i order to meet targets for successfully completed calls. This matter was subject to ivestigatio by the Miistry of Defece Police ad has ow bee cocluded. BT has made retrospective service credit paymets to the Departmet of 1,021,000 ad has reimbursed the Departmet for the exteral cost icurred by the ivestigatio totallig 122,000 excludig VAT ad for the value of the overstated calls totallig 197,000 excludig VAT. The maagemet of this project has bee chaged followig the lapse i service performace reportig 81 Followig the fraud the Departmet ad BT have imposed a ew maagemet structure to the Defece Fixed Telecommuicatios project icludig ew goverace arragemets. BT is ow required to provide more detailed reportig ad the Departmet will carry out regular detailed audits of the ew BT reportig system to esure its itegrity. This will be supplemeted by regular service audits by the Departmet. Certai BT staff ivolved i the activities of Ketterig call cetre have bee released by BT. 82 Despite this icidet, the Departmet s team ruig the cotract are cofidet that the curret good workig relatioships ca be maitaied ad that it has bee resolved i a professioal maer by seior staff. Allocatio ad maagemet of risk i Miistry of Defece PFI projects 65

68 appedix oe Case study 8 Tidworth Water & Sewerage Views of the Tidworth Sewerage works. Fact Box Category Other Utilities Siged February 1998 Term 20 years plus the optio to exted 5 years Iitial service date September 1998 Full service date September 1998 Capital value Aual Uitary Charge ( ) Balace sheet treatmet Cotractor Objective Nil there were o ew major capital works c 0.7 millio Off balace sheet Thames Water Provisio of water ad sewerage services i Tidworth Garriso The Departmet eeded to trasfer the service provisio to the private sector 83 The Tidworth Water ad Sewerage cotract provides services to the Tidworth Garriso icludig civilia residets of the tow. The system cosisted of departmetally owed ad operated water ad sewage systems. By 1993, the system was ureliable ad i poor repair. At this time the Departmet was losig crow immuity ad the facilities would ot have bee legally compliat. 84 Prior to the iitiatio of the project, the Property Maager s techical cosultats advised that capital works to the value of aroud 7 millio would be required to make it compliat. This would be achieved by closig oe of the two sewage treatmet works, rebuildig the other ad istallig a pumpig mai to lik the sites. At this stage a project team was established to cosider all the optios ope. The scope was wideed to iclude the sewage treatmet works ad also sewers, water abstractio, treatmet, distributio ad billig of o MOD customers. This optio was used as the Public Sector Comparator, retaiig the facilities withi MOD owership. This was a live optio. Negotiatios with the Preferred Bidder failed ad the Departmet called o the reserve bidder 85 The project was advertised i May Wessex Water was selected as preferred bidder i July However, durig the preferred bidder egotiatios agreemet could ot be reached o the trasfer of risk, particularly aroud the extet of cotractor liabilities ad potetial paymets to the cotractor i the evet of breach or termiatio. As a result, the Departmet termiated its egotiatios with Wessex ad brought i the reserve bidder, Thames Water. Agreemet was reached with Thames i March The Departmet retaied some risks 86 It was agreed that the Departmet would retai risks for treatig poly-aromatic hydrocarbos origiatig from the liigs of the old MOD water mais. EU legislatio was subsequetly chaged ad the particular substace foud at Tidworth was removed from the stadard followig the evidece that there were o health effects at the levels foud i drikig water. Therefore the Departmet icurred o additioal costs with respect to this risk. 66 Allocatio ad maagemet of risk i Miistry of Defece PFI projects