A METHOD FOR ESTIMATING CONTINGENCY BASED ON PROJECT COMPLEXITY. Jucun Liu. The Department of Civil and Environmental Engineering.

Transcription

1 A METHOD FOR ESTIMATING CONTINGENCY BASED ON PROJECT COMPLEXITY Master s Thesis by Jucun Liu to The Department of Civil and Environmental Engineering In partial fulfillment of the requirements for the degree of Master of Science in Civil Engineering In the field of Construction Management Northeastern University Boston, Massachusetts April, 2015

2 ii ABSTRACT Accurate cost estimates are important in every construction project for owners to prepare their budgets and construction plans. In transit projects, estimators for construction projects make estimates from historic data on every detailed level of construction needs like rails, ties and vehicles and then sum up these costs to get the final estimates for projects. However, transit projects usually experience cost overrun and budgets are rarely sufficient. This paper proposes a methodology to enhance estimates for transit projects and analyze the cause of general lack of accuracy in cost estimates. By analyzing the fundamental background of transit project phases, some of the reasons for insufficient estimates are identified. Then by analyzing the actual cost data from transit projects provided by the TCRP Final-G07 report, a new methodology is developed to try to help estimators to make better judgments. Federal Transit Administration s Standard Cost Categories (SCC) for Capital Projects, divides a transit project s cost into 10 categories. Previous research has shown that the cost for the details in each category (rails, tracks and ties etc.) follows a lognormal distribution. For each detailed estimate, it is suggested that the estimator provides the most likely value (mode) for the distribution as the cost estimate. In other words, the estimate is the mode of that particular lognormal distribution. However, in reality, the average cost of the whole project cost is the mean value of the cost distribution. Since the mode and mean of a lognormal distribution is not the same, a difference occurs. By performing mathematical analysis on the normal distribution for the whole project cost and lognormal distribution of the detailed estimates, a new methodology is

3 iii developed. The results are verified using actual project costs provided by the TCRP Final-G07 report and the FTA-2007 report.

4 iv TABLE OF CONTENTS LIST OF TABLES... vi LIST OF FIGURES... viii ABSTRACT... ii Introduction Source of Error New Method to Develop More Accurate Contingencies Background Information and Assumptions Lognormal distribution Coefficient of variation Central limit theorem TCRP-G07 Report Assumptions Theories and formula Complexity of a Project Breakdown Analysis Example of Breakdown Analysis Confidence Level Inverse Normal Distribution Method Improve Rating Criteria Validation FTA-2007 Report Final-G Determination of c.o.v values Summary Suggestions for Future Work... 85

5 v APPENDIX A: Results of Breakdown Analysis APPENDIX B: Results for Contingency Percentages REFERENCE:... 95

6 vi LIST OF TABLES Table 1: Scores for 28 projects Table 2: Total score of 28 projects Table 3: c.o.v of 28 projects Table 4: Delay and Overrun of projects Table 5: Average delay and overrun for different c.o.vs Table 6: Score and c.o.v for 14 projects in analysis Table 7: Score and c.o.v for 14 projects in analysis Table 8: Breakdown Analysis for Tren Urbano Table 9: Systematic errors Table 10: Systematic errors Table 11: Corresponding values of probabilities Table 12: Contingency percentages Table 13: Contingency percentages Table 14: Scores for different criteria Table 15: New scores after one reduction (seven criteria) Table 16: Final score (five criteria) Table 17: Multi-linear regression result Table 18: Old standard (with nine criteria) Table 19: New standard (with five criteria) Table 20: Rating of projects in FTA Table 21: Contingencies for 75% confidence level Table 22: FFGA and Base value for projects Table 23: New estimates and reported costs Table 24: Percentages of projects having no overrun Table 25: Percentages of projects having no overrun for different confidence levels. 69

7 vii Table 26: Ratings of projects in Final G-07 report Table 27: Scores and c.o.vs for projects in Final G Table 28: FFGA and base values for projects in Final G Table 29: Contingencies, final estimates and reported costs Table 30: Percentages of projects having no overrun Table 31: Contingencies for the first set of c.o.vs Table 32: Errors for c.o.v of 0.05, 0.15 and Table 33: Errors for c.o.v of 0.1, 0.2 and Table 34: Errors for c.o.v of 0.1, 0.25 and Table 35: Errors for c.o.v of 0.05, 0.25 and Table 36: Errors for c.o.v of 0.1, 0.2 and Table 37: Breakdown analysis result for c.o.v of Table 38: Breakdown analysis result for c.o.v of Table 39: Corresponding values of different probabilities for c.o.v of Table 40: Corresponding values of different probabilities for c.o.v of

8 viii LIST OF FIGURES Figure 1: Assigned c.o.v vs. delay Figure 2: Assigned c.o.v vs. overrun Figure 3: Score vs. delay for the 28 transit projects Figure 4: Score vs. overrun or the 28 transit projects Figure 5: Contingency percentages for c.o.v of Figure 6: Contingency percentages for c.o.v of Figure 7: Contingency percentages for c.o.v of Figure 8: Score vs. delay after combination Figure 9: Score vs. overrun after combination Figure 10: Score vs. overrun after one reduction (seven criteria) Figure 11: Final score vs. delay (five criteria) Figure 12: Final score vs. overrun (five criteria) Figure 13: Confidence level vs. percentages of projects having no overrun... 71

9 1 Introduction Transit cost categories: Cost estimating is essential for construction projects in the bidding and design stage of a project. The most common way estimators prepare a detailed cost estimate is to estimate costs using a bottom-up approach. From FTA s Standard Cost Categories (SCC) for Capital Projects, a transit project s cost is broken down into 10 categories: SCC10: Guideway and track elements, SCC20: Stations, stops, terminals and intermodal, SCC30: Support facilities: Yards, shops, admin, BLDGS, SCC40: Sitework and special condition, SCC50: Systems, SCC60: Row, land, existing improvements, SCC70: Vehicles, SCC80: Professional services, SCC90: Unallocated contingencies and SCC100: Financial charges (Ye Zhang 2014). These 10 categories can be further broken down into more detailed subsections such as earthwork, steel, reinforced concrete, etc. Estimators usually provide estimates for these detailed components and then sum these estimates up to get the total estimate for the whole project. Funding process for transit projects: There are various ways of funding transit projects from federal, state, local and private sources. The most common way of funding transit infrastructure projects is primarily based on a combination of state and local taxes, while for major projects, federal funding also plays an important role. The federal aid usually funds projects on a pay-as-you-go basis, meaning that projects have been built in phases or increments as funds become available over a period of time. Private funding is labeled innovative because of the involvement of private sectors in developing,

10 2 constructing and operating transportation facilities. A contract between a private sector and a public agency will be signed forming a partnership which allows more participation of private sectors in transit projects. This arrangement is called Public- Private Partnership (PPP) and its use is on the rise given the limitation of funds for transportation projects in the United States. Simply put, funding process involves initiation of a state transit agency proposing a project which will be examined by federal government (Federal Transit Administration (FTA)) to see if it is worth funding. If the project is approved, it will be eligible for federal financial support. If a project is hoping to get funded, its design, scope, purpose and a primary cost estimate should be provided to the FTA to examine its feasibility and this is where the importance of cost estimate comes in. (Diekers and Mattingly 2009) The traditional way the estimators used can be explained as following. When faced with a new transit project, estimators will examine what items are needed. For every detailed component, such as procurement of steel, concrete and plywood, estimators usually keep a record of previous prices. Then from these historic records, estimators normally choose a value for use in the cost estimate. It is the position of this thesis that the cost estimates used by the estimators are the most likely costs (modes). After obtaining all the modes for every component, estimators sum up these modes and apply contingencies and financial charges to the final value to get the final estimate. Given the uncertainty of costs, it can be assumed that cost components are random variables following certain distributions. Past research shows that the cost for a detailed component in construction costs follows a lognormal distribution (Touran

11 3 and Wiser 1992, Moret 2011). Assuming this to be true, the estimator, without knowing about the underlying distribution, is using the mode of the distribution to come up with its total cost estimate. Most transit projects experience cost overruns. In the Final-G07 report by Booz.Allen (2005), out of 28 projects, 21 projects experienced cost overrun. This phenomenon indicates that the traditional estimating methods usually underestimate the cost needed for a project.

12 4 1. Source of Error As was discussed before, cost overruns in transit projects are commonplace. There are several reasons for this but in this thesis we concentrate on a fundamental error that is the result of the estimating approach. This phenomenon can be explained by the central limit theorem and the nature of transit projects. Normally, the engineering-related parts of a transit project can be divided into 8 Standard Cost Categories from SCC10 to SCC80 as presented previously. These 8 categories contain numerous sub-phases as well and these sub-phases may contain even more sub-components. For example, SCC10-Tracks may include purchasing steel, which can be divided into purchasing rebar, pre-stressed steel or post-stressed steel. The lognormal assumption is only valid when the most detailed category is concerned. For example, when talking about the procurement of rebar alone, the cost of the purchase itself can be considered as following a lognormal distribution. At each detailed stage, estimators normally choose the most common value for each item which is referred to as the mode of the lognormal distribution. However, the total estimate will be the sum of each detailed stage and when talking about the sum of distributions, the lognormal assumption will no longer hold. Then a question should be raised: what will the sum of independent random variables obtained from a number of lognormal distributions follow. Several studies have been conducted trying to approximate the sum of random variables obtained from independent lognormal distributions. One assumption is that the sum of independent lognormal random variables should also follow a lognormal distribution.

13 5 Beaulieu and Xie (2004) tried to develop a new method to approximate the sum of independent lognormal random variables in their paper. There were three traditional ways to approximate the sum of independent lognormal random variables methods by Schwartz and Yeh (1982), Wilkinson (1960), and Farley (1982). The method by Schwartz and Yeh, and method by Wilkinson are both based on the assumption that the sum of independent lognormal random variables follows a lognormal distribution. The paper by Beaulieu and Xie presented a simpler and in some sense optimal approach to approximate the sum of lognormal distributions. They approximate the sum of lognormal distributions using a transformation that linearizes a lognormal distribution and then deriving the minimax linear approximation in the transformed domain. In order to support their investigation, Beaulieu and Xie compared their new method with traditional methods developed by Schwartz and Yeh, Wilkinson, and Farley. After comparison in regard of sums of i.i.d. lognormal distributions, Beaulieu and Xie found that Schwartz and Yeh s approximation performed poorly on the tail region and Wilkinson s approximation was even worse at tail region. The approach developed by Beaulieu and Xie does reduce errors in tails but still cannot work well with sum of random variables from large number of lognormal distributions. It was found that this assumption performs well for sums of N=2 i.i.d. summands, but is poor when the number of summands increases. Farley s approximation is better in general for large values of arguments, but worse than other methods for smaller values of arguments.

14 6 Results in Beaulieu and Xie s analysis show that their minimax approach reduces the relative error in the tails of the approximating distribution and their approach can be better than other approximations in some applications. In the process of developing their minimax approach, Beaulieu and Xie also examined the validity of the assumption that the sum of lognormal random variables follows a lognormal distribution. Their result shows that this assumption works well for sum of N=2 i.i.d. summands, but is poor when the number of summands increase. Other papers have also tried to approximate the sum of random variables from independent lognormal distributions. Mehta, et al (2007) proposed a general method that uses MGF (Moment Generating Function) as a tool to approximate the sum of lognormal distributions. The simulation result also shows that if random variables from a limited number of independent lognormal distributions are added, the sum can be approximated by a lognormal distribution, but this lognormal approximation cannot hold when the number of lognormal distributions increase. However, in construction projects, the number of items that require cost estimate usually exceeds the number of 2. In most cases, transit projects can contain hundreds of thousands of items that follow lognormal distributions. Under these circumstances, the assumption that the sum of random variables obtained from independent lognormal distributions will remain lognormal should not be appropriate anymore. Another theorem that can be applied is the Central Limit Theorem, which states that the sum of several random variables from independent distributions can be well-

15 7 approximated by a normal distribution, when the number of random variables is large. This theorem is more legitimate for the purpose of this thesis because as mentioned above, transit projects normally contain a large number of items, which can be considered as random variables from independent lognormal distributions. Though this approximation may not be as accurate as the approximations performed in previous papers specifically for lognormal distributions when the number of summands is small, this is the most appropriate assumption that can be made in this regard. According to central limit theorem, when summing random variables from several lognormal distributions together, the summation will become a normal distribution, assuming independence between distributions. During the transition process, the mode of a lognormal distribution will become the mode of a combined normal distribution. When summing distributions together, the mean after summation will simply be the summation of means of each lognormal distribution. However, the same cannot be applied to the value of mode. Therefore as the summation of distributions proceeds, the mode of the final normal distribution will not be equal to the sum of modes of those original lognormal distributions. The total estimate for a project should be the mode of a statistical distribution for sure. However, the mode is no longer the sum of modes of lognormal distributions but the mode of a normal distribution. Furthermore the estimate is equal to the mean of that normal distribution because mode and mean are the same value in a normal distribution. With the knowledge that the sum of means of several distributions is equal to the mean of the

16 8 combined normal distribution, it can be concluded that the mean of that normal distribution, which is the final estimate, is calculated by summing up all the means of those lognormal distributions. Errors occur in this process as there is difference between mode and mean of a lognormal distribution. Quantified verification will be presented in the Breakdown Analysis chapter (Chapter 5) later.

17 9 2. New Method to Develop More Accurate Contingencies Once realizing that the old method has its flaws, a new method to provide more accurate estimates is developed in this thesis. From the previous chapter, it is known that the errors come from the difference between the sum of expected values for the project as a whole and the sum of individual modes for detailed components. In order to eliminate this difference, originally derived estimates from summing all modes need to be transformed into the sum of their expected values when all the components are added together. The whole project cost will be considered following a normal distribution and the total expected value is its mean. This transformation process is referred to as Breakdown Analysis and will be the foundation of this thesis. It is called Breakdown Analysis because in the process, costs from SCC10 to SCC80 will be broken down into sub-phases and transformation will be done on each sub-phase. With this method, the theoretical error in the old method will be eliminated and with the help of the assumption of normal distribution, contingencies needed for various confidence levels can also be derived. This analysis will be explained more thoroughly in later chapters.

18 10 3. Background Information and Assumptions Lognormal distribution In order to perform breakdown analysis on the project cost, some assumptions and background knowledge are required. For instance, construction activity durations generally are assumed to follow beta distributions and method of time estimation has already been developed and well recognized in PERT. (Cook 1966) It has been shown that cost of various components in a building project follows a lognormal distribution (Touran and Wiser 1992, Moret 2011). Lognormal is a skewed distribution that provides for a longer tail on the right side and allows a more accurate modeling of construction costs which are non-negative, typically bounded at the low side and less bounded on the high side. Lognormal distribution is the distribution of a set of random variables whose logarithms follow a normal distribution. The mean, mode and variance of a lognormal distribution are: Mode=e μ σ2 (1) E(x) =e μ+σ2 2 (2) Var(x) =(e σ2 1)e 2μ+σ2 (3) Where μ is the mean of the underlying normal distribution and σ is the standard deviation of that normal distribution. With these parameters known, a statistical analysis can be done to find a better way to enhance estimates.

19 11 Coefficient of variation This analysis also makes use of the Coefficient of Variation (referred to as c.o.v in the rest of the thesis). This parameter is obtained by dividing the standard deviation of a distribution by its mean. Coefficient of variation represents the level of uncertainty in the cost of a component and might be an indication of complexity of a project. Here the assumption is the more complex the project, the harder to estimate the costs accurately. This parameter will play an important role in the analysis and the reason is demonstrated in later chapters. Central limit theorem According to the Central Limit Theorem, if Sn is the sum of n mutually independent random variables, then the distribution function of Sn is wellapproximated by a certain type of continuous distribution known as Normal Distribution (Grinstead, Snell 2003). In this project, the cost of the each detailed subsection follows a lognormal distribution. The traditional way of estimating is to sum these costs of all the detailed subsections. Therefore after summing these independent costs together, the whole project follows a normal distribution by Central Limit Theorem. This is useful when making confidence estimates. TCRP-G07 Report TCRP-G07 Report (Final G-07 report), Managing Capital Costs of Major Federally Funded Public Transportation Projects, was published by Transit Cooperative Research Program in This report contains recommendations for strategies, tools, and techniques to better manage major transit capital projects over

20 12 $100 million. It also presents estimates and final as-built costs for 28 transit projects. This report will be used extensively in this thesis. Assumptions Before the analysis, some assumptions should also be made. The first assumption is that, since projects often last a rather long period of time, the time value of money, inflation index and other factors affecting value of money will change. In this analysis we transfer all the money to the same value of money in a certain year to simplify the analysis. Then the analysis makes use of c.o.v. in order to capture the uncertainty in cost. In order to do this, a complexity rating is assigned to different projects. However, this complexity rating process was designed by taking the criteria described in the limited project descriptions provided in the G7 Report and may not be 100% accurate. After establishing the complexity level of the projects, μ and σ are calculated making use of the estimates provided by the TCRP-G07 report. However, the report provides 3 estimates according to three different phases of the projects. In this paper, only the estimates prepared after the final design phase are used. Theories and formula From lognormal distribution it can be concluded that: The sum of modes = modes = e μ i σ i 2 (4) The sum of means = μt = sum of E(x) = E(x) = e μ i+ σ i 2 2 (5)

21 13 Then the difference between sum of modes and μt: Δ = μt- modes = e μ i+ σ i e μ i σ i 2. (6) After simplification, the following formula can be obtained: 3σ2 i Δ= e μ i (e 2 1) e σ i 2 (7) From Equations (2), (3) and (4), σ 2 and μ can be obtained. σ 2 =ln[1 + Var(x)/E(x)2] = ln[1 + c. o. v 2 (x)] (8) μ=ln(mode) + σ 2 (9)

22 14 4. Complexity of a Project As mentioned previously, coefficient of variation (c.o.v) is crucial to this analysis and c.o.v is a representation of a project s cost uncertainty. At least part of cost uncertainty (or the inaccuracy of cost estimates) can be attributed to the complexity of the project. Complexity of a project can originate from several aspects. Everything from scope to construction can influence a project s complexity. Gidado (1996) suggests that there seems to be two perspectives on project complexity in the industry: the managerial aspect, which involves the planning of bringing together numerous parts of work to form work flow; and the operative and technical aspect. Wood and Ashton (2010) suggest that high number of trades involved and long timescale will increase projects complexities. Creedy (2005) found that change in design is an important factor for cost escalation. The Fulton Street project report prepared by Timo Hartmann et al. (2007) discussed factors which can increase the subway project complexity. In the report, existing heavy traffic, having large portions of construction underground, public concern, tight site conditions, and multiple contractors all make the project complex. In this thesis, 9 criteria were selected to represent a project s complexity because we think these criteria most properly summarize how complex a project is. The Final G-07 report contains several possible causes that may increase a project s cost. From these causes and factors mentioned in the descriptions of those 28 projects in the report, 9 criteria were chosen to represent a project s complexity. These complexity

23 15 factors were decided after the projects were completed, so they had the benefit of the hindsight. These criteria are listed below: 1. Project Change: This criterion includes change of scope usually initiated by owners and change orders initiated by contractors during the construction phase. It adds uncertainty and generally increases cost and time for a project. 2. Unforeseen Site Conditions: Unforeseen site conditions are one of the most frequent causes for contractor s claims and always add difficulty to the project construction with potential to increase costs. 3. Duration of project: A longer project often is affected by inflation and this makes a long project unpredictable in terms of cost. If a project lasts more than 6 years (assumed value), it will be considered a long-duration project and scored accordingly. 4. Third Party Factors: A public transit project is often funded by public agencies or government so the opinions and interference of the public are quite common. The other case is that during the construction, protests against construction work may also happen which would require changes to design and scope. These all can add to project complexity. 5. Heavy Traffic: If heavy traffic is a common phenomenon around the construction site, it can add great time and cost to a project. Traffic detours and their costs have always been difficult to estimate.

24 16 6. Multiple Contracts: When a single project is divided into several packages and bid with different prime contracts, it would be natural that it requires more time and money to coordinate. 7. Underground Work/Complexity of Stations: Underground work constantly adds uncertainty to a project for its lack of accurate site prediction and characteristics. It can be considered as the most unpredictable elements in a project. Though a common feature in transit projects, stations can also be complex because of different design requirements, site conditions and aesthetic considerations. They are lumped together with underground work because in most transit projects, at least some stations are located underground. 8. Utility Relocation: In large projects, especially in older urban locations, information about the location of utility lines is sketchy at best. This makes estimating the cost of utility relocation subject to large uncertainties. 9. Elevated Structure: Elevated structures are in general more complex compared to at-grade line construction. After the above criteria were selected, a rating system based on numerical scores was created. Each project s description was then reviewed and depending on the complexity a score of 0 or 1 was assigned for each of the parameters. Therefore, the total score a project will have based on how many features are described in the project description, can vary between 0 and 9. This process was performed on all the projects reported in G7 Report and the result is presented in Table 1 below:

25 17 Project change Unforeseen site condition Duration of Project Table 1: Scores for 28 projects. Third party Heavy factors traffic Multiple Contracts Underground work/complexity of stations Utility relocation Elevated structure Atlanta North-line extension Boston old colony rehabilitation Boston silver line phase Chicago southwest extension Dallas south oak cliff extension Denver southwest line Los Angeles red line (MOS 1) Los Angeles red line (MOS 2) Los Angeles red line (MOS 3)

26 18 Minneapolis Hiawatha line New Jersey Hudson- Bergen MOS New York 63rd street connector Pasadena gold line Pittsburgh airport busway phase Portland airport MAX extension Portland Banfield corridor Portland interstate MAX Portland Westside/Hillsboro MAX Salt lake north-south line 1 0 San Francisco SFO airport line 1 1 1

27 19 San Juan Tren Urbano Santa Clara Capitol line Santa Clara Tasman east line Santa Clara Tasman west line Santa Clara Vasona line Seattle busway tunnel St. Louis-St. Clair corridor Washington Largo extension

28 20 The total score for each project is presented in Table 2 (sorted according to score): Name Table 2: Total score of 28 projects. Total Salt Lake North-south Line 1 Portland Interstate MAX 2 Denver Southwest Line 3 San Francisco SFO Airport Line 3 Santa Clara Capitol Line 3 Dallas South Oak Cliff Extension 4 Minneapolis Hiawatha Line 4 Boston Old Colony Rehabilitation 4 New Jersey Hudson-Bergen MOS 1 4 Portland Airport MAX Extension 4 Portland Banfield Corridor 4 Portland Westside/Hillsboro MAX 4 Santa Clara Tasman East Line 4 Santa Clara Tasman West Line 4 Seattle Busway Tunnel 4 St. Louis-St. Clair Corridor 4 Washington Largo Extension 4 Atlanta North-line extension 5 Los Angeles Red Line (MOS 2) 5 Los Angeles Red Line (MOS 3) 5

29 21 New York 63rd Street Connector 5 Pasadena Gold Line 5 Pittsburgh Airport Busway Phase 1 5 San Juan Tren Urbano 5 Santa Clara Vasona Line 5 Boston Silver Line Phase 1 6 Chicago Southwest Extension 6 Los Angeles red line (MOS 1) 6 Then, these projects were categorized into 3 classes with different c.o.vs. C.o.v of 0.1, 0.2 and 0.4 are selected. It is considered that a project with score lower than 3 will have a c.o.v of 0.1 since its complexity is relatively low. A project with score of 5 or greater is considered to be having a c.o.v of 0.4 because this project is affected by too many complexity parameters. Projects with a score of 3 or 4 fall into the class with c.o.v of 0.2. The reason these three values are used are explained in Validation Chapter. The result is as shown in Table 3: Table 3: c.o.v of 28 projects Salt Lake North-south Line 0.1 Portland Interstate MAX 0.1 Denver Southwest Line 0.2 San Francisco SFO Airport Line 0.2 Santa Clara Capitol Line 0.2 Dallas South Oak Cliff Extension 0.2

30 22 Minneapolis Hiawatha Line 0.2 Boston Old Colony Rehabilitation 0.2 New Jersey Hudson-Bergen MOS Portland Airport MAX Extension 0.2 Portland Banfield Corridor 0.2 Portland Westside/Hillsboro MAX 0.2 Santa Clara Tasman East Line 0.2 Santa Clara Tasman West Line 0.2 Seattle Busway Tunnel 0.2 St. Louis-St. Clair Corridor 0.2 Washington Largo Extension 0.2 Atlanta North-line extension 0.4 Los Angeles Red Line (MOS 2) 0.4 Los Angeles Red Line (MOS 3) 0.4 New York 63rd Street Connector 0.4 Pasadena Gold Line 0.4 Pittsburgh Airport Busway Phase San Juan Tren Urbano 0.4 Santa Clara Vasona Line 0.4 Boston Silver Line Phase Chicago Southwest Extension 0.4 Los Angeles red line (MOS 1) 0.4

31 23 In order to verify the validity of this categorization process, each project s delay and overrun at the end of final design are also obtained from the Final G-7 report. Table 4: Delay and Overrun of projects Name Delay Overrun Atlanta North-line extension 9.50% 24.00% Boston old colony rehabilitation 0.20% 2.40% Boston silver line phase % 46% Chicago southwest extension 5.90% 4.80% Dallas south oak cliff extension -5.30% 35% Denver southwest line % Los Angeles red line (MOS 1) 8.80% 55.20% Los Angeles red line (MOS 2) 9.60% 26.00% Los Angeles red line (MOS 3) 2.60% 8.80% Minneapolis Hiawatha line 3.70% 6.00% New Jersey Hudson-Bergen MOS % 13.90% New York 63rd street connector -0.20% -2.00% Pasadena gold line -0.20% -2.30%

32 24 Pittsburgh airport busway phase % 0% Portland airport MAX extension 0% 1.60% Portland Banfield corridor -2.40% % Portland interstate MAX 0% 11.00% Portland Westside/Hillsboro MAX 1% 5.90% Salt lake north-south line -2.30% 0.00% San Francisco SFO airport line 8.90% 32.80% San Juan Tren Urbano 12.50% 80.00% Santa Clara Capitol line 0% 1.70% Santa Clara Tasman east line 0.10% 0.00% Santa Clara Tasman west line -1.80% % Santa Clara Vasona line 2% 1.00% Seattle busway tunnel 5.60% 67.20% St. Louis-St. Clair corridor 0.00% -0.70% Washington Largo extension 0.10% 5.10% Then plot diagrams are drawn for Delay vs. c.o.v and Overrun vs. c.o.v.

33 overrun Delay % c.o.v vs. delay 20.00% 15.00% 10.00% 5.00% 0.00% -5.00% % y = x R² = c.o.v Figure 1: Assigned c.o.v vs. delay % c.o.v vs. overrun 80.00% 60.00% 40.00% 20.00% 0.00% % % y = x R² = c.o.v Figure 2: Assigned c.o.v vs. overrun Graphs for scores vs. Delay and scores vs. Overrun are also drawn.

34 overrun delay % Score vs. Delay 20.00% 15.00% 10.00% 5.00% 0.00% -5.00% % R² = score Figure 3: Score vs. delay for the 28 transit projects % 80.00% 60.00% 40.00% 20.00% 0.00% % % Score vs. Overrun R² = Score Figure 4: Score vs. overrun or the 28 transit projects All graphs show upward trends which means that delay and overrun increase as c.o.v or score increases. This makes sense since when a project is complex, it is likely to be

35 27 more difficult to estimate the cost and duration. Besides, average delay and overrun for each c.o.v are also calculated and presented in Table 5: Table 5: Average delay and overrun for different c.o.vs c.o.v Delay -1.15% 1.16% 7.15% Overrun 7.10% 30.04% 49.42% The results also show an upward trend as c.o.v increases which complies with the conclusion observed in the graphs making this categorization process reasonable. 5. Breakdown Analysis In order to eliminate the theoretical error discussed in chapter 1: Source of Error, a breakdown analysis is performed on each project. Breakdown Analysis involves the following steps: 1. Rate projects according to pre-set criteria; 2. Assign coefficient of variation to projects based on their scores; 3. Perform the transformation process described. Since the error actually occurs at the very detailed stage and the error is the difference between mean and mode, transforming the mode to mean from the detailed stage is considered. Project rating system needs to be applied, so data from the Final-G07 report is used for illustration. Besides, cost data from another FTA project cost database also needs to

36 28 be used since it contains detailed cost of each phase of a project. Projects in both of these databases are selected to do the breakdown analysis. FTA Transit Database: The FTA Cost Database contains costs for both LRT and HRT projects. These costs are allocated from SCC10 to SCC LRT projects and 30 HRT projects are included in this database. Equation (5), (8) and (9) from the previous chapter are used. = Mean mode (10) With these equations, as long as mode and coefficient of variation are known, the expected value of that item can be calculated. Finally the difference between original estimate and final cost can be calculated. However, the FTA cost database is not detailed enough. It contains categories of SCC10 to SCC80 but it does not provide costs for the sub-components of these categories. In other words, the costs of categories of SCC10 to SCC80 are already sums of several distributions and the components of these categories are nowhere to be known. If final changes as the constitution of components changes, the s of the detailed stages cannot be replaced by the of the categories as a whole thus making the analysis inaccurate. Therefore, the assumption that for the same categories, as long as the sum of modes remains the same, remains the same even though the composition of components changes needs to be verified.

37 29 Composition of components involves the number of components and the cost of each component. Different compositions of components can have different numbers of components or different costs of components. Since this proof aims at verifying that compositions of components will not affect the value of final, let mi be the modes of the components in the first composition so that the sum of modes for the first composition of components is m i. Let m j be the modes of components in a different composition where the sum of modes is m j. As discussed in the previous paragraph, we are verifying as long as the sum of modes remain the same, remains the same even composition of components changes. Therefore, m j and m i should be equal. These two compositions have the same total value of modes but vary in components or the cost of components. For example, the first composition may have 10 components, while the second composition has 20 components; or the first composition can have $100 as the cost of each component while the second composition has component cost of $50. Then their i and j are calculated to see if they are different. Coefficient of variation is the same since it is the same cost category. For mi, σ i 2 = ln (1 + c. o. v 2 ) μ i = ln(m i ) + σ i 2 = ln [m i (1 + c. o. v 2 )] Expected values=e μ i+ σ 2 i 2 = e ln[m i(1+c.o.v 2 )]+ ln (1+c.o.v2 ) 2 =m i (1 + c. o. v 2 ) 3 2 i = Expected values mode = m i (1 + c. o. v 2 ) 3 2 m i So the sum of i = [m i (1 + c. o. v 2 ) 3 2 m i ]

38 30 In order to continue with this verification, coefficient of variation needs to be proven unchanged as mode changes. From the definition of coefficient of variation for lognormal distribution, Variance C.o.v 2 = 1 Expected values 2=eσ2 From this equation, it can be seen that coefficient of variation does not relate to mode so it can be assumed that c.o.v remains unchanged as long as σ remains the same. With this assumption, the verification can proceed. i = [(1 + c. o. v 2 ) 3 2 1] m i (11) Then the same calculation process is applied to mj. The following can be obtained: j = [(1 + c. o. v 2 ) 3 2 1] m j Since the total cost is the same as assumed before, meaning that m i is equal to m j, final s for m i and m j are also the same thus verifying that final is independent of compositions of components. This verification is essential because it makes the breakdown analysis possible to perform. With this verification, it is known that costs of sub-components of all the categories in SCC10 to SCC80 are not required to perform the transformation process. Now that the basic assumption is verified, it means that the cost of each category can be transformed to its expected value even though the component combination is not known. The FTA transit database provided breakdown costs for each SCC phase, each

39 31 estimate cost is transformed to its expected value using the process shown above and difference between mode and mean, which is the error of the estimate, is calculated. 6. Example of Breakdown Analysis Following is an example of breakdown analysis and how the transformation process is performed with a project s cost estimate. One of the two databases available to us is the Final-G07 report which contains 28 projects with project descriptions which are crucial to determining projects c.o.vs. The other database is the HRT and LRT database from transit department which provides detailed actual costs for 59 projects broken into SCC10 to SCC80. SCC90 and SCC100 are intentionally left out because these two phases are related to financial charges and contingencies which do not fit in the area of engineering and construction. Though mostly different, these databases contain some mutual projects. 14 projects are found in both databases and these 14 projects will be used to perform the breakdown analysis and develop the model. The basic procedure for the analysis contains several steps. The first step is to review the project descriptions in the Final-G07 report and rate these 14 projects. Second step is to assign proper c.o.v to each project based on its score obtained from step 1. Third step is to go to the HRT and LRT database and perform the procedures described in the Breakdown Analysis chapter. The last step is to find increase in cost in terms of percentage of the original total cost estimate. The 14 projects with scores and c.o.vs are in Table 6 and 7:

40 32 Name Chicago Table 6: Score and c.o.v for 14 projects in analysis Denver Los Los Los Minneapolis New Jersey Southwest Southwest Angeles Angeles Angeles Hiawatha Hudson-Bergen Extension Line Red Line Red Line Red Line Line MOS1 MOS1 MOS2 MOS3 Score C.O.V Name Portland Portland Table 7: Score and c.o.v for 14 projects in analysis Salt Lake San San St Louis Atlanta Interstate Westside/Hillsboro North-South Francisco Juan Saint Clair North line MAX Max Line SFO Tren Corridor extension Airport Urbano Ext. Score C.O.V San Juan Tren Urbano is randomly selected and will be used as an example of how breakdown analysis is done. It is known that San Juan Tren Urbano has a score of 5 with c.o.v equal to 0.4. The breakdown cost of this project is obtained from HRT and LRT database and breakdown analysis is performed. The results are:

41 33 Table 8: Breakdown Analysis for Tren Urbano Estimate σ 2 μ Expected values SCC10 At-grade guideway Elevated structure guideway Underground bored earth tunnel Direct fixation track Total SCC20 Light maintenance facility-depot Heavy maintenance facility Maintenance of storage building Administrative facility Total SCC30 Train control-way side Electrification-substations Electrification-third rail Communications Revenue collection-in station Central control Total

42 34 SCC40 At-grade center platformmedium At-grade side platform-medium Cut and cover center platformmedium Elevated center platformmedium Elevated side platform-medium Parking lots Signature and graphics Total SCC50 Revenue vehicles-order A Total SCC60 Utility relocation-asis(urban) Total SCC70 (no sub-phase) Total 0 SCC80 Planning/feasibility study

43 35 Preliminary engineering and design Final design Construction management Project management Project management oversight Project initiation-insurance Training/start-up/testing certificate Other soft costs Total Total expected values Reported cost Increase In the table above, take At-grade guideway for instance, recall the equations in Chapter 3: Background Information and Assumptions σ 2 is calculated using equation (8) σ 2 = ln(1 + c. o. v 2 ) In this case where c.o.v=0.4 σ 2 = ln( ) =

44 36 μ is calculated with equation (9) μ = ln(mode) + σ 2 Where mode is the cost in the second column In this case μ = ln( ) = Finally, expected value is calculated with equation (5) Mean=e μ+σ 2 2 In this case mean = e ( ) 2 = Apply these procedures to all the components and sum up all the expected values, the final expected value is equal to $2,560,485,171. The adjustment error resulting from using mode instead of mean is calculated by the equation: Increase = Expected Value Estimate Estimate (12) For San Juan Tren Urbano,

45 37 Increase = = This increase is considered to be the theoretical error or system error in the total estimate of this project. In other words, theoretically, this percentage error is supposed to happen no matter how careful estimators are, while making estimates. This increase is the quantitative verification of the source of error described in chapter 1: Source of Error. Apply this whole procedure to all the 14 projects, the increases are presented in Table 9 and 10: Name Chicago Denver Table 9: Systematic errors Los Los Los Minneapolis New Jersey Southwest Southwest Angeles Angeles Angeles Hiawatha Hudson- Extension Line Red Line Red Line Red Line Line Bergen MOS1 MOS1 MOS2 MOS3 Score C.O.V Increase

46 38 Name Portland Portland Table 10: Systematic errors Salt Lake San San St Louis Atlanta Interstate Westside/Hillsboro North-South Francisco Juan Saint Clair North line MAX Max Line SFO Tren Corridor extension Airport Urbano Ext. Score C.O.V Increa se It can be noticed that projects with same c.o.v have the same percent increases. This phenomenon can be explained as below. Recall equation (11) from chapter 5: Breakdown Analysis = [(1 + c. o. v 2 ) 3 2 1] m Where is the difference between total estimate and total expected value and m is the total estimate (mode). The increase is calculated using equation (12) Increase = This equation can be transformed into Expected Value Estimate Estimate

47 39 Increase = 3 = [(1+c.o.v2 ) 2 1] m m m = (1 + c. o. v 2 ) (13) It can be seen that using the proposed approach, as long as projects have the same c.o.v, their percent increases will be the same. This finding actually will help the future process of creating charts for estimators to use because c.o.v is the only determining factor. Charts can be developed in correspondence with different c.o.vs. This again proves that this increase is only related to the very nature of a project itself. No matter how careful estimators are when making cost estimates, the increase in cost will theoretically happen as long as the project nature, which includes complexity, does not change. 7. Confidence Level Since all the analysis and calculation were based on the data without contingency, how much more should be added to each project to ensure that the project budget is adequate? In other words, the contingency needs to be determined. There are several ways to define contingency. In fact, there are three types of contingencies: tolerance in the specification, float in the schedule, and money in the budget. In this thesis, the focus is on the budget contingency. In this case, contingency is defined as the money needed above the estimate to reduce the risk of overruns to a level acceptable to the organization (Baccarini 2005; Jackson 2012). Different owners will have different requirements for confidence level; some owners think more conservatively and do not want to regret later, they may choose a higher confidence level while other

48 40 owners do not mind risking a little bit thus choosing a lower confidence level. Moreover, large contingency can deprive other projects from funding. Therefore it is crucial to have a proper contingency for a certain project. The development of contingencies can be satisfied by the nature of probability distribution, in this case a normal distribution, itself. In this analysis, we assumed that the total cost of a transit project follows a normal distribution. Therefore with the help of a normal distribution, a particular number, in this case a cost estimate, with a predetermined probability can be acquired. The predetermined probability is referred to as confidence level. 8. Inverse Normal Distribution Method In order to find the contingency for a particular c.o.v, the inverse normal distribution method is used. Though starting as lognormal distributions for every detailed component cost, when all these details get added together, the whole cost follows a normal distribution as discussed before. Therefore, the process for finding contingency for the project as a whole must use a normal distribution. In this chapter, the corresponding value of a given probability with known mean and standard deviation needs to be found. The process contains the following steps: first create a set of consecutive numbers from 0 to 1, which will be the probabilities used later. In this thesis, 0.01 to 0.99 were created. The mean for a project is its total expected value calculated before. Its standard deviation can be calculated using the definition of c.o.v:

49 41 standard deviation = c. o. v mean (14) With mean and standard deviation known, the corresponding values of those probabilities created before can be calculated. Then contingency of a project is calculated by equation (15) below: contingency = Final budget Total estimated value (mode) Total estimated value (mode) 100% (15) San Juan Tren Urbano will be used as an example of how to develop a cost curve. We classified San Juan Tren Urbano as a highly complex project and thus assigned a c.o.v of 0.4. Estimate (mode) = $2,250,000,000 Expected Value (mean) = $2,811,056,029 So its Standard Deviation = = $1,024,194,068 Then create a set of consecutive numbers as described before and the corresponding values of those consecutive numbers (probabilities) can be calculated: