IDENTIFYING THE PREFERRED OPTION COMPLEX DECISION-MAKING FOR LARGE PROJECTS

Transcription

1 Presented to NZPI Conference, Rotorua, May 2009 IDENTIFYING THE PREFERRED OPTION COMPLEX DECISION-MAKING FOR LARGE PROJECTS By Sylvia Allan, Senior Consultant to MWH & Steve Oldfield, MWH Technical Risk Expert

2 What I cover: Generic issues around choosing preferred options for major projects Basic multi-criteria analysis methodology Application of multi-criteria analysis methodology for three large projects Demonstrating adaptability of the methodology

3 At the Planning stage, large and complex projects inevitably involve: the need to assess alternatives in consenting processes (4 th Schedule RMA; S171 for NoRs) a wide range of costs and benefits, including many intangibles a wide range of environmental / social / cultural effects a wide range of risks (consenting, financial, technological, physical) a range of time-related considerations (climate change effects, reversibility, reconsenting) a range of internal and external decision criteria (corporate objectives, RMA, LGA, etc)

4 Identifying a preferred option to proceed with must: demonstrate alternatives have been considered demonstrate the application of adequate and appropriate information and values reasonably transparent/auditable robust Because projects are so varied, any methodology must be adaptable to a wide range of circumstances

5 The Multi-criteria analysis methodology Multi-criteria analysis (MCA) is now widely used. It involves scoring different options in terms of a range of aspects, attributes or criteria which reflect the issues that need to be considered to achieve the best possible outcome, recognising that there may be constraints on what can be done. MCA provides a open and traceable method of weighing up the advantages and disadvantages of the different options taking account of both tangible and intangible issues. Ref: Optimised Decision Making Guidelines A Sustainable Approach to Managing Infrastructure. Pub. By NZ National Asset Management Steering (NAMS) Group, 2004

6 Investigations and modelling Decision Process Short-list options and define assessment criteria Studies Stakeholders Identify and assess risks Information Views Explore issues & refine options using Multi-criteria Analysis Establish broad costs, benefits & feasibility Decision Consensus Process Multi-criteria Analysis Of options Through the application of the multi-criteria analysis method

7 The MCA Process 1. Develop options 2. Screen out non-viable options Broad objectives that define an ideal solution Screening criteria to remove show-stoppers 3. Agree programme of elicitation workshops Number of workshops depends on diversity of stakeholder groups 4. Develop or workshop assessment criteria & scoring scheme 5. Determine make up of decision making group Ideally the same set of criteria should be used throughout 6. Develop briefing material on each options 7. Workshop to obtain option scores & criteria weights Sufficient information for informed decision to be made Scores Measures of fit of option against each criterion Weights Relative importance of criteria 8. Evaluate options 9. Sensitivity testing & robustness of results to alternative views How sensitive are results To small changes 10. Feedback and/or peer review results to decision makers 11. Identification of overall preferred option Iterative refinement of options may be required. Post-adjustment of scores & weights only if justifiable In the event of no clear winner use cost Criterion to decide or use Benefit Cost Analysis on short-listed options

8 Options Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5 Criteria m-1 Criteria m MCA - Methodology Various ways of establishing weights Assessment Criteria w 1 x S 11 + w 2 x S 12 + w 3 x S 13 + W 1 W 2 W 3 W 4 W 5 W m-1 W m Option 1 Option 2 S 11 S 12 S 13 S 14 S 1m-1 S 1m S 21 S 22 S 23 S 24 W k.s 1k W k.s 2k Option Scores Option 3 Option 4 Option n-1 Option n S 31 S S nm W k.s 3k W k.s 4k W k.s n-1k W k.s nk Highest or Lowest Average Combined Score Indicates Preferred Option Scores typically between 1 and 5

9 Scores and weights What do they represent? Scores represent how favourable an option is considered to be in terms of a particular criterion, aspect or attribute. They avoid the need to define a common scale of measurement (eg, cost) across a diverse set of criteria, some which may be difficult to quantify. Typically, scores between 1 and 5 are used. More becomes difficult for people. Weights represent the importance of a particular criterion in relation to other criteria. The weights are typically normalised to sum to 1. If all criteria are considered to be equally important then all weights are the same, and a weighted analysis is not needed. Different groups may have different views about the relative importance of criteria. Different sets of weights should be recorded for each group involved in the decision making process.

10 Criteria and scores Criteria must be written down, so it is clear to all just what is encompassed by a criterion. The assigned scores become the base data for the analysis Score 1 Best / Worst in respect of a particular criteria Score 3 Average neither particularly good nor bad in respect of a particular criteria Score 5 Worst / Best in respect of a particular criteria

11 Setting the Weights An interactive graphical technique is effective in enabling consideration of the relative importance of the different criteria in a group setting. 6 Change from 2 to 6 Weights are initially assigned values between 0 and 10, then scaled to sum to 1 Different weighting schemes to be explored.

12 Practical methodology Multi-criteria analysis usually involves: structured, facilitated, recorded workshop process client and expert participation sometimes interest group involvement presentation and discussion of comprehensive relevant information consensus approach to scoring and weighting (differences recorded) sensitivity analysis Review after further information (if needed) However, it is an adaptable process.

13 Basis of Methodology People are essentially rational. Given the same information, thorough discussion will reach the same decision. Applies regardless of personal attributes of group. Multi-criteria analysis is an aid in decisions; it does not on its own make a decision. Other factors may over-ride.

14 Use of multi-criteria analysis in three major projects North Island Grid Upgrade Overhead Line Corridor and Route Choice Coromandel Power Line Overhead Line Route Review Wellington Bulk Water Supply Reservoir

15 NIGUP Corridor Stage Second stage of ACRE process (Area Corridor Route Easement) Identified corridors like a braided river possible to mix and match sections MCA issues dividing corridors into sections, number of sections and section length Eleven aspects taken into account A range of weightings applied

16 Corridor analysis matrix Central Corridor A B C D E F G H Comments Lot Size (5) A: Getting easier to the south of this section D: Cambridge 5 but less down to south Special Ownership A: Army base (Papakura) C: Quarry Landscape Quality/ Natural Character (4) E: 4 over lake F: Possible sensitive communities Absorption Capacity Archaeological/Hist orical A: 3 Southern end B: 2 to the north C: Has some areas 4 and F: More of a problem in central part Tangata Whenua D: Uncertainty concerning isolated section Tourism/Recreation C: Not all corridors visible from SH27 Q: Some preferred 1 Ecological Settlements (3) 1 1 D: Cambridge but less population either side C: Morrinsville to be avoided F: Mangakino to be avoided Life Lines A: Ardmore Airport Papakura Army Base B: Possible issues of crossing existing lines D: To get over 220kV line need to move corridor boundary E: 2 if possible to remove existing 110kV line District Plans (1) A: Ardmore Airport restriction C: Some felt should be 1 Totals

17 Route Sector (see Fig 3) Ae+Ben +Bes+C e+fc Ae+Ben +Bes+D e Ac+Cc+ Hc+Fc Ac+Cc+ Hc+Ec+ Gc Ac+Bc+ Hc+Fc Ac+Bc+ Hc+Ec+ Gc Ac+Bc+ Dc+Gc Aw+Bw+ Dw+Ac+ Gc Aw+Ew+ Cw+Dw+ Ac+Gc Aw+Cw+ Dw+Ac+ Gc RMA Weighting (1) East Corridor Central Corridor West Corridor Sector totals for all aspects Average Rank

18 Preferred Corridor Part of Central Corridor identified for next Route stage analysis Substantial public input, further detailed work, options identified, some discarded MCA issues route sections, no option in some areas, aspects for evaluation different from Corridor (ie, some aspects avoided), use of proxies/indicators

19 Routes and route sections

20 Route choice aspects QBL Attribute Social Cultural Environmental Economic Aspect / Indicator Number of dwellings Sensitive landuses Tourism / recreation Tangata Whenua issues / values Archaeological / heritage Ecology / valued vegetation Visual impact Construction impacts Impact on aerial topdressing Impacts on other rural landuses Engineering degree of difficulty Property degree of difficulty Property costs

21 Component Label Number of Dwellings Sensitive Landuses Tourism/ recreation Impact on aerial topdressing / spraying Impact on other rural landuses Impact on historical / archaeological heritage Impact on ecological values and valued vegetation Visual impact / landscape values Property degree of difficulty Engineering degree of difficulty Construction impacts Tangata Whenua metaphysical and other values Compensation Payments Scores for aspects by route section S1E S1W S2E&S2W S3E&S3W S4E&S4W S5E S5W S6E S6W S7E S7W S8E S8W S9E S9W S10E&S10W S11E S11W S12E S12W S13E S13W S14E S14W S15E S15W

22 Alternative weighting schemes used in analysis Social Environmental Economic Cultural Top-level Attributes Number of Dwellings Sensitive Landuses Tourism/ recreation Impact on aerial top-dressing / spraying Impact on other rural landuses Impact on historical / archaeological heritage Impact on ecological values and valued vegetation Visual impact / landscape values Property degree of difficulty Engineering degree of difficulty Construction impacts Tangata Whenua metaphysical and other values Compensation Payments Totals

23 Alternative weighting schemes used in analysis (cont) RMA S6 Heavy UWeights1 RMA S6+S5 UWeights2 Community Uweights3 Industry Uweights4 Top-level Attributes Number of Dwellings Sensitive Landuses Tourism/ recreation Impact on aerial top-dressing / spraying Impact on other rural landuses Impact on historical / archaeological heritage Impact on ecological values and valued vegetation Visual impact / landscape values Property degree of difficulty Engineering degree of difficulty Construction impacts Tangata Whenua metaphysical and other values Compensation Payments Totals

24 Comparison of alternatives by preferred route section and numerically Components Social Environ'tal Economic Cultural Consensus RMA S6 RMA S6+S5 Community Industry Consensus S1E S1W West S5E S5W West West S6E S6W West West S7E S7W West S8E S8W West S9E S9W East S11E S11W S12E S12W S13E S13W West S14E S14W East East S15E S15W East S6-9E S6-9W West S11-14E S11-14W West

25 Interim (and final) route Preferred option subject to final review following public submissions / informal hearings and some additional specific investigations Small number of scores slightly changed as a result No change in preference

26 Coromandel power line Replacement line in sensitive location MCA applied as a review mechanism Delphi technique replaced workshop method, several iterations Ten aspects or criteria agreed on Weighting systems ordinary person and RMA analysis, plus four well-beings

27 Assessment criteria weighting Ordinary Person analysis Assessment criteria weighting Ordinary Person analysis

28 Assessment criteria weighting RMA analysis Assessment criteria weighting RMA analysis

29 Affected dwellings / buildings Impact on rural / other landuses Recreational / tourist impacts Impacts on ecological values Visual impacts Tangata whenua values Property degree of difficulty Engineering degree of difficulty Construction impacts Diversity / security Affected dwellings / buildings Impact on rural / other landuses Recreational / tourist impacts Impacts on ecological values Visual impacts Tangata whenua values Property degree of difficulty Engineering degree of difficulty Construction impacts Diversity / security Overall multi-criteria assessment matrix Overall Multi-criteria Assessment Matrix - "Ordinary Person" Analysis Max Route Options Overall Score Score Profile Option Rank Option 1 (Replace) Option 1 (Parallel) Option Option Option 3A Option 3B Option Overall Multi-criteria Assessment Matrix Part 2 RMA Analysis Max Route Options Overall Score Score Profile Option Rank Option 1 (Replace) Option 1 (Parallel) Option Option Option 3A Option 3B Option

30 Scores Scores Scores Scores Results presented against the four well-beings separately RMA assessment 2.0 MCA Options Selection - Social Weighting Only 2.0 MCA Options Selection - Economic Weighting Only Option 1 Option 1 Option 2 Option 3 Option 3AOption 3BOption 4 (Replace) (Parallel) 0.0 Option 1 Option 1 Option 2 Option 3 Option 3AOption 3BOption 4 (Replace) (Parallel) Options Options 2.0 MCA Options Selection - Environmental Weighting Only 2.0 MCA Options Selection - Cultural Weighting Only Option 1 Option 1 Option 2 Option 3 Option 3AOption 3BOption 4 (Replace) (Parallel) Options 0.0 Option 1 Option 1 Option 2 Option 3 Option 3AOption 3BOption 4 (Replace) (Parallel) Options

31 Wellington New bulk water supply reservoir Live storage options for further investigation Whakatikei (various sizes) Pakuratahi Skull Gully (augmented from Orongorongo R) Different areas and issues associated with each Extensive studies / consultation undertaken.

32 Context of options Whakatikei Pakuratahi Skull Gully

33 Associated infrastructure

34 Skull Gully

35 Pakuratahi

36 Whakatikei

37 Options - Comparison Whakatikei Pakuratahi Skull Gully Spillway level: 146m Water depth at dam: 31m Dam height: 36m Lake surface area: 52 ha Catchment area: 4350 ha Spillway level: 292m Water depth at dam: 32m Dam height: 37m Lake surface area: 44 ha Catchment area: 3750 ha Spillway level: 203m Water depth at dam: 38m Dam height: 40m Lake surface area: 40 ha Catchment area: 590 ha

38 Making the process robust Phase 1 study determined three groups of people should participate: GW Councillors, Customers, GW Senior Staff Phase 1 workshops (Nov 2006) to identify and weight criteria/aspects Each of the three workshops shared information presentations on options and information to date (environmental, engineering, risk planning) Comments/discussion from Phase 1 workshops also identified information gaps became part of Phase 2 investigations

39 Phase 1 workshop outcomes Each workshop generated a very large number issues and considerations relevant to choice of storage options 30 to 40 Each workshop then grouped the detailed issues/considerations into more general criteria/attributes (10 or 11). The description of each attribute was then written up to include the specifics Each workshop then weighted each of its attributes (relative to other attributes) Each workshop came up with a slightly different set of criteria/attributes, and quite different weightings

40 Example outcome of phase 1 workshop criteria

41 Workshop outcomes - Councillors

42 Workshop outcomes - Customers

43 Workshop outcomes - Staff

44 Workshop outcomes - Comparison

45 Reflections on MCA process Very adaptable Robust if applied with integrity and sufficient information Efficient process if well-managed However not a decision on its own an aid to a decision some aspects change over time costs, planning framework careful documentation is essential.