NERC Probabilistic Assessment

Transcription

1 NERC Probabilistic Assessment Addendum to the 2012 Long-Term Reliability Assessment June of Peachtree Road NE Suite 600, North Tower Atlanta, GA

2 Table of Contents Introduction... 3 Overview... Error! Bookmark not defined. Probabilistic Assessment Methods... 6 Summary Results... 8 ERCOT... 9 FRCC MISO MRO-MANITOBA HYDRO MRO-MAPP MRO-SASKPOWER NPCC MARITIMES NEW ENGLAND NEW YORK ONTARIO QUÉBEC PJM SERC SERC-E SERC-N SERC-SE SERC-W SPP WECC Observations and Recommendations Recommendations Appendix I: Summary of Methods Appendix II: Assessment Area s of 35

3 Introduction A comprehensive understanding of the complexity of the changing bulk power system (BPS) is key for the development of prompt industry actions that achieve effective reliability outcomes. NERC s reliability assessments provide a technical platform for important policy discussions on the challenges facing the interconnected North American BPS. The trends indentified in previous Long-Term Reliability Assessments (s) highlight the changing nature of the resource mix from predominately capacity-based resources to largely energy-limited resources (i.e., wind, solar, and in some cases gas-fired generation). By identifying and quantifying these emerging reliability issues, NERC is able to provide risk-informed recommendations and leading-edge indicators that support a learning environment for industry to pursue improved reliability performance. The purpose of this assessment is to provide enhanced resource adequacy metrics for NERC s. Historically, NERC has gauged resource adequacy through the use of a deterministic assessment metric planning reserve margin. Planning reserve margin is a measure of available capacity over and above the capacity needed to meet normal (50/50) forecast peak demand levels. In order to withstand higher than expected demand and unplanned forced outages of generation and transmission facilities, regulatory bodies generally require producers and transmission facilities to maintain a peak planning reserve margin between 10 to 20 percent of normal peak demand. In 2010, the Generation and Transmission Reliability Planning Models Task Force (GTRPMTF) concluded that existing reliability models could be used to develop one common composite generation and transmission assessment. Within the context of the task force s review, a composite generation and transmission assessment is an assessment of generation (resource) adequacy, which considers the ability of to receive power supplied by aggregate resources. 1 The level of detail of transmission modeling is a practical constraint. Although there are some models with the capability of modeling transmission in detail, the use of such models across large electrical areas may not be practical at this time. However, the task force concluded that a probabilistic assessment should recognize transmission constraints. The task force also noted the importance of having complete coverage of the North American BPS, as well as the elimination of overlaps. As this premise is already adopted and executed annually in the, the approach for the probabilistic assessment follows suit. The assessment areas (i.e., Regions, Planning Coordinators (PCs), independent system operators (ISOs), and regional transmission organizations (RTOs)) used for this assessment are identical to those used for the. Using the GTDPMTF s on Methodology and Metrics, 2 the NERC Planning Committee approved a common set of reliability indices to supplement the. The task force ultimately chose two primary metrics to appropriately represent a consistent measure across the different assessment areas. The chosen metrics Expected Unserved Energy (EUE) and lossof- hour (LOLH) are defined below as well as other probabilistic metrics: EUE is a measure of the generation and transmission system s capability to continuously serve all s at all delivery points while satisfying all planning criteria. EUE is energy-centric and analyzes all hours of a particular year. Results are calculated in megawatthours (MWh). EUE is the summation of the expected number of megawatthours of that will not be served in a given year as a result of demand exceeding the available capacity across all hours. Additionally, this measure can be normalized based on the assessment area s total Net Energy for Load. Normalizing the EUE provides a measure relative to the size of a given assessment area. LOLH is generally defined as the number of hours per year where system demand will exceed the generating capacity. LOLH is usually expressed in hours per year. Any outage caused by inadequate resources regardless of geographic extent or interrupted (it could be 1 MW for a single customer or the loss of the whole area ) counts as a LOLH. An LOLH of 0.1 means that an hour of loss of is expected for every 10 years. For a particular 1 Transmission adequacy is generally assessed using deterministic methods. This assessment takes place in two different situations: (1) when new resources or s are connected to the grid (see FAC-001 and -002), and (2) through the TPL series of standards. Such studies model the impact of contingencies; stability analysis assesses the immediate impact of contingencies (cycles to minutes), while power flow models assess their steady state impact. 2 The GTDPMTF s Final on Methodology and Metrics is located on NERC s website: Metrics final _w%20_pc_approvals_revisions_12%2008%2010.pdf 3 of 35

4 Introduction year (e.g. 2014), the resources available and shapes for that year are used to calculate the metrics for that year. Loss-of- expectation (LOLE) is generally defined as number of days per year for which the available generation capacity is insufficient to serve the demand at least once during that day. This metric is not being reported as part of this assessment. A criterion of one day in 10 years is an often used standard for LOLE. Loss-of- probability (LOLP) is the probability of system demand exceeding the generating capacity during a given period. If LOLP is the number of days on which there is expected to be a shortfall per year, it is the same as LOLE. However, other definitions of LOLP are also used. Loss-of- events (LOLEV) is the number of events in which some system is not served in a given year. A LOLEV can last for one hour or for several contiguous hours and can involve the loss of one or several hundred megawatts of. TRE-ERCOT is the only Assessment Area to provide this metric in its regional assessment. LOLE, LOLEV, and LOLP are often used by local areas to define a target metric of reliability. The classic definition of reliability as one day in 10 years is an LOLP target and is often translated into a LOLE target of 0.1 day/year or LOLEV of 0.1 event/year. These metrics are not provided in this report to avoid potential conflicts with regional practices based on different methods. Overview The 2012 Probabilistic Assessment (2012 ) is designed to complement the 2012 by providing additional probabilistic statistics of EUE and LOLH. This assessment includes the results of each assessment area s third and fifth year forecasts (i.e., 2014 and 2016 results) from the The assessment areas represented are shown in Figure 1. For the most part, the same base case was used for both the 2012 and the 2012 model runs; however, for some assessment areas, different base case data may have been used due to the vintage of the data available or other analytical restrictions. NERC s is a compilation of 11 separate assessments conducted by five Regional Entities (REs) and six individual assessment areas, covering the total 26 assessment areas. These studies varied in scope from the whole of WECC to individual PCs (e.g., MISO, PJM, and SaskPower). These individual assessments were then combined into NERC s for the whole NERC footprint. A July 2012 conference call and a following formal letter in August that requested supporting information and data initiated the To minimize the burden on industry, the assessment was designed to coincide with the finalization of the data for the The analysis was conducted over the next six months with initial individual drafts due in December 2012 and final reports in February The final comprehensive NERC report combining these assessments will be presented for the approval of the NERC Planning Committee at its June 2013 meeting. Last year NERC conducted the Pilot Probabilistic Assessment 2011, 3 which was based on 2010 data; this initial effort was approved by the NERC Planning Committee in June Only FRCC, the NPCC Assessment Areas, the SERC Assessment Areas, Manitoba, MISO, and PJM were capable of conducting such studies as part of the pilot report. NERC compared the results of the pilot report and the 2012 for the areas in both studies. With the approval of the pilot report, the Planning Committee concluded that the probabilistic assessment complemented and enhanced the efforts of the Reliability Assessment Subcommittee (RAS) and should be a biennial report with the participation of all assessment areas across the whole NERC footprint. This is the first such report in the series of probabilistic assessments covering all of the NERC Assessment Areas. 3 The Pilot Probabilistic Assessment 2011 is located on the NERC website: 4 of 35

5 Introduction FIGURE 1: NERC ASSESSMENT AREAS 5 of 35

6 Probabilistic Assessment Methods While consistent metrics and indices were produced for this assessment, several different models were used to perform the necessary computations. However, common approaches ensured the development of a consistently defined method across all assessment areas. All assessment areas must adhere to the following minimum requirements: Use an hourly model that includes forecast uncertainty. Create a limitation on what generation is included in modeling. To be included, future generation must have associated transmission. Model random outages for all units as random variables as opposed to derating the unit s capacity when modeling dispatchable capacity. Use a transmission modeling method to incorporate major transmission constraints and limitations that is consistent with the assessment area s planning processes. The assessment area may select the transmission modeling method at its discretion. Each assessment area must document: o o How that approach takes in to account transmission constraints within and outside of the assessment area, and How it developed the data needed for modeling. Generate three metric results: Annual LOLH EUE EUE as a percentage of Net Energy for Load (normalized EUE) for two common forecasted years (i.e., years 3 and 5). 4,5,6 Document of all modeling assumptions. Adhere to a common report format. While this approach provides commonality in methods, differences exist in how each assessment area performed its own probabilistic assessment (i.e., modeling software, detailed assumptions, etc). This did not lead to the direct comparability of metrics between assessment areas, but the method allowed for consistency of the inputs to the metrics. More importantly, the comparison of metrics could not be used to determine which assessment area is more reliable than another assessment area. all comparability, however, is lost. By requiring each assessment area to forecast metrics for two common years, the trend for an assessment area s reliability over time could be observed. A conclusion from the GTRPMTF final report indicated that a single model or software package was not warranted. As such, the probabilistic assessment used three different types of software packages. A detailed matrix showing the assumptions and methods used for each assessment area can be found in Appendix I. 4 Although the spans a 10-year period, the selection of these two years provides the greatest value while reducing the reporting burden of calculating metrics for each year of the. 5 Computation of LOLE is not required. This metric is generally defined as the summation of the LOLP for each daily peak hour. Since LOLE only evaluates resource reliability across the daily peak hour, the reliability of energy-limited systems or systems with significant variable and energy-limited resources may not be accurately modeled. Therefore, the metric calculation is based on 365 hourly values. 6 Some entities use only the weekday peak s, or 260 hourly values. These entities generally assume that the LOLP for the weekend peak s are expected to sum to zero and will not contribute to the LOLE for the year. 6 of 35

7 Probabilistic Assessment Method Manitoba, MISO, MAPP, NPCC, PJM, and SERC used the Multi-Area Reliability Simulation (MARS) program, which was developed by General Electric (GE). MARS uses a sequential Monte Carlo simulation technique to calculate the reliability indices of a generation system that is made up of a number of interconnected areas. MARS performs a chronological hourly simulation of the interconnected system, comparing the hourly in each area to the total available generation in the area, taking into account the random outages of thermal generating units, availability of interconnection tie lines, and the energy-limited nature of hydro and wind resources. If an area s available generation, including assistance from other areas, is less than its, the area will be in a loss-of- state for that hour, and the statistics required to compute the reliability indices are determined. ERCOT used ProMaxLT to calculate LOLEV, LOLH, and EUE. ProMaxLT is stochastic in nature; it uses a Monte Carlo random outage scheduler to create full and partial outage states for generating units. An exponential distribution is used in ProMaxLT to randomly determine the outage and repair times for each unit. Four exponential distributions are used: Full Time to Failure, Full Time to Repair, Partial Time to Failure, and Partial Time to Repair. FRCC did not perform a Monte Carlo simulation. FRCC used the Tie Line and Generation Reliability Program (TIGER) for the computation of LOLH and EUE metrics. The simulation software is based on the method of recursive convolution for the calculation of generating capacity reliability indices that employs an algorithm tested compliant with the standard Reliability Test System modeled by IEEE Power System Engineering Committee. SPP used GridView a software application developed by ABB Inc. to simulate the economic dispatch of an electric power system while monitoring key transmission elements for each hour to perform the probabilistic analysis. A key advantage of using the GridView application is the ability to model a detailed transmission system in the study region, not just a transportation model. SaskPower and WECC used PROMOD for reliability planning indices. The software simulates the operation of an electric utility generation system. Although the same software was used for this assessment, different methodologies and assumptions were used in each area. SaskPower was not able to calculate the LOLH metric. A summary of the methods used in the studies is contained in Appendix I. Additionally, Appendix II contains a link to individual area reports where more detail on the modeling approaches is available. 7 of 35

8 Summary Results This section contains the results of the 2012 by area. Within each assessment area, NERC presents the probabilistic metrics and the reserve margin from this assessment. The corresponding Anticipated and Prospective Reserve Margins are also included. Results from the pilot report and the corresponding 2010 are provided where available. Detailed results from each of the 11 studies are available through the links in Appendix II. The 2012 calculated reserve margin and probabilistic metrics for 2014 and 2016, which are presented with the corresponding 2012 reserve margins. The pilot assessment calculated metrics for 2011 and 2014, which are presented with the corresponding 2010 reserve margins. The data from the pilot report and 2010 is included in the tables under The 2010 data is left blank if the footprint of the assessment area changed between 2010 and 2012 or the area did not participate in the pilot assessment. Reserve margin is used as the main link between the 2012 and this assessment. The reserve margins calculated for the submitted data sheets were in some cases quite different from the reserve margins because of different reporting of demand, demand- and supply-side demand reduction, and capacity transfers. Where possible, the Adjusted Reserve Margin was calculated for the by changing how the data was included in the reserve margin calculation. Demand reduction may be included as an Emergency Procedure and excluded from the reserve margin calculation in the, but in the it was reported as a demand-side program (reducing Total Internal Demand) or as a supply-side program (included as a resource). The demand response capacity was moved in the Adjusted Reserve Margin to have the same effect on reserve margin as in the ; the differences in reserve margin reflect only data differences and, to the extent possible, modeling differences. Generally, the data used in this assessment was quite similar to that in the 2012 with only minor differences from updates that occurred between the completion of the data submission and the beginning of the. Thus, the 2012 Adjusted Reserve Margin is very close to the 2012 reserve margin in most cases. Any large differences in reserve margin reflect characteristics of the probabilistic analysis where the data cannot be reported in the same way as in the. 8 of 35

9 Summary Results ERCOT 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh The ERCOT probabilistic assessment was not performed using data similar to the Instead, the assessment was based on a system with sufficient capacity to obtain a LOLE of one event in 10 years. The required reserve margin for this was 16.1 percent compared to the 8.9 percent (2014) and 5.7 percent (2016) Anticipated Reserve Margins in the Therefore the reliability indices calculated in the probabilistic assessment could not be compared to the 2012 reserve margins. Also, ERCOT did not participate in the pilot assessment based on the 2010, so no comparison over time was possible. Data comparison will be possible when the new probabilistic assessments are completed in A notable characteristic of the ERCOT study is that external areas, contributions from the dc ties (ERCOT is an isolated interconnection), energy efficiency and demand response programs, hydro generations and solar generations were not considered. Private Use Networks (PUNs) were considered as behind-the-meter for the but explicitly modeled for the. ERCOT also ran a second version of the study, which considered internal transmission limits in addition to the case of no internal transmission limits. By conducting the study at the target LOLE of one event in 10 years, there is a benchmark comparison across the various metrics. For 2014, a 16.1 percent reserve margin results in a 0.1 event/year LOLE, a 0.24 hours/year LOLH, and a 267 MWh/year EUE. For 2016, the probabilistic metrics are all lower but not different in character. Subsequent studies will be needed before any kind of trend may be identified. ERCOT ran a sensitivity case that included the internal transmission constraints. The results indicated that the metrics increased by hours (LOLH) or 12.5 MWh (EUE) in 2014 and by 0.07 hours (LOLH) or 3.5 MWh (EUE) in of 35

10 Summary Results FRCC 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Pilot Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh FRCC s membership includes 30 Regional Entity (RE) Division members and 24 Member Services Division members composed of investor-owned utilities, cooperative systems, municipal utilities, power marketers, and independent power producers. The FRCC Region is divided into 10 Balancing Authorities (BAs) with 72 registered entities (both members and non-members) performing the functions identified in the NERC Reliability Functional Model and defined in the NERC Reliability Standards. The Region contains a population of more than 16 million people and has a geographic coverage of about 50,000 square miles over peninsular Florida. The reserve margin in the FRCC study corresponded with the Anticipated Reserve Margin rather than the Prospective Reserve Margin, as in most other Regions. Even so, the reserve margin was sufficient for all of the probabilistic metrics to be zero. The FRCC study used convolution to calculate unsupplied energy and hours. FRCC participated in both the pilot study and the 2012 using a well-tested analysis, but because all of the probabilistic metrics were zero in both studies no inferences could be drawn. 10 of 35

11 Summary Results MISO 40% % 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year EUE (ppm) MWh/TWh Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) 0.30 EUE (ppm) MISO is an essential link in the safe, cost-effective delivery of electric power across all or parts of 15 U.S. states and the Canadian province of Manitoba. As an RTO, MISO provides consumers with unbiased regional grid management and open access to the transmission facilities under its functional supervision. MISO is modeled as a single combined entity made up of 28 local BAs. MISO used the MARS program to calculate the probabilistic metrics. For the, Manitoba was analyzed separately. The MISO results matched up exactly with the Prospective Reserve Margin. The probabilistic metrics were small in both years as expected by the high reserve margin used. MISO was not defined as a reporting area (reporting area was MRO) in the Though it covers an extensive geographic area and contains over 100 GW of generation, MISO is modeled as a single area in this study. This footprint will be expanded further before the next probabilistic assessment. 11 of 35

12 Summary Results MRO-MANITOBA HYDRO 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh Manitoba Hydro is a Provincial Crown Corporation that provides electricity to 542,000 customers throughout Manitoba and natural gas service to 267,000 customers in various communities throughout southern Manitoba. Manitoba Hydro also has formal electricity export sale agreements with more than 35 electric utilities and marketers in the midwestern United States, Ontario, and Saskatchewan. Manitoba Hydro is its own Planning Authority (PA) and BA. MISO is the Reliability Coordinator (RC) for Manitoba Hydro. Manitoba Hydro participated in both the pilot report and 2012, and the probabilistic metrics were calculated to be zero in both studies. Manitoba did not have data for 2010 for comparison as it was included as part of MRO in that study. Its 2012 results matched closely with the. 12 of 35

13 Summary Results MRO-MAPP 40% % 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year EUE (ppm) MWh/TWh Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) The Mid-Continent Area Power Pool (MAPP) is an association of electric utilities and other electric industry participants operating in all or parts of Iowa, Minnesota, Montana, North Dakota, and South Dakota. Currently, the MAPP PA includes entities in two BA areas and thirteen Load-Serving Entities (LSEs). The MAPP PA covers an area of approximately 200,000 square miles and serves a population of about 3.5 million. MAPP typically experiences its annual peak demand in summer. There have not been any changes to the MAPP Assessment Area boundary in the last two years, and none are expected. MAPP did not participate in the 2010 pilot study. In the 2012 report, the MAPP data matched closely with the data. MAPP had surplus reserves above its target reserve margin in all study periods. When compared with other Regions, MAPP had a lower reserve margin and higher probabilistic metrics. It is interesting to note the reserve margin decreases from 2014 to 2016, but the probabilistic metrics also decrease where they would be, under some circumstances, expected to increase. 13 of 35

14 Summary Results MRO-SASKPOWER 40% % 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year EUE (ppm) MWh/TWh Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) The Saskatchewan Power Corporation (SaskPower) is the PA, RC, and principal supplier of electricity in the province of Saskatchewan. It is a Provincial Crown Corporation and, under provincial legislation, is responsible for the reliability oversight of the Saskatchewan Bulk Electric System (BES) and its interconnections. Saskatchewan s analysis showed low reserve margins in both 2014 and 2106 as consistent with the LOLH is currently not reported by Saskatchewan. Saskatchewan utilized a conservative forecasting methodology for the and This forecast is similar to a 90/10 forecasting methodology, which results in lower reserve margins. A 50/50 forecast will be utilized for the Saskatchewan has capacity being added to the system in 2013, 2015, and 2017 respectively, which equates to higher reserve margins after the new generation goes into service. Saskatchewan is a small system and does not model assistance from outside its assessment area. Saskatchewan is currently reviewing their reliability planning methodology. 14 of 35

15 Summary Results NPCC The NPCC Region covers nearly 1.2 million square miles and is populated by more than 55 million people. NPCC United States includes the six New England states (New Hampshire, Vermont, Massachusetts, Connecticut, Rhode Island, and Maine) and the state of New York. NPCC Canada includes the provinces of Ontario and Québec and the Maritime provinces of New Brunswick and Nova Scotia. The six Assessment Areas (including PJM) involved in the study assembled the NPCC data, so specific assumptions within each area varied somewhat. However, the resulting data is consistent for all Regions in the final study. The NPCC study had a large number of modeled transmission interfaces within each assessment area modeled, and these had a substantial impact on the results. The NPCC probabilistic assessment process contained data updates that occurred after the 2012 submission, meaning the differences were larger in this area than others. As differences were only in the few hundreds of MW on supply or demand, the probabilistic metrics were still comparable to the NPCC has conducted this study for many years so the results are based on a well practiced and stable analysis. Because of this analysis history, NPCC is one of the few areas where the intended serial comparison of probabilistic metrics and their relationship with the can be investigated. The Maritimes reserve margins decreased from the pilot study but remained above the target margin of 20 percent. The probabilistic metrics increased as expected in the 2012 study but are still quite low. This can be seen through the lower probabilistic metrics in 2016 even with the lower reserve margin. This indicates that differences in the probabilistic metrics are still within the range of analytical accuracy. New England s market structure caused the reserve margin assumed for 2016 to be lower and the associated probabilistic metrics to be slightly higher in 2016 because the capacity auction to acquire capacity for 2012 had not been conducted. When the common year of the two assessments (i.e., 2014) was compared, it showed that New England s reserve margins increased with no significant change in the reported metrics. When the common year was compared for New York, it showed that the reserve margins slightly decreased with no significant change in the reported metrics. Whereas when the common year was compared for Ontario, it showed that the reserve margins increased but resulted in probabilistic metrics that were zero for all years studied. Quebec s reserve margins are close to NERC s generic 10 percent reference reserve margin for hydro-electric systems. Quebec s probabilistic metrics were essentially zero for all years studied. 15 of 35

16 Summary Results MARITIMES 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) 0.60 EUE (ppm) EUE (ppm) MWh/TWh NEW ENGLAND 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh 16 of 35

17 Summary Results NEW YORK 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh ONTARIO 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh 17 of 35

18 Summary Results QUÉBEC 40% % 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year EUE (ppm) MWh/TWh Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Res. Margin (%) Adjusted Planning Res. Margin (%) EUE (MWh) 0.70 EUE (ppm) 18 of 35

19 Summary Results PJM 40% % 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year EUE (ppm) MWh/TWh Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) PJM Interconnection is an RTO that coordinates the movement of wholesale electricity in all or parts of 13 states and Washington, D.C. Acting as a neutral, independent party, PJM operates a competitive wholesale electricity market, manages the high-voltage electricity grid to ensure reliability of an area that spans 214,000 square miles, and serves more than 60 million people. The PJM probabilistic assessment was conducted with the NPCC study so the analytical issues were similar. The assessment included a detailed representation of the interconnections to the north and west of PJM but did not model the impact of systems to the south. Though differences were larger in some areas, the 2012 data was still close to the The 2016 data in the 2012 showed less generation than the 2016 data in the 2012 ; therefore, the 2016 probabilistic metrics in the 2012 were higher than what was indicated by the 2012 data. The 2014 forecast planning reserve margin modeled in the 2012 was higher than the margin modeled in the pilot report due to new generation and demand response resources that were added to the 2012 case since Unfortunately, trending from the is not available since the PJM footprint reported in the 2012 is different from the footprint reported in the 2010 study, which was used as basis for the pilot assessment. In both the 2010 and 2012, the probabilistic metrics are higher in the second year studied. This is consistent with the lower reserve margins seen in the second reporting year in the corresponding s. These lower reserve margins are partially a result of the PJM market structure: the PJM auction that ensures adequate capacity in the second study year had not been conducted at the time the and data was compiled. As presented in the above table, the probabilistic metrics (EUE and LOLH) will increase in 2016 as reserve margin shrinks. However, the metrics are still low, which is consistent with the reserve margin remaining above the requirement. 19 of 35

20 Summary Results SERC SERC is a summer-peaking Region that covers all or portions of 16 central and southeastern states and serves a population of more than 60 million. Owners, operators, and users of the BPS in these states cover an area of approximately 560,000 square miles. In the SERC Region, there are 33 BAs and more than 200 registered entities under the NERC functional model. The following section is a summary describing the assessment areas within the SERC footprint: SERC-E, SERC-N, SERC-SE, and SERC-W. SERC conducted a single study to calculate the probabilistic metrics for all four areas. The reserve margins in the did not align well with the mainly because the analysis structure used did not allow imports and exports to be reported in a manner similar to the. SERC used the latest available data to conduct its analysis; therefore, minor updates also created differences from the reserve margins. Otherwise, the data used in the corresponded with the SERC participated in the pilot assessment, so two sets of probabilistic metrics were available. Due to the reporting changes in SERC Assessment Areas, the 2010 dataset used in the probabilistic pilot study differed from data reported in the The SERC-N reserve margins increased significantly between the pilot report and 2012 studies. This was due to additional generation being added between the two studies. All four areas have high reserve margins and consequently low probabilistic metrics. SERC-E had the lowest reserve margins and showed measurable though still low LOLH and EUE. SERC-E had lower LOLH and EUE for the 2012 study than for the pilot, though reserve margins were similar. The cause of this difference could not be determined from the probabilistic assessment data or reports and may be below the threshold of analytical noise. 20 of 35

21 Summary Results SERC-E 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh SERC-N 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) 0.01 EUE (ppm) MWh/TWh 21 of 35

22 Summary Results SERC-SE 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) 0.01 EUE (ppm) EUE (ppm) MWh/TWh SERC-W 80% 70% 60% 50% 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh 22 of 35

23 Summary Results SPP 40% 30% 20% 0% Anticipated Reserve Margin Prospective Reserve Margin Adjusted Planning Res. Margin hours/year Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) Adjusted Planning Reserve Margin (%) EUE (MWh) EUE (ppm) EUE (ppm) MWh/TWh Southwest Power Pool (SPP) is a NERC RE that covers 370,000 square miles and encompasses all or part of Arkansas, Kansas, Louisiana, Mississippi, Missouri, New Mexico, Oklahoma, and Texas. The SPP RE reporting footprint includes the MRO RE members that are part of the SPP Planning Coordinator, which consists of the Nebraska entities. 7 SPP s footprint consists of 20 BA areas, including 48,368 miles of transmission lines, 915 generating plants, and 6,408 substations at 100 kv and above. Due to the conservative resource forecast used in the SPP analysis, the resulting reserve margin more closely aligned with the Anticipated Reserve Margin than the Prospective Reserve Margin. However, reserve margins were over 20 percent; therefore, the probabilistic metrics were very low in both the pilot and 2012 studies. Due to the data discrepancies in the pilot report, the results were not included in the pilot report or the In 2010, NERC created a Reliability Assessment Procedure that realigned the reporting areas for the REs. Beginning in 2011, SPP RE assumed the reporting responsibilities of the Nebraska entities (NPPD, OPPD, and LES) that are part of the SPP Planning Coordinator. The realignment of footprints increased the demand forecast for the SPP RE footprint. 23 of 35

24 Summary Results WECC The Western Electricity Coordinating Council (WECC) is one of eight electric reliability councils in North America and is responsible for coordinating and promoting BES reliability in the Western Interconnection. WECC s 329 members, including 38 BAs, represent a wide spectrum of organizations with an interest in the BES. Serving an area of nearly 1.8 million square miles and approximately 81 million people, it is the largest and most diverse of the NERC Regions. WECC s service territory extends from Canada to Mexico and includes the Canadian provinces of Alberta and British Columbia, the northern portion of Baja California in Mexico, and all or portions of the 14 western states in-between. All nine WECC Assessment Areas were analyzed in one study using 37 nodes to evaluate transmission. As the whole interconnection was included in the study, modeling of other assessment areas was not an issue. WECC used exactly the same data for the as the anticipated resources (projected resources are the same). As reporting differences between the two studies precluded useful comparison, the table for WECC does not include the 2010 data. All probabilistic metrics were zero in all nine assessment areas for both years analyzed. Even the CALN and NORW showed zero as their EUE or LOLH in 2014; both areas are at their target reserve margins. 24 of 35

25 Assessment Area AESO BASN BC CALN CALS DSW MEXW NORW ROCK Summary Results Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) EUE (ppm) Anticipated Reserve Margin (%) Prospective Reserve Margin (%) Forecast Planning Reserve Margin (%) EUE (MWh) - - EUE (ppm) of 35

26 Observations and Recommendations This report is the first of a series of biennial reports presenting a consistent set of probabilistic reliability metrics for all NERC Assessment Areas in an effort to augment the deterministic reserve margin metrics of the. Complete and non-overlapping coverage in these studies for the whole NERC area is captured. All assessment areas, except ERCOT, completed probabilistic studies on the systems as reported in the. ERCOT conducted a more comprehensive probabilistic study to determine reserve requirements. It was apparent during the completion of this effort that a number of areas are still getting familiar with the intricacies of the probabilistic modeling. As this effort progresses, these areas will become more comfortable with their methods and results. The indicates that most of the assessment areas have adequate resources based on reserve margin targets. The results are consistent with the 2012 data. Most of the Regions show small values of LOLH and EUE, which is consistent with these reserve margins. The WECC, NPCC, PJM, and SERC studies demonstrated it is possible through regional and interregional coordination to run a probabilistic analysis over a wide area with multiple assessment areas. The NPCC-PJM analysis showed that a certain amount of regional differences can also be accommodated in probabilistic assessments. It is hoped that as a result of this North American-wide compilation of assessments, NERC will be able to extend the cooperation and coordination of reliability assessment across the whole NERC footprint. One of the main reasons the GTRPMTF proposed this probabilistic assessment was to integrate transmission limits into resource adequacy assessment. The extent to which this has been done by the various Regions varies considerably. NPCC averages nine internal nodes for each of its five assessment areas, and also breaks down external Regions into subareas. Seven of the studies only represent the studied assessment area as a single node (often referred to as a copper sheet analysis). Transmission in neighboring assessment areas is usually less detailed. The appropriate number of transmission limits to model depends on the transmission in that Region. If the internal transmission system is strong enough, modeling as a single node may be the correct representation. ERCOT performed its study with and without internal transmission constraints and showed there is a slight increase in the probabilistic reliability metrics when internal transmission limits are considered. In many aspects, the probabilistic models had considerable diversity. While the model used often strongly influenced the modeling of such things as uncertainty, intermittent generation, demand-side resources, and emergency operating procedures, there should be opportunities to learn from the approaches used by other areas. The peer reviews conducted for this report and future assessments will facilitate this progress. Recommendations The wide diversity in modeling of such areas as forecast uncertainty, intermittent resources, demand-side resources, and emergency operating procedures indicates an opportunity to learn and improve the next analysis. Leading up to the start of modeling for the next report in summer 2014, discussion and analysis related to probabilistic resource and transmission modeling should continue within the RAS. One aspect of the 2012 that will need to be improved next time is the detailed input data reporting. It is used to describe the probabilistic study and as a comparison to the. In this study, some areas emphasized consistent reporting with the, while others focused more on reporting data specifically as used in the probabilistic analysis. Future reporting will have to be structured to better meet both objectives. It is still early to pull much information from these results on probabilistic metric trends particularly differences between the probabilistic trend and the reserve margin trend. More detailed analysis including the impact of specific generation (e.g., wind) or due to including the effect of transmission in this analysis may be possible in the future, but cannot be investigated with the results produced so far. 26 of 35

27 Observations and Recommendations Both the RAS and the Planning Committee expressed a desire to leverage the analysis in this report to provide more value and insights to the future state of reliability. The RAS should consider the value of adding an additional scope to the probabilistic assessment and report suggested enhancements to the Planning Committee. One suggestion is to include a scenario analysis as part of the probabilistic assessment. Scenario analysis can help identify leading indicators of system stress and determine where vulnerabilities in the system exist. Scenario assumptions such as non-responsive demand response or increased forced outage rates of gas-fired generation due to natural gas unavailability could help gauge an area s resilience to emerging BPS risks. 27 of 35

28 Appendix I: Summary of Methods A summary of modeling assumptions and methods is presented on subsequent pages. 2 3 b 3 c 3 d Model Used Load Load Forecast Uncertainty Behind-the-Meter Probabilistic Assessment Modeling Assumptions and Methods Summary Description ERCOT FRCC MISO Manitoba MAPP SaskPower NPCC Areas PJM SERC Areas SPP Name ProMaxLT TIGER GE MARS GE MARS GE MARS PROMOD GE MARS GE MARS GE MARS GridView PROMOD IV # Trials Total Run Time 35 hours 1 minute 39 hours 30 minutes 20 minutes 1 minute 30 minutes; 300 CPUs 30 minutes; 300 CPUs 17 minutes; 240 CPUs Model Type Monte Carlo Convolution Monte Carlo Monte Carlo Monte Carlo Convolution Monte Carlo Monte Carlo Monte Carlo Internal Load Shape External Load Shape Adjustment to Forecast Modeling Standard Deviation Uncertainties Considered Thermal Generation Variable Generation 15 years; Historical Weather data; years; years Seasonal Peaks Typical Year; 2005 Typical Year; 2005 Peak & Energy 7-step Discrete Distribution Typical year; 2002 Typical year; 2002 Peak & Energy 7-step Discrete Distribution Typical year; 2011 Peak & Energy see above 4.92% 4% Weather, forecast Resource Resource Weather, economic, forecast Within the Within the Weather, economic, forecast Weather, economic, forecast Typical year Resource Resource Within the or Annual Peaks & Energy High and Low forecast based on a 90% confidence interval Weather, economic, forecast Summer 2002 Winter 2004 Summer 2002 Winter 2004 Peak & Energy 7-step Discrete Distribution (not normal for all Areas) Varies by Area; asymmetrical Weather, economic, forecast Within the Within the Summer 2002 Winter 2004 Summer 2002 Winter 2004 or Annual Peaks & Energy 7-step Discrete Distribution Varies by PJM sub-areas, ~0.06 Weather, economic, forecast Within the Within the 10 years; Typical year; 2008 or Annual Peaks & Energy 7-step Discrete Distribution WECC Areas 27 hours 72 hours Monte Carlo Typical Year; 2005 see above 5% Weather, forecast Within the Within the Load Multiplier 7-step Discrete Distribution Weather, forecast 50% capacity; 50% within Within the Monte Carlo Typical year Seasonal Peaks Normal Distribution peak, 4% Weather, economic, forecast Within the Within the Future Resource 5 c Inclusion Criteria Same as with additional CCs & CTs with Approved GIAs with with with with with with with with 28 of 35

29 Appendix I: Summary of Methods Probabilistic Assessment Modeling Assumptions and Methods Summary Description ERCOT FRCC MISO Manitoba MAPP SaskPower NPCC Areas PJM SERC Areas SPP WECC Areas Retirement Criteria More retirements than with Attachment Y of the MISO Tariff with with with Exact definition varies by Area with with with with 6 a 4 Future Transmission Controllable Demand Response Inclusion Criteria Retirement Criteria Modeling Load shape / Derates /FOR ERCOT's 5 year plan Planned, permitted, and under construction Variable resource derated for utilization Energy- Limited Resource by # of times they could be called upon with with Interruptible modeled as a modifier Flat profile on weekly base None with with Energy- Limited Resource with Area planning processes with Area planning processes Applied as EOP steps or modeled as a resource Per PJM's Regional Transmission Expansion Plan (RTEP) process. Per PJM's Regional Transmission Expansion Plan (RTEP) process. Applied within EOP steps All permitting, under construction Consistenc e with SPP MDWG Powerflow models. Applied within EOP steps Nonvariable resource with Correlation to When modeled as EOP 29 of 35