Socio-Technical Risk Analysis Industry Affiliates Program International Topical Meeting on Probabilistic Safety Assessment and Analysis 2017 Protective Systems: Definitions and Terms in the Regulated Risk Assessment Setting Ha Bui a,g, Tatsuya Sakurahara a,g, Justin Pence a,g,h, ZiHui Gu a,f, Zahra Mohaghegh a,g,h, Seyed Reihani a,g, Martin Wortman b, Ernie Kee a,g,*, Vera Moiseytseva c, David Johnson d a University of Illinois at Urbana-Champaign b Texas A&M University, Industrial Engineering c YK.risk, LLC, Bay City, TX d B. John Garrick Institute for the Risk Sciences, UCLA, Los Angeles, CA e South Texas Project Nuclear Operating Company (STPNOC), Wadsworth, TX f College of Law g Nuclear, Plasma, and Radiological Engineering h Illinois Informatics Institute soteria.npre.illinois.edu
soteria.npre.illinois.edu Who Are We Working With? Department of Energy (DOE) funding for Systematic Enterprise Risk Management by Integrating the RISMC Toolkit and Cost-Benefit Analysis [2017-2020] National Science Foundation (NSF) funding for A Big Data-Theoretic Approach to Quantify Organizational Failure Mechanisms in Probabilistic Risk Assessment [2015-2020] Industry-Sponsored research for Risk-Informed Solution for Generic Safety Issue 191 (GSI-191) [2013-2017] 2
What Are We Doing? soteria.npre.illinois.edu 3
soteria.npre.illinois.edu 4 Outline I. Introduction II. Setting A Common Language for Protective System Risk Analysis A. A System of Protective Systems B. Definitions and Terms 1. Conservative 2. Design-Basis 3. Beyond Design-Basis 4. Deterministic 5. Prescriptive 6. Risk III. Conclusions
soteria.npre.illinois.edu 5 A Need for A Common Language for Protective System Risk Analysis A primary challenge to profit-making in a regulated setting is the unavoidable tension that arises between the owner/investor and the regulator when evaluating the necessary cost for protection against catastrophic failure: Owner/Investor: profit-seeking Regulator: expecting the owner to meet an adequate protection standard In some cases, the lack of a common language may lead to: Misinterpretation or misunderstanding between the two parties. Root causes: Lack of clear definitions Potential Impacts: Inefficiencies in operating procedures, Undesirable costs, Inadequate protection management.
soteria.npre.illinois.edu 6 A Need for A Common Language for Protective System Risk Analysis The NRC should develop and implement guidance for use in its security regulatory activities that uses a common language with safety activities and harmonizes methods with risk assessment and the proposed risk-informed and performance-based defense-in-depth framework. Apostolakis, G., "A Proposed Risk Management Regulatory Framework (NUREG-2150)" (2012) In pursuit of this common language, the terms used in describing risk-related activities should be well defined. beyond design basis, conservative, design basis, deterministic, performance-based, prescriptive, risk, risk-informed, uncertainty, etc.
soteria.npre.illinois.edu 7 Outline I. Introduction II. Setting A Common Language for Protective System Risk Analysis A. A System of Protective Systems B. Definitions and Terms 1. Conservative 2. Design-Basis 3. Beyond Design-Basis 4. Deterministic 5. Prescriptive 6. Risk III. Conclusions
soteria.npre.illinois.edu 8 A System of Protective Systems A System of Protective Systems: A combination of protective systems that are designed with redundancy (successive, independent barriers to catastrophic failure) A system of protective systems that includes four protective systems (A, B, C, and D) in a multi-barrier DID arrangement
soteria.npre.illinois.edu 9 A System of Protective Systems Example System Failure and Success Probabilities Risk curves for 2 alternatives An event tree representation of the system of protective systems
soteria.npre.illinois.edu 10 Outline I. Introduction II. Setting A Common Language for Protective System Risk Analysis A. A System of Protective Systems B. Definitions and Terms 1. Conservative 2. Design-Basis 3. Beyond Design-Basis 4. Deterministic 5. Prescriptive 6. Risk III. Conclusions
soteria.npre.illinois.edu 11 Conservative Context: Faced with designing a protective system or system of protective systems, an engineer will need to decide on one or more functional performance metrics that must be met for success of the system. Example: Pump design for a protective system: V = Q ρ in (h out h in ሻ The pump should be sized to provide the V flow rate to meet the design. Q: required energy removal rate (J/s) h: enthalpy (inlet, outlet) (J/kg) ρ in : inlet density (kg/m 3 ) V: required pump flow rate (m 3 /s) But the engineer might want to ensure that the design will be robust, hence, make assumptions about the design conditions that would help ensure adequacy of the calculated flow requirement. Assumptions on the performance metrics can be made so that the pump volumetric flow is determined to be greater than if optimistic values were used for one or more performance metrics.
soteria.npre.illinois.edu 12 Conservative When values assumed for performance metrics used in developing requirements for the design of protective systems or systems of protective systems are biased to overestimate the protective systems' required performance for a specific functional requirement, they are thought of as conservative. The NRC s glossary only defines Conservative Analysis : An analysis that uses assumptions such that the assessed outcome is meant to be less favorable than the expected outcome. U.S. Nuclear Regulatory Commission, "Glossary of Risk-Related Terms in Support of Risk-Informed Decision making (NUREG-2122)" (2013) However, the term conservative (and conservatism) is used in different contexts without being clearly defined: level of conservatism, conservative safety margins. demonstrably conservative analysis, conservative methods, unnecessary conservatism, conservative deterministic failure margin method,
soteria.npre.illinois.edu 13 Conservative We propose a narrower definition in which conservative or ( conservatism ) can be thought of as: the practice (or describes the practice) wherein one or more parametric values used in the determination of a specific functional requirement for a protective system or system of protective systems are intentionally biased for the purpose of increasing the capability of the as designed system against the functional requirement. To be more specific, conservative (or conservatism): Applies to Protective systems or systems of protective systems Intentionally biasing parametric values when determining the assumed capability for a specific functional requirement, such that the capability against the requirement is increased The parameters used in the model of functional capability Does not applies to Systems having no protective function Unbiased values (intentional or otherwise) assumed in determining the requirement for a specific function The actual or expected performance in service
soteria.npre.illinois.edu 14 Design-Basis The NRC s glossary defines: Design-basis (DB) accident : A postulated accident that a nuclear facility must be designed and built to withstand without loss to the systems, structures, and components necessary to ensure public health and safety. Design-basis phenomena : Earthquakes, tornadoes, hurricanes, floods, etc., that a nuclear facility must be designed and built to withstand without loss of systems, structures, and components necessary to ensure public health and safety. Design-basis threat : A profile of the type, composition, and capabilities of an adversary. The NRC and its licensees use the design basis threat (DBT) as a basis for designing safeguards systems to protect against acts of radiological sabotage and to prevent the theft of special nuclear material. The DBT is described in detail in Title 10, Section 73.1(a), of the Code of Federal Regulations [10 CFR 73.1(a)]. Nuclear facility licensees are expected to demonstrate they can defend against the DBT. For further detail, see Protecting Our Nation. https://www.nrc.gov/reading-rm/basic-ref/glossary/design-basis-accident.html
soteria.npre.illinois.edu 15 Design-Basis The term Design-basis is used in those different contexts but is itself undefined. It is desirable to understand the relationship between Design-basis and protective systems (or system of protective systems) and NPP regulated activities. Basic relationship between Design-basis and protective systems could be summarized as follows: Applies to Protective systems or systems of protective systems under regulation The regulated design and operational activities of a protective system or system of protective systems' in-service function Does not applies to Protective systems or systems of protective systems that are not under regulation Actual in-service performance of a protective system or systems of protective systems' in-service function Design-basis: refers to the scope of regulated design or operational characteristic of a protective system or system of protective systems.
soteria.npre.illinois.edu 16 Beyond Design-Basis The NRC s glossary defines: Beyond Design-basis (BDB) accident : This term is used as a technical way to discuss accident sequences that are possible but were not fully considered in the design process be-cause they were judged to be too unlikely. (In that sense, they are considered beyond the scope of design-basis accidents that a nuclear facility must be designed and built to withstand.) As the regulatory process strives to be as thorough as possible, beyond designbasis accident sequences are analyzed to fully understand the capability of a design. https://www.nrc.gov/reading-rm/basic-ref/glossary/design-basis-accident.html To bring more clarity, it should be stated that it is not possible for all accident sequences to be analyzed, and unreasonable to assume that the actual capability of any particular NPP design can be fully understood under all scenarios. It seems reasonable to narrow the scope of BDB by saying what it is not. That is, having defined DB, BDB is what lies outside of DB. Beyond Design-Basis: refers to any characteristic, or characteristics, of a protective system or system of protective systems whose characteristics are outside of the DB scope (not under regulation).
soteria.npre.illinois.edu 17 Deterministic Regulatory requirements that add a Factor of Safety (FoS) are common for protective systems. Like the discussion on conservative, the FoS intends to increase the capability against a specific performance requirement. However, in regulation and regulatory guidance on regulation, where epistemology may be lacking, a method referred to as deterministic design is indicated. Deterministic design can be accomplished by adopting an alternative physical model or by adopting coefficients and/or bias in a well-understood physical model of performance or by adopting both. That is, if a well-understood model of performance is g( ሻ and an alternative model is f( ሻ then conservative results can be defined as: Where: xҧ is a vector of coefficients, f χҧ g f( ሻ may be either functionally equivalent to g( ሻ or it can be an alternative model χҧ is a vector of coefficients chosen to maximize f( ሻ Conservative may be thought of as deterministic when the two concepts produce equivalent results. xҧ
soteria.npre.illinois.edu 18 Deterministic Deterministic design requirements are most commonly discussed in contrast to performancebased, risk-based, or risk-informed design approaches; for example, the following quote is from the Federal Register: The NRC established its regulatory requirements to ensure that a licensed facility is designed, constructed, and operated without undue risk (with adequate protection) to the health and safety of the public. These requirements are largely based on deterministic engineering criteria. Simply stated, this deterministic approach establishes requirements for engineering margin and for quality assurance in design, manufacturing, and construction. In addition, it assumes that adverse conditions can exist (e.g., equipment failures and human errors) and establishes a specific set of design-basis events. It then requires that the licensed facility design include safety protective systems capable of preventing and/or mitigating the consequences of those design-basis events to protect the public health and safety. The deterministic approach contains implied elements of probability (qualitative risk considerations), from the selection of accidents to be analyzed as design-basis accidents (e.g., reactor vessel rupture is considered too improbable to be included) to the requirements for emergency core cooling (e.g., safety train redundancy and protection against a single failure). U.S. Nuclear Regulatory Commission, "Use of Probabilistic Risk Assessment Methods in Nuclear Regulatory Activities; Final Policy Statement" (1995)
soteria.npre.illinois.edu 19 Deterministic A more general perspective, developed through classic engineering risk analysis, may differ from the NRC's perspective regarding deterministic design, where prescriptive and deterministic may be differentiated. Accepted engineering FoS provide a clear quantitative definition of margin to failure from the design point including the type of failure. We suggest that Deterministic term: Applies to Does not applies to Uncertainty in a design choice regarding a specific performance requirement Uncertainty where performance cannot be determined from well-known physical design principles that make use of a FoS Protective systems Design requirements that include a FoS 1.0, that is no margin to failure The design itself or the way a design is fashioned Accepted methods for FoS (i.e., accepted engineering standards) Systems having no protective function Design requirements without FoS Deterministic: refers to a performance-specific protective system design requirement that attempts to account for uncertainty where well-accepted engineering FoS are lacking or nonexistent.
soteria.npre.illinois.edu 20 Prescriptive The term Prescriptive can be aligned with a risk assessment perspective, where scenarios developed in an event sequence consist of combinations of various functionally unreliable elements representing systems, components, or subcomponents. That is, risk is reduced as systems are added in parallel arrangements (systems of systems) such that larger numbers of failure must occur for catastrophic failure. Therefore, prescriptive, as defined below, is not directly related to deterministic or FoS. Instead, we suggest that the Prescriptive term: Applies to A protective system or system of systems design A required design configuration intended to reduce the probability a protective system or a system of systems will not function when needed Accomplishing an objective protective function Does not applies to Systems having no protective function The probability of failure for individual elements within a system or system of systems Functions without a protective objective Prescriptive: refers to a protective system or system of systems functional-level design configuration requirement that would increase the probability a protective function would be met.
soteria.npre.illinois.edu 21 Risk A regulatory objective is protecting the public, local populations, and near neighbors against consequences associated with catastrophic failures of protective systems. Catastrophic failure can occur, creating negative externalities for near neighbors. The regulator attempts to act on their behalf such that protective system design, operation, and maintenance are conducted in a way that those exposed are provided with adequate protection from harm. In this setting, risk refers to the adequate protection afforded; a measure of protection adequacy. The extent to which regulatory guidance drives a profit-making enterprise away from a riskneutral operating point is an indicator of the risk that may be considered by investors seeking to reduce the cost of production. Any costs associated with inspection, design, and operation that could be avoided in the absence of regulatory oversight would be associated with the level of adequate protection, and therefore, risk. A logical view, based on the belief that the regulator would rationally choose options that increase the level of adequate protection regardless of cost.
soteria.npre.illinois.edu 22 Risk Applies to The difference between the probability of failure of a particular functional requirement of a protective system or system of protective systems under regulation and under risk-neutral design, operation, or maintenance Is correlated with The cost of design, operation, or maintenance of a protective system or system of protective systems under regulation Does not applies to The probability of failure of protective systems or systems of protective systems below the riskneutral operating point Is uncorrelated with The cost of design, operation, or maintenance of a protective system or system of protective systems below costs incurred up to the risk-neutral operating point Risk: refers to the difference between the probability of failure of a protective system under regulation and under risk-neutral design, operation, or maintenance.
soteria.npre.illinois.edu 23 Outline I. Introduction II. Setting A Common Language for Protective System Risk Analysis A. A System of Protective Systems B. Definitions and Terms 1. Conservative 2. Design-Basis 3. Beyond Design-Basis 4. Deterministic 5. Prescriptive 6. Risk III. Conclusions
soteria.npre.illinois.edu 24 Conclusions This paper reports on the initial results of ongoing research to develop definitions for some common, but unclarified, terms that are used in settings where risk assessments or risk analyses are conducted by profit-making entities in regulated, high-consequence industries. Efficient communication among different organizations (i.e., industry, the regulatory agency, and academia) was, at least partly, impaired by the lack of a common understanding of the terms being used. Misunderstandings of the terminologies used in risk-informed applications usually occur at the interface of domains (e.g., deterministic vs. probabilistic, technical vs. social/human/organizational) and/or because organizations have different objectives. The lack of common language may result in unexpected safety issues, delays, rework, and economic losses; therefore, the efficiency and effectiveness of risk-informed decisionmaking may be negatively affected. Risk analysis is emerging as an important tool in the NEI Nuclear Promise initiative to reduce costs and establish clear communication paths, which are central in engineering work; hence, developing a common language is important for future development and multidisciplinary collaborations.