THE USE OF RISK ANALYSIS IN REGULATORY ANALYSIS. US Perspective. Clare Narrod

Transcription

1 THE USE OF RISK ANALYSIS IN REGULATORY ANALYSIS US Perspective Clare Narrod Joint Institute for Food Safety and Applied Nutrition University of Maryland

2 Outline of Presentation Risk analysis framework Major components of the Regulatory Impact Assessment Rulemaking process (establishment, requirements, and procedural flow) Reason why rules are challenged Example of cost to society of getting the analysis wrong

3 Red Book 1983

4 Risk Assessment seeks to answer What can go wrong? How likely is it to happen? What are the consequences if it does happen? Risk management seeks to answer How best mitigate a risk given the costs and benefits and the effectiveness of risk mitigation measure

5 Establishment of Rulemaking Process EO (All agencies) Required agencies to estimate net impact of benefits and costs of a regulation on society Required analysis to include the effect of proposed rule on state, local, tribal governments and businesses of different size Federal Crop Insurance Reform (FCIR) and Dept Agriculture Reorganization Act 1994 USDA required to conduct a regulatory risk assessment and costbenefit analysis under any proposed rule designated as major (over $100 mill. In 1994 $) Analysis had to make clear Nature of risk Alternative ways to mitigate the risk Reason for justifying the proposed rule Comparison of the likely costs and benefits of reducing the risk of the proposed rule

6 Rule Making Process (cont.) OMB Circular A-4: Encourages agencies to conduct a cost-effectiveness analysis whenever possible Required for all major rulemakings for which primary benefits are improved public health and safety where valid effectiveness measures can be developed and benefits can not be measured. Proposed rules as of Jan Final rules as of Jan

7 MAJOR COMPONENTS OF THE REGULATORY IMPACT ASSESSMENT Proposed Regulation Combined these analyses, consistent or not, makeup important components of Cost-Benefit decision-making. Analysis E.O Regulatory Risk Assessment 7 U.S.C. 2204e (b)(1) Cost-Effective Analysis Circular A-4

8 Informal Rulemaking process Step 1: Step 2: Step 3: Step 4: Step 5: Initiating Event Determination whether a rule is needed Preparation of proposed rule OMB Review of proposed rule Publication of proposed rule Step 6: Step 7: Step 8: Step 9: Public comment on rule Preparation of final rule/ Interim Final Rule OMB Review of final rule/ Interim Final Rule Publication of Final Rule/Interim Final Rule From ICF consulting

9 Rule Making Process Transparent Proposed Rules, Final Rules, and emergency rules are published During the comment period (and afterwards) they are often criticized by special interest groups Domestic/international Public/private actors

10 Rules Reality Developed within tight deadlines. Efforts may be conducted under separate chains of command, even within the same agency. INCONSITENCIES open the rule making to attack.

11 Ex. Baur v. Veneman, 12/16/03 Statute Federal Meat Inspection Act; Federal Food, Drug and Cosmetic Act Ban on use of downer livestock requested Justification based on Risk Assessment The appellate court found Baur had standing to bring a federal suit. The % chance of exposure to BSE from contaminated beef is a sufficient injury in fact.

12 Ex. Harlan Land Co. v. USDA, 9/27/01 Statute Plant Quarantine Act Petitioner concerned about rule allowing importation of citrus from Argentina Justification based on Supplemental plant risk assessment Cause of action- Admin. Proc. Act - arbitrary and capricious agency action based on Sec. 706 because APHIS failed to identify a negligible level of risk Action - Suspension of APHIS rule

13 Reasons Rules Attacked Failure to consider important data Data consideration was in the wrong part or in the wrong manner Basis for assumptions not articulated Wrong assumptions made (fruit, box, carton, shipment, quota) Impossible to duplicate manner and method of arriving at number (formula is not evident or transparent) Faulty math (OGC s views)

14 Timing of the Analyses Risk assessments often conducted ex ante to aid in development of regulatory policy Cost/benefit analysis most often done ex post to justify for regulatory action (Executive Order 12866) Rarely is attempt made to integrate the two to aid in rational decision making from the start

15 REGULATORY ANALYSES MUST BE CONSISTENT Regulatory Risk Assessment Baseline (no rule) Risks Proposed Rule Option Other options Risk Reduction Cost-Benefit/Effectiveness Analysis Baseline (no rule) Costs Proposed Rule Option Other options Benefits/Effectiveness

16 Optimal Regulatory Policy Acceptable policies: E(Benefits) E(Costs) Consistent w/ Executive Order What OMB/White House looks for Optimal policies: E(MB) = E(MC) Maximizing expected net benefits

17 Optimal Quarantine Policy W(φ) Figure 1 Optimal quarantine policy Efficient policy frontier W(φ*) A C D F B E Sub-optimal C(φ*) C(φ) E(Marginal Benefits) = E(Marginal Costs)

18 Karnal Bunt (KB) example KB is a disease affecting wheat, rye, and triticale (a hybrid of wheat and rye) caused by Tilletia indica Mitra Poses no risk to human health Can cause production losses to wheat reduced yields and reduction of quality of wheat flour generally wheat containing more that 3% of bunted kernels is considered unsatisfactory for human consumption because of fishy odor

19 Spread of KB detected in Haryana, India near city of Karnal s spread to Pakistan, Iraq, Afghanistan, Nepal, and Iran 1970 appeared in Mexico (Sonora, Sinaloa, and Baja California Sur) 1982 wheat kernel infected with KB were intercepted in wheat imported from Mexico

20 Impact of this Detection at the Borders of the US USDA took action to prevent the importation of host plant material and any articles that might spread the disease - as preventive action (7 CFR ) actions made permanent and restrictions on wheat made for all countries where Karnal bunt was known to occur (7 Code of Federal Regulations ) March 8, 1996, KB detected in Arizona during a seed certification inspection March 20, a Declaration of Extraordinary Emergency signed authorizing the Secretary to take emergency action with regard to KB within Arizona, New Mexico and Texas

21 Early Risk Assessments because of the close proximity of wheat growing areas of Arizona and California to infested areas in Mexico transport of the KB pathogen is extremely likely KB was a high risk pest, primarily because wheat from infested areas would probably be denied or restricted access in the export market Recommended that in the event of introduction of the KB pathogen national quarantines should be established to restrict distribution so as to protect integrity of U.S. wheat export market.

22 Protocols 1) Restrictions on the movement of positive-tested grain and seed outside the quarantine area, but allows all negative-tested grain and seed to move without significant additional restrictions 2) Requirements that all railcars be cleaned after delivery of wheat from the quarantined area 3) Restrictions on the movement of negative-tested seed outside of the quarantine area 4) Requirements for heat treatment of millfeed from quarantine-area wheat

23 Estimating Benefits and Cost of the Federal Quarantined Program Calculate the welfare effects of an outbreak of KB outside of the regulated area Main goal of the regulation was to protect export markets During 1997/98 US typically exports 1.2 billion bushels annually (approx. $3-4 billion) 1/2 countries when KB was discovered had restrictions against wheat imports from countries in which KB was known to occur USDA argued that failure to implement the quarantine would jeopardize trade with those countries

24 Final KB Regulations on Compensation Concluded: our quarantine measures were appropriate and justifiable when compared with the magnitude of the benefits achieved. Even a 10-percent reduction in wheat exports would have a significant effect on wheat sector income. It is estimated that a 10-percent decline in wheat exports would cause a decline in wheat sector of over $500 million. (62 FR 24765) But can these conclusions be justified if one examines the expected costs and benefits of the regulations?

25 Alternative Approach Applied the alternative model of quarantine policy Utilized the original risk assessment models to capture the potential spread of Karnal bunt Consider potential benefits and costs of the regulations based on the original analysis Examine the expected costs and benefits of regulations incorporating information on the distributions of potential outcomes given various regulatory actions using the alternative model of quarantine policy

26 The effects of various protocols on the risk of Karnal Bunt outbreak Protocol Probability of an outbreak 1/ For individual pathway Overall Railcar cleaning: - with 6.43 x x 10-3 Greatest overall effect - without 5.18 x x 10-2 Restrictions on the movement of negativetesting seed: - with x without 1.40 x x 10-2 Millfeed treatment: - with 1.66 x x without 6.59 x x / Evaluated at mean. 2/ Assumes no smuggling Greatest individual risk reduction, but doesn't change overall risk at all Original analysis only considered individual pathway

27 Table 7 Expected costs and benefits of alternative quarantine actions (million dollars) Quarantine Option Expected benefits Expected costs Net Option 1--Baseline 1/ 1, ,896.1 Option 2--Railcar cleaning 2, ,005.5 Option 3 Restrictions on seed movement 1, ,892.9 Option 4 Millfeed treatment 1, ,868.3 Option 5 Railcar cleaning; restrictions on seed movement Option 6 Railcar cleaning; millfeed treatment Option 7 Restrictions on seed movement; millfeed treatment 2, , , , , ,864.9 Option 8 Railcar cleaning; restrictions on seed movement; millfeed treatment 2, , / Includes prohibition of movement of positive testing grain and seed from quarantined area; all negative testing grain and seed moved in sealed hopper cars; all combines disinfected before leaving quarantined area.

28 Figure 2 Karnal Bunt Quarantine Alternatives 2.1 RR RR + S RR + MF RR +S+ MF Benefit (Bil $) Cost (Mil $) Railroad car cleaning most beneficial

29 Conclusions Original risk assessments focused on individual pathways without regard to the effect on the overall level of risk; overstated effects of individual protocols Original regulatory impact analyses ignored the effects of the quarantine policies on consumers; overestimated the benefits of the quarantine Original RIA failed to look at the expected marginal benefits and costs of various quarantine alternatives If expected costs and benefits considered, two of the more controversial protocols--seed restrictions and the millfeed requirements--may have received closer scrutiny Cost to society of these actions (including compensation) was between $350 million and $390 million

30 Implications for the future Where possible, integrate risk assessments to analyze expected costs and expected benefits Proper measurement of baseline risks Explicit assumptions on risk premium Compensation should be used to ensure compliance not to offset suboptimal policy