Model Governance: Is YOUR Company There Yet? Professional Interests & Stakeholder Perspectives. Moderator: Trevor C. Howes, FSA, FCIA, MAAA

Transcription

1 Model Governance: Is YOUR Company There Yet? Professional Interests & Stakeholder Perspectives Moderator: Trevor C. Howes, FSA, FCIA, MAAA Presenters: Larry J. Bruning, FSA, MAAA Trevor C. Howes, FSA, FCIA, MAAA David K. Sandberg, FSA, CERA, MAAA

2 Model Governance Is YOUR Company There Yet? Session 2 Professional Interests and Stakeholder Perspectives

3 What will you say if your models fail? All models are wrong. Some models are useful. and sometimes wrong models are dangerous. 2

4 Chemistry is not an exact science. Rio 2016 Spokesman Mario Andrada Photo by Agência Brasil 3

5 We need to be concerned about Model Risk What is model risk? the risk of adverse consequences resulting from reliance on a flawed model or a model that is misused or misinterpreted Effective Model Governance provides comfort to the intended users of model results that the model risk is understood and being effectively managed. 4

6 Who is concerned with Model Governance? Ratings Agencies Consumer Critics General Public Policyholders Regulators. Model Risk? Insurance Commissioners (NAIC) Federal Insurance Office (FIO) Federal Stability Oversight Council (FSOC) Federal Reserve Board (FRB) International Association of Insurance Supervisors (IAIS) Insurance Industry Actuarial Profession IASB (Global) Senior Management Boards of Directors Auditors AAA/ASB (US) CIA/ASB (Canada) FIA/FRC (UK) 5

7 Who will we hear from today? 1. Regulatory Viewpoint Larry J Bruning FSA, MAAA Hampton Finer 2. Industry/Company Perspective Dave Sandberg FSA, MAAA, CERA 3. Actuarial Profession Trevor Howes FCIA, FSA, MAAA 6

8 Model Governance Session 2 Professional Interests & Stakeholder Perspectives Wednesday, August 31, 2016 Larry J. Bruning FSA, MAAA 7

9 Model Governance Regulatory Perspectives U.S. State Insurance Regulator Perspective International Association of Insurance Supervisors (IAIS) Perspective 8

10 Modeling Authorized by Law & Regulation Life Insurance Reserves Asset Adequacy Testing ML 820 (SVL) & VM 30 Principle Based Valuations ML 820 (SVL) Life Insurance VM 20 Variable Annuities VM 21 AG 38 ML 820 (SVL) & VM C AG 48 ML 820 (SVL) & VM 30 Capital C 3 Phase I (Interest Rate Risk C 3a) ML 312 C 3 Phase II (Market Risk C 3c) ML 312 ORSA ML 505 9

11 Regulator Guides to Analysis & Examination Financial Analysis Handbook Purpose is to provide a uniform risk focused analysis approach for insurance departments to, among other things, identify prospective risks, within a firm, which could cause financial problems Financial Condition Examiners Handbook Purpose is to assist department examiners in conducting risk focused financial examinations to assess and monitor current financial condition and prospective solvency of a firm. 10

12 Regulator Guides to Analysis & Examination Both Handbooks are currently under revision to: Address Principle Based Valuations Model Governance Corporate Governance Financial Condition Examiners Handbook Maintained by Financial Examiners Handbook (E) Technical Group Financial Analysis Handbook Maintained by Financial Analysis Handbook (E) Working Group 11

13 Regulatory Expectations around Model Governance Model Governance is Focused on 5 Major Areas Security Process Model Software Change Process Model Parameter Setting Process Model Validation Process Oversight of Model Governance Process 12

14 Regulatory Expectations around Model Governance Security Process Controls on Access to Models (login, passwords) Specific Role Definitions Global Model Parameters Model Assumptions Model Inputs (Inforce, new business, assets, products) Model Outputs Model Execution Documentation Model Software Change Process Test vs. Production Controls around Testing (e.g. back testing, forward testing) Controls on moving test version to production Documentation 13

15 Regulatory Expectations around Model Governance Model Parameter Setting Process Global Model Parameters Model Assumptions Model Inputs Model Outputs Model Execution Documentation Model Validation Process Seriatim vs. Grouping (Assets, In force, New Business) Model validation criteria (counts, amounts, cash values account values ) Source of control totals (general ledger, administration systems) Documentation 14

16 Regulatory Expectations around Model Governance Oversight of Model Governance Process Board Level Committee or Individual Senior Management Level Committee or Individual Information Technology Corporate Actuary CRO.. 15

17 IAIS Expectations around Model Governance Insurance Core Principle (ICP) 14 Valuation The reliability of model results is enhanced through the use of insurers and supervisors best practices surrounding model governance, controls and independent review. Supervisory comparisons or benchmarking of modelling practices can further enhance the reliability of modelled results. ICP 16 Enterprise Risk Management for Solvency Purposes As such, the use of models itself generates risk (modelling and parameter risk) which, if not explicitly quantified, at least needs to be acknowledged and understood as the insurer implements its ERM framework, including the insurer s Board and Senior Management. 16

18 IAIS Expectations around Model Governance ICP 16 Enterprise Risk Management for Solvency Purposes Supervisors should review an insurer s internal controls and monitor its capital adequacy, requiring strengthening where necessary. Where internal models are used to calculate the regulatory capital requirements, particularly close interaction between the supervisor and insurer is important. In these circumstances, the supervisor may consider the insurer s internal model, its inputs and outputs and the validation processes, as a source of insight into the risk exposure and solvency position of the insurer. 17

19 IAIS Expectations around Model Governance ICP 17 Capital Adequacy Where a supervisor allows the use of an internal models to determine regulatory capital requirements, the supervisor requires: Prior supervisory approval for the insurer s use of an internal model for the purpose of calculating regulatory capital requirements; The insurer to validate an internal model to be used for regulatory capital purposes by subjecting it, as a minimum, to 3 tests: statistical quality test, calibration test, and use test ; and The insurer to demonstrate that the model is appropriate for regulatory capital purposes and to demonstrate the results of each of the three tests. 18

20 Session 2 Professional Interests and Stakeholder Perspectives Update on the Federal Reserve Views on Model Governance Hampton Finer 19

21 Session 2 Professional Interests and Stakeholder Perspectives A Life Company Perspective on Model Governance Dave Sandberg FSA, MAAA, CERA 20

22 Session 2 Professional Interests and Stakeholder Perspectives Model Governance and the Actuarial Profession Trevor Howes FCIA, FSA, MAAA 21

23 My Agenda Professional perspectives on Model Governance Guidance for regulators and external stakeholders oiaa Risk Book Chapter on Model Governance omodel Governance Work Group of the Academy Professional Standards of Practice on Modeling oglobal overview ohighlights of ASB ASOP Exposure Draft Significance to practicing actuary 22

24 IAA Risk Book Governance, Management and Regulation of Insurance Operations A collection of papers produced and approved by the Insurance Regulation Committee of the International Actuarial Association (IAA) to help ensure both the sustainability of insurance programs and the protection of their policyholders. 11 chapters published in past 12 months; 9 more in progress Draft of Chapter 15 Governance of Models is close to approval o Authors: Godfrey Perrott, Sheldon Selby, David Sherwood, Trevor Howes ( 23

25 IAA Risk Book Chapter 15 Governance of Models Target Audience is Regulators and other stakeholders Goal is to explore Models and Model Risk specifically from an insurance enterprise perspective, and describe the importance and nature of Model Governance Provide an actuarial perspective Review and extract key points from wealth of materials available on the subject; add specific points reflecting insurance risk models (Life and P&C) Not a comprehensive textbook Not professional guidance 24

26 IAA Risk Book Chapter 15 Governance of Models Structure of Chapter 1. Overview 2. Definitions 3. Background 4. Introduction to Models and Model Risk 5. Model Governance 6. Model Inventory 7. Model Validation 8. Summary and Conclusions Appendix Model Inventory Bibliography of Relevant Reference Papers 25

27 Life Practice Council oversees the Academy's public policy work on life insurance issues establishes and improves standards of actuarial practice for actuaries practicing in the U.S. 26

28 Life Practice Council (Jeff Johnson) Life Valuation Committee (Alice Fontaine) Life Principle based Approach Practice Note Work Group (Karen Rudolph) PBR Strategy Subgroup (Cande Olsen) Model Governance Work Group (Donna Claire) PBR Checklist Subgroup (Mike Failor) PBR Intensive Seminar Subgroup (Donna Claire) PBR Model Governance Practice Note Subgroup (Nadeem Chowdhury) 27

29 Model Governance Work Group Formed by PBR Strategy Subgroup in Nov 2015 Driven by view that modeling and model governance were becoming increasing important as introduction of PBR came closer Need to expand Life and Health Qualifications Seminar to include PBR and Model Governance? Hence formation MGWG under Donna Claire with 3 subgroups 28

30 Model Governance Work Group Activities PBR Intensive Seminar Subgroup Role of the Actuary what is responsibility of Qualified (PBR) actuary vs. appointed actuary vs. senior management? Running PBR boot camps PBR Model Governance Practice Note Questions and Answers on PBR Modeling PBR Model Governance Checklist Non exhaustive checklist for practicing actuaries and others involved in actuarial model governance

31 Global Progress on Actuarial Modeling SOPs 3 rd Exposure Draft ASOP on Modeling released July, 2016 comment by 10/31/ nd Exposure Draft of changes to Standards of Practice and an Ed Note released July 2016 comment by 9/30/2016 Final Drafts ISAP 1A Model Governance and ISAP 5 Insurer Enterprise Risk Models released 8/18/2016 comment by 9/18/2016 Technical Actuarial Standard M Modelling (effective April, 2010) 30

32 Proposed ASOP Modeling Key Definitions 2.7 Model A simplified representation of relationships among real world variables, entities, or events using statistical, financial, economic, mathematical, or scientific concepts and equations. Models are used to help explain a system, to study the effects of different parts of a system, and to derive estimates and guide decisions Simple Model A model wherein, in the actuary s professional judgment, the model results are transparent and can be predicted without an actual model run or readily obtained from an external source that is not another model. 31

33 Proposed ASOP Modeling Key Definitions 2.7 Model Risk The risk of adverse consequences resulting from reliance on a model that does not adequately represent that which is being modeled or that is misused or misinterpreted. 32

34 Scope of Proposed Modeling ASOP 1.2 Scope This ASOP applies to actuaries in all practice areas performing actuarial services when selecting, designing, building, modifying, developing, using, reviewing, or evaluating all types of models that are not simple models. If the model results are not heavily relied upon by the intended user, or do not have material financial effect, the requirements of this ASOP are limited to certain disclosure requirements in section

35 Application of Proposed ASOP 3.1 Application of ASOP Guidance The guidance in this ASOP applies to actuarial practice regarding models that are not simple models when, in the actuary s professional judgment, intended users of the model rely heavily on the results, and the use of the results of the model has a material financial effect for the intended user. 34

36 Key Principles of Modeling ASOP 3.4 Model Meeting the Intended Purpose The actuary should select, design, build, develop, or use a model that meets the intended purpose. 3.5 Mitigation of Model Risk The actuary should examine the potential for model risk and undertake reasonable and appropriate steps to mitigate such risk, using validation, governance, and controls, as appropriate to the intended purpose. 35

37 What does this mean to us? Models are increasingly complex, risky and critical to insurance company compliance and management Actuaries are used to being sole owners/managers of their models Actuarial workload to update, enhance, use models is increasing leaving little time to respond to external governance requirements We must recognize the personal and professional reputational risk we are exposed to from model risk 36

38 What does this mean to us? Interest and instigation for enhanced model governance and management of model risk are emerging from multiple stakeholders at once odifferent purpose, timing, focus, use of terminology? orisk of conflicting or redundant requirements and loss of ownership? Opportunity to influence policy and practice oavoid excessive and inappropriate regulation oserve the public interest oprotecting ourselves and our profession 37

39 Questions? 38