Re: ASB Comments Comments on Second Exposure Draft of the Modeling ASOP

Similar documents
Re: ASB Comments Comments on Third Exposure Draft of the Modeling ASOP

Re: Proposed Actuarial Standard of Practice, Capital Adequacy Assessment for Insurers, Second Exposure Draft

Re: Comments Regarding Coordination Between Actuarial Standards of Practice (ASOPs) Involving Retirement Benefits.

Re: Comments on ORSA Guidance in the Financial Analysis and Financial Condition Examiners Handbooks

Re: Review of International Standard of Actuarial Practice 4 IFRS 17 Insurance Contracts Exposure Draft

Please consider the following comments on the Second Exposure Draft of the ASOP on Modeling.

May 8, Assessment and Disclosure of Risk Actuarial Standards Board 1850 M Street NW, Suite 300 Washington, DC Dear Sir or Madam:

January 30, Harlan Weller Government Actuary Department of the Treasury 1500 Pennsylvania Avenue, NW Room 4024 Washington, DC 20220

August 15, Submitted via to Annual Funding Notice Under ERISA Section 101(f) Dear Mr. Good:

Limited Guidance for Selecting Reasonable or Acceptable AVMs

August 07, Re: Regulation Identifier Number RIN 1210 AB20. To Whom It May Concern:

Re: Proposed Regulation 31 CFR Part 10 (REG ) [75 FR 51713]

Re: Pre-consultation comments on draft ICP revisions 4, 5, 7 and 8

RE: Discussion Draft of Statements of Principles Regarding Property and Casualty Insurance Ratemaking

Research Report. Premium Deficiency Reserve Requirements for Accident and Health Insurance. by Robert W. Beal, FSA, MAAA

June 30, Technical Director Financial Accounting Standards Board 401 Merritt 7 PO Box 5116 Norwalk, CT Dear Ms.

Pricing of Life Insurance and Annuity Products

November 6, Variable and Indexed Annuities in QLACs. Dear Mr. Iwry:

May 12, RE: Projection of Cash Balance Benefits. Dear Ms. Judson and Mr. Neis:

ASOP No. 41: Actuarial Communications and the Actuarial Standards Board

ASOP No. 1 March Appendix 2. Comments on the Exposure Draft and Responses

July 9, Office of Federal Procurement Policy th Street, N.W. Room 9013 Washington, DC Attn: Raymond J. M. Wong

Re: Exposure Draft on Pension Accounting and Financial Reporting by Employers

Michael Saunders Acting Director, Employee Plans Rulings & Agreements Market Street Philadelphia, PA 19104

December 20, Re: Notice of Benefit and Payment Parameters for 2015 proposed rule. To Whom it May Concern,

Please contact Bill Rapp assistant director of Public Policy at the Academy, if you have any questions.

We do have a few comments about the Exposure Draft which we believe should be considered.

Re: Comments on proposed rule for the Medicare Shared Savings Program: Accountable Care Organizations

RE: Comment Letter on APF to Keep Term and ULSG Separate in VM-20 Calculation to Reduce Allocation Concerns

Scenario and Cell Model Reduction

August 29, Dear Mr. Bean:

Interim Final Rule Health Insurance Issuers Implementing Medical Loss Ratio (MLR) Requirements under the Patient Protection and Affordable Care Act

RE: Preliminary Views on Economic Condition Reporting: Financial Projections

August 11, Fred Anderson Chair Indexed Universal Life Illustration Subgroup National Association of Insurance Commissioners

C1 Work Group Updated Recommendation of Corporate Bond Risk-Based Capital Factors

Responses to Request for Comments

May 2015 DISCUSSION DRAFT For Illustrative Purposes Only Content NOT Reviewed or Approved by the Actuarial Standards Board DISCUSSION DRAFT

October 16, The Honorable Nick Gerhart Chair, Variable Annuities Issues (E) Working Group National Association of Insurance Commissioners

October 4, Sent via to Julie Gann. Re: Exposure Draft Dear Mr. Bruggeman:

Concept Release on possible revisions to PCAOB Standards related to reports on audited financial statements

Annual statements for years 2012 and prior did not provide sufficient granular data for us to perform similar analyses.

RE: Recent FASB Educational Sessions on Long-Duration Insurance Contracts

Re: Proposed Operational Risk Factors and Growth Charge for the Life RBC Formula

Methods and Assumptions for Use in Life Insurance Company Financial Statements Prepared in Accordance with U.S. GAAP

Adopted by the Actuarial Standards Board September 2008 Updated March (Doc. No. 161)

Pricing of Life Insurance and Annuity Products

July 14, RE: Request for Feedback on the IAIS MOCE Proposal and the C-MOCE. Dear Tom,

Re: Interim Final Rules Relating to Coverage of Preventive Services

Re: Proposed changes to the Annuity Disclosure Model Regulation (#245)

CAS Task Force to Reply to the Modeling ASOP Exposure Draft

Document Identifier CMS CMS Medical Loss Ratio (MLR) Annual Reporting Form

July 31, Submitted electronically via

The Honorable Nancy Pelosi Speaker, House of Representatives H-232 U.S. Capitol Washington, DC November 5, Dear Madam Speaker:

Cavanaugh Macdonald. The experience and dedication you deserve

Alternatives for Pension Cost Recognition: Implementation Issues

September 27, Susan M. Cosper Technical Director Financial Accounting Standards Board 401 Merritt 7 P.O. Box 5116 Norwalk, CT

RED 2.1 & 4.2: Quantifying Risk Exposure for ORSA. Moderator: Presenters: Lesley R. Bosniack, CERA, FCAS, MAAA

February 14, Re: Regulator Questions on Proposed Factors for Bonds. Dear Mr. Fry,

Modeling by the Ceding Company and/or Reinsurer

Documentation in Health Benefit Plan Ratemaking

Actuarial Standard of Practice No. 24: Compliance with the NAIC Life Insurance Illustrations Model Regulation

Actuarial Standard of Practice No. 4. Measuring Pension Obligations and Determining Pension Plan Costs or Contributions.

MEMORANDUM. The Joint Committee on the Code of Professional Conduct. Revised Code of Professional Conduct. DATE: January 1, 2001

Office of the Secretary Public Company Accounting Oversight Board 1666 K Street, N.W. Washington, DC December 11, 2013

Property/Casualty Unpaid Claim Estimates

EXPOSURE DRAFT. Modeling. Comment Deadline: September 30, 2013

SUBJECT: Comments on the 2018 Proposed Revision of Actuarial Standard of Practice (ASOP) No. 4

Article from. In the Public Interest. January 2016 Issue 12

September 30, Technical Director Financial Accounting Standards Board 401 Merritt 7 PO Box 5116 Norwalk, CT

Synthetic GIC Reserve Proposal Supplement to November 2012 Proposal. Deposit Fund Subgroup of the. Annuity Reserves Work Group (ARWG)

Actuarial Opinions and ASOP Nos. 36 and 43

Minnesota Legislative Commission on Pensions and Retirement. Actuarial Review of Retirement Systems as of July 1, 2016

PROFESSIONALISM AND THE PRACTICING ACTUARY

The Trustees Report for the Old-Age, Survivors, and Disability

Presenters* Agenda. Copyright 2009 by the American Academy of Actuaries

May 19, Re: Investment Risk-Based Capital: A Way Forward. Dear Commissioner Fry:

Submission on the Solvency Standard Re-issue 2014

Comparison of ACA and STLD Coverage Requirements and Implications for the ACA Markets

APPLICATION OF PROFESSIONAL STANDARDS IN INTERNATIONAL PRACTICE

INTERPRETATIVE OPINION 3: PROFESSIONAL COMMUNICATIONS OF ACTUARIES. and INTERPRETATIVE OPINION 4: ACTUARIAL PRINCIPLES AND PRACTICES

Frequently Asked Questions on the U.S. Qualification Standards

EXPOSURE DRAFT. Measuring Pension Obligations and Determining Pension Plan Costs or Contributions

May Link Richardson, CERA, FSA, MAAA, Chairperson

Professionalism and the Practicing Actuary

Sent electronically through the IASB Website (

December 13, 2018 Internal Revenue Service Room 5205 Ben Franklin Station Washington, DC 20044

Re: Informational Bulletin: Notice to Actuaries Submitting Actuarial Summaries and Studies for Private Self-Insured Employers

Wipfli LLP 11 Scott Street, Suite 400 Wausau, WI PO Box 8010 Wausau, WI fax

Re: Comments on ASOP No. 6, Measuring Retiree Group Benefit Obligations

File Reference No Exposure Draft of a Proposed Accounting Standard Update - Revenue from Contracts with Customers

Issue 1 Required Procedures When an ERISA-Permitted Audit Scope Limitation is Imposed

Society of Actuaries Finalizes New Mortality Assumptions

III.B. Provisions and Parameters for the Permanent Risk Adjustment Program

TIC has reviewed the ED and is providing the following comments for your consideration. GENERAL COMMENTS

Comments on the Exposure Draft of the Proposed Revision of Actuarial Standard of Practice Number 4

Discounting of Property/Casualty Unpaid Claim Estimates

Mary D. Miller, MAAA, FCAS Academy Past President

The American Academy of Actuaries Duration Blanks Work Group Response to the NAIC Blanks Working Group Proposal. May 2011

Session 03PD: PBR Reporting and Disclosures Thinking About the End at the Beginning. Moderator: James Russell Collingwood ASA,MAAA

Statements of Actuarial Opinion Regarding Health Insurance Liabilities and Assets

Transcription:

March 1, 2015 Modeling (Second Exposure) Actuarial Standards Board 1850 M Street NW, Suite 300 Washington, DC 20036 Re: ASB Comments Comments on Second Exposure Draft of the Modeling ASOP Members of the Actuarial Standards Board: The American Academy of Actuaries 1 Pension Committee is pleased to present the following comments to the Actuarial Standards Board (ASB) regarding the second exposure draft of the proposed Actuarial Standard of Practice (ASOP) on Modeling. We have three primary areas of concern with the proposed standard scope, lack of clarity, and interaction with other standards as well as some miscellaneous comments that we wish to bring to your attention. Scope Creating an ASOP on modeling presents unique challenges because modeling is so integrated into the work actuaries do on a day-to-day basis. As a result, it is difficult to isolate modeling from functions, such as the selection of actuarial assumptions and methods that for pension actuaries are already thoroughly addressed in other ASOPs. Much of what is addressed in the exposure draft is an inherent and fundamental part of all actuarial work. We are concerned that by attempting to enumerate the components of modeling to develop what is essentially a checklist, it creates the false impression of uniformity of the modeling process. At the same time, this enumeration could either lead actuaries to do unnecessary validation work, or provide unhelpful disclosure as to why such additional work is inappropriate. While we do not think this standard will result in meaningful changes in the way actuaries do their work, it could lead to additional and unnecessary validation and documentation out of concern that the ASOP might otherwise provide a roadmap for another party to challenge the actuaries work, even where those challenges lack substance. Given that this is the first general standard on modeling, and in light of these concerns, we think it would be appropriate to soften the language throughout the standard to make it less 1 The American Academy of Actuaries is an 18,000+ member professional association whose mission is to serve the public and the U.S. actuarial profession. The Academy assists public policymakers on all levels by providing leadership, objective expertise, and actuarial advice on risk and financial security issues. The Academy also sets qualification, practice, and professionalism standards for actuaries in the United States. 1

prescriptive. Alternatively, if we are interpreting the language of the exposure draft to be more prescriptive than intended, then we think others also may do so and the draft would benefit from more detailed examples to clarify what is intended. For example, much of the content of the proposed standard is an elaboration of how an actuary might go about selecting and using models, or data, assumptions, methods and procedures as described in ASOP No. 41. The proposed standard includes many useful considerations of what constitutes sufficient detail so that another actuary practicing in the same area could form an objective appraisal of the reasonableness of the actuary s work, it is unlikely that every item in the list will apply to every actuarial project. Less prescriptive language could more appropriately identify the role of professional actuarial judgment in the process. Also, as we indicated in our response to the first exposure draft, we believe that only models above some minimal level of complexity should be explicitly covered by the ASOP. As an alternative, the actuary should be able to take the complexity of the model into consideration when using his or her professional judgment to determine whether full application of the guidance of the ASOP is appropriate. It is unclear why this ASOP is needed in situations involving straightforward calculations. For instance, in calculating a deterministic present value of a member s benefit entitlement, which can be done using pencil and paper, the actuary is using a model, as defined in the exposure draft. In situations such as these, the exposure draft does not seem to have any positive effect and might instead add cost and exposure to routine work. Therefore, we believe full application of the guidance may not be warranted due to the simplicity of the model, even if the financial effect could be material to the individual plan member. Alternatively, if the intent is that full application is not meant to be onerous in these more straightforward situations, then it would be helpful to provide some clarification to illustrate what specific actions and disclosures might apply in this situation and how, if at all, these might differ from current practice. In addition, we suggest splitting the standard into two or more parts (not all of which would necessarily ultimately become ASOPs). Lack of Clarity While we appreciate the work the ASB has put into updating the Modeling exposure draft, we believe that it still lacks clarity in several key areas: Section 2.7 was expanded from the prior exposure draft to add additional details. However, in doing, so it lost a lot of clarity. By expanding the words around intended application and project objective, it is no longer clear when each is applicable. In fact, the use of the phrase modifying, reviewing, or evaluating the model is used for both, which is confusing. The differences both in terms of definition and their application to issues throughout the ASOP, if meant to differentiate between creating a model and using a model, are not clear. Perhaps several examples covering several practice areas would clarify this, as would the use of strident phrases that give a clearer picture of the differences instead of intended application and project objective, such as creating a model and using a model. 2

For pension actuaries, one example of a common model that is used is the actuarial valuation program to calculate pension liability measures, the outputs of which are used to determine cash contributions for a pension plan. For this project most pension actuaries use a model that is created by a third party either from within their company or from an external vendor. In connection with this type of model, there are several specific roles actuaries assume, including: Developer - An actuary may be involved in the creation of the model itself, either in a development role, or in testing or reviewing the development. However, this actuary generally does not use the model to create results that are delivered to the Principal. They just deliver the model to another actuary.. A similar role might be an actuary responsible for validating a model and providing guidance for other users. One example might be an actuary (or group of actuaries) at a firm that purchases a pension valuation system from a developer, and who are responsible for understanding the workings of the model and providing guidance to other users at their firm. Although this is not technically a development role, it is similar in that other users may rely on this individual or group in the same way that they might rely on a developer. Customizer - An actuary may be involved in setting certain parameters in the model or using other computer coding techniques to adjust the model to be specific to pension plan and purpose (e.g., cash funding). In doing so, they rely on the work done by the Developer and produce output used in deliverables to the Principal. This can involve programming, or testing or reviewing the programming, of the plan provisions, assumptions and methods used in the model. Although not involved in developing the model itself, this person is involved in developing the project-specific version of the model. User- An actuary may rely on the work done by both the Developer and the Customizer via work that is done using the outputs from the model. We believe that the standards of practice for each of these role categories should be different. However, it is not clear from the exposure draft what the standards are for each of these categories or what terms in Section 2.7 are applicable to each category. One might be able to identify times during this project when all three role categories of actuaries are involved in modifying, reviewing or evaluating the model both before it has been selected or used in a specific project and when it is being selected or used in a specific project. We believe that the intent of the draft standard is not to have every actuary using a model be subject to the standards for both the intended application and project objective. However, it would be helpful to provide examples that clarify how actuaries paying the different roles described above might be expected to comply. In using models, there is often an actuary who relies on others involved at earlier stages in the project. Sections 3.1.2 and 3.1.3 refer to circumstances when actuaries rely on others. We agree with the response in Appendix 2 that says The reviewer believes that different guidance is warranted for actuaries using models developed by others. 3

However, it is not clear the extent to which the actuary described in each of these Sections is subject to the remaining requirements of the ASOP. It would be helpful to have clarification, perhaps in the form of examples, of how these roles limit the application of the remaining sections of the ASOP. Another input to our actuarial models has to do with actuarial methods, such as the method used to allocate the cost of pension benefits to the past, the current year and future years. There are several generally accepted methods, and some are prescribed by law or another party. This should be incorporated into the discussion of models in the standard. Section 3.17.3 of ASOP No.4 states: Inability to Evaluate Prescribed Assumption or Method If the actuary is unable to evaluate a prescribed assumption or method set by another party without performing a substantial amount of additional work beyond the scope of the assignment, the actuary should disclose this in accordance with section 4.2(b). We think this is a very important concept that is missing from the Modeling exposure draft and recommend that something similar be added. An actuary should not have to perform substantial additional work outside the scope of the assignment, and should only have to disclose that fact. This should be applicable to the discussion on assumptions, mitigation of model risk, and applicability of guidance. The disclosures in Section 4.3 of ASOP 41 could be expanded to cover models and the items listed in that section should be disclosed for prescribed assumptions or methods set by another party. Interaction with Other ASOPs It is still not clear how this standard interacts with some of the other ASOPs to which pension actuaries are subject. As structured, we believe this standard will engender confusion as to the appropriate standard to follow. Also, if there are important issues that should be included in the other ASOPs that were identified when drafting this standard, they should be added to the appropriate ASOP and not be added here just because this one is currently being drafted. Section 3.2.6 is very clear about how this ASOP interacts with ASOP No. 23; this should serve as an example of how to treat the interaction with other ASOPs. Pension actuaries have spent significant effort in providing input during the updating of ASOP Nos. 4, 6, 27 and 35 to discuss method and assumption setting. Wording similar to that of Section 3.2.6 should be used in Section 3.2.7 when actuaries are subject to ASOP Nos. 4, 6, 27 and 35. The ASOP could maintain the drafted guidance in the exposure draft for actuaries who aren t subject to these pension standards. However, having to follow two different sets of guidance for setting the same assumptions for the same model would be confusing and difficult for actuaries to follow. Section 3.4 and Section 4 both deal with communications and disclosures. The ASB has an entire ASOP, No. 41, dealing with this topic, which is applicable to the 4

communication of results and disclosure in connection with models for all practice areas and Section 3.6 of this exposure draft appropriately refers to it. If the ASB believes there is additional guidance needed on this topic for all practice areas, it should be added to ASOP No. 41. Having information in Section 3.4 and 4, instead of referring to ASOP No. 41 is confusing and would be difficult for actuaries to follow. Actuaries should be able to go to one ASOP for all issues applicable to communications in connection with general actuarial practice, which is why we expect the ASB produced such a standard; however, anything related to communications that is not applicable to general actuarial practice, and is specific to a particular practice area, should be found in ASOPs for that particular practice area. ASOP No. 4 also provides a lot of guidance about measuring pension costs and contributions, which are generally done using models. We believe reference should be made to this ASOP indicating that it is what should be followed for the discussion of specifications and implementation of models included in its scope. Other Comments In addition, we have a few miscellaneous comments: Section 2.6 uses the word designer and that word is not used elsewhere in the document. It is not clear what this is supposed to mean. If this is meant to be the individual who created the model, other options that might resonate better could be developer or programmer, or this could benefit from a clarifying example. Section 2.12 defines the word parameter as inputs to a model. In the pension actuarial valuation program example noted above, the items in the current definition are inputs to the model. However, another model input is related to the pension plan provisions, and common wording used to describe that input is parameter. For example, if a pension plan provides a flat dollar benefit of $45 per month per year of service, then the valuation model may have a parameter that can be set to that flat dollar benefit. We believe the definition should be expanded to include this concept. The second paragraph of Section 1.2 and the second paragraph of Section 3.1.1 have examples that may need further consideration. Both of these items can be very important steps in pension models and we are concerned with the wording that implies they are examples of steps that may not be important. Data validation may mean various things depending on the reader. We agree with the ASB, as stated in Appendix 2 The standard uses (and the reviewers use) validation to include a wide range of processes or even perspectives, including checking, recognizing that a wide range of models and terminology to describe them exists. The example in the two paragraphs may hold true if one considers data validation as going back to the original source of the data and independently validating every data element being used. However, one might also read data 5

validation to be a higher level review of the data for reasonableness and consistency. Clarification should be considered since the second type of validation is an important step that is covered in ASOP No. 23, while the first type of validation is generally be more rigorous than necessary. Consideration should also be given as to whether sensitivity testing should be used as an example. Sensitivity testing may not be appropriate or may not be included in the scope of services for which the actuary has been engaged. However, interest in the profession around the assessment of risk is growing. We suggest it might make sense to address the issue of sensitivity testing as something that the actuary should consider whether, in their professional judgment, it is appropriate for the given circumstance, instead of an example of something that might be too rigorous. We do not believe that sensitivity testing should always be required, since it may be too rigorous or beyond the scope of the project. Consideration should be given to adding something to Section 3.1.2 in regard to confirming the expertise of those who developed the model and assessing the level of care that was taken in developing the model. Although there are situations where the actuary may not have relevant information on qualifications and process, if the actuary does have this information it will be relevant in evaluating the model. Compliance with the guidance in section 3.13 may be impractical, if not impossible, in real world situations. Large complex models may be developed over extended periods of time, possibly years. It would not be possible to confirm that the applicable guidance from the standard has been applied. By that point, the actuary may no longer be part of the team. Even if still part of the team, the actuary may not have the authority to ensure compliance. As an example, consider the actuary who is part of a large project to build an actuarial valuation system that is expected to span multiple years. It is unlikely that the actuary will be the project manager and have the authority to force significant changes to the system throughout the terms of the model application. This requirement should be modified to clarify that it applies to the extent practicable and include a disclosure where it is not. Appendix 2 of the ASOP states The reviewers intend section 3.2.1 to be applicable to actuaries creating a model and section 3.2.2 to be applicable to actuaries using an existing model. That is not clear in the wording of both the headers and the text, especially due to the overlap of words like Reviewing and Evaluating. Consideration should be given to changing the headers to something easy to understand like Creating a Model and Using an Existing Model. The text could then expand on the concept in the header, for example, by indicating that Creating a Model includes designing, building, developing, reviewing or evaluating the model. 6

The example in Section 3.2.7(e) refers to financial reporting and needs to be more specific (for example, financial reporting for insurance companies ). Models used in financial reporting for pension plans do not offer frequent opportunities to compare assumptions and parameters to emerging experience. We are concerned by the phrase in Section 3.3.1(b)(4) that suggests that significant additional work might be appropriate to compare results of various models, which can be extremely labor intensive and expensive. This step is qualified only by the project objective and is not subject to limitations of the scope of the assignment or appropriateness in the actuary s professional judgment, both of which we think would be good qualifiers. The Pension Committee appreciates the opportunity to comment on this matter and would like to discuss these items with you at your convenience. Please contact Matthew Mulling, the Academy s pension policy analyst (202-785-7868 or mulling@actuary.org) to arrange a convenient time to discuss these items further. Sincerely, Michael F. Pollack, FSA, MAAA, EA, FCA Chairperson, Pension Committee American Academy of Actuaries 7