Session 8: The Latest on Practical Uses of Big Data and Predictive Analytics. Moderator: Phil Murphy

Transcription

1 Session 8: The Latest on Practical Uses of Big Data and Predictive Analytics Moderator: Phil Murphy Presenters: Ron Schaber Tim Hill Derek Kueker Jean Marc Fix Chris Stehno

2 PRACTICAL USES OF BIG DATA AND PREDICTIVE ANALYTICS August 4 th, 2015 Ron Schaber, 2 nd Vice President IUS

3 INNOVATION If you always do what you always did, you will always get what you always got. Albert Einstein

4 Thoughts, observations and predictions The future of predictive analytics Thoughts and observations: Easy to see how it should be done; not nearly so easy to do New data sources will help Effective use of business analytics is essential Some distribution channels work better than others Be prepared to make adjustments you won t get it right the first time Practical Uses of Big Data and Predictive Analytics - Ron Schaber August 4,

5 Is North America ready for big data? If insurance companies can t modernize their legacy mainframe systems, they will not be able to keep pace with fast-paced, rapidly changing, mobile, consumers. Those companies that modernize will rise to the top and create new products and services that the industry hasn t even seen yet. I think competition will bury un-modernized companies that continue to manage via legacy systems and intuition. We are using multiple systems that aren t working with each other and this presents many issues for our data management and analysis. There is data available that we aren t able to use... and data that we aren t capturing that would be valuable if only we had a place to store it and access it. Access Anytime, Anywhere: 2012 Emerging Technology Top Trends, LIMRA Practical Uses of Big Data and Predictive Analytics - Ron Schaber August 4,

6 Thoughts, observations and predictions The future of accelerated underwriting Predictions: As an industry we will get to fully underwritten pricing using accelerated underwriting If we don t figure it out, somebody else will (i.e. watch out for Google and Amazon) Practical Uses of Big Data and Predictive Analytics - Ron Schaber 5 August 4,

7 Published with permission Practical Uses of Big Data and Predictive Analytics - Ron Schaber August 4,

8 Published with permission Practical Uses of Big Data and Predictive Analytics - Ron Schaber August 4,

9 Predictive modeling and underwriting Streamlined/automated underwriting paradigm Big data vendors: thousands of data points Reinsurers: decades of mortality data Fully underwritten decisions Real-time basis Utilize non-fcra data 80% accuracy Practical Uses of Big Data and Predictive Analytics - Ron Schaber August 4,

10 Example of a new rule in theory Hypertension today App/interview questions Self-reported Exam, Rx, MIB, Inspection, and/or APS Run findings against rules Hypertension tomorrow Signed authorization No questions or other requirements Check info from wearable/internet of Things, MIB, MVR, Rx Run findings against rules

11 Hurdles between theory and reality Reliable Marketing Time of application Ongoing monitoring Legal/discriminatory Reportable Ethics & values Regulations Adoption

12 Looking forward for rules impact of data Wearables Internet of things Social media

13 WHAT WILL TOMORROW BRING? Practical Uses of Big Data and Predictive Analytics August - Ron Schaber 4,

14 Post-Level Term Period Lapses and Predictive Modeling Underwriting Issues & Innovation Seminar August 2015 Jean-Marc Fix, FSA, MAAA, VP, R&D, Optimum Re Insurance

15 Agenda Predictive modeling as another tool Pros and Cons An example: post-level term lapses Lessons Next steps 14

16 Predictive Yada Yada Yada Predictive modeling fatigue Nothing magical 15

17 Maslow s Hammer or Birmingham Screwdriver When all you have is a hammer, everything looks like a nail. or Give a small boy a hammer and he will find that everything he encounters needs pounding * * Abraham Kaplan, The Conduct of Inquiry: Methodology for Behavioral Science 1964 via Wikipedia 16

18 Kaplan Corollary It comes as no particular surprise to discover that a scientist formulates problems in a way which requires for their solution just those techniques in which he himself is especially skilled * 17 * Abraham Kaplan, The Conduct of Inquiry: Methodology for Behavioral Science Chandler Pub Co 1964, 4 th printing p28 (can be found on Google Books)

19 Just Another Tool But a flexible tool Based on applied statistics Good at measuring systematically relationship and interaction (Reasonably) easy to use Can be used for traditional actuarial modeling 18

20 Pros and Cons A systematic approach at analysis and modeling Handy set of tools: simplify life Pygmalion: over investment in model s neatness Not a substitute for judgment But actuaries are content expert 19

21 Study Lapse Modeling for the Post-Level Period, a Practical Application of Predictive Modeling November 2014 Richard Xu and his team at RGA Look at the appendices (in Excel) 20

22 Research Team Richard Xu Dihui Lai Minyu Cao Scott Rushing Tim Rozar 21

23 Project Oversight Group Jean-Marc Fix (chair) William Cember Andy Ferris John Hegstrom Christine Hofbeck Steve Marco Dennis Radliff Steve Siegel (SOA) Barbara Scott (SOA) 22

24 Model Generalized Linear Model (GLM) Think of it as a generalization of a linear regression by transforming the variables 23

25 GLM Three elements A dependent variable Y sampled from an exponential family distribution A linear predictor β of the explaining variables X A link function such that: E(Y)= (β X) 24

26 Model Here assumes Y (lapses) follow a Poisson distribution Probability Mass Function Poisson is good for modeling number of independent events per unit of time Result follows a multiplicative model (link function for Poisson is log) Poisson pmf" by Skbkekas - Own work. Licensed under CC BY 3.0 via Wikimedia Commons 25

27 Data Need clean data No different than any other modeling Data was from a previous study on post level term lapses Split data in two : train and test sets Guard against over fitting 26

28 Train and Test Split the data in two: Train set to build the model Test set to validate the model Can split randomly or by year Proportion varies between 50/50 and 80/20 depending on how much data you have (more data => smaller validation set size) Biggest danger in modeling is over fitting 27

29 Over Fitting Illustrated: Data

30 Over Fitting Illustrated: Model R^2=

31 Over Fitting Illustrated: Model R^2=

32 Over Fitting Illustrated: The Truth Y=X^1.5 31

33 Over Fitting Illustrated: What s Next?

34 Over Fitting Illustrated: What s Next? Model 1 : 103 Model 2: 45 True Value:

35 Variable Selection Pick most important variable Run model Add a variable Run model Is model better? Repeat until satisfied Also try non linear, pre-temporal and interaction variables : art comes in 34

36 Warning Numbers Ahead 35

37 Model Model Variables Coefficients Beta (a) Sample Value of x i (b) Sample Calculation (c) = (a) * (b) Variable 1 Beta 1 Input variable 1 Beta 1 x IV 1 Variable 2 Beta 2 Input variable 2 Beta 2 x IV 2 Variable 3 Beta 3 Input variable 3 Beta 3 x IV 3 Result (tranformed) aka Linear predictor Sum of this column 36

38 Model Input Ten Year Level Term Assumption Value Issue Age 40 Duration(>=10) 11 Risk Class NS (Other Non-Smoker) Face Amount 250K-1M Premium Mode Monthly Premium Jump Ratio 3.01x-4x 37

39 Model Model Variables Coefficients Beta (a) Sample Value of x i (b) Sample Calculation (c) = (a) * (b) Risk Class NS (Other Non-Smoker) BCNS (Best Class NS) SM (Smoker) Face Amount <50K K K-250K K-1M >1M Premium Mode Monthly Semiannual/Quarterly Annual Other/Unknown

40 Model Model Variables Coefficients Beta (a) Sample Value of x i (b) Sample Calculation (c) = (a) * (b) Intercept Issue Age (Duration - 9)^(-1) (11-9)^(-1) (6.0956) (Premium Jump Ratio)^(-1) ^(-1) (0.8195) 39

41 Model Model Variables Coefficients Beta (a) Sample Value of x i (b) Sample Calculation (c) = (a) * (b) (Issue Age)^ ^2 (1.0478) log(issue Age) ln(40) (9.9073) (Duration - 9)^(-2) (11-9)^(-2) (Duration - 9)^(-3) (11-9)^(-3) (2.5610) (Premium Jump Ratio)^(-2) ^(-2) (0.2402) (Premium Jump Ratio)^(-3) ^(-3) Cross Term Issue Age:(Premium Jump Ratio)^(-1) *3.5^(-1) Issue Age:(Duration - 9) *(11-9) (0.2530) 40

42 Model Result Results Linear Predictor = Sum(Beta i * x i ) = Sum (c) (0.9642) Modeled Lapse Rate = e Linear Predictor 38.1% 41

43 Fit by Duration 100% 120,000 90% 80% 100,000 Lapse Rate 70% 60% 50% 40% 30% 80,000 60,000 40,000 Lapse Count 20% 10% 20,000 0% Lapse Count Actual Lapse Rate Predicted Lapse Rate 0 42

44 Fit by Premium Jump 100% 90% 80% 70% Lapse Rate 60% 50% 40% 30% 20% 10% 0% Dur 10 Act Lapse Rate Dur 11 Act Lapse Rate Dur 12 Act Lapse Rate Dur 10 Pred Lapse Rate Dur 11 Pred Lapse Rate Dur 12 Pred Lapse Rate 43

45 Validation Lapse Rates Model Variables % of data actual predicted Risk Class NS (Other Non-Smoker) 74% 51.5% 51.4% BCNS (Best Class NS) 13% 75.0% 74.7% SM (Smoker) 13% 53.6% 53.4% Face Amount <50K <1% 38.1% 37.2% K 8% 43.9% 43.9% 100K-250K 45% 52.1% 52.1% 250K-1M 40% 57.8% 57.5% >1M 7% 69.0% 68.9% Premium Mode Monthly 42% 40.3% 39.7% Semiannual/Quarterly 37% 61.5% 61.7% Annual 19% 70.2% 70.3% Other/Unknown 2% 87.0% 86.9% 44

46 Better Mousetrap? Is unexpected result logical or spurious? Goldilocks principle: model should be just right 45

47 Appendix Goodies An Excel spreadsheet of the model to play with and use How to build a model The R language code for an educational analysis based on sample dummy data provided 46

48 Additional Resources- SOA Report: Predictive Modeling: A Modeler s Introspection Investment/2015-predictive-modeling.aspx Predictive modeling page of SOA website at Publications/Newsroom/Emerging-Topics/Predictive- Analytics/default.aspx Advanced Analytics Seminar of SOA Calendar/2015-advanced-business-analytics-chicago.aspx 47

49 Additional Resources John Hopkins University s MOOC on Data Science on Coursera m_medium=catalog The R language programming community 48

50 QUESTIONS? Jean-Marc Fix is Vice President, Research and Development, at Optimum Re and has over 25 years of life insurance and reinsurance experience. He studies all aspects of mortality and longevity, from both an actuarial and underwriting side. He is a member of the Longevity Advisory Group and of the Living to 100 Symposium Committee. He is also a member of the Reinsurance Section research team and is interested in any tools that may make his research job easier. 49

51 Trademark of Optimum Group Inc. 50

52 Applications of Credit in Life Insurance Underwriting Issues & Innovation Seminar Session 8: The Latest on Practical Uses of Big Data & Predictive Analytics Derek Kueker, FSA MAAA August 4,

53 Introduction

54 Introduction Purpose RGA & TransUnion partnered together to better understand the value of credit data to life insurers and potential applications Background Credit-Based Insurance Scores (CBIS) used in P&C since the 1990 s Wide adoption in pricing & underwriting for auto and home insurance Predictive models built and validated using de-personalized credit data Goal of the Model To predict mortality The model is also extremely good at predicting lapses 53

55 Credit Reporting Process Data flow from the consumer transactions / behaviors to the credit report Consumer Collection Agencies Courts Lender / Creditor #1 Lender / Creditor #2.. Lender / Creditor #9 Utilities Etc. TransUnion Consolidates data, builds models Comprehensive reports on individuals (Scores, Attributes or Full file) Consumers Landlords P&C Insurers Life Insurers Lenders Utilities Collection Agencies Employers (new hires) 54

56 Credit Report Contents Demographics What is on a Credit Report Name, Age, SSN, DOB, Phone Trades (Loans) Credit Cards, Mortgages, Auto Loans, Personal Loans, Home Equity Loans, Other Debt Hard Inquiries Consumer-initiated applications for credit What is NOT on a Credit Report Specific Credit Card Purchase Transactions IRS Data Income, Race, etc. Checking/Savings Account Data Public Records Bankruptcies, Tax Liens, Civil Judgments 55

57 Introduction Credit Regulations and Insurance Adoption Federal Statute Fair Credit Reporting Act (FCRA) Section 604 specifies permissible purposes for use of consumer reports to a person which it has reason to believe intends to use the information in connection with the underwriting of insurance involving the consumer Thus, consumer reports may be used in connection with the underwriting of life insurance. State Regulations 50% of states adopt Model Act for P&C; sensible restrictions on credit variables Most other states propose their own rules for credit use in rating/underwriting P&C Insurers 99% of insurers use credit in connection with underwriting Credit-Based Insurance Scores (CBIS) in use since the 1990 s Life Insurers Underwriters are currently using credit reports Early adopters are using in connection with underwriting Used in direct marketing by several life carriers 56

58 Model Creation and Validation

59 Model Creation Building the Model Starting Data Built the model on 44 million lives and over 3 million deaths Started with over 800 variables offering features of individual s credit history Variable Selection Selected variables that were: Most predictive of the outcome Stable over time Non-gameable Not too correlated with the other variables Model Process Binary Logistic Regression Model validated internally using an additional 30 million lives Age, Gender and Region used as control variables External Validation of Model Tested model using traditional mortality and lapse studies Used a random holdout dataset of another 18 million lives TrueRisk Life presented as a score from: Data comes from de-personalized 1998 credit archive (90% of the US population) Model calibrated to actual deaths occurring over a 12-year period TrueRisk Life Score TransUnion & RGA built and tested TrueRisk Life on 92 million individuals 1 to 100 Low Risk High Risk 58

60 Model Creation Credit Report Variables Scores Credit Seeking Activity Derogatory Credit Information Credit Tenure Credit Usage New trades Inquiries Recency & Frequency Severity Recency Count Amount File thickness (e.g. number of active trades) Longevity (e.g. months since oldest trade) Utilization percentage Recency of use Usage patterns The attributes encompass the breadth of the data with the most emphasis on long-run behavior rather than short-term patterns 59

61 TrueRisk Life Deliverables TrueRisk Life Score Reason Codes Credit Report What value is a model if you don t understand the drivers? 60

62 Model Validation Population Study Overall Mortality Details Mortality study performed on holdout sample of 18 million lives using a 1998 TransUnion archive and studying the lives during Score buckets are set to be uniform across the population Study shows 5 times segmentation ( compared to 1-5) SSMDF used as source of deaths; used population mortality tables 61

63 Model Validation Population Study Mortality by Age Group Mortality by Duration 62

64 Insured Lives Validation

65 Model Validation Insured Lives Study Distribution of Insureds (Compared to Population) Details of the Study Important to test the value of TRL on an insured block of business Business Studied: Full UW (term, UL, VUL) and small face WL Study Period: Mortality and Lapse result studied on a count basis Relative mortality and relative lapse results reported 64

66 Model Validation Insured Lives Study Fully Underwritten Mortality Study Mortality of the group is 2.6 times higher than the 1-10 group Term, UL & VUL; Face Amounts $100,000; Issue Ages < 70 65

67 Model Validation Insured Lives Study Fully Underwritten Mortality Study Segmentation exists within risk classes; Mortality for worst TRL scores (71-100) are about double that of best risks (1-10); Non-smokers are shown, but results are similar for smokers. Term, UL & VUL; Face Amounts $100,000; Issue Ages < 70 66

68 Model Validation Insured Lives Study Fully Underwritten Lapse Study Details Overall Lapse Results (Durations 1-2) Term, UL and VUL Face amounts $100k Issue Ages < 70 Results Overall Lapse Results (Durations 3+) Lapse rate of group is 6 times higher than 1-10 group in durations 1-2 Continued segmentation seen in later durations, but less dramatic Similar results seen when looking at the curves by issue age band 67

69 Model Validation Insured Lives Study Small Face Whole Life Mortality Study Overall Mortality (Issue Age < 70) Results Mortality about 6 times higher for worst scores Segmentation at higher scores for this business 14% of exposure & 29% of claims have a score > 95 > 10% of the claims have a score of 100 Value also seen beyond issue age 70 Includes Whole Life products < $100k face; most of this business is under $25k-$50k; Issue ages < 70; Scores above 90 are further split out 68

70 Model Validation Insured Lives Study Small Face Whole Life Lapse Study Details Overall Lapse Results (Durations 1-2) Includes Whole Life products Face amounts < $100k; most of this business is under $25k-$50k Issue Ages < 70 Results Overall Lapse Results (Durations 3+) Significantly higher lapse rates at the higher scores Raw lapse rates are much lower for durations 3+, but there is little segmentation by score 69

71 Applications within Life Insurance

72 Applications within Life Insurance Dynamic Underwriting Further Risk Segmentation within a Fully Underwritten Environment Further Risk Segmentation within a Simplified Issue Environment Batch segmentation ( pre-approval ) for new firm life offers Cross-sell or up-sell existing customers Inforce Policy Management (lapse & mortality) 71

73 Key Takeaways

74 Key Takeaways Keep it simple Transparency is key Not all data is created equal We all have a responsibility to the industry Through the responsible use of new underwriting evidence such as credit, we can not only provide better risk selection but also enhance the consumer experience as well as grow the underserved life insurance industry. 73

75 Applications of Credit in Life Insurance Underwriting Issues & Innovation Seminar Session 8: The Latest on Practical Uses of Big Data & Predictive Analytics Derek Kueker, FSA MAAA August 4,

76 Milliman Overview of Predictive Analytics Model Developed for Underwriting Underwriting Issues and Innovation Seminar Tim Hill, FSA, MAAA August 4, 2015

77 Uses of Predictive Analytics in Life Insurance Selection of agents Lead generation / marketing Underwriting Policyholder retention Detection of fraud at claims time In force management 76

78 What Data is available today? Insurance data 77 Gender Age Height/weight Geographic location Medical history Financial information Lifestyle information Driving record Consumer data Thousands of records on every individual While geographic data exists and is predictive, desire is generally to use individualized consumer data Will discuss consumer data in more detail in Milliman example

79 Milliman Underwriting Example Goals of Project A client asked us to develop a model, using consumer data to rapid issue a significant percent of policies Goal was to waive paramedical exam in significant percent of cases Product sold as a fully underwritten policy Full underwriting was the default method if model not conclusive Reasons for this request: Reduce underwriting costs by eliminating need for medical exams, MVR, and other tests in some cases Reduce issue time Improve customer satisfaction Reduce not-taken rates, leading to additional savings 78

80 Milliman Example Data and Tools Worked closely with client, who provided us data (about 70,000 lives) Some data used to develop a model and rest saved to later validate the model Used a machine-learning program Finds non-linear behavior and interactions that a generalized linear model (GLM) cannot Recognizes variables that have strong fit Decision trees used At each node, many factors are determined and those that are the strongest drivers are used to split the policies at that node 79

81 Milliman Example Node Split Example of a node split All Policies: 10,000 Probability of Best Preferred: 35.0% BMI = 29 or more: 2,162 policies Probability of Best Preferred: 6.6% BMI = < 29: 7,838 policies Probability of Best Preferred: 42.9% 80

82 Milliman Example Traditional Tools Ultimate goal was to develop a model that produced a score for each applicant Score was used to determine if the applicant could be issued a policy without further underwriting Considered both traditional and non-traditional variables Examples of traditional variables Age BMI Gender MIB MVR Rx histories 81

83 Model Build What are the variables that have predictive value? Most Predictive Consumer Variables in Scoring Model Brand Name Medicine Propensity Score Rx Online Search Propensity Advertised Medicine Inquirer Propensity Score Credit Card Indicator Prescriptions by Mail Propensity Number of Sources UnderBanked Indicator Consumer Prominence Indicator Casino Gaming Propensity Score Economic Stability Indicator Financial Home Assessed Value Consumer Hits: Exercise items Predictive at higher ages, but not lower ages Media Channel Usage Home Lot Square Footage Household Income Total Consumer Hits 82

84 Milliman Example Scoring Process Want to determine whether the policy can be rapidly issued based on the score (without further underwriting) Need to establish cut-off points for bucketing the lives into each of the underwriting classes (e.g., best preferred, preferred, standard, decline) Limits were set to reduce the number of cases where the applicant received a lower rating than from the normal best preferred underwriting risk class We used two thresholds: No more than 20% of applicants one class below class being studied could score above the threshold being tested No more than 10% of applicants more than one class below class being studied could score above threshold being tested 83

85 Milliman Example Validation Process 70% of data was used to construct the model and 30% was set aside to validate the model after it was constructed Both models (probability of best preferred and probability of decline) had a validation correlation of over 99% Correlations over 90% are considered good fits to the underlying data 84

86 Outliers Set reasonable expectations and be able to explain result General In cases where your model classifies poorly, find reasons why Does the result make sense, given the inputs? Which makes more intuitive sense, the model outcome or the actual outcome? Case Study Cross-Models: see how much of each underwriting class made it into good/bad groups Reason Codes: need to be able to illustrate reasons for a particular score Have to have confidence in model, otherwise underwriters choose to either not even use model (fully underwrite everything) or ignore model insights 85

87 Milliman Example Scoring Results A small percentage of policies would be issued under this program who would have otherwise been declined However, this should be more than offset by the underwriting savings from the policies that are rapid issued 86

88 Ongoing Efforts - Monitor Results Going Forward General Ideally, predictive modeling is not a once and done process More data means more robust models If model is not tracking well, need to be able to adjust 87 Case Study Refresh model with new data extract Portfolio Manager: examines how scores move over time, illustrates any drift Distribution Manager: a spreadsheet grid to examine scores by agent, area, or any other breakdown Compare model scores with underwriting decisions on new policies

89 Milliman Example Other Findings Traditional factors are stronger predictors for determining the best preferred class Consumer and financial factors are more influential in determining whether or not to decline It was estimated that almost 80% of the top applicants could be rapid issued under this program However, score level could be set wherever company chooses The additional non-insurance data proved predictive, but was most valuable when used with the insurance data (i.e., MIB, MVR, Rx) 88

90 Enhance Target Marketing by Adding Mortality Insights Target marketing using consumer data common in direct life insurance sales, but less common in majority of agent driven sales Develop a pre-qualify process using consumer data Determine which lives in a group are most likely going to qualify for life insurance Determine which risk class to offer so that applicant not disappointed by underwriting decision Not taken rates higher for applicants that get worse rating class than applied for Agent more confident talking to client because less likely to disappoint them Improves overall mortality by only offering to lives that tend to be healthier which allows company to have more competitive product and ultimately more high-quality sales 89 September 1, 2015