Big Data, Small Data, Medium-sized Data

Transcription

1 Big Data, Small Data, Medium-sized Data Making the most of what you ve got 19 April 2016 Phil Joubert William Chan

2 A Big Data timeline Google trends Big Data Big Data a thing Statistics a thing Mortality tables and modern life insurance invented Six Sigma Actuaries embrace PC s Japan Inc. applies statistical quality control Prudential buys first commercial computer Tesco starts analysing loyalty card data Credit card companies pioneer fraud detection Big Data / Analytics: Wired or Tired? Page 2

3 Cross industry use of Big Data EY studies 1 show insurers lagging the other industries Large retailers Internet natives Retail banks Logistics specialists Life Insurers Life insurers are asking three questions: 1. Do I have big data? 2. Do I need big data? 3. If so, how do I get some? Page 3 1: EY 2016 Sensor Data Survey - disrupt or be disrupted

4 How big is your data? One human genome 200 Gb The 1000 Genomes project 200 Tb Twitter fire-hose 1 Tb / day NSA s Massive Data Repository 1,000+ Tb 1 Your policy database 10 Gb 2 Full stochastic run of RBC model 1 Gb So the answer to question one is: Probably not Page 4 1: Guestimate by experts 2: Guestimate by me

5 Do you need Big Data? Three common fallacies and some facts Fallacy One: Big Data is sexy Big Data is mysterious If I had big data, I would be sexy and mysterious Fallacy Two (The Big Data fusion fallacy): if I collect enough data together, the truth will reveal itself Fallacy Three (The Data is out there fallacy): The data exists, I just need to get hold of it Facts: Big Data requires specialist IT infrastructure more than specialist math Getting any kind of data before you know exactly what you want to do with it is the Wrong Thing To Do So the answer to question two is: it depends what you want to do and how much you re willing to invest Page 5

6 How do I get Big Data? Why you don t have Big Data: How often do you check Facebook? How often do you go to the supermarket? How often do you talk to your life insurer? 4.5 billion likes generated daily as of May 2013 (Source: Facebook) Ideas for becoming more like Facebook and Tesco: Increase frequency of customer contact (Vitality) Increase breadth of customer relationship (Become a health insurer / composite insurer) Change the way you interact with your customers (digital channels) Completely change your business model (?) Over 80M shopping trips per week in 2015 (Source: Tesco annual report) Annual statement of policy value So the answer to question three is: with a great deal of effort Page 6

7 Becoming data driven Page 7

8 Becoming a data-driven organisation Start from the top Vision / Strategy High level objectives Authority from senior management Data champion / visionary / sponsor Policies Data gathering Data storage Data use Data privacy Data security Data inventory People, Technology, Projects Right combination of technical capability and business sense Right tools, right level of support from IT Short projects to start delivering insight / value fast, longer term ones to fundamentally change the business. Supporting meta projects to build the needed information and infrastructure Long term, does this transfer to BAU? Is the project team permanent? Page 8

9 Next evaluate what data you have Decisions Systems Data 1 Your existing stock of data (data you have already) Generally your biggest data asset, but the most expensive and difficult to change and improve: Enhance access Improve awareness Create trust by cleaning / validating Combine datasets such that the sum is greater than the whole Preprocess datasets to produce useful calculated sets 3 Data from external sources Easy to change, costs vary: Get a better data set (more fields, more accurate, more timely, more history, more coverage) Improve your internal processing, including checking / cleansing and provision 2 Your delta of data (data you are gathering or creating) Quicker to change. The impact will be smaller over the short term, but fundamental over the long term: Gather more data Store more data Have dedicated people assigned to data innovation Provide access to data Page 9

10 Then you need to work out what to do with it Design Phase Implementati on phase A typical data project has a design phase and an implementation phase: D1 D2 D3 D4 D5 Precise statement of problem (hypothesis) Data definition & collection Methodology definition Technology design Insight delivery design Possible restatement of problem I1 I2 I3 Prototype delivery Industrialisation Embed into business Bad hypothesis Customers from different channels lapse at different rates Good hypothesis Customers from different channels lapse at different rates, and we can differentiate charging to minimise the expected value from each This is a fundamental difference between academic data analysis and business driven data analysis Page 10

11 Making it work closing the loop How do we extract value from data? Hypothesis Data Analytics Action Actionable Relevant data Insights Valuable Appropriate data sources Decision rules & Algorithms Transaction / behavior history Success = end user doing something different Page 11

12 Data security and privacy New risks are a side-effect of becoming data-driven Don t be like these guys: Issues Customer data security and privacy Actions High-profile data breaches may lead customers to demand more value for sharing information Engage customers to identify what value they will seek in exchange for their data Generic, blanket privacy policy statements will no longer suffice Revisit data protection protocols and new, emerging architectures Don t store or maintain customer data that is readily available Acquire and internalize data not replicable externally Turn to external parties to maintain certain types of PII Consider engaging regulators and industry peers in developing robust or auditable generally accepted privacy principles Page 12

13 Tools of the trade Storage / Manipulation Database, data warehouse, data lake?!? NoSQL? Analysis Open-source vs Commercial Presentation RShiny Tableau Page 13

14 Case study life analytics Page 14

15 Compare Actuarial Approach vs Regression Approach Actuarial Approach Choose a base table Experience study or benchmarking Apply percentage on a base table Problem: difficult to set assumptions to decrements influenced by multiple factors Regression Approach Identify available independent variables Choose a regression model Select significant variables and estimate parameters with the model Validation of the parameters and model Page 15

16 Logistic Regression vs Linear Regression Consider fitting a limited or binary variable e.g. yes or no, dead or alive Linear regression do not provide a good fit, output out of the range [0,1], which probability could take value Logistic regression eliminates the problem, output produced must be within the range [0,1] Linear Probability Model Logistic Regression Model P = 1 P = 1 P = 0 P = 0 The predicted probability p x 1,, x n = eβ 0+β1x1+ +βnxn 1+e β 0+β1x1+ +βnxn Estimated by most of the statistical packages, e.g. SPSS, R or SAS Page 16

17 Logistic Regression Model Validation Hypothesis testing: Wald statistic, Log likelihood ratio. Comparing actual vs expected number of incidents Model Challenges Choosing independent variables need creativity Treating per policy per time period as one exposure, time period can be selected as daily, monthly or annual, censoring of data may be needed Removing independent variables with multicollinearity Application Examples Variable Annuity dynamic lapse rate Reverse mortgage refinancing rate Page 17

18 Logistic Regression Case Study Case Study Lump Sum Reverse Mortgage Loan Termination: Refinancing, Death and Mobility 1000 loans with 30 years experience Regression Result on Refinancing Rate 3 Significant factors Current Age Property Price Index change since inception Initial Property Price 2 Excluded factors Initial Loan-to-Value Ratio due to high correlation with current age Gender not significant Page 18

19 Annualized Refinancing Rate Number of Refinanced Logistic Regression Case Study Number of Refinanced Annualized Refinancing Rate Number of Refinanced Annualized Refinancing Rate 5% Regression Results 4% Actual 3% Logistic Model 2% 1% 0% Current Age of the Borrower 35% 30% Actual 25% 20% 15% 10% 5% 0% 0% 20% 40% 60% 80% % change in property Price 4% 3% Actual 2% Logistic Model 1% 0% Initial Propery Price (Million HKD) Actual vs Predicted Logistic Model Actual Logistic Model Actual Logistic Model Actual Current Age of the Borrower % change in property Price >8 Initial Property Price (Million HKD) Page 19

20 Case study Bayesian techniques Page 20

21 Small Data I don t like cricket Adam Voges is an Australian cricketer with an impressive batting average, based on 20 innings What odds would you offer on him making 100 in his next test innings? Voges data at face value Historic data on all international batsmen to date Blend of historic and specific data Page 21

22 Small Data Bayesian statistics, a recap 1 Not enough data, dependent on an unknown parameter 2 A likelihood function 3 4 A statement summarising what we know about the unknown parameter The Bayesian estimate of the unknown parameter 5 A problem Page 22

23 Small Data So how do we deal with the problem? Until recently Bayesian methods were more of a philosophy than a viable technique Two related advances changed this: Computing power Markov Chain Monte Carlo (MCMC) 1. Regular Monte-Carlo integration (independent samples) 2. Integration by sampling along a random walk path Page 23

24 Small Data What if I really don t like cricket? Small portfolio? Distribution of estimated losses based on own data alone Few year s of experience? Standard error estimates too wide? Worried about ignoring 200 years of actuarial experience, but not sure how to reconcile it with your data? Distribution of estimated losses based on Bayesian blend of population and own data Choose your prior such that you can control the level of belief Page 24

25 Learn more Page 25

26 Some links / articles Advanced analytics for insurance EY 2016 Sensor Data Survey - disrupt or be disrupted Data and Analytics Impact Index Digital security: a Financial Services perspective Page 26

27 EY Assurance Tax Transactions Advisory About EY EY is a global leader in assurance, tax, transaction and advisory services. The insights and quality services we deliver help build trust and confidence in the capital markets and in economies the world over. We develop outstanding leaders who team to deliver on our promises to all of our stakeholders. In so doing, we play a critical role in building a better working world for our people, for our clients and for our communities. EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients. For more information about our organization, please visit ey.com. 20XX Ernst & Young, China All Rights Reserved. APAC no. ED MMYY This material has been prepared for general informational purposes only and is not intended to be relied upon as accounting, tax, or other professional advice. Please refer to your advisors for specific advice. ey.com/china