Aggressive Retrospec.ve Tes.ng of Stochas.c Loss Reserve Models What it Leads To

Transcription

1 Aggressive Retrospec.ve Tes.ng of Stochas.c Loss Reserve Models What it Leads To Glenn Meyers Presenta.on to 2 nd Interna.onal Conference on Actuarial Science and Quan.ta.ve Finance June 17, 2016

2 Outline of Talk CAS Loss Reserve Database provides the data that enables aggressive retrospec.ve tes.ng. Iden.fy some shortcomings in two currently popular stochas.c loss reserve models. Propose models that address these shortcomings. Use Bayesian MCMC. Show what can be done with MCMC models. Address the dependencies issue.

3 The CAS Loss Reserve Database Created by Meyers and Shi With Permission of American NAIC Schedule P (Data from Parts 1-4) for several US Insurers Private Passenger Auto Commercial Auto Workers Compensa.on General Liability Product Liability Medical Malprac.ce (Claims Made) Available on CAS Website hxp://

4 Illustra.ve Insurer Incurred Losses

5 Illustra.ve Insurer Paid Losses

6 Criteria for a Good Stochas.c Loss Reserve Model Using the predic.ve distribu.ons, find the percen.les of the outcome data for several loss triangles. The percen.les should be uniformly distributed. Histograms PP Plots and Kolmogorov Smirnov Tests Plot Expected vs Predicted Percen.les KS 95% cri.cal values = 19.2 for n = 50 and 9.6 for n = 200

7 Illustra.ve Tests of Uniformity

8 Illustra.ve Tests of Uniformity

9 Data Used in Study List of insurers available in the Appendix of the monograph. 50 Insurers from four lines of business Commercial Auto Personal Auto Workers Compensa.on Other Liability Criteria for Selec.on All 10 years of data available Stability of earned premium and net to direct premium ra.o Both paid and incurred losses

10 Test of Mack Model on Incurred Data Conclusion The Mack model predicts light tails.

11 Test of Mack Model on Paid Data Conclusion The Mack model is biased upward.

12 Test of Bootstrap ODP on Paid Data Conclusion The Bootstrap ODP model is biased upward.

13 What the Retrospec.ve Tests Tell Us Mack predic.on of vola.lity tend to be too low on incurred loss models. Both Mack and bootstrap ODP predic.ons tend to be biased upward.

14 Possible Explana.ons A different set of unique events (aka Black Swans) regularly occur.

15 Possible Explana.ons A different set of unique events (aka Black Swans) regularly occur. We are using the a bad model!

16 Possible Explana.ons A different set of unique events (aka Black Swans) regularly occur. We are using the a bad model! What follows is a quest for a bexer model.

17 A Tool for Building New Models Bayesian MCMC MCMC stands Markov Chain Monte Carlo A Markov chain is a sequence of vectors {X t } where X t depends only on X t-1. Under certain condi.ons, the sequence converges to a limi.ng distribu.on.

18 Bayesian MCMC Defined in terms of a prior distribu.on, p(θ) and a condi.onal distribu.on f(y θ). The MC limi.ng distribu.on is the posterior distribu.on f(θ y). That is to say { f ( θ t y) } N+M can be regarded t=n as a random sample from the posterior distribu.on for sufficiently large N.

19 Soqware for Bayesian MCMC BUGS Bayesian analysis Using Gibbs Sampler WINBUGS, OpenBUGS JAGS Just Another Gibbs Sampler Stan Named aqer Stanislaw Ulam Early user of MCMC in nuclear physics. My current favorite JAGS and Stan are callable from R with the appropriate package.

20 Bayesian MCMC is a Game Changer for Stochas.c Modeling If you can code the model, you can get a large sample from the posterior distribu.on. No fundamental limit to the number of parameters. Ideal for stochas.c loss reserving!

21 The Situa.on Prior to My Re.rement We have the CAS Loss Reserve Database Hundreds of loss reserve triangles with outcomes. The current best prac.ce models do not correctly predict the distribu.on of outcomes.

22 The Situa.on Prior to My Re.rement We have the CAS Loss Reserve Database Hundreds of loss reserve triangles with outcomes. The current best prac.ce models do not correctly predict the distribu.on of outcomes. We have a game changing sta.s.cal model firng methodology Bayesian MCMC!

23 Nota.on w = Accident Year w = 1,,10 d = Development Year d = 1,,10 C w,d = Cumula.ve (either incurred or paid) loss

24 Bayesian MCMC Models Use R with runjags or rstan packages Get a sample of 10,000 parameter sets from the posterior distribu.on of the model 10 Use the parameter sets to get 10,000, C w,10, simulated outcomes! w=1 Calculate summary sta.s.cs of the simulated outcomes Mean Standard Devia.on Percen.le of Actual Outcome

25 Begin with Incurred Data Models

26 The Correlated Chain Ladder (CCL) Model logelr ~ uniform(-5,0.5) α w ~ normal(log(premium w )+logelr, 10 ) β 10 = 0, β d ~ uniform(-5,5), for d=1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ 1,d = α 1 + β d C 1,d ~ lognormal(µ 1,d, σ d ) ρ ~ uniform(-1,1) µ w,d = µ w + β d + ρ (log(c w-1,d ) µ w-1,d ) for w = 2,,10 C w,d ~ lognormal(µ w,d, µ d )

27 Key Statements in the CCL Model µ w,d = α w + β d + ρ (log(c w-1,d ) µ w-1,d ) for w = 2,,10 C w,d ~ lognormal(µ w,d, σ d ) Subject to the constraint that σ d decreases as d increases.

28 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters.

29 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters. CCL uses Bayesian es.ma.on to get a posterior distribu.on of parameters.

30 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters. CCL uses Bayesian es.ma.on to get a posterior distribu.on of parameters. Chain ladder applies factors to last fixed observa.on.

31 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters. CCL uses Bayesian es.ma.on to get a posterior distribu.on of parameters. Chain ladder applies factors to last fixed observa.on. CCL uses uncertain level parameters for each accident year.

32 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters. CCL uses Bayesian es.ma.on to get a posterior distribu.on of parameters. Chain ladder applies factors to last fixed observa.on. CCL uses uncertain level parameters for each accident year. Mack assumes independence between accident years.

33 The Correlated Chain Ladder Model Predicts Distribu.ons with Thicker Tails Mack uses point es.ma.ons of parameters. CCL uses Bayesian es.ma.on to get a posterior distribu.on of parameters. Chain ladder applies factors to last fixed observa.on. CCL uses uncertain level parameters for each accident year. Mack assumes independence between accident years. CCL allows for correla.on between accident years, Corr[log(C w-1,d ),log(c w,d )] = ρ

34 Predic.ng the Distribu.on of Outcomes Use MCMC soqware to produce a sample of 10,000 {α w }, {β d },{σ d } and {ρ} from the posterior distribu.on. For each member of the sample µ 1,10 = α 1 + β 10 For w = 2 to 10 C w,10 ~ lognormal (α w + β 10 + ρ (log(c w-1,10 ) µ w-1,10 )),σ 10 ) Calculate 10 w= 1 w,10 Calculate summary sta.s.cs, e.g. C " 10 % " 10 % E$ C w,10 '!and!var $ C w,10 '! # w=1 & # w=1 & Calculate the percen.le of the actual outcome by coun.ng how many of the simulated outcomes are below the actual outcome.

35 Results of Tail Pumping

36 Results of Tail Pumping

37 Posterior Distribu.on of ρ for Illustra.ve Insurer Frequency ρ is highly uncertain, but in general positive ρ

38 Generally Posi.ve Posterior Means of ρ for all Insurers Commercial Auto Frequency Mean ρ Personal Auto Frequency Mean ρ Workers' Compensation Frequency Mean ρ Other Liability Frequency Mean ρ

39 Compare SDs for All 200 Triangles

40 Test of Mack Model on Incurred Data Conclusion The Mack model predicts light tails.

41 Test of CCL (ρ = 0) Model on Incurred Data Conclusion Predicted tails are too light

42 Test of CCL Model on Incurred Data Conclusion Plot is within KS Boundaries

43 Short Term Conclusions Incurred Loss Models Mack model predic.on of variability is too low on our test data.

44 Short Term Conclusions Incurred Loss Models Mack model predic.on of variability is too low on our test data. CCL model correctly predicts variability at the 95% significance level.

45 Short Term Conclusions Incurred Loss Models Mack model predic.on of variability is too low on our test data. CCL model correctly predicts variability at the 95% significance level. The feature of the CCL model that pushed it over the top was between accident year correla.ons.

46 Iden.fied Improvements with Incurred Data Accomplished by pumping up the variance of Mack model by inser.ng a dependency between accident years. What About Paid Data? Start by looking at CCL model on cumula.ve paid data.

47 Test of Bootstrap ODP on Paid Data Conclusion The Bootstrap ODP model is biased upward.

48 Test of CCL on Paid Data Conclusion Biased upward.

49 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

50 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

51 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) Log of loss ra.o is a parameter. The α w varies by accident year. Forces σ d to decrease as d increases µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

52 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

53 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) The speedup w parameters allow the development factors to change with the accident years. Forces σ d to decrease as d increases µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

54 The Speedup w Parameters β d s are almost always nega.ve! (β 10 = 0) speedup 1 = 1 speedup w = speedup w-1 (1 γ (w-2)δ) Speedup rate = γ + (w-2)δ If positive, claim settlement speeds up If negative, claim settlement slows down Can change over time

55 Distribu.on of Mean γs General Speedup of Claim Settlement

56 Test of CSR on Paid Data Conclusion Validates within KS Boundaries

57 Short Term Conclusions Paid Loss Models Mack and Bootstrap ODP models are biased upward on our test data.

58 Short Term Conclusions Paid Loss Models Mack and Bootstrap ODP models are biased upward on our test data. Models that allow for changes in claim sexlement rates predicts the range of the outcomes correctly at the 95% confidence level.

59 Short Term Conclusions Paid Loss Models Mack and Bootstrap ODP models are biased upward on our test data. Models that allow for changes in claim sexlement rates predicts the range of the outcomes correctly at the 95% confidence level. Claims adjusters have important

60 Dependencies Between Lines of Insurance

61 A Recent Development Predic.ng Mul.variate Insurance Loss Payments Under a Bayesian Copula Framework by Yanwei (Wayne) Zhang FCAS and Vanja Dukic Awarded the 2014 ARIA Prize by CAS

62 The General Idea in Zhang/Dukic Given X 1 ~ MCMC Model 1 Given X 2 ~ MCMC Model 2 Fit the joint (X 1,X 2 ) ~ MCMC Models 1 and 2

63 The General Idea in Zhang/Dukic Given X 1 ~ MCMC Model 1 Given X 2 ~ MCMC Model 2 Fit the joint (X 1,X 2 ) ~ MCMC Models 1 and 2 Problem when I tried this with the CSR Model. Marginal samples were different from the univariate samples.

64 log C log C ( ) X wd ( ) Y wd Joint Lognormal Distribu.on ~Multivariate Normal Step 1 Use MCMC to get univariate sample of 10,000 μ wd s and σ d s for each line X and Y = CA, PA, WC or OL. X µ wd Y µ wd, ( X σ ) 2 wd ρ σ X Y wd σ wd ρ σ X Y wd σ wd ( X σ ) 2 wd

65 log C log C ( ) X wd ( ) Y wd Joint Lognormal Distribu.on ~Multivariate Normal Step 1 Use MCMC to get univariate sample of 10,000 μ wd s and σ d s for each line X and Y = CA, PA, WC or OL. Step 2 For each parameter set in the univariate sample for each line, use MCMC to get a single ρ from the bivariate distribu.on of (log(c X wd ), X µ wd Y µ wd, log(c Y wd )) ( X σ ) 2 wd ρ σ X Y wd σ wd ρ σ X Y wd σ wd ( X σ ) 2 wd

66 Got Samples of ρ for 102 Pairs of Triangles in CAS Database

67 Distribu.on of the Sum of Losses for Two Lines of Insurance 10 w=1 X C w,10 10 w=1 Y + C w,10 From the 2-step bivariate model From the independent model formed as a random sum of losses from the univariate models.

68 Test 2-Step Bivariate Model on 102 Pairs of Lines in CAS Database

69 Test Independent Model on 102 Pairs of Lines in CAS Database

70 Model Selec.on Choosing between 2-Step and Independent If we fit model, f, by maximum likelihood define ( ) AIC = 2 p 2 L x ˆθ

71 Model Selec.on Choosing between 2-Step and Independent If we fit model, f, by maximum likelihood define Where p is the number of parameters ( ) L x ˆθ ( ) AIC = 2 p 2 L x ˆθ is the maximum log-likelihood of the model specified by f.

72 Model Selec.on Choosing between 2-Step and Independent If we fit model, f, by maximum likelihood define Where p is the number of parameters ( ) L x ˆθ AIC = 2 p 2 L x ˆθ is the maximum log-likelihood of the model specified by f. Lower AIC indicates a bexer fit ( ) Encourages larger log-likelihood Penalizes for increasing the number of parameters

73 Model Selec.on with the WAIC Sta.s.c If we have an MCMC model with parameters { θ } 10,000 i i=1 WAIC = 2 ˆp WAIC 2 { L( x θ )} 10,000 i i=1

74 Model Selec.on with the WAIC Sta.s.c If we have an MCMC model with parameters { θ } 10,000 i i=1 WAIC = 2 ˆp WAIC 2 { L( x θ )} 10,000 i i=1 Where ˆp WAIC is the effec6ve number of parameters Decreases as the prior distribu.on becomes more informa.ve i.e. less influenced by the data.

75 Model Selec.on with the WAIC Sta.s.c If we have an MCMC model with parameters { θ } 10,000 i i=1 WAIC = 2 ˆp WAIC 2 { L( x θ )} 10,000 i i=1 Where ˆp WAIC is the effec6ve number of parameters Decreases as the prior distribu.on becomes more informa.ve i.e. less influenced by the data. 10,000 { ( )} i=1 L x θ i = Average log-likelihood of the model

76 Model Selec.on Choosing between 2-Step and Independent WAIC sta.s.cs indicate that the independent model is preferred for ALL 102 pairs of lines! Counterintui.ve to many actuaries. Infla.on affects all claims Cyclic effects I think I owe an explana.on.

77 Model Selec.on Choosing between 2-Step and Independent WAIC sta.s.cs indicate that the independent model is preferred for ALL 102 pairs of lines! Counterintui.ve to many actuaries. Infla.on affects all claims Cyclic effects I think I owe an explana.on.

78 Model Selec.on Choosing between 2-Step and Independent WAIC sta.s.cs indicate that the independent model is preferred for ALL 102 pairs of lines! Counterintui.ve to many actuaries. Infla.on affects all claims Cyclic effects I think I owe an explana.on.

79 The Changing SeXlement Rate (CSR) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ w,d = log(premium w )+logelr + α w + β d speedup w C w,d ~ lognormal(µ w,d, σ d )

80 The Stochas.c Cape Cod (SCC) Model logelr ~ uniform(-5,0) α 1 = 0, α w ~ normal(0, 10) for w = 2,,10 β 10 = 0, β d ~ uniform(-5,5), for d = 1,,9 a i ~ uniform(0,1) 10 i=d σ d = Forces σ d to decrease as d increases a i µ w,d = log(premium w )+logelr + β d C w,d ~ lognormal(µ w,d, σ d )

81 The Stochas.c Cape Cod (SCC) Model Simpler than the CSR model Resembles an industry standard BornhueXer Ferguson with a constant ELR Source Dave Clark and Jessica Leong in the references 2-Step SCC model is preferred for some insurers Look at a sample of standardized residual plots Insurer 5185 for CA and OL favors 2-Step Picked as an illustra.on

82 Posterior Distribu.on of ρ

83 Sample of Standardized Residual Plots Insurer CA and OL 2-Step AY 1 borders are black AY 3 borders are blue

84 Sample of Standardized Residual Plots Insurer CA and OL 2-Step AY 1 borders are black AY 3 borders are blue In general, SCC residuals tend to find their own corner. If many are in the NW-SE corner, we see a nega.ve mean ρ.

85 Implica.ons of Independence Cost of capital risk margins should have a diversifica.on credit. As an example, the EU Solvency II adds risk margins by line of business implicitly denying a diversifica.on credit. With a properly valid MCMC stochas.c loss reserve model, one can get 10,000 stochas.c scenarios of the future and calculate a cost of capital risk margin, and reflect diversifica.on. I am preparing a paper on risk margins. Session at the 2016 CLRS

86 A Proposed Law for Dependency Modeling Using the 2-Step procedure, we can fit mul.variate distribu.ons. We can compare the 2-Step model to a model that assumes independence.

87 A Proposed Law for Dependency Modeling Using the 2-Step procedure, we can fit mul.variate distribu.ons. We can compare the 2-Step model to a model that assumes independence. The Law If your dependent bivariate model is bexer than the independent model, you should look for something that is missing from your model.

88 Long Term Recommenda.ons New Models Come and Go Transparency - Data and soqware released

89 Long Term Recommenda.ons New Models Come and Go Transparency - Data and soqware released Large scale retrospec.ve tes.ng on real data Move from Actuarial Mathema6cs to Actuarial Science While individual loss reserving situa.ons are unique, knowing how a model performs retrospec.vely on a large sample of triangles should influence one s choice of models.

90 Long Term Recommenda.ons New Models Come and Go Transparency - Data and soqware released Large scale retrospec.ve tes.ng on real data Move from Actuarial Mathema6cs to Actuarial Science While individual loss reserving situa.ons are unique, knowing how a model performs retrospec.vely on a large sample of triangles should influence one s choice of models. Bayesian MCMC models hold great promise to advance Actuarial Science. Illustrated by the above stochas.c loss reserve models.

91 Long Term Recommenda.ons New Models Come and Go Before aggressive retrospec.ve tes.ng (ART), Bayesian MCMC models for stochas.c loss reserving have been applied to familiar models. (???) For example the Chain Ladder Model (Mack).

92 Long Term Recommenda.ons New Models Come and Go Before aggressive retrospec.ve tes.ng (ART), Bayesian MCMC models for stochas.c loss reserving have been applied to familiar models. (???) For example the Chain Ladder Model (Mack). Nega.ve ART results on current models lead to formula.ng new models with Bayesian MCMC. e.g. CCL and CSR models

93 Long Term Recommenda.ons New Models Come and Go Before aggressive retrospec.ve tes.ng (ART), Bayesian MCMC models for stochas.c loss reserving have been applied to familiar models. (???) For example the Chain Ladder Model (Mack). Nega.ve ART results on current models lead to formula.ng new models with Bayesian MCMC. e.g. CCL and CSR models I predict bexer models in the future.

94 References by Glenn Meyers Stochas.c Loss Reserving Using Bayesian MCMC Models CAS Monograph Series hxp:// Dependencies in Stochas.c Loss Reserving Models CAS eforum, Winter This is a working paper, an updated paper has been submifed to Variance. hxp://