Reverse Sensitivity Testing: What does it take to break the model? Silvana Pesenti Silvana.Pesenti@cass.city.ac.uk joint work with Pietro Millossovich and Andreas Tsanakas Insurance Data Science Conference, 16 July 2018 http://openaccess.city.ac.uk/18896/
Motivation
Motivation: Real-data example Proprietary model of a London insurance market portfolio Y = g(x) Facts 500,000 Monte Carlo simulations of inputs X = (X 1,..., X 72 ) and output Y no knowledge about distributional assumptions Reverse Sensitivity Testing, Silvana Pesenti 1
Risk measures Risk assessment of Y through the risk measures: VaR α (Y ) = inf{y R P (Y y) α}, ES α (Y ) = 1 1 α 1 α VaR u (Y ) du. Reverse Sensitivity Testing, Silvana Pesenti 2
1. Which input factor is most important? Reverse Sensitivity Testing, Silvana Pesenti 2
1. Which input factor is most important? 2. Which is the most plausible alternative model that leads to an increase in the risk measure? Reverse Sensitivity Testing, Silvana Pesenti 2
Reverse sensitivity testing
Method 1. Define a stress on the output Y : - increase of VaR or/and ES 2. Derive weights (change of measure) such that - re-weighted output fulfils the required stress - most plausible (minimal Entropy) 3. Analyse the stressed model - sensitivity measure Reverse Sensitivity Testing, Silvana Pesenti 3
Model and applicability Applicable in a Monte Carlo setting for n large for any distribution of X i for any dependence structure of X under no restrictions on g Reverse Sensitivity Testing, Silvana Pesenti 4
Monte Carlo setup M Monte Carlo simulations of Y = g(x) Find weights w (1),..., w (M), such that the re-weighted output has the required stress X 1... X n Y w x (1) 1... x (1) n y (1) = g(x (1) 1,..., x(1) n )..... x (M) 1... x (M) n. y (M) = g(x (M) 1,..., x (M) n )? Weights w are analytical functions of the output. Reverse Sensitivity Testing, Silvana Pesenti 5
Stress on VaR Weights for a stress on VaR Reverse Sensitivity Testing, Silvana Pesenti 6
Stress on VaR and ES Weights for a stress on VaR and ES Reverse Sensitivity Testing, Silvana Pesenti 7
Numerical example
Insurance portfolio Non-linear insurance portfolio Y = L (1 X 4 ) min { (L d) +, l } L = X 3 (X 1 + X 2 ), where X 1, X 2 different lines of business X 3 positive multiplicative risk factor, e.g. inflation X 4 percentage lost due to default of the reinsurance company reinsurance limit l and deductible d Reverse Sensitivity Testing, Silvana Pesenti 8
Insurance portfolio - Weights Stress: VaR 0.95 (Y ) by 10% ES 0.95 (Y ) by 13% RN density 0 10 VaR(Y) under Q 300 400 500 Y Reverse Sensitivity Testing, Silvana Pesenti 9
Insurance portfolio - Output empirical distribution 0.0 0.2 0.4 0.6 0.8 1.0 Baseline Model Stressed Model 300 400 500 Y Reverse Sensitivity Testing, Silvana Pesenti 10
Insurance portfolio - Input 1.0 0.25 1.0 0.25 0.8 0.20 0.8 0.20 0.6 0.15 0.6 0.15 0.4 0.10 0.4 0.10 empirical distr. function 0.2 0.0 1.0 0.8 50 100 150 200 250 300 X 1 0.05 0.00 0.25 0.20 0.2 0.0 1.0 0.8 150 200 250 X 2 0.05 0.00 0.25 0.20 difference of empirical distr. 0.6 0.15 0.6 0.15 0.4 0.10 0.4 0.10 0.2 0.05 0.2 0.05 0.0 0.00 0.0 0.00 1.00 1.05 1.10 X 3 0.0 0.2 0.4 0.6 0.8 1.0 X 4 Reverse Sensitivity Testing, Silvana Pesenti 11
Insurance portfolio X 1 X 2 X 3 X 4 Y Mean 150 200 1.05 0.10 362 Mean, stressed 157 202 1.05 0.14 371 Relative increase 5% 1% 0% 44% 3% Standard deviation 35 20 0.02 0.20 36 Standard deviation, stressed 43 21 0.02 0.26 50 Relative increase 25% 5% 1% 30% 38% Reverse Sensitivity Testing, Silvana Pesenti 12
Which input factor is most important? Reverse Sensitivity Testing, Silvana Pesenti 12
Sensitivity measures
Sensitivity measure Sensitivity measure Γ i = Estressed (X i ) E(X i ) normalised depends on Y through the weights w. Reverse Sensitivity Testing, Silvana Pesenti 13
Real-data example Proprietary model of a London insurance market portfolio Y = g(x) Stress: VaR 0.95 (Y ) by 8% ES 0.95 (Y ) by 10% Reverse Sensitivity Testing, Silvana Pesenti 14
Real-data example empirical distribution functions 0.0 0.2 0.4 0.6 0.8 1.0 0.5 1.0 1.5 2.0 Y Figure 1: Empirical distribution of the output under the baseline model and the stressed model. Reverse Sensitivity Testing, Silvana Pesenti 15
Real-data example Sensitivity measure 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 18 35 3 47 12 59 38 68 58 4 16 34 52 69 1 41 2 61 5 40 67 27 44 31 32 24 50 23 66 45 25 57 15 8 22 65 64 43 17 21 60 28 13 29 39 20 55 46 9 42 10 30 14 7 63 54 51 62 53 70 71 36 49 72 19 37 48 6 33 11 56 26 Input risk factor Reverse Sensitivity Testing, Silvana Pesenti 16
Thank you! Reverse Sensitivity Testing, Silvana Pesenti 16
Appendix
Insurance portfolio - Assumptions Assumptions: X 1 (truncated) LogNormal with mean 150 and sd 35. X 2 Gamma with mean 200 and sd 20. X 3 (truncated) LogNormal with mean 1.05 and sd 0.02. X 4 Beta with mean 0.1 and sd 0.2. X 1, X 2, X 3 are independent. X 4 dependent on L through a Gaussian copula with correlation 0.6. d = 380, l = 30. Reverse Sensitivity Testing, Silvana Pesenti