2002 Statistical Research Center for Complex Systems International Statistical Workshop 19th & 20th June 2002 Seoul National University

Transcription

1 2002 Statistical Research Center for Complex Systems International Statistical Workshop 19th & 20th June 2002 Seoul National University Modelling Extremes Rodney Coleman Abstract Low risk events with extreme consequences occur from flooding, terrorism, epidemics, fraud, etc. Measuring and modelling these rare but extreme events is needed to answer questions of how high to build a flood barrier, how much to spend on railway safety, how much to put into reserves to save a bank from collapse. We look at probability models appropriate for modelling extreme losses. Model uncertainty is shown to arise in fitting the long tail from just a small sample of tail data. A methodology is shown which has been applied to extreme banking losses. Key words: Return values, quantiles, operational risk, extreme value distributions, probability weighted moments, small sample estimation. Table of Contents 1. Extreme risk events Return values and quantile models 2 3. Statistical modelling of loss data 3 4. A case study Pricing the risk Alternative methodologies Appendices References Dr Rodney Coleman Department of Mathematics, Imperial College 180 Queen s Gate, London SW7 2BZ, UK

2 Modelling extremes Rodney Coleman Page 2 1 Extreme risk events Extreme risk events are rare. The consequences can be catastrophic. We have only to remind ourselves of Chernobyl, nvcjd, the human form of mad cow disease, with under 150 cases so far, but untreatable and identifiable only after death. We might add: winning the lottery jackpot. The UK national lottery has odds of less than 1 in 14 million given by choosing 6 numbers from 49. When considering extreme loss we must also bear in mind balance of risk, such as the rare complications in children from inoculations despite their benefit in saving lives. Consideration will be directed to modelling the consequences of low risk events when these consequences are expressed in monetary terms. Prior to the World Trade Center destruction in September 2001, the largest insured loss was from Hurricane Andrew in 1992 which caused $16 billion of insurance payouts. In 1999, supercomputers were able to successfully predict the path of Hurricane Lenny, though its severity was overestimated. The low risk means that we cannot demand that the residents of Florida go to live in the interior of the continent, and insurance cover cannot be denied them if they continue to live in Florida. But how should insurance for rare but extreme losses be priced? Worldwide insured losses for the years , reported in Embrechts et al. (1997), illustrates the problem facing the insurance industry. While the increase in the sizes of the losses throughout this time is clearly attributable to increased economic development, the rate at which the extreme losses occur may be a reflection of global warming. After recent fatal railway train crashes in the UK, the question of the appropriate amount to spend on safety devices has political and public dimensions. Railway companies were criticised for even suggesting that safety might be measured in lives saved per million pounds. The crashes have driven passengers onto the roads, where the number of deaths each week exceeds the annual rate on the railways. When Barings Bank collapsed overnight in 1995 due to fraud and mismanagement, the financial world found that despite having a good understanding of credit risk (nonrepayment) and market risk (price variability) its risk from business activities, such as loss of key personnel, computer shut-down, and fraudulent transactions, these risks were not being measured or managed. Records were not even being collated of losses arising from this so-called operational risk. Today stockmarket listed companies report on operational risk to shareholders in their annual reports. 2 Return values and quantile models Return values are the basic risk measure for extremes. Building regulations require building standards that ensure that catastrophic building failure will occur in any year with a less than a 1 in 50 chance.

3 Modelling extremes Rodney Coleman Page 3 Nuclear plants, dams, bridges and sea dykes generally require a less than 1 in 10,000 chance of catastrophic loss in any year. These are return values. The probability of a loss larger than x in any year being less than p gives, assuming independence of loss events, P (No loss > x over k years) > (1 p) k. Example 1. Reactor safety If the probability of meltdown in any given nuclear plant in a year is , and we consider the 600 nuclear plants in the USA, then the probability of meltdown in one of the plants sometime in the next 20 years is 44%. Return values are defined via quantiles. The 100 p % quantile is the loss x exceeded in 100 (1 p)% of the years. We write Q(p) for the quantile function, where Q(p) = x is the inverse function of the loss size distribution function F (x) = P (X x) = 1 p. We are familiar with quantile measures such as the lower and upper quartiles, Q(0.25) and Q(0.75), the median, Q(0.5), etc. For any non-decreasing function T, T (Q(p)) is a quantile function. For example, if Q, Q 0, and Q 1 are quantile functions, so are µ + σ Q 0 (p) (σ > 0) αq 0 (p) + βq 1 (p) (α > 0, β > 0) ln Q(p) (Q(p) > 0) Q(1 p) 1 Q(1 p) These can be combined to create ever more elaborate models (Gilchrist 2000). Some standard quantile models are given in Table 1. A location parameter µ and scale parameter σ give Q(p) = µ + σ Q 0 (p). Since the moments of heavy-tailed distributions may not exist, location and scale are not necessarily expectation and variance. Model Q 0 (p) Model Q 0 (p) Uniform p Fréchet ( ln p) ξ (ξ > 0) Exponential ln(1 p) Weibull ( ln p) ξ (ξ < 0) Normal Φ 1 (p) Pareto (1 p) ξ (ξ > 0) Power p ξ (ξ > 0) Beta (1 p) ξ (ξ < 0) Logistic ln{(1 p)/p} GEV {( ln p) ξ 1}/ξ (ξ 0) Gumbel ln( ln p) GPD {(1 p) ξ 1}/ξ (ξ 0) Table 1: Standard quantile models Q 0 (p) are shown, where ξ is a shape parameter (Gilchrist 2000). 3 Statistical modelling of loss data The normal distribution that forms the basis of much of statistical inference needs to be replaced by a loss distribution showing a thicker upper tail. The lognormal distribution

4 Modelling extremes Rodney Coleman Page 4 had this role historically in econometrics theory and the Weibull in reliability modelling. Our experience is that the tail of the lognormal is insufficiently thick, failing to model the extremely large losses. Example 2. An illustrative small data set 7, 10, 15, 18, 20, 21, 22, 24, 25, 32, 36, 52, 80, 120. The spot diagram suggests that we look for a skew distribution to model their distribution. Table 2 shows the results of probability plots given by the Minitab statistics package. We see from the table that none of the distributions offers good estimation of the tail. With a sample of just 14 values the confidence bands are very wide, and the largest value 120 is way beyond every fitted 95% quantile. The fitted p-value (1 F (120)) for the largest value 120 is also given in the table, and demonstrates its status as an outlier. Distribution Q(0.95) 95% CI Q(0.99) 95% CI 1 F(120) Normal 83 (60, 107) 104 (74, 134) Lognormal 86 (48, 154) 141 (68, 295) Gumbel 94 (73, 116) 111 (87, 135) < 0.01 Weibull 87 (56, 134) 121 (74, 194) 0.01 Logistic 70 (45, 95) 94 (59, 129) < 0.01 Loglogistic 83 (43, 161) 162 (64, 411) 0.02 Table 2: Estimated tail quantiles are given for the data of Example 2, together with their estimated 95% confidence intervals, for various distributions Figure 1: The probability density function of a generalised extreme value distribution GEV (ξ, µ, σ) = GEV (0.4, 1500, 900).

5 Modelling extremes Rodney Coleman Page The small sample problem An extreme loss in a small sample is over-representative of its 1 in a 100 or 1 in a 1000 chance. Extreme losses in a small sample are under-represented if there are no extreme losses observed. We must conclude that we cannot, through fitting data, model the true distribution of a heavy-tailed distribution. This is true whatever model we try to fit. 3.2 Extreme value distributions Two heavy-tailed models that can allow large observations arise out of Extreme Value Theory. These are the Generalised Extreme Value distribution (GEV) and the Generalised Pareto Distribution (GPD) (see Appendix 7.1). The GEV is the limit distribution for the sample maxima, the largest observation in each period of observation. The GPD is the limit distribution of losses which exceed a high enough threshold. Nevertheless, there is no reason why the GEV should not be used for modelling threshold exceedances, or the GPD for sample maxima, or any other heavy-tailed distribution, as we will not necessarily be approaching the asymptotic position unless we have a very large data set. In this latter case the small sample problem does not arise, and tests made for model fit can be powerful enough to identify the lack thereof. 3.3 Parameter estimation Errors in estimating the shape parameter ξ can make estimates of the loss severity distribution and its quantiles quite unstable, especially its high quantiles. Table 3 shows how relatively small changes in the estimated shape parameter can lead to significant errors in high quantile estimation. In reading quantile function plots, differences are shown through changes in y-axis values for fixed x-axis values. Several methods have been developed for the estimation of the shape parameter. These can lead to estimates that differ considerably more than is shown in Table 3. ξ 100p% % % % Table 3: Quantiles Q(p) for GEV (ξ, 0, 1). Maximum likelihood estimation weights each value equally. Interest centred on the tail suggests that more weight should be given to the largest observations. Hosking and Wallis (1997) have developed a probability weighted method of moments (PWM) (see Appendix 7.2). Hill estimation for the shape parameter (Hill 1975) uses just the data in the tail (see Appendix 7.3). Other tail data methods give Dekkers-Einmahl-DeHaan and Pickands estimators (Embrechts et al. 1997), but these would require larger data sets than the 12 or 14 values used to illustrate this article.

6 Modelling extremes Rodney Coleman Page Figure 2: Quantile functions for GEV (ξ, 0, 1), ξ = 0.42, 0.46, Let the order statistics from a random sample of size n be denoted by x 1:n > x 2:n > > x n:n. Table 4 gives the Hill estimates of ξ for the GEV fit of Example 2. Choice of how many tail observations to use can lead to problems of reproducibility of the estimation. My practice with a small sample is to use a trimmed mean ξ = 1 n 6 n 3 k=4 ξ k to estimate ξ using Hill estimates. The trimming should be adjusted so that, from inspection of the Hill estimates, averaging is over a flattish subset of estimates. This trimmed mean is not appropriate for large data sets when a comparison can be made with other estimation methods. 3.4 Fitting the data of Example 2 k x k: ξ k Table 4: Hill s estimation applied to the data {x k:14 } of Example 2. In practice, we often use PWM+H estimates, probability weighted moments for estimating µ and σ, with Hill estimation for ξ. A comparison can be made with using PWM for all three parameters and this is done in Table 6 for the same data. In the table: GEV1 and GPD1 have all three parameters fitted by PWM; GEV2 and GPD2 use the Hill trimmed mean estimate for ξ, with the other two parameters fitted by PWM; GEV3 gives the three parameters fitted by maximum likelihood (using the Xtremes package of Reiss and Thomas (2001)); GEV4, GEV5 and GEV6 use arbitrarily chosen values of ξ, with the other two parameters fitted by PWM. The GPD and maximum likelihood fitted models are here

7 Modelling extremes Rodney Coleman Page 7 to demonstrate the difficulty of fitting a long-tailed model to small samples. Recall, our objective is to model the extreme tail. Banking regulators are proposing that the 99.9% fitted quantile be reported as a measure of risk. In Figure 3, the three probability density functions (GEV1, GEV3 and GPD1 of Table 6) appear to have converging tails. However the quantile function plots show significant differences. For the GPD, the location parameter µ is the lower bound on the distribution. GEV1 GEV2 GEV3 GPD1 GPD2 GEV4 GEV5 GEV6 x ξ = Table 5: Fitted values from GEV and GPD for Example 2. GEV1 GEV2 GEV3 GPD1 GPD2 GEV4 GEV5 GEV6 µ σ ξ Q(0.5) Q(0.9) Q(0.95) Q(0.99) Q(0.999) Table 6: Fitted parameters and quantiles from GEV and GPD for Example 2.

8 Modelling extremes Rodney Coleman Page Figure 3: a) The probability density functions for the fitted distributions GEV1 (black), GEV3 (red) and GPD1 (green) of Table 6. b) and c) The quantile functions for high quantiles. 3.5 A simulation study A simulation study using data from 30 independent 12-samples from GEV(0.5, 0, 1) shows through the large estimated standard errors (Table 7) that PWM+H gives unstable estimation of high quantiles. In this simulation the probability plotting points p j:n were {u j:n }, the order statistics for a random sample of size n from Uniform(0,1), with a new random sample taken for fitting each GEV sample (see Appendix 7.2). Sample sizes of about 100 seem to be required. Other estimation methods fare worse. Quantile 95% 99% True value Average estimate Est. standard error Table 7: A simulation study using data from 30 independent 12-samples from GEV(0.5, 0, 1). Simulations (Embrechts et al. 1997) demonstrate that problems of estimation for heavytailed distributions can arise even when the exact model is known and there are lots of data. Tests of fit for any particular heavy-tailed distribution appear to lack the power to detect a lack of fit for any realistically sized data set, as was seen in our studies (Cruz et al. 1998). One advantage of PWM over maximum likelihood is its easier implementation, and its greater applicability in small samples (Landwehr et al. 1979). As Hosking et al. (1985) noted, although PWM estimators are asymptotically inefficient compared with maximum likelihood estimators (MLEs), no loss of efficiency is detectable in samples of 100 or less. The biases of PWM estimators are small and they decrease rapidly as the sample size increases. The standard deviations of the PWM estimators are comparable with those of MLEs for moderate sample sizes (n=50, 100) and are often substantially less than those of MLEs for small samples.

9 Modelling extremes Rodney Coleman Page 9 4 A case study One mechanism for estimation is to fit a GEV distribution to the sample maxima of loss data over each of the preceding 12 months. The estimation process can be applied daily, weekly or monthly on a rolling 12-month basis. In view of the heavy-tail characteristics just think of the size of a potential catastrophic loss a very high quantile such as 99% can yield a figure which implies an economic capital allocation beyond that which would be feasible. Recall the Basel Committee proposal that a 99.9% quantile value over a one year holding period be calculated. Severe fraud loss events will show the fitted parameters to vary in time, since each large loss event will distort the shape of the fitted distribution. Historic data cannot be assumed to be like recent or future data, so long data series cannot be assumed to improve the estimation. My view is that although no estimates will be reliable, using rolling 12-month data will highlight the impact of each extreme loss, and allow its effect to decline in time. It also provides a pricing mechanism which reflects the occurrence of extreme loss and which can be compared with hedge prices, such as insurance costs. Example 3. Retail banking fraud The data are the order statistics for the 12 monthly maxima of the losses from fraud during 1995 at a large UK retail bank. The data come from Cruz (2002). 600, , , , , , , , , , , , Calculating losses to the nearest penny cannot easily be explained. Ignoring the variety of precisions gives the following table of fitted distributions and quantile estimates (values given in units of $1000), with fitted values (rounded) for the five largest order statistics µ σ ξ Q(0.5) Q(0.9) Q(0.95) Q(0.99) Q(0.999) GEV GEV GPD GEV1 GEV3 GPD Table 8:

10 Modelling extremes Rodney Coleman Page 10 Wide differences in parameter values of GEV1 (PWM) and GEV3 (MLE) can be seen, with resulting high variability in the fitted 99% and 99.9% quantiles Figure 4: Fitted probability density functions and those parts of the quantile function plots in the long tail. GEV1 (black), GEV3 (red) and GPD1 (green). 5 Pricing the risk We have seen how the lack of good data in the tail of the loss severity distribution creates uncertainty in the model, and in the high quantile values. This carries through to any price that would be put on the risk when transferring it to insurance or cat bonds. How we should be addressing this pricing is to be the subject of another paper, but once again illustrates this uncertainty. Let us assume an excess-of-loss insurance contract, with an excess of u ($100,000 for example), with losses over a calendar year to form the basis of the premium for the following year. Then pricing requires us to estimate the shortfall, the excess of loss over the threshold u, when a failure has resulted in such a loss. For a random variable X from the loss distribution, this is the conditional distribution of X u given that X > u. Estimation procedures and tests for the appropriateness of the choice of model (mean excess estimation and Q-Q plots) are found in Embrechts et al. (1997). A useful consequence of using the GPD model for fitting loss data is that its mean excess is linear in u. For ξ < 1 and u > 0, E(X u X > u) = σ + ξu 1 ξ A plot of sample values of X u against u will give approximately a straight line with slope ξ/(1 ξ). In practice the few sample values with large u can make the GPD fitting unsatisfactory. To obtain annual premiums we need to take into account the frequency distribution, so that we can multiply the single event price by the number expected in any year. In practice we might choose to use the frequency over the previous calendar year and the estimated mean shortfall for that year s losses.

11 Modelling extremes Rodney Coleman Page 11 6 Alternative methodologies 6.1 Resampling techniques The jackknife and bootstrap (Efron and Tibshirani 1993) can be used to obtain sampling properties of the estimates such as confidence intervals. However this will not correct for the small sample problem. Similarly for the hierarchical Bayes resampling of Medova (2000) and Kyriacou & Medova (2001). This latter treats non-stationarity by letting the GPD parameters be random from distributions which themselves have parameters (hyperparameters). The frequency process, generally modelled as a Poisson process independent of the loss size process, can be given hyper-parameters. They simulate a loss event (severity plus time of occurrence) from the GPD and Poisson process having these estimated hyperparameters. The estimates are all updated as new simulated data are created. 6.2 Econometric modelling of loss data The direct use of linear predictive modelling can, for example, relate the frequency and severity of loss to underlying predictor variables (risk indicators) such as system downtime (SD), number of employees (E), and number of transactions (T). It is essential that this be accompanied by model checking and validation. Values of R 2 of 95% or more (showing that the model has accounted for at least that amount of variability in the data) should not be taken as evidence for the model s predictive power. Backtesting for predictive power is important. 6.3 Dynamic financial analysis (DFA) DFA refers to enterprise-wide integrated financial risk management (see Kaufmann et al. (2001)). It involves (mathematical) modelling of every business line, with dynamic updating in real time. Research for the insurance industry is centred at RiskLab, sponsored by ETH Zürich, Credit Suisse, Swiss Re, and UBS. 6.4 Bayesian Belief Networks (BBNs) A BBN is an acyclic graph of nodes connected by directed links of cause and effect, with a probability table at each node. Their development arose out of concern that cause and effect relationships are not incorporated into statistical/econometric modelling. Also concern that risk assessment needs to incorporate uncertainty, diverse information, expert judgement, and partial information. Probability relationships via Bayes Theorem allow scenarios to be transmitted through the network. 7 Appendices 7.1 The GEV and GPD distributions Generalised Extreme Value Distribution (GEV)

12 Modelling extremes Rodney Coleman Page 12 For a random variable X from GEV (ξ, µ, σ), where µ and σ are the location and scale parameters and ξ is the shape parameter, let z = (x µ)/σ, then P (X x) = F ξ,µ,σ (x) = F ξ,0,1 (z) = { exp{ exp( z)}, for all z (ξ = 0) exp{ (1 + ξz) 1/ξ }, for 1 + ξz 0 (ξ 0) As ξ 0, the ξ 0 case gives the Gumbel distribution, for ξ > 0 we have the Fréchet distribution, and for ξ < 0 the Weibull distribution. Generalised Pareto Distribution (GPD) For a random variable X from GP D(ξ, µ, σ), P (X x) = F ξ,µ,σ (x) = F ξ,0,1 (z) = 1 exp( z), for z 0 (ξ = 0) { for z 0 1 (1 + ξz) 1/ξ (ξ > 0), 0 < z < 1/ξ (ξ < 0) As ξ 0, the ξ 0 cases give the Exponential distribution, for ξ > 0 we have the Pareto distribution, and for ξ < 0 the Beta distribution. 7.2 Method of moments with probability weighted moments The rth PWM is ω r = 1 0 Q(p) p r dp = E{Q(p) p r } where p has a uniform distribution over [0, 1]. For the order statistics x 1:n > x 2:n > > x n:n from a random sample of size n, the corresponding sample values are ω r = 1 n n x j:n p r j:n j=1 where p j:n = n j n j + 1 or n n + 1 p r j:n = (p j:n ) r or E{(U j:n ) r } ( = E(U j:n ) ) or u j:n and {u j:n } are the order statistics u 1:n > u 2:n > > u n:n for a random sample of size n from Uniform(0,1). The PWM estimates of µ, σ and ξ are found by substituting the sample values ω 0 = x, ω 1 and ω 2 for their theoretical values. Let m 1 = 2ω 1 ω 0, m 2 = 3ω 2 ω 0. For the GEV having ξ < 1, ξ is the solution to m 2 = 3ξ 1 m 1 2 ξ, ie ξ = c c2 1

13 Modelling extremes Rodney Coleman Page 13 approximately, where c = ln 2 ln 3 m 1 m 2 σ = m 1 ξ (2 ξ 1)Γ(1 ξ), µ = ω 0 + σ {1 Γ(1 ξ)} ξ where Γ(ν) = 0 t ν 1 e νt dt is the Gamma Function. For the GPD having ξ < 1, ( ) 1 m2 ξ = m 1 σ = m 1 (2 ξ) (1 ξ), µ = ω 0 σ 1 ξ These formulae for GEV and GPD agree with those in Hosking and Wallis (1997) and Reiss and Thomas (2001). Those in Cruz (2002, page 70) need to be corrected. 7.3 Hill estimates For the order statistics x 1:n > x 2:n > > x n:n from a random sample of size n, the Hill estimates of the shape parameter are ξ k = 1 k 1 k 1 j=1 ln x j:n ln x k:n (k = 2,..., n) The form ξ k = 1 k k ln x j:n ln x k:n (k = 2,..., n) j=1 is sometimes used, but this does not improve the fit in respect of the data sets of this paper. References Basel Committee on Banking Supervision. Regulatory Treatment of Operational Risk. September (See (2001a) Working Paper on the Bank for International Settlements Basel Committee on Banking Supervision. (2001b) Consultative Paper 2. Bank for International Settlements Basel Committee on Banking Supervision. (2001c) Consultative Paper 2.5. Bank for International Settlements British Bankers Association. (1999) BBA/ISDA/RMA Research Study on Operational Risk. (See Coleman R. (2000) Using modelling in OR management. Conference Operational Risk in Retail Financial Services (London, June 2000) (Unpublished: rcoleman/iir.pdf)

14 Modelling extremes Rodney Coleman Page 14 Coleman R & Cruz M. (1999) Operational risk measurement and pricing (Learning Curve). Derivatives Week VIII, No. 30, July 26, 5 6. (Revised preprint: rcoleman/oprisklcv.html) Coles S & Powell E. (1996) Bayesian methods in extreme value modelling: A review and new developments. International Statistical Review Cruz M G. (2002) Modeling, measuring and hedging operational risk. Wiley Cruz M, Coleman R & Salkin G. (1998) Modeling and measuring operational risk. Journal of Risk Daníelsson J, Embrechts P, Goodhart C, Keating C, Muennich F, Renault O & Shin H S. (2001) Submitted in response to the Basel Committee for Banking supervision s request for comments. May, ( Efron B & Tibshirani R. (1993) An Introduction to the Bootstrap. Chapman & Hall Embrechts P. (Editor) (2000) Extremes and Integrated Risk Management. Books, London Embrechts P, Klüppelberg C & Mikosch T. (1997) Modelling Extremal Events. Springer Gilchrist W. (2000) Statistical modelling with quantile functions. Hall/CRC Risk Chapman & Hill B M. (1975) A simple general approach to inference about the tail of a distribution. Annals of Statistics Hosking J R M & Wallis J R. (1997) Regional Frequency Analysis: An Approach Based on L-Moments. Cambridge University Press Johnson N L, Kotz S & Balakrishnan N. (1995) Continuous Univariate Distributions. Volume 2. (2nd Edition). Wiley Jorion P. (1997) Value at Risk: The New Benchmark for Controlling Market Risk. McGraw-Hill Kaufmann R, Gadmer A & Klett R. (2001) Introduction to Dynamic Financial Analysis. ASTIN Bulletin 31 (May) (Available also from Kyriacou M N & Medova E A. (2000) Extreme values and the measurement of operational risk II. Operational Risk 1, Landwehr J, Matalas N & Wallis J R. (1979) Probability weighted moments compared to some traditional techniques in estimating Gumbel parameters and quantiles. Water Resources Research Medova E A. (2000) Extreme values and the measurement of operational risk I. Operational Risk 1, Reiss R-D & Thomas M. (2001) Statistical Analysis of Extreme Values. (2nd Edition). Birkhäuser