A VaR too far? The pricing of operational risk Rodney Coleman Department of Mathematics, Imperial College London

Transcription

1 Capco Institute Paper Series on Risk, 03/2010/#28 Coleman, R, 2010, A VaR too far? The pricing of operational risk, Journal of Financial Transformation 28, A VaR too far? The pricing of operational risk Rodney Coleman Department of Mathematics, Imperial College London Abstract This presentation is a commentary on current and emerging statistical practices for analysing operational risk losses according to the Advanced Measurement Approaches of Basel II, the New Basel Accord. In particular, the limitations of the ability to model operational risk loss data to obtain high severity quantiles when the sample sizes are small are exposed. The viewpoint is that of a mathematical statistician. Introduction Value-at-Risk (VaR) entered the financial lexicon as a measure of volatility matched to riskiness. Basel I saw that it could be a risk-sensitive pricing mechanism for computing regulatory capital for market and credit risks. Basel II extended its scope to operational risk, the business risk of loss resulting from inadequate or failed internal processes, people, systems, or from external events. The European Parliament agreed and has passed legislation ensuring that internationally active banks and insurers throughout the EU will have adopted its provisions from However, to put the problem of applying the Basel II risk-sensitive advanced measurement approaches (BCBS, 2006) in perspective, consider the role of quantitative risk modelling in the recent global financial crisis. This began with the credit crunch in August 2007 following problems created by sub-prime mortgage lending in the USA. A bubble of cheap borrowing was allowed to develop, with debt securitised (ie off-loaded) and insured against default, creating a vicious spiral. In their underwriting, the insurers failed to adjust for the risk that property values might fall, with consequent defaults on mortgage repayment. This bad business decision caused havoc at banks who lent the money, and with investors who bought the debt, and insurers without the means to pay out on the defaults. Where was VaR when it was needed? Which is to say, what part did financial modelling play in preventing this? The answer has to be nothing much. Further, it can only marginally be put down to operational risk, since operational risk excludes risks which result in losses from poor business decisions. Oprisk losses will often stem from weak management and from external events. Basel II, the New Basel Accord, set out a regulatory framework for operational risk in internationally active banks. In so doing, it gave a generally accepted definition for operational risk, previously categorised as other risks. More importantly, it is Basel II s role in driving developments in operational risk management practices, the search for robust risk measurement, and the prospect of being subject to regulatory supervision and transparency through disclosure (Pillars 2 and 3 of the Accord), that have led in a short time to operational risk becoming a significant topic in business and management studies. Basel II The Accord sets out a risk sensitive way of calculating reserve capital to cover possible defaults. Institutions are required to categorise operational risk losses by event type, promoting identification of risk drivers. There is no mandated methodology. Pillar 1 of Basel II gives three ways of calculating the operational risk capital charge, with increasing complexity, but benefiting from a reducing charge. We shall be considering its requirements in respect of its highest level, the Advanced Measurement Approaches (AMA), which requires that the banks model loss distributions of cells over a business line/loss event type grid using operational risk loss data that they themselves have collected, supplemented as required by data from external sources. 1

2 Pillar 2 of the Accord requires banks to demonstrate that their management and supervisory systems are satisfactory. Pillar 3 relates to transparency, requiring them to report on their operational risk management. It is these two latter pillars that are probably going to have a greater impact in protecting global finance than loss modelling. Solvency II, the European Union s regulatory directive for insurers, has adopted the same three pillars. directive will come into force throughout the EU in In November 2007 the US banking agencies approved the US Final Rule for Basel II. Banks will be grouped into the large or internationally active banks that will be required to adopt AMA, those that voluntarily opt-in to it, and the rest who will adopt an extended version of the earlier Basel I. We note that in the rule book, summarised in BCBS (2006), operational risk sits in paragraphs 644 to 679, occupying the final 12 pages, pages 140 to 151. Advanced measurement approaches Attention is directed to the following passages taken from BCBS (2006) that cover the modelling requirements for the advanced measurement approaches. ( 665) A bank s internal measurement system must reasonably estimate unexpected losses based on the combined use of internal and relevant external loss data, scenario analysis and bank-specific business environment and internal control factors (BEICF). ( 667) The Committee is not specifying the approach or distributional assumptions used to generate the operational risk measure for regulatory capital purposes. However, a bank must be able to demonstrate that its approach captures potentially severe tail loss events. Whatever approach is used, a bank must demonstrate that its operational risk measure meets a soundness standard [...] comparable to a one-year holding period and a 99.9th percentile confidence interval. ( 669b) Supervisors will require the bank to calculate its regulatory capital requirement as the sum of expected loss (EL) and unexpected loss (UL), unless the bank can demonstrate that it is adequately capturing EL in its internal business practices. That is, to base the minimum regulatory capital requirement on UL alone, the bank must be able to demonstrate [...] that it has measured and accounted for its EL exposure. ( 669c) A bank s risk measurement system must be sufficiently granular to capture the major drivers of operational risk affecting the shape of the tail of the loss estimates. ( 669d) The bank may be permitted to use internally determined correlations in operational risk losses across individual operational risk estimates [...]. The bank must validate its correlation assumptions. ( 699f) There may be cases where estimates of the 99.9th percentile confidence interval based primarily on internal and external loss event data would be unreliable for business lines with a heavy-tailed loss distribution and a small number of observed losses. In such cases, scenario analysis, and business environment and control factors, may play a more dominant role in the risk measurement system. Conversely, operational loss event data may play a more prominent role in the risk measurement system for business lines where estimates of the 99.9th percentile confidence interval based primarily on such data are deemed reliable. ( 672) Internally generated operational risk measures used for regulatory capital purposes must be based on a minimum five-year observation period of internal loss data. ( 673) A bank must have an appropriate de minimus gross loss threshold for internal loss data collection, for example, 10,000 euro. ( 675) A bank must use scenario analysis of expert opinion in conjunction with external data to evaluate its exposure to high-severity events. [...] These expert assessments could be expressed as parameters of an assumed statistical loss distribution. Event types and business lines Table 1 shows the seven designated loss event types (ETs) given in Basel II, also adopted by Solvency II. Table 2 shows the eight broad business lines (BLs) within the banking sector given in Basel II. Together they create an 8 7 grid of 56 BL/ET cells. This 2

3 Event types (ET) Internal fraud External fraud Employment practices and workplace safety Clients, products and business practice Damage to physical assets Business disruption and system failures Execution, delivery and process management Table 1: Event types for banking and insurance under Basel II and Solvency II Business unit Investment banking Banking Others Business line Corporate finance Trading and sales Retail banking Commercial banking Payment & settlement Agency services Asset management Retail brokerage Table 2: Business units and business lines for international banking activities under Basel II The Operational Risk Consortium Ltd (ORIC) database of operational risk events, established in 2005 by the Association of British Insurers (ABI) has nearly 2000 events showing losses exceeding $10,000 in the years 2000 to Table 3 based on a report for ABI (Selvaggi, 2009) gives percentages of loss amounts for those BL/ET cells having at least 4% of the total loss amount. This information tells little about the actual events. For this we need Level 2 and Level 3 categories, the seven event types being Level 1. To illustrate this, again from Selvaggi (2009), Table 4 shows the most significant Level 2 and Level 3 event types in terms of both severity and frequency (values over 4%) from ORIC. This table excludes losses arising from the widespread mis-selling of endowment policies in the UK in the 1990s. The victims were misled with over-optimistic forecasts that their policies would pay off their mortgages on maturity. Among the insurers, Abbey Life was fined a record $1 million in December 2002, and paid $160 million in compensation to 45,000 policy holders. The following March Royal & Sun Alliance received a fine of $950,000. Later that year, Friends Provident was fined $675,000 for mis-handling complaints, also an operational risk loss. A widely accepted approach in risk management is to identify the major risks faced by an organisation, measured by their impact in terms of their frequency and severity. In many cases, not much more than a handful of operational risks will give rise to the most serious of the loss events and loss amounts. However, the day-to-day management requires bottom up as well as top down risk control. There needs to be an understanding of the risk in all activities. Aggregating losses over business lines and activities would tend to conceal low impact risks behind those having more dramatic effect. The bottom up approach is thus a necessary part of seeing the complete risk picture. Aggregation at each higher level will inform management at throughout. 3

4 Event type (Level 1) Business line CP&BP ED&PM BD&SF Others Total Sales and distribution Customer service/policy Accounting/finance IT Claims Underwriting Others Total Table 3: Business line / event type grid showing percentages of loss amounts (min 4%) in the ORIC database ( ) (Source: Selvaggi, 2009) Level 2 Level 3 Size % Freq % Advisory activities Mis-selling (non-endowment) 13 9 Transaction capture, Accounting error 12 execution, maintenance Inadequate process documentation 8 Transaction system error 8 6 Management information error 7 Data entry errors 7 5 Management failure 5 Customer service failure 4 16 Suitability, disclosure, fiduciary Customer complaints 6 4 Systems Software 6 Customer/client account Incorrect payment to client/customer 9 management Payment to incorrect client/customer 4 Theft and fraud Fraudulent claims 4 Total Table 4: Levels 2 and 3 event categories from insurance losses in ORIC ( ) showing loss amount or loss frequency of 4% or more. (Source: Selvaggi, 2009) Expected loss and unexpected loss An internal operational loss event register would typically show high impact events at low frequency among events of high frequency but low impact. A financial institution might therefore sort its losses into: expected loss (EL) to be absorbed by net profit, unexpected loss (UL) to be covered by risk reserves (so not totally unexpected), and stress loss (SL) requiring core capital or hedging for cover. The expected loss per transaction can easily be embedded in the transaction pricing. It is the rare but extreme stress losses that the institution must be most concerned with. This structure is the basis of the loss data analysis approach to operational risk. Hard decisions need to be made in choosing the EL/UL and UL/SL boundaries. As well as these, a threshold petty cash limit is needed to set a minimum loss for recording it as an operational loss. Loss events with recovery and other near 4

5 miss events will also need to be entered into the record. We see that, for regulatory charges, Basel specifies EL to be the expectation of the fitted loss distribution and UL to be its 99.9th percentile. For insurers, Solvency II sets UL at the 99.5th percentile. Basel further permits the use of UL as a single metric for charging. Let us compare this with the classical VaR, the difference between UL and EL on a Profit-and-Loss plot. Expectation measures location, and VaR measures volatility. Oprisk losses are non-negative, so Basel appears to be using zero as its location measure, making it possible to rely on just UL as its opvar metric. Basel s referring to confidence interval for the confidence limit UL adds further weight to thinking it refers to the interval (0, UL). Now UL gives no information about potential losses larger than itself. In fact the very high once-in-a-thousand-years return value is deemed by Basel to capture pretty much all potential losses. This will not always be the case for heavy-tailed loss distribution models. In survival analysis and extreme value theory we would also estimate the expected shortfall, the expectation of the loss distribution conditioned on the values in excess of UL. This has been called CVaR (conditional VaR). There being a minimum threshold to registered losses, we really need to reconstruct the loss distribution for under-the-threshold losses to include them in the expectation and 99.9th percentile. I have always felt that the use of the difference between a moment measure (expectation) and a quantile measure (percentile) in the VaR calculation unnecessarily complicates any investigation of its statistical properties. Why not have median and high percentile? External data During a recent supervisory visit to an insurance company, the FSA was told that they had only two operational risk loss events to show, with another six possibles. This extreme example of the paucity of internally collected oprisk data, particularly the large losses that would have a major influence in estimating reserve funding, means that data publicly available or from commercial or consortia databases needs to be explored to supplement internal loss events. Fitch s OpVar is a database of publicly reported oprisk events showing nearly 500 losses of more than ten million dollars between 1978 and 2005 in the US. The 2004 Loss Data Collection Exercise (LDCE) collected more than 100 loss events in the US of 100 million dollars or more in the ten years to The Operational Riskdata exchange Association (ORX) is well established as a database of oprisk events in banking. It is a consortium collecting data from thirty member banks from twelve countries. It has more than 44 thousand losses, each over 20,000 euro in value. Apart from ORIC, individual insurance companies will have their own claims records containing accurate settlement values. Combining internal and external data When data from an external database is combined with an in-house data set, the former are relocated and scaled to match. The external data are standardized by subtracting its estimated location parameter value (eg its sample mean m) and dividing the result by its estimated scale value (eg its sample standard deviation s) from each datum. Then we adopt the location m and scale s of the internal data. So, if y is a standardised external datum, the new value of the external datum is z = m + s y. Finally we check the threshold used for the internal database against the transformed threshold for the relocated and rescaled external data, and set the larger of the two as the new threshold, eliminating all data points that fall below. This can lead to strange statistical results. Table 5 shows that sample statistics for pooled data can lie outside the range given by the internal and external data sets (Giacometti et al, 2007, 2008). We note that the same statistics using logscale data show this phenomenon for commercial banking skewness and kurtosis. Perhaps the location and scaling metrics are inappropriate, and we need model specific metrics for these. We are told that the sample size of the external data is about 6 times that of the internal set. The small sample problem An extreme loss in a small sample is over-representative of its 1 in a 1000 or 1 in a chance, yet underrepresented if not observed. Indeed we find the largest losses are overly influential in the fitted model. So, we 5

6 Business line sample statistic internal data external data pooled data Retail banking mean 15,888 37,917 11,021 std deviation 97, ,787 59,968 skewness kurtosis min 500 5,000 1,230 max 2,965,535 8,547,484 2,965,535 Commercial banking mean 28,682 40,808 19,675 median 2,236 12,080 1,347 std deviation 133, ,409 83,253 min 500 5, max 1,206,330 20,000,000 2,086,142 Table 5: Some descriptive statistics for pooled data (abstracted from Giacometti et al, 2007) must conclude that fitting a small data set cannot truly represent the loss process whatever model is used. As a statistician I am uncomfortable with external data. An alternative device which I feel to be more respectable is to fit a heavy-tailed distribution to the data and then simulate a large number of values from this fitted distribution, large enough to catch sufficient high values for statistical analysis. A basic principle in statistics is that inferences about models in regions far outside the range of the available data must be treated as suspect. Yet here we are expected to do just that when estimating high quantiles. Stress and scenario testing Scenario analysis has as its object to foresee and consider responses to severe oprisk events. However merging scenario data with actual loss data will corrupt that data and distort the loss distribution. These scenario losses would be those contributing to a higher capital charge than otherwise would be the case. Stress testing is about the contingency planning for these adverse events based on the knowledge of business experts. A potential loss event could arise under three types of scenario: Expected Loss (optimistic scenario) Unexpected Serious Case Loss (pessimistic scenario) Unexpected Worst Case Loss (catastrophic scenario). This mimics the Expected Loss, Unexpected Loss, and Stress Loss of the actuarial approach. Probability modelling of loss data Loss data is not gaussian. The normal distribution model that backs so much of statistical inference will not do. Loss data by its nature has no negative values. There are no profits to be read as negative losses. Operational losses are always a cost. Further a truncated normal distribution taking only positive values gives too little probability to its tail, making insufficient allowance for large and very large losses. The lognormal distribution has been used instead historically in econometrics theory, and the Weibull in reliability modelling. In practice even the lognormal will fail to pick up on the extremely large losses. Two models that can allow large observations come from Extreme Value Theory. These are the Generalised Extreme Value distribution (GEV) and the Generalised Pareto Distribution (GPD) They are limit distributions as sample sizes increase to infinity. These distributions are used in environmental studies (hydrology, pollution, sea defences, etc.) as well as in finance. The GEV and GPD each have three parameters, μ giving location, σ scale, and ξ shape, which we vary to obtain a good fit. The location μ and shape ξ are not to be identified with the population mean and population variance. For the GPD μ is the lower bound of the range. Figure 1 shows the 6

7 form of their respective probability density functions. The lognormal and Weibull have just two parameters, and so lack flexibility. Four and more parameter models such as Tukey s g-and-h class of distributions, are also gaining users, but require more data than is usually available, though they have been seen to capture the loss distribution of aggregated firm-wide losses. An extensive list with plots and properties can be found in Young and Coleman (2009) Figure 1: The probability densities of GEV (0.70, 230, 100) and GPD (0.70, 150, 125) Figure 2: The sample cumulative distribution function with the four fitted models of Table 4. (Source: Young and Coleman, 2009, p.401) Fitting severity models We fit the GEV and GPD to the 75 losses given in Cruz (2002, p.83). For each model we use two fitting processes: maximum likelihood for all three parameters, and maximum likelihood for the location and scale, but the Hill estimator for the shape parameter (Hill, 1975). Figure 2 shows the sample cumulative distribution function (the observed proportion of values less than x) shown as steps, together with four fitted cumulative distribution functions (the height y is the probability of obtaining a future value less than x). The range of observation is (143, 3822). From Figure 2 we can see a good fit in each case. In Table 6 what we also see is that the estimated losses at large quantiles (reading x-values from fitted y-values) differ greatly between the fitted models, that is to say, way beyond the largest observation. Estimation far outside a data set is always fraught and can lead to significant errors in high quantile estimation. Basel II asks for the 99.9 percentile, Q(0.999), Solvency II for the 99.5 percentile, Q(0.995). These quantiles are the opvar. 7

8 Fitted model GEV GEV GPD GPD Parameter estimates μ σ ξ Quantiles Q(0.9) Q(0.95) Q(0.975) Q(0.99) Q(0.995) Q(0.999) Data Model values Table 6: The parameters, quantiles, and fitted values of GEV and GPD models when fitted to loss data. (Source: Young and Coleman, 2009, pp ) A simulation study of GPD (0.70, 150, 125) gave an estimated 95% confidence interval for Q(0.999) of (5200, 9990), very wide indeed. The computations were made using Academic Xtremes (Reiss and Thomas, 2007). The statistical operations were carried out without seeking any great precision. The data in thousands of dollars were rounded to the nearest thousand dollars, the parameters of the fitted models are rounded to two significant digits, the fits were judged by eye, the simulation for the estimated confidence interval was based on a simulation of only 4000 values. My first big surprise was that a good fit could be achieved, secondly that it could be achieved so easily, my third that we could have four close fits, and with such a variety of parameter values. Table 7 shows parameter estimation variability through four simulations of 1000 values from GEV (0.53, 230, 130). We see that 1000 values can be a small sample in that it may not be enough to provide precise estimation. Why such lack of concern for precision? Highly sophisticated methods can do no better when we have such variability in the resulting output at high quantiles far beyond the data. We see this variability right away in the estimated parameter values. The fit was judged by eye. Why were goodness-of-fit tests not used? They are by their nature conservative, requiring strong evidence for rejecting a fit, evidence not available here. Modelling body and tail separately With sufficient data we may see that the tail data needs to be modelled separately from the main body. We might consider fitting a GEV to the body and a GEV to the tail, reflecting the Extreme Value theory properties. Three parameters each and one for the location of the join makes seven, but having the two probability densities meeting smoothly provides two relations between them, and using the same shape parameter, brings the problem to four unknowns (Giacometti et al, 2007; Young and Coleman, 2009, p.397). Modelling frequency Standard statistical methods can be used to fit Poisson or negative binomial probability distributions to frequency 8

9 μ σ ξ GEV Simulation Simulation Simulation Simulation Average Table 7: Parameter estimates of GEV (ξ, μ, σ) from four simulations of 1000 values from GEV (0.53, 230, 130). counts. Experience shows that the daily, weekly or monthly frequencies of loss events tend to occur in a more irregular pattern than can be fitted by either of these models. Basel asks that we combine the fitted frequency model with the fitted severity model to obtain a joint model. This loses the correlation structure from the loss events. Some topics left out Basel asks for correlation analysis: a big problem with small data sets. Mathematical finance has provided us with a correlation theory based on copulas, but not useful here. Validation techniques, such as the re-sampling methods of the jackknife and bootstrap (Efron and Tibshirani, 1993) can be used to obtain sampling properties of the estimates such as confidence intervals. Bayes hierarchical modelling (Medova, 2000; Kyriacou and Medova, 2001; Coles and Powell, 1996) treats non-stationarity by letting the GPD parameters be themselves random from distributions with parameters (hyperparameters). Dynamic financial analysis refers to enterprise-wide integrated financial risk management. (mathematical) modelling of every business line, with dynamic updating in real time. It involves Bayes belief networks (BBNs) are acyclic graphs of nodes connected by directed links of cause and effect. The nodes are events, and the states are represented by random variables. Each node event is conditioned on every path to it. For the link A to B, the random variable X associated with event A is given by the multi-on-multivariate history leading to A. The BBN requires starting probabilities for each node, and the calculation for P (B A) where A incorporates all links to it. This is a formidable task. The introspection forces risk assessment for every activity, and, once the network is up and running, it can be used for stress testing. To sum up The point being emphasised is that no methodology on its own can provide an answer. Multiple approaches should be used, both qualitative and quantitative, to aid management in acquiring a sensitivity to data and its interpretation and its use in decision making. References Basel Committee on Banking Supervision., 2006, International Convergence of capital measurement and capital standards, Bank for International Settlements Coles S., and E. Powell, 1996, Bayesian methods in extreme value modelling: A review and new developments, International Statistical Review, 64, Cruz, M.G., 2002, Modeling, measuring and hedging operational risk, Wiley 9

10 Efron, B., and R. Tibshirani, 1993, An introduction to the bootstrap, Chapman & Hall Embrechts P. (Editor), 2000, Extremes and integrated risk management, Risk Books Giacometti, R., S. Rachev, A. Chernobai, and M. Bertocchi, 2008, Aggregation issues in operational risk, The Journal of Operational Risk, 3:3, 3-23 Giacometti, R., S. Rachev, A. Chernobai, M. Bertocchi, and G. Consigli, 2007, Heavy-tailed distributional model for operational risk, The Journal of Operational Risk, 2:1, Hill, B.M., 1975, A simple general approach to inference about the tail of a distribution, Annals of Statistics, 3, Kyriacou M.N., and E.A. Medova, 2000, Extreme values and the measurement of operational risk II, Operational Risk, 1:8, Lloyd s of London, 2009, ICA: 2009 Minimum Standards and Guidance, Lloyd s Medova, E.A., 2000, Extreme values and the measurement of operational risk I, Operational Risk, 1:7, Reason, J., 1997, Managing the risk of organisational accidents, Ashgate Reiss, R.-D., and M. Thomas, 2007, Statistical analysis of extreme values (3rd edition), Birkhauser Selvaggi, M., 2009, Analysing operational risk in insurance, ABI Research Paper 16. Young, B., and R. Coleman, 2009, Operational risk assessment, Wiley 10