Integration of Qualitative and Quantitative Operational Risk Data: A Bayesian Approach

Integration of Qualitative and Quantitative Operational Risk Data: A Bayesian Approach 6 Paolo Giudici University of Pavia The aim of this chapter is to provide a Bayesian model that allows us to manage operational risk and measure internally the capital requirement, compliant with the Advanced Measurement Approaches (AMA) recommended by Basel Committee on Banking Supervision (Basel II) for internationally active banks (see, eg, Basel Committee on Banking Supervision, 2003). In general, the objective is to estimate a loss distribution and to derive functions of interest from it (such as the value-at-risk, or VAR). More precisely, losses in operational risk are realisations of a convolution between a counting process (frequency) and a number of continuous ones (severities). For a review of statistical models used in operational risk management see, eg, Cruz (2002) and Cornalba and Giudici (2004). A general problem for such models is the lack of appropriate historical databases, which makes difficult to apply statistical inference techniques to squeeze in a correct manner information to check the tail of loss distribution. Bayesian networks offer a solution to this problem, combining in a coherent way qualitative and quantitative data, as well as risk indicators and external databases. Indeed such an approach seems to well reflect the requirements of the AMA to measuring operational risk. Consider the following quotation from the 2001 working paper on the regulatory 131

OPERATIONAL RISK MODELLING AND ANALYSIS treatment of operational risks: Any risk measurement system must have certain key features to meet the supervisory soundness standard set out in this section. These elements must include the use of internal data, relevant external data, scenario analysis and factors reflecting the business environment and internal control systems. From the citation it appears clearly that a valid AMA approach should take into account, in a correct way, internal and external loss data, scenario-based expert opinions and causal factors reflecting the business environment and control systems. The Bayesian network approach proposed here can model coherently all these elements. In fact, a Bayesian statistical approach allows us to integrate, via Bayes s theorem, different sources of information, for example coming from loss data collection, self-assessment and external consortium loss data, to give a unified knowledge that allows us to manage operational risk and, at the same time, to determine a better op VAR, reflecting a holding period of one-year and a confidence level of 99.9%. In addition, a Bayesian network model, based on a Bayesian statistical approach, can give us more: we can also consider the correlation between losses of different business lines and risk types and we can evaluate the impact of causal loss factors, such as the effectiveness of internal/external control systems and the available key risk indicators (measured on people, IT and processes). For a more technical review on this proposed approach see Bilotta and Giudici (2004). 132 THE MODEL A Bayesian network is a set of nodes representing random variables and a set of arrows connecting these nodes in an acyclic manner (for a more precise definition see, eg, Jensen, 1996, or Giudici, 2003). Each node has assigned a function that describes how the state of a node depends on the parents of the node. The topology of the graph that relates the nodes defines the probabilistic dependencies between the node variables, by means of a set of conditional distributions. In this context, a Bayesian network will typically contain three type of node: loss nodes, corresponding to the desired granularity

INTEGRATION OF QUALITATIVE AND QUANTITATIVE OPERATIONAL RISK DATA level (in the simplest case the bank can decide to measure losses for each business line/risk type combination; a larger bank can add specification of its processes and owned companies); control nodes, describing the perceived efficacy of internal/external control systems; key risk indicator nodes, describing the levels of loss drivers. A Bayesian network can contain both categorical and continuous variables. In practice, to ease the computation and the interpretation of the results, most networks are either continuous (eg, contain only nodes of the continuous type) or discrete (eg, contain only nodes of the categorical types). In general, discrete Bayesian networks can describe more accurately the (multivariate) loss distribution than continuous ones; however, they require more detailed information and data that may not often be available, especially for small and medium-sized banks. In this latter case continuous networks more parsimonious models are strongly suggested. In our actual experience in implementing Bayesian networks in banks, we have used both networks, a discrete one for the larger bank and a continuous one for the smaller one. The results reported in this chapter will refer to discrete Bayesian networks, typically easier to describe. Each loss node in a discrete Bayesian network is a discrete random variable, for each combination of business lines/risk type/process. Data on such nodes will come from internal and external loss data collection as well as from self-assessment questionnaires. Each key risk indicator is a discrete random variable, for the loss combination that it affects. Such variables will be estimated using available bank performance data (typically at the monthly level). Finally, the variables corresponding to the effectiveness of the internal and external controls (CI and CE) node in each process can be formulated on the basis of expert opinions, which give information about the quality of the internal and external control system of the organisation. In our operational risk Bayesian network, each loss datum gives one piece of information to the system in the form of an operational loss for a fixed time period. This means that, if the period is the day, that the operational loss for that day is the product between the frequency and severity of the corresponding node for that day. On the other hand, each loss opinion accounts for one piece of information, in the form of an (expected) operational loss. Notice that data and opinions are both defined at the same level of operational loss. This allows their interchangeability in the model. 133

OPERATIONAL RISK MODELLING AND ANALYSIS 134 APPLICATION In order to apply the Bayesian network previously described, a loss database is needed, as in all operational risk models. In this case, the actual database can be made of four related component tables, representing: the internal loss data collected; the external loss data available (from pooled consortium data in Italy DIPO); the key risk indicators database; and the self-assessment database. The last of these is typically produced by the audit department and can thus contain expert opinion data both for the losses and the effectiveness of the controls. Bayesian networks are a rather general methodology: they can be applied to only one of the previous four datasets, or to any combination of the four, without loss of generality for the model. We can thus compare, using the same model, the quality and the value of the different datasets available. The previous datasets must of course be compatible and normalised; once they are ready we can apply Bayesian network modelling. First of all, the network is initialised with expert opinions coming from the self-assessment (qualitative or structural learning). This means that the topology of the network (that is, the relationships between the different nodes) is at first learned only with expert opinions. Each expert does not typically know how his opinions will be used but implicitly expresses in his opinions the dependencies he perceives between the different nodes. A Bayesian network is therefore a good tool to elicit and value expert prior knowledge, especially in the form of correlations between different losses, between losses and control and also key risk indicators. Figure 1 gives an example of an operational risk Bayesian network, learned exclusively with experts opinions. In the figure each node represents losses (or CI/CE) at the business line/risk type/owned bank/process. This is because the analysed data belong to a banking group, with about 60 owned companies. The figure is only a portion of the large network built. As we have seen, qualitative learning consists in determining the graphical description of the dependencies (the links between nodes) induced by the data. Such data can be of any kind: internal losses, external losses, opinions or combinations of them. Figure 1 is obtained on the basis of expert opinions only. A typically correct, and stepwise, way to proceed is to first build a network with expert

INTEGRATION OF QUALITATIVE AND QUANTITATIVE OPERATIONAL RISK DATA Figure 1 Example of a learned operational risk Bayesian network CI1/3/4/... 1/3/4/31 1/3/4/1564 1/3/4/165 1/3/4/1660 1/3/4/239 1/3/4/12a 1/3/4/672 1/3/1/1660 CI1/3/4/12 1/3/7/883 CE1/3/4/... 1/3/5/1564 1/3/6/1660 1/3/7/1468 1/3/7/1564 1/3/7/1660 CI1/3/7/... 1/3/7/31 1/3/4/674 1/3/4/12 1/3/4/883 1/3/5/91 1/4/2/1660 1/4/4/1564 1/4/4/1660 1/4/4/239 Cl1/5/6/... CE1/3/7/... 1/4/7/31 1/3/7/939 CE1/4/7... CE1/3/7... 1/4/2/1564 1/4/7/1660 CI1/3/4/91 1/5/1/463 CI1/5/1/... 1/5/6/1280 CE1/5/6/... CI1/4/7/31 1/5/4/463 1/3/7/91 CI1/5/4/... 1/5/4/464 1/5/6/520 CE1/3/4... CI1/5/6/... CE1/5/1... 1/3/4/91 CI1/5/2/... CE1/5/4... CI1/3/7/91 CE1/5/6... 1/5/6/466 1/6/7/1660 1/5/6/674 1/5/2/465 1/5/6/464 CI1/5/6/... CI1/5/6/... CE1/5/6... CE1/5/2... CE1/5/6... 1/6/2/1280 1/5/7/1660 1/5/6/1660 CE1/5/6... 1/5/7/672 1/6/2/463 1/3/7/12 1/6/7/672 1/5/7/463 1/5/6/463 CI1/5/6/... 1/6/4/674 1/6/6/1280 1/6/4/463 1/6/4/1280 1/5/7/12 1/6/3/1280 1/5/7/1280 1/5/7/674 1/6/6/674 1/8/7/91 1/5/7/464 1/6/6/1660 1/5/7/670 1/6/6/464 1/6/7/463 1/6/7/674 1/6/7/1280 1/6/7/670 1/8/7/672 1/8/6/474 1/8/4/672 opinions only, then with internal data only and then with external data. Comparison of the three gives good hints on data (and opinion) quality. If the three networks are retained based on valid data we can proceed in building up a network based on two on all three data sources. The network that best represents the company structure can then be chosen. Alternatively, one can choose the network with best performances (eg, with the least number of predictive errors, or the lowest total VAR). Once a topology is chosen, we can learn from it the conditional probabilities linking the different nodes and, from them, the marginal probability of each loss node, from which the VAR can be assessed. Given the chosen Bayesian network topology, we build the conditional probability distribution, or local distribution, P(X X ), for each variable X, given its parents X. This process is called quantitative learning. Marginal distributions (discrete distributions of probability) for each node of the network can be calculated from the local distributions, ie, for each variable X given each combination of states of its parents P. Figure 2 gives an example of such a marginal distribution, for a node appearing in Figure 1. On the basis of Figure 2 we can calculate an annual operational VAR. Note that data might not have been collected and inserted in the model at the yearly level. Indeed, while opinions can be usefully referred to a year ahead, loss data should be inserted for shorter 135

OPERATIONAL RISK MODELLING AND ANALYSIS Figure 2 Marginal distribution of loss node 1/3/4/1660 49.2447 1 5.1964 2 19.8792 3 5.1964 6 10.0906 7 5.1964 12 5.1964 15 Table 1 Result of simulation for node 1/3/7/91 VAR OP Scenarios 1.324.612,310 with 10000 1.379.085,286 with 25000 1.377.908,000 with 25000 1.357.427,000 with 50000 1.357.430,072 with 50000 1.360.500,435 with 60000 periods (days or weeks). Therefore, in order to calculate the required capital coverage for the correct holding period, it is necessary to simulate total losses from each marginal loss distribution, by summing over days (or weeks). In Table 1 we can see the result of simulation for the node 1/3/7/91 (firm/retail banking/execution, delivery and process management/process) according to different simulation sizes. From Table 1 notice that the results, as expected, become more stable as more scenarios are being considered. The previous calculation can be repeated, in a similar fashion, for all nodes of the network, thus obtaining all operational VARs. Such measures can be simply summed to obtain the overall VAR (this is because correlations have been implicitly taken into account). As we saw earlier, a Bayesian network can naturally model dependency effects of the operational losses from causal factors, such as key risk indicators and control systems. In other words, a Bayesian network can make a Bayesian model also capable of becoming a causal model. For example, in the audit context it is often important to evaluate the effectiveness of the internal and external controls in reducing operational risks. From the proposed Bayesian network we 136

INTEGRATION OF QUALITATIVE AND QUANTITATIVE OPERATIONAL RISK DATA Table 2 Evaluation of the causal effect of the control system Loss node 1/3/4/91 Loss node 1/3/4/91 Internal controls Not existing Effective External controls Not existing Not existing 8 0,10 0,20 11 0,15 0,35 13 0,20 0,40 18 0,30 0,03 23 0,25 0,02 Median class 18 11 VAR 23 23 can obtain the probability distribution of the node s effectiveness of internal controls and effectiveness of external controls, described above, as for any other node. If there are losses nodes that have control nodes as parents (that is, loss nodes that, on the basis of the structural learning process, are estimated to significantly depend on control nodes), we can evaluate the effectiveness of the control system through a summary index of the conditional probability distribution of such losses given the values of the control parents. An example of such conditional distributions is shown in Table 2. Table 2 reports two conditional distributions of the losses arising in the node 1/3/4/91: one conditional on the absence of both internal and external controls on that node; the other conditional on the internal controls being effective, and external controls absent. We remark that, in this application, the loss variable is categorical, with 25 increasing levels of severity, numbered from 1 to 25 (plus the zero class). From Table 2 notice that the introduction of effective internal controls has decreased considerably the median of the distribution (not the VAR, the latter being calculated in correspondence of the 99.99% percentile it is rather difficult to move). REFERENCES Basel Committee on Banking Supervision, 2003, The New Basel Capital Accord, Consultative Document, URL: http://www.bis.org/publ/bcbsca.htm. Bilotta, A., and P. Giudici, 2004, Modelling operational losses: a bayesian approach, Quality and reliability engineeering internationa, forthcoming. 137

OPERATIONAL RISK MODELLING AND ANALYSIS Cornalba, C., and P. Giudici, 2004, Statistical Models for Operational Risk Management, Physica A: Statistical Mechanics and its applications, forthcoming. Cruz, M., 2002, Modelling, Measuring and Hedging Operational Risk (Chichester: John Wiley & Sons). Giudici, P., 2003, Applied Data Mining: Statistical Methods for Business and Industry (Chichester: John Wiley & Sons). Jensen, F. V., 1996, An Introduction to Bayesian Networks (London: UCL Press). 138