Advanced Operational Risk Modelling Building a model to deliver value to the business and meet regulatory requirements Risk. Reinsurance. Human Resources.
The implementation of a robust and stable operational risk model for regulatory (AMA, ICAAP, Solvency II) and business management purposes presents significant challenges. These include issues concerning lack of data, difficulty in parameter selection or stability of results over time. Several firms are currently dealing with these challenges. Aon has helped more than 25 AMA banks and other types of financial institutions in the past five years in order to solve these issues. Overcoming the challenges in operational risk modelling In 2013 and 2014 fines and compensations related to mis-selling, index-rigging, operations with blacklisted countries and tax evasion moved into the eleven-figure range, putting operational risk to the top of the agenda for risk managers. Metrics like the 99.9% VaR, calibrated by the Basel Committee on losses recorded in the 2000-2006 period, went under severe stress, forcing banks to review AMA models that were generating unsustainable capital charges, well above the ones calculated by standard or basic methods. It is worth noting that these methods are going to be replaced with a new simplified approach that will take into account the size of the financial institution and will lead to increased regulatory operational risk capital for larger firms. We would like to highlight some of the key areas and lessons learned that can help firms build their next generation model. This document summarises some of the key challenges that typically need to be overcome in building, and obtaining regulatory approval, for an AMA or internal capital model. Typical characteristics that an operational risk model should have to inform the business and gain buy-in To learn how Aon can empower results for your organisation contact: Giorgio Aprile Director, Operational Risk & Capital Management +39 02 4543 4092! giorgio.aprile@aon.it Evan Sekeris Financial Institutions Global Practice Leader + 1 310 490 0243 evangelos.sekeris@aon.com aon.com Aon UK Limited is authorised and regulated by the Financial Conduct Authority. 2
The four data elements Internal Loss Data Internal loss data, including its comprehensive, accurate and consistent preparation, is the key building block to informing frequency (by event type), expected losses and the severity distributions. It represents the idiosyncratic component of the operational risk profile of a financial institution. To enable this, firms need to: (i) put in place robust and clear risk/event definitions and procedures; (ii) ensure losses captured in the database reflect the full occurrence of losses in the business (through mechanisms, such as a reconciliation against the general ledger, validation and review, etc); and (iii) verification of data comprehensiveness and quality. On the base of a comprehensive exploratory data analysis and benchmarking, Aon developed a set of tools to help our clients understand the completeness and quality of their internal loss data. We can also support to design and implement a robust process to collect and reconcile operational risk data. Scenario Analysis Scenarios are an important tool for the assessment of exposure to unexpected loss, utilising a combination of internal and external loss experience together with expert opinion and business engagement. However, their use to derive meaningful data for input into an AMA model is challenging. Key considerations are: (i) the level within the business for use (hub selection); (ii) risk coverage within each hub; (iii) metrics to capture; (iv) how to prepare for and engage with stakeholders; (v) collation of realistic and consistent parameters; and (vi) output. Aon has developed a structured approach to scenario analysis drawing on a combination of detailed loss data analysis, utilising internal and external events information. This approach involves:! Business segmentation and selection of risk classes, e.g. at Basel event type or by model risk category! Stakeholder identification for subsequent engagement! Pre-interview analysis and preparation, including statistical analysis of historical internal and external data, storyline development and selection of industry loss examples! Scenario interviews and workshop undertaken to gather a detailed understanding of how losses or risk within each event type being evaluated may materialise and to assess and agree exposure informed by robust data analytics! Post-workshop refinement and sign-off We have also developed a proprietary calculation engine to derive capital estimates related to scenario analysis. The model could be used both for Pillar 2 estimations and to integrate the loss distribution approach for AMA Pillar I models. External Loss Data Every external data source has a bias of some sort, whether obtained from historical insurance claims files, the public domain or a data consortium. As a consequence the intricacies of how to use these different data types need to be carefully considered when parameterising a model, in terms of impact on the tail and capital stability over time. Our approach to operational risk modelling clearly identifies the contribution of the systemic and idiosyncratic components, as required by the recent European Banking Authority and FED guidelines. Aon UK Limited is authorised and regulated by the Financial Conduct Authority. 3
Business Environment and Internal Control Factors There is a perception in some jurisdiction that BE&ICFs is not an important input into an AMA or Internal model, and in fact not sought by some regulators. This has largely been driven by negative feedback from regulators on BE&ICFs data, which are usually focused on qualitative assessment of risk and controls. Banks collect a high amount of information that could be actively used for operational risk modelling. As an example cyber risk could be assessed more efficiently by using data inventory, logs, and failed penetration data, instead of using traditional loss data. Few banks use BE&ICFs in their model, and where they do, they do not impact capital in a way expected by regulators. In most jurisdictions regulators are encouraging firms to develop structured BE&ICFs approaches that facilitate the realistic assessment of exposure to specific operational risk components. We have developed a proprietary methodology to incorporate the BE&ICFs in AMA modelling that uses such a component as a weighting mechanism between quantitative and qualitative estimates. Model design, build & calibration Complexity vs parsimony Models being built by AMA banks are often technically advanced, relying on new advances in the field of stochastic and operational risk modelling, but with limited experience of their application across the industry. This can lead to issues in comprehension of the theorems used in practice, calibration routes, goodness of fits tests and stability. Careful consideration needs to be taken as to how a model is going to be used. The key to building a robust and stable model is to adopt a pragmatic and proportionate approach. This will enable a firm to meet its objectives, invariably including capital stability over time, comply with regulatory requirements and understand their business and model risks. Model units of measure Models often have many units of measure resulting from a desire to develop a structure that reflects the risk matrix of standard Basel II taxonomy. However, model granularity on this basis typically suffers from a lack of data to inform parameterisation, leads to instability of results and capital numbers driven by aggregate losses rather than single large losses. It also results in a need to apply considerable diversification to generate capital values considered to be appropriate. The start of a model (re)build involves the testing of options for model granularity, taking into account data availability, and differences in the risk profile and reporting considerations. This often overlooked phase saves considerable time and problems at a later stage of development. It ensures the respective strengths and limitations of choices made are understood in terms of implications for capital and business output, around units of measure or granularity chosen. Integration Complex models could potentially work as black boxes, where the effect of the different model components is not properly and intuitively understandable. We have worked on a package of integration techniques to overcome this challenge, both for the integration of internal and external data and for the integration of quantitative (LDA) and qualitative (scenario) components. The use of BE&ICFs in the integration process could guarantee the fourth data element to be actively considered in the model itself. Aon UK Limited is authorised and regulated by the Financial Conduct Authority. 4
Reporting, use test, & risk mitigation Reporting The recent EBA guidelines focused the issue of modelling assumptions reporting, requiring adequate disclosure and sensitivity on model assumptions. Our approach is to clearly identify at least three sets of documents: (i) the methodological document, describing the assumptions adopted for modelling (ii) a First Time Adoption document, reporting the analysis underlying the methodological assumptions and model sensitivity, (iii) the capital calculation report. A thorough model validation, undertaken by appropriately skilled internal or external resources should address the documentation challenge. Review and challenge or validation work should pay particular attention to results and whether business judgement is consistently applied throughout the framework and in line with the documentation produced. Use test & risk mitigation Demonstrating that the model helps business decisions aimed at reducing and managing operational risks is often a challenge. When well designed, taking into consideration the reporting needs of the each stakeholder, model outputs can be used to: (i) prioritise and manage the reduction of expected losses through investment in the control environment; (ii) inform new and existing product risk evaluation and control, in conjunction with a scenario analysis framework; and (iii) compensation management strategies. Our team Our experts have a combination of industry and market knowledge, technical and consulting expertise and regulatory insight, supported by a unique team of actuaries to enable efficient and effective delivery. We have extensive experience in delivering complex projects, within tight timeframes and budgets, as documented by our engagement in more than 25 projects with AMA banks. We provide our clients with:! A tailored approach! Highly experienced consultants with strong market knowledge and best practice! Deep technical understanding of modelling methodologies and application combined with a focus on pragmatic, value focused solutions! Consistently high level of client satisfaction by leaving a positive legacy! Proprietary data, tools, platforms and models Aon UK Limited is authorised and regulated by the Financial Conduct Authority. 5
Client case studies 2 nd Generation AMA model design & build One of Europe s Systemically Important Financial Institutions with operations in 40 countries, which had achieved AMA accreditation in 2008, needed to develop a new (2 nd Generation) AMA model to meet changing regulatory requirements, whilst also providing a means to use the model to inform decision making in the business at a strategic and operational level. Aon was engaged to:! Consult with key stakeholders and define required model design and output requirements! Perform core model build for both loss distribution and scenario analysis approaches! Design and roll-out a new structured scenario analysis framework! Complete documentation! Model output validation and testing! Insurance integration to optimise capital financing Our client received two models for potential use and then once it had chosen the preferred option took ownership for final parameterisation and calibration. As an outcome, a model has been put in place, which incorporates all the elements of internal data, scenario analysis and external data in line with leading market practices (including Bayesian inference). The model was built to provide Risk Weighted Assets (RWA) stability over time; report metrics to inform business decision making; and to facilitate optimal capital financing including the use of insurance. The bank has also won a number of awards for its work. Impact assessment of recent large losses One of the first banks to achieve AMA accreditation suffered a number of large losses from the 2008 global financial crisis. Aon was engaged to perform an independent review of the bank s analysis to assess the implications resulting from operational risk boundary, timing and fraud events. The scope of the review comprised:! Review and challenge of loss classification and impact assessment, considering a range of potential outcomes financial loss and event classification. This required a deep knowledge of the regulatory framework and market best practice! Evaluate suitability and relative merit of the various statistical approaches considered, by the bank, for impact assessment and capital calculation. Our blend of financial engineering, computer science and actuarial expertise combined with extensive experience across over 25 AMA banks globally enabled us to provide a robust challenge of the bank s initial choices and advice on more applicable techniques! Report on results for submission to the bank s lead regulator, including a roadmap for refinement and enhancement to the AMA framework Our client was able to use our work and outputs to develop a clear plan for incorporating the large losses experienced into its AMA model, implement a roadmap for enhancement to the overall framework in line with its regulator s expectations and leading industry practices, and understand how other firms are also handling large losses. Aon UK Limited is authorised and regulated by the Financial Conduct Authority. 6 FP.AGRC.33.CL