EBF ref. 006433/006409 Brussels, 30 January 2014 Launched in 1960, the European Banking Federation is the voice of the European banking sector from the European Union and European Free Trade Association countries. The EBF represents the interests of some 4,500 banks, large and small, wholesale and retail, local and cross-border financial institutions. Together, these banks account for over 80% of the total assets and deposits and some 80% of all bank loans in the EU alone. EBF comments 1 on the supervisory benchmarking concept established in article 78 of the Capital Requirements Directive (CRD IV) BACKGROUND The Capital Requirements Directive (CRD IV), in its article 78, requires competent authorities to conduct, at least annually, a supervisory benchmarking assessment of internal approaches for the calculation of own funds requirements except for operational risk. Such an assessment shall show significant differences in own funds requirements for the same exposure as well as internal approaches showing extreme diversity or signs of systematic underestimation of own funds requirements. EBA is mandated to develop guidelines and regulatory technical standards (RTS) for the assessment and procedures involved including the template, the definitions and the benchmark portfolios. For this purpose, EBA organised a workshop with representatives of competent authorities and banks on 19 December 2013. The EBF has actively participated in the review of the characteristics of internal approaches from 2010. It has contributed to the review with the publication of a thorough study of the characteristics of internal rating based models for residential mortgages 2 conducted by the EBF with the involvement of 43 banks from 14 EU countries. Several meetings have been held with the EBA including at the abovementioned workshop as well as with investors and other stakeholders. There is broad common understanding that differences in the outcome of internal approaches stem from three main sources: Firstly, market characteristics and legal framework; secondly, supervisory practices; and thirdly, banks choices within the options available in the international standards established by the Basel Committee and transposed to the EU in the former CRD. 1 Prepared by the EBF Risk Assessment Working Group and approved by the EBF Banking Supervision Committee. Contact person at the EBF Secretariat: Gonzalo Gasos (g.gasos@ebf-fbe.eu). 2 EBF Study on IRB models for residential mortgages in Europe (July 2012). EBF a.i.s.b.l ETI Registration number: 4722660838-23 56, Avenue des Arts B-1000 Brussels +32 (0)2 508 37 11 Phone +32 (0)2 511 23 28 Fax www.ebf-fbe.eu
Against this background, the EBF would like to convey the following messages to the EBA for the preparation of the RTS. GENERAL REMARKS The concept of benchmarking or comparing the output of one model against another could be meaningless unless the objectives of the two models are similar. We know (e.g. from the Third interim report on the consistency of risk-weighted assets, EBA, 17 December 2013) that any two models can differ significantly in respect of their (i) model philosophy (through-the-cycle, hybrid, point-in-time); and (ii) the length of the data series used to calibrate the models. Constructing benchmarks using aggregates of models which differ on these two points, or using a fixed benchmark, cannot account for these differences. At issue here is the lack of regulatory consistency across jurisdictions, and within jurisdictions, on what a PD model output must represent. For example, rules could be set to determine model philosophy; rules could be set to ensure that model calibrations represent a full economic cycle view of the risk. In order to successfully and meaningfully benchmark models in a perfect world, this lack of consistency must be remediated, time given for banks to adapt their models and then sensible and meaningful benchmarks could be constructed. This would be unlikely to be compatible with the timeframes envisaged to complete the benchmarking exercise and report ultimately to the European Commission. Given that models must and should reflect banks own experiences, their processes, risk policies and risk management performance, benchmarks should provide a range of acceptable outputs. This should preserve the objective of risk sensitivity of the previous Basel accord, and also remind supervisors and other stakeholders, that benchmarking is no substitute for the internal back testing and validation of models, which still must be regarded as the key factors in assessing model performance. An additional concern is that banks construct models that operate statistically. Acceptable models, therefore, are models that operate effectively on the bulk of that individual bank s borrowers. Errors will be tolerated for outlier borrowers. Benchmarking approaches that do not take account of this run the risk of rejecting model outputs based upon the inadvertent selection of an outlier population for the comparison. It is possible and perfectly acceptable for two banks to rate the same borrower completely differently using two powerful and correctly calibrated statistical models. The clustering approach may serve to mitigate this concern to some degree. The EBF s members have a common interest with the EBA in seeking to provide confidence in banks risk-weighting approaches. Notwithstanding the besetting difficulties described, the EBF welcomes the intent of the benchmarking concept and intends to examine the benchmarking problem with a view to present a workable solution to the EBA in the not too distant future. 2
KEY POINTS Type of benchmark We note that the EBA has identified two possible types of benchmarks, namely a relative benchmark or an absolute benchmark. Regarding the option of absolute benchmarks, we note that the EBA points to 2 possible benchmarks for capital requirements, i.e. the standardised approach and the mapping to rating agency default rates (for the PD) and Foundation regulatory values for the LGD and the EAD. The EBA should refrain from using a blunt standardised measure as a benchmark as it could unwarrantedly replace the true observation of risk. In the view of the EBF, it would be preferable to use relative benchmarks subject to certain conditions for the following reasons: It would be a more natural way of benchmarking, as recognized in the EBA paper, however some caveats have to be considered (see the section on clusters below). It allows certain degree of flexibility to explain the reasons of the observed variability for the sake of understanding. More importantly, a relative benchmark can give recognition to other qualitative aspects associated with risk models such as risk policies and risk management practices. These are the conditions that the use of relative benchmarks should meet: It should be broken down in several dimensions. No straight comparison should be made but a fair appraisal should seek a minimum degree of homogeneity in the populations subject to evaluation. Parameters should be consistent across banks. We understand how difficult it is to compare the results of models that contain significant methodological differences. For this reason the EBA should communicate that the results of relative benchmarking carry the bias associated with the different options available to banks modelling. So long as convergence is apparent the relative benchmark will become more meaningful. The exercise should always be based on actual parameters and exposures. Qualitative explanations of the relevant factors that may have an impact on the variability of risk weights should be studied and published together with the results. Clusters It is worth mentioning that the comparison made in a benchmarking exercised is focused on the model. Therefore the definition of the model matters. One of the key characteristics of a model is about the number and nature of the clusters in which borrowers are classified. The estimated parameters and the observed default frequency (ODF) refers to every cluster. This makes it 3
difficult to draw fair conclusions from a comparison between average regulatory PDs and default rates at a cluster level. We find two main sources of problem. From a practical viewpoint, banks typically calculate the ODF at model level, with the clients or contracts that meet the features to belong to that model as defined in every bank. The problem is that the EBA would use its own definition of clusters that will not match those of banks. In fact, banks may not have historical records available with the features that define a specific cluster designed by EBA. EBA will have to take into account this limitation in the calculation and in the communication of results. From a conceptual standpoint, if banks had the possibility to calculate the ODF at every defined cluster level with and EAD-averaged regulatory PD, the conclusions drawn from a simple comparison could still be questionable. For instance, the exposures of an EBA cluster could belong to two different internal models of a bank, as shown in this picture: MODEL 1 MODEL 2 CLUSTER 1 It follows that back-testing needs to be interpreted. The potential determination of homogeneous clusters at European level lies behind. In the meantime, the EBA should take this limitation into account for instance by accepting in each cluster only PD reported values with low dispersion. These facts should be made known when laying out the conclusions of the assessment. Floors The question was raised whether relative benchmarks should apply before or after the impact of floors and other model limits. The EBA rightfully indicates in its preliminary examination that the purer capital requirement pre-adjustment would be more complicated to implement due to the fact that the supervisory floor requirements are in many cases embedded into the banks rating systems and calculation processes. The EBF agrees with the EBA analysis but would like to underscore how important it is to allow for a qualitative interpretation of the post-adjustment results. The most significant differences in the national application of floors should be roughly estimated and explained to duly complement the findings of the hard comparison. 4
Hypothetical transactions The EBF has long warned about the fundamental confusion behind the use of hypothetical transactions. There seems to be a widespread belief that the same transaction should be assigned exactly the same risk weight in every bank. Evidence suggests that the general public oftentimes jump to the conclusion that there is something wrong in the models. For this reason, we urge regulators to be extremely cautious with the publication of such outcomes. In particular, the EBF does not favour the use of hypothetical transactions for several reasons: They are less meaningful than actual transactions of real borrowers. Models work on a statistical basis. Hence the accuracy of a model should only be measured against the whole target population but no conclusion should be drawn from a single item. Hypothetical transactions do not take into account relevant factors involved in the risk experience of the institution such as its processes, risk policies and risk management performance. The recurrent use of hypothetical transactions would entail a certain bias as models will look better the closer they are to the proposed transactions. Models could end up accommodating to a set of hypothetical transactions instead of focusing on its performance on the actual transactions. Timescales The benchmarking exercises will be put in place when the definition of the underlying regulatory criteria is still underway. This is particularly the case as regards divergent modelling options and supervisory practices in the field of risk models. Concerning the implementation of benchmarking of market risk models in accordance with article 78, we urge the EBA to take into consideration the fundamental review of the trading book that will be taking place simultaneously. The fundamental review of the trading book will entail considerable work for banks on impact studies and model development and will most likely result in changes of the current internal market risk models or implementation of new ones. For this reason we question the merit in introducing a very detailed benchmarking exercise of existing market risk models at this stage. Data collection In order to ensure a consistent data collection over time for the benchmarking exercise we recommend the EBA to apply a systematic approach to data collection and the use of information already provided in the existing reporting frameworks (COREP and FINREP) to the furthest possible extent. Ad hoc data collection draws heavily on banks resources and enhances the risk of inconsistency in data from different banks and over time. 5
Communication The EBA already showed a cautious approach when editing former papers on risk weighted methodologies and risk analyses and we would expect that it follows the same line in this occasion as the correct interpretation of risk weight benchmarking involves considerable degree of understanding of the fundamentals of risk measurement. In this vein, the EBA should be very careful in the communication of results to ensure a correct interpretation. A private communication to individual banks should be sought. In the case that a future public disclosure is made, the relevant circumstances affecting the outcome of models should be reflected in the report so that banks could give an educated explanation to analysts and stakeholders. These circumstances should include market characteristics, legal framework aspects and significant modelling options. It will also be important to lay out the results in terms of acceptable ranges of variation. Straight comparisons could mislead the general public. Instead, we should speak in terms of Acceptable ranges of variation given the circumstances of the institution. 6