Acceptance criteria for external rating tool providers in the Eurosystem Credit Assessment Framework

Acceptance criteria for external rating tool providers in the Eurosystem Credit Assessment Framework 1 Introduction The Eurosystem credit assessment framework (ECAF) defines the procedures, rules and techniques which ensure that the Eurosystem s requirement for high credit standards is met for all eligible assets. The details of the ECAF have been published in Title V of Guideline of the ECB of 19 December 2014 on the implementation of the Eurosystem monetary policy framework (recast) (ECB/2014/60) (hereafter referred to as the General Documentation ). The ECAF builds on credit assessment information from four sources, one of which is the external rating tool providers (RTs). RTs refer to a credit assessment source which is outside the Eurosystem, i.e. not operated by a central bank, and which consists of applications which rate obligors using, among other relevant information, audited annual accounts. The general acceptance criteria for the use of RTs within the ECAF are set out in Article 124 of the General Documentation. This document outlines these criteria. This document is structured as follows: Section 2 describes the acceptance procedure for RTs to be approved for ECAF purposes; Section 3 provides some details on acceptance criteria; Section 4 contains criteria defining the ongoing relationship with the Eurosystem after ECAF-approval; and Section 5 details the documentation requirements. An annex includes the application form. 2 Acceptance procedure An RT provider wishing to participate in ECAF has to submit a request to the national central bank(s) (NCBs) of the jurisdiction(s) in which the RT would like to be recognised, using the form provided in the annex to this document supplemented by the documentation specified in Section 5. A counterparty wishing to use an RT that is not yet accepted may ask the RT provider to apply for ECAF acceptance. After receiving a request, the Eurosystem will contact the RT provider to obtain any additional information that is required to conduct the assessment. The assessment may also include an on-site visit to the premises of the RT provider. The Eurosystem analyses the request within a reasonable time frame and reserves the right to decide whether it accepts an RT for the purposes of the ECAF based on an evaluation of compliance with the acceptance criteria deemed necessary by the Eurosystem, including the criteria set out in this document. The decision is communicated to the requesting RT provider as soon as possible. in the Eurosystem credit assessment framework (ECAF) 1

An RT is always accepted per jurisdiction. The ECB and NCBs will make available on their respective websites the relevant information concerning the RTs that have already been accepted in a certain country. The Eurosystem reserves the right to revoke the acceptance of an RT at any time if the RT provider fails to comply with the requirement to ensure the high credit quality standards of the Eurosystem, in particular by cooperating continuously in providing all the required information (e.g. for the ECAF performance monitoring process, as described in Article 126 and Annex IX of the General Documentation, as well as the provision of back-testing data). 3 Acceptance criteria for rating tools In its evaluation, the Eurosystem will assess: (i) the aims and functions of the system; (ii) the organisation, resources and governance of the system; (iii) the types of entity assessed; (iv) the output of the credit assessment system; (v) auditing and reporting; (vi) the definition of default; (vii) the data and IT-system architecture; (viii) the rating methodology, process and model development; and (ix) the one-off validation and ongoing performance monitoring. The Eurosystem requires continuous fulfilment of the following requirements: Definition of default RTs should follow the ECAF s definition of default, which follows the definition of default laid down in Directive 2013/36/EU and Regulation (EC) No 575/2013. This definition must be applied consistently in all credit assessment-related tasks (in particular, in modelling, use, and validation). Output of the credit assessment system the output of the credit assessment systems should be a one-year, issuer-specific PD calibrated using the ECAF definition of default and mapped to a rating scale that reflects exclusively the quantification of the risk of obligor default. The rating scale should have a minimum of seven grades or rating classes for non-defaulted obligors and one for defaulted obligors, rendering the default status of an obligor clearly identifiable from the rating grade. Coverage the RT should provide sufficient coverage so as to ensure the efficient implementation of the ECAF. For example, it should ideally cover a wide range of euro area countries. Nevertheless, for regional operators, ample coverage within at least one euro area country may suffice. Governance the RT provider must meet a minimum level of internal governance standards reflecting internationally accepted governance practices. The provider should be economically independent from the rated entities and should not be linked to RT clients. Data the sources of information should be chosen to ensure that the data is appropriate, obtainable, accurate, complete and of a high quality. With specific regard to financial statement data, balance sheet data are deemed valid only if in the Eurosystem credit assessment framework (ECAF) 2

they are delivered within 12 months of the closing date of the debtor s last financial year. The credit assessment should be conducted on the basis of the most recent financial and non-financial information available. Annual accounts should comply with local or international accounting standards and be audited. Rating methodology and validation the methodology shall assess the credit quality of debtors by using primarily quantitative models in a systematic and mechanical manner, which may be complimented by a human expert assessment, including an analysis of the group that the rated entity belongs to. 1 The methodology for assigning ratings must be systematic, continuous, rigorous and subject to validation (based primarily on historical experience via systematic back-testing). In particular, the models/methods used should undergo a rigorous one-off validation after their development, and the RT should have systematic approaches in place for the detection of any deficiencies in the estimation system that need to be corrected by the RT. Updates of credit assessments the RT provider updates the credit assessment of all debtors or guarantors on a regular basis at least once a year and on an ad hoc basis whenever material information on the debtor becomes available. Historical performance data information on the historical record of the RT s observed default rates per rating grades covering the five calendar years preceding the request for acceptance for use within ECAF shall be provided. The historical annual data on the observed default rates and potential additional information shall comply with the provisions for performance monitoring outlined in Article 126 of the General Documentation, as if the RT had been subject to these provisions over this time period. Additional operational criteria will be communicated to the RT provider upon application as deemed necessary. 4 Criteria defining the ongoing relationship with the Eurosystem Availability of ratings ratings should be available to all parties having a direct interest. Direct interest is understood as the interest of those counterparties that use the credit assessments for their banking business and of the Eurosystem (i.e. the NCBs that use the credit assessments in the context of accepting collateral). Methodological changes the RT provider should notify the ECB and the NCB(s) of the country or countries where the RT has been recognised at least one month in advance if it intends to implement significant changes to the methodology (in particular, the rating scale), legal or governance structure or coverage of the RT. 1 If the RT includes a qualitative analysis by an analyst, the analyst needs to check whether the rated entity is part of a group and take this information into consideration when finalising his/her rating decision. in the Eurosystem credit assessment framework (ECAF) 3

Performance monitoring an RT provider participating in the ECAF is subject to the Eurosystem performance monitoring process (see Article 126 and Annex IX of the General Documentation). The RT provider is obliged to set up and maintain the necessary infrastructure for monitoring the static pool. The construction and evaluation of the static pool have to be in line with the general requirements on performance monitoring under the ECAF. The RT provider shall undertake to inform the Eurosystem of the results of the performance evaluation on the date requested by the Eurosystem. Together with the performance monitoring data submitted, a signed certification from the CEO, or authorised signatory with responsibility for the audit or compliance function within the RT confirming the accuracy and validity of the performance monitoring data must also be submitted. The RT provider also has to undertake to keep internal records of static pools and default details for five years. In order to ensure transparency and to enable continued monitoring, as part of the annual performance monitoring process, RT providers should provide the ECB annually with: an updated version of all documentation maintained for ECAF purposes (in line with the requirements set out in this document); any updates to such documentation should be marked up using tracked changes; a concise summary of any updates to methodologies that took place in the preceding 12 months, including information on changes to default definitions, and any updates foreseen for the near future. In particular, if the minimum documentation consists of several documents, the summary should indicate clearly where it has been updated; recommendations of audits, action plans detailing how they are to be followed and a final report on any action taken. These are to be provided at the latest a year after the last audit, if an audit has taken place in the year covered by the annual performance monitoring process and, if reports are available, by the deadlines for the annual performance monitoring process. As part of the annual performance monitoring process, RT providers should test the discriminatory power and calibration of their model and submit the results to the ECB. In this regard, RT providers should conduct annually two tests for each criterion one mandatory and one of their own choosing, as follows. Discriminatory power the ability of a credit model to differentiate defaulting and non-defaulting entities is referred to as its discriminatory power. The Accuracy Ratio or Gini Coefficient should be estimated annually. Additional tests, from among which one may be chosen, are the CAP Curve (from which the Gini Coefficient derives), the ROC Curve, AUC, hit rates and false alarm rates. Calibration calibration refers to the assignment of default probabilities to a credit model s output. To test the calibration quality, RT providers should conduct annually the Hosmer-Lemeshow test. Additional tests, from among which one may be chosen, are the binomial test, the Brier Score or multivariate Sterne test. in the Eurosystem credit assessment framework (ECAF) 4

The Eurosystem reserves the right to request any additional information required to conduct the performance monitoring and acceptance process and to visit the RT provider and/or to hold teleconferences with a view to assessing the compliance of the RT with the acceptance criteria over time. 5 Documentation requirements RT providers should maintain a standard (minimum) level of documentation in English as a precondition for initial and ongoing acceptance within the ECAF. The documentation should be made available to the Eurosystem. The documentation maintained by RT providers for ECAF-purposes should, as a minimum, cover each of the aspects outlined in Sections 5.1 to 5.5 below. 5.1 General overview of the system 5.1.1 Aims and functions of the system This should include a description of the main uses of RT ratings and a high-level overview of the system. 5.1.2 Organisational set-up, resources and governance of the system The RT provider should describe the organisational and governance structure of the system. In particular, it should describe the organisation, the ownership and group structure, affiliation, legal structure and specificities under which the RT provider operates (including major shareholders and subsidiaries). In addition, it should list the business areas responsible for: (a) rating entities; (b) developing methodologies; (c) validation and performance monitoring; (d) data gathering; (e) maintaining the IT infrastructure. It should also provide the number of full-time equivalents (FTEs) dedicated to performing tasks (a) to (e). Details of the RT provider s degree of independence in terms of financial means and decision-making power should be provided. Moreover, details on market share should be made available. 5.1.3 Types of entities assessed A description of the types of entity that can be assessed by the system should be provided. The criteria used to determine whether an entity is assessed should be described, for example criteria based on sector, the size of the entity in terms of market capitalisation, turnover, etc. This description should also specify whether the RT provider initiates the assessment or whether the request for assessment comes from elsewhere. in the Eurosystem credit assessment framework (ECAF) 5

5.1.4 Output of the credit assessment system The output of the RT should be described in detail. In particular, the exact rating approach, e.g. point-in-time versus through-the-cycle PD, issuer-pd versus issue- PD, individual versus class-wise PD, etc, should be clarified. Furthermore, if a rating scale is used, detailed information on the scale s structure, its granularity and on the individual PDs/PD-intervals associated with each class should be provided. 5.1.5 Reporting and auditing The date, scope, outcome and recommendations of the last internal or external audit should be provided, as well as the frequency of audits either by an external third party or by the internal compliance function. In addition, any further reports should be listed, with their timetable and a short abstract describing each report, and made accessible to the ECB on demand. 5.2 Definition of default The RT provider should provide a description of the precise definition of default used. Detailed information on the data sources used to assess whether the assessed obligor falls into default following the Eurosystem definition of default, including reporting size thresholds, reporting frequency and exact coverage, and the IT systems and process to ensure compliance with the default definition should be provided. 5.3 Data and IT-system architecture 5.3.1 Input data and collection process (including quality controls) Information on all the different input data types employed by the RT should be provided, e.g. base data, annual accounts, quarterly financial statements, forecasts, default data, rating data, etc. The explanations should include a description of the sources, e.g. public sources such as national statistical offices versus private or confidential sources, such as credit register, agencies providing specialised information, private vendors, etc. The collection method should be explained for each data type, i.e. manual or automated. Furthermore, details of the timetable and frequency of data collection should be provided. Finally, an overview of the quality control processes in place should be included. 5.3.2 Information flow and IT-system architecture A detailed overview of the IT infrastructure, for example databases and software applications, should be provided including details of the function and use of each component, as well as the information stored. It should be made clear whether an individual component is used solely for RT purposes, and who is ultimately in the Eurosystem credit assessment framework (ECAF) 6

responsible for system maintenance and quality control. In addition, the business continuity plan in case there is a major failure of the IT infrastructure should be presented. 5.3.3 Outsourcing arrangements Details of any services or functions or services outsourced to third parties and the names of the companies providing these services, as well as a summary of the general guidelines regarding the identification, management and monitoring of the risks posed by the outsourcing of important functions should be provided. 5.4 Rating methodology and process 5.4.1 Description of the rating process A description of the rating process should be provided, including a high-level description of the method, e.g. heuristic, expert-based, statistical, combination. The risk assessment decision should be detailed by explaining the different phases and stages in the assessment process. A description of the parties involved in the rating process and the steps involved in the approval process is to be provided. Details should also be provided regarding the main areas analysed, e.g. enterprise and business characteristics, financial analysis such as ratio analysis, management evaluation, financial flexibility, the industrial and economic environment, etc. If different approaches are taken for different entities, this should be highlighted and the reasoning for using the different approaches should be provided. Finally, information should be provided on how often entities are re-assessed along with a timetable for reassessment, including details regarding any procedures in place to determine whether an extraordinary or unscheduled assessment needs to be carried out. 5.4.2 Methods/models used A detailed description of the methods and models used for each segment of the rating portfolio should be provided, including a detailed description of the predictors, e.g. financial ratios, base data, alternative ratings, together with their weights. If the method differs by portfolio segment, the differences should be clarified. The rationale supporting the choice of rating method and criteria should also be explained. If qualitative and quantitative information is employed in the assessment, a description of how these data are combined in order to obtain a final rating is also required. This description should include a summary of the general guidelines in place for the handling of discretionary changes to ratings stemming from statistical rating systems if applicable and a note on how groups are handled. in the Eurosystem credit assessment framework (ECAF) 7

5.4.3 Procedures to ensure consistency, accuracy and comparability of credit assessments A description of procedures in place to ensure consistency, accuracy and comparability of credit assessments should be provided. The description may include: (a) the four-eye principle to ensure accuracy and comparability of data collection or ratings; (b) guidelines followed by the rating committee and analysts in different regional offices (if applicable); (c) documentation preservation procedures for audit purposes. 5.4.4 Overview of the method development process This should provide a description of the process used to develop the method, for example, if statistical models are employed, how different models are evaluated and relevant variables identified, etc. 5.5 One-off validation and ongoing performance monitoring 5.5.1 One-off validation of the model Details of the validation procedures carried out when a model is first introduced should be provided. 5.5.2 Ongoing performance monitoring Details of the validation procedures carried out to ensure that the models remain adequate for purpose should be provided. A description of any other performance monitoring process besides the standard process for ECAF performance monitoring should be provided if such processes exist. in the Eurosystem credit assessment framework (ECAF) 8

Annex Eurosystem application form information requirements relating to counterparties and RT providers Request for rating tool acceptance 1. RT provider s name, address, contact details: [Please indicate the name of the RT provider together with all contact details] 2. RT: [Name of the RT (product name)] 3. To: [Name of home central bank] 4. Date of registration with ESMA as CRA 2 : [Indicate the date of registration with ESMA as CRA, if RT is registered as CRA] 5. Number of customers: [Indicate the number of customers structured according to regional relevance to the RT provider s business] 6. Yearly turnover: [Indicate the figure structured according to regional relevance to the RT provider s business] 7. Country (or countries) to be covered by RT: [List of country names] 8. Exposure category to be covered: [Details on the exposure category to be covered by the RT] 9. Model coverage: 9a Geographical: [List the countries covered by the RT] 9b Class of debtors: [Specify the coverage in terms of sectors of economic activity covered by the RT] 9c Minimum/maximum turnover of entities rated: [Specify the requested range of turnover] 10. Definition of default: [Specify the default definition underlying the RT] 11. Probability of default: [Specify the probability of default definition underlying the RT] 12. Additional documentation: 3 [Documentation in line with requirements outlined in this document] 13. Five-years of disaggregate performance information: [Attach disaggregate performance information (covering at least five years] 2 3 CRA registered or certified in accordance with Regulation (EC) No 1060/2009 of the European Parliament and of the Council of 16 September 2009 on credit rating agencies. Additional documentation should cover, as a minimum, detailed information on each of the aspects described under the heading Documentation requirement in Section 5, i.e. on general overview of the system, definition of default, data and IT system architecture, rating methodology and process, as well as one-off validation and ongoing performance monitoring. in the Eurosystem credit assessment framework (ECAF) 9