Academy of Economic Studies Doctoral School of Finance and Banking - Dissertation paper - Early Warning Models for Banking Supervision in Romania

Transcription

1 Academy of Economc Studes Doctoral School of Fnance and Bankng - Dssertaton paper - Early Warnng Models for Bankng Supervson n Romana MSc Student: Radu Muntean Supervsor: Ph.D. Professor Mosă Altăr Bucharest, July 29

2 Table of contents Abstract...2. Introducton Practce and lterature revew Methodology Bnary dependent varable model Ordered logstc model Varable Selecton Valdaton Predctons Data Results Ratng downgrade Weak overall poston Conclusons...38 Bblography...4 Annex Varables and Sgns...43 Annex 2 KS Test...46 Annex 3 Monotony...49 Annex 4 Unvarate Framework...5 Annex 5 Multcolnearty...5

3 Abstract In ths paper we propose an early warnng system for the Romanan bankng sector, as an addton to the standardzed CAAMPL ratng system used by the Natonal Bank of Romana for assessng the local credt nsttutons. We am to fnd the determnants for downgrades as well as for a bank to have a weak overall poston, to estmate the respectve probabltes and to be able to perform ratng predctons. Havng ths purpose, we buld two models wth bnary dependent varables and one ordered logstc model that accounts for all possble future ratngs. One result s that ndcators for current poston, market share, proftablty and assets qualty determne ratng downgrades, whereas captal adequacy, lqudty and macroeconomc envronment are not represented n the model. Banks that wll have a weak overall poston n one year can be predcted usng also ndcators for current poston, market share, proftablty and assets qualty, as well as, n ths case, captal adequacy and macroeconomc envronment, the latter only for the bnary dependent varable model, leavng lqudty ndcators out agan. Based on the ordered logstc model s capacty for ratng predcton, we estmated one year horzon scores and ratngs for each bank and we aggregated these results for predctng a measure of assessng the local bankng sector as a whole. 2

4 . Introducton Bankng s one of the most ntensvely supervsed ndustres world-wde due to the hgh mpact of bank falures on economc actvty. Fnancal stablty, a wde varety of markets, nfrastructure and even people s personal comfort and safety depend on the credt mechansm and the soundness of the bankng sector. Therefore, all over the world, governments grant authorty to fnancal supervsory bodes and put them n charge wth the regulaton, authorzaton and supervson of the fnancal nsttutons, n order to lmt the rsks they undertake and the negatve effect they mght have on other economc sectors. Bank supervsors develop ther knowledge about the banks runnng n ther jursdctons by the means of on-ste examnatons and off-ste survellance. Although useful n order to provde current and detaled data, the on-ste examnatons can be costly for the supervsory authorty by requrng the on-ste teams to be sent at the premses of the examned bank n order to have meetngs, access fles, check data qualty, analyze systems ntegrty and obtan results that can be used n assessng the bank s current stuaton. Moreover, on-ste examnatons can be burdensome to bankers because of the human and logstcal resources that they need to wthdraw from ther current actvtes and make avalable to the on-ste team demands. Off-ste survellance ams at makng the supervsor aware also of the bank s stuaton between on-ste examnatons. Fnancal data and other changes that occur at the bank s level are reported to the supervsory authorty where are recorded and analyzed. Ths way, the assessment of each bank s contnuously updated and the supervsors may decde whether and when another on-ste examnaton s needed. Ths dual approach mght save resources for both, supervsor and supervsed bank, and mght provde a clear pcture of the rsks undertaken by each bank as well as nputs to assessng stablty of the bankng sector. Some of the tools hghly used n the off-ste survellance refer to the gatherng of relevant nformaton wthn screens such as rsk matrces and other specfc tables that assess each bank after a wde range of crtera. Hstorcal experence and expert opnon are some of the methods for selectng relevant crtera and ther benchmarks. Each bank s granted wth an assessment as low-to-hgh, or a numercal ratng for each of the crtera employed. Then, all crtera-based ratngs lead to the overall ratng of the bank. 3

5 Ratng systems used by supervsory authortes provde valuable nformaton about the credt nsttutons analyzed wthn the same framework. Separatng problematc of well performng banks allow the supervsor to save resources by havng the possblty to focus more on the banks that are currently n dstress. However, ratngs carry nformaton about past stuatons and are more of an ex-post measure of the banks status. Therefore, supervsors always need to consder expert opnon and recent developments n order to have a better assessment of the credt nsttuton. Addtonally, supervsors have come up wth a class of tools that are able to predct negatve future events, thus ganng more tme to act. Early warnng (EW) models have been used to predct negatve events lke bank falure, ratng downgrade and nadequate captalzaton. For the purpose of ths paper we am to fnd the determnants for ratng downgrades and the ones for a bank s future overall poston, n order to be able to estmate probabltes for downgrades, bad ratngs and also for each possble ratng; these results wll then allow for ratng predcton. Ths paper s structured as follows: the next chapter s a bref ntroducton to current practces and some of the lterature relevant for the presented subject and chapter 3 presents the methodology hghlghtng the used models, the varable selecton process and model valdaton. Next chapter refers to data, as analyzed varables, perods and dscretons and s followed by Chapter 5 whch presents the results focused on both ratng downgrade and weak overall poston as man dependent varables. In ths chapter we show some of the ntermedary results, the fnal models, valdaton, predcton and other results. Chapter 6 hghlghts the most mportant results and conclusons. 4

6 2. Practce and lterature revew Supervsory authortes around the world have developed ther own ratng systems amng for a standardzed approach to the dfferent banks runnng busness n ther jursdctons as presented by BIS (2). The CAMELS ratng system was mplemented n 98 n the Unted States of Amerca by all three supervsory authortes: Federal Reserve System (Fed), Offce of the Comptroller of the Currency (OCC) and Federal Depost Insurance Corporaton (FDIC). The ratng system has sx components, referrng to captal protecton (C), asset qualty (A), management competence (M), earnngs strength (E), lqudty rsk (L) and market rsk senstvty (S); to each of these components a grade from (best) trough 5 (worst) s assgned. CAMELS was followed by other ratng systems lke ORAP (Organzaton and Renforcement of Preventve Acton) mplemented by French Bankng Commsson n 997, RATE (Rsk Assessment, Tools of Supervson and Evaluaton) mplemented by UK Fnancal Servces Authorty n 998, RAST (Rsk Analyss Support Tool) mplemented by Netherlands Bank n 999, etc. In the Unted States, the FDIC mplemented the SCOR (Statstcal CAMELS Off-ste Ratng) model n 995. SCOR s quarterly run based on data reported by credt nsttutons and uses an ordered logt model of CAMELS ratngs to estmate lkely downgrades of banks wth a current composte CAMELS ratng of and 2. Ths s explaned by the hgher attenton already gven by the supervsor to banks wth on-ste examnaton ratng of 3, 4 or 5. The model flags for revew banks that are currently strong or satsfactory but have a probable downgrade. The current ratng s compared to the one-year pror fnancal data and the coeffcents found are employed to estmate future ratngs. The assumpton s that the relaton between current ratng and pror data wll hold for future ratng and present data. SCOR uses a step-wse estmaton n order to elmnate not statstcally sgnfcant varables. Many of the varables that are nput to ths model are also nput to the SEER (System for Estmatng Examnaton Ratng) model of the Federal Reserve, although the pror CAMELS ratng s ncluded only n the SEER ratng model. The tme horzon for ratng estmaton under SCOR s between four and sx months. Accuracy of the output has been shown to decrease beyond the sx month perod. The output of ths model s a table gvng the probabltes that the next ratng wll be each of the fve possble ratngs. A downgrade appears when a bank wth a ratng of 5

7 or 2 goes to a ratng of 3,4 or 5. Also, the model provdes a SCOR ratng as the sum of the possble ratngs weghted wth ther probablty. Areas of concern are hghlghted by comparng the bank wth a Medan 2 Bank, whch s a typcal bank wth a ratng of 2. Ratng downgrade models share strong smlartes wth bankruptcy predcton models. Beaver (966) performed an early unvarate dscrmnant analyss usng 3 fnancal ratos for 58 frms, whch found that cash-flows/equty and debt/equty can be useful n default predcton. Altman (968) developed a scorng functon, usng multvarate dscrmnant analyss (MDA), n order to dscrmnate between the two possble events. The varables used together wthn the functon were also specfc for the purpose of bankruptcy predcton. The logstc regresson was frst used wthn a bankruptcy predcton framework by Ohlson (98). The varables are used n a multvarate framework as t s the case for MDA but the scorng functon s lnear wth regard to the log odd of default. Logt models are preferable to MDA as the latter assumes that the covarance matrces are the same for bankrupt and non bankrupt frms, t also assumes normally dstrbuted varables and, most mportant, are not able to provde a framework for performng sgnfcance tests for the model parameters. Over the last decades, the ncreasng nterest of both supervsors and academcs n ratng models and early warnng systems has led to an economc lterature able to provde new methods and to rase new ssues wth respect to models used n bank supervson. As credt nsttutons are not usually defaultng often enough n order to provde for a sgnfcant data base and therefore a sgnfcant statstcal model, many papers refer to nadequate captalzaton, ratng downgrades or other lesser negatve events that are also of hgh nterest for assessng the stablty of a bank. In ths respect, Jagtan et al (2) tested the effcacy of EW models as tools for the predcton of captal nadequate banks usng a sample of U.S. banks wth captal between $3 mllon and $ bllon. Logt and trat recognton analyss (TRA) models were generated and compared trough a testng perod. Fndngs showed the mportance of TRA n hghlghtng complex nteracton varables useful n predctng banks wth defcent captal. Both the logt and the TRA models had a reasonable 6

8 degree of accuracy and they were consdered a powerful tool for detectng one year n advance nadequately captalzed banks. Kolar et al (2) used logt and TRA models to predct large U.S. bank falures. The models were developed from an orgnal sample and tested for predctve ablty n the holdout sample. Both models performed well, but TRA outperformed logt models n overall accuracy, large bank falure accuracy, weghted effcency scores. The paper concluded that TRA models can dentfy varables nteractons relevant for predcton and therefore can provde valuable nformaton about the future large bank falures. Regardng ratng downgrade, Glbert et al (2) compared such a model wth a currently employed bankng falure predcton model (SEER) n use at the Federal Reserve. Because of the small number of bank falures, the SEER coeffcents are mostly frozen and over tme, the ablty of the downgrade model n predctng downgrades mproves relatve to that of the SEER model n predctng falures. Ths paper concludes that a downgrade model may be useful n bankng supervson and shows the hgher accuracy of a frequently re-estmated model. Other studes lke Glbert et al (22) argue that ratng downgrade predcton models may not clearly outperform falure predcton models, especally n tranqul perods. However, t should be noted that there s a consensus over the fact that a ratng downgrade predcton model s an mportant nformatonal supplement to supervsors and even though t should not rule out expert opnon and other supervsory tools, t should be used for hghlghtng possble problematc banks. The models employed for ratng systems can be valdated through varety of technques. Engelmann et al (23) analyzed useful tools for dscrmnatory power such as the Cumulatve Accuracy Profle (CAP) and the Recever Operator Characterstc (ROC). The summary statstcs of CAP and ROC were proved to be equvalent and the comparablty of dfferent models accordng to both statstcs s stated only for the same nput data. For ths reason, one could use Area Under ROC (AUROC) alone n order to capture the dscrmnatory power of a model. Wth respect to statstcal ssues concernng early warnng models we refer to Hosmer and Lemeshow (2) who have thoroughly presented practcal steps, problems and dscretons avalable when workng wth a logstc regresson. Studes performed on U.S. banks or cross-european banks samples have met wth the choce between dfferent types of early warnng system models. That s because on 7

9 such samples one could dentfy bank falures or nadequate captalzed banks and therefore develop a model for predctng these events. Bankng sectors n most emergng countres have fewer banks and the data hstory s shorter. Supervsors n these jursdctons also employ tools based on current assessment of banks but due to ths ssue they are usually not able to predct bank falures or nadequate captalzaton as early warnng models, as these knd of negatve events have not happened enough to provde for a sgnfcant database. However, mplementaton of ratng downgrade predcton models s possble. In Romana, n accordance wth the Government Emergency Ordnance no. 99/26, the bankng supervsory authorty s granted to the Natonal Bank of Romana (NBR). Wthn the NBR there are several Departments drectly connected to the bankng sector, wth respect to regulaton, authorzaton, fnancal stablty and prudental supervson. Changes n management, shareholders, fnancal stuaton of banks, as well as current and past fnancal data and other relevant nformaton are all actvely analyzed by the NBR, manly wthn the Supervson Department (SD). Commercal banks are assessed regardng the rsks they undertake both by on-ste examnatons and by off-ste survellance. The CAAMPL unform ratng system refers to sx components that are checked by the supervsor and rated n order to obtan a fnal score and then an overall ratng of the bank: - captal adequacy (C); - shareholders qualty (A); - assets qualty (A); - management (M); - proftablty (P) and - lqudty (L). Banks are rated from (best) to 5 (worst) for each ndcator ncluded n each of the sx components and then the supervsor calculates aggregated ratngs for the components and an overall ratng for the bank. 8

10 9 3. Methodology 3.. Bnary dependent varable model In order to buld an early warnng model for the predcton of CAAMPL ratng downgrades we have employed a logt methodology. Then, the same methodology has been appled for predcton of banks recevng a bad ratng n one year horzon. Frstly, we assume an unobservable dependent varable y* related to a bnary observed varable y, whch represents a CAAMPL ratng downgrade (y=) versus a constant or upgraded CAAMPL ratng (y=). () = > = otherwse y y f y, :, * The latent varable y* s explaned by the vector of bank s fnancal ratos and other ndvdual fgures as well as macroeconomc envronment x and the vector of estmated coeffcents β. (2) n n x x x y or x y ε β β β β ε β = + =... :, 2 2 * ' * The term ε s logstc dstrbuted, thus havng the logstc cumulatve dstrbuton functon: (3) x e x F + = ) ( The probablty that a bank wll have a downgraded ratng can be expressed as follows: (4) β β β β ε β ' ) ( ), ( ) ( ) ( ), ( ' * x e x F x y P x P y P x y P + = = < > = = The model s coeffcents are contaned n the β vector and they need to be estmated. The maxmum lkelhood method (MLE) assumes that each observaton s extracted from Bernoull s dstrbuton. Therefore, a ratng downgrade event has the attached probablty F(x β) makng the probablty of a non-downgraded ratng event - F(x β). The probablty mass functon s the product of the ndvdual probabltes: (5) = = = = = = ' ' 2 2 )) ( ( ) ( ),...,, ( y y n n x F x F y Y y Y y Y P β β

11 The lkelhood functon should be maxmzed wth respect to the vector of coeffcents. (6) N = DN ' ' [ F( x β )] [( F( x β ] NDN L ( β ) = )) In order to obtan a more convenent expresson to maxmze we employ the logarthm: N ' ' (7) ln L( β ) = { y ln( F( xβ )) + ( y ) ln( F( xβ ))} = The coeffcents have been estmated usng the quadratc hll clmbng algorthm, whch, n order to acheve convergence, employs the matrx of secondary dfferentals of the log lkelhood functon. The estmated coeffcents should be analyzed carefully notng that ther sze does not necessarly carry sgnfcant economc nformaton. However, the sgn of each coeffcent s mportant as t shows how the dependent varable s nfluenced by a varaton n each varable. For nstance, postve coeffcents show that ther respectve varables varatons nfluence the downgrade probablty n the same drecton as that of the varatons whch took place. The margnal effect of the explanatory varables x j on the dependent varable s gven by β j weghted wth a factor f dependng on all the values n x. E( y x, β ) ' (8) = f ( x β ) β j, where x correspondng to F. j df( x) f ( x) = s the densty functon dx 3.2. Ordered logstc model Secondly, we consdered an ordered logstc model. In ths approach, the dependent varable s assumed to represent ordered or ranked categores. The one year future CAAMPL ratng s mapped nto the dfferent values of y. The dependent varable n an ordered logstc model s consderng a latent varable, lke n the case of the bnary dependent varable model prevously presented. (9) y * = x ' β + ε The observed response y s obtaned from y *, based on the followng rule:

12 () y *, f : y γ * 2, f : γ < y γ 2 = * M, f : γ M < y The probabltes for the dependent varable to take each of the values allowed for are gven as follows: () P( y P( y P( y ' = x, β, γ ) = F( γ x β ) ' ' = 2 x, β, γ ) = F( γ 2 x β ) F( γ x β ), where F s the cumulatve ' = M x, β, γ ) = F( γ x β ) M dstrbuton functon of ε. For the purpose of ths applcaton, F was selected as beng the logstc cumulatve dstrbuton functon. The threshold values γ are mportant by determnng the value of the dependent varable, based on the score x β. In order to estmate the threshold vector γ, as well as the β coeffcents, the log lkelhood functon has to be maxmzed. N M (2) ln L( β ) = ln( P( y = j x, β, γ )) ( y = j) = j=, where (x) s an ndcator functon whch takes the value for a true argument and for a false argument Varable Selecton Whle buldng a logt model, a key ssue s the selecton of explanatory varables. In ths regard, we consdered to steps structured by Hosmer and Lemeshow (2) for the process of varable selecton as well as other useful flters amed to dscrmnate between relevant and rrelevant explanatory varables. For the frst flter we consdered the attrbute of the explanatory varables to dscrmnate between downgrades and non-downgrades. In ths respect, we employed a two-sample one-sded Kolmogorov-Smrnov test to determne whether the two groups are drawn from the same underlyng populaton, the null hypothess of the K-S test. We calculate the percentage of X ND and X D less than each value x of the tested varable and we record x for whch the dfference between the two fgures s maxmum. The K-S statstc equals the maxmum dfference between X ND and X D. (3) KS = max [ X X ] x ND D The p-value of the test s p=e -2*λ^2, where λ s gven by:

13 (4) ND D. λ = max KS,, where ND s the number ND + D ND D ND + D of non-downgrades and D s the number of downgrades. The man purpose of the K-S test flter s to elmnate varables that clearly do not dscrmnate between downgrades and non-downgrades. However, ths test s also used n order to obtan the sgn of the dscrmnaton, explctly whether the varable s generally hgher for future downgraded banks or lower. Ths result wll be compared to the followng tests so that the sgn of the explanatory varable wth regard to the dependent varable could be examned more carefully. The threshold for ths test was set at the. level of the p-value so that varables wthout a clear economc sense to be elmnated, but was not set lower to avod excludng potentally relevant varables. As second flter, we analyzed the monotony assumed by a logt model. In ths respect, for each explanatory varable, we bult a lnear regresson between the logarthm odd aganst the mean values for several data subsets and checked f the assumptons made for the relaton between the dependent varable and explanatory varable are respected. RD (5) ln = β + βx RD, where RD s the hstorcal rate of Downgrade and x s the average of the explanatory varable, both bult on the data subsets. The varables selected after these two flters are analyzed wthn a unvarate logt model framework. Hosmer and Lemeshow (2) proposed a threshold of.25 for the p-values of varables n unvarate models. Varable selecton can take nto account p- values, lkelhood values, as well as AUROC calculated for each unvarate model. PD (6) ln = β + βx PD, where PD s the probablty of Downgrade and x s the explanatory varable, both chosen over the entre estmaton sample. A fourth flter s gven by colnearty tests. It should be noted that any correlaton between selected varables should make economc sense. Varables wth a correlaton coeffcent above a threshold are analyzed and the one whch has a hgher performance n unvarate models s selected. 2

14 Explanatory varables whch have passed all the flters are subsequently analyzed n a multvarate framework. Backward selecton method mples contnung wth all the selected varables n a multvarate model. Lke structured n Hosmer and Lemeshow (2), we examned the Wald statstc for each varable and we compared the coeffcent of each varable wth coeffcents obtaned n unvarate models. It s mportant to see whether the sgns of the coeffcents change or whether ts sze s hghly volatle. Varables that pass these tests are employed n a new multvarate model, for whch agan the coeffcents are examned. A new model s compared wth the prevous larger model and n case the analyzed varable s consdered not provdng addtonal nformaton to the model, t wll be rejected. Ths process of elmnatng, refttng and comparng contnues untl all the varables ncluded n the model are statstcally sgnfcant as well as economcally sgnfcant, also checkng whether other relevant varables remaned outsde the model. In a forward selecton method, after we decded whch varables wll be used n a multvarate model, we ntroduced one varable at a tme, n ther unvarate performance order. If the new model s superor to the old model and f all the estmates are sgnfcant, the varable s accepted and therefore other varable s analyzed for selecton n the new model. Varable selecton for predctng banks recevng a bad ratng n one year horzon s smlar to the methodology presented for the predcton of ratng downgrades. In ths case, a Kolmogorov Smrnov test s appled to observe the dscrmnaton of each varable between the dfferent one year future ratngs. Analyzng varables n ths respect s more complex as t s requred that they dscrmnate between banks n hgh and low ratngs, also havng the opton to check the downgrades dscrmnaton. Monotony must also be respected for each selected varable compared to the logarthm odd of hstorcal rate of each future ratng. A thrd test s based on the unvarate models and checks the sgnfcance of each parameter estmated n the unvarate model bult for each tested varable. The colnearty flter s smlar to the flter employed for ratng downgrades predcton, therefore the varables wth a correlaton above the threshold are analyzed and the one wth a lower performance n a unvarate framework s rejected. The varables selected after the four flters are used n order to buld a multvarate model. Both backward and forward selecton methods are smlar to the methods used 3

15 for the ratng downgrades predcton, notng that the dependent varable s n the case of bnary dependent varable model the probablty to receve a bad ratng, respectvely the probablty of each ratng, n a one year horzon, for ordered logstc model Valdaton When the steps wthn the varable selecton are completed, the remanng varables enter the fnal model, whch has to be valdated n order to be consdered proper for the ntended purpose and to be used for predcton. Whle the models are useful n estmatng probabltes of downgrades, t s necessary to select a threshold above whch the dependent varable wll be estmated as, meanng a ratng downgrade. Ths threshold wll be estmated based on the mnmzaton of a loss functon whch assesses the loss of the supervsory authorty usng the model, dependng on the Type I (downgrades occurred when nondowngrades were estmated) and Type II (non-downgrades occurred when downgrades where estmated) errors. Estmated Equaton Dependent varable = Dependent varable = Total estmated Estmated dependent varable = Correctly estmated non-downgrades Unexpected downgrades (Type I Error) Estmated non-downgrades Estmated dependent varable = Unexpected non-downgrades (Type II Error) Correctly estmated downgrades Estmated downgrades Total Non-downgrades Downgrades Total Sample Therefore, the loss functon of the supervsory authorty has the followng specfcaton: (7) ϕ () c = ε ( c) ω + ε 2 ( c) ω2, where ε(c) are Type I and Type II errors, dependng on the cutoff value c and ω are ther respectve weghts. These weghts wll be selected by the decson maker and the cutoff wll have the value of c when the loss functon s mnmzed. 4

16 Another tool used for valdaton s the Recever Operator Characterstc (ROC) Curve. Ths method has the advantage of an easly understandable graphc representaton as an area part of the area of a square, whch represents the performance of the perfect model. In order to calculate the area under the ROC Curve (AUROC), we need the followng relatons: H ( c) (8) HR( c) =, where HR(c) s the ht rate for cutoff c, H(c) s the number of ND ratng downgrades estmated correctly wth cutoff c and ND s the total number of ratng downgrades. Ht rate s correspondng to the concept of senstvty, as the probablty of detectng a true sgnal. F( c) (9) FAR( c) =, where FAR(c) s the false alarm rate for cutoff c, F(c) s the NND number of false alarms wth cutoff c and NND s the total number of ratng nondowngrades. False alarm rate s correspondng to the concept of specfcty, as FAR=-Specfcty s the probablty of detectng a false sgnal. Havng calculated the ht rate and the false alarm rate and plottng them together we obtan the ROC Curve and the AUROC s subsequently calculated. (2) AUROC = HR ( FAR) d( FAR) ROC Curve Senstvty/Specfcty Probablty cutoff Specfcty Senstvty Senstvty (HR) Specfcty (FAR) ROC Curve s obtaned plottng HR and FAR over all possble probablty cutoffs. Area under ROC ranges from zero to one and provdes a measure of how the model dscrmnates between the realzaton of the dependent varable and the opposte event. As general rule for model performance, we use the followng thresholds for AUROC: 5

17 If AUROC<.5 If.5<=AUROC<.6 If.6<=AUROC<.7 If.7<=AUROC<.8 If.8<=AUROC<.9 If.9<=AUROC Faled test less than chance Faled test Poor test Far test Good test Excellent test Whle the AUROC ndcates the dscrmnatory power of the model, ths fgure alone may need to be analyzed wth respect to the sample used n ts calculaton. Therefore, we used a Bootstrap methodology, generatng AUROC fgures based on dfferent samples from a dstrbuton dentcal to the emprcal dstrbuton of the orgnal sample Ths method allows us to assess the stablty of the AUROC around the orgnal estmated value and to obtan varaton ntervals around ths value. In order to assess the goodness-of-ft of the model we used a Hosmer-Lemeshow test. In ths respect, we dvded the sample n g groups and we compared the estmated probablty of downgrade wth the emprcal percentage of downgrades for each group. The HL Test statstc for a model wth correct specfcaton follows a Chsquare dstrbuton wth (g-2) degrees of freedom and s calculated as follows: (2) Cˆ = g ' ( ok nkπ k ) ' ( π ) k = nkπ k k 2, where n k s the total number of subjects n the k th group, k o k = y j j=, y j s the ndcator of ratng downgrade, π k s the average estmated probablty for group k. 6

18 3.5. Predctons Once the varables have been selected and the model has been valdated, for both bnary dependent varable and ordered logstc models, the dependent varable s calculated. In sample, ths s done usng the values of the ratos already used n estmaton, whle out of tme the dependent varable s calculated based on values of the ratos not ncluded n the estmaton. The estmated dependent varables are compared to the realzed values n sample and, partcularly, out of tme. Whle for the bnary dependent varable model the dependent varable s easly comparable wth the percentage of ratng downgrades/number of banks recevng a bad ratng, for the ordered logstc model the probabltes calculated for each possble ratng have to be manpulated n order to obtan values comparable wth an observable varable. Frstly, the ordered logstc model can be used for the same purpose as the bnary dependent varable model, for nstance, n calculatng a probablty of ratng downgrade. M (22) PD = P, j + j = r, where M s the total number of ratngs, r s the current ratng of the observed bank and P,j s the probablty that the observed bank wll have the ratng j n one year horzon. Moreover, the probablty of downgrade estmated wth the ordered logstc model can be compared to the one estmated wth the bnary dependent varable model and the valdaton results can be analyzed as well. Ths can also be done n the case of predctng bank recevng a bad ratng. The ordered logstc model can also provde for a shadow ratng whch s the average of the possble ratngs, weghted wth ther respectve estmated probabltes. M (23) SR = P j =, j R j Consderng a nave model predctng the one year horzon ratng to be the current ratng, the estmated shadow ratng s expected to perform better. Both predctons are comparable wth the realzed ratng, observable n one year, usng a dstance functon as followng: N (24) ( ) 2 D = R R, where R s the estmated ratng for observaton, N s the = number of analyzed observatons and R s the observed respectve ratng. 7

19 4. Data For the purpose of ths paper, the nput data contans both mcroeconomc and macroeconomc varables (Annex ) from December 22 to December 28. Most of the mcroeconomc data s taken from the reports provded by a sample of about 3 Romanan banks to the Natonal Bank of Romana, ther supervsory authorty. These fnancal ratos are structured on the followng four man components: assets qualty, captal adequacy, proftablty and lqudty. Other varables wth specfc values for each bank and therefore consdered to be mcroeconomc are the CAAMPL ratng and the bank s poston n the market, as both assets and loans based market share. The other part of the nput data conssts on several ndcators at macroeconomc level whch have the same values for dfferent banks at the same moment n tme. These varables are current values and last varatons of ndcators related to nterest rates, exchange rates, wage, ndustral producton, unemployment rate and nflaton. It should be noted that the fnancal ratos are also comparable because of the reportng regulatons and procedures mantaned by the Natonal Bank of Romana. Moreover, the data only takes nto account banks whch are Romanan legal enttes, exceptng the savngs banks for housng. We have not selected branches of foregn banks that are not Romanan legal enttes because these banks have dfferent reportng regme and also a dfferent overall status, due to the drect nvolvement of the parent bank and home country supervsory authorty. The banks selected nto analyss have a cumulatve assets market share between 9.95% and 94.66% over the perod makng the results obtaned for ths sample relevant for the entre Romanan bankng sector. The avalable data was dvded nto three samples. The frst perod from December 22 to December 26 contanng 48 observatons for fnancal ratos and ndcators s used to estmate the parameters, wth the help of the one year future CAAMPL Ratng. The models bult based on these parameters are then tested n the followng perod, 4 observatons untl December 27, wth the help of the one year future CAAMPL Ratng, untl December 28. Subsequently, 6 observatons data untl December 28 s used to make predctons for the followng perod. 8

20 5. Results 5.. Ratng downgrade Frstly, we have bult an early warnng model for predctng CAAMPL ratng downgrades n one year horzon. In ths respect, we used a set of tests n order to elmnate varables that do not comply wth the assumptons made for them. The purpose of ths method s to obtan a set of varables that explan future ratng downgrades reasonably well ndvdually and to use them n a multvarate framework so that, at the end, to buld a early warnng model for ratng downgrade. Kolmogorov Smrnov Test Followng the steps presented n the methodology secton, we have started wth a Kolmogorov Smrnov test to fnd the varables able to dscrmnate between banks that wll have ther ratngs downgraded n one year horzon and banks that wll have at least the same ratng after that perod. The results show that for a threshold of. for the test p-value, only 5 varables wll be selected (see Annex 2). We show n a graphc representaton for two of the selected varables a)loans and deposts placed wth other banks/ Total assets (v4) and b)assets market share (CotaActve) compared to the graph of a rejected varable c)customer loans/customer deposts (v44) the dfference between the cumulatve dstrbuton functons F(x) for downgrades (blue lne) and non-downgrades (red lne). a) b) Emprcal CDF Emprcal CDF.9.8 F(x) F2(x).9.8 F(x) F2(x) F(x).5 F(x) x x c).9.8 F(x) F2(x) Emprcal CDF.7.6 F(x) x 9

21 Monotony For the purpose of the monotony test we bult subgroup regressons for the logarthm odd over the explanatory varables whch passed the frst test. The result s hghly dependent on the number of subgroups used; therefore the test results wll not be gven categorcal power of varable rejecton. Nevertheless, for a small number of subgroups, such as ten, we selected a threshold for p-value at.. The values for p- value reached a wde varety of values for selected and rejected varables, lke for a) Loans and deposts placed wth other banks/ Total assets (v4) and b)assets market share (CotaActve) compared to the graph of a rejected varable c) Customer loans/total labltes (v3). a) b) c) Ths test for monotony s used to reject only those varables whch clearly do not fulfll the logt assumptons. Both the number of subgroups and the threshold of p- value were selected n such manner to allow for varables wth present but weaker monotony to pass and enter the next flter of unvarate models. Unvarate framework The varables tested n a unvarate framework performed well, wth only one beng elmnated because of a p-value of.23 and a relatvely small AUROC. The tests already performed elmnated varables that clearly do not explan future ratng 2

22 downgrades, so that now s possble to analyze to correlaton between the selected varables and then buld a multvarate model. Multcolnearty The correlaton matrx for the so-far selected varables shows hgh correlatons between some of them. Ratng v3 v33 v4 v32 DIPI CotaCredte CotaActve Ratng v v v v DIPI CotaCredte CotaActve The threshold set for ths step s a correlaton coeffcent of maxmum.7. However, hgh values wll be further analyzed even f wthn ths threshold. Frst of all, the loans market share (COTACREDITE) s hghly correlated wth the assets market share (COTAACTIVE) but t wll be selected frst due to a hgher unvarate AUC: 57.5% compared to 55.6%. The correlatons between ROA (v3), ROE (v32) and Operatonal return rate (v33) are very hgh, but they wll be accepted for now, hghlghted and analyzed n the model buldng. It should be noted that the CAAMPL Ratng s hghly correlated wth ths three proftablty ratos as expected, the hgher the proftablty, the lower the Ratng (lower ratng ndcates better performng banks). Multvarate model The remanng varables were ntroduced n a multvarate logt model. In a backward selecton methodology, varables wth the hghest p-values were elmnated one at a tme, examnng the values of the model s lkelhood and Akake Informaton Crteron. If the new model s better, the varable s elmnated and a new teraton s done. After several teratons, and after reconsderng the elmnated varables n order to assess whether they perform better n a multvarate framework, we decded to replace 2

23 the loans market share wth the assets market share, as the latter was statstcally sgnfcant and allowed for a model wth hgher lkelhood and smaller AIC. The fnal model has the followng specfcatons: Varable Coeffcent Std. Error z-statstc Prob. C RATING ROE Loans and deposts placed wth other banks/ Total assets Assets market share Mean dependent var.647 S.D. dependent var S.E. of regresson Akake nfo crteron Sum squared resd Schwarz crteron.6952 Log lkelhood Hannan-Qunn crter Restr. log lkelhood Avg. log lkelhood LR statstc (4 df) McFadden R-squared Probablty(LR stat). As expected, better CAAMPL Ratngs ncrease the probablty of ratng downgrade, meanng that banks wth modest performance have lower probablty to downgrade than the better performng banks. In fact, ths can be explaned by the drect nvolvement of the stakeholders as well as the ncreased supervsory measures always appled to a bank wth poor performance. A ratng downgrade from Ratng to 2 or even from Ratng 2 to 3 s accepted wth more ease than a downgrade n the lower end of the scale. The nfluence of ROE ndcator on the probablty of downgrade s negatve, meanng that the hgher ROE, the lower the probablty, whch can be explaned by the fact that banks wth hgher proftablty have a stronger fnancal poston and therefore are less lkely to encounter a ratng downgrade. Hgher Loans and deposts placed wth other banks/ Total assets may ncrease the contagon rsk but may also be evdence that the bank has some problems n the 22

24 customer loan sectors or n other areas that are commonly more effcent and have a hgher return rate. The sgn of the Assets market share varable ndcates that smaller banks are more lkely to downgrade. Usually, bgger banks have sold portfolos and are safe of tensons generated by fast development or smply operatonal problems that are more costly to smaller banks. Valdaton and results After the model was bult, we checked ts dscrmnatory power usng the ROC curve and the Area under ROC. True postve rate (Senstvty) ROC curve False postve rate (-Specfcty) The AUROC of ths Early Warnng Model s 85.84%, statstcally greater than.5, whch s the value for a random model. In order to check the stablty of ths fgures, compared to the estmaton sample, we employed a bootstrap exercse, wth teratons. The 95% confdence nterval s (8.4%, 9.27%), so that the model s assessed as havng a good dscrmnatory power, wth lttle varatons due to the estmaton perod. The EW Model was assessed as good wth respect to ts dscrmnatory power and stablty n sample. However, n order to be used n predctng future ratng downgrade, the model should be tested for predctve power, n out of tme settngs. Usng the values of the selected varables n the year 27, we predcted downgrades for the year 28 and compared them wth the observed downgrades n the respectve perod. The AUROC for the out of tme sample s 74.8% wth 95% confdence nterval of (56.74%, 92.87%). The model s far n predctng out of tme downgrades, wth AUROC statstcally greater than.5. 23

25 ROC curve.9 True postve rate (Senstvty) False postve rate (-Specfcty) Due to the small out of tme sample of downgrades, the ROC curve s not concave and t should be nterpreted wth care. The ordered logstc model developed n next secton delvers probabltes for each possble ratng so that we can compute a probablty of downgrade by addng all the probabltes for each ratng worse than the current value. ROC curve True postve rate (Senstvty) False postve rate (-Specfcty) The AUROC of ths Early Warnng Model s 84.87% and the 95% confdence nterval s (79.3%, 9.45%), so that the model s assessed as havng a good dscrmnatory power, wth lttle varatons due to the estmaton perod. The AUROC for the out of tme sample for ths model s 8.39% wth 95% confdence nterval of (63.6%, 97.7%). ROC curve.9.8 True postve rate (Senstvty) False postve rate (-Specfcty) Due to the small out of tme sample of downgrades, the respectve ROC curve s not concave and t should be nterpreted wth care. However, the out of tme AUROC values for the ordered logstc model are hgher than those of the bnary dependent varable model, ndcatng that even though the models perform closely n sample, the 24

26 ordered logstc model predcts more accurate the downgrades of the CAAMPL Ratng n one year horzon. Wth respect to the loss functon of the supervsory authorty, the model was used n order to assess a threshold for a probablty of downgrade, whch wll be, wth the same notaton: c = argmnε ( c) ω + ε 2( c) ω2 c The weghts used for Type I and Type II Errors depend on the mportance gven by the supervsory authorty to unexpected downgrade events. If ths weght s.5 the probablty threshold wll be 2.2%, where as for the weght of.6667 t s.%. Included observatons: 48 Predcton Evaluaton (success cutoff C =.22) Estmated Equaton Constant Probablty Dep= Dep= Total Dep= Dep= Total P(Dep=)<=C P(Dep=)>C Total Correct % Correct % Incorrect Total Gan* Percent Gan** NA Included observatons: 48 Predcton Evaluaton (success cutoff C =.) Estmated Equaton Constant Probablty Dep= Dep= Total Dep= Dep= Total P(Dep=)<=C P(Dep=)>C Total Correct % Correct % Incorrect Total Gan* Percent Gan** 7.22 NA

27 Usng the ordered logstc model presented n the next chapter to estmate probabltes of ratng downgrades, the thresholds wll be 8.7% and.5%, respectvely. For the frst case, the errors wll be 9.48% (Type I) and 25.3% (Type II) and for the second, n whch the weght for Type I error s hgher, the errors wll be.69% (Type I) and 36.23% (Type II). Bnary dependent varable model Ordered logstc Model (see next Type I for probablty of downgrade secton) for probablty of downgrade Error Type I Type II Type I Type II Weght Cutoff Cutoff Error Error Error Error 5.% 2.2% 2.78% 2.84% 8.7% 9.48% 25.3% 66.67%.% 4.29% 29.78%.5%.69% 36.23% Wth respect to the probablty of downgrade, both types of models can provde useful results. Generatng the Kernel denstes for the two models allow us to draw the followng representatons for estmaton perod and for test perod. 5 Kernel Densty (Epanechnkov, h =.99) 5 Kernel Densty (Epanechnkov, h =.83) _PDBIN _PDMULTI 2 Kernel Densty (Epanechnkov, h =.374) 6 Kernel Densty (Epanechnkov, h =.75) _PDBINTEST _PDMULTITEST 26

28 Comparng the average probabltes of downgrade estmated by the two models, we fnd that the ordered logstc model s more conservatve to the end of the analyzed perod. Probablty of Downgrade.25.2 Value.5. Multnomal model Bnary model Month 5.2. Weak overall poston At ths pont, we have bult a model desgned for predctng CAAMPL ratng downgrades, whch can be a useful tool n bankng supervson. However, ths model should always be doubled by expert opnon and used just as the early warnng model whch t s. In fact, the output of the model s a probablty of ratng downgrade, wthout specfyng how many grades the downgrade could be and what could be the probablty that the one year horzon ratng wll be better. A much more useful tool wll be a model that can not only predct ratng downgrades, but can also provde a probablty for each possble ratng. Ths ssue s partcularly helpful as one can obtan an estmated one year horzon CAAMPL ratng, weghtng the possble ratngs wth ther estmated probabltes. For these reasons we employed an ordered logstc model, consderng the theoretcal background presented n the methodology secton as well as the general flters used n the varable selecton for ratng downgrades. In ths case, the varable selecton methodology seeks varables that explan a bad future ratng n one year horzon. Kolmogorov Smrnov Test Both models have to dscrmnate between banks wth hgher ratngs and banks wth lower ratngs n one year horzon. We used a Kolmogorov Smrnov test to check whether the varables fulfll ths requrement and we dvded the possble ratngs nto good (-2) and bad (3-4) ratngs. The test was passed by 4 varables, at a. threshold for the test p-value. For ths mode, we also present a graphc overvew of 27

29 two selected varables a)roe(v32) and b)solvency rato(v23) and one rejected varable c)customer loans/customer deposts(v44): a) b).9 Emprcal CDF FG(x) FB(x).9 Emprcal CDF FG(x) FB(x) F(x).5 F(x) x x c) Emprcal CDF FG(x) FB(x) F(x) x The maxmum dfference between the dstrbuton of good banks (red lne) and bad banks (blue lne) s vsbly hgher for selected varables compared to the varables rejected at ths step. Monotony Wth respect to monotony, we consdered dependent varable takes the value f the bank wll have a bad ratng and otherwse, n one year horzon. Smlar to the methodology for predctng probablty of downgrade, we used a regresson for the average logarthm odd of the dependent varable wth the average of each explanatory varable, on the ten created subgroups. The number of subgroups was selected consderng the sze of the estmaton sample and the purpose of buldng the monotony test, wth lmted dscrmnaton, so that to be sure we wll not exclude varables that mght perform well n a multvarate framework. We selected a threshold for p-value at. and the test showed the a wde varety of values for selected and rejected varables, lke for a)roe (v32), b)solvency rato (v23), respectvely c) Level Own Funds Index (v26). 28

30 a) b) c) Ths test was passed by 24 varables whch entered the unvarate models. Unvarate framework The next flter used was smlar to the case of the ratng downgrade predcton. We generated unvarate logstc models for the tested varables and we set up a threshold at. for ther p-values. The unvarate models assume a dependent varable gven by the one year horzon ratng, as for a good ratng and for a bad ratng. We then construct logt models wth ths dependent varable and each tested explanatory varable and we also check the AUROC for these models. Most of the varables performed well so that 23 varables passed ths test, havng only two varables rejected. Multcolnearty Next, the selected varables were analyzed based on ther correlatons. The competng varables have been ordered wth respect to ther AUROC n the unvarate models and then the correlaton matrx has been used to elmnate varables wth a correlaton hgher than the.7 threshold when compared to varables wth hgher unvarate AUROC. However, varables elmnated at ths step were hghlghted and compared n model buldng wth the varables they were correlated to. 29

31 Multvarate models At ths step, we elmnated varables that clearly do not explan the dependent varable, whch n ths case s the probablty of a bank to be bad n one year horzon. As presented before for ratng downgrades, we buld a multvarate bnary dependent varable model for ths partcular case. In a backward selecton methodology, varables wth the hghest p-values were elmnated one at a tme, examnng the values of the model s lkelhood and Akake Informaton Crteron. If the new model s better, the varable s elmnated and a new teraton s done. After several teratons, and after reconsderng the elmnated varables n order to assess whether they perform better n a multvarate framework. The fnal bnary dependent varable model, for good/bad banks has the followng specfcatons: Varable Coeffcent Std. Error z-statstc Prob. ROE Ratng Loans market share Solvency rato General rsk rate Consumer prce ndex Mean dependent var S.D. dependent var S.E. of regresson Akake nfo crteron Sum squared resd Schwarz crteron Log lkelhood Hannan-Qunn crter Avg. log lkelhood The varables selected after the above presented methodology were also ntroduced n a multvarate ordered logstc model, wth the one year horzon ratng beng the dependent varable, whch resulted n the followng fnal model: 3

32 Varable Coeffcent Std. Error z-statstc Prob. ROE Ratng Loans market share Solvency rato General rsk rate Lmt Ponts LIMIT_2:C(6) LIMIT_3:C(7) LIMIT_4:C(8) Akake nfo crteron Schwarz crteron Log lkelhood Hannan-Qunn crter Restr. log lkelhood Avg. log lkelhood LR statstc (5 df) LR ndex (Pseudo-R2).365 Probablty(LR stat). These selected varables passed all the tests n a constant manner, havng the same sgn both n unvarate settngs and n multvarate framework. The dependent varable of ths ordered logstc model s the one year horzon ratng so that the varables sgns have dfferent meanng than n the ratng downgrade model. As expected, the relaton between current CAAMPL Ratng and the one year horzon ratng s drect so the better the current CAAMPL Ratng, the better the one year horzon ratng. Ths ssue can also be explaned by the fact that the ratng s not so volatle n tme. A bank wth strong current poston s less lkely to be weak n one year tme than a bank that s currently already weak. Banks wth hgh proftablty, as ndcated by ROE, are more lkely to have a strong poston n one year horzon. The same goes for banks wth hgher market share whch, due to ther sze, have the means to properly manage ther portfolo n order to fnd ways n mantanng a strong overall poston. 3