BETA DISTRIBUTED CREDIT SCORE - ESTIMATION OF ITS J-DIVERGENCE

Transcription

1 The 7 th International Das of Statistics and Economics Prague September BETA DISTRIBUTED CREDIT SCORE - ESTIMATION OF ITS J-DERGENCE Martin Řezáč Abstract It is known that Beta distribution could provide a reasonable approimation of distribution of a credit scores which are the outcome of credit scoring models i.e. models which are used to determine the probabilit of default i.e. when the client fails to meet his or her credit obligations. Credit scoring models are used in practice in the majorit of decisions related to the granting of credits and are thus inherentl part of the majorit of processes approval encement commercial etc. in the financial sector. Besides Gini coefficient or K-S statistic J-divergence also called Inmation value is widel used to assess discriminator power of credit scoring models. However empirical estimator using deciles of scores which is the common wa how to compute the J-divergence ma lead to strongl biased results. The main aim of this paper is to describe properties of alternative both parametric and nonparametric estimators of J-divergence credit scoring models with Beta distributed scores. As we show the parametric and ESIS estimators are much more appropriate to use considering both the bias and mean squared error. Indeed better estimator leads to better assessment of models what ma lead to better credit scoring models used in practice. Ke words: J-divergence Inmation Value Credit Scoring Beta Distribution. JEL Code: E5 C4 C63 Introduction J-divergence is one of the frequentl used was of describing the difference between two probabilit distributions. When considering the qualit discriminator power of a classification model maimall different conditional probabilit distributions is eactl what one aims to. Thus the J-divergence is ver suitable and also widel used to assess the qualit of classification models. It is also known under the name of Inmation value in the case of its use the purpose of scoring models e.g. credit scoring models that are used to determine the probabilit of default i.e. when the client fails to meet his or her credit obligations. Credit scoring models are used in practice in the majorit of decisions relating to the granting 82

2 The 7 th International Das of Statistics and Economics Prague September of loans and are inherentl part of the processes approval collection sales etc. in the financial sector. Development methodolog of credit scoring models and methods assessing their qualit can be found in articles such as Hand and Henle 997 Thomas 2 Vojtek and Kočenda 26 or Crook et al. 27 and books like Anderson 27 or Thomas 29. This paper deals primaril with the J-divergence which is one of the widel used indices net to the Gini inde and KS statistics see Crook et al. 27 or Řezáč and Řezáč 2 details assessment of the qualit of credit scoring models. Usuall it is calculated b a discretization of the score into intervals using deciles with the requirement a nonzero number of observations in all intervals. However this could lead to strongl biased estimate of the J-divergence. As an alternative method to the empirical estimates one can use the kernel smoothing theor which allows to estimate unknown densities and consequentl using some numerical method integration to estimate value of the J-divergence. Another alternative is the empirical estimates with supervised interval selection ESIS proposed and discussed in Řezáč 2. Details connected to the kernel estimates and a discussion concerning both these approaches one can find there as well. The main objective of this paper is to describe the behaviour of the J-divergence estimates of credit scoring models with Beta distributed score. The second and the third chapter deal with the methodolog of these estimates including algorithms or reference to the relevant literature. The fourth chapter is then devoted to justif appropriateness of Beta distribution on a real data though the choice of this distribution tpe was justified in the literature e.g. in Jankowitsch et al. 27 or Morau 2. Furthermore this chapter is devoted to computation of the J-divergence on the real data. There are also using a simulation stud discussed the properties of estimates described within the paper. J-divergence Beta distributed score Jeffre divergence J-divergence of two random variables X and X with densities f and f is defined as smmetrised Kullback-Leibler divergence i.e. : : f ln D J X X DKL X X DKL X X f f d f where Kullback-Leibler divergence X : X is given b D KL : f ln. D KL X X f d 2 f 83

3 The 7 th International Das of Statistics and Economics Prague September Consider thus two random variables X and X representing suitabl transmed outputs of the credit scoring model bad clients in default and good clients. Let these random variables be Beta distributed with densities f and f defined b or B f 3. or B f 4 It is eas to show that the transmations i i i = convert random variables X and X to random variables Y a Y with densities otherwise B g 5 otherwise. B g 6 For such distributed random variables one can find analtical epression of the Kullback- Leibler divergence and hence also of the J-divergence. We get theree Y Y D J 7 where t is the digamma function. See Gradshtein and Rzhik 965 or Medina and Moll 29 details about the digamma function. The J-divergence can also be calculated b approimative mula using the relationship 5 ln t t see Johnson Kotz and Balakrishnan 995. Then it holds ln Y Y D J 8

4 The 7 th International Das of Statistics and Economics Prague September Furthermore one still needs to estimate the parameters and the practical estimation of the J-divergence. Tpicall this is done using the MLE estimates. Those are available eample in Univariate procedure in SAS sstem. Computational schemes of these MLE estimates can then be found in Johnson Kotz a Balakrishnan 995. Overall we obtain the parametric estimates b this procedure. One point has to be mentioned here. The J-divergence is not generall invariant with respect to transmations. Indeed this holds transmations used converting fourparameters Beta distributed variables given b 3 and 4 to two-parameters Beta distributed variables given b 5 and 6. Nevertheless when comparing the discriminative power of several credit scoring models on the same data then this propert disadvantage does not matter. And what is quite important estimation of parameters in 3 and 4 and consequent computation of the J-divergence is quite complicated. From this perspective it seems to be appropriate to use parametric estimate given b 7 or 8. 2 Non-parametric estimates of J-divergence In practice the most commonl used non-parametric estimators of the J-divergence are the empirical estimates. These are based on the idea of replacing unknown densities b empirical estimates of these densities de facto using appropriate relative frequencies. Let's have n score values i n bad clients and n score values i n good clients s i and denote L resp. H as the minimum resp. maimum of all values. Let's divide the interval [L H] up to r subintervals [q q] q q2] qr- qr] where q L qr H q i i r and q i i r are suitable border points e.g. appropriate quantiles of score of all clients. Set I s q The empirical estimate of the J-divergence is given b 85 n j n j n i n i I s i i q j j q ] j j q ] s i 9 j r observed counts of bad or good clients in each interval. Denote fˆ j the contribution to the inmation value on j th interval calculated b fˆ n n n n j j j j ln j r. n n n n j

5 The 7 th International Das of Statistics and Economics Prague September r D ˆ fˆ j. J j A special case is the decile estimate which uses scores of all clients and r = to determine the boundaries of the interval qi. Further algorithms are then e.g. ESIS see Řezáč 2 ESIS see Řezáč and Koláček 2 or ESIS2 see Řezáč 22. Further possible wa how to estimate the J-divergence is the usage of the theor of kernel densit estimates. For given M+ equidistant point of the score L we have M ˆ H L ~ ~ ~ 2 DJ f L f i f H 2 2M i ~ where f are estimated contributions to the J-divergence given b appropriate kernel i estimates of the unknown densities of bad and good clients scores. See Řezáč 2 more details. 3 Results The logical question is wh to consider just the Beta distribution. Some arguments were given in Jankowitsch et al. 27 and Morau 2. Furthermore the answer can be found in the following Figures and 2 and Tables and 2 obtained using the SAS UNARIATE procedure. There was some real data provided b a financial institution including output of a credit scoring model inverse logit transmation of minus the probabilit of default and a good/bad indicator of the client. The data range was observations see Řezáč a Řezáč 2 more details. M H Fig. : Fitted Beta distributions of scores. a bad b good clients. 86

6 The 7 th International Das of Statistics and Economics Prague September From the Figure fitted Beta densities and histograms and especiall from the Figure 2 Q-Q charts it is obvious that the choice of the Beta distribution was appropriate. Fig. 2: Q-Q charts fitted Beta distributions of scores. a bad b good clients. Source: Own construction The following Table contains the results of goodness of fit GoF tests of the eamined data. In case of bad clients score all considered tests did not reject the hpothesis of a Beta distribution. On the other hand in case of good clients score there was approimatel tenfold growth of test statistics the Cramer-von Mises and Anderson- Darling tests but the test statistics of the Kolmogorov-Smirnov test remained at approimatel the same value. Overall all three tests rejected the hpothesis of a Beta distributed score in the case of good clients score. Tab. : Goodness of fit tests of scores. a bad b good clients. a b The problem results of GoF tests good clients score lies in the ver large range approimatel 6 observations of data. It is commonl known that with a large data set the GoF test becomes ver sensitive to even ver small inconsequential departures from a distribution. Due to this phenomenon it is recommended large data set to follow the result of Q-Q chart rather than results of GoF tests. Even if we did not want to follow this recommendation it applies that when perming the same tests more specificall we did one thousand tests on a random sample of good clients score comprising % of the original 87

7 The 7 th International Das of Statistics and Economics Prague September number of observations we got approimatel the same results as bad clients score even with higher p-values all three tests. Overall we stated that scores of bad and good clients could be considered Beta distributed. Table 2 contains parameters of the fitted Beta distributions. The most striking difference between good and bad clients score is given b the parameter beta 4.8 vs. 6.5 and the parameter sigma 7.66 vs Tab. 2: Parameters of Beta distributed scores. a bad b good clients. a b The following Table 3 shows values of D J estimated b the algorithms mentioned above. The last line contains the parameter estimate given b 7 with MLE estimates of parameters. As the best non-parametric estimate seems to be the value 3.7 given b the algorithm ESIS. Tab. 3: Estimates of DJ. D J decil kern esis esis 3.73 esis param The question however is the general behaviour of these algorithms. A ver common wa to assess the qualit / properties of some parameter estimates or statistics are bias bias and mean square error MSE defined b ˆ bias E D J D J 3 2 Dˆ D. MSE E J 4 88

8 The 7 th International Das of Statistics and Economics Prague September The following Figures 3 and 4 show the properties of these algorithms from this perspective. Simulation stud leading to these results was carried out as follows. We consider n clients n bad and n p good p B is the relative frequenc of bad clients. In our case we p B B choose pb =. which is the closest to the aementioned real data. Furthermore we consider the parameters of Beta distribution resulting the value D J =. and Range of data sample we choose n = and n =. First we generate scores of bad and good clients depending on the selected parameters. Then we calculate all of the aementioned estimates. This process was repeated one thousand times. Mean values the bias and the MSE are then calculated as the arithmetic means. Fig. 3: Bias of estimates of Dˆ J Beta distributed of scores. Fig. 4: Logarithm of MSE of estimates of Dˆ J Beta distributed of scores. 89

9 The 7 th International Das of Statistics and Economics Prague September From the Figures 3 and 4 it is apparent that the decile estimate is significantl biased specificall undervalued. The value of log MSE became quite quickl stabilized and with increasing number of observations did not fall. Overall this estimate is thus not ver suitable. In contrast algorithms ESIS and ESIS2 led in the case of a weaker model DJ =. to almost unbiased estimate. For a stronger model DJ = 2.25 are their properties worse. However the were the best of all considered methods of estimating DJ. Conclusion The aim of this paper was to describe properties of the selected estimates of J-divergence also called Inmation value of credit scoring models with Beta distributed scores. It was given a mula the theoretical value of the J-divergence assuming this tpe of distribution. Its knowledge enabled the both compute parametric estimates but also to assess the qualit of non-parametric estimates. On real data it was presented an estimate of the J- divergence. In addition properties of aementioned estimates have been demonstrated on simulated data from the Beta distribution. Namel the were the bias and the MSE the data ranges from to. It quite obviousl turned out the weaknesses of traditional decile empirical estimation. Conversel it seemed that the algorithms ESIS and ESIS2 were good estimates of J-divergence with Beta distributed score. References Anderson R. The Credit Scoring Toolkit: Theor and Practice Retail Credit Risk Management and Decision Automation. Od: Od Universit Press 27. Crook J.N. Edelman D.B. Thomas L.C. Recent developments in consumer credit risk assessment. European Journal of Operational Research 833 pp Gradstein I.S. and Rzhik I.M. Tables of integrals sums series and products. New York and London: Academic Press 965. Hand D.J. and Henle W.E. Statistical Classification Methods in Consumer Credit Scoring: a review. Journal. of the Roal Statistical Societ Series A 6No.3 pp Jankowitsch R. Pichler S. Schwaiger W.S.A. Modelling the economic value of credit rating sstems. Journal of Banking & Finance 3 pp Johnson N. L. Kotz S. Balakrishnan N. Continuous Univariate Distributions volume 2 2nd edition. New York: Wile

10 The 7 th International Das of Statistics and Economics Prague September Medina L.A. and Moll V. H. The integrals in Gradshten and Rzhik. Part : The digamma function. Scientia Series A: Mathemaical Sciences 7 pp Morau R. Sensitivit Analsis of Credit Risk Measures in the Beta Binomial Framework. The Journal of Fied Income 93 pp Řezáč M. Advanced empirical estimate of inmation value credit scoring models. Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis LIX 2 s Řezáč M. Inmation Value Estimator Credit Scoring Models. Proceedings of ECDM 22 Lisboa s Řezáč M. and Koláček J. Computation of Inmation Value Credit Scoring Models. Workshop of the Jaroslav Hájek Center and Financial Mathematics in Practice I Book of short papers pp Řezáč M. and Řezáč F. How to Measure the Qualit of Credit Scoring Models. Finance a úvěr - Czech Journal of Economics and Finance 6 5 pp Thomas L.C. A surve of credit and behavioural scoring: ecasting financial risk of lending to consumers. International Journal of Forecasting 6 2 pp Thomas L.C. Consumer Credit Models: Pricing Profit and Portfolio. Od: Od Universit Press 29. Vojtek M Kočenda E. Credit Scoring Methods. Finance a úvěr-czech Journal of Economics and Finance pp Contact Martin Řezáč Masark Universit Facult of Science Department of Mathematics and Statistics Kotlářská Brno mrezac@math.muni.cz 9