Forecastig bad debt losses usig clusterig algorithms ad Markov chais Robert J. Till Experia Ltd Lambert House Talbot Street Nottigham NG1 5HF {Robert.Till@uk.experia.com} Abstract Beig able to make accurate forecasts of future losses has obvious beefits to a compay. These losses are ot oly of iterest to shareholders i terms of impigig o future profits, but also allow appropriate provisios for future bad debt to be made. We describe the aalysis udertake as part of a recet project to forecast bad debt losses for a major UK Plc. Segmetatio algorithms ad Markov chais were applied ad the resultig forecasts were favourable over the cliet s curret methodology. However, the improvemet from the iclusio of clusterig algorithms varied substatially depedig o the time period beig modelled. Backgroud A major UK plc approached Experia to ivestigate usig Markov chai models to calculate bad debt losses o their portfolio. The cliet wished bad debt to be calculated to year ed, ad also up to 3 years i the future. Rather tha modellig losses directly, the project took a idirect approach ad modelled a accout s behavioural score. A accout s behavioural score is a measure of the probability of default withi the subsequet moths. It is built usig other behavioural characteristics ad hece ecompasses a accout s history i its make-up. Give this measure, expected losses for a portfolio ca be foud. The aalysis ivolved two key stages. Firstly, the populatio was segmeted accordig to future performace of the behavioural score. This complemets the Markov chai aalysis. Secodly, Markov chai models were built ad applied to each sub-populatio to predict behavioural score. The behavioural score was segmeted ito five equal sized bads with three further bads for whether a accout was yet to ope, closed for bad debt or closed for reasos other tha bad debt. Our iterest is i fidig whe a accout will close for bad debt. This will eable volumes of accouts eterig bad debt withi the ext few moths to be foud. Modellig the behavioural scorebad allows a accout s history to be used i the modellig procedure, whilst still beig able to utilise the
statioary Markov models to obtai valuable results. Covertig bad debt volumes ito a loss value for the accout portfolio is the fial step. The efficacy of the segmetatio routie, together with the accuracy of the results from the Markov models, will be described. This paper is split ito five sectios outliig the aalysis udertake. The ext sectio details the data that was used ad the sample creatio. The secod ad third sectios address the segmetatio ad Markov chai routies. The fourth ad fial sectios give details of the various models built ad compare the results of the optimum MC model to that curretly employed by the cliet. For reasos of cofidetiality % improvemets will be give rather tha actual figures. The Data The form of the agreemet betwee the cliet ad their customers is that at the ed of each moth the customer must repay i full the amout owed to the cliet for ay expeses they have accrued that moth. If the customer fails to repay the they eter deliquecy. Each customer also gets a behavioural score each moth ad it is the performace of the behavioural score over time that we will model. Whe certai coditios have bee met regardig derogatory performace a accout will be discoected ad collectios activity may commece. Data was collated over the time frame October 1999 to October 2002. These data were split ito two groups accordig to whether they had a discoectio reaso code for bad debt or whether they were closed for reaso other tha bad debt / still active. A sample of accouts was take to perform the aalysis. A much larger proportio of accouts that were discoected for bad debt was take tha accouts that were ot discoected for bad debt. The sample comprised approximately 140,000 accouts ad various checks were made to esure that the weighted sample mirrored the populatio. The data to be aalysed were i the form of a time series. At each of the 37 moths we had the behavioural scorebad for a accout (icludig the bad for yet to ope) ad certai other behavioural characteristics. The scorebads were labelled 0-7, with 0 beig ot yet ope, 1-5 represetig actual behavioural score partitios (the higher the bad the lower the probability of default), 6 closed for bad debt ad 7 closed for other reaso. Example behavioural scorebad profiles are give below
Moth 1 2 3 4 5 6 7 8 Obs 1 0 0 5 5 4 5 4 6 Obs 2 3 3 2 2 1 1 7 7 Obs 3 0 0 0 0 0 0 0 4 Table 1 Segmetatio Markov chais model mothly trasitios betwee the behavioural scorebads of a group of accouts. By applyig Markov chais to a group of accouts that behave similarly with respect to their behaviour score profiles more accurate predictios will be obtaied. Creatig such groups is the remit of the segmetatio algorithms. Two stadard segmetatio techiques were ivestigated: fastclus - a partitioig algorithm i base SAS; ad CHAID - a decisio tree algorithm available i the Eterprise Mier module of SAS. - Fastclus is a usupervised techique that seeks to classify data such that the groups are as iterally cohesive ad exterally isolated as possible. Each of the behavioural characteristics for a accout is treated as a dimesio i a multi-dimesioal space; this allows each accout to be plotted ad hece the separatio of accouts to be quatified. All characteristics are stadardised first ad categorical data are coverted ito dummy variables. - CHAID is a supervised techique that segmets accouts by meas of choosig splits o characteristics that creates groups that differ as much as possible with respect to a objective. Here, the objective fuctio is the future behavioural score bad. A split produces two or more braches, each of which cotais homogeeous data. To each brach further splits are made i a iterative maer. The more effectively a variable ca split the data ito heterogeeous groups, the earlier it will be used i the splittig procedure. Whe further splits do ot sigificatly icrease the separatio the process is stopped. Differet models were built allowig for the objective fuctio to be defied at differet moths i the future. This mirrors the aalysis of predictig bad debt at various times i the future. Various behavioural characteristics were looked at for these segmetatio algorithms, icludig curret behavioural score, moths sice ope, customer age, curret deliquecy ad time at curret address. Two further algorithms were ivestigated. A routie based solely o segmetig the accouts accordig to their curret (statioary) behavioural scorebad ad a routie based o the movemet of the behavioural score over a pre-defied period of previous moths. To test the performace of the segmetatio algorithms we eed to quatify how homogeeous the accouts withi each cluster are i terms of the
behavioural scorebad profiles. Let S tc be the stadard deviatio of the behavioural scores i moth t ad cluster c. Here, t rages from 1 (curret moth) to T (T-1 moths time), c rages from 1 to C (max # clusters). A overall measure of how homogeeous the clusters are, lookig over time, is give by: H t C = TC H will take small values for clusters cotaiig accouts with similar behavioural scores; large values for clusters cotaiig accouts with dissimilar behavioural scores. To allow for o-statioarity i the data H was foud at various time periods i the 37 moths, ad for various values of t. That is whether we are predictig the level of homogeeity of the accouts over the ext t=4 moths or t=24 moths, say. Aalyses showed that CHAID ad usig the statioary behavioural score gave the most favourable results. It is worth otig that i all cases the segmetatio algorithms gave better results tha if o segmetatio was employed. S tc Markov chai aalysis A Markov chai will be built for each group created by the segmetatio algorithm. The aim is to predict future behavioural scorebads ad i particular whe a accout will close for bad debt. This will allow umbers of accouts eterig bad debt withi the ext t moths to be foud. The fial step is to covert this volume to a actual bad debt value. We will restrict attetio to a first order statioary Markov chai. A oe-step trasitio matrix determies the behaviour of this type of Markov chai. The trasitio matrix is built usig historical data ad cotais probabilities of a accout movig i the ext time period to each of the scorebads give the scorebad it is curretly i. Let ij (t) be the umber of accouts makig a trasitio from scorebad i to scorebad j at moth t. The mle s for etries i the trasitio matrix defiig movemets from time t are defied by: ij ( t) pij ( t) = i. ( t) The statioary assumptio meas that the uderlyig true distributio of trasitios is the same over all time periods. Therefore, to obtai the best estimate of this true distributio we must average over all trasitios over all time periods. The mle s for this global oe-step trasitio matrix are: p ij = t t ( t) ij ( t) i.
The trasitio matrix is applied to curret accouts to predict their future behaviour. A example trasitio matrix, P, is give below: Moth t+1 1 2 3 4 5 Closed bad Closed ot bad 1 0.85 0.09 0.01 0.00 0.01 0.01 0.03 2 0.05 0.76 0.12 0.04 0.01 0.00 0.02 Moth t 3 0.03 0.08 0.70 0.15 0.03 0.00 0.01 4 0.01 0.04 0.03 0.70 0.12 0.00 0.00 5 0.00 0.02 0.07 0.03 0.88 0.00 0.00 Closed bad 0.00 0.00 0.00 0.00 0.00 1.00 0.00 Closed ot bad 0.00 0.00 0.00 0.00 0.00 0.00 1.00 Table 2 This trasitio matrix shows that if a accout is curretly i scorebad 1 the there is a probability of 0.85 of the accout still beig i scorebad 1 ext moth ad a probability of 0.01 of the accout closig for bad debt ext moth. Note that the scorebads closed for bad debt ad closed for ot bad debt are absorbig, meaig oce a accout has etered these bads it caot leave. This has the immediate cosequece that volumes of accouts i these bads are cumulative over time, ad some simple maipulatio is therefore required to fid the volumes of accouts goig bad at each moth. The Chapma-Kolmogorov equatios are used to show that P t gives the probability of a accout movig from oe scorebad at time 1 to each of the scorebads at time (t+1). Note that the scorebad referrig to ot yet ope is excluded from the oe-step trasitio matrix. The distributio of ew accouts amogst the scorebads will be determied via the cliet rather tha a Markov trasitio matrix usig historical iformatio. We have dealt so far with lookig at mothly trasitios i buildig the trasitio matrix, but this eed ot be the case. The oe-step trasitio matrix could easily be describig a trasitio over 2 or 3 moths, say. I fact, the behavioural score gives the probability of default withi the succeedig 4 moths ad hece lookig at trasitios over a loger period could well be preferable. Although the trasitio matrix may be defied i terms of trasitios over 3 moths say, it is still possible to fid the behaviour of accouts i ay moth. For example, the behaviour i oe moth time will be give by the matrix P 1/3. A trasitio matrix is used to describe how accouts curretly active will behave i t moths time, but we must also take ito accout how accouts that are curretly ot ope, but will ope i time period before t, will behave.
Suppose C 1 defies the distributio of accouts amogst the behavioural scorebads at moth 1. The C 1 P gives the expected distributio of accouts at moth 2. Similarly, C 1 P 2 gives the distributio of the accouts from moth 1 at moth 3. However, we must also take ito accout ew coectios at each moth. If defies the distributio of these ew coectios at moth 2 the C 2 C 2 P will defie their behaviour at moth 3. For those accouts ew at moth 3 0 C 3 2 P the defies their distributio at moth 3. This is equivalet to C. Therefore, a estimate of the distributio of accouts at moth t i each of the behavioural scorebads,, is give by: Clearly, Ĉ t Cˆ t = C P 1 t 1 t + C i= 2 C i, 1 i t, eed to be estimated. This was doe via the cliet. i P t i Model buildig Various models were looked at i fidig the best predictios of loss volumes. I particular, the followig issues were addressed: The time period to which the predictios are made over. Are differet amouts of historical data required depedig o how may moths i the future we are predictig? Modellig a seasoal effect i the data, e.g. Christmas period. Here, differet trasitio matrices will be built for the differet times of the year, ad the combied. This is a form of o-statioary Markov chai. Whether to build the trasitio matrices defiig movemets over 1 moth or over a greater period of time. Whether to build separate trasitio matrices for ew ad existig accouts ad the combie them i a fial model. To test the statioarity assumptio, plots were made of the mothly trasitio probabilities over time. The plots show little seasoality although a high level of volatility over the moths of the trasitios. This will have the cosequece of predictio accuracy ot beig uiform over the moths. I choosig which models are better, predicted volumes of accouts eterig bad debt were compared to actual volumes of accouts eterig bad debt each moth. Comparisos betwee actual volumes ad predicted volumes of bad debt were i the form of a residual: ( Observed volumes - Actual volumes) Actual volumes
Results Iitially, o segmetatio procedure was employed. This will allow us to quatify the improvemet i results attributable to the segmetatio procedure. The optimal model was foud for fidig bad debt volumes at various times i the future. Comparig the Markov aalysis residuals to those from the cliet s curret methodology was complicated because of a disparity betwee the datasets that the two models were built o. The dataset used for the Markov aalysis was far more volatile ad therefore the % icreases i accuracy that the Markov model achieves over the curret methodology ca be treated as a lower boud of what the true improvemet is. The results ca be see i Table 3. For reasos of commercial sesitivity we will look at data over a year ago. The table shows the differeces i residual betwee the Markov aalysis ad the curret approach. Positive differeces idicate a improvemet with the Markov aalysis. We are treatig October 2000 as the last date for which we kow actual bad debt volumes ad hece are predictig bad debt volumes after this date. Predict at Moth % improvemet Nov-00 20% Dec-00 34% Ja-01 1% Feb-01-9% Mar-01-5% Apr-01 4% May-01-6% Ju-01 24% Jul-01 26% Aug-01 4% Sep-01 4% Oct-01 12% Table 3 It is clear that the Markov aalysis has led to improved accuracy for the predictios. The improvemet is ot uiform over the moths for reasos give earlier. Curiously, applyig segmetatio to the Markov aalysis did ot lead to a cosistet improvemet i performace. While over some time frames the results were sigificatly improved, over other time frames the segmetatio led to oly a slight improvemet. The ext step is to covert the predicted umbers of accouts goig bad to a actual loss value. The methodology for this is similar to the cliet s curret approach ad hece we will ot elucidate.
Coclusio This project looked at the effectiveess of employig clusterig ad Markov chai algorithms to forecast bad debt. For reasos of cliet cofidetiality absolute values of accuracy for this model caot be give. However, as a compariso to the cliet s curret methodology the results from this techique were favourable.