Harnessing Traditional and Alternative Credit Data: Credit Optics 5.0 March 1, 2013
Introduction Lenders and service providers are once again focusing on controlled growth and adjusting to a lending environment that has forever changed. Today s regulatory and competitive pressures make it more important than ever for credit providers to make informed, effective risk assessments. To make the most attractive, yet profitable offers to increasingly in-demand consumers, it is no longer sufficient to use only traditional credit history data. Conventional credit scores have been shown to provide a limited view of consumer behavior and its associated risk. To deliver smart, targeted, offers for credit and services, organizations need comprehensive and current visibility into consumer risk, which is attainable only through the combination of traditional and alternative forms of credit data. Credit Optics from ID Analytics combines the power of traditional and alternative data to develop optimal credit decisions, allowing organizations to grow their portfolios while controlling exposure to risk. Credit Optics is a FCRA-compliant alternative credit score that provides the powerful, differentiated insights organizations require to develop a more complete picture of an individual s creditworthiness across the entire customer lifecycle. The score is designed to function either as a standalone credit score or as a plus one score capable of meaningfully enhancing the strength of existing custom and traditional credit scores. The newest version of the Credit Optics score leverages proprietary cross-industry event and performance data to provide a significant leap forward in predictive power, while continuing to be uncorrelated with traditional scores ensuring it contributes additive benefit when inserted into existing credit strategies and policies. This paper discusses how the newest and most powerful version of Credit Optics was built, including the analytic development process, model performance, stability, and impact of performance for ID Analytics clients. Credit Optics has been deployed across several industries including financial services, auto lending, subprime lending, and telecommunications. Using Credit Optics in combination with traditional credit scores, clients can achieve improvements in all of the credit decisions which impact a lender s bottom line, including prescreen, approval, credit line, and account management decisions. 2
The Advantages of the ID Network The unique risk perspective of Credit Optics is driven by the ID Network, a repository of consumer behavior data from a wider range of industries than other leading sources. A real time, cross-industry compilation of consumer risk information, the ID Network enables ID Analytics to deliver reliable, high-resolution visibility into how a consumer behaves across industries over time. By combining the traditional and alternative credit data found in the ID Network with world class analytics, Credit Optics is able to deliver a more accurate assessment of credit risk for more consumers than traditional credit scores alone. There is no question that traditional data assets used to create credit scores provide value in assessing a consumer s credit risk. Usage data, payment behavior and delinquency information on lending products provide valuable indicators into the likelihood of future consumer behavior on loans and services. What makes Credit Optics special is the addition of alternative data not typically provided in a credit score. Alternative data assets provide a broader view of a consumer s credit behavior beyond financial services accounts into other industries and payment vehicle types. This broader view, when input into advanced analytic models provides a more robust prediction of a consumer s creditworthiness. Credit Optics includes usage and payment behavior on wireless phones, utilities and cable, as well as use and payment behavior on payday loans and other sub-prime lending vehicles. Credit Optics also includes data on demand deposits accounts and alternative payment data, which can indicate a preference for non-traditional lending institutions. 3
The use of both traditionally predictive and alternative credit data enables Credit Optics to deliver a unique and predictive new perspective on consumer credit risk that is highly effective when used as a plus one score in addition to an existing credit bureau score or custom credit score. 4
Building Credit Optics 5.0 Credit Optics has been developed similarly for use in the bankcard, auto lending, and telecommunications industry. This whitepaper addresses the bankcard version of Credit Optics 5.0, which was developed specifically for use in prescreen, acquisition, and portfolio management credit decisions. To develop this model, ID Analytics employed a modeling technique known as LogitBoost, an extension of the popular AdaBoost method. This method uses a series of small segmentation trees that negate the need for upfront population segmentation. The training population contained bankcard inquiries that resulted in active trades. The timeframe selected for the model development sample included the first quarter of 2008 through the second quarter of 2011. This time period is important because it covers a rapidly changing credit environment containing elements of a deep recession as well as its ensuing recovery. The development sample included a representative population of over 9.4 million inquiries that resulted in active trades and combined eight different data sources. Having such a broad sample of data helps deliver the most robust model possible, which provides high performance during varying economic conditions. Characteristics for Credit Optics were calculated based on a view of an applicant s information from the ID Network, thus not restricted to just financial trades. The layers and complexity of the data and interactions available through the ID Network mean that many thousands of variables are generated as candidates to the final model. This data was combined with the many traditional credit bureau-type attributes, also from the time of application. These may include elements such as number of bankcard lines open, utilization, and highest credit limit. The unique insight available through the combination of traditional and alternative credit data leads to a credit score that performs better than either approach alone. 5
a. Performance Data Credit Optics uses a bad tag consistent with industry standards: ninety days past due (DPD) within twelve months of the date of inquiry. Charge-offs, bankruptcies and collections are all also included in the definition of bad. Good accounts were defined as accounts with no more than thirty days past due within a twelve month time window. All other accounts were excluded from the modeling population as indeterminate. The 9.4 million inquiry sample included portfolios with vastly different bad rates, coming from all denominations of creditworthiness. The portfolios included in the final consortium had bad rates ranging from below 1% to over 10%. b. Population Segmentation Testing the robustness of the model warranted dividing the population into three distinct time periods. The early holdout set and the late holdout set were complete out-of-time samples not used in the development of the model. The early holdout set was selected and Q1 of 2008, representative of applications made during a declining credit landscape. The late holdout set was selected from Q2 of 2011. The remaining applications consisted of the model development set; this was subdivided into various training, testing and holdout sets as shown in Figure 1. All candidate models were evaluated independently on the three sets: early out-of-time, late out-of-time and in-time holdout. The only non-time dependent dimension of segmentation was dividing the population into hit versus no-hit segments based on the applicant s presence or lack of traditional credit relationships. Where an account was deemed a hit, traditional credit variables were used to enhance the ID Network-derived variables. In the no-hit group the ID Network-derived from nontraditional financial relationships were used. Data such as cell phone and payday loan information from the U.S. consumer base is an example of non-traditional data. While these no-hits would typically be underserved by traditional credit scores, Credit Optics 5.0 provides a very reasonable determination as to how these applications would perform. Figure 1 Credit Optics 5.0: Model Development Sample ( 000s) SEGMENTATION HIT NO HIT TOTAL Early Holdout 2,529 279 2,808 Model Development Set 5,534 531 6,065 Late Holdout 528 35 563 Total 8,591 845 9,436 6
c. Modeling Algorithm The foundational algorithm used to train the Credit Optics 5.0 model is known as LogitBoost, which is an extension of the popular AdaBoost modeling method. This method uses a log likelihood loss based on a logit representation of the probability of bad as a function of the optimized quantity. The key feature of the method is that the solution of the optimization problem is represented as a sum of simple classifiers (in this case, regression trees). Each classifier in the sequence is determined by an optimization of the loss function with a prior determined by the sum of the previous classifiers, which effectively reweight the examples, focusing on those that were poorly classified. This weighting is not ad hoc; rather it derives directly from the optimization of the log likelihood loss function in the presence of a prior. The distinctive characteristics of LogitBoost relative to AdaBoost are the use of the log likelihood rather than exponential loss function, as well as implementing a Newton update with each iteration, which provides a more robust approach to the optimal solution. The final algorithm contained a set of proprietary modifications to the LogitBoost approach that added quantitative improvement to the model. d. Model Training Many combinations of parameters were explored in building the latest Credit Optics model. These parameters included differential bad sampling, time-decay weighting, scoring initialization, adjusting model parameters and using novel proprietary transformations of the data. Presegmentation strategies were also tested, however a model using two segments, hit versus nohit, as described above, was found to be the best performing and most robust. Each model was tested on four different training datasets, selected from the model development set time period and including several hundreds of thousands of accounts chosen from each of the different datasets. The models were also compared to other models built on custom versions of the datasets and were shown to be comparable in terms of performance. Candidate variables were considered in several rounds of testing, to ensure that the model included a set of variables that contributed significantly to the model s performance and to ensure that only variables that did not appear to have an interaction with the calendar or credit landscape at the time were selected this step was critical in ensuring a model that was the most robust. The final candidate model was one that performed best on population weighted average across the datasets on the held-out population. 7
e. Variable Selection The model s performance across time periods and portfolios was enhanced by a careful selection of the variables to be included in the final model. For variable selection, ID Analytics employs a backward selection methodology with a model-driven performance metric. In addition to the typical scrutiny applied to any variables to be included in credit models, each candidate variable was rigorously tested to see if there was any disparate effect with regards to credit regime effectively whether a variable had a different effect on the score produced during the recession rather than at other times. Figure 2 provides an example of one such variable, which was excluded from the model. In this example, the relative ratio of bads and goods is dependent on time much higher during the recessionary period - which could adversely affect the model s performance over time. A number of such variables were excluded. Figure 2 8
4. Model Performance Results The overall model performance, considering all hits and no-hits over the entire holdout set (much of it out-of-time) had a maximum overall KS of 39. Breaking down the performance data by credit class, the model has KS values ranging from 27 to 46, with a good performance in all three time periods, as shown in Figure 3. Figure 3 Credit Optics 5.0: Maximum KS CREDIT CLASS EARLY HOLD-OUT IN TIME LATE HOLD-OUT Prime 45 46 38 Near-Prime 41 39 34 Sub-Prime 28 31 27 It s worth noting that the KS values shown within each credit class are lower than the maximum KS values observed on files spanning all credit classes. The lower KS values within credit classes are expected, as the KS statistic will typically decrease as populations become more homogenous. While different credit classes exhibit different levels of performance, they were consistently within a percent or two of a custom model generated using a specific dataset/time period combination. There was no significant bias in performance by credit class. As seen below, prime portfolios have a KS in the mid-40s, a near prime portfolio performs in the low-40s and a mid-subprime also performs in the low-30s. The more subprime (as determined by overall bad-rate) dataset had a generally lower performance, but this was a function of the greater number of no-hits present in that portfolio. Indeed, this specific portfolio had a no-hit rate of thirty percent, against the less than ten percent for the entire population. Figure 4 breaks down the Figure 3 results to show performance on the hit and no-hit populations: Figure 4 Credit Optics 5.0: Maximum KS CREDIT CLASS HIT NO HIT Prime 46 Near-Prime 39 Sub-Prime 39 31 9
Perhaps as important as the model s overall predictive strength, is the lift that it provides when used in conjunction with a traditional credit score as a plus one. Figure 5 demonstrates the lift provided to a Top 10 Bankcard issuer who tested Credit Optics both as a standalone credit score and as a plus one with its existing credit score. Credit Optics, when used as a plus one, provided a significant lift over using either Credit Optics or the traditional credit score as a standalone risk assessment tool. Figure 5 Credit Optics 5.0: KS Results for a Top 10 Issuer MODEL MAXIMUM KS Traditional Score 42 Traditional Score + Credit Optics 53 While the bureau score the issuer was using performs well, combining it with Credit Optics makes a powerful difference. Figure 6 shows additional detail illustrating how Credit Optics provides meaningful new information capable of significantly improving an organization s risk decisions. Figure 6 Credit Optics 5.0: Maximum KS High Risk 2 3 4 5 6 7 8 9 Low Risk Total Traditional Risk Scores Below 595 6.1% 3.9% 3.6% 3.5% 3.2% 3.1% 3.2% 3.0% 2.3% 1.3% 3.3% 595 654 5.2% 4.1% 3.9% 3.8% 2.6% 2.5% 2.1% 2.0% 1.7% 1.5% 2.9% 655 694 4.7% 3.0% 2.5% 1.8% 1.8% 1.6% 1.5% 1.4% 1.0% 1.2% 2.1% 695 719 4.4% 2.0% 1.6% 1.5% 1.4% 1.0% 1.0% 0.8% 0.8% 0.5% 1.5% 720 734 3.3% 1.5% 1.2% 1.0% 0.8% 0.7% 0.7% 0.5% 0.5% 0.5% 1.0% 735 749 2.5% 1.0% 0.8% 0.8% 0.6% 0.5% 0.6% 0.3% 0.5% 0.2% 0.8% 750 759 1.6% 0.8% 0.7% 0.5% 0.3% 0.5% 0.3% 0.3% 0.2% 0.2% 0.6% 760 769 1.2% 0.6% 0.5% 0.3% 0.3% 0.2% 0.3% 0.2% 0.1% 0.1% 0.5% 770 785 1.0% 0.3% 0.5% 0.5% 0.3% 0.2% 0.2% 0.2% 0.2% 0.1% 0.3% Above 785 0.8% 0.3% 0.2% 0.2% 0.1% 0.1% 0.1% 0.1% 0.1% 0.1% 0.2% Total 4.3% 2.9% 1.6% 1.3% 0.9% 0.7% 0.6% 0.5% 0.3% 0.2% 1.3% Accounts with similar performance and vastly different Risk Scores As seen above, Credit Optics and the traditional risk score are not highly correlated (correlation coefficient = 0.642). This means that Credit Optics has additive power to differentiate high risk consumers who score low risk with a traditional credit score (important for risk assessment) and low risk consumers who score high risk with a traditional credit score (important for expanding growth populations). Figure 6 illustrates the ability of the Credit Optics score to separate risk within traditional score 10
bands. This can be seen by examining each row, where a population traditionally viewed as homogenous by a traditional score is further segmented into more granular populations of varying risk as identified by the Credit Optics score. Across rows one can discern many examples similar to the one highlighted where the overlay of the Credit Optics risk assessment identifies populations of similar risk but significantly different traditional scores. In each of these instances, Credit Optics is providing the issuer with refined insight into consumer credit risk required to make more informed, profitable lending decisions. Further evidence of the robust performance of Credit Optics is that the model s odds remain consistent through time. Figure 7 demonstrates the log-odds ratio by decile of the entire scored population, ordered by score (from high to low) using an equal binning method. A log-odds of two corresponds to an odds ratio of 100 (very low probability of going bad as per the definition), which is consistent with being given a high score. What s important to note is that these lines are reasonably consistent across time. 11
Figure 7 715 640 590 510 475 435 375 300 <300 Observing the results at the portfolio-level, as seen in Figure 8 for the subprime portfolio, the lines are even more consistent. The odds are slightly worse at each decile with respect to the overall population, especially in later deciles, which would be expected for a subprime portfolio. The decrease in odds by decile is consistent through time, showing slightly improving overall odds between 2008 and 2010, which reflects the credit environment. Figure 8 715 640 590 510 475 435 375 300 <300 12
Conclusion As lenders, telecommunications providers and utility companies focus on acquiring and retaining highly profitable consumer relationships while controlling credit risk and complying with all regulatory requirements, there is a need to achieve a new level of visibility into a consumer s credit profile. Traditional credit scores cannot provide a complete view of the consumer s credit history due to the limited data used to calculate these scores. To identify, acquire and cultivate the right consumers for credit and service offers, organizations need more complete, up-to-date access into a consumer s risk assessment. This is achieved through the unique combination of traditional and alternative credit data. Credit Optics is designed to accurately predict credit risk on its own, boost the power of traditional credit scores, and provide actionable intelligence on the emerging market. Using the newest version of Credit Optics, clients can achieve increased revenue and decreased credit losses through improvements in prescreen, approval, credit line, pricing, and portfolio management decisions. Credit Optics is built to be FCRA compliant, while also being transparent and consumerfriendly, enabling organizations to incorporate Credit Optics into current credit risk strategies with complete confidence. For more information on how Credit Optics can help your company confidently make more informed, profitable lending decisions, contact us today at marketinginfo@idanalytics.com, 858-312-6200, or visit www.idanalytics.com 13