DFAST Modeling and Solution

Transcription

1 Regulatory Environment Summary Fallout from the financial crisis included the emergence of a new regulatory landscape intended to safeguard the U.S. banking system from a systemic collapse. In 2012, the Federal Reserve Board of Governors (Fed), began to require the largest U.S. Bank Holding Companies (BHCs) to file a Comprehensive Capital Analysis and Review (CCAR), with stress tests intended to assess the capital adequacy of these BHCs in times of crisis. By 2015, CCAR and stress tests, now known as DFAST (after the Dodd-Frank Act Stress Tests) were expanded to include U.S. BHCs with between $10 and $50 billion in consolidated assets and foreign banks, whose exempt status expired. Requirements for CCAR reporting & DFAST stress testing result in complex data challenges for many banks. Stress test models must consider current regulations & updates to forecast scenarios. For banks, CCAR reporting and DFAST stress testing are complex and data intensive endeavors with some of the following challenges: DFAST requires credit modeling and risk assessment at a granular level over vast amounts of data There is often a need for third-party data from sources such as Trepp to supplement internal data Retrieving, maintaining, & standardizing both internal and external data is usually difficult and time-consuming Subsets of data selected for reporting and testing must reflect the existing portfolio of loans at the bank Most financial institutions simply do not have the expertise nor personnel necessary to efficiently meet their regulatory requirements and thus require outside statistical modeling and reporting assistance. In 2013, a Fed report on the financial industry s compliance progress noted that several banks revenue estimates were inaccurate due to data limitations, the use of sub-optimal predictive models, and weak information management systems. In order to comply with increasing stringent Why do banks have inaccurate revenue estimates? Data Limitations Poor Predictive Models Weak IMS Technology regulatory requirements, banks will need state of the art stress test models that can consider current regulations along with any updated forecast scenarios that may be introduced by regulators. In addition to more efficiently meeting DFAST requirements, high quality, granular models generated from the stress test process help banks discover previously undetected risks in loan portfolios, gain a better understanding of risk across all positions and lead to better overall management decision making. PG 1

2 Through project work and development of a software solution, Opex Analytics has gained substantial experience developing robust statistical loan loss models for bank portfolios using a combination of internal historical loan data, economic factors, and third-party data. By working with clients, Opex has established the following process for commercial loans which is also reflected in the delivered software solution. STEP 1 We perform exploratory analysis on both Trepp and the Company s internal data to produce a model that predicts risk based on historical loan data. Stepwise logistic regression is typically used to predict risk. All models are tested on recent holdout data and using 10-fold crossvalidation. 1 Suitability & Exploratory Analysis A suitability analysis is performed to screen external data and the Company s loan tape data to assess whether the loan loss experience in the Trepp data is comparable to the Company s default and loss experience. For example, Trepp data is filtered to find the percentage of loans defaulted by property type during a given time period. The correct filter is identified so that the percentage of loans defaulted in Trepp matches the Company s historical observations. The results of this analysis are used to confirm the suitability of using the Trepp data as a proxy for the Company s loan loss history. All subsequent analyses are then performed only on the selected subset of Trepp data. In conjunction with the suitability analysis, Opex also performs data exploration analyses on both Trepp and the Company s internal data. This includes examining distributions on various attributes, detection of possible outliers, and ensuring the data exhibits expected patterns (for example, many defaults during the recession of ). Logistic regression is used to correlate predictors with risk and in term probabilities of default. If the Company does not possess enough historical data, the analysis is performed on the Trepp subset. The model establishes a relationship between the risk score and indicators with each indicator associated with a weight coefficient that is determined based on the so-called maximum log likelihood. Proper use of logistic regression techniques is validated using appropriate diagnostics (e.g. correlation, autocorrelation, heteroscedasticity, etc.). Additionally, we also employ stepwise regression, which automatically determines an appropriate set of predictors. Model quality is evaluated with standard metrics: t-tests, F-tests and R-squared. Alternative models such as support vector machines, random forests, and boosted trees are also tested to evaluate predictive power. A hold-out subset of data is also used to evaluate the models. Historical loans over a more recent period (e.g., 1 quarter) are predicted as test data. The discrepancy between what is predicted and what actually occurred should not cross a certain threshold. The comparison is done with respect to recent Trepp and the bank s loans. To make the analysis even more robust, Opex also performs 10-fold cross-validation on models, where randomly selected subsets of data are considered as hold-out subsets (rather than only the most recent loans). PG 2

3 STEP 2 Each bank loan is scored based on the derived risk model s indicator weights. 2 Risk Model Scoring After a risk model has been developed for loans, each current (and future) loan has a risk score computed from the relationships derived using historical data. From this point on, the current and future loans of the bank are considered. Each individual loan is scored based on the values of indicator variables derived using the Company s loan data and other economic indicators. At the completion of Step 2, each of the bank s loans has an associated score that represents the probability of default. Very few loans relative to the total are expected to be tagged as default historically. An imbalance between the number of current loans and loans in default can cause complications. Model evaluation metrics such as the ROC curve become questionable. For this reason we also validate the results with metrics that are better suited for such imbalanced data sets, such as precision and recall. The maximum log likelihood algorithm might be altered to reflect the imbalance by either using only a subset of loans in the current status, or creating artificial loans in default using known statistical techniques. STEP 3 Stepwise logistic regression is typically used to predict risk. All models are tested on recent holdout data and using 10-fold crossvalidation. 3 Loan Transition Probabilities Scored loans can be classified in one of three states: Current, Delinquent, and Default. This depends on not only the risk score computed in Step 2, but also on the time and the loan s state in the previous quarter. Loans may remain in the same state as the previous quarter, or transition to a different state. The key task in this step is to calculate the transition probabilities for each loan, which determine the likelihood of a loan moving from one state to the other. Consider for example a loan that is being maintained and remains Current due to low computed risk value: Probability of a stable loan state movement from Q1 to Q2 PG 3

4 Loans might remain in the same state due to economic conditions or changes in property value. A loan might remain in Current state for various reasons such as positive economic conditions, stable employment on the borrower s part, or increasing property values. Under such conditions, where LTV is dropping, it is reasonable to expect that the probability of the loan remaining Current will be considerably higher than the chance of a move to Delinquent. The probabilities of the loan moving between states is thus illustrated in the figure above. The aforementioned methodology from Step 1 relies on logistic regression and the weights reflected in the risk score. This is only one possible way to derive these probabilities. Opex also tests other methodologies such as random forests and support vector machines. These approaches give probabilities without a notion of a loan s risk (they correlate indicators to the probability of a default in different ways). STEP 4 Markov chain models are derived that illustrate the movement from state to state for each loan over time. 4 Markov Chain Models Using the logic mentioned above, the different probabilities of a loan being in a given state in a future quarter can be computed. The result of steps 1-3 is the ability to calculate the probability of a loan being in a given state in a given future quarter. This figure illustrates the general process: Illustration of movement between loan states over time PG 4 Here, we see that a loan in Default can never become Current or Delinquent. On the other hand, a loan being Delinquent can remain Delinquent or transition to Current or Default. The models behind this step are known as Markov chains and the required set of full calculations involve matrix multiplication of the underlying probabilities.

5 STEP 5 Expected loss is calculated and all results can be visualized in a dashboard for further insights. 5 Loss Given Default Expected monetary loss of a loan is computed as follows: Expected Loss = (Probability of Default) * (Monetary Amount of a Loss Given Default) The probability of default was derived in the previous step. Loss given default (LGD) is defined as the estimated loss for each loan that defaults. LGD may be estimated as a function of collateral deficiency, which is defined as the difference between the outstanding loan balance and collateral balance at the moment of default. In Step 1, the risk of a loan is evaluated based on indicators such as GDP. Clearly only estimates on future growth of GDP can be made. For this reason, as required by DFAST, three scenarios are considered: Base Case, Adverse Case and Severely Adverse Case. Under each scenario a different GDP growth assumption is made. This in turn yields different risk scores and thus, different expected losses. The results to be leveraged by the end user are available in a dashboard containing the following functionalities: Dashboard Solution Functionality PG 5

6 Expected loss is calculated and all results can be visualized in a dashboard for further insights. In the dashboard above, the Probability of Defaulting module in the lower left corner shows the total loss in a portfolio broken down by probability of default after five quarters in the future. The dashboard also displays the cumulative loss of loans by future quarter ( Total Predicted Possible Portfolio Loss ). The predictive power of models with respect to the different scenarios is depicted in the bottom right corner with a precision-recall curve. The Important Variables module allows the end user to examine the average value of the selected model variables across all loans with a certain default probability. For example, eleven quarters from now, all loans with a default probability of 95% or higher have an average original loan balance of approximately $10 million and the confidence interval between $5 million and $20 million. This interactive dashboard also allows users to select values for a variety of economic indicators (CPI, GDP and Housing shown here) and measure different scenarios to compare to Federal Reserve baseline economic conditions. The radar charts shown below depicts how the changes in different economic indicators impact a loan portfolio measured against the base case scenario. The left panel shows the impact of a drop of 10 percent in an individual economic indicator. The blue line represents the baseline, while the green line indicates that such a drop in unemployment would result in a decrease of the portfolio loss of $175 million (approximately $5.575 billion for the baseline minus $5.4 billion corresponding to the drop in the unemployment). Similarly, if housing expenditures decrease by 10%, the portfolio loss increases by approximately $25 million. The right panel shows similar consequences with respect to individual indicators increasing by 10%. PG 6

7 Process Flow Summary Diagnostic tests confirm if the regression equations are appropriate for use in stress tests. Finally, after risk scores, loan state probabilities and loss given default are calculated, Opex conducts a series of diagnostics tests to confirm that the regression coefficient equations are appropriate for use in stress tests. Following diagnostic testing, Opex performs risk management analytics in accordance with regulatory guidance. The tests include back testing, sensitivity analysis and in and out-of-data sampling. Opex Analytics and its staff of PhDs, data scientists and financial services industry experts has the knowledge and experience to enable a bank to meet its current and future regulatory requirements in an efficient and cost effective manner. PG 7

8 Opex DFAST Leadership Team Diego Klabjan, Ph.D. is a founder of Opex Analytics. He serves as a chief data scientist and technology officer. Diego is a leader in the field of analytics. As a full professor at Northwestern, he is the Founding Director, Master of Science in Analytics. He was also in the first group of people to be recognized as Certified Analytics Professionals (CAP) by INFORMS. Diego is a full professor in Northwestern s Department of Industrial Engineering and Management Sciences. Macario Lullo graduated with a Masters in Analytics (MSiA) from Northwestern University in Cario also earned an MBA from the University of Chicago and a BS in Economics from Northwestern. While in the MSiA program, Cario applied advanced data science concepts to projects in the healthcare, credit scoring and money management industries. Prior to joining Opex Analytics, Cario worked as a trader, manager and salesperson in the financial services industry. He held senior positions in the derivatives divisions of firms including Deutsche Bank and JP Morgan. PG 8