Bridging the Gap of Missing Company Financials to Estimate Credit Risk

Transcription

1 Imputation of Missing Company Financial Ratios Bridging the Gap of Missing Company Financials to Estimate Credit Risk Overview One of the biggest challenges faced by analysts assessing credit risk of a large portfolio of counterparties is the lack or incompleteness of company financials. This issue is particularly common in the case of private companies, because they are usually not subject to the same strict accounting standards as their public counterparts and thus may not report some common financials; notable in this sense is the omission of cash flow items, that are not required from small and/or medium private companies in some countries [1-2], unless the company follows the International Financial Reporting Standards ( IFRS ) reporting standard [3]. However, even public entities may sometimes recognize and report specific items at different times, or omit them altogether, if they follow different accounting standards; for example, in the Income Statement reported under IFRS, extraordinary items are prohibited, while they are defined and may be reported (even if infrequently) under US Generally Accepted Accounting Principles ( US GAAP ); other cases include the way leases, financial liabilities and equity may be differently classified under IFRS and under US GAAP. [4] Missing financials can hinder proper risk assessment by the analyst, for example by inhibiting generation of outputs from a quantitative model, that uses those items as statistically significant variables for discriminating good from bad companies. This also affects the coverage of any prescored database that collects outputs (scores and/or probability of defaults) calculated from a credit risk model using a database of company financials such as S&P Capital IQ. Derivation of missing items from other financials; it is the most accurate, but it can be applied in very limited instances, e.g. when there is a clear and well defined relationship between missing items and other financials that the company is reporting. Using proxy items; it can be considered only when suitable proxies are available, and it is affected by how close the proxy is to the actual item. For example, using EBITDA as a proxy for EBIT. Replacement by industry / country averages; it is probably the most common approach used by financial practitioners, but does not take into account the actual differences and characteristics of each company. Advanced mathematical techniques; they are highly accurate approaches assuming there is a large enough set of companies with complete financials to be able to calibrate a robust statistical relationship between them. S&P Capital IQ has developed an innovative mathematical framework that derives missing financial inputs for its credit risk models, based on actual financials available and reported by each company. This advanced imputation framework is based on a statistical relationship established by looking at over 10 years of complete financials of several thousand companies across the globe and optimized by mathematical techniques, which ultimately ensures the estimates are as close as possible to the actual values as if they were reported by the companies themselves. S&P Capital IQ s approach is fully automated and generates the most probable value for each missing financial input for its credit risk models, thus allowing to estimate the model output 1. The methodology is very flexible, because it accommodates any number of missing financial inputs, provided the user specifies at least one input (usually, a size-related variable). This paper describes how S&P Capital IQ developed its imputation framework, the methodology scope and its performance, and how it can add value to users of S&P Capital IQ s credit risk models. 1 In addition to this, the imputation is able to calculate an appropriate confidence interval, thus giving clients a measure of the potential variability of the estimate provided. This confidence interval is currently not made available to clients. 1 1

2 Many Alternatives While developing an advanced imputation framework for missing financials, several alternatives were considered: Gaussian Copula: determines the dependence relationship by joining normal marginal distributions together to form a joint distribution, which is one of the most common approaches used in the industry. Gaussian Mixture Model (GMM): this is also commonly used for imputation purposes. The approach yields a multivariate distribution that consists of a mixture of one or more multivariate Gaussian distribution components that are linearly added together. Gaussian Mixture Fit: this is a simplified version of the GMM, where only two components exist in the mixture model for example, for defaulters and non-defaulters. K-Nearest Neighbors (KNN): this method identifies a subset of companies that look as close as possible to the company with missing financials. o KNN unconditional mean fills missing values of a company with the means of its neighbors o KNN conditional mean assumes a company s missing financial is linearly dependent on its non-missing financial variables, and this implied relationship could be captured from its nearest neighbors in the same industry. Table 1: Advantages and Limitations of Different Methodologies Methodology Group Gaussian Methods Advantages Limitations Gaussian Copula Gaussian Mixture Gaussian Mixture Fit 1. Popular in high-dimensional dataset 2. Easily model a wide range of distributions 3. Estimate marginal and copula separately 1. Fails to capture extreme values 2. Sensitive to very heterogeneous sets of data, like those typical of private companies K-Nearest Neighbors KNN unconditional mean KNN Conditional mean 1. Simple and easy to learn 2. Robust to noisy training data 3. Effective if training dataset is large [5] 1. Computation Complexity 2. Requires a large dataset to yield good results Source: S&P Capital IQ as of March 31, 2015 Each method, in turn, was calibrated to align as much as possible to the segmentation of S&P Capital IQ credit risk models, partitioning the training dataset into regional and/or industry sub-models. Ultimately, each alternative was rigorously tested and reviewed by looking at specific performance measures, tailored to the type of model considered (whether a PD model or a scoring model), to ensure the overall robustness and statistical soundness as follows: Closeness of imputed values to actual values calculated when all financial inputs are available. Model performance with imputed values (discriminatory power for PD models and Ratings agreement for scoring models). Stability of model performance with increasing number of imputed values. Ease of implementation and intuitiveness of selected approach for different company types. 2 2

3 S&P Capital IQ s Imputation Framework S&P Capital IQ s imputation framework implements advanced mathematical techniques to estimate missing model inputs for all credit risk models (PD models, such as PD Model Fundamentals and scoring models, such as Credit Model), and focuses specifically on financial ratios, because the results are less affected by currency conversion effects (ratios are currency-independent), more statistically robust (easy to calibrate), and less time-intensive than estimating missing raw financial items. S&P Capital IQ s imputation mechanism can accommodate any number of missing ratios used by our models, provided the user specifies at least one common financial, (such as Total Revenue for PD Model Fundamentals Private Corporates). In order to calibrate and test each method, the following steps were adopted: Take only companies with a complete set of financial inputs; Randomly remove a subset of financials to simulate the case of missing inputs; Impute, e.g. estimate, the missing values and compare them to the actual values; Use the imputed values as inputs into our credit risk models and assess the overall model performance. To ensure the robustness of the imputation mechanisms, we removed increasing percentages of financials across all companies (missing at random), or removed increasing number of inputs from each company (missing by variable number). Methodology Selection Criteria To select the best imputation approach for S&P Capital IQ Credit Risk Models, PD Model Fundamentals and Credit Model, among all possible alternatives mentioned above, we looked at two dimensions: how well the imputed financial ratio approximates the actual value; we used the mean squared error (MSE) as the primary metric; how does the model perform with the imputed values: to this purpose, we looked at the ability of PD Model Fundamentals to distinguish healthy from risky companies (as measured by the receiver operating characteristic, ROC, and average log-likelihood 2 comparison) and for Credit Model at the agreement between scores obtained with imputed ratios and with actual ratios. In the following tables 3, we show the performance of alternatives, such as Gaussian Copula and KNN by industry, and discuss our selection rational. For PD Model Fundamentals Public Corporates, the Gaussian Copula approach was chosen, because public corporates exhibit a reasonable degree of homogeneity in terms of financial ratio ranges and appear reasonably well modelled within a normality assumption. Yet, a distinction was made by region and industry, calibrating the copula based on two regions (NA and non-na) and 21 industry groups, to better align to the model segmentation. Consequently, as shown in Table 2, Gaussian Copula has a smaller (better) MSE compared to KNN method, particularly in the North America region. Overall, the model was calibrated using more than 27,000 observations, from companies with a complete set of financial inputs. 2 Average Log-likelihood test is used to understand if all the parameters together, along with imputed financial variables, are useful to estimate the dependent variable, i.e. probability of default in PD Model Fundamentals. 3 S&P Capital IQ has developed and tested this approach for non-financial corporations. All performance analysis in this paper is with respect to nonfinancial corporations. 3 3

4 Table 2: Model Performance MSE 4 Comparison PD Model Public Gaussian Copula by Industry KNN by Industry Missing Schemes NA Non-NA NA Non-NA 10% missing % missing % missing Variable Missing Variables Missing Variables Missing Variables Missing Note: selected framework is highlighted in bold. Source: S&P Capital IQ as of March 31, 2015 For PD Model Fundamentals Private Corporates, the KNN approach was chosen instead. Indeed, for private companies, it is quite evident that the Gaussian Copula approach fails to perform appropriately, as shown by the quick deterioration of the model performance when additional missing ratios need to be estimated. For example, under the 50% missing scheme, the ROC of the Probability of Default model using imputed ratios from Gaussian Copula has fallen by more than 3%, compared to ROC with no missing variables (Table 3). This is quite intuitive because private companies have very heterogeneous operating profiles, and span a much larger universe of sizes in the S&P Capital IQ database, going from the small and medium enterprises (>$5m Revenues) up to the large private corporations (several billion dollars in Revenues) and follow less strict accounting and reporting standards. In contrast, the KNN approach proves more resilient and confirms a posteriori the need for a different approach for private companies. Table 3: Model Performance ROC and Average Log-likelihood 7 Comparison PD Model Private Gaussian Copula by Industry KNN by Industry Missing Schemes ROC Average Loglikelihood ROC Average Loglikelihood No missing value % missing % missing % missing Variable Missing Variables Missing Variables Missing Variables Missing Note: selected framework is highlighted in bold. Source: S&P Capital IQ as of March 31, 2015 For scoring models, such as Credit Model 2.6 ( CM 2.6 ), we find that the KNN approach performs better, in line with what we see for PD Model Fundamentals Private Corporates. This is consistent with the fact that CM 2.6 does not score public and private companies using separate submodels, and the universe of private companies that CM 2.6 covers is much larger than that of public companies; as already found with PD Model Fundamentals Private Corporates, the heterogeneous characteristics of private companies are appropriately captured by the KNN method. For scoring models, we evaluated the agreement between scores obtained with imputed ratios or with actual ratios. Taking CM 2.6 NA sub-model as an example, the KNN approach demonstrates strong performance compared to Gaussian Copula (Table 4), and we observe similar stability of performance across other sub-models. 4 The smaller the MSE is, the better the model performs. 5 Missing at random (specify a global missing percentages from 10% - 50%) 6 Missing by variables (e.g. 1 Variable Missing means each record will miss 1 variable out of a total of selected variables in PD Model or CreditModel, depending on industry). 7 The higher the ROC (or average log-likelihood) is, the better the model performs. 4 4

5 Table 4: Model Performance Score Agreement Comparison (CM2.6 scores with imputed financials vs CM2.6 scores with actual financials) CM 2.6 North American Gaussian Copula by Industry KNN by Industry Missing Schemes Exact Within 1 Match 8 Within 2 Exact Match Within 1 Within 2 10% missing 86.3% 96.6% 98.4% 91.4% 98.1% 99.2% 30% missing 64.9% 89.0% 94.7% 75.0% 93.0% 96.9% 50% missing 48.1% 80.2% 90.3% 58.7% 85.9% 93.3% 1 Variable Missing 85.1% 96.8% 98.5% 91.4% 98.3% 99.3% 3 Variables Missing 63.7% 89.5% 95.1% 75.0% 93.9% 97.4% 5 Variables Missing 47.3% 80.9% 90.8% 58.9% 87.1% 94.2% 7 Variables Missing 33.8% 70.4% 85.1% 42.2% 76.5% 88.6% Note: selected framework is highlighted in bold. Source: S&P Capital IQ as of March 31, 2015 In addition, we compared the CM 2.6 scores with imputed financials vs the actual S&P Ratings (Table 5). Taking CM 2.6 NA sub-model as an example, the rating agreement remains at a good level for increasing missing financials, and the limited deterioration in performance compared to the baseline case (no missing financials) confirms the robustness of the imputation approach; similar performance stability is observed across other sub-models. Table 5: Model Performance Score Agreement Comparison (CM2.6 scores with imputed financials vs actual S&P Ratings) CM 2.6 North American Gaussian Copula by Industry KNN by Industry Missing Schemes Compared Pairs Exact Match Within 1 Within 2 Exact Match Within 1 Within 2 Baseline CM2.6 vs. S&P Ratings 24.8% 64.3% 84.3% 24.8% 64.3% 84.3% 10% missing CM2.6 Imputation vs. S&P Ratings 23.6% 61.8% 81.9% 24.4% 63.4% 83.6% 20% missing CM2.6 Imputation vs. S&P Ratings 22.9% 59.6% 79.6% 24.2% 62.6% 82.6% 30% missing CM2.6 Imputation vs. S&P Ratings 21.9% 57.2% 77.0% 23.6% 61.2% 81.3% 40% missing CM2.6 Imputation vs. S&P Ratings 20.6% 54.7% 74.6% 22.9% 59.3% 79.6% 50% missing CM2.6 Imputation vs. S&P Ratings 19.4% 51.8% 71.7% 21.5% 56.9% 77.1% Note: selected framework is highlighted in bold. Source: S&P Capital IQ as of March 31, 2015 In summary, S&P Capital IQ imputation framework for public and private companies yields good estimates for missing inputs ensuring a robust model performance in terms of discriminatory power or ratings agreement, even at very high levels of missing financials, when the uncertainty of estimated values increases. In the next section, an actual case study highlights the benefits offered by S&P Capital IQ advanced imputation framework. 8 The higher the match percentage is, the better the model performs. 5 5

6 Case Study SeraCare Life Sciences, Inc. SeraCare Life Sciences, Inc., a private life sciences tools and services firm, provides products and services to facilitate the discovery, development, and production of human and animal diagnostics, and therapeutics. The company sells its products and services through direct sales force and independent distributors primarily in the United States, Europe, and Asia. It was founded in 1984 and is headquartered in Milford, Massachusetts. From 2002 to mid-2005, SeraCare s probability of default (PD) was around 0.2%, indicating a low risk of default. However, the company stopped releasing its full financial statement since mid-2005, leaving investors and other counterparties in complete darkness, unable to assess the company s credit-worthiness. Their worst nightmare became reality on March 23, 2006, when SeraCare filed for bankruptcy (Chapter 11) and was pushed into reorganization on May 17, Thanks to our imputation methodology, the company s key inputs to S&P Capital IQ s PD Model Fundamentals can be appropriately estimated and the credit risk profile is then quickly evaluated, even when companies submit incomplete information. Based on the only financials available, Total Revenue and Net Income (Sep. 30, 2005), our imputation approach first identifies 50 companies with the closest revenue size and net margins in the pharmaceuticals industry, and then estimates the linear relationship between these two available factors and the missing factors, thus generating estimates to all missing inputs needed to generate a probability of default. As shown in Table 6, the imputed credit factors for SeraCare on 9/30/2005 are much worse compared to the prior quarter (6/30/2005), due to the actual low profit and high leverage. Table 6: Key Credit Factor Comparison SeraCare Life Sciences, Inc. 6/30/2005 As Reported 9/30/2005 9/30/2006 Point Estimate [C.I.] 2 As Reported Total Equity / Total Assets [-0.45, 0.08] 0.58 Net Income / Total Liabilities [-4.52, -0.26] Current Liabilities / Net Worth [4.83, ] 0.57 Return on Net Capital Proxy [-1.02, -1.02] Cash / Total Assets [0.01, 0.07] 0.19 Total Revenues ($ in Millions) $52.59 $ $46.71 Net Income / Revenues PPE / Total Assets [0.11, 0.34] 0.07 Probability of Default (PD) 0.20% 27.5% [23.0%, 35.6%] 21.1% 1 Definition: Net Income / (Total Liabilities + Total Equity Cash & ST Invest) 2 C.I. = Confidence interval at 95% 3 Reported by the company Source: S&P Capital IQ as of March 19, 2015 In fact, the PD on 9/30/2005, based on imputed credit factors, is around 27.5% with a 95% confidence interval between 23.0% and 35.6%, signaling potentially serious credit issues for this company.9 Indeed, a few months later, the company filed for bankruptcy. Interestingly, the PD estimated by S&P Capital IQ s imputation mechanism is in line with the actual PD that could be generated when complete financials became again available on Sep. 30, As shown in Figure 1, the dramatic increase in PD from 0.2% (6/30/2005) to more than 27.5% (9/30/2005) would have already highlighted the deterioration in SeraCare s creditworthiness even with little financial information, six months before bankruptcy, thus allowing credit risk managers to enact a different plan. 9 The confidence interval is not currently made available to clients. 6 6

7 Figure 1: Filling In the Gap Prior to Bankruptcy SeraCare Life Sciences, Inc. Imputation of Missing Company Financial Ratios Source: S&P Capital IQ as of March 19, 2015 Risk analysts often face the challenge of incomplete financials information reported by public and private companies, especially in emerging markets. This renders the risk assessment particularly difficult, especially when the analyst relies on quantitative models that need those financials to generate meaningful outputs. S&P Capital IQ has developed a unique imputation framework that provides financial estimates for missing inputs and enables our models to generate risk assessments that can help our models users discriminate the good from the bad companies, and/or rank them in a similar way to the S&P Ratings approach. This enables analysts to generate a credit assessment for companies with missing financials and also to study historical (estimated) trends of creditworthiness for companies that previously could not be assessed due to lack of data. In addition, S&P Capital IQ s imputation framework provides users with the opportunity to obtain estimates and implied projections of missing financials, by running scenarios on the fly and specifying a subset of pre-defined model inputs to understand the potential implications on other missing financials and the impact on the final assessment of credit-worthiness. Thanks to this robust imputation framework, S&P Capital IQ s prescored universe can now be extended to any company with at least one model input. 7 7

8 References [1] For Finland: Doing Business in Finland, UHY (November 2013, from [2] For India: [3] For a review of permitted/required use of IFRS standards in 150+ countries, worldwide, see: (Deloitte). [4] IFRS and US GAAP: similarities and differences, PwC (October 2014). [5] Survey of Nearest Neighbor Techniques, International Journal of Computer Science and Information Security, Vol. 8, No. 2,

9 Copyright 2015 by Standard & Poor s Financial Services LLC, a subsidiary of McGraw Hill Financial, Inc. All rights reserved. No content (including ratings, credit-related analyses and data, valuations, model, software or other application or output therefrom) or any part thereof (Content) may be modified, reverse engineered, reproduced or distributed in any form by any means, or stored in a database or retrieval system, without the prior written permission of Standard & Poor s Financial Services LLC or its affiliates (collectively, S&P). The Content shall not be used for any unlawful or unauthorized purposes. S&P and any third-party providers, as well as their directors, officers, shareholders, employees or agents (collectively S&P Parties) do not guarantee the accuracy, completeness, timeliness or availability of the Content. S&P Parties are not responsible for any errors or omissions (negligent or otherwise), regardless of the cause, for the results obtained from the use of the Content, or for the security or maintenance of any data input by the user. The Content is provided on an as is basis. S&P PARTIES DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE OR USE, FREEDOM FROM BUGS, SOFTWARE ERRORS OR DEFECTS, THAT THE CONTENT S FUNCTIONING WILL BE UNINTERRUPTED OR THAT THE CONTENT WILL OPERATE WITH ANY SOFTWARE OR HARDWARE CONFIGURATION. In no event shall S&P Parties be liable to any party for any direct, indirect, incidental, exemplary, compensatory, punitive, special or consequential damages, costs, expenses, legal fees, or losses (including, without limitation, lost income or lost profits and opportunity costs or losses caused by negligence) in connection with any use of the Content even if advised of the possibility of such damages. Credit-related and other analyses, including ratings, and statements in the Content are statements of opinion as of the date they are expressed and not statements of fact or recommendations to purchase, hold, or sell any securities or to make any investment decisions. S&P assumes no obligation to update the Content following publication in any form or format. The Content should not be relied on and is not a substitute for the skill, judgment and experience of the user, its management, employees, advisors and/or clients when making investment and other business decisions. S&P s opinions and analyses do not address the suitability of any security. S&P does not act as a fiduciary or an investment advisor except where registered as such. While S&P has obtained information from sources it believes to be reliable, S&P does not perform an audit and undertakes no duty of due diligence or independent verification of any information it receives. S&P keeps certain activities of its business units separate from each other in order to preserve the independence and objectivity of their respective activities. As a result, certain business units of S&P may have information that is not available to other S&P business units. S&P has established policies and procedures to maintain the confidentiality of certain non-public information received in connection with each analytical process. S&P provides a wide range of services to, or relating to, many organizations, including issuers or underwriters of securities or obligors, investment advisers, broker-dealers, investment banks, other financial institutions and financial intermediaries. As a result, S&P may receive fees or other economic benefits from organizations whose securities or services it may recommend, analyze, rate, include in model portfolios, evaluate, price or otherwise address. STANDARD & POOR S, S&P, CREDITSTATS DIRECT, RATINGSDIRECT, and GICS are registered trademarks of Standard & Poor s Financial Services LLC. S&P CAPITAL IQ and CREDITMODEL are trademarks of Standard & Poor s Financial Services LLC. MICROSOFT, WORD, POWERPOINT, and EXCEL are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other product or service names may be the property of their respective owners. Americas EMEA APAC (0) credit_analytics@spcapitaliq.com 9 9