Detecting Abnormal Changes in Credit Default Swap Spread

Transcription

1 Detecting Abnormal Changes in Credit Default Swap Spread Fabio Bertoni Stefano Lugo January 15, 2015 Abstract Using the Credit Market Analysis (CMA) dataset of Credit Default Swaps (CDSs), this paper investigates the power and specification of statistical tests and adjustment methods used to detect Abnormal CDS Spreads Changes (ASCs). We obtain three main results. First, we find that among statistical tests, the sign test is better specified than the signed-rank test and more powerful than both the signed-rank test and the t-test. Second, we find that adjustment method to compute the ASCs that is most used in the literature (i.e., the rating-matched method) often result in statistical tests that are not correctly specified. The misspecification is more relevant for speculative grade firms and after the beginning of the financial crisis. Third, we propose an adjustment method to compute ASCs (the spread-matched method), which, when combined with the sign test, results in a model that is always correctly specified and has in most circumstances a better power than any other correctly specified test and adjustment method. 1 Introduction A Credit Default Swap (CDS) is a derivative instruments allowing a protection buyer to be insured over a predetermined period of time against the default of a reference entity (e.g., a firm, a government) on its liabilities. Until the contract expires, or a credit event occurs, the CDS buyer pays regularly a premium to the CDS seller for being insured. The premium is expressed as a percentage of the notional amount covered by the contract and is referred to as the CDS spread. In case a default occurs, the protection sellers pays to the protection buyer the notional amount insured minus the amount recovered. The first CDS contract was created by JP Morgan in 1995 and, the according to the Bank for International Settlements, in June 2014 the total notional amount of single-name CDS contracts was about USD 11 trillion (BIS, 2014). As the market developed and data were made available from a variety of sources, CDS spreads became increasingly popular among academics, practitioners and regulators as a market measure of credit risk. CDS spreads present a series of advantages over bond yields, which have been traditionally used for this goal. First, whereas bond yields require to make crucial assumptions over the benchmark interest curve to derive credit premiums, CDS spreads are already a direct measure of such premiums (Hull et al., 2004). Second, CDS spreads data refer to new, fixed-maturity contracts issued every day, whereas bond yields data refer to outstanding bonds whose time to maturity naturally evolves over time. Third, previous studies have shown that, compared to bond yields, CDS spreads respond more quickly to changes in credit conditions (e.g., Blanco et al., 2005; Zhu, 2006) and are less affected by liquidity (Longstaff et al., 2005). EMLYON Business School - Department of Economics, Finance and Control. bertoni@em-lyon.com. Utrecht University - School of Economics. s.lugo@uu.nl. 1

2 For all these reasons, CDS spread data are particularly suited for studies investigating the impact of specific events on credit risk. Table 1 presents a selection of the event studies on credit risk using CDS spreads. [Insert Table 1 about here] All of the studies in Table 1 use a matching-portfolio (or diff-in-diff) approach to compute abnormal CDS spread changes (ASC) around the event. The statistical significance of such changes is then addressed using parametric or non-parametric tests on the cross section of the ASCs. The most common matched-portfolio approach used in the literature is a rating-based method, in which the change in the CDS spread of a focal company is adjusted by the change in the CDS spread in a portfolio of firms in the same rating category. Despite its popularity, the rating-based adjustment method present two limitations. First, not all of the firms with outstanding CDS contracts are rated, or rated by the same credit rating agency. Because non-rated firms may be significantly different from rated firms (Faulkender and Petersen, 2006), a rating-based approach might in principle introduce a selection bias in the event study. Second, and most importantly, ratings tend to be sticky (Altman and Rijken, 2004), and to adjust slowly to changes in credit risk (Norden and Weber, 2004). Thus, rating-based matched portfolios could include firms with significantly different levels of default risk from the firm experiencing an event, in particular during turbulent periods when the credit risk is more volatile. In this paper, we propose an alternative adjustment technique (the spread-matched method) to compute ASCs, which uses the pre-event level of CDS spread as to identify the companies in the matched portfolio. Being based on CDS spreads rather than credit ratings, the spread-matched method could be more effective in identifying a matched portfolio of companies with similar credit risk, and thus better estimate the ASC for each event under consideration, resulting in more precise estimates. We use simulation methods to address the size and power of three statistical tests (i.e., t-test, sign test and signed-rank test) in a number of different sub-samples, and compare the relative performance of these tests when ASCs are computed using a spread-matched versus a rating-matched method. We also consider, as a further benchmark, two other approaches used so far in the literature, namely the unadjusted approach (i.e., spread changes around the event with no further adjustment) and the all-spreads approach (i.e., the matched portfolio includes all available CDSs for which the CDS spread is available). We obtain three main findings. First, we show that, regardless of the adjustment method used, the sign test dominates the t-test in terms of Type II errors, and that it dominates the signed-rank test both in terms of Type I and Type II errors for virtually all of the subsamples considered. This result is in line with the findings of Bessembinder et al. (2009) on bonds returns. The Type II errors for the t-test remain remarkably high even in situations where Type II errors for non-parametric tests are virtually null. These results suggest that a sign test should be the primary test to detect abnormal CDS spreads changes. Second, we show that the rating-matched method and the unadjusted method result in sign tests which are misspecified in several of the subsamples considered. The misspecification is more pronounced for speculative grade (SG) firms and after the beginning financial crisis. Third, the spread-matched method results in sign tests that are correctly specified for virtually all of the subsamples considered. Moreover, this approach generally dominates rating-based methods also in terms of the power of statistical tests. For virtually all subsamples, abnormal changes in CDS spreads computed using a spread-matched method result in the most powerful tests among the correctly specified ones. Our paper contributes to the literature addressing the adequacy of statistical tests used to identify abnormal variations in financial data such as stocks short-term returns (Brown and Warner, 1980, 1985) 2

3 and long-term returns (Barber and Lyon, 1997; Lyon et al., 1999), firms operative performance (Barber and Lyon, 1996), and bond returns (Bessembinder et al., 2009). Our work is closely related to that of Andres et al. (2013), who also address event study methodologies using CDS data. Their study however differs from ours in a number of significant ways. First, out study focuses specifically on matching-portfolio models, and tries to identify the best method to compute abnormal spread changes within this class of models, whereas Andres et al. (2013) compare completely different model approaches (including mean-adjusted and factor models) considering only one variant per model. Our choice of focusing on matching-portfolio models has two motivations. First, this class of models is the most parsimonious, because it requires data only in the days surrounding the event, whereas other models such as mean-adjusted models or factor models require data availability during a much longer period. These methods might result in an excessive number of events dropped in a event study. Moreover, the data selection imposed by the use of a factor model might not be random: for example, Hull et al. (2004) report that CDS on investment-grade reference entities tend to be more frequently traded than on speculative ones. Second, as illustrated in Table 1, matching-portfolio models have been dominating among CDSs event studies so far, and it is thus important to address how to correctly implement them. A third substantial difference between the study by Andres et al. (2013) and ours is the dataset used for CDS spreads. Andres et al. (2013) base their analysis on Markit data, while we use the Credit Market Analyses (CMA). These two dataset are the most commonly used in event studies using CDS data, as Table 1 illustrates. However, Mayordomo et al. (2013) have shown that significant discrepancies exist between the two dataset, and have found that CMA data lead the price discovery process, which makes it a better candidate for event study analysis. The rest of this paper proceeds as follows. Section 2 describe the dataset used. In Section 3, the different methods to compute abnormal spread changes, as well as the approach used to address the size and power of resulting statistical tests are described. Our main results are presented in Section 4. In Section 5, we illustrate the results of an event study on ASC following Moody s downgrades based on the different statistical tests and adjustment methods considered in this work. We report additional results about size and power for alternative specifications and samples in Section 6. Section 7 concludes. 2 Data Our source of CDS data is Credit Market Analysis (CMA). The CMA dataset includes daily observations for every firm with at least one recorded quote from a consortium of buy-side and sell-side institutions. For each entity, maturity and seniority, CMA reports average daily CDS spreads. Our dataset spans from January 2003, when CMA started collecting data, to December As is customary for event studies on CDS data, we focus on 5-year maturity senior contracts. The dataset initially contains 2,391,426 daily observations for 1,641 firms located in 52 different countries. The most represented country is the US, with 1,179,404 observations for 786 firms. We focus our primary analysis on US firms and investigate how statistical tests perform on the whole sample in Section 6.1. CMA reports the veracity score of each CDS spread. A veracity score of 1 indicates that actual trades data are used to compute the spread, a score of 2 indicates that buy-side and sell-side commitment-to-trade quotes are used, a score of 3 indicates the use of non-binding quotes. Scores higher than 3 indicates derived, theoretical spreads and are thus discarded for our analysis. In the spirit of Bertoni and Lugo (2014) and Hull et al. (2004), we retain all the observations for with a veracity score of 3 or lower, which are those for which the CDS spread is at least representative of actual bid and offer quotes. The number of observations 3

4 with a veracity score of 3 or lower for US firms is 882,870, 676,159 of which exhibits a veracity score of 1. We address the impact of data quality constraints on our results in Section 6.2. In order to compute the ASC, we need two consecutive data-points. As shown in Table 2, it is possible to compute daily changes in spreads for 748 distinct US firms, for a total of 875,438 potentially usable firm-day events. [Insert Table 2 about here] We hand-collect from Moody s website the rating history for each firm in the dataset. We were able to identify a rating history for 586 of the 748 US firms in our sample, bringing the total number of usable firm-day observations to 681,661, of which: 480,299 (70.5%) are rated investment grade (IG) and 201,362 (29.5%) are rated as SG. 1 In order to address the performance of tests relying on rating-based adjustment methods versus alternative methodologies, we use for our primary analysis only observations where ratings are available. Results obtained re-including unrated firms are presented in Section 6.4. Finally, in order to address whether the relative performance of different tests has changed since the financial crisis, we split our sample in two separate periods: a Pre-Crisis and a Crisis period. Following Mayordomo et al. (2013), we set the beginning of the crisis on August the 1st, Table 2 reports the descriptive statistics for daily changes in CDS spreads for the different samples considered. CDS spreads tend to increase over the whole period, a result of course driven by the crisis years included in the dataset: for rated US firms, the mean daily change in spreads is 0.75 basis points (bps) since August the 1st, 2007, whereas its only 0.03 bps during the period before. Also the volatility differs significantly; the standard deviation of daily changes is bps for the Pre-crisis period and bps since the advent of the crisis. A similar contrast is observed when IG and SG firms are separated; the mean change in CDS spreads is 0.08 bps for US IG firms and 1.09 bps for SG firms; the standard deviation is bps and bps respectively. 3 Methodology 3.1 Assessing size and power of statistical tests To address the size and power of statistical tests used to identify abnormal changes in CDS spreads, we use a simulation-based approach (e.g., Barber and Lyon, 1996, 1997; Bessembinder et al., 2009). Following Bessembinder et al. (2009), we draw 5,000 samples of 200 randomly selected firm-date observations for each of the samples considered. The number of observations for each random sample is consistent with the number of events included in the studies surveyed in Table 1. Consistently with Bessembinder et al. (2009), we focus on Type I errors, which are of particular importance in the typical research setting. A statistical test is correctly specified if the proportion of random samples affected by Type I error (i.e., the test rejects the null hypothesis of no abnormal change in CDS spreads when the latter is true) is not significantly greater than the theoretical value associated with the desired level of statistical confidence. For each test and measure of abnormal CDS spread changes we compute rejection rates using all of the three customary levels of statistical 1 Including non-us firms, the total number of observations of daily changes in spreads where ratings are availalable is 1,232,812, of which: 853,333 (69.22%) are IG and 379,479 (30.8%) are SG. 2 Mayordomo et al. (2013) set the beginning of the crisis using a Bai and Perron (2003) test to identify a structural break in the time series of corporate CDS. 4

5 significance (i.e., 10%, 5% and 1%). Moreover, for each confidence level we compute the upper and lower tails rejection rates; in this way we can identify asymmetries in the rejection rate in the two tails. We define a test to be correctly specified if there is no statistically significant overrejection of the null hypothesis for any of the rejection regions and confidence levels considered. If the tests performed on each of the 5,000 drawn samples are independent and correctly specified, the proportion of samples where the null hypothesis is rejected follows a Bernoulli process, with a mean (µ) equal to the expected rejection rate µ(1 µ) (e.g., 2.5% for each tail with a 5% confidence level), and standard deviation equal to: sd = At the 5% confidence level, the null hypothesis that the test is well specified for a specific rejection region and confidence level can be rejected if the rejection rate is greater than µ sd. The maximum acceptable rejection rates for each tail are thus 5.60%, 2.93% and 0.70% with a confidence level of 10%, 5% and 1% respectively. To address the power of the tests, we impose to a group of randomly selected observations an abnormal positive or negative ASC, and use a two-sided test at the 5% confidence level for the null hypothesis that ASC = 0. The higher the proportion of random samples where the null hypothesis is rejected, the more powerful the statistical test (i.e., the lower the Type II error). Since the power of any test tends to increase with the magnitude of the imposed shock, it is important to use abnormal levels of changes in spreads that are realistic. We decide to use a conservative value of 0.50 bps for ADS for our primary analysis. This value is of the same order of magnitude as what found by Jorion and Zhang (2007) and Jorion and Zhang (2009) in their studies about the ADS following the default of competitors and counterparties (the 1-day ADS in the range of 0.81 bps bps). However, as noted by Bessembinder et al. (2009) in the context of corporate bonds, what constitutes an economically significant abnormal change is likely to depend on the level of risk of the company. We thus use different shocks when performing our analyses on separated samples for IG and SG CDSs. For the IG sample, we keep the ADS at 0.50 bps. For the SG sample, we use a 1.0 bps shock. The effect of the magnitude of the shock imposed and of the number of usable events on the power of statistical tests is addressed in Section Computing Abnormal Spread Changes The CDS spread change (SC) corresponding to an event occurred for a company i in day t can be computed as follows: SC i,t = CDS i,t = CDS i,t CDS i,t 1 (1), where t and t 1 are two consecutive trading days and CDS i,t is the CDS spread of company i in day t. The SC i,t computed in Equation (1) reflects both changes in the idiosyncratic risk of firm i and changes in the general market risk. The goal of researchers is typically to determine the idiosyncratic component of SC i,t, which is normally done using the matching-portfolio method. The first step of this method is to identify a portfolio of companies that are similar to the focal company and give us an indication of how the CDS spread would have changed for the focal company if no event occurred. Once the matched portfolio has been identified, the average CDS spread of the companies in the portfolio (I t ) is computed. The change in the CDS spread of the matched companies ( I t ) represents the value of SC i,t that we would expect if no event occurred (i.e., the systematic component of SC i,t ). As such, the difference between the observed and 5

6 the expected change, as computed as in Equation (2), constitutes the ASC i,t for firm i in day t, as follows: ASC i,t = SC i,t I i,t (2) The choice of how to form the portfolio of matched companies used to compute I t clearly influences the estimate of ASC i,t. In this paper we consider three different adjustment methods (the all spreads method, the rating-matched method and the spread-matched method) illustrated in the remainder of this section together with the unadjusted method, which is based directly on Equation (1) The Unadjusted Method The unadjusted method is based on the assumption that the SC is an unbiased predictor of ASC, which is equivalent to imposing that E [ I Event] = 0. The majority of the studies surveyed in 1 apply this method as a robustness check. The unadjusted method has advantages and disadvantages. On the one hand, it does not require to build a matching portfolio, which makes its calculation easier and somewhat less arbitrary. On the other hand, the unadjusted method relies on the rather strong assumptions that the distribution of events is uncorrelated with changes in systematic risk, and that the unconditional expected change in systematic risk is null. The extent to which these assumptions are valid is ultimately an empirical question The All-Spreads Method One possible way to control for the systematic component of SC i,t is to compare it to the change in spread of the market as a whole (i.e., to an equally-weighted portfolio of all of the available CDS spreads). This approach is used by Bertoni and Lugo (2014) and by Ismailescu and Kazemi (2010). To avoid a potential bias induced by firms entering or exiting the portfolio (Lyon et al., 1999), the index is measured using only the firms whose CDS spread is available both in t and in t 1. We refer to this particular adjustment method to compute ASCs as the all-spreads method. The main advantage of the all-spreads method is that (unlike the rating-matched method discussed in the next section) it does not require any additional information than the CDS spreads. The main drawback is that this method assumes that the credit risk of all of the firms varies in the same way as the credit risk of the companies for which an event occurs The Rating-Matched Method The most common method to build the portfolio of matched companies is based on credit rating. In the rating-matched method an equally-weighted index of all of the firms within the same rating category as firm i is used to compute I. This method has been used in the two seminal event studies by Hull et al. (2004) and Norden and Weber (2004) and has been extensively used in event studies on corporate bonds (e.g., Bessembinder et al., 2009). As for the all-spreads method, in the rating-spread method only firms whose CDS spread is available both in t and t 1 are considered in order to avoid a potential bias (Lyon et al., 1999). Moreover, firms whose rating changes in t 1 or t are further excluded. This approach requires a decision about how rating categories are defined. As illustrated in 1, previous studies use in fact different definitions of rating categories: for example, Jorion and Zhang (2007) use 5 rating categories (AAA/AA; A; BBB; BB; B and 3 In the context of stock returns, for instance, Brown and Warner (1980) show that in several of the samples they consider, unadjusted returns lead to well-specified and powerful tests. 6

7 below), while Jorion and Zhang (2009) use only 2 categories: IG (above BBB) and SG (BBB and below). 4 Since the number of categories can affect the results, we consider two different methods to compute ASC in the rating-matched method. The first method uses 5 categories defined as in Jorion and Zhang (2007); we refer to this as the Rating(5)-matched method. The second approach only divides firms into IG versus SG, and we refer to this approach as the Rating(2)-matched method. The rationale behind using a rating-based portfolio is that firms with a similar level of credit risk are expected to be affected in a similar way by changes in the systematic risk (Hull et al., 2004). However, as anticipated in Section 1, firms in the same rating category can vary substantially in their credit risk, because ratings tend to be sticky and adjust slowly to changes in credit conditions. For this reason, in the next Section we propose an alternative method to adjust for default risk in a matched-portfolio model The Spread-Matched Method Because CDS spread is a measure of credit risk, we suggest the use a matching procedure based on CDS spread as an alternative to credit rating. We refer to this approach as the spread-based method. Specifically, we use as matched portfolio the firm whose pre-event CDS spread is closest to that of the focal firm. 5 4 Results In this Section we report the main results of our analysis. We begin, in Section 4.1, by reporting the analysis on the whole US sample. In the following two sections, we split this sample by time period (Section 4.2) and by credit rating (Section 4.3). Finally, in Section 4.4 we show how our results are affected by the size of the sample and of the ASC. 4.1 Analysis on the Whole Sample Size and Type I Error In Table 3, we report the three statistical tests considered for randomly drawn samples of observations from the dataset of usable US rated firms. As illustrated in Section 3.1, we consider rejection rates for each tail at the 1%, 5% and 10% confidence level, and consider as correctly specified any test and method not resulting in a significant (at the 5% confidence level) over-rejection for any of the six pairs of region and significance level. [Insert Table 3 about here] The only method that does not result in over-rejection regardless of the test used is the spread-matched method. For the t-test, only the All-spreads method results in a misspecified test in one of the six pairs of regions and significance levels. For all of the other methods, the t-test results in rejection rates well below the 4 Event studies on CDS spreads using the rating-matched method also differ in the credit rating agency they rely upon. For example, Hull et al. (2004) use Moody s ratings, while Jorion and Zhang (2007) use S&P ratings. 5 In unreported estimates we use different versions of the spread-matched method, using more than one firm for the matched portfolio, or including in the matched portfolio all companies with a CDS spread within a pre-specified range from the focal company s. The more companies we include in the matched portfolio (or the wider is the acceptable range of CDS spread) the worse the spread-matching gets in terms of Type I and Type II errors. We thus report here the version based on the closest match, for the sake of brevity. 7

8 theoretical expected value, suggesting that, whereas Type I error does not appear to be a serious concern, Type II error might. We will address this concern in the next section. Among non-parametric tests, the sign test appears in general to be better specified than the signed-rank test. All of the the adjustment methods, with the exception of spread-matched method, exhibit rejection rates above the theoretical values in the left tail. For the sign test, only the unadjusted method results in a severely misspecified tests for the negative regions. The all-spreads exhibit instead over-rejections in the positive region, albeit the result is statistically significant only when the 5% confidence level is considered. The sign test appears to be correctly specified when rating-based adjustment methods are used Power and Type II Error We now examine the Type II error for the the different statistical tests and methods. Table 4 reports rejection rates when a shock of 0.5 bps (either positive or negative) is imposed to ASCs computed using different adjustment methods. [Insert Table 4 about here] As expected, the t-test is substantially less powerful than non-parametric tests. The t-test rejects the null hypothesis only between 3.34% and 6.72% of the cases. The sign test is the most powerful. Taking the Rating(5)-matched method as a reference, the percentage of random samples where the null hypothesis is rejected (at the 5% confidence level) using the sign test is on average greater than the signed-rank test by 19% when the shock is negative and by 26% when the shock is positive. If we also take into account that, in the previous section, we have shown that the sign test is the best specified, we may conclude that the sign test dominates the other two statistical tests. Accordingly, we will focus on this test for comparing the different adjustment methods. The all-spreads method is the least powerful method, with a rejection rate around 30% lower than the Rating(5)-matched method. The most powerful method is unadjusted method, exhibiting rejection rates higher than 99% both for positive and negative shocks. However, as shown in the previous section, this method is misspecified when the sign test is used. Among the three correctly specified methods (i.e., the spread-matched and the two rating-matched methods) the two rating-matched methods appear to be slightly more powerful, albeit the differences in rejection rates are small (4% to 6%) and found only for negative shocks. When positive abnormal changes are imposed, the power of the three methods is virtually identical. The results of Table 3 and 4 clearly indicate the sign test as the best approach to identify abnormal CDS spreads changes; as for the methods, rating-based and the spread-based adjustment methods are both well suited when applied to a sample of US rated firms. 4.2 Analysis on Different Sub-Periods In Section 4.1 we have shown how both rating-based and Spread-based methods result in a well specified and powerful Sign test. As discussed in Section 1, rating-matched methods could be less adequate when credit risk changes become more extreme; this is because of the tendency of credit rating agencies to slowly adjust their evaluations. In this Section we thus address the size and power of tests for two period-based set of random samples, one before the advent of the 2007 financial crisis and one after. 8

9 4.2.1 Size and Type I Error The size of tests for the pre-crisis and the crisis periods is presented in Table 5. Panel A reports rejection rates for random samples drawn from observations before the crisis (before August 1, 2007), whereas Panel B reports rejection based for random samples drawn from observations after the crisis (from August 1, 2007). Results for the pre-crisis period are similar to those observed for the full sample. No method results in overrejections when the t-test is used, and the sign test is misspecified only when the unadjusted method or the all-spreads method are used. In the crisis period, both the t-test and the sign test are misspecified also when the Rating(2)-matched method is used, whereas the sign test is correctly specified using the all-spreads. As for the signed-rank test, only the all-spreads and Rating(2)-matched methods are correctly specified in the pre-crisis period, whereas in the crisis period the test is correctly specified only with the spread-matched method. In the all Period-based samples show that the signed-rank test is most affected by overrejection of the null hypothesis for almost all the methods and period considered. Between the t-test and the sign-test, the most preferable test in terms of size depends on the method used and the period under consideration. Comparing the different adjustment methods, only the spread-matched method and the Rating(5)-matched methods result in a t-test and a sign-test that are well specified for both the sub-periods Power and Type II Error Table 6 shows rejection rates when abnormal shocks are imposed on random samples from the pre-crisis period (Panel A) and the crisis period (Panel B). [Insert Table 6 about here] In the pre-crisis period, the sign tests and signed-rank tests appear to be equally powerful, with very high rejection rates. Combined with the spread-matched method, the sign test appears to have a rejection rate higher by about 5%. Rejection rates for the t-test are substantially lower than for the two non-parametric tests, regardless of the method used. Focusing on the methods for the Sign test, none of the correctly specified methods appear to be dominate in terms of power. Again, the all-spreads method is not only misspecified, but also the weakest among those considered. Rejection rates are generally lower in crisis period, as expected because of the increased volatility of spreads. In this period, the sign test (rejection rate of 81%) outperforms the signed-rank (rejection rate of 42%). Again, the t-test is confirmed to be the weakest of the three methods. As done in Section 4.1.2, we thus focus on the sign test in comparing the different adjustment methods. Among the correctly specified methods, the spread-matched method dominates the others in terms of power, especially for positive shocks: the rejection rate is 22% higher than with the Rating(5)-matched method and 39% higher than with the all-spreads method. The only method that outperforms the spreadmatched method is the unadjusted method, which is misspecified, as shown in Table 5. In sum, the sign test combined with the spread-matched method, represents the best choice both in terms of size and power to identify abnormal changes in CDS spreads since the advent of the crisis. 4.3 Analysis on Different Credit Ratings In the spirit of Bessembinder et al. (2009), in this section we address the size and power of tests for random samples drawn for IG and SG firms separately. It is important to underline that the division between the two 9

10 subsamples only affects the selectable events, and not how ASCs are computed. For example, the matchedportfolio for the all-spreads method still includes all US rated firms, regardless of whether it is used for IG or SG events. 6 Section addresses the size of tests, whereas the power is discussed in Section Size and Type I Error Table 7 reports the analysis on over-rejection for all of the methods and tests distinguishing between IG (Panel A) and SG (Panel B). [Insert Table 7 about here] The sign test is correctly specified for both the IG and the SG firms when used with the spread-matched method. With the rating-based methods, the sign test results in overrejection in the negative region for IG firms and in the positive region for SG firms. The signed-rank test is never correctly specified, regardless of the method used. The t-test is correctly specified for both IG and SG only with the spread-matched, the unadjusted and Rating(5)-matched methods. In sum, the two most commonly used methods, unadjusted and the Rating(5)-method, are correctly specified only when used together with a t-test. If non-parametric tests are used, the only suitable option appears to be the spread-matched method Power and Type II Error The results on the analysis about the power of the different tests and methods for IG and SG are reported in Table 8. As described in Section 3, we apply shocks of different magnitude to the sample of IG (Panel A) and SG (Panel B) firms. [Insert Table 8 about here] The sign test clearly dominates the signed-rank test and the t-test in terms of power. Looking at Panel B, the highest rejection rate for the sign test with negative shocks (97.66%) is greater than the signed-rank test s (64.44%), and the difference is even more pronounced when the shock is positive. Again, the t-test is weak, especially for SG firms. Recalling that the sign test is correctly specified for both subsamples only when the spread-matched method is used, it appears evident how this empirical strategy is dominant also in this case. Rejection rates with the spread-matched method are 20% to 25% greater than those obtained with the only other correctly specified method for IG firms (i.e., the all-spreads method), and virtually identical to those of the other methods. For SG firms the spread-matched method, which is the only correctly specified method, dominates the rating-based methods also in terms of power: rejection rates are 5% to 10% greater than with the Rating(5)- matched method and 15% to 25% greater than with the Rating(2)-matched method. In summary, this analysis confirms that the best empirical strategy is the combination of sign test and spread-matched method. 4.4 Different Sample Sizes and Abnormal Spread Changes The results illustrated in the previous sections are consistent in showing that the spread-matched method is superior to other approaches currently used in the literature. In this section, we investigate further how the 6 Otherwise, the all-spreads method and the Rating(2)-matched method would be identical. 10

11 power of the sign test, the signed-ranked test and the t-test vary method varies with the number of events and the magnitude of the shock imposed, when the spread-matched method is used. Table 9 illustrates rejection rates in US firms for all ratings (Panel A), IG (Panel B), and SG (Panel C). To gauge the impact of the number of usable events, we apply the same positive and negative shocks as in previous analysis and report the results obtained with random sample of different sizes (namely: 50, 100 and 250 events). In Panel A, we address the effect of the magnitude of the shocks by increasing the size shock to ±0.7 bps. In Panel B we reduce the size of the shock to ±0.3 bps, because a ±0.5 bps already results in rejection rates close to 100% for the sign test in IG firms. In Panel C we increase the shock to ±1.5 bps, because the power of the tests is still relatively low using ±1.0 bps in SG firms. [Insert Table 9 about here] Results in Table 9 confirm that the sign test is always the most powerful. The t-test remains remarkably weak even when larger shocks and more events are considered. Even with as little as 100 events, the sign test is able to identify a 0.5 bps shock with a rejection rate of 60%, compared to less than 40% for the signed-rank test and less than 4% for the t-test. The power of non-parametric tests increases substantially with the number of events considered. For the full sample and a 0.5 bps shock, the rejection rate for the sign test increases from around 34% when the number of events is 50 to to 61% when the number of events is 100. A similar increase is observed in the samples of IG and SG firms. As for the magnitude of shocks, with 100 events the rejection rate for the sign test increases by 20% with an increase of ASC from 0.5 bps to 0.7 bps. For IG firms, the rejection rates increases by 35% with an increase of ASC from 0.3 bps to 0.5 bps. Finally, for SG firms the rejection rate increases by 22% with an increase of ASC from 1.0 bps to 1.5 bps. To better illustrate how the power of the sign test varies with the number of events and magnitude of the shock, we repeated our simulations to cover a broader range of parameters. We introduce ASC varying from -2.0 bps to +2.0 bps, in increments of 0.1 bps, for random samples ranging between 0 and 250 events, in increments of 50. All of the the simulations are repeated for all US rated firms, IG firms and SG firms. The results are shown in Figure 1. [Insert Figure 1 about here] Figure 1-A displays the power curve when both IG and SG firms are included. With only 50 events, the test is able to identify abnormal changes at least 95% of the time when the shock is bigger than 1.6 bps. With 250 events, the same power is obtained with shocks as small as 0.5 bps. As Figure 1-B and 1-C illustrate, for any given number of events and magnitude of shocks, IG firms and SG firms result in more and less powerful tests than the whole sample, respectively. For IG firms, Figure 1-B shows that the sign test identifies shocks of 0.4 bps more than 95% of the times when 250 events are used. Figure 1-C shows that the same levels of rejection is associated to shocks of at least 1.5 bps for SG firms. With 50 events and a 2.0 bps shock, the rejection rate is 43% in SG firms and 99% in IG firms. In summary, given the range of abnormal daily changes in spreads found by event studies in the literature, we may conclude that the use of a sign test with the spread-matched method should realistically allow to identify an economically relevant effect. 11

12 5 Illustrative example: reaction to downgrades To illustrate how the choice of the statistical test and of the adjustment method affect the empirical analysis based on ADS, we perform an event study using the tests and methods considered in this study. As shown in Table 1, one of the topics most investigated with CDS-based event studies is how rating actions determine significant changes in CDS spreads. Both Hull et al. (2004) and Norden and Weber (2004) address this question mainly (if not exclusively) with a t-test on ADS computed using a rating-based adjustment method. 7 The two works come to different conclusions about the effect of downgrades. Both studies find that the CDS market anticipates the rating action, but only Norden and Weber (2004) find a significant change in CDS spreads around the announcement. In this section we investigate how CDS spreads change on the day Moody s downgrades a firm. To avoid confounding events, we follow Hull et al. (2004) and eliminate all of the downgrades preceded in the previous 90 business days by other rating event (including upgrades and rating revisions). We identify 203 usable observations, 74 of which refer to firms with a pre-event rating of IG and 129 refer to firms with a pre-event rating of SG. For each sample and method, we address the statistical significance of the impact of downgrades using each statistical test. We report the the t-statistic, and the z-statistics for the sign and signed-rank tests in Table [Insert Table 10 about here] Two results emerge from Table 10. First, the null hypothesis is never rejected using a t-test, regardless of the sample and method used. As suggested by our analysis, the t-test is excessively conservative, being characterized by a low Type I error but a very high Type II error. Inferring that downgrades do not have an impact based on results for the t-test could thus be misleading. The second result is that different methods result in different conclusions when non-parametric tests are used. As shown in Panel A, for the pooled sample the sign test rejects the null hypothesis at the customary confidence levels only when the spread-matched method, the unadjusted method and the all-spreads methods are used. Of these three methods, however, the two latter are not well specified. This leaves us with two correctly specified methods, which lead to different conclusions as to whether the null hypothesis is rejected at customary confidence levels. To better understand this apparent inconsistency, we repeat the event study for the two subsamples of IG and SG firms. For IG firms, the only method resulting in a rejection of the null hypothesis with the sign test is the unadjusted method, which is not correctly specified. Both spread and rating-based methods thus lead to the same conclusion that downgrades do not have a a significant impact on the CDS spreads of IG firms. This finding is consistent with the results of Hull et al. (2004). Panel C shows that the difference between the spread-matched and matched methods observed for the pooled samples is driven by SG firms. The null hypothesis is rejected at the 1% confidence level by the spread-matched method, at the 5% confidence level by the Rating(5)-matched method and is not rejected 7 Hull et al. (2004) focus on the rating class of the company before the rating action, while Norden and Weber (2004) change the index on the day of the event to reflect the new rating class of the company. We follow here the approach by Hull et al. (2004). Norden and Weber (2004) also perform a sign test and a signed-rank test; however, they base their inference mainly on the results from the t-test. 8 The z-statistic for the Sign test is computed as Z = S n 2/ n 0.5 where S is the maximum between the number of positive and negative ASCs and n is the sum of the two. The statistic is reported for illustrative purposes; the statistical significance is addressed using the p-value computed as p = 2P r [X S], where X Binomial (n, 0.5). The two approaches are asymptotically equivalent. 12

13 at customary confidence levels for Rating (2)-method. Recalling that the sign test for SG firms has proven to be substantially weaker with rating-based methods than with the sign method (see Table 8), we may conclude that downgrades most likely do have a significant impact on CDS spreads for SG firms, but that this effect is not large enough to be caught by rating-based methods. All in all, it appears that: a) downgrades do have a significant impact on CDS spreads, and; b) this result is mainly driven by SG firms. Studies using a t-test or a rating-matching method are likely to underestimate the statistical relevance of the impact of rating downgrades on CDS spreads, in particular for SG firms. At the same time, studies focusing on an unadjusted method might overestimate the statistical relevance of the impact for IG firms. 6 Additional Results 6.1 Analysis of the Global Sample In this Section, we study the size and power of statistical tests and methods on the global sample, including non-us firms. As in Section 4.3, we split the sample between IG and SG firms reports overrejection rates for IG (Panel A) and SG (Panel B) firms. [Insert Table 11 about here] Compared to the samples including US firms only, statistical tests for IG companies appear to be better specified on average; the t-test is misspecified only when the all-spreads method is used, and the sign test is correctly specified with the spread-matched method (as in the US sample) and with both the ratingmatched methods. The signed-rank test is the worst in terms of size, and is correctly specified only when the spread-matched method is used. For global SG firms, the size of tests is instead similar to what observed in Table 7 in the US sample. For the sign test, both rating-based measures exhibit significant overrejection in the positive region, whereas the unadjusted method overrejects the null hypothesis in the negative region. The signed-rank test is misspecified for all of the methods considered. In the global sample the t-test results in an overrejection of the null hypothesis also for the Rating(5)-matched method (albeit only for the negative region and at the 10% confidence level). In summary, only the spread-matched method results in both the t-test and the sign test to be correctly specified for both IG and SG. The power of tests and methods for the global sample is reported in Table 12. [Insert Table 12 about here] Once again, we find that the sign test dominates both the signed-rank test and the t-test in terms of power, and especially so for SG firms. Accordingly, we focus on the sign test to address differences among methods. For IG firms, rejection rates are very high and comparable for all of the methods excluding the all-spreads method, which exhibits lower rejection rates. Among the correctly specified methods, both spread-matched and the rating-matched methods are suitable approaches for the global IG companies. 9 We replicate the analysis presented in Section 4.1 for the global sample and obtain similar results as those obtained for the US firms. The results, which are not reported here for the sake of brevity, are available upon request. 13

14 For SG firms, the only two correctly specified methods (i.e., the spread-matched and the all-spreads) exhibit high and similar levels of power. The most powerful method is the unadjusted method which is however the worst in terms of Type I errors, as illustrated in Panel B of Table 7. As for rating-based methods, not only they are not correctly specified for SG firms, but they are also substantially weaker than the spread-matched and the all-spread methods. Altogether, the analysis on the global sample confirms that the spread-matched method is the best option to detect abnormal changes in spreads. 6.2 Veracity As discussed in Section 2, we have retained all the observation with a veracity of 3 or better. In this Section we present the results obtained when only observations with a veracity equal to 1 are retained. Results for the size and power of tests are reported in Tables 13 and 14. [Insert Tables 13 and 14 about here] No appreciable differences in terms of size and power are found compared to our primary analysis. The Sign test on ASC computed using either the spread-matched method or the rating-based method remain the best option in terms of Type I and Type II errors. 6.3 Cumulative Abnormal Spread Change In our main analysis we focused on CDS spread changes occurring over one day trading day. However, event studies might concentrate on a longer period before or after the event to take into account that the markets may anticipate the event or adjusting slowly. 10 Thus we address how the different tests and methods perform when a Cumulative ASC (CASC) is considered. We compute CASCs for a [ 1; +1] event window, which is the most commonly used multi-day event window. Results for the size and power of tests are reported in Tables 15 and 16. [Insert Tables 15 and 16 about here] We find that overrejections become more common when tests are performed on CASCs. Only the spread-adjusted method and the all-spreads method result in a correctly specified sign test, and no adjustment method is associated with a correctly specified signed-rank test. Rating-based methods results in overrejections of the null hypothesis for the t-test. Focusing on the sign test, the spread-matched method is more powerful than rating-based methods when the shock is positive, and dominates the all-spreads for both positive and negative shocks. The unadjusted method, which is more powerful than the spread-matched method only method, is strongly misspecified. The spread-matched method thus appears to be the best option also when investigating CASCs. 6.4 Unrated Firms Aside from resulting in lower Type I error and in the lowest Type II error using the sign tests for most of the subsamples considered, the spread-matched method, unlike the rating-matched method, does not require 10 For example, Jorion and Zhang (2007) find that the ASC associated to the intra-industry contagion effect of Chapter 11 filings is actually observed in the day following the event. 14

15 rating information, which in turn entails three advantages. First, simplicity, because it does not require the matching between the CDS dataset and a credit rating dataset. Second, sample size, because the firms with no rating may be included in the sample. Third, no sample selection, because it does not entail the exclusion of firms without a rating. With respect to this last advantage of the spread-matched method, to the extent to which rated and unrated firms are systematically different, the inclusion of observations of unrated firms might affect the results of the analysis. Accordingly, we repeated the analysis on the size and power of tests including in the dataset all observations, regardless of the availability of Moody s credit rating. We compare the performance of spread-matched method with those of the other two methodologies that do not require ratings to be computed (namely, the unadjusted and the all-spreads methods). Results for size and power of the tests are reported in Tables 17 and 18. [Tables 17 and 18 about here] Results for the size of tests are virtually identical to those obtained on rated firms only: only the spreadmatched method is correctly specified for all of the three tests; the all-spreads and the unadjusted methods result also in a misspecified t-test and sign test, respectively. The sign test with the spread-matched method dominates the only other correctly specified method (the all-spreads method) in term of power. The rejection rates are virtually identical to those obtained on rated firms only. 6.5 Relative Abnormal Changes in CDS Spread In this work we have decided to focus on absolute changes in CDS spreads. However, some studies (e.g., Dittmann et al., 2013) have investigated relative changes in CDS spreads (i.e., the change in spread as a proportion of the initial CDS spread). Both absolute and relative changes in spreads are an acceptable unit of measure from a theoretical point of view, and which one is most suited is ultimately depending on the nature of the event under study. We thus replicate our primary analysis on the relative ACS (RACS) computed as the abnormal change in the a logarithm of the CDS spreads. To address the power of tests, we impose a 0.5% shock. Results are reported in Tables 19 and 20. [Insert Tables 19 and 20 about here] Again, we find that only the spread-matched method results in all of the three considered tests to be correctly specified. Moreover, we find that rating-based tests are affected by overrejection also for the t-test. Albeit still dominated by non-parametric tests, the t-test is clearly more powerful with RASC changes than with ASC: the maximum rejection rate is 38.50%, whereas the maximum value obtained with ASC was 6.72%. As for the sign test, the test is correctly specified only with the spread-matched method and the all-spreads method. All in all, the spread-matched method still results in acceptable Type I errors when RASC is considered, but appears to be characterized by an excessive Type II error. 7 Conclusions In recent years CDS spreads have become an important tool for academics, practitioners and policymakers to assess how specific events affect the credit risk of companies. These studies used different statistical tests 15

16 and different adjustment methods to compute ASCs. A first objective of this work is to understand if these tests and adjustment methods are correctly specified, and if they have an acceptable power. Our results suggest that the signed-rank test is often misspecified, leading to high Type I error, and that the t-test is very weak, leading to high Type II error. The sign test is always correctly specified and is more powerful than the other two tests whenever they also happen to be correctly specified. Our results also suggest that the adjustment method to compute the ASCs that is most used in the literature (i.e., the rating-matched method) often results in statistical tests that are not correctly specified. Interestingly, the misspecification of the rating-matched method is more pronounced when used in circumstances in which the credit is more volatile, such as for speculative grade firms and during the financial crisis. This evidence is consistent with one of the fundamental limitations of the rating-matched method, which is its reliance on credit rating to identify companies in the matched portfolio. Because credit ratings are sticky and slow to adjust, they may be poor indicators of the credit risk of a company in turbulent times. To overcome this limitation of the rating-matched method, we suggest a new method to identify the companies in the matched portfolio: the spread-matched method. CDS spreads adjust more rapidly to changes in credit risk, which means that they should be better than credit rating in identifying companies with similar credit risk even when credit risk is volatile. The spread-matched method has three additional advantages over the rating-based method: it is more parsimonious, it allows analyses on larger samples, and does not suffer from the potential sample selection deriving from the exclusion of firms without a credit rating. Our analysis confirms that the combination of the spread-matched method with the sign test results in tests which are virtually always well specified and more powerful than any other well-specified combination of tests and methods. A simple application to the ASC following a credit downgrade illustrates the importance of the choice of the statistical test and matching methods for the analysis. Specifically, we show that the null hypothesis that downgrades have no effect on CDS spreads can be rejected when the spread-matched method is used but not when the rating-matched method is used. This difference is due to the fact that the CDS spread changes especially in SG companies, for which the rating-matched method is substantially weaker than the spread-adjusted method. Accordingly, by using the rating-matched method we would have failed to reject the null hypothesis because of the limitations of this method. We thus strongly recommend the use of the spread-matched method, which would not suffer from such limitations. 16

17 References Altman, E. I., Rijken, H. A., How rating agencies achieve rating stability. Journal of Banking & Finance 28 (11), , recent Research on Credit Ratings. URL Andres, C., Betzer, A., Dourmet, M., Measuring abnormal credit default swap spreads. Working Paper. Bai, J., Perron, P., Computation and analysis of multiple structural change models. Journal of Applied Econometrics 18 (1), URL Barber, B. M., Lyon, J. D., Detecting abnormal operating performance: The empirical power and specification of test statistics. Journal of Financial Economics 41 (3), URL Barber, B. M., Lyon, J. D., Detecting long-run abnormal stock returns: The empirical power and specification of test statistics. Journal of Financial Economics 43 (3), URL Bertoni, F., Lugo, S., The effect of sovereign wealth funds on the credit risk of their portfolio companies. Journal of Corporate Finance 27, Bessembinder, H., Kahle, K. M., Maxwell, W. F., Xu, D., Measuring abnormal bond performance. The Review of Financial Studies 22 (10), pp URL BIS, Otc derivatives statistics at end-june Tech. rep., BIS. Blanco, R., Brennan, S., Marsh, I. W., An empirical analysis of the dynamic relation between investment-grade bonds and credit default swaps. The Journal of Finance 60 (5), pp URL Brown, S. J., Warner, J. B., Measuring security price performance. Journal of Financial Economics 8 (3), URL Brown, S. J., Warner, J. B., Using daily stock returns: The case of event studies. Journal of Financial Economics 14 (1), URL Callen, J. L., Livnat, J., Segal, D., The impact of earnings on the pricing of credit default swaps. The Accounting Review 84 (5), pp URL Dittmann, I., Norden, L., Zhu, G., Expectations of executive risk-taking and preferences: Evidence from ceo stock grants. Working Paper. Faulkender, M., Petersen, M. A., Does the source of capital affect capital structure? Review of Financial Studies 19 (1), URL 17

18 Horvath, B., Huizinga, H. P., Does the european financial stability facility bail out sovereigns or banks? an event study. CEPR Discussion Papers 8661, C.E.P.R. Discussion Papers. URL Hull, J., Predescu, M., White, A., The relationship between credit default swap spreads, bond yields, and credit rating announcements. Journal of Banking & Finance 28 (11), , <ce:title>recent Research on Credit Ratings</ce:title>. URL Ismailescu, I., Kazemi, H., The reaction of emerging market credit default swap spreads to sovereign credit rating changes. Journal of Banking & Finance 34 (12), , <ce:title>international Financial Integration</ce:title>. URL Jorion, P., Zhang, G., Good and bad credit contagion: Evidence from credit default swaps. Journal of Financial Economics 84 (3), URL Jorion, P., Zhang, G., Credit contagion from counterparty risk. The Journal of Finance 64 (5), pp URL Longstaff, F., Mithal, S., Neis, E., Corporate yield spread: Default risk or liquidity? New evidence from the credit-default swap market. Journal of Finance 60 (5), Lyon, J. D., Barber, B. M., Tsai, C.-L., Improved methods for tests of long-run abnormal stock returns. The Journal of Finance 54 (1), pp URL Mayordomo, S., Peña, J. I., Schwartz, E. S., Are all credit default swap databases equal? European Financial Management, n/a n/a. URL Norden, L., Weber, M., Informational efficiency of credit default swap and stock markets: The impact of credit rating announcements. Journal of Banking & Finance 28 (11), , <ce:title>recent Research on Credit Ratings</ce:title>. URL Pop, A., Pop, D., Requiem for market discipline and the specter of {TBTF} in japanese banking. The Quarterly Review of Economics and Finance 49 (4), URL Zhang, G., Zhang, S., Information efficiency of the u.s. credit default swap market: Evidence from earnings surprises. Journal of Financial Stability (0),. URL Zhu, H., An empirical comparison of credit spreads between the bond market and the credit default swap market. Journal of Financial Services Research 29 (3),

19 Figure 1: Power curve for the sign test for (A) random samples; (B) investment grade samples; (C) speculative grade samples 19