Prepayments in depth - part 2: Deeper into the forest

Transcription

1 : Deeper into the forest Anders S. Aalund & Peder C. F. Møller October 12, 2018 Contents 1 Summary 1 2 Pool factor and prepayments - a subtle relation In-sample analysis Out-of-sample analysis Is the market smart? 5 4 What is a random forest? Decision trees Tree bagging Finalizing the random forest Summary Prepayments are typically estimated by closed form expressions where each variable is assumed to have a smooth and monotonous shape. In this analysis we used Machine Learning to investigate if the assumptions in the prepayment expression are consistent with the observations. For instance whether a larger pool f actor always results in a higher prepayment all else being equal. We also test if the market price of the bonds contains information about prepayments in the near future (that is; whether market participants manage to use other information than NP V and pool factor to better predict prepayments or not). We show that the traditional interpretation of the effect of the pool factor with regards to prepayments is not justified for bonds that are in-the-money, but not deep in the money. For bonds deep-in-the money however, we find that prepayment is a monotonically increasing function of the pool factor as normally expected. These findings are important when investing in bond above par-value (but not deep-in-themoney) since the short and medium term return will be affected by prepayments and thus the influence from the pool factor is crucial. In other words, the interpretation of the effect of the pool factor should be done carefully. We also show that adding the market price (three months prior to term) to a model using only NP V and pool factor does in fact improve prepayment prediction, meaning that market participants competently use other parameters than N P V and pool f actor to predict prepayments. Our findings raise further questions that will be answered in the next part of this series. 1

2 2 Pool factor and prepayments - a subtle relation Prepayment models and investors typically assume that prepayments are a monotonically increasing function of the pool factor at a given value of NP V. This feature is also build into rdea s prepayment model. But is this assumption consistent with empirical data? We have tested this using a random forest and the answer is, no! In this section we first make a full in-sample random forest and look at the properties of this model. Secondly the model is tested on out-of-sample data. 2.1 In-sample analysis First we fit the prepayment model and a random forest on the full data set ( ) using only NP V and pool factor (pf) as input. It s clear from figure 1 that the two models are very consistent and that both models fail to capture the big spike in prepayments in 2005, which were a result of the introduction of new loan types. We also notice that both models over estimate prepayments in recent years. Despite the consistency between the two models the random forest is overall the better model which can also be seen in the histogram in figure 2. Figure 1: Average prediction error on the in-sample-data set. Both models use N P V and pool f actor. (Source: rdea) Figure 2: Observed minus estimated in-sample prepayments using the prepayment model and the random forest model. Both methods use NP V and pool factor as variables. The data sets and procedures are identical for the two methods. (Source: rdea) To investigate why the random forest is better at predicting prepayments we investigate how the pool f actor influence prepayments in the random forest. The advantage of having only two variables in the models is that it s possible to illustrate the effect of the pool factor. This is done in figures 3, 4 and 5. Page 2

3 Figure 3: Estimated prepayment as a function of pool factor given an NPV of 115 using the prepayment model and the random forest. Both models calibrated to same data set. (Source: rdea) Figure 4: Estimated prepayment as a function of pool factor given an NPV of 110 using the prepayment model and the random forest. Both models calibrated to same data set. (Source: rdea) Figure 5: Estimated prepayment as a function of pool factor given an NPV of 100 using the prepayment model and the random forest. Both models calibrated to same data set. (Source: rdea) We observe that prepayment is (roughly) an increasing function of the pool f actor when the borrower saving (NP V ) is high in which case the random forest is overall following the classical prepayment model. For lower borrower savings though, the relation is reversed! For the case where there is no or limited saving by prepaying and the price is around par (NP V 100) prepayment is decreasing for pool factors above 0.8 and flat otherwise. Our interpretation of this is that it s a classic case of seasoning (not to be mistaken with Page 3

4 seasonality). Bonds with a pool factor close to 1 often consist of many new loans and people who have prepaid recently. These borrowers (especially the first group) are less likely to prepay their loans for reasons like moving to a new house, getting divorced, dying etc. This means that the structural prepayments typically are lower for newer bonds where there have been few redemptions, buybacks and prepayments and this effect requires some economical incentive before the relations are switched around, i.e. before the burnout effect becomes relevant. For intermediate NP V values the curve is very flat except for very low pool factors. Overall the random forest curves are less smooth than the prepayment function which is a result of the limited data set resulting in some over fitting. If more data was available the random forest method would most likely be able to fit more smooth relations. We however find it promising that the method can detect these more subtle findings. The conclusion is that the impact of the pool factor is a function of the NP V. When the prepayment option is deep-in-the-money the normal interpretation of pool f actor and burnout seems to be valid but for bonds where the embedded prepayment option is in-the-money, but not deep in-the-money, the relation is more subtle. 2.2 Out-of-sample analysis To ensure that the above conclusion is not just an example over over fitting we have tested the model out-of-sample (figure 6). We conclude that the conclusion for the in-sample analysis (figure 2) also holds for the out-of-sample analysis (figure 7). Thus we conclude that the findings above is not a result of over fitting and that the subtle interpretation of the pool f actor relation to prepayment is valid. Figure 6: Set up for out-of-sample test. (Source: rdea) Figure 7: Observed minus estimated out-of-sample prepayments using the prepayment model and the random forest model. Both methods use NPV and pool f actor as variables. The data sets and procedures are identical for the two methods. (Source: rdea) Page 4

5 3 Is the market smart? In the section above we did a qualitative analysis of the pool factor, which was possible since we only had two variables in the model. As we increase the amount of variables in the random forest the price we pay is that the intuition becomes murky or even disappears. Thus, the best way of testing whether any variable have an explanatory power is to do the out-of-sample analysis and see if improves the models ability to predict prepayments. A hypothesis could be that the market price of the bond contains information about the upcoming prepayment. To test this we add the bond price to the random forest and test whether it improves the outof-sample prediction. The results are shown in figures 8 and 9. We notice that the prediction improves and becomes more balanced. The models that only use NP V and pool factor (both prepayment model and random forest) have a tendency to overestimate prepayments in the out-of-sample period whereas the model that also have price as a variable overall are more, but not completely, balanced. Our interpretation of this is that the market price contains information about the prepayments, i.e. the market is able to predict some of the factors not captured by a model only using NP V and pool factor. One reason why the market price contains information about prepayments is that the price factors in the weekly preliminary prepayments that in itself can improve prepayment prediction. Another might be using the previous terms prepayment in predicting the next. The next natural step is to include the preliminary prepayments and/or previous terms prepayment (and see if bond prices still hold predictive power), which will be covered in the next part of this series. Figure 8: Distribution of observed minus estimated preayments using the random forest model. Both models use NP V and pool factor but the red data set also has the price 3 month before term as a variable. The data is from out-of-sample. (Source: rdea) Figure 9: Average of observed minus estimated preayments per term using the random forest model. All models use NP V and pool factor but the red data set also has the price 3 month before term as a variable. The data is from out-of-sample. (Source: rdea) Page 5

6 4 What is a random forest? A random forest model consists of many decision trees created in a clever way, and the prediction of the random forest is the average of the prediction of the decision trees in the forest. 4.1 Decision trees In figure 10 a simple, schematic example of a decision tree can be seen. NP V 100? pp = 0% NP V 105? pf 0.7? NP V 115? pp = 5% pp = 3% pp = 10% pf 0.2? pp = 15% pp = 25% Figure 10: A schematic example of a single decision tree predicting prepayments from the variables N P V and pf. Deep decision trees can have arbitrarily complex dependencies on the variables. tice e.g. that in part of the tree higher values of pf gives higher prepayments and in another part the opposite is true. (Source: rdea) Single decision trees are intuitive to understand, can capture arbitrarily complex behaviour, but are prone to over fitting. For example, given a dataset consisting related values of NP V, pf, and pp it would be possible to make a decision tree that exactly predicted the dataset 1. The tree could simply map each observed pair of (NP V, pf) to the corresponding pp. Such a model would look great when testing on the training set, but would perform poorly when predicting on new data 2. It would be extremely over fitted. The idea behind a random forest is to retain the trees ability to model complex behaviour while eliminating over fitting. 4.2 Tree bagging Bagging (short for bootstrap aggregating ) is a fairly simple, yet powerful technique. When applied to random forests, it consists of creating different datasets from one original dataset and fitting different trees to the different data sets and then taking the prediction average of those trees. For example, given a training set of observations of variables, X = { x 1, x 2,..., x m }, and results, Y = {y 1, y 2,..., y m }, one can create a new dataset of size n by n times selecting a random pair ( x i, y i ) with replacement (meaning that the same i can be used several times). Typically n = m, but n can be both larger or smaller. By doing this T 1 Assuming that there were no observations with exactly identical values of NP V and pf and differing values of pp. 2 This is why we test the predictive power our models on samples not included in the training set. Page 6

7 times and each time fitting the optimal tree to the resulting dataset, one gets an ensemble predictor: pred ensemble ( x) = 1 T T pred t ( x) (1) t=1 4.3 Finalizing the random forest Though the trees generated by bagging differ, they are still a bit too alike. This is because the optimal trees for each subset, tend to be using the same variables to branch on at each level of the tree. A clever way of reducing this problem is to allow the tree to branch only on a random subset of the available features at each branch. Say e.g. that x consists of values for (x a, x b, x c,...x z ); then for the first split, a decision tree might be allowed only to chose the optimal variable from the subset (x b, x f, x j, x k, x o ). It turns out that the combination of tree bagging and selecting random variable subsets into a random forest greatly reduces problems with over fitting and is an extremely powerful method for making non-parametric predictive models that are both robust and able to capture arbitrarily complex behaviour. The disadvantages of a non-parametric method are: n-parametric models that can capture complex behaviour are prone to over fitting (but not random forests, when used correctly) It can be hard to get an intuitive understanding of non-parametric models, but it can be done (as seen in section 2.1) The advantage of using a non-parametric model is two-fold: The analyst does not need to figuring out what functional form is able to fit the data - saving analyst time The analyst does not need to figuring out what functional form is able to fit the data - preventing analyst prejudges from entering into the model and allowing data to speak for itself Using the random forest technique correctly one is able to quickly and without prejudges build a robust model that can capture arbitrarily complex behaviour, yet does not over fit the data. This is perfect when one has a large number of variables that one suspects might be used for modelling, but to an unknown extent and in an unknown way. This exactly what we are exploiting in this article. Page 7

8 DISCLAIMER rdea Markets is the commercial name for rdeas international capital markets operation. The information provided herein is intended for background information only and for the sole use of the intended recipient. The views and other information provided herein are the current views of rdea Markets as of the date of this document and are subject to change without notice. This notice is not an exhaustive description of the described product or the risks related to it, and it should not be relied on as such, nor is it a substitute for the judgement of the recipient. The information provided herein is not intended to constitute and does not constitute investment advice nor is the information intended as an offer or solicitation for the purchase or sale of any financial instrument. The information contained herein has no regard to the specific investment objectives, the financial situation or particular needs of any particular recipient. Relevant and specific professional advice should always be obtained before making any investment or credit decision. It is important to note that past performance is not indicative of future results. rdea Markets is not and does not purport to be an adviser as to legal, taxation, accounting or regulatory matters in any jurisdiction. This document may not be reproduced, distributed or published for any purpose without the prior written consent from rdea Markets. Page 8