Backtesting of a Monte Carlo Value at Risk simulation based on EWMA volatility forecasting and Cholesky decomposition of asset correlations

Transcription

1 UNIVERSIDADE FEDERAL DO RIO DE JANEIRO INSTITUTO DE ECONOMIA MBA EM FINANÇAS E GESTÃO DE RISCO TRABALHO DE CONCLUSÃO DE CURSO Backtesting of a Monte Carlo Value at Risk simulation based on EWMA volatility forecasting and Cholesky decomposition of asset correlations Author: Jascha Andri Forster UFRJ Id. Number: Supervisor: Prof. Dr. Manuel Alcino Ribeiro da Fonseca May, 2015

2 UNIVERSIDADE FEDERAL DO RIO DE JANEIRO INSTITUTO DE ECONOMIA MBA EM FINANÇAS E GESTÃO DE RISCO TRABALHO DE CONCLUSÃO DE CURSO Backtesting of a Monte Carlo Value at Risk simulation based on EWMA volatility forecasting and Cholesky decomposition of asset correlations Author: Jascha Andri Forster UFRJ Id. Number: Supervisor: Prof. Dr. Manuel Alcino Ribeiro da Fonseca May, 2015

3 The views expressed or implied in this document are solely those of the author and do not necessarily reflect the views of the Instituto de Economia, Universidade Federal do Rio de Janeiro.

4 Acknowledgements I would like to express my gratitude to my supervisor Prof. Dr. Manuel Alcino Ribeiro da Fonseca for the useful comments and remarks that helped me write this monograph. In addition, I would like to thank Prof. Dr. Wilson Calmon Almeida dos Santos for his support. My knowledge in statistics and time series analysis improved a lot with his lectures. I would also like to thank all other professors of the MBA who helped me acquire the knowledge and mathematical skills to complete this work. Furthermore, I would like to thank Jürg Forster for his helpful comments on this monograph. Finally yet importantly, I would like to thank Melina Rutishauser for her support.

5 Contents CONTENTS... 5 INDEX OF FIGURES AND TABLES... 6 ABSTRACT... 7 INTRODUCTION CONCEPTS IN RISK MANAGEMENT... 9 HISTORICAL VIEW ON RISK MANAGEMENT... 9 VALUE AT RISK... 9 Definition... 9 Criticism of VaR SIMULATION TECHNIQUES TO ESTIMATE VAR Variance-Covariance Simulation (parametric) Historical Simulation (non-parametric) Monte Carlo Simulation (non-parametric) CORRELATIONS AMONG ASSETS VOLATILITY FORECASTING EWMA GARCH (1,1) BACKTESTING Kupiec POF-test (unconditional) Kupiec TUFF-test (unconditional) Haas Mixed-Kupiec test (conditional) METHODS: EMPIRICAL VAR MODELLING AND SUBSEQUENT BACKTESTING...17 DATA GATHERING AND PROCESSING EXTRACTION OF PARAMETERS FOR SIMULATIONS Correlation: Cholesky decomposition Volatility Forecast: Exponentially Weighted Moving Average MONTE-CARLO SIMULATION ROLLING ANALYSIS BACKTESTING RESULTS...20 DESCRIPTIVE STATISTICS CORRELATION ANALYSIS MONTE CARLO SIMULATION CHOLESKY DECOMPOSITION OF THE CORRELATION MATRIX VOLATILITY FORECAST USING EWMA FEASIBILITY OF THE SIMULATION OUTCOME ROLLING ANALYSIS BACKTESTING OUTCOMES DISCUSSION...25 INTERPRETATION OF THE RESULTS POSSIBLE ADJUSTMENTS OF THE MODEL GENERALIZATION IS IT USEFUL AT ALL TO KNOW THE VAR? CONCLUSION...28 REFERENCES...29 APPENDIX...30

6 Index of Figures and Tables FIGURE 1: ILLUSTRATION OF DIFFERENT VAR CONFIDENCE LEVELS, BASED ON A NORMAL DISTRIBUTION WITH µ = 0 AND Σ = FIGURE 2: A DECAY FACTOR OF 0.94 WAS USED FOR THE EXPONENTIALLY WEIGHTED MOVING AVERAGE VOLATILITY FORECAST. THIS GRAPH SHOWS THE WEIGHTS THAT HAVE BEEN ASSIGNED TO HISTORICAL RETURNS TO FORECAST VOLATILITY OF SEPT. 1, THE SPACES IN BETWEEN THE BARS ARE DUE TO THE WEEKEND DAYS WHICH DID NOT RECEIVE ANY WEIGHT FIGURE 3: CUMULATIVE DISTRIBUTION FUNCTION OF THE SIMULATED PORTFOLIO RETURNS FOR SEPT. 1, THE RETURNS RESEMBLE THE GAUSSIAN DISTRIBUTION FIGURE 4: HISTORICAL RETURNS OF THE PORTFOLIO (BLACK), WITH VAR ESTIMATES AT 95% (GREEN), 99% (BLUE) AND 99,9% (RED) CONFIDENCE LEVELS. THE PERIODS THAT WERE BACKTESTED ARE DENOTED WITH P1, P2, P3 AND P4. AN EXCEPTION OCCURS WHEN THE HISTORICAL RETURN FALLS BELOW THE PROJECTED VAR TABLE 1: MEAN, STANDARD DEVIATION AND VARIANCE OF THE LOG-RETURNS ACROSS THE WHOLE SAMPLE ( ) TABLE 2: CORRELATION OF LOG-RETURNS ACROSS THE WHOLE SAMPLE ( ). THE ENTIRE MATRIX CAN BE FOUND IN APPENDIX TABLE 3: CHOLESKY DECOMPOSITION OF THE MATRIX OF CORRELATION COEFFICIENTS ON SEPT. 1, THE ENTIRE MATRIX CAN BE FOUND IN APPENDIX TABLE 4: CORRELATION MATRIX OF THE LOG-RETURNS THAT WERE GENERATED BY THE MONTE CARLO SIMULATION ON SEPT 1, WHILE THE INITIAL SIMULATION GENERATES UNCORRELATED RETURNS, IT CAN BE SEEN THAT THE CORRELATION AMONG THE ASSETS HAS BEEN CORRECTLY REINTRODUCED BY THE CHOLESKY DECOMPOSITION. THE ENTIRE MATRIX CAN BE FOUND IN APPENDIX TABLE 5: BACKTESTING RESULTS: ALL BACKTESTS WERE PERFORMED AT P=0.05. VALUES IN THE FIRST COLUMN REPRESENT THE VAR CONFIDENCE LEVELS Page 6

7 Abstract In this monograph, a Monte Carlo based model is developed to estimate Value at Risk (VaR). Based on exponentially weighted moving averages to forecast volatility, and Cholesky decomposition to adjust for correlations among assets, the model estimates returns of a portfolio of exchange traded funds (ETFs). By backtesting the proposed model over a set of timeframes, the author shows that this method is only suitable to estimate VaR in calm markets as it is unable to make reliable estimates in turbulent times. Page 7

8 Introduction Value at Risk (VaR) is one of the most popular risk measures in the financial industry. In order to estimate VaR, a broad set of techniques is available. These techniques are examined in Ch. 1. By backtesting VaR models against historical data, one can assess whether they reliably predict risk. In this monograph, a Monte Carlo based model is developed using exponentially weighted moving averages to forecast volatility, and Cholesky decomposition to adjust for correlations among assets. The methods are presented in Ch. 2. The Monte Carlo model is then backtested with a portfolio of country specific exchange traded funds. The question that the author would like to answer is whether such a model just performs well during normal times, or if it is actually able to estimate risks during a financial crisis. The results (Ch. 3) show that the model works well in normal times, but that it might underestimate risks in turbulent markets. Especially extreme events are difficult to model. Hence, anybody who is using VaR models to generate risk estimates should use them with care. Page 8

9 1. Concepts in Risk Management Historical view on Risk Management The study of risk management started gaining traction in the second part of the twentieth century. Large changes in the financial system, such as the end of the Bretton Woods era, an ever increasing amount of financial transactions, financial crises and company scandals made financial professionals have a close look at the risks they are exposed to. New theories and techniques were developed, and they continue to evolve up to this day. In order to successfully manage financial portfolios, it is important to know how much risk is taken. Not only because an investor strives for returns that are risk adjusted, but also because regulating bodies require that managers continuously monitor the risks their portfolios are exposed to. There are many types of risks: liquidity risks, market risks, credit risks, legal risks and operational risks. Each of those risks has to be approached with a specific set of techniques and methods. Market risks can be further subdivided into equity price risk, exchange rate risk, commodity price risk, and interest rate risk. One of the most widely used approaches to quickly grasp the market risk involved in an asset, portfolio or even a whole organisation, is called value at risk (VaR). Value at Risk Definition Value at Risk (VaR) is a method that is based on Markowitz modern portfolio theory (Markowitz 1952), and tries to give investors a single number that is an approximation of how much they could lose with an investment at a given confidence level and time period. This can be calculated either for a single asset or for entire portfolios. Before the VaR can be calculated, one has to determine two parameters: time horizon and confidence level. If one is, for example, interested in the 5% worst outcomes for the return within a day, a confidence level of 95% and a time interval of 1 day would be selected. The basic idea is that one can compare the potential loss with capital reserves, and thus ensure that one does not put the further existence of the firm at risk by investing in a risky asset. Under the assumption that returns are normally distributed, a basic mathematical definition of VaR is: VaR α = W + z σ Where α represents the confidence level, z the (1 α)-quantile of the standard normal deviation, σ the standard deviation of the asset and W the expected (or average) value of the asset. Assuming standard normal distribution of returns, VaR can be depicted as follows: Page 9

10 Figure 1: Illustration of different VaR confidence levels, based on a normal distribution with µ = 0 and σ = 1 The most appropriate confidence level depends on the situation: For example in regulatory settings where capital requirements for financial institutions are defined, high confidence levels should be used. Choosing a confidence level of only 95% would not make any sense, as the capital requirements would just be high enough for 95% of all trading days. A bank would hence be allowed to go bankrupt more than 15 times per year As we will see later, very high confidence levels such as 99.9% have to be backtested with a lot of historical data. As market conditions change over time such backtests might yield wrong estimates. Therefore it makes sense to always estimate and backtest VaR at various confidence levels. If a normal distribution of returns and an expected value of 0 are assumed, VaR values can be transformed both in the time- as well as in the confidence level dimension. In order to transform the VaR time period, one can use the following formula (Hull 2012): VaR t2 = t2 t1 VaR t1 where t1 and t2 are the VaR time periods. For the 1-day VaR t1 would be equal to 1, for the 10-day VaR equal to 10. The confidence level can be changed using the following formula (Dowd 1998): VaR 0.99 = ( ) VaR 0.95 There is no clear inventor of the VaR approach, probably because from a mathematical perspective it is just a simple probability measure to estimate quantiles of a distribution. The first time VaR was used in a regulatory setting was in 1980, where the Securities Exchange Commission (SEC) of the United States set capital requirements of financial firms in perspective of potential losses (Dale 1996). In 1995, J.P. Morgan published RiskMetrics, which finally led to a wide acceptance of VaR (Hull 2012). In 1996, just a year after the RiskMetrics publication, the Basel Capital Accord introduced the VaR concept in its framework (BIS 1996). Criticism of VaR In order to calculate the VaR of a portfolio one has to make assumptions and simplifications. In combination with the fact that past returns can just predict future returns to a very limited degree, one has to accept that it is impossible to truly know the market risk of a portfolio. Page 10

11 Even if the risk were known, it would not be possible to represent the whole complexity of the market risk in a simple number. The biggest drawback of the VaR method is that it only allows to estimate the X% worst outcomes, and not how severe the worst outcome could be. Depending on the underlying distribution of the returns, the VaR can be misleading, and the approach has therefore been widely criticized. This is the case when actual return distribution shows fat tails (normal distribution underestimates extreme values) but the method that is used to estimate VaR is based on the assumption that returns are normally distributed. Assumptions of normality are especially not applicable to assets such as options. Here the returns are non-linear functions of the risk variables, which implicates that returns are not normally distributed. As a result, two portfolios that have the same VaR might actually bear different risks depending on the underlying distribution of returns (Tsai 2004). Therefore other methods have been developed, such as expected shortfall (ES), also referred to as conditional value at risk (Longin 2001). Another criticism is that VaR can just predict risk accurately during normal market conditions (Jorion 2007). It is hard to define what normal market conditions are, but to illustrate this problem one could look at the recent financial crisis. As the trust among financial institutions evaporated within days, markets suddenly lost their liquidity, which distorted the equity prices, fuelled uncertainty among investors and led to even higher volatilities. It is almost impossible to include compounding effects into value at risk models, as such extreme events are so rare that one cannot infer from past events. The aim of the study presented in this monograph is to compare the performance of a VaR model based on data before and during the financial crisis of One last potential shortfall of this technique is the type of data that is usually used. The standard method is to use the daily prices (e.g. closing prices of equities), and then to calculate the daily VaR based on this dataset. Daily closing prices however do not necessarily reflect the actual risk of a portfolio, since all intra-day trading activity is ignored. All the critiques make clear that VaR should not be used as the only risk management tool, as it has a couple of severe shortcomings. Nevertheless, VaR can give valuable insights if applied properly, which explains why the technique is still very popular, and is being used both by financial institutions as well as by regulatory bodies around the world. Simulation techniques to estimate VaR A broad set of techniques is available for computing the VaR of an underlying asset. Each technique is based on different assumptions. In order to interpret the resulting VaR it is therefore paramount to understand the computation methods. The techniques can be divided into parametric and non-parametric methods. While non-parametric models are based on historical prices or simulations, parametric methods use statistical parameters to determine the VaR. For completeness, this chapter will briefly describe the most widely use methods. However, the actual VaR estimates that are performed by the author are purely based on Monte Carlo simulations. Variance-Covariance Simulation (parametric) In parametric methods, an assumption has to be made about the underlying distribution of the returns. It is often assumed that returns are normally distributed, although there is significant evidence that most return series (for example the S&P500) show fatter tails (Cont 2001). The actual potential loss will therefore be underestimated. Especially returns of more complex assets such as derivatives are badly approximated with a normal distribution. As a next step, historical data is used to estimate the mean and the variance of the probability distribution. Page 11

12 Here the assumption is made that historical data is a good indicator of future price movements. This is of course just true to a limited degree. Once those parameters are estimated, one can simply estimate the value for a given confidence level. In order to calculate VaR for entire portfolios, the correlation among assets has to be estimated as well. Based on the correlation-matrix, the VaR can then be computed for the entire portfolio. A third assumption that is made is that these variables are stationary. For example, it is assumed that the variance remains constant across a time-series (homoscedasticity). This often is not the case, and therefore any simulation that is based on this assumption has to be analysed with caution. While this method is useful due to its simplicity, it does not lead to good VaR estimates because the underlying assumptions (e.g. normal distribution) are often inadequate. Historical Simulation (non-parametric) Historical simulations are the simplest to perform. In a first step, a time window of historical returns deemed relevant is chosen. Second, depending on the confidence level that one is interested in, one selects the X% worst outcomes. For example, if the time window is 100 days, and one is interested in the 95% VaR, the Value at R isk would be the fifth-worst outcome of the historical return. In the case of a portfolio, the historical price changes have to be adjusted to the asset weights. This approach is popular in the financial industry because it is simple to calculate: correlations among assets do not have to be computed, and future volatility does not need to be forecast. In addition, the historical simulation approach makes no assumptions about the probability distribution of the returns. Hence, it considers the actual distribution and is automatically adjusted for fat tails. However, the simulation is based on two other assumptions: First, it assumes that the past is a good indicator for the future. Second, it assumes an equal weight for each day of the past, which is a problem if the variance of the time series is not consistent across the selected period (that is, heteroscedasticity is present). A further disadvantage of historical simulations is that a long price history is needed. This can complicate VaR calculations for new instruments or assets since not enough market data is available (Damodaran 2007). Especially for high VaR confidence levels, one needs very large datasets to get an accurate estimate of rare events, the so-called tails of the distribution. Large datasets however have a disadvantage: if the data is too old, it might not be relevant anymore for the current market conditions, for example if the market is heteroscedastic. A last problem is that the historical method can lead to VaR bumps. Once a historical date with a high loss falls out of the dataset because it is too far in the past, the VaR estimates will shift drastically. This is an indicator that the model is not robust for changes in the dataset. Monte Carlo Simulation (non-parametric) Monte Carlo simulations are probably the most popular methods to make VaR estimates, but they are complicated to implement. This method involves several steps: First, correlations among assets have to be determined, which is usually done based on historical market data. Second, volatilities have to be forecast, which can be done using several techniques. Third, large numbers of simulation rounds are created. In each round, risk factors that influence the underlying price assume different values, and hence the outcome varies. By combining all simulation outcomes, a joint distribution of potential outcomes can be calculated. Based on these estimates, the value at risk for the whole portfolio can be estimated. As a Monte Carlo simulation will be used later in this monograph, the exact details of the calculations will be outlined in Chapter 2. Page 12

13 The most evident advantage of Monte Carlo simulations is that the assumptions about the distribution of the returns can be flexibly adopted. It is hence possible to apply this technique as well to non-linear instruments such as options (Damodaran 2007). As the estimated joint distribution is depending on the underlying parameters, it is possible to generate different scenarios with different parameters. For example, each input factor could have a different distribution. Compared to the other methods, this brings in a huge flexibility. It is common practice to run at least 5000 simulations to get a good estimate of the joint distribution. But as the simulation converges to the real distribution as 1/ N (Wiener 1999), results are improved with each additional run. Especially large portfolios might require a very potent infrastructure to keep the computation time at reasonable levels. Another disadvantage of this method is that the model might be completely wrong (so-called model risk), as the assumptions that one makes about correlations, volatilities, etc. might be wrong. It is therefore important to thoroughly backtest every Monte Carlo simulation (Hull 2012). Correlations among assets To simulate returns of a portfolio accurately one has to know the correlation among the assets in a given portfolio. The correlation among two assets X and Y is denoted ρx,y, and is calculated as follows: ρ X,Y = cov(x, Y) σ X σ Y with cov(x,y), the covariance among X and Y defined as cov(x, Y) = E[(X E[X])(Y E[Y])] and σx,, the standard deviation of asset X defined as where E denotes the expected value of X. σ X = E[X 2 ] (E[X]) 2 Volatility Forecasting For both the Variance-Covariance approach as well as for the Monte Carlo simulations, a solid volatility forecast is necessary. The simplest way to estimate future volatility is to assume that it is constant across time (homoscedasticity); hence, one just needs to calculate the variance of historical data. The variance of asset X is equal to the squared standard deviation of asset X, and is calculated as follows: Var(X) = σ X 2 = E[X 2 ] (E[X]) 2 However, by just looking at the distribution of financial returns, one can clearly see that there are periods with higher, and periods with lower return variance. Hence, methods had to be developed that can deal with heteroscedasticity in time series. The two most broadly used methods to account for heteroscedasticity are called Exponentially Weighted Moving Average (EWMA) and Generalized Autoregressive Conditional Heteroscedasticity (GARCH). Page 13

14 EWMA To deal with heteroscedasticity, one has to weight recent events more than events that are more distant in the past when calculating the variance. The exponentially weighted moving average (EWMA), assigns each date a weight, which exponentially decreases the further in the past the event is. The following formula is used: σ 2 t (ewma) 2 2 = λ σ t 1 + (1 λ) r t 1 Lambda (λ) is a smoothing parameter that has to be defined. RiskMetrics uses a lambda of 0.94, which became an industry standard. Using this value, the most recent return is weighted (1 0.94) = 6%, the previous day (1 0.94) = 5.64%, etc. GARCH (1,1) The generalized auto regressive conditional heteroscedasticity (GARCH(p,q)) method has first been developed by Bollerslev (Bollerslev 1986), and was a generalization of Engle s ARCH-approach (Engle 1982). The most widely used method is a GARCH (1,1) model, The first number in the parenthesis defines how many autoregressive lags appear in the equation, and the second number defines how many lags are included in the moving average. The GARCH(p,q) model is defined as: p σ 2 2 t = ω + α i r t i i=1 Page 14 q 2 + β j σ t j where p is the number of past observations and q the number of variance rates that are taken into account. The empirical parameters ω, αi and βi can be estimated by the maximum likelihood method, which will not be described here. Consequently, the GARCH(1,1) model is defined as: j=1 σ t = ω + αr t 1 + βσ t 1 The variance forecast of day t is hence based on three factors: The first is a constant (omega), the second is a weighted (alpha) form of yesterday's squared errors, and the last one is yesterday's weighted (beta) variance. Backtesting The idea of backtesting is to test if a model would have yielded reliable risk estimates when using historical data. It hence functions like a reality check for any model based on assumptions that might or might not be true (Jorion 2007). There are different methods for backtesting, but they all look at how often an actual return exceeded the one day VaR. The term exception is used for a day where the return is below the VaR estimate. When we use a confidence level of 99%, the one-day VaR should be exceeded approximately on 1% of all days. When there are significantly more (or less) exceptions observed, the model might not be suitable to estimate risk. A broad set of backtests has been described in academic literature, each of it with its advantages and disadvantages. A good overview is provided by Nieppola (Nieppola 2009). In general, one can divide the backtests into two categories: unconditional coverage tests and conditional coverage (Jorion 2007). Unconditional coverage tests just check whether the total amount of exceptions is in line with predicted outcomes. Conditional coverage tests also check whether clustering of exceptions

15 occurs. Good models have an even distribution of exceptions, which means that the model is accurate for different market conditions. If exceptions are clustered in a specific time window, then the VaR estimate is not accurate for this period. This chapter will describe three tests in detail, as they are later used to backtest the developed model. Kupiec POF-test (unconditional) Paul H. Kupiec developed the point of failure test (POF-Test) in 1995 (Kupiec 1995), which tests the frequency of exceptions, also known as failure rate. As described above, the failure rate should roughly correspond to the chosen VaR confidence level. If there are more exceptions than expected, the VaR underestimates risk, if there are fewer exceptions than expected it overestimates risk. The failure rate can be defined as x/t, where x is the number of exceptions, and T the total number of observations. For a selected VaR confidence level of 99%, the failure rate should converge to 1% (= p =1- confidence level) when the total number of observations is increased (Jorion 2007). The total amount of VaR violations follows a binomial probability distribution: p(x) = ( T x ) px (1 p) (T x) This can be approximated by a normal distribution: f(x) = x pt p(1 p)t N(0,1) Based on this distribution, one can test the null hypothesis that H 0 : p = p = x T If we can show that x/t is significantly different from p, we should reject the VaR model. The POF-test is a likelihood-ratio test, where LR POF = 2ln [ (1 p)t x p x (1 x T )T x ( x T )x] LRPOF should be χ 2 -distributed, using one degree of freedom. The VaR model will be rejected if the LRPOF statistic exceeds the critical value of the χ 2 distribution. This test has two shortcomings: a) the smaller the sample size, the weaker will be the test. And b) clustering of exceptions cannot be observed, as already described above. Hence other backtests have been developed. Kupiec TUFF-test (unconditional) In order to take the independence of exceptions into account, Kupiec has developed a second test, called TUFF-test (time until first failure) (Kupiec 1995). The Likelihood-ratio for the TUFF test is: p(1 p)v 1 LR TUFF = 2ln [ ( 1 v ) (1 1 v 1 ] v ) Page 15

16 where p is equal to 1- VaR confidence level, and v is the time until the first exception. LRTUFF is χ 2 distributed with one degree of freedom. As this test only takes into account the first failure, it is not capable of discovering the clustering of exceptions in the middle of the dataset. The test was however adopted by Haas (Haas 2001), who combined the POF and the TUFF test into a conditional coverage test, the so-called Mixed-Kupiec test. Haas Mixed-Kupiec test (conditional) Haas proposed a backtest that takes into account the time between successive exceptions. He takes advantage of the TUFF test to measure the time in-between exceptions, and combines it with the POF test to check if the overall rate of failure is accurate. Haas proposes the following test statistic: p(1 p) v i 1 LR Ind = 2ln [ ( 1 v ) (1 1 v i 1 i v ) i ] where vi represents the time interval between two exceptions. Hence, a test statistic has to be calculated for each exception. By combining the different likelihood-ratios, one gets LRind: n p(1 p) v i 1 p(1 p)v 1 LR ind = ( 2ln [ ( 1 v ) (1 1 v i 1 ) 2ln [ i=2 i v ) i ] ( 1 v ) (1 1 v 1 ] v ) which is also χ 2 distributed, with n degrees of freedom. By further combining it with the POF test, one finally gets the Mixed-Kupiec test. Its likelihood ratio is defined as: LR Mixed = LR POF + LR ind LR Mixed = 2ln [ (1 p)t x p x (1 x T )T x ( x + ( 2ln p(1 p) v i 1 T )x] [ ( 1 v ) (1 1 i=2 i v ) i p(1 p)v 1 2ln [ ( 1 v ) (1 1 v 1 ] v ) LRMixed is again χ 2 distributed with n+1 degrees of freedom. n v i 1 ] ) Page 16

17 2. Methods: Empirical VaR modelling and subsequent backtesting The idea of the empirical section of this monograph is to generate a model that simulates the VaR of a portfolio of exchange-traded funds (ETFs). The model is then backtested with historical data. The aim is to check whether such a model is suitable to accurately estimate the value at risk in different market conditions. The following chapter will describe each step from data gathering to the final analysis. Data gathering and processing Value at risk can be calculated for any type of portfolio. The author wanted to use equities, and selected exchange traded funds (ETFs) due to their recent increase in popularity. ETFs are tradable funds that reproduce a specific index, thereby offer investors the possibility to make highly diversified investments. ETFs have gained popularity in recent years as they offer diversification with much smaller fees compared to the fees of an asset management firm. As the available computing power was limited to a laptop, a relatively small portfolio of 17 ETFs was selected. The data used in this monograph was downloaded from The following tickers were used, and the daily closing prices of the ETFs were downloaded for the period from April 1, 1996 to January 30, Tickers: EWJ (Japan), EWG (Germany), EWU (United Kingdom), EWH (Hong Kong), EWC (Canada), EWW (Mexico), EWA (Australia), EWP (Spain), EWL (Switzerland), EWI (Italy), EWS (Singapore), EWM (Malaysia), EWD (Sweden), EWQ (France), EWK (Belgium), EWN (Netherlands), EWO (Austria). These ETFs have been selected for three reasons: they were among the first on the market (hence a lot of historical data is available), they are all based on the MSCI index, and they are all managed by the same company (Blackrock). Returns have been calculated using log-returns. Return on day t (rt) is defined as: r t = ln ( S t S t 1 ) where St is the stock price on day t, and St-1 the stock price of the previous trading day. All simulations were performed in Microsoft Excel, using Visual Basic for Applications (VBA ). Figures were generated with GRETL version Extraction of parameters for simulations As a separate simulation is run for each day, correlations among assets as well as the volatility were calculated for each day separately. It was decided to take the last 252 trading days into account in order to estimate the parameters for the Monte Carlo simulation (i.e. to estimate volatility of day t, days t-252 to t-1 were taken into account). Correlation: Cholesky decomposition A Cholesky decomposition (also called Cholesky factorization) was used to account for correlation among the assets. The application in the process of Monte Carlo simulations is described, for example, in the work of Jorion (Jorion 2007). This procedure decomposes the Page 17

18 symmetric, positive-definitive matrix of correlation coefficients into the product of a lower triangular matrix and its conjugate transpose: If Then M = U T DU = U T DU = (U T D)( DU) = ( DU) T ( DU) Where C = DU satisfies C T C = The decomposed matrix can then be used to reintroduce correlation in a set of simulated asset prices by multiplying the matrix of simulated returns with the lower triangular matrix that is generated by the Cholesky decomposition. As Excel has no built-in function to perform this decomposition, a user defined function (UDF) has been set up in VBA. The original VBA code was developed by Charles Dapaah and shared on his site The complete VBA code can be found in Appendix 1. Volatility Forecast: Exponentially Weighted Moving Average The author implemented both EWMA and GARCH(1,1), but it turned out that the available computer was not powerful enough to perform good estimates of the GARCH(1,1) parameters inside the rolling analysis of the Monte Carlo simulation. The author therefore decided to focus on EWMA for the empirical model, which was described in detail in the introduction chapter. The decay factor was set to 0.94, according to RiskMetrics recommendation in 1996 (Jorion 2007). Monte-Carlo Simulation From all possible methods to model VaR, the Monte Carlo simulation has been selected as it is the most flexible one, and one does not have to make assumptions about underlying distributions. A VaR estimate is being calculated for each day (1-day VaR) and for each ETF individually. The potential price move of each asset is based on a simulation of 5000 price realizations, where the input factors are the correlation among assets (based on Cholesky decomposition) and the forecast volatility (based on EWMA). The last 252 trading days were selected as a period for the historical price data that is used for the estimation of the parameters. In order to transform log-returns into USD, the log-returns were first converted into arithmetic percentage changes and subsequently into currency values. It was assumed that USD were invested into each asset at each point in time. This simplification was done because it significantly speeds up computing time. A VaR is always dependent on the weights of each asset in the portfolio; it is therefore a risk measurement that only applies to the analysed portfolio with all its specific characteristics. Selecting different weights would have been as arbitrary as selecting equal weights. It should however have no influence on the backtesting results, as the backtesting procedure is not dependent on the individual weights if it is performed at the level of the portfolio-returns. Page 18

19 Once the percentage returns of the individual assets had been transformed into US dollars, the single VaR estimates were aggregated using the following formula: VaR Portfolio = VaR i VaR j ρ ij where ρ ij denotes the correlation among asset i and j. Rolling analysis The methods described so far will calculate the VaR for a single day. A VBA script has been written to automate the process: In a first step, the VaR of the portfolio at the starting day is calculated. Second, all relevant values are extracted to separate sheets where they are stored for later analysis. Third, the timeframe that is used to estimate the VaR (i.e. input data that is used to forecast volatility, calculate correlations among assets, etc.) is moved forward by one trading day. The VBA script then loops through these three steps until a previously specified end date is reached. Through this process, a time series of VaR estimates is generated, where each VaR estimate is based on historical data of the previous 252 trading days. The complete VBA code can be found in the Appendix 2. While the code works fine, it is not yet structurally optimized, and the author assumes that the calculations can be streamlined in future versions. Backtesting The time-series of extracted VaRs have then been used for backtesting of the Monte Carlo simulation. For comparison, the following four time periods where defined: P1 = trading days: The period immediately before the financial crisis broke out. P2 = trading days: The first part of the financial crisis ( subprime mortgage crisis ) P3 = trading days: The second part of the financial crisis ( sovereign debt crisis ) P4 = trading days: The markets seem to be normal again (low volatility) Two backtests have been used: Kupiec s POF test and Haas Mixed Kupiec test (which are described in detail in the introduction). While the Kupiec s POF test focuses on the total number of VaR exceptions, Haas Mixed Kupiec test also analyses whether the exceptions are clustered, a condition that should be avoided. All tests have been performed at a significance level of p=0.05. Critical χ 2 values depend on the available degrees of freedom. While all POF tests can be calculated with 1 degree of freedom, the Mixed-Kupiec tests have different degrees of freedom, depending on how many exceptions occur. Page 19

20 3. Results Descriptive Statistics To get an overview of the data at hand, the downloaded data was subjected to descriptive statistics. The following table shows the mean, the standard deviation and the variance of the logreturns of each ETF. Japan Germany UK HongKong Canada Mexico Australia Spain Switzerland - MEAN 0, , , , , , , , , STDEV 0, , , , , , , , , VAR 0, , , , , , , , , Italy Singapore Malaysia Sweden France Belgium Netherlands Austria - MEAN 0, , , , , , , , STDEV 0, , , , , , , , VAR 0, , , , , , , , Table 1: Mean, standard deviation and variance of the log-returns across the whole sample ( ). Correlation Analysis Subsequently, correlations among the ETFs have been calculated, which is shown in the following table. Japan Germany UK HongKong... Sweden France Belgium Netherlands Austria Japan 1 0, , , , , , , , Germany 0, , , , , , , ,69905 UK 0, , , , , , , , HongKong 0, , , , , , , , Canada 0, , , , , , , , , Mexico 0, , , , , , , ,7349 0, Australia 0, , , , , , , , ,59954 Spain 0, , , , , , , , , Switzerland 0, , , , , , , , , Italy 0, , , , , , , , , Singapore 0, , , , , , , , , Malaysia 0, , , , , , , , , Sweden 0, , , , , , , , France 0, , , , , , , , Belgium 0, , , , , , , , Netherlands 0, , , , , , , , Austria 0, , , , , , , , Table 2: Correlation of log-returns across the whole sample ( ). The entire matrix can be found in Appendix 3. Monte Carlo Simulation In the next step, a simulation has been set up as described in the Methods section. The simulation that was finally used was based on a Monte Carlo simulation with 5000 simulation rounds. The author is aware of the fact that simulation rounds would yield slightly more accurate results. With the available infrastructure it was however not feasible to further increase the amount of simulations. The correlation among assets was based on a Cholesky decomposition of the initial correlation matrix that was based on the observed returns, and the volatility was estimated with an exponentially weighted moving average approach (EWMA). As a first step, the size of the time-window was determined as being 252 trading days (which roughly corresponds to a full calendar year). Next, a correlation matrix was generated for this time frame, based on the historical log-returns. This correlation matrix was then decomposed using Cholesky decomposition as described in the methods. Page 20

21 It was hypothesized that the correlation among assets would vary depending on the time windows. Jorion already suggested that assets are correlated more strongly in turbulent markets compared to normal market conditions (Jorion 2007). A brief analysis of the correlation means showed that the crisis periods of our dataset also had higher correlations (Appendix 4). This is of course no proof of Jorion s hypothesis, but it indicates that the dynamics are different depending on the conditions of the market. Cholesky Decomposition of the Correlation Matrix As we observed varying correlations across the whole time series, the correlation matrices of the past 252 returns have been decomposed with a Cholesky decomposition for each trading day. Table 3 shows a sample of the decomposition that was done for one period. Japan Germany UK HongKong... Sweden France Belgium Netherlands Austria Japan Germany 0, , UK 0, , , HongKong 0, , , , Canada 0, , , , Mexico 0, , , , Australia 0, , , , Spain 0, , , , Switzerland 0, , , , Italy 0, , , , Singapore 0, , , , Malaysia 0, , , , Sweden 0, , , , , France 0, , , , , , Belgium 0, , , , , , , Netherlands 0, , , , , , , , Austria 0, , , , , , , , , Table 3: Cholesky decomposition of the matrix of correlation coefficients on Sept. 1, The entire matrix can be found in Appendix 5. Volatility Forecast using EWMA In a second step, future volatility was forecast using the EWMA approach. For the decay factor Gamma, a value of 0.94 was used as recommended by the authors of RiskMetrics. Figure 2: A decay factor of 0.94 was used for the exponentially weighted moving average volatility forecast. This graph shows the weights that have been assigned to historical returns to forecast volatility of Sept. 1, The spaces in between the bars are due to the weekend days which did not receive any weight. Page 21

22 The Monte Carlo simulation of the ETF returns was then performed. It was based on these two estimates, where in each simulation round, a random number was generated (based on a normal distribution), taking the estimated volatility and correlation into account. Feasibility of the Simulation Outcome The correlation matrix of the simulated returns was compared with the correlation matrix of the historical log returns to see if the Cholesky decomposition yielded feasible results. Japan Germany UK HongKong Sweden France Belgium Netherlands Austria Japan 1 0, , , , , , , , Germany 0, , , , , , , , UK 0, , , , , , , , HongKong 0, , , , , , , , Canada 0, , , , , , , , , Mexico 0, , , , , , , , ,26743 Australia 0, , , , , , , , , Spain 0, , , , , , , , , Switzerland 0, , , , , , , , , Italy 0, , , , , , , , , Singapore 0, , , , , , , , , Malaysia 0, , , , , , , , , Sweden 0, , , , , , , , France 0, , , , , , , , Belgium 0, , , , , , , , Netherlands 0, , , , , , , , Austria 0, , , , , , , , Table 4: Correlation matrix of the log-returns that were generated by the Monte Carlo simulation on Sept 1, While the initial simulation generates uncorrelated returns, it can be seen that the correlation among the assets has been correctly reintroduced by the Cholesky decomposition. The entire matrix can be found in Appendix 6. As a last step, each individual ETF log-return estimate was converted back to the arithmetic return estimate. Taking into account the weight of each ETF in the portfolio, and the correlation among the assets, the individual estimates where combined to a portfolio estimate. The VaR was then estimated for three confidence levels: 0.95, 0.99, and The following graph shows the cumulative distribution function of the simulated returns for Sept. 1, Figure 3: Cumulative distribution function of the simulated portfolio returns for Sept. 1, The returns resemble the Gaussian distribution. Rolling Analysis In order to backtest the VaR simulations, a VBA script was written that repeated the calculations so far described across a large timeframe. Thereby, VaR estimates where generated for each trading day for the time frame between January 1, 2006 and December 11, 2013 (in total 2000 trading days). The goal of this monograph was to compare the Page 22

23 performance of the simulation between normal market conditions and the recent financial crisis. Therefore, four time frames were selected, and the simulation was backtested using each of the four periods. Figure 4 depicts the whole timeframe that was analysed. Graphs for each individual period can be found in Appendix 7. Figure 4: Historical returns of the portfolio (black), with VaR estimates at 95% (green), 99% (blue) and 99,9% (red) confidence levels. The periods that were backtested are denoted with P1, P2, P3 and P4. An exception occurs when the historical return falls below the projected VaR. Backtesting Outcomes As described in the Methods section, each period was tested for overall proportion of failure (Kupiec s POF-Test) as well as for bunching of the exceptions (Haas Mixed Kupiec-Test). The critical χ 2 level was calculated based on the p-value of For the POF test, all periods and VaR confidence levels have the same critical χ 2 value, as all tests are performed with 1 degree of freedom. For the Mixed-Kupiec test the critical χ 2 value has to be calculated on an individual basis, as the degrees of freedom depend on how many exceptions occur in the analysed period. The results in Table 5 show that in the time before the crisis, the simulation provided accurate results (model should not be rejected), whereas during the crisis, it had to be rejected at some confidence levels. A comparison between the two analysed crisis-periods shows that they have a different profile in terms of their effect on the VaR. While VaR estimates were at least partially accurate during the first crisis, they clearly failed in the second crisis: During the first crisis, the model cannot be rejected with both tests at the 0.95% VaR confidence level. At higher VaR confidence levels, the model fails in both tests. During the second crisis however, it has to be rejected on all VaR confidence levels, both in the POF test as well as in the Mixed-Kupiec test. In comparison with P2, P3 renders much higher values of χ 2. This shows that the model is more inaccurate during the second part of the crisis. Those results are interesting and will be further analysed in the Discussion section. Page 23

24 P1: Before Financial Crisis KUPIEC: POF-TEST HAAS: MIXED KUPIEC-TEST ( days) X2 Crit. X2 TestResult X2 Crit. X2 TestResult 0,95 4,95 3,84 Reject 36,57 33,92 Reject 0,99 0,75 3,84 Do Not Reject 7,35 11,07 Do Not Reject 0,999 No Exception 3,84 No Exception No Exception 3,84 No Exception P2: First Financial Crisis KUPIEC: POF-TEST HAAS: MIXED KUPIEC-TEST ( days) X2 Crit. X2 TestResult X2 Crit. X2 TestResult 0,95 2,98 3,84 Do Not Reject 28,71 31,41 Do Not Reject 0,99 5,42 3,84 Reject 26,08 15,51 Reject 0,999 9,40 3,84 Reject 34,81 9,49 Reject P3: Sovereign Debt Crisis KUPIEC: POF-TEST HAAS: MIXED KUPIEC-TEST ( days) X2 Crit. X2 TestResult X2 Crit. X2 TestResult 0,95 11,52 3,84 Reject 59,60 40,11 Reject 0,99 29,09 3,84 Reject 76,90 26,30 Reject 0,999 14,65 3,84 Reject 41,08 11,07 Reject P4: Normal Markets KUPIEC: POF-TEST HAAS: MIXED KUPIEC-TEST ( days) X2 Crit. X2 TestResult X2 Crit. X2 TestResult 0,95 0,89 3,84 Do Not Reject 27,00 27,59 Do Not Reject 0,99 1,92 3,84 Do Not Reject 5,95 12,59 Do Not Reject 0,999 No Exception 3,84 No Exception No Exception 3,84 No Exception Table 5: Backtesting results: All backtests were performed at p=0.05. Values in the first column represent the VaR confidence levels. Page 24

25 Discussion Interpretation of the Results A first thing that we observe when looking at Table 5 is that the model seems to predict value at risk more or less accurately during P1 and P4, which were selected based on the low volatility (i.e. calm markets). At VaR confidence level 0.95% the model had to be rejected in P1, but the χ 2 value was just slightly above the critical χ 2 level. At the other VaR confidence levels the model could not be rejected by using both the POF and the Mixed-Kupiec test. When a model works well in calm markets, this simply means that it is built in a way that a given input yields an accurate output. If we assume for example that the correlation among assets, the volatility as well as the return distribution remain constant, then a correctly designed model should have no problems to estimate the VaR at a given confidence level. One interesting point to discuss is that both in P1 as well as in P4, there were no exceptions at VaR confidence level of This should not be further surprising as we just backtested the model over a period of 252 trading days. At a confidence level of 0.999, we would expect 1/1000*252 exceptions, which is equal to 0,252. Nevertheless, it makes sense to check if an exception at 0.95% is also an exception at 0.999% as a model might either underestimate the risk just at one confidence level such as in P1, or at all confidence levels such as in P3. When we look at P2, the first part of the financial crisis (triggered by subprime mortgages), the model could not be rejected at 0.95%, but failed to correctly estimate the tail losses at higher confidence levels. This occurred because the volatility shot up very fast during this period, and the model is programmed in a way that it only gradually adopts to those new market conditions. Hence, such suddenly occurring high losses appear as exceptions at 99% and 99.9% levels. In P3, which was also termed the beginning of the sovereign debt crisis, the model fails to predict the accurate VaR values. Overall exceptions were underestimated at all confidence levels. The results were also clustered at all confidence levels, as we can see from the mixed Kupiec test. Interestingly the model passed at the confidence level. One can clearly see that the χ 2 values are much higher in P3 compared to P2, so the model was highly inaccurate during the sovereign debt crisis. But what could be the reason that there is a difference between P2 and P3? One factor that could be important is the geographic exposure of the analysed portfolio. ETFs gained popularity on the US market as they offered investors a cheap and convenient way to diversify their portfolio with foreign assets that showed little correlation with the US market (compared to stocks of US companies). While the mortgage crisis emerged from the US, the sovereign debt crisis had its epicentre in Europe. As the portfolio has no US ETF included, it s return distribution will react differently in each of these crises. Possible adjustments of the model The model that was used to estimate the VaR was based on several parameters, as described in the Methods section. It is not possible to know the true value of the parameters, as their optimum would change over time, depending on the underlying assets. The input values that were used for those parameters are estimates that should yield good results across different portfolios, across time frames, etc. To get better VaR estimates, those parameters could be changed: One could for example try to modify lambda, the decay factor of the exponentially weighted moving average that is used to forecast volatility. The downside of a model that adjusts too Page 25