Fin285a:Computer Simulations and Risk Assessment Section 9 Backtesting and Stress Testing Daníelson, , 8.5, 8.6

Transcription

1 Fin285a:Computer Simulations and Risk Assessment Section 9 Backtesting and Stress Testing Daníelson, , 8.5, 8.6

2 Overview What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Fall 2017: Fin285: 9 2 / 25

3 What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing What is backtesting? Fall 2017: Fin285: 9 3 / 25

4 Backtest basics Test VaR and ES estimates Use past data Forecast into the future (windows) Important for regulation (Basel II and III) Out of sample forecasting Clean future forecasts VaR estimates at time t use only data before time t Fall 2017: Fin285: 9 4 / 25

5 What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Regulatory issues Fall 2017: Fin285: 9 5 / 25

6 Market risk capital (reserves) Capital (liquid) held in reserves by banks and other financials Required by regulation Important during crisis Previously: Set by fraction of portfolio in different asset classes Sovereign debt Corporate debt Home mortgages Equity Now (> 2005): Also, involves some aspect of internal risk (VaR) measures (Internal models) Needs to be carefully monitored (backtesting) Fall 2017: Fin285: 9 6 / 25

7 Example Capital t = C t = A t max(var t (1%), VaR t (1%))+Constant (9.1) VaR t (1%) = Average over previous 60 days (9.2) A t = 3 if v t (v t 4) if 5 v t 9 (9.3) 4 if 10 v t v t = VaR(1%) violations over past 250 trading days Think about the dynamics of this: Volatility goes up v t rises Need to shift portfolio to cash Fall 2017: Fin285: 9 7 / 25

8 What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Backtesting details Fall 2017: Fin285: 9 8 / 25

9 Estimation windows (length = 500) Full Sample [1,500], VaR 501 [2,501], VaR 502 [3,502], VaR 503 Fall 2017: Fin285: 9 9 / 25

10 Estimation and Testing Estimation Start End Test Fall 2017: Fin285: 9 10 / 25

11 Recursive windows Windows can extend all the way back to the start of the data: Estimation Start End Test Fall 2017: Fin285: 9 11 / 25

12 VaR violations (exceptions) η t = { 1 if y t VaR t 0 if y t > VaR t (9.4) v 1 = η t (9.5) VR = v 1 pw T = v1 Observed violations Expected violations = p W T (9.6) W T testing length If working, VR 1 Fall 2017: Fin285: 9 12 / 25

13 What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Backtest examples Fall 2017: Fin285: 9 13 / 25

14 Null hypothesis VaR(p) v t is binomial (like coin) Two features to test: VaR probability: Pr(η t = 0) = 1 p Pr(η t = 1) = p Variability of p estimates in windows (see our old examples) Independence: η t independent over time η t are unpredictable Fall 2017: Fin285: 9 14 / 25

15 Backtest code Python: backtest.py Rolling historical VaR Estimate VaR using past 500 days Report violations and violation ratio Simpler than code in textbook Statistical test: like coin, and election polls Fall 2017: Fin285: 9 15 / 25

16 Independence VaR violations (exceptions) should be independent Find conditional probability that violation (η t ) follows violation(η t 1 ) For independence this is equal to the VaR probability level Can also look at correlations of violations η t should be unpredictable using any past data Fall 2017: Fin285: 9 16 / 25

17 Testing independence Pr(η t = 1 η t 1 = 1) = Pr((η t = 1)and(η t 1 = 1)) Pr(η t = 1) (9.7) Python: backtest.m Find conditional probability (above) For independence this is equal to the VaR probability level A violation today contains no information on tomorrow Fall 2017: Fin285: 9 17 / 25

18 Testing expected shortfall This can be done A little trickier than VaR Requires more data See textbook Fall 2017: Fin285: 9 18 / 25

19 Testing expected shortfall First, assume rolling expected tail return: ES t (p) = R t (9.8) Now, key ratio: E(R t R t VaR(p)) ES t (p) = 1 (9.9) For correct ES estimate this ratio should be 1 Python: backtestes.py Difficult to do analytic statistical tests Bootstrap confidence bands Generally requires more data than testing VaR Fall 2017: Fin285: 9 19 / 25

20 Problems with backtesting Window/time selection Enough extreme events? Regime changes True out of sample (data snooping) Need clean tests This rarely happens Multiple model problems Critical for policy and reserves Fall 2017: Fin285: 9 20 / 25

21 What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Stress testing Fall 2017: Fin285: 9 21 / 25

22 Scenario analysis Test important scenarios Find portfolio sensitivity to key risk factors V t = K β j f t,j (9.10) j=1 Extreme moves in key risk factors Types of tests Replay actual events (crashes of 1987/2008) Generate new extreme events (Greece leaves Euro, Brexit) Monte-carlo with extra weight on extremes Structural breaks in the times series (changing correlations) Scenarios often dictated by regulation Fall 2017: Fin285: 9 22 / 25

23 Smooth incorporation into risk models (f,f stress ) simulations for normal and stressed situations. { f with probability (1 ǫ) f new = f stress with probability ǫ (9.11) Fall 2017: Fin285: 9 23 / 25

24 Stress test problems Finding scenarios Probabilities for worst case scenario?? Preparing for a 10,000 year flood What if institutions plan around given scenarios, but not ready for other events? This is like our manipulating VaR examples. Endogenous feedback: What happens when everyone is hit with this shock Fire sales across markets Fall 2017: Fin285: 9 24 / 25

25 Overview What is backtesting? Regulatory issues Backtesting details Backtest examples Stress testing Fall 2017: Fin285: 9 25 / 25