Using AFT and FSI Tools to Validate Components of Interest Rate Risk Models

Transcription

1 Using AFT and FSI Tls t Validate Cmpnents f Interest Rate Risk Mdels Page 1

2 Agenda Validatin Tests f Prepayment Mdels Mdels applicable t securities Mdels applicable t prtfli lans Validatin Tests f Stchastic Rate Generatrs using FSI Lattice Mnte Carl Page 2

3 Regressr and Back-testing Regressr is applicable t AFT prepayment mdels: Securities MBS CMOs MBS data at the CUSIP level require extractin rutines frm Blmberg r Intex. CMO data at the CUSIP level require license with Intex, Chasen r ther cash flw mdel vendr. Agency CMO data can be extracted frm Blmberg. Page 3

4 Regressr and Back-testing Batch Back-test in the Regressr Mdel Output created: Actual CPR AFT frecast f CPR using actual rates Output is written t text files with number f files equal t the number f cllateral types. Example: 300 CUSIPs f 15 and 30 FNMA, FHMLC, and Whle lans might reside in six text files All CUSIPs can be run at nce in Regressr with the text files created autmatically. Text files can be pened in EXCEL and standard gdness f fit r ther statistics applied. Page 4

5 Regressr and Back-testing Once a macr is written, prcess takes less than an hur fr the entire list f CUSIPs (We did this fr 300 CUSIPs.) Prcedure can be learned in abut 15 minutes via phne supprt, if all required extract files are available. Regressr allws the aggregatin acrss CUSIPs drawn frm same cllateral. Perfrmance n sub-prtflis can be assessed. Page 5

6 Why can AFT Claim Regressr Prvides a Back-test? AFT Prepayment mdels are estimated using cllateral data Securities are small samples f the cllateral. Output in Regressr at CUSIP level is based n bth verlapping histrical perids and prjected perids, depending n the length f histry and when cllateral mdel was estimated. Over the histrical data mdel perid this is an in-sample backtest. When CPRs are prjected beynd histrical data perid used fr building mdels, the test is an ut-f-sample back-test. If the cnnectivity in the ALM system and AFT mdels has been validated, then the ALM mdel is nt needed fr back-testing AFT prepayment mdels. Page 6

7 Nv-02 Jan-03 Mar-03 May-03 Jul-03 Sep-03 Nv-03 Jan-04 Mar-04 May-04 Jul-04 Sep-04 CPR (%) Back testing Sample Output frm Regressr KTS R 2 =73% Actual Predicted 80 One f 180 graphs created fr an ALCO Partners client Page 7

8 Dynamic Aggregatr and Back testing Dynamic Aggregatr is applicable t prtfli lans Similar utput created as in Regressr AFT prepayment mdels prvide the basis fr mdeling. Discrete prepayment factrs can be scaled t imprve mdel fit. Histrical data used t fit mdel and mdel statistics can be created as part f the estimatin prcess. In-sample and ut-f-sample mdel testing can be applied with similar statistics. If AFT-ALM mdel link has been validated, then the prcess can substitute fr back test using ALM mdel. Page 8

9 Findings frm Dynamic Aggregatr Prjects Dynamic Aggregatr has cincidentally slved a classic data bttleneck in many banking institutins IT departments nw regularly archive histrical data relevant fr testing prepayment mdels They are very efficient at prviding raw data. They are inefficient at mapping data Internal resurces fr research and analysis are frequently lw n a pririty list External prjects where nly raw data is required appear nt t face the same internal hurdles because they demand very few IT resurces Page 9

10 Findings frm Dynamic Aggregatr Prjects A part f prepayment mdel validatin prjects applicable t prtfli mrtgages is btaining and mapping data. But this prcess is n lnger a large cst. Mapping csts have nt been large. Key variable is number f systems Mapping rules typically remain cnstant ver time, s annual updates becme very cst effective Once the data has been mapped int Dynamic Aggregatr, the prcess fr testing and is very cst-effective, particularly when the prcess is dne annually as required by banking regulatins Page 10

11 Testing Stchastic Rate Generatrs In many banks stchastic rate generatrs are being utilized t measure value based IRR Base case is calibrated t market instruments There are knwn nuances and prblems assciated with this Hw des a bank validate a stchastic rate mdel? Page 11

12 Testing Stchastic Rate Generatrs Apprach I Obtain mdel dcumentatin Build a separate mdel independently Perfrm duble-blind tests and cmpare inputs and utputs Issues with Apprach I Vendrs may be unwilling t prvide prprietary cde dcumentatin Building a separate mdel can be very expensive, including its validatin Specialized cding/math expertise required t build a stchastic rate generatr Page 12

13 Testing Stchastic Rate Generatrs Apprach II Obtain a qualified stchastic rate generatin system Calibrate t the same set f market instruments Cmpare inputs and utputs fr tested samples FSI: A qualified stchastic rate generatin system Page 13

14 Testing Stchastic Rate Generatrs FSI is a fully specified tw factr lattice & Mnte-Carl system designed as a prtfli management prduct with validatin capabilities Prvides a basis fr cmfrt r discmfrt depending n the mdel results and hw the mdel results are integrated with prtfli management decisins Since it cntains bth single factr (BK) and tw-factr prcesses, it prvides a basis fr understanding the sensitivity f the risk measure t the underlying prcess utilized Helps imprve calibratin prcedures and allws a lk int the black bx f stchastic rate generatin capability Page 14

15 Test Prcedures Testing Stchastic Rate Generatrs Cmpared stchastic utput frm a widely used ALM mdel with a stchastic generatr Cmpared three prtflis in tw perids MBS & CMO Prtfli FRMs Callable Bnds Greeks are cmputed at the instrument level Market values and market values under rate shcks can be cmputed Market values changes cmpared ver time at instrument level Results used t refine and understand calibratin prcess and reasnableness f risk results Page 15

16 % D MV Callables: MV Sensitivity 12/31 3/30 8% Callables: MV Sensitivity Cmpared 8% Callables: MV Sensitivity Cmpared 6% 4% FSI1F FSI2F ALM 6% 4% FSI1F FSI2F ALM 2% % D MV 2% -2% -2% -4% -4% -6% Shck (bp) -6% Shck (bp) Market value sensitivity measures fr callable bnd prtfli where very similar in bth time perids Page 16

17 Duratin (ALM) Cnvexity (ALM) Callables: Duratin & Cnvexity Duratin Cnvexity 8 Callables: FSI 2F vs. ALM Mdel 8 Callables: FSI 2F vs. ALM Mdel Duratin (FSI 2Factr) Red line represents parity Cnvexity (FSI 2Factr) Page 17 Results indicated sme nn-randm differences n individual securities. Further investigatin revealed pattern assciated with at the mney calls. Crss checks with market values shwed sensitivity t calibratin mechanics

18 % D MV (ALM) Callables: FSI 2 Factr vs. ALM Mdel: 12/31 t 3/31 Hw des Market Value Change ver Time? 4% Callables: 3/31 vs. 12/31, FSI vs. ALM Mdel 3% 2% 1% -1% -1% 1% 2% 3% % D MV (FSI 2Factr) Red line represents parity Page 18

19 %D MV %D MV FRMs: MV Sensitivity 12/31 3/31 4% FRMs: MV Sensitivity 6% FRMs: MV Sensitivity 2% 4% 2% -2% -4% -6% FSI1F FSI2F ALM -2% -4% -6% -8% FSI1F FSI2F ALM -8% Shck (bp) -12% Shck (bp) Prepayment mdels were identical Page 19

20 Duratin (ALM) Cnvexity (ALM) FRMs: Duratin & Cnvexity Duratin Cnvexity 6 FRMs 0 FRMs Duratin (FSI 2Factr) Cnvexity (FSI 2Factr) Red line represents parity Page 20

21 % D MV (ALM) FRMs: FSI (2 Factr) vs. ALM: 12/31 t 3/31 FRMs -1% -2% -3% -3% -2% -1% % D MV (FSI 2Factr) Red line represents parity FRMs where there are gd market value indicatrs cmbined with the OAS calculatins lead t similar results Page 21

22 MV MV CMOs & MBS: MV Sensitivity 12/31 3/31 4% CMOs &MBS: MV Sensitivity Cmpared (12/31/2004) 4% CMOs &MBS: MV Sensitivity Cmpared (12/31/2004) 2% 2% -2% -2% -4% FSI1F FSI2F -4% FSI1F FSI2F -6% ALM -6% ALM -8% Shck (bp) -8% Shck (bp) Page 22

23 Duratin (ALM) Cnvexity (ALM) CMOs & MBS: Duratin & Cnvexity Duratin Cnvexity 8 CMOs & MBS: Duratin 1 CMOs & MBS: Cnvexity Duratin (FSI 2Factr) Cnvexity (FSI 2Factr) Red line represents parity Circled utliers were later identified as input errrs in the ALM mdel Page 23

24 %D MV (ALM) CMOs & MBS: FSI (2 Factr) vs. ALM: 12/31 t 3/31 1% CMOs& MBS: 3/31 vs. 12/31-1% -2% -3% -4% -5% -5% -4% -3% -2% -1% 1% %D MV (FSI 2Factr) Red line represents parity Fur utliers are the same CUSIPs as in the prir graph Page 24

25 Yield Curve (%) 1M LIBOR Rate (%) 5Yr LIBOR Rate (%) 3 Yield Curve Distributins: Single Factr vs. Tw Factr Stchastic Prcesses MC Shrt Rate Max 25% MC Lng Rate Max 25% Percentile 2 Percentile 2 15% % % 90th 75th Mean Median 25th 10th 2.5 Min Single Factr 3-31 Mnth 1 5% 75th Mean Median 25th 10th 2.5 Min Single Factr 3-31 Mnth 90th 3% MC Yield Curve Single Factr Prcess: Yield Curve Distributin 2% 1% -1% -2% -3% -4% Percentil e 75th 90th 97.5 Max Median Mean 25th 10th 2.5-5% Page 25-6% Mnth Single Factr 3-31 Min

26 1M LIBOR Rate (%) Yield Curve Distributins: Single Factr vs. Tw Factr Stchastic Prcesses Yield Curve (%) 5Yr LIBOR Rate (%) 3 MC Shrt Rate 25% MC Lng Rate 25% Percentil emax 2 Percentil emax 2 15% % % Tw Factr Prcess: Yield Curve Distributin 90th 75th Mean Median 25th 10th 2.5 Min Tw Factr 3-31 Mnth 3% 2% 1% -1% -2% -3% -4% MC Yield Curve 1 5% 90th 75th Mean Median 25th 10th 2.5 Min Tw Factr Percentil e Max th 75th Median Mean 25th 10th 2.5 Min Mnth -5% Page 26-6% Mnth Tw Factr 3-31

27 Yield Curve (%) Yield Curve Distributins: Single Factr vs. Tw Factr Stchastic Prcesses Yield Curve (%) Single Factr Prcess: Yield Curve Distributin Tw Factr Prcess: Yield Curve Distributin 3% 2% 1% -1% MC Yield Curve Percentil e 75th 90th 97.5 Max Median Mean 25th 10th 3% 2% 1% -1% MC Yield Curve Percentil e Max th 75th Median Mean 25th 10th -2% -2% 2.5-3% -3% 2.5-4% -5% -4% -5% Min -6% Mnth Single Factr 3-31 Min -6% Mnth Tw Factr 3-31 Page 27

28 %D MV %D MV FRMs: MV Sensitivity: Single vs. 2 Factr 12/31 3/31 4% FRMs: MV Sensitivity 6% FRMs: MV Sensitivity 2% 4% 2% -2% -4% -6% FSI1F FSI2F ALM -2% -4% -6% -8% FSI1F FSI2F ALM -8% Shck (bp) -12% Shck (bp) Impact f prcess is limited in this case even thugh distributin f yield curves is nticeable Page 28

29 Observatins frm Test Stchastic mdels will frequently prduce different measures f risk at the instrument level fr instruments with embedded ptins We believe but it is difficult t cnfirm withut access t surce cde that this is largely due t differences in calibratin prcedures as well as differences in mdeling methds. With larger prtflis that include instruments with a wide variety f ptin structures, the differences in results will tend t cancel each ther ut if the verall levels f implied vlatility used in the calibratins are similar. Page 29

30 Observatins frm Test Validatin tests f stchastic mdels using a qualified stchastic mdel, such as FSI, can: Identify mdeling errrs, because large differences in results invite further investigatin Enhance understanding f the black bx f ALM mdel calibratin prcedures, when calibratins aren t all that clse Prvide guidance fr the prduct characteristics that are stchastic mdeling technique sensitive Validatin tests f stchastic mdels lead t the fllwing warning: Beware f relying n a single stchastic mdel when measuring risk at the instrument level fr instruments with embedded cmplex ptins Page 30

31 Observatins frm Test Tests f stchastic mdels can identify mdeling errrs, because large differences invite further investigatin Wrd t the warning abut using stchastic mdels fr risk management invlving individual transactins Page 31