Connecting Markets East & West Standard Initial Margin Model (SIMM) How to validate a global regulatory risk model RiskMinds Eduardo Epperlein* Risk Methodology Group * In collaboration with Martin Baxter and James McEwen (GM Quantitative Research) The analysis and conclusions set forth are those of the author. Nomura is not responsible for any statement or conclusion herein, and opinions or theories presented herein do not necessarily reflect the position of the institution. December 9, 2015 Nomura
Before embarking on the validation framework it is important to appreciate the differences between a margin model and a capital model An Initial Margin (IM) model is designed to estimate how much collateral we need to post to cover a potential increase in the value of our derivative contracts over the Margin Period-of-Risk (MPoR) within a netting set Basel regulation 1 stipulates that IM should be calculated at a 99% confidence level, with MPoR is set at a minimum of 10 business days The calculation is repeated daily, thus capturing any change in the portfolio and, hence, any change to its variability Both counterparties conduct equivalent calculations of IM The bilateral IM is segregated, such that in the event of a counterparty defaulting its posted collateral provides the necessary protection Reconciliation and agreement on the amount of posted/called collateral is crucial for this process to work smoothly Hence, a standardization of the method to calculate IM is vital In a capital model we calculate the Expected Positive Exposure (EPE) to our counterparty in order to estimate the amount of credit risk capital we need to hold given the counterparty s probability of default (PD) Regulatory counterparty exposure models, such as the Internal Model Method (IMM), are designed to calculate the EPE of derivative contracts traded with the counterparty The credit risk capital is then estimated via the EPE, the PD of the counterparty, and the loss-given-default Unlike the risk mitigation provided by IM, the credit risk capital model requirement is imposed on the surviving counterparty The capital calculations need not be reconciled with the counterparty and, hence, don t require the same level of standardization as IM (though regulators may think otherwise to promote uniform financial safety) 1. Margin Requirements for non-centrally cleared derivatives, September 2013, BCBS 261 1
The financial industry, through the auspices of ISDA 1, agreed on a Standardized IM Model (SIMM TM ) and proposed it to the regulators SIMM is based on a variant of the Sensitivity Based Approach (SBA), which was developed by the regulators as risk-sensitive yet conservative standard model for market risk capital under FRTB 2 Equally important, the financial industry needed to propose a common approach for validating the SIMM and propose that the national regulators adopt that approach uniformly The gold standard for validating risk models is Backtesting. But, once again, it is important to highlight the differences between backtesting a capital model and an IM model: Risk Model Type Backtesting Approach Participation Frequency Corrective Actions Value-at-Risk VaR (market risk capital) Stand-alone All individual firms Daily Capital multiplier/model updates IMM (counterparty credit risk capital) Stand-alone All individual firms Quarterly Capital multiplier/model updates SIMM (Initial Margin) Global via central coordination Systemically important firms, covering systemically important portfolios Annually SIMM updates via central coordination 1. ISDA: International Swaps and Derivatives Association Inc 2. FRTB: Fundamental Review of the Trading Book 2
1 15 29 43 57 71 85 99 113 127 141 155 169 183 197 211 225 239 The regulatory backtesting framework currently used to validate VaR models 1 appeared to be the most suitable candidate to validate SIMM Basel regulation stipulates Red-Amber-Green (RAG) zones for establishing the validly of the VaR model Backtesting is performed by comparing the one-day VaR ex-ante (t) against the P&L ex-post (t to t+1) over 250 business days A VaR exception occurs when P&L < - VaR (i.e. loss exceeds VaR) RAG: Green (0-4 exceptions) Amber (5-9 exceptions) Red (10 or more exceptions) a.k.a. Basel Traffic light test RAG zones correspond to type 1 errors (falsely rejecting an accurate model): Green (<95%), Amber (95%<99.99%), Red (>=99.99%) Exceptions follow a binomial distribution P&L Sample VaR backtesting 3 2 1 0-1 -2-3 VaR 3 Exceptions 1. https://www.bis.org/publ/bcbsc223.pdf 3
The adopted SIMM backtesting framework needed to be done globally across systemically important firms, in a coordinated fashion The last exercise concluded in July 30 th, 2015, involving 16 institutions, across 19 legal entities, generating 280 portfolios, via the following 4 simple steps: 1 1. Calculate the SIMM (post and call) by taking a snapshot of the portfolios as of April 30 th, 2015, 2. Generate about 7 years of historical P&L data, from January 1 st, 2008 to April 30 th, 2015, by shocking the frozen portfolio with a 10 business day market move, thus generating approximately 1900 (overlapping) P&Ls, 3. Conduct an extensive reconciliation exercise to help minimize operational errors 4. Perform backtesting analysis 1. This backtesting exercise was coordinated by ISDA 4
Before starting the actual backtesting exercise every firm conducted extensive reconciliations on a bilateral basis Two sample tests involved calculating: a) The correlation of pairs of P&L vectors (between two firms) perfect reconciliation would imply -100% correlation b) The relative difference between Own Entity Call IM and Counterparty Post IM As shown below, results were generally considered successful 5
The risk across the 280 portfolios was primarily driven by delta exposure In order to make the portfolio more representative of future state of when SIMM goes live 1 each selected portfolio contained uncleared OTC derivative trades executed between June 30th, 2013 (inclusive) and April 30th, 2015 (inclusive) and open as of April 30th, 2015. 1. Go live expected September 2016 6
The ratio of the calculated IM to the 99% and 1% percentile of the historical 10-day P&L distribution gave the first indication of the validity of the SIMM The sum of all SIMM values is over 2x larger than the sum of all historical VaR measures. This indicates that for the actual portfolios the calculated IM is likely to be conservative and pass backtesting. 7
The standard 1-day VaR backtesting had to be modified to for the 10-day SIMM backtesting using overlapping windows and multiple portfolios The first modification to the backtesting involved the transition from taking independent samples of 1-day P&Ls to overlapping samples of 10-day P&Ls We can illustrate the effect of auto-correlation introduced by the overlapping windows by conducting a Monte Carlo simulation of 250 IID random variables and generating the overlapping P&Ls to empirically estimate the RAG zones: Please see below: Zone Number of exceedances 1-day 10-day overlapping Green 0-4 0-8 Amber 5-9 9 25 Red 10+ 26+ The second modification involved taking into account the fact that the 280 backtesting portfolios were conducted across common time slices and therefore were not necessarily independent We can also illustrate this effect calculating the empirical correlation across the portfolios and repeating the Monte Carlo simulation with the same correlation structure. Please see below an example with 3 time series with identical pair-wise correlation of 50%: Number of exceedances Zone 10-day overlapping, 1-day independent 50% correlation Green 0 11 0 19 Amber 12 19 20 51 Red 20+ 52+ 8
We can now backtest a single portfolio by comparing the Call and Post IM against the historical time series of 10-day P&Ls Here we show a sample plot of 782 overlapping 10-day P&Ls (from Feb 29 th, 2008 to Apr 8 th, 2015) against IM to Post and IM to Call. We observe 9 exceedances against IM to Call and 3 against IM to Post, which fall well within the Green zone of up to 18 exceedances As one might expect, the majority of the exceedances occurred during the 2008-09 crisis period. 8
By conducting the modified backtesting at legal entity level the number of exceedances beyond the IM level where all within the GREEN zone It should be noted that each legal entity had multiple portfolios with different correlation structure and different numbers of historical data points. Hence, the RAG zones had to be estimated separately Backtesting covered in- and out-of-sample test periods since SIMM is calibrated with a 1 year stress period and recent 3 year period 1 More granular backtesting was also performed at the 280 individual portfolios for both called and posted IM, and only two out 280 portfolios (less than 1%) had exceedances in the Amber zone Hence, overall, a successful result Legal Entity Number of Observations Green up to Amber up to Exceedance Count (to call) Traffic Light (to call) Exceedance Count (to post) Traffic Light (to post) A 1913 427 594 8 Green 42 Green B 1903 398 553 38 Green 23 Green C 1904 415 555 15 Green 42 Green D 1775 409 625 122 Green 44 Green E 1832 419 600 39 Green 62 Green F 1497 323 459 63 Green 40 Green G 1913 404 522 31 Green 35 Green H 1911 274 373 76 Green 59 Green I 1911 115 181 25 Green 17 Green J 1911 384 505 21 Green 2 Green K 1903 221 295 35 Green 9 Green L 1903 322 439 40 Green 8 Green M 1913 403 569 27 Green 37 Green N 782 168 255 22 Green 46 Green O 1211 279 398 37 Green 55 Green P 1913 394 535 60 Green 48 Green Q 1852 377 499 9 Green 29 Green R 1903 418 567 23 Green 115 Green S 1903 369 543 47 Green 66 Green 1. SIMM is designed to be non pro cyclical so it makes sense to backtest annually over a an extended test period, even it involves some level of in sample testing 9
Summary and conclusions How to validate a global regulatory risk model In particular, How to validate SIMM IM models, unlike capital models, require a much higher level of standardization. Hence, the need to a Standard IM Model or SIMM To preserve the standardization of IM the validation needs to be applied uniformly by national regulators The standard VaR backtesting framework has been adapted to test the SIMM over a 10-day overlapping window and across multiple portfolios A global validation framework has been successfully developed and tested across 16 major financial institution SIMM successfully passed the global backtesting exercise as of April 30 th, 2015 1 1. Submitted to regulators on July 31 st, 2015 10
Appendix
Glossary BCBS: Basel Committee on Banking Supervision. Founded in 1974 by regulators in the G-10 countries, with mandate to strengthen regulation, supervision and practices of banks worldwide to enhance financial stability, but without any formal supranational authority. EPE: Expected Positive Exposure. Average positive exposure calculated across a netting set over a 1 year horizon.. FRTB: Fundamental Review of the Trading Book. Fundamental review of the market risk framework introduced under Basel 2.5. IMM: Internal Model Method. Internal model used for calculating counterparty exposure at netting set level for both OTC derivative and Securities Financing Transactions (SFT) MPoR: Margin Period-of-Risk PD: Probability of Default SBA: Sensitivity Based Approach SIMM: Standard Initial Margin Model VaR: Value at Risk. Trading loss calculate at a given confidence level and time horizon. For regulatory capital calculation we use 99% confidence level and 10 days. 11