www.pwc.ch The non-backtestability of the Expected Shortfall
Agenda 1 Motivation 3 2 VaR and ES dilemma 4 3 Backtestability & Elicitability 6 Slide 2
Motivation Why backtesting? Backtesting means model validation or also comparison of different models against realized observations Backtestability is the only way to say if your model is doing a good job in forecasting risk Basel Committee replaces VaR 1% with ES 2.5% (for market risk only) ES cannot be backtested because it fails to satisfy elicitability... If you held a gun to my head and said: We have to decide by the end of the day if Basel 3.5 should move to ES, with everything we know now, or do we stick with VaR, I would say: Stick with VaR Paul Embrechts, March 2013, RISK.net Slide 3
VaR and ES dilemma Background VaR: the best of worst x% losses threshold of x% losses ES: the average of worst x% losses expected x% loss Slide 4
Backtestability & Elicitability Backtestable what s that? What do we mean when we say a risk measure is backtestable? A risk measure is backtestable if there exists an observable test statistic, that allows to say whether predictions are over/under-estimated H 0 : The risk measure estimates at hand are correct model validation against realized observations H 0 : The risk measure estimates at hand are at least as good as the ones from the standard procedure or H 0 : The risk measure estimates at hand are at most as good as the ones from the standard procedure comparison of different models against realized observations Slide 5
Backtestability & Elicitability Backtestable = Elicitable? Elicitability = a possibility for verification and comparison of risk measures by using scoring or loss functions Elicitability is a very important property in risk management (statistics), mainly for two reasons: it provides a natural methodology to perform backtesting model validation against realized observations it provides a natural way of comparing different procedures for computing a risk measure comparison of different models against realized observations Slide 6
Backtestability & Elicitability Backtestable = Elicitable? (Cont d) Elicitability implies backtestability (in both cases) If a measure is elicitable, we can rank models by their mean score VaR is elicitable However, backtesting (model validation) methods for VaR that are used in practice do not use any structure from elicitability ES is not elicitable However, there are practical and easily implementable tests to backtest (model validate) ES (coverage test similar to the one for VaR, s. ref [1]) ES is jointly elicitable with VaR (s. ref [2]) This means that you can backtest ES (compare different models) only if you have previously backtested VaR, bilaterally This opens the way for valid practical backtesting of ES 1 N. Costanzino, M. Curran. Backtesting general spectral risk measures with application to ES. February 21, 2015 2 T. Fissler, J. F. Ziegel, T. Gneiting. Expected Shortfall is jointly elicitable with Value at Risk Implications for backtesting, July 14, 2015 Slide 7
Backtestability & Elicitability Backtestable = Elicitable? (Cont d) Lack of elicitability need not imply lack of backtestability Elicitability is not the only way to backtest a risk measure Is there a risk measure which is coherent, elicitable and is appropriate for regulatory purposes? Backtestable Elicitable VaR, expectiles ES Slide 8
Thank You This publication has been prepared for general guidance on matters of interest only, and does not constitute professional advice. You should not act upon the information contained in this publication without obtaining specific professional advice. No representation or warranty (express or implied) is given as to the accuracy or completeness of the information contained in this publication, and, to the extent permitted by law, PricewaterhouseCoopers AG, its members, employees and agents do not accept or assume any liability, responsibility or duty of care for any consequences of you or anyone else acting, or refraining to act, in reliance on the information contained in this publication or for any decision based on it. 2016. All rights reserved. In this document, refers to PricewaterhouseCoopers AG which is a member firm of PricewaterhouseCoopers International Limited, each member firm of which is a separate legal entity.