Week 2 Quantitative Analysis of Financial Markets Hypothesis Testing and Confidence Intervals Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036 October 27, 2017 Christopher Ting QF 603 October 27, 2017 1/45
Table of Contents 1 Introduction 2 Confidence Intervals 3 Hypothesis Test 4 Chebyshev s inequality 5 VaR 6 Problems with VaR 7 Takeaways Christopher Ting QF 603 October 27, 2017 2/45
Introduction Impracticality to analyze an entire population. Random sampling Estimation of sample statistics: Mean and Variance Hypothesis formation and procedures used to conduct tests concerned with population means and population variances. Construction of confidence intervals for population parameters based on sample statistics. t test, chi-square test Chebyshev s Inequality Value at risk: Extremely useful in risk management Christopher Ting QF 603 October 27, 2017 3/45
Learning Outcomes of QA05 Chapter 7. Michael Miller, Mathematics and Statistics for Financial Risk Management, 2nd Edition (Hoboken, NJ: John Wiley & Sons, 2013). Calculate and interpret the sample mean and sample variance. Construct and interpret a confidence interval. Construct an appropriate null and alternative hypothesis. and calculate an appropriate test statistic. Differentiate between a one-tailed and a two-tailed test and identify when to use each test. Interpret the results of hypothesis tests with a specific level of confidence. Demonstrate the process of backtesting VaR by calculating the number of exceedances. Christopher Ting QF 603 October 27, 2017 4/45
Recall: Sample Mean Sample mean µ = 1 n n x i = i=1 n i=1 x i n. Consider the random variable X i /n. Sample mean is equivalent to the sum of n i.i.d. random variables, X i /n, each with a mean of µ/n and a standard deviation of σ/n. The central limit theorem suggests that the distribution of the sample mean converges to a normal distribution with mean µ and variance σ 2 /n, i.e., ) µ N (µ, σ2. n Sample mean is unbiased. Christopher Ting QF 603 October 27, 2017 5/45
Recall Sample Variance The sample variance σ 2 = 1 n 1 n ( xi µ ) 2 is unbiased. The variance of the sample variance is ( E ( σ 2 σ 2) ) ( 2 2 = σ 4 n 1 + κ 3 ). n If we have n sample points, then the sample variance σ 2 follows a chi-squared distribution with (n 1) degrees of freedom: i=1 (n 1) σ2 σ 2 χ2 n 1. This relationship between the ratio of variances and the chi-squared distribution is true only when the data generating process (DGP) results in normally distributed data points. Christopher Ting QF 603 October 27, 2017 6/45
t-statistic If we first standardize our estimate of the sample mean using the sample standard deviation, the new random variable follows a Student s t distribution with (n 1) degrees of freedom. t = µ µ. σ n This standardized version of the population mean is so frequently used that it is referred to as a t-statistic, or simply a t-stat. Christopher Ting QF 603 October 27, 2017 7/45
Confidence Interval By looking up the appropriate values for the t distribution, we can establish the probability that our t-statistic is contained within a certain range: P x L µ µ x σ U = γ. n where γ is the probability that our t-statistic will be found within the lower and upper bounds. Typically γ is referred to as the confidence level, and 1 γ is known as the significance level and is often denoted by α. Christopher Ting QF 603 October 27, 2017 8/45
Estimation of Confidence Interval In practice, the population mean, µ, is often unknown. After rearranging, we obtain an equation with a more interesting form: ( P µ x L σ µ µ + x ) U σ = γ. n n We are now giving the probability that the population mean will be contained within the defined range. We call this range the confidence interval for the population mean. A problem with confidence intervals is that they require us to settle on an arbitrary confidence level. Christopher Ting QF 603 October 27, 2017 9/45
Hypothesis Testing What is the probability that the population mean is greater than y. In the form of a null hypothesis, we are interested in knowing whether the expected return of a portfolio manager is greater than 10%: H 0 : µ r > 10%. Even though the true population mean is unknown, for the hypothesis test we assume that the population mean is 10%. Christopher Ting QF 603 October 27, 2017 10/45
Hypothesis Testing (cont d) If the true population mean were 10%, what would be the probability that we would see a given sample mean? t = µ 10% σ/ n. We can then look up the corresponding probability from the t distribution. In addition to the null hypothesis, we can offer an alternative hypothesis. In the previous example, where our null hypothesis is that the expected return is greater than 10%, the logical alternative would be that the expected return is less than or equal to 10%: H 1 : µ r 10%. Christopher Ting QF 603 October 27, 2017 11/45
Hypothesis Test in Practice Many practitioners construct the null hypothesis so that the desired result is false. If we are an investor trying to find good portfolio managers, then we would make the null hypothesis µ r 10%. That we want the expected return to be greater than 10% but we are testing for the opposite makes us seem objective. In the case where there is a high probability that the manager s expected return is greater than 10% (a good result), we have to say, We reject the null hypothesis that the manager s expected return is less than or equal to 10% at the x% level. Christopher Ting QF 603 October 27, 2017 12/45
Sample Problem 1 At the start of the year, you believed that the annualized volatility of XYZ Corporation s equity was 45%. At the end of the year, you have collected a year of daily returns, 256 business days worth. You calculate the standard deviation, annualize it, and come up with a value of 48%. Can you reject the null hypothesis, H 0 : σ = 45%, at the 95% confidence level? Answer: The appropriate test statistic is (n 1) σ2 = (256 1)0.482 σ2 0.45 2 = 290.13 χ2 255. For a chi-squared distribution with 255 degrees of freedom, 290.13 corresponds to a probability of 6.44%. Hence, we fail to reject the null hypothesis at the 95% confidence level. Christopher Ting QF 603 October 27, 2017 13/45
One Tail or Two? In risk management, more often than not we are more concerned with the probability of extreme negative outcomes, and this concern naturally leads to one-tailed tests. A two-tailed null hypothesis could take the form H 0 : µ = 0, H 1 : µ 0. In this case, H 1 implies that extreme positive or negative values would cause us to reject the null hypothesis. A one-tailed null hypothesis could take the form H 0 : µ > c, H 1 : µ c. In this case, reject H 0 only if the estimate of µ is significantly less than c. Christopher Ting QF 603 October 27, 2017 14/45
Some t Statistics at Different α α t 10 t 10 t 1000 t 1.00% -2.76-2.36-2.33-2.33 2.50% -2.23-1.98-1.96-1.96 5.00% -1.81-1.66-1.65-1.64 10.00% -1.37-1.29-1.28-1.28 90.00% 1.37 1.29 1.28 1.28 95.00% 1.81 1.66 1.65 1.64 97.50% 2.23 1.98 1.96 1.96 99.00% 2.76 2.36 2.33 2.33 For a normal distribution 95% of the mass is within ±1.96 standard deviations. For a one-tailed test, 95% of the mass is within either +1.64 or -1.64 standard deviations. Using 1.96 instead of 1.64 is a common mistake. Christopher Ting QF 603 October 27, 2017 15/45
Gold Standards of Confidence Level In practice, 95% and 99% confidence levels are gold standards. If we can reject a null hypothesis at the 96.3% confidence level, some practitioners will simply say that the hypothesis can be rejected at the 95% confidence level. The implication is that, even though we may be more confident, 95% is enough. This convention can be convenient when testing a hypothesis repeatedly. In statistics, there is no such thing as a sure bet; there is no such thing as absolute certainty. Christopher Ting QF 603 October 27, 2017 16/45
Chebyshev s Inequality For the random variable X with mean µ and variance σ 2, and given any positive constant k, P ( X µ kσ ) = E ( 1 X µ kσ ) = E Hence E ( (X µ kσ P ( X µ kσ ) 1 k 2 1 ( X µ kσ ) 2 1 ) 2 ) = 1k 2 E ( (X µ) 2) σ 2 = 1 k 2. Christopher Ting QF 603 October 27, 2017 17/45
Chebyshev s Inequality (cont d) Interpretation: The probability of the magnitude of dispersion from the mean being greater than kσ is no greater than 1/k 2. Intuitively, Chebyshev s inequality suggests that outliers are rare. Instead of µ and σ 2, suppose we can only estimate with unbiased sample mean µ and sample variance σ 2, respectively. It is easy to prove that E ( (X µ) 2) = E ( (X µ) 2). It follows that Chebyshev s inequality becomes P ( X µ k σ ) 1 k 2 σ 2 σ 2. Christopher Ting QF 603 October 27, 2017 18/45
Chebyshev s Inequality (cont d) Assume a large number of data points so that σ 2 / σ 2 = 1. Then P ( X µ k σ ) 1 k 2. Now, P ( X µ < k σ ) = 1 P ( X µ k σ ). It follows that P ( X µ < k σ ) 1 1 k 2. Christopher Ting QF 603 October 27, 2017 19/45
Chebyshev s Inequality in QF Interpretation: X µ < k σ is equivalent to µ k σ < X < µ + k σ. The probability for X to realize a value in the interval is at least 1 1/k 2. The larger k is or the wider the interval is, the minimum probability will be larger. Christopher Ting QF 603 October 27, 2017 20/45
Application of Chebyshev s Inequality For a given level of variance, Chebyshev s inequality places an upper limit 1/k 2 on the probability of a variable being more than a certain distance from its mean, for any statistical distribution (with well defined variance and mean). For a given distribution, the actual probability may be considerably less. Take, for example, a standard normal variable. Chebyshev s inequality tells us that the probability of being greater than two standard deviations from the mean is less than or equal to 25%. The exact probability for a standard normal variable is closer to 5%, which is indeed less than 25%. Christopher Ting QF 603 October 27, 2017 21/45
Tutorial on Chebyshev s Inequality What is the upper bound for the probability of a stock return to be 3 standard deviations away from the mean? Answer: On October 19, 1987, the S&P 500 index dropped about 23% (log return). The daily standard deviation is 0.9722%. How many standard deviations is the Black Monday away from the mean? Answer:. According to Chebyshev s inequality, what is the upper bound of the probability of the Black Monday? Answer:. Christopher Ting QF 603 October 27, 2017 22/45
Tutorial on Chebyshev s Inequality (cont d) The number of daily returns used for estimating the sample standard deviation is 16,525. What is the expected number of Black Nondays? Answer:. What is the actual probability of a Black Monday? Answer:. Christopher Ting QF 603 October 27, 2017 23/45
Value at Risk Value at risk (VaR) is one of the most widely used risk measures. VaR was popularized by J.P. Morgan in the 1990s. The executives at J.P. Morgan wanted their risk managers to generate one statistic at the end of each day, which summarized the risk of the firm s entire portfolio. What they came up with was VaR. If the 95% VaR of a portfolio is $400, then we expect the portfolio will lose $400 or less in 95% of of the scenarios, and lose more than $400 in 5% of the scenarios. We can define VaR for any confidence level, but 95% has become an extremely popular choice. VaR is decidedly a one-tailed confidence interval. Christopher Ting QF 603 October 27, 2017 24/45
Value at Risk Time Horizon The time horizon also needs to be specified for VaR. On trading desks, with liquid portfolios, it is common to measure the one-day 95% VaR. In other settings, in which less liquid assets may be involved, time frames of up to one year are not uncommon. Christopher Ting QF 603 October 27, 2017 25/45
Formal Definition of VaR If an actual loss equals or exceeds the predicted VaR threshold, that event is known as an exceedance. Let the random variable L represent the loss to your portfolio. L is simply the negative of the return to your portfolio. If the return of your portfolio is -$600, then the loss, L, is +$600. For a given confidence level, γ, then, we can define value at risk as P ( L VaR γ ) = 1 γ. If a risk manager says that the one-day 95% VaR of a portfolio is $400, it means that there is a 5% probability that the portfolio will lose $400 or more on any given day (that L will be more than $400). Christopher Ting QF 603 October 27, 2017 26/45
Sample Problem 2 The probability density function (PDF) for daily profits at Triangle Asset Management can be described by the following function: 1 p(x) = 10 + 1 x, 10 x 0; 100 1 10 1 x, 0 < x 10. 100 What is the one-day 95% VaR for Triangle Asset Management? Answer: To find the 95% VaR, we need to find a, such that Hence, a 10 a 10 p(x) dx = 0.05. 1 10 + 1 100 x dx = 1 10 a + 1 200 a2 + 0.50 = 0.05. Christopher Ting QF 603 October 27, 2017 27/45
Sample Problem 2 (cont d) We obtain a quadratic equation a 2 + 20a + 90 = 0. Using the quadratic formula, we can solve for a a = 10 ± 10. Because the distribution is not defined for x < 10, we can ignore the negative, giving us the final answer a = 10 + 10 = 6.84. The one-day 95% VaR for Triangle Asset Management is a loss of 6.84. Christopher Ting QF 603 October 27, 2017 28/45
95% vs. 99.9% At the 95% confidence level will, on average, an exceedance event occurs every 20 days. Is an event that happens once every 20 days really something that we need to worry about? VaR at the 99.9% confidence level expects to see an exceedance only once every 1,000 days. It is tempting to believe that the risk manager using the 99.9% confidence level is concerned with more serious, riskier outcomes, and is therefore doing a better job. But this is a common mistake for newcomers choosing a confidence level that is too high. If we have 1,000 data points, then there are 50 data points to back up our 95% confidence level, but only one to back up our 99.9%. Christopher Ting QF 603 October 27, 2017 29/45
Backtesting Backtesting entails checking the predicted outcome of a model against actual data. Any model parameter can be backtested. In the case of one-day 95% VaR, there is a 5% chance of an exceedance event each day, and a 95% chance that there is no exceedance. P ( K = k ) ( ) n = p k (1 p) n k. k Here, n is the number of periods that we are using in our backtest, k is the number of exceedances, and (1 p) is our confidence level. Christopher Ting QF 603 October 27, 2017 30/45
Sample Problem 3 As a risk manager, you are tasked with calculating a daily 95% VaR statistic for a large fixed income portfolio. Over the past 100 days, there have been four exceedances. How many exceedances should you have expected? What was the probability of exactly four exceedances during this time? The probability of four or less? Four or more? Answer: Over 100 days we would expect to see five exceedances: (1 95%) 100 = 5. The probability of exactly four exceedances is 17.81%: P ( K = 4 ) ( ) 100 = 0.05 4 (1 0.05) 96 = 17.81%. 4 Christopher Ting QF 603 October 27, 2017 31/45
Sample Problem 3 (cont d) Like we compute for zero, one, two, and three exceedances, and the probability of excedences less than or equal to four is P ( K 4 ) = 4 ( ) 100 0.05 k (1 0.05) 100 k k k=0 = 0.0059 + 0.0312 + 0.0812 + 0.1396 + 0.1781 = 43.60% The probability of four exceedances and above is P ( K 4 ) = P ( K > 4 ) +P ( K = 4 ) = (1 0.4360)+0.1781 = 74.21% Christopher Ting QF 603 October 27, 2017 32/45
Serial Correlation in Exceedences To model serial correlation in exceedances, look at the periods immediately following any exceedance events. The number of exceedances in these periods should also follow a binomial distribution. For example, suppose you are calculating the one-day 95% VaR for a portfolio, and you observed 40 exceedances over the past 800 days. You count the number of back-to-back exceedances by looking at the 40 days immediately following the exceedance event. Because you use 95% confidence level, you would expect that 2 of them, 5% 40 = 2, would also be exceedances. The actual number of these day-after exceedances should follow a binomial distribution with n = 40 and p = 5%. Christopher Ting QF 603 October 27, 2017 33/45
Serial Correlation in Exceedences (cont d) Another common problem with VaR models in practice is that exceedances tend to be correlated with the level of risk. Positive correlation between exceedances and risk levels can happen when a model does not react quickly enough to changes in risk levels. Negative correlation can happen when model windows are too short. To test for correlation between exceedances and the level of risk, you divide exceedances into two or more buckets, based on the level of risk. Christopher Ting QF 603 October 27, 2017 34/45
Example As an example, consider again the one-day 95% VaR for a portfolio over the past 800 days. You divide the sample period in two, placing the 400 days with the highest forecasted VaR in one bucket and the 400 days with the lowest forecasted VaR in the other. After sorting the days, you would expect each 400-day bucket to contain 20 exceedances: 5% 400 = 20. The actual number of exceedances in each bucket should follow a binomial distribution with n = 400, and p = 5%. Christopher Ting QF 603 October 27, 2017 35/45
Subadditivity One criticism is that VaR is not a subadditive risk measure. Subadditivity is basically a fancy way of saying that diversification is good, and a good risk measure should reflect that. Assume that our risk measure is a function f that takes as its input a random variable representing an asset or portfolio of assets. Higher values of the risk measure are associated with greater risk. If we have two risky portfolios, X and Y, then f is said to be subadditive if f(x + Y ) f(x) + f(y ). In other words, the risk of the combined portfolio, X + Y, should be less than or equal to the sum of the risks of the separate portfolios. Christopher Ting QF 603 October 27, 2017 36/45
Sample Problem 4 A portfolio consists of two bonds, each with a 4% probability of defaulting. Assume that default events are uncorrelated and that the recovery rate of both bonds is 0%. If a bond defaults, it is worth $0; if it does not, it is worth $100. What is the 95% VaR of each bond separately? What is the 95% VaR of the bond portfolio? Answer: For each bond separately, the 95% VaR is $0. For an individual bond, in (over) 95% of scenarios, there is no loss. In the combined portfolio, however, there are three possibilities, with the following probabilities: p(x) x 0.16% -$200 7.68% -$100 92.16% $0 Christopher Ting QF 603 October 27, 2017 37/45
Sample Problem 4 (cont d) There are no defaults in only 92.16% of the scenarios, (1 4%) 2 = 92.16%. In the other 7.84% of scenarios, the loss is greater than or equal to $100. The 95% VaR of the portfolio is therefore $100. For this portfolio, VaR is not subadditive. Because the VaR of the combined portfolio is greater than the sum of the VaRs of the separate portfolios, VaR seems to suggest that there is no diversification benefit, even though the bonds are uncorrelated. It seems to suggest that holding $200 of either bond would be less risky than holding a portfolio with $100 of each. Clearly this suggestion is not correct. Christopher Ting QF 603 October 27, 2017 38/45
Sample Problem 4 (cont d) This example makes clear that when assets have payout functions that are discontinuous near the VaR critical level, we are likely to have problems with subadditivity. By the same token, if the payout functions of the assets in a portfolio are continuous, then VaR will be subadditive. In many settings this is not an onerous assumption. In between, we have large, diverse portfolios, which contain some assets with discontinuous payout functions. For these portfolios subadditivity will likely be only a minor issue. Christopher Ting QF 603 October 27, 2017 39/45
Expected Shortfall Another criticism of VaR is that it does not tell you anything about the tail of the distribution. Two portfolios could have the exact same 95% VaR but very different distributions beyond the 95% confidence level. More than VaR, then, what we really want to know is how big the loss will be when we have an exceedance event. Using the concept of conditional probability, we define the expected value of a loss, given an exceedance, as follows: E ( L L VaR γ ) = S. We refer to this conditional expected loss, S, as the expected shortfall. Christopher Ting QF 603 October 27, 2017 40/45
Expected Shortfall (cont d) If the profit function has a probability density function given by f(x) and VaR is at the γ confidence level, then we can find the expected shortfall as: S = 1 1 γ VaR x f(x) dx. Christopher Ting QF 603 October 27, 2017 41/45
Sample Problem 5 For the same confidence level and time horizon of Sample Problem 2 in Slide 27, what is the expected shortfall? Answer: VaR S = 1 x p(x) dx = 20 0.05 10 = (x 2 + 115 ) VaR x3. 10 VaR 10 With VaR = 10 10, we find that S = 7.89. Thus, the expected shortfall is a loss of 7.89. ( 1 x 10 + 1 ) 100 x dx Christopher Ting QF 603 October 27, 2017 42/45
Sample Problem 5 (cont d) Intuitively, the expected shortfall must be greater than the VaR, 6.84, but less than the maximum loss of 10. Because extreme events are less likely (the height of the PDF decreases away from the center), it also makes sense that the expected shortfall is closer to the VaR than it is to the maximum loss. Christopher Ting QF 603 October 27, 2017 43/45
Summary The hypothesis testing process requires a statement of a null and an alternative hypothesis, the selection of the appropriate test statistic, specification of the significance level, a decision rule, the calculation of a sample statistic, a decision regarding the hypotheses based on the test, and a decision based on the test results. Hypothesis testing compares a computed test statistic to a critical value at a stated level of significance, which is the decision rule for the test. Christopher Ting QF 603 October 27, 2017 44/45
Summary (cont d) A hypothesis about a population parameter is rejected when the sample statistic lies outside a confidence interval around the hypothesized value for the chosen level of significance. Backtesting is the process of comparing losses predicted by the value at risk (VaR) model to those actually experienced over the sample testing period. If a model were completely accurate, we would expect VaR to be exceeded with the same frequency predicted by the confidence level used in the VaR model. In other words, the probability of observing a loss amount greater than VaR should be equal to the level of significance. Christopher Ting QF 603 October 27, 2017 45/45